Dmitriy Setrakyan on Apache Ignite Beyond the Data Grid | GOTO 2015
Hi, it's Mike with UGtastic . I'm here at GOTO Conference 2015 and I 'm sitting here with Dmitriy Setrakyan who gave a talk about the data grid, well, bigger than the data grid. Hi, it's Mike with UGtastic . I'm here at GOTO Conference 2015 and I 'm sitting here with Dmitriy Setrakyan who gave a talk about the data grid, well, bigger than the data grid. Meet Dmitriy Setrakyan at GOTO Conference 2015! He dives into Apache Ignite, an in-memory data fabric that extends beyond traditional data grids. Learn about its features, history, and how it was donated to Apache. ://just3ws.github.io/interviews/dmitri-goto-conference-2015
The Interviewer
Mike Hall
Interviewer, UGtastic
The Guest
Dmitriy Setrakyan
Apache Ignite beyond the data grid
Mike Hall
Interviewer, UGtastic
▶ Speaking
Hi, it's Mike with UGtastic . I'm here at GOTO Conference 2015 and I 'm sitting
here with Dmitriy Setrakyan who gave a talk about the data grid, well, bigger than the data
grid. Beyond the data grid. Beyond the data grid with Apache Ignite. Well, first
I want to say thank you very much for taking the time to speak with me and you
can tell me a little bit about what your presentation was about and what is this
Apache Ignite product. All right, thank you Mike. So yeah, I want you to notice
I'm wearing an Apache Ignite t-shirt, first of all, so it's very cool. So whenever
you see green t-shirts, I want you to know it's Apache Ignite. So I just gave
a talk about Apache Ignite. So it's an Apache project, currently undergoing incubation
process at Apache, but I'm getting a feeling it 's gonna graduate pretty soon,
probably within a couple of months. And what it is, it's a memory data fabric
and essentially you use Apache Ignite whenever you face performance or scalability
problems within your application. And at that point you decide to introduce an
in-memory architecture, you want to cache a lot of data. You're gonna be faced
with quite a few use cases, but a lot of them are common and we see it for every
project that moves in memory. For example, you are going to have some clustering,
you are going to have some scale out, horizontal scalability issues, you are going
to probably want to cache data in memory, maybe partition or shard data in memory,
and you probably want to transact on that data or maybe query that data. You may
want to stream that data into the system or continuously ingest large amounts
of data into the system and use streaming, maybe computing using distributed MapReduce,
or for joint type APIs. So all of those components are available within Apache
Ignite in- memory data fabric. So from the get-go, we wanted Apache Ignite to
become a one-point solution for most of the in-memory use cases, and that's why
we cover most of these use cases. However, the project is still very simple to
use. It comes as one zip file. You just unzip it off your go. It uses all the
standard APIs, probably you're most familiar with in Java, like maps, queues,
sets, most of the concurrent Java Util concurrent APIs are also ported to Apache
Ignite, ThreadPool, distributed executor service. All of those standard APIs are
still used within Apache Ignite. So is Apache Ignite a database competitor to a NoSQL database, or is it a layer that sits between applications and persistent data stores?
Dmitriy Setrakyan
Apache Ignite beyond the data grid
▶ Speaking
That's a good question actually. So the answer is it's probably not a simple yes or no answer, because if you look at distributed cache within Apache Ignite, it is a key value store. So from that standpoint, you can think about it from a NoSQL standpoint, because you're working with basic Java objects. It's a key value store. However, Ignite has full SQL capabilities, and it allows you to index those, index inside of those objects. For example, your class names become tables, your fields inside of the classes become columns, and from there, and then you can create indexes in memory, and you can start querying it using standard SQL. So from one side, you can think about it as NoSQL, but it does provide SQL. It's SQL for NoSQL. SQL for NoSQL, there you go, that's a good way to put it.
Mike Hall
Interviewer, UGtastic
▶ Speaking
What does it provide, like a way to write back to persistence, or is it always only in memory?
Dmitriy Setrakyan
Apache Ignite beyond the data grid
▶ Speaking
Actually, it does. It does provide, it does automatically write through to a persistent store, so if you have like an Oracle database or MySQL database already installed, Ignite will automatically integrate with that data store, automatically will detect all the indexes and put them in memory, or if you can also integrate it, put it on top of NoSQL disk-based databases like Mongo, React, Couchbase, or Cassandra for that matter, and it will also work on top of those as well. So it sounds like an extremely advanced caching layer that provides some direct query capabilities that you can save a call to the actual persistent on-disk databases. That sounds about right for the data grid portion. However, as I mentioned, Ignite is a fabric. Data grid and distributed caching is just one component, and it probably is the biggest component within Ignite, and that's where most of the people actually will use. But there's also streaming of data, there's also sliding window support, there's also distributed map reduce for joint kind of computations, so it provides a lot more than just that. But yeah, from a data grid standpoint, it's a fairly feature-rich project. Okay, and I'm going to ask you to just move a little bit. The project itself, I mean, how did you go about, is it something that you kind of cooked up from projects you were working on or some thesis?
Mike Hall
Interviewer, UGtastic
▶ Speaking
Where did the idea come to exist?
Dmitriy Setrakyan
Apache Ignite beyond the data grid
▶ Speaking
All right, so actually I'm one of the co-founders of GridGain systems. So we started out probably seven or eight years ago, and that time we wanted, we had a dream of creating this easy-to-use grid computing system. And we started from compute grid, and probably three years down the road we added data grid capabilities to it. And GridGain actually always had an open-source component, open-source version, and also there was an enterprise version. And in September of last year we decided to donate code to Apache Software Foundation. So it started going through incubation in Apache, and we are, I'm hoping, approaching the graduation in Apache as well.
Mike Hall
Interviewer, UGtastic
▶ Speaking
So what is the process?
Dmitriy Setrakyan
Apache Ignite beyond the data grid
▶ Speaking
I mean, when you have an idea and you want to spin out an open-source project, do you just, did you go straight to Apache? Or was it that you had a project that you released and Apache contacted you and said, "Hey, let's pull this in.
Mike Hall
Interviewer, UGtastic
▶ Speaking
How did that, how did you end up with Apache then?
Dmitriy Setrakyan
Apache Ignite beyond the data grid
▶ Speaking
" So, well, I mean, there are two parts to the question. We wanted to join Apache because we wanted to grow community around open-source projects. So once you join Apache, you become part of this Apache process, what they call an Apache way, which is very community friendly. It introduces all sorts of outside contributions. So right now , believe it or not, we have, it used to be only GridGain commuters, we probably have more outside contributors working on Ignite than from the side of GridGain. So you get community growth. But however, Apache does not pull the code in. It will take your code, it will take your project and make sure you grow community around it, make sure you work with it the open-source way. But it will not take the code. You're still managing it. So you're still responsible for the project development. But it's just that it's kind of sold through the Apache marketplace. No, it's actually more of an Apache way. I mean, that's what it's called. It's Apache way, which is very open everything. All the discussions happen in the open. All the tickets happen in the open. All the design discussions and future of the project is all in the open and everybody is equal. It doesn't belong to one person. It belongs to the community now. Okay, and when you describe you're in an incubation phase, how do you go from incubation and what's the difference between incubation and being, well, not an incubation? Top-level project, you graduate. So the difference would be that, from a product standpoint, there's a very little difference. From an Apache standpoint, you have to, first of all, license compliance. You have to only have certain allowed licenses in your project, like definitely Apache 2. 0 is allowed as a license. There's BSD license and MIT licenses are allowed. Then you have to have a well-oiled build procedure, well-oiled community process set up within Apache. So once you actually show that you can work with an open- source community and follow this kind of path, at that point you're ready to graduate. And usually that happens when you do like two or three releases under Apache. So we're currently on two. I think after the third one we will be ready to graduate. We're already complying. We're already embracing the Apache way.
Mike Hall
Interviewer, UGtastic
▶ Speaking
Okay, well thank you very much for taking the time to speak with me and for supporting open-source. Thank you. Thanks.
Critical Insights
durable
"Apache Ignite provides a comprehensive in-memory data fabric, addressing various use cases including clustering, scalability, caching, and more."
durable
"GridGain Systems was founded with the goal of creating an easy-to-use grid computing system, which evolved to include data grid capabilities."
durable
"Donating the code to Apache Software Foundation for incubation allows the project to grow and benefit from a larger community."