Demystifying Big Data with Scala and Akka

30
Big Data Computation Technology And Analysis With Scala Vikas Hazrati, Knoldus Software LLP Big Data Innovation Conference (c)
  • date post

    17-Oct-2014
  • Category

    Technology

  • view

    4.260
  • download

    1

description

Big Data with Scala Presentation at National Big Data summit 2013

Transcript of Demystifying Big Data with Scala and Akka

Page 1: Demystifying Big Data with Scala and Akka

Big Data Computation Technology And Analysis With Scala

Vikas Hazrati, Knoldus Software LLP

Big Data Innovation Conference (c)

Page 2: Demystifying Big Data with Scala and Akka

About Me

CTO at Knoldus Software

Writing a book titled “Building performant and concurrent systems

with Scala and Akka”

Working with Scala and Big Data for the last 3 years

Big Data Innovation Conference (c)

Page 3: Demystifying Big Data with Scala and Akka

what?

Add content

Big Data Innovation Conference (c)

2011 - 1.8 zettabytes (1.8 trillion gigabytes)

big-data market is expected to grow from US$3.2 billion in 2010 to US$16.9 billion in 2015

Page 4: Demystifying Big Data with Scala and Akka

Big Data Innovation Conference (c)

Lot of data

Lot of compute

Long time to compute it

so what?

Page 5: Demystifying Big Data with Scala and Akka

About Me

Big Data Innovation Conference (c)

Page 6: Demystifying Big Data with Scala and Akka

hadoop

Big Data Innovation Conference (c)

Batch processing, non interactive

X real time

Page 7: Demystifying Big Data with Scala and Akka

alternates

Big Data Innovation Conference (c)

Lucene

CEP

NoSQL

Combinationof

parallel processing

Page 8: Demystifying Big Data with Scala and Akka

to build it :-{

Difficult!

Domain + Language + Performance

Big Data Innovation Conference (c)

Page 9: Demystifying Big Data with Scala and Akka

to build it :-{

Domain Expertise

Language Expertise

Performance Expertise

Big Data Innovation Conference (c)

Page 10: Demystifying Big Data with Scala and Akka

to build it :-{

Domain Expertise

Language Expertise

Performance Expertise

Big Data Innovation Conference (c)

Page 11: Demystifying Big Data with Scala and Akka

scala

Add content

Big Data Innovation Conference (c)

Page 12: Demystifying Big Data with Scala and Akka

scala

Big Data Innovation Conference (c)

Page 13: Demystifying Big Data with Scala and Akka

Big Data Innovation Conference (c)

Page 14: Demystifying Big Data with Scala and Akka

Akka

Big Data Innovation Conference (c)

Page 15: Demystifying Big Data with Scala and Akka

actor message passing

Add content

Big Data Innovation Conference (c)

Page 16: Demystifying Big Data with Scala and Akka

actor message passing

Add content

Big Data Innovation Conference (c)

Page 17: Demystifying Big Data with Scala and Akka

tooling?

Big Data Innovation Conference (c)

Page 18: Demystifying Big Data with Scala and Akka

Spark

Big Data Innovation Conference (c)

Page 19: Demystifying Big Data with Scala and Akka

Spark

Big Data Innovation Conference (c)

Page 20: Demystifying Big Data with Scala and Akka

Spark

Big Data Innovation Conference (c)

Page 21: Demystifying Big Data with Scala and Akka

Spark

Big Data Innovation Conference (c)

Page 22: Demystifying Big Data with Scala and Akka

Delite

DSL for Heterogenous Parallel Architectures

Big Data Innovation Conference (c)

Page 23: Demystifying Big Data with Scala and Akka

Delite

Big Data Innovation Conference (c)

Page 24: Demystifying Big Data with Scala and Akka

Delite

Big Data Innovation Conference (c)

Page 25: Demystifying Big Data with Scala and Akka

decooda liquid data platform

Big Data Innovation Conference (c)

Page 26: Demystifying Big Data with Scala and Akka

Real time parallel execution

Big Data Innovation Conference (c)

Page 27: Demystifying Big Data with Scala and Akka

Big Data Innovation Conference (c)

Page 28: Demystifying Big Data with Scala and Akka

Big Data Innovation Conference (c)

Page 29: Demystifying Big Data with Scala and Akka

conclusions

Big Data Innovation Conference (c)

Multiple real time alternativesbeyond Hadoop

Scala and Akka facilitatinggrowth

Page 30: Demystifying Big Data with Scala and Akka

[email protected]@knolspeakfb - KnoldusSoftware

Big Data Innovation Conference (c)