Big data hype or reality
-
Upload
e2-partners -
Category
Technology
-
view
244 -
download
1
description
Transcript of Big data hype or reality
![Page 1: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/1.jpg)
![Page 2: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/2.jpg)
Agenda
• What is Big Data?
• Technology Radar
• Technologies in scope.
• Architecture
• Wanted!
• Next steps.
![Page 3: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/3.jpg)
The world of data is changing.
![Page 4: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/4.jpg)
Data has a chaotic nature.
![Page 5: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/5.jpg)
Big Data <> Big DataBig Data == Big in Data.
![Page 6: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/6.jpg)
Big Data = 4 V’s.
![Page 7: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/7.jpg)
Volume = Dealing with the size.
![Page 8: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/8.jpg)
Variety =
Handling the multiplicity of types, sources and formats.
![Page 9: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/9.jpg)
Velocity=
Reacting to the flood of information in the time required by the application.
![Page 10: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/10.jpg)
Veracity =
How can we cope with uncertainty, imprecision, missing values or untruths.
![Page 11: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/11.jpg)
Big Data 1.0=
Building the capabilities to process large dataIn support of their current operations
(efficiency improvement).
![Page 12: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/12.jpg)
Big Data 2.0=
What can I now do that I couldn’t do before, or do better then I could do before.
![Page 13: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/13.jpg)
Polyglot persistence
• Relational databases are not dead.
• Enterprises should expect multiple data-storage technologies for different applications.
• Even for a single application, polyglot persistence is good.
• Do not replace one database solution with another to expect wonders.
![Page 14: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/14.jpg)
![Page 15: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/15.jpg)
![Page 16: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/16.jpg)
Technologies in the picture
• Hadoop and technologies build on top of it.
• ElasticSearch.
• neo4J.
![Page 17: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/17.jpg)
Hadoop
• Apache Foundation
• Commercial solutions
• Hortonworks
• Cloudera
• MapR
![Page 18: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/18.jpg)
![Page 19: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/19.jpg)
![Page 20: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/20.jpg)
And many more...
![Page 21: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/21.jpg)
ElasticSearch
• Based on lucene.
• ElasticSearch is also the name of the company.
• Search, analyze and index in realtime.
• Distributed.
• High availability.
• Document-oriented.
• Schema free
• RESTful api
![Page 22: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/22.jpg)
![Page 23: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/23.jpg)
![Page 24: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/24.jpg)
neo4j
• Graph database.
• Ideal for metadata and relationships.
• Not for large content.
• Not for large graphs.
![Page 25: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/25.jpg)
![Page 26: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/26.jpg)
![Page 27: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/27.jpg)
Polyglot persistence
• Relational databases are not dead.
• Enterprises should expect multiple data-storage technologies for different applications.
• Even for a single application, polyglot persistence is good.
• Do not replace one database solution with another to expect wonders.
![Page 28: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/28.jpg)
![Page 29: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/29.jpg)
![Page 30: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/30.jpg)
Next steps.
![Page 31: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/31.jpg)
Learning and case study group.
![Page 32: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/32.jpg)
I need datastores:- Openstreetmap.- NASA- ....
![Page 33: Big data hype or reality](https://reader033.fdocuments.us/reader033/viewer/2022051610/5487ba4ab47959e70c8b555a/html5/thumbnails/33.jpg)
Q&A