The Big Data Combat
-
Upload
sarajstanford -
Category
Documents
-
view
10 -
download
0
description
Transcript of The Big Data Combat
![Page 1: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/1.jpg)
THE BIG DATA COMBAT
SPEC INDIA
![Page 2: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/2.jpg)
WHAT IS IT ALL ABOUT?
The size of the data generated in the world explodes
Data is being constantly gathered by various sources
The Data keeps increasing many folds every day
Technology gears up to combat BIG DATA
![Page 3: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/3.jpg)
BIG DATA
A large unstructured big volume data set
Too complex to be handled by commonly used database management systems
RDBMS DBMS
Big data uses statistical inference to determine parameters from a large volume of data
Regressions Nonlinear Relationships Data Dependencies
![Page 4: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/4.jpg)
SOURCES OF DATA TODAY
The Internet
Mobile Devices
Remote Sensing
Software Logs
Cameras
Microphones
Radio Frequency Identification (RFID)
Wireless Sensor Networks
![Page 5: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/5.jpg)
THE CHALLENGESIn the Growth & Digitization of This Global Information
Storage
![Page 6: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/6.jpg)
VOLUME
BIG Volumes The unceasing increase in the amount of data Created everyday Overwhelming in size
![Page 7: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/7.jpg)
VELOCITY
Velocity @ The Speed Of Light… Speed of Data in and out Transactions Business Analysis
![Page 8: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/8.jpg)
VARIETY
Variety Spices up Big Data too
Data Types Data Sources Challenges in
Capture Curate Store
Interpretation Meaningful Analys Search Data Visualization
![Page 9: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/9.jpg)
BIG DATA ROLLOUT
Steps for a mature and meaningful data set Data Profiling Data Cleansing Data Integration of structured and unstructured data Data Merging Data Migration Data Replication ETL / ELT / ETLT Design and Development Interfacing legacy systems with the modern approach
![Page 10: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/10.jpg)
BIG DATA TOOLS
Hadoop, a distributed file system
MapReduce, a framework for data abstractions
Hive for data summarization and adhoc queries
Pig for parallel processing
HBase, a structured storage for large tables
Sqoop for data integration of Hadoop with RDBMS
Flume for data transfers of log data to centralized data repositories
![Page 11: The Big Data Combat](https://reader036.fdocuments.us/reader036/viewer/2022082817/568128da550346895d8b7b37/html5/thumbnails/11.jpg)
IT IS BIG & IS GETTING BIGGER
TOO!
Visit
http://www.spec-india.com/services/bi-bigdata-database-services.html
to request a FREE POC to Test Drive our services