(ATD 9) Microsoft Big Data Platform
-
Upload
luka-lovosevic -
Category
Technology
-
view
112 -
download
10
description
Transcript of (ATD 9) Microsoft Big Data Platform
![Page 1: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/1.jpg)
Big Data: I Microsoft ima slona za utrkuLuka Lovošević, Antonio FaletarMicrosoft Hrvatska
• MICROSOFT HRVATSKA
![Page 2: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/2.jpg)
SadržajUvod u Big DataPregled MS platformeHadoopDemo
![Page 3: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/3.jpg)
Što je Big Data?
![Page 4: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/4.jpg)
MICROSOFT CONFIDENTIAL – INTERNAL ONLY
![Page 5: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/5.jpg)
Što je Big Data?Podaci koji su vam bitni, ali ih tradicionalnim alatimane možete procesirati.
VOLUME(Količina)
VARIETY (Struktura)
VELOCITY (Brzina, real-
time)
![Page 6: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/6.jpg)
Izvori podataka
Logovi Text
Pametne kuće Senzori
Vrijeme i lokacija RFID
Telemetrija Društvene mreže
![Page 7: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/7.jpg)
Big Data algoritmi
Analiza na društvenim mrežama
Slični artikli (npr. web shop) Real-time analiza Česti skupovi artikala
Reklamiranje na webu
Analiza povezanih pojmova
Sustavi preporukaKlastering (grupiranje)
c
![Page 8: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/8.jpg)
Microsoft Big Data platforma
![Page 9: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/9.jpg)
Microsoft Big Data platforma
Hadoop – HDInsight
(Windows ili Azure)
SQL Server 2012 Parallel Data Warehouse
SQL Server StreamInsight
Self-service BI alati
![Page 10: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/10.jpg)
Malo više o Hadoopu
![Page 11: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/11.jpg)
Što je Hadoop?Platforma za procesiranje velike količine podataka
Apache, open source
Google GFS i MapReduce
Visoko skalabilan i distribuiran
Commodity hardver
2013
Yahoo!
EnterpriseHadoop
Apache projekt
2004 2008 2010 20122006
![Page 12: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/12.jpg)
Hadoop arhitektura
![Page 13: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/13.jpg)
Node
NodeNode
Podaci
Node
MapReduce
![Page 14: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/14.jpg)
// Map Reduce function in JavaScript
var map = function (key, value, context) {var words = value.split(/[^a-zA-Z]/);for (var i = 0; i < words.length; i++) {
if (words[i] !== "")context.write(words[i].toLowerCase(),1);}}};
var reduce = function (key, values, context) {var sum = 0;while (values.hasNext()) {sum += parseInt(values.next());
}context.write(key, sum);};
NodeNode
NodeNode
Program
MapReduce
![Page 15: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/15.jpg)
Primjer za MapReduce
![Page 16: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/16.jpg)
Alati za uspješno Hadoopiranje
![Page 17: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/17.jpg)
Pig
Procesiranje i oblikovanjepodataka
ETL tool
MapReduce
![Page 18: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/18.jpg)
Hive
Strukturiranje podataka
SQL sintaksa
ODBC, Excel …
MapReduce
![Page 19: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/19.jpg)
MahoutBiblioteka gotovih algoritama
Strojno učenje (npr. clustering, recommendation, …)
MapReduce
![Page 20: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/20.jpg)
HDInsight
Hadoop
Programiranje u .NET-uSecurity, HA & managementPodrška za virtualizacijuIntegracija s Microsoft BI alatimaIsto iskustvo za on-premise i cloud
Hadoop za Windows ServerHadoop za Windows Azure
![Page 21: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/21.jpg)
Demo
Windows Azure HDInsight
![Page 22: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/22.jpg)
Hadoop 2.0
HortonWorks Stinger inicijativa
Tez (interactive) vs. batch
Streaming (Storm project), itd.
![Page 23: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/23.jpg)
ZaključakBig data trendHadoop de facto standardWindows Azure HDInsightOpen source
![Page 24: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/24.jpg)
Pitanja?
![Page 25: (ATD 9) Microsoft Big Data Platform](https://reader036.fdocuments.us/reader036/viewer/2022062419/5585753dd8b42a4c2c8b4e05/html5/thumbnails/25.jpg)
Hvala!