Building the Modern Analytics Architecture
29
Transcript of Building the Modern Analytics Architecture
Building the Modern Analytics Architecture
Karen López
DBI-B210
Karen Lopez
InfoAdvisors.com @datachick
Senior Project Manager
& Architect
Love Your Data
Modern Analytics Architectural ComponentsArchitecting versus buildingCost, benefit, risk
Building the Architecture in Microsoft AzureAnalytics
Importing external dataUsing Power Query & HDInsight for analysisUsing Power View
Q&A and Call to Action
What we will be doing
Modern Analytics Architectural Components &Concepts
Why Hadoop?BIG DATA
Unstructured data
Cloud
Analytics
Scale
Data LakeData Reservoir
Variable Data
Data versus ProcessingSchemaless
Hadoop Ecosystem (a Zoo)
Distributed Storage(HDFS)
Query(Hive)
Distributed Processing
(MapReduce)
Scripting(Pig)
NoSQL Database(HBase)
Metadata(HCatalog)
Data Integration( ODBC / SQOOP/ REST)
Relational
(SQL Server)
Machine Learning(Mahout)
Graph(Pegasus
)
Stats processin
g(RHadoo
p)
Event Pipeline(Flum
e)
Active Directory (Security)
Monitoring & Deployment
(System Center)
C#, F#, .NET
JavaScript
Pipeline / Workflow
(Oozie)
Azure Storage Vault (ASV)
PDW Polybase
Business Intelligence
(Excel, Power View,
SSAS)World's Data (Azure Data Marketplace)
Event Driven
Processing
LegendRed = Core HadoopBlue = Data processingPurple = Microsoft integration points and value addsYellow = Data MovementGreen = Packages
Via Hadoop Ecosystem pptx, Cindy Gross. Used with permission
Shuffle and sort dataGet from large data to smaller dataParallel processing
MapReduce
SQL-like query language Abstraction on top of MapReduceMetastructure on top of HDFS
Hive
Traditional DW ArchitectureOLTP DB
OLTP DB
OLTP DB
EDWstaging
On Premises
Data
Mart
Data
Mart
Modern DW ArchitectureOLTP DB
OLTP DB
OLTP DB
EDWstaging
On Prem
Microsoft Azure
Analytics Mart
Data
Mart
On Prem
Scale up versus scale out Nodes, not beefier servers Data stays, clusters come and go Quick Create for playing Custom Create for architecting a solution
Scale and Nodes
“Every design decision should include cost, benefit and risk”
- Karen Lopez
Building the Architecture in Microsoft Azure
Blob StorageStorage
Container HDInsight
Cluster
HDInsight Demo Components
PowerShell Azure SQL DB
Demo: HDInsight Components….Let’s get started
Let’s talk billing….Per hourPer node
head nodesecurity nodedata nodes
Data egress, not ingressGeo-redunancy
HDInsight AdvantagesSeparate Blob Storage from ClusterBilling
Pay as you go6 or 12 month plans
Tools you knowSkills you haveSkills users have
Analysis
The Data Story
*http://www.dsireusa.org/incentives/incentive.cfm?Incentive_Code=US37F
http://www.irs.gov/file_source/pub/irs-soi/zipcode08flds.xls
IRSEnergy Tax
Credit Data*
Retail Data Warehouse
HDInsight
Power Query & Power
View
Power BI Stack
Power Query
Power View
Power Pivot
Power Map
Demo: Analysis
Bringing it all together
HDInsight + SQL Server + External Data + Power BI
Wrap up
Breakout Sessions DBI-B213 Details of Microsoft SQL Server 2012 Parallel Data Warehouse Appliance Update 1
DBI-B328 From Zero to Data Insights Using HDInsight on Microsoft Azure
DEV-B341 Using Big Data to Advance Your Cloud Service + Device Solutions
Related content
LabsDBI-H306 What's New in Microsoft SQL Server 2014 for DBAs
Find Me Later At MSE Data Platform
Track resourcesDownload Microsoft SQL Server 2014
http://www.trySQLSever.com
Try out Power BI for Office 365! http://www.powerbi.com
Sign up for Microsoft HDInsight today! http://microsoft.com/bigdata
ResourcesLearning
Microsoft Certification & Training Resourceswww.microsoft.com/learning
msdnResources for Developers
http://microsoft.com/msdn
TechNetResources for IT Professionals
http://microsoft.com/technet
Sessions on Demandhttp://channel9.msdn.com/Events/TechEd
Complete an evaluation and enter to win!
Evaluate this session
Scan this QR code to evaluate this session.
© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.
