Building the Modern Analytics Architecture

Karen López

DBI-B210

Karen Lopez

InfoAdvisors.com @datachick

Senior Project Manager

& Architect

Love Your Data

Modern Analytics Architectural ComponentsArchitecting versus buildingCost, benefit, risk

Building the Architecture in Microsoft AzureAnalytics

Importing external dataUsing Power Query & HDInsight for analysisUsing Power View

Q&A and Call to Action

What we will be doing

Modern Analytics Architectural Components &Concepts

Why Hadoop?BIG DATA

Unstructured data

Cloud

Analytics

Scale

Data LakeData Reservoir

Variable Data

Data versus ProcessingSchemaless

Hadoop Ecosystem (a Zoo)

Distributed Storage(HDFS)

Query(Hive)

Distributed Processing

(MapReduce)

Scripting(Pig)

NoSQ

L Data

base

(HB

ase

)

Metadata(HCatalog)

Data

Inte

gra

tion

( OD

BC

/ SQ

OO

P/ R

EST)

Rela

tion

al

(SQ

L Serve

r)

Machine Learning(Mahout)

Graph(Pegasus

)

Stats processin

g(RHadoo

p)

Eve

nt Pip

elin

e(Flu

me)

Active Directory (Security)

Monitoring & Deployment

(System Center)

C#, F#, .NET

JavaScript

Pipelin

e / W

orkfl

ow

(Oozie

)

Azure Storage Vault (ASV)

PD

W Po

lybase

Busin

ess

Inte

lligence

(E

xcel,

Pow

er V

iew

, SS

AS)World's Data

(Azure Data Marketplace)

Eve

nt

Drive

n

Proce

ssing

LegendRed = Core HadoopBlue = Data processingPurple = Microsoft integration points and value addsYellow = Data MovementGreen = Packages

Via Hadoop Ecosystem pptx, Cindy Gross. Used with permission

Shuffle and sort dataGet from large data to smaller dataParallel processing

MapReduce

SQL-like query language Abstraction on top of MapReduceMetastructure on top of HDFS

Hive

Traditional DW Architecture

OLTP DB

OLTP DB

OLTP DB

EDWstaging

On Premises

Data

Mart

Data

Mart

Modern DW Architecture

OLTP DB

OLTP DB

OLTP DB

EDWstaging

On Prem

Microsoft Azure

Analytics Mart

Data

Mart

On Prem

Scale up versus scale out Nodes, not beefier servers Data stays, clusters come and go Quick Create for playing Custom Create for architecting a solution

Scale and Nodes

“Every design decision should include cost, benefit and risk”

- Karen Lopez

Building the Architecture in Microsoft Azure

Blob StorageStorage

Container

HDInsight Cluster

HDInsight Demo Components

PowerShell Azure SQL DB

Demo: HDInsight Components

….Let’s get started

Let’s talk billing….Per hourPer node

head nodesecurity nodedata nodes

Data egress, not ingressGeo-redunancy

HDInsight AdvantagesSeparate Blob Storage from ClusterBilling

Pay as you go6 or 12 month plans

Tools you knowSkills you haveSkills users have

Analysis

The Data Story

*http://www.dsireusa.org/incentives/incentive.cfm?Incentive_Code=US37F

http://www.irs.gov/file_source/pub/irs-soi/zipcode08flds.xls

IRSEnergy Tax

Credit Data*

Retail Data Warehouse

HDInsight

Power Query & Power

View

Power BI Stack

Power Query

Power View

Power Pivot

Power Map

Demo: Analysis

Bringing it all together

HDInsight + SQL Server + External Data + Power BI

Wrap up

Breakout Sessions DBI-B213 Details of Microsoft SQL Server 2012 Parallel Data Warehouse Appliance Update 1

DBI-B328 From Zero to Data Insights Using HDInsight on Microsoft Azure

DEV-B341 Using Big Data to Advance Your Cloud Service + Device Solutions

Related content

LabsDBI-H306 What's New in Microsoft SQL Server 2014 for DBAs

Find Me Later At MSE Data Platform

Track resources

Download Microsoft SQL Server 2014 http://www.trySQLSever.com

Try out Power BI for Office 365! http://www.powerbi.com

Sign up for Microsoft HDInsight today! http://microsoft.com/bigdata

Resources

Learning

Microsoft Certification & Training Resources

www.microsoft.com/learning

msdn

Resources for Developers

http://microsoft.com/msdn

TechNet

Resources for IT Professionals

http://microsoft.com/technet

Sessions on Demand

http://channel9.msdn.com/Events/TechEd

Complete an evaluation and enter to win!

Evaluate this session

Scan this QR code to evaluate this session.

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries.The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.