Darwin Schweitzer

Big Data Analytics

C LO U D

D ATA A I

Organizations that harness Data, Cloud, and AI outperform

Security and performanceFlexibility of choiceReason over any data, anywhere

Data warehouses

Data Lakes

Operational databases

Hybrid

Data warehouses

Data Lakes

Operational databases

SocialLOB Graph IoTImageCRM

T H E M O D E R N D A T A E S T A T E

Classic AnalyticsTransaction-driven

Cloud-born AnalyticsEvent-driven

COSMOS DB Databricks

SQL DB/DW and Analysis Services

Big Data & Advanced Analytics in Azure

Model & ServePrep & Train

Databricks

HDInsight

Data Lake AnalyticsCustom

apps

Sensors

and devices

Store

Blobs

Data Lake

Ingest

Data Factory(Data movement, pipelines & orchestration)

Machine

Learning

Cosmos DB

SQL Data

Warehouse

Analysis Services

Event Hub

IoT Hub

SQL Database

Analytical dashboards

Predictive apps

Operational reports

Intelligence

B I G D ATA & A D VA N C E D A N A LY T I C S AT A G L A N C E

Business

apps

1001

SQLKafka

Azure DatabricksPowered by Apache Spark

A fast, easy and collaborative Apache® Spark™ based analytics platform optimized for Azure

Best of Databricks Best of Microsoft

Designed in collaboration with the founders of Apache Spark

One-click set up; streamlined workflows

Interactive workspace that enables collaboration between data scientists, data engineers, and business analysts.

Native integration with Azure services (Power BI, SQL DW, Cosmos DB, Blob Storage)

Enterprise-grade Azure security (Active Directory integration, compliance, enterprise -grade SLAs)

Get started quickly by launching

your new Spark environment with

one click.

Share your insights in powerful

ways through rich integration with

Power BI.

Improve collaboration amongst

your analytics team through a

unified workspace.

Innovate faster with native

integration with rest of Azure

platform

Simplify security and identity control

with built-in integration with Active

Directory.

Regulate access with fine-grained user

permissions to Azure Databricks’

notebooks, clusters, jobs and data.

Build with confidence on the trusted

cloud backed by unmatched support,

compliance and SLAs.

Operate at massive scale

without limits globally.

Accelerate data processing with

the fastest Spark engine.

ENHANCE PRODUCTIVITY BUILD ON THE MOST COMPLIANT CLOUD SCALE WITHOUT LIMITS

Optimized Databricks Runtime Engine

DATABRICKS I/O SERVERLESS

Collaborative Workspace

Cloud storage

Data warehouses

Hadoop storage

IoT / streaming data

Rest APIs

Machine learning models

BI tools

Data exports

Data warehouses

Azure Databricks

Enhance Productivity

Deploy Production Jobs & Workflows

APACHE SPARK

MULTI-STAGE PIPELINES

DATA ENGINEER

JOB SCHEDULER NOTIFICATION & LOGS

DATA SCIENTIST BUSINESS ANALYST

Build on secure & trusted cloud Scale without limits

Demo Azure Databricks

https://github.com/Azure/data-ai-iot/tree/master/databricks

https://gallery.azure.ai/Solution/Azure-Databricks-Spark-Streaming-4

Additional Sample Azure Databricks Notebooks

Use Cases

Business / custom apps

(Structured)

Logs, files and media

(unstructured)

Azure storage

Polybase

Azure SQL Data Warehouse

Data factory

Data factory

Azure Databricks

(Spark)

Analytical dashboards

Model & ServePrep & TrainStoreIngest Intelligence

Web & mobile appsAzure Databricks

(Spark Mllib,

SparkR, SparklyR)

Azure Cosmos DB

Business / custom apps

(Structured)

Logs, files and media

(unstructured)

Azure storage

Polybase

Azure SQL Data Warehouse

Data factory

Data factory

Analytical dashboards

Model & ServePrep & TrainStoreIngest Intelligence

Unstructured data

Azure storage

Polybase

Azure SQL Data Warehouse

Azure HDInsight

(Kafka)

Azure Databricks

(Spark)

Analytical dashboards

Model & ServePrep & TrainStoreIngest Intelligence

Pricing @General Availability

Release Standard Premium

General availability Data analytics: $0.40/DBU/hr + VM

Data engineering: $0.20/DBU/hr + VM

Includes:

Compliance: SOC2, HIPAA, AAD Integration

Data connectors (Blob Storage, Data Lake, SQL DW,

Cosmos DB, Event Hub), GPU Instances

Data analytics: $0.55/DBU/hr + VM

Data engineering: $0.35/DBU/hr + VM

Includes:

Everything from standard +

Fine grained control for notebooks & clusters, structured

data controls

JDBC/ODBC endpoint

Governance logs

.NET Integration

Integrations with Azure apps like Power BI, etc.

Engage Microsoft experts for a workshop to help identify

high impact scenarios

Try a Quickstart or Tutorial at:

https://docs.microsoft.com/en-us/azure/azure-databricks/

https://gallery.azure.ai/Solution/Azure-Databricks-Spark-Streaming-4

https://github.com/Azure/data-ai-iot

Learn more about Azure Databricks www.azure.com/databricks

https://github.com/Azure/data-ai-iot

Accelerate learning and using Try → Learn → Build:

Try• Demos (IDEA)

• Introduce

• Demo

• Explain

• Attend

Learn• GitHub Samples

• Solution Templates

• Data Science VM

Build

• Documentation &

Solution Architectures

Transform individuals to transform business

Transformation of individuals → teams → organizations

Darwin Schweitzer | WW INTELLIGENT CLOUD – Big Data / AI Advanced Workload LeadWorldwide Commercial Business (WCB) – Intelligent Cloud(425) 638-9068 | darsch@microsoft.com | @DataSnowman | GitHub DataSnowmanPlease check out Data and AI and IoT resources at https://github.com/Azure/data-ai-iot

Thank you

Appendix