Windows Azure Conference 2014 Real Time BI with Windows Azure.

Windows Azure Conference 2014

Deepthi AnantharamTechnology Evangelist

Real Time BI with Windows Azure


Agenda

Why Cloud BI Change Data Capture – A BI Scenario Cloud BI Self Service Overview


Reduced Organizational IT costs

Built-in scalability

High Availability

Maximum “future proofing”

Why move to the Cloud ?

This applies to Business Intelligence Solutions too

Addresses several key concerns in DW / BI hardware estimates

Reduces significant hardware costs and associated planning effort

Potentially shortens implementation time


Common Cloud Services Model Infrastructure as a Service (IaaS) Platform as a Service (Paas) Software as a Service (SaaS)


Business Intelligence in the CloudPublic Cloud Delivery Options

IaaS

Hosted Virtual

Server DB

PaaS

Build Your Own Platform

SaaS

BI as a “service”

apps• Deploy your own Solution in the Cloud• Low upfront costs (licensing of VM, Software etc)• No cloud based horizontal Scaling; must be done in DB and Application architecture

• Cloud based platform services• Low cost, pay per use• Allows / requires custom development to some extent

• Pre Packaged BI Solutions• “Easy” management and Implementation• No design flexibility• Custom apps are off the table


Business Intelligence in the CloudOther delivery options

Private Cloud

Hybrid Cloud

• Offers the most control but also Greater cost• Design your own solution• Manage your own cloud• An option if your organization Owns and maintains a private cloud

• Combination of Private Cloud and Public Cloud• Leverage the public for “spikes” in either availability Or Processing • More complicated solution to accomplish load balance


BI Lifecycle

Data Sources Data Preparation Data Storage Data Analysis Data Access

OLTP Database

ERP Database

Departmental Database

Extract

Transform

Load

Filter

Cleanse

Schedule

Data Warehouse

Data Mart

OLAP

In-Memory Analytics

Data Mining

Reports- Tablix- Drill- Visualisations

Graphs- Charts- Ad-hoc- Dynamic

Dashboard- Mash-up- Collaborate- Search


Real-time BI Platform - CDC Overview

Change-Data-Capture is a mechanism to trickle feed changes in data from a source database to multiple destinations in near real-time.

Approaches such as Replication (which are trigger based) can induce performance overhead on source systems and be intrusive in implementation.

Change-Data-Capture (which is log based) mitigates the above by capturing changes directly from source system logs and propagating the change delta.


• SELECT [name], database_id, is_cdc_enabled FROM sys.databases

• EXEC sys.sp_cdc_enable_db

• EXEC sys.sp_cdc_enable_table @source_schema = N'dbo', @source_name = N'TestTable1', @role_name = NULL GO

Setting up CDC

Creates CDC SchemaCreates CDC System Tables

Creates CDC JobsCreates CDC Functions


Enabling CDC

Returns list of captured columns

Returns list of tables which are enabled for CDC

Changes of all DDLs after CDC is enabled

Contains indexes associated with Change tables


What if

http://blogs.msdn.com/b/mattm/archive/2012/03/26/cdc-for-oracle-in-sql-server-2012.aspx


CDC Concept ArchitectureSQL Server 2012 has built-in CDC for Oracle source

ACTIVE Archive REDO Log File A_1

INACTIVE Archive REDO

Log File A_n

::

Oracle Log Miner

SQL Server CDC

Service SQL Server CDC Staging

Database

CDC SSIS Package

SQL Server / DB 2 / Oracle /

PDW

1. 2.3. 4.

5.

6.


Value Proposition

Technology Benefit• Low-overhead, granular (column level) change propagation in real-time• No installation, temporary staging or manual customization on source system

Implications• Elimination of delay resulting from DW refresh in batch cycles• Improved ROI on data analytics/BI investments due to greater relevance,

accuracy and completeness of information

Business Benefit• Competitive differentiation through real-time business visibility and improved

decision-making agility


AzureOn-Premise

On-Premise

14

Oracle(OLTP)

SQL Server DW

/ PDW / DB2 DW

etc.

Oracle(OLTP)

SQL Server DW

SQL Server CDC

SQL Server CDC

AzureOn-Premise

Oracle(OLTP)

SQL Server

DW

SQL Server CDC

Azure

Oracle(OLTP)

SQL Server DW

SQL Server CDC

Real-time BI Concept Architectures – Azure & On-Premise


Oracle OLTP -> DB2 DW Case-StudyPunjab National Bank with more than 119 years of strong existence and over 6000 branches including 5 overseas branches, 6460 ATMs, 5047 Business Correspondents and 2165 Ultra Small Branches, serves more than 82 million esteemed customers.

Microsoft TeamVivek Ravindra (STU Lead)Raghavendra Prabhu (PPS)Sandeep Katyal (SSP)Prajyot Saran (TSP)Himanshu Gupta (TSP) Arun Sharma (HCL PAM)Udaya Shankar (IMDM)Rakesh Sinha (IMDM)N. Raja (DCM)Sanjeev Sharma (CATM)

End-of-day bulk upload from Online Payments System (OPS) to Enterprise Data Warehouse (EDW) took 6 hours

Minimum 6 hour delay in reconciliation of actual cash-flow position in EDW thus resulting in delayed decision making

IT personnel have to work till mid-night to refresh EDW thus incurring additional cost and inconvenience

Business Requirements

Customer:Up-to-the minute refresh of EDW resulting in timely decision making and analytics

Reduction in labor cost due to elimination of extended work hours

Microsoft: $500K+ SQL License Revenue

Quantified Results


Collaborate in Power BI for Office 3651 in 4 enterprise customers on Office 365

Share Question

Q&A

Mobility

Analyze in Excel1 Billion Office Users

Analyze VisualizeDiscover

Scalable | Manageable | Trusted

A powerful new way to work with dataSelf-service business intelligence with familiar Excel and the power of the cloud


• Power Query• Easily search and access public and organizational data within excel

Power BI components


• Power Pivot• Tools for creating relationships• hierarchies • Calculated Measures• KPIs

• Power View

– Tools for creating interactive charts, graphs and other visual representatives of data

Power BI components


• Power Map• 3D Visualization tool for mapping, exploring and interacting with

geographic data

Power BI components


Share & Collaborate with Power BI for Office 365

Quickly create collaborative BI sitesBI Sites

Natural Language Questions

Data Refresh

Data Search

Mobile Access

Share Queries


• SharePoint Online sharing of Power BI content via new site type• Reports can be set to refresh data on schedule or demand• Power BI sites can be browsed from mobile device or laptop• Add reports to mobile favorites to view on tablet or smartphone

Power BI Sites


To conclude

Not all business intelligence needs are the same; if they were, this world would be a very boring place. With Microsoft's BI offerings in Azure, you have the flexibility to create the BI environment that best fits your needs. Whether you need traditional SQL Server Reporting Services, PowerPivot, or full-blown Power View, Azure provides the building blocks for your solution.

Windows Azure Conference 2014Windows Azure Conference 2014

Customer Interaction Zone (CiZ)

Any question on Anything Azure!

Book your slot now! @ the registration desk at the CiZ

Windows Azure Conference 2014Windows Azure Conference 2014

© 2014 Microsoft Corporation. All rights reserved. Microsoft, Windows, and other product names are or may be registered trademarks and/or trademarks in the U.S. and/or other countries. The information herein is for informational purposes only and represents the current view of Microsoft Corporation as of the date of this presentation. Because Microsoft must respond to changing market conditions, it should not be interpreted to be a commitment on the part of Microsoft, and Microsoft cannot guarantee the accuracy of any information provided after the date of this presentation. MICROSOFT MAKES NO WARRANTIES, EXPRESS, IMPLIED OR STATUTORY, AS TO THE INFORMATION IN THIS PRESENTATION.

Windows Azure Conference 2014EPG | MICROSOFT CONFIDENTIAL 25

Competing against Oracle Golden-Gate

Cost• Golden-gate is an Oracle DB option, that needs to be licensed for both Source and Target• What really hurts customers is that they need to pay Oracle even when the Source is Oracle DB!• SQL Server CDC (or our partner solution from Attunity) is not licensed by Source or Target

Risk• Golden-gate requires installation of services or temporary tables or temporary files on both Source and

Target• For mission-critical or sensitive environments, customers don’t want to risk installing anything

on the Production server• SQL Server CDC (or our partner solution from Attunity) does not require installation of services or

temporary tables or temporary files on either the Source or the Target

Ease-of-Use• SQL Server CDC can be set-up with a few clicks• Using SSIS CDC components, processing of change-data can be automated to refresh OLAP cubes

Windows Azure Conference 2014 Real Time BI with Windows Azure.

Documents

Transcript of Windows Azure Conference 2014 Real Time BI with Windows Azure.