Introduction to Analysis Introduction to Analysis Services and OLAP Services and OLAP TechnologyTechnology

Tom Conlon and Rob ZareTom Conlon and Rob Zare

Program ManagersProgram ManagersSQL Server Business Intelligence UnitSQL Server Business Intelligence Unit

Microsoft Corp.Microsoft Corp.

TopicsTopics• What is OLAP? What is OLAP? • 3 Why’s of OLAP3 Why’s of OLAP• Performance case studyPerformance case study• Architecture summaryArchitecture summary• Lots of demos!Lots of demos!

– ExcelExcel– Office web componentOffice web component– Microsoft data analyzerMicrosoft data analyzer– Building a cube with Analysis ManagerBuilding a cube with Analysis Manager

The First Database

The First Data Warehouse

The First DBA

OLTP Versus Data WarehousingOLTP Versus Data Warehousing

• OLTP supportsOLTP supports– Granular transactionsGranular transactions– Real time production systemsReal time production systems– Current, changing dataCurrent, changing data

• Data warehousing supportsData warehousing supports– Summarized queriesSummarized queries– Consistent, heterogeneous dataConsistent, heterogeneous data– Voluminous, historical, stable dataVoluminous, historical, stable data

OperatingOperatingBusinessBusiness

ManagingManaging BusinessBusiness

What is OLAP?What is OLAP?

• OOn n LLine ine AAnalytical nalytical PProcessingrocessing• OLAP is FASMIOLAP is FASMI

– FFastast

– AAnalysisnalysis

– SSharedhared

– MMultidimensionalultidimensional

– IInformationnformation

The 3 Why’s of OLAPThe 3 Why’s of OLAP

• Intuitive Metadata modelIntuitive Metadata model for business users for business users– Just show them the data, and very quickly they Just show them the data, and very quickly they

‘just get it’.‘just get it’.• Fast queriesFast queries

– Orders of magnitude improvement over Orders of magnitude improvement over relational.relational.

– Can completely change usability of systemCan completely change usability of system• Dedicated Language for AnalysisDedicated Language for Analysis and and

Modeling (MDX)Modeling (MDX)– Built-in functions for handling time and group Built-in functions for handling time and group

comparisonscomparisons

OLAP Brief historyOLAP Brief history

• 1970s Multidimensional database1970s Multidimensional database– Boutique industry of specialistsBoutique industry of specialists– Very expensive, rare deploymentVery expensive, rare deployment– Considered very “advanced”Considered very “advanced”

• 1980s EIS1980s EIS– Very simple UIVery simple UI– Still expensive, uncommon, tailored for a few Still expensive, uncommon, tailored for a few

usersusers

• 1989 – named OLAP by E. F. Codd1989 – named OLAP by E. F. Codd

OLAP in the 90s – Key ChangesOLAP in the 90s – Key Changes

• Orders of magnitude decline in disk Orders of magnitude decline in disk storage pricesstorage prices

• Internet and intranet is ubiquitousInternet and intranet is ubiquitous• Key changes in corporate cultureKey changes in corporate culture

– ‘‘Analysts’ at all levels in corporationAnalysts’ at all levels in corporation– Give decision makers direct access to data, Give decision makers direct access to data,

cannot wait for IT to deliver a canned reportcannot wait for IT to deliver a canned report

• 1998 – Microsoft Analysis Services1998 – Microsoft Analysis Services– OLEDB for OLAP StandardOLEDB for OLAP Standard

OLAP ArchitecturesOLAP Architectures

• Multidimensional view of dataMultidimensional view of data• MOLAP, ROLAP, HOLAPMOLAP, ROLAP, HOLAP

Why #1: Why #1: Intuitive MetadataIntuitive Metadata

FASFASMMII

OLAP ObjectsOLAP Objects

• CubesCubes• DimensionsDimensions• HierarchiesHierarchies• LevelsLevels• MeasuresMeasures• MembersMembers• PropertiesProperties

ProductProductPeasPeas CornCorn BreadBread MilkMilk BeerBeer

MMaarrkkeett

BosBos

NYCNYC

ChiChi

SeaSeaJanJan

MarMarFebFeb

TimeTime

Units of beer sold in Units of beer sold in Boston in JanuaryBoston in January

CubeCube

300300 100100 225225 425425 775775

600600 210210 345345 900900 6565

300300 125125 175175 400400 215215

415415 690690 630630 325325 510510

DimensionsDimensions

• Edges of a cubeEdges of a cube• Usually entities familiar to the end userUsually entities familiar to the end user

– Customer, product, time, etcCustomer, product, time, etc

• Typically 5 to 10 per cube, but sometimes Typically 5 to 10 per cube, but sometimes in the 10’s or even morein the 10’s or even more

• HierarchicalHierarchical– E.g. Year/month/day, Country/State/CityE.g. Year/month/day, Country/State/City

Hierarchies & LevelsHierarchies & Levels

• USAUSA– WAWA

• SeattleSeattle– 9810998109

• Tom ConlonTom Conlon

• 20012001– Quarter1Quarter1

• FebruaryFebruary– February20February20

Country

State

City

Zip

Customer

YearQuarter

Month

Day

MeasuresMeasures

• ““Quantities” stored in the cells of cubesQuantities” stored in the cells of cubes• Generally numeric, but not alwaysGenerally numeric, but not always• Examples:Examples:

– Units soldUnits sold– PricePrice

OLAP puts Data In The Hands Of OLAP puts Data In The Hands Of UsersUsers

End user analysis activitiesEnd user analysis activities

• PivotingPivoting– Swapping page/row/column layoutSwapping page/row/column layout

• SlicingSlicing– Select specific dimension members on an axisSelect specific dimension members on an axis

• DrilldownDrilldown– Navigate from summary to detail dataNavigate from summary to detail data

• Drill throughDrill through– Retrieve granular data from Fact TableRetrieve granular data from Fact Table

• CalculationsCalculations– Adding derived dimension members or measuresAdding derived dimension members or measures

• VisualizationVisualization– Charting, mapping, etc.Charting, mapping, etc.

Microsoft OfficeMicrosoft Office

demodemo

Excel and Office Web Component

Microsoft Data Analyzer

Why #2: OLAP is FastWhy #2: OLAP is FastFFASMIASMI

•Why is OLAP Fast?Why is OLAP Fast?•Case StudyCase Study

Aggregations – The Magic Behind Aggregations – The Magic Behind OLAP PerformanceOLAP Performance

• Calculate ahead of time the results of the Calculate ahead of time the results of the aggregate cellsaggregate cells– Avoid scanning the detailed data during user Avoid scanning the detailed data during user

interactioninteraction– Gain constant response time to queries Gain constant response time to queries

independent of DB size or logic complexityindependent of DB size or logic complexity

16 64 256 10244096

16384

65536

0

10000

20000

30000

40000

50000

60000

70000

2 3 4 5 6 7 8

Data Explosion SyndromeData Explosion Syndrome

Number of Dimensions

Nu

mbe

r of

Agg

rega

tion

sOLAP Problem: Data ExplosionOLAP Problem: Data Explosion

(4 levels in each dimension)

Aggregation DesignAggregation Design

Fact TableFact Table

Highest level of aggregationHighest level of aggregation

Most detailed Most detailed AggregationsAggregations

Show me all sales for all products for all . . .Show me all sales for all products for all . . .

Aggregation DesignAggregation Design

Fact TableFact Table

Highest level of aggregationHighest level of aggregationShow me all sales for all products for all . . .Show me all sales for all products for all . . .

Aggregation DesignAggregation Design

Fact TableFact Table

Mo

nth

Mo

nth

ProductsProducts

Qu

arterQ

uarter

Pro.Pro. Family Family

Qu

arterQ

uarter

ProductProductMo

nth

Mo

nth

Pro. Pro. Family Family

Using Analysis Using Analysis Manager to Design a Manager to Design a CubeCube

demodemo

T3 - Data OverviewT3 - Data Overview

• Expanded version of a real production databaseExpanded version of a real production database• DimensionsDimensions

– Market (80 Markets)Market (80 Markets)– Period (268 Weeks, 67 Months, 5 Years)Period (268 Weeks, 67 Months, 5 Years)– Product (710,000 Products, 130,000 Brands, 1000 Product (710,000 Products, 130,000 Brands, 1000

Classes, 500 Subgroups, 100 Groups, 9 Departments)Classes, 500 Subgroups, 100 Groups, 9 Departments)

• Eight fact tables: different levels of aggregationEight fact tables: different levels of aggregation– Therefore eight cubes, joined into one Virtual CubeTherefore eight cubes, joined into one Virtual Cube– Exactly the tables used by production system todayExactly the tables used by production system today

• Partitioning by monthsPartitioning by months

Storage RequirementsStorage Requirements

Table Rows MBytes Cubes Mbytes

detail_brand_* 1030093872 163377 Week_Brand 17835detail_prodmod_* 20222730 3182 Week_Class 235detail_subcat_* 11030444 1739 Week_Subgroup 31detail_upc_* 4881479622 793767 Week_Item 434670month_brand_* 294967010 47055 Month_Brand 4862month_prodmod_* 5082050 802 Month_Class 66month_subcat_* 2725398 433 Month_Subgroup 29month_upc_* 1428626606 225316 Month_Item 24486

Total 7674227732 1235670 Market Research 482214

7.7 Billion 1.2 TB 471 GB

Table storage (relational) Cube storage

39%

PerformancePerformance

• ProcessingProcessing– 7.7 billion rows, 50 hours7.7 billion rows, 50 hours– 153 million rows/hr153 million rows/hr– 42K rows/sec42K rows/sec– 60-70% CPU utilization60-70% CPU utilization

• QueryingQuerying– 50-user workload, 1350 queries, 30-sec think time50-user workload, 1350 queries, 30-sec think time– Cold cacheCold cache– Median response 0.08 sec, mean 1.2 secMedian response 0.08 sec, mean 1.2 sec– Low CPU load - we didn’t have enough queries!Low CPU load - we didn’t have enough queries!– You’ve got to experience this to appreciate it!You’ve got to experience this to appreciate it!

Why #3: MDX Language for Why #3: MDX Language for AnalysisAnalysis

FFAASMISMI

What is MDXWhat is MDX

• MDX = Multi Dimensional ExpressionsMDX = Multi Dimensional Expressions• A syntax for modeling and querying an A syntax for modeling and querying an

OLAP databaseOLAP database• Part of the OLE DB for OLAP SpecPart of the OLE DB for OLAP Spec• Supported by multiple providers (OLAP Supported by multiple providers (OLAP

Services, TM1, SAS, WhiteLight, SAP…)Services, TM1, SAS, WhiteLight, SAP…)• It is the key for all advanced analytical It is the key for all advanced analytical

capabilities of OLAP Services.capabilities of OLAP Services.

Typical MDX ExpressionsTypical MDX Expressions

• Modeled after SQLModeled after SQL

SelectSelect

{[Measures].[Store Sales],[Measures].[Unit Sales]} on 0{[Measures].[Store Sales],[Measures].[Unit Sales]} on 0

from Salesfrom Sales

where ([Time].[1997], [Customers].[USA])where ([Time].[1997], [Customers].[USA])

Typical MDX ExpressionsTypical MDX Expressions

• Share of TotalShare of Total[Sales]/(Sales,Product.CurrentMember.Parent)[Sales]/(Sales,Product.CurrentMember.Parent)

• Sales growth from the same period last year:Sales growth from the same period last year:([Sales], [Time].CurrentMember) – ([Sales], [Time].CurrentMember) – ([Sales], ParallelPeriod([Year], 1)([Sales], ParallelPeriod([Year], 1)

• 3 Periods moving average3 Periods moving averageAvg( Time.CurrentMember.Lag(2): Time.CurrentMember, Avg( Time.CurrentMember.Lag(2): Time.CurrentMember,

Measures.Inventory )Measures.Inventory )

• YTD sales:YTD sales:Sum( YTD(Time.CurrentMember), [Sales] )Sum( YTD(Time.CurrentMember), [Sales] )

Security in OLAP CubesSecurity in OLAP Cubes

SecuritySecurity

• Integrated NT securityIntegrated NT security• Roles: groups of NT accounts with the Roles: groups of NT accounts with the

same privilegessame privileges• Database RoleDatabase Role

– Defines the list of usersDefines the list of users

• Cube RoleCube Role– Cube access: Read, WriteCube access: Read, Write– Cell security: Read, Write, Contingent ReadCell security: Read, Write, Contingent Read

All Drinks 665 200 75 55 70 465 125 145 195

Soft Drinks 530 140 45 40 55 390 100 125 165

Coke 160 35 10 10 15 125 35 40 50

Pepsi 175 50 15 15 20 125 30 40 55

Mountain Dew 195 55 20 15 20 140 35 45 60

Alcoholic Drinks 135 60 30 15 15 75 25 20 30

Beer 60 20 10 5 5 40 10 10 20

Wine 75 40 20 10 10 35 15 10 10

2001

20

01 -

Q1

200

1 -

Jan

200

1 -

Feb

200

1 -

Mar

20

01 -

Q2

2001

- A

pr

2001

- M

ay

200

1 -

Jun

Heterogeneous coordinates systemHeterogeneous coordinates system

SummarySummary• OLAP is about empowering analysts OLAP is about empowering analysts

within the corp with direct access to datawithin the corp with direct access to data• Intuitive business UIIntuitive business UI

– Drill down, up, through, acrossDrill down, up, through, across– Slice and dice, Formatting and graphicsSlice and dice, Formatting and graphics

• FAST response timeFAST response time• Multidimensional Expression Language Multidimensional Expression Language

(MDX) for queries(MDX) for queries• Rapid adoption in 90s and beyondRapid adoption in 90s and beyond

Architecture – Single ServerArchitecture – Single Server

OLAPOLAPStoreStore

Ap

plicatio

nA

pp

lication

AD

O

AD

O

MD

MD

Pivo

tTab

le P

ivotT

able

Service

Service

OL

ED

B fo

r O

LE

DB

for

OL

AP

OL

AP

AnalysisAnalysisServerServer

AnalysisAnalysisManagerManager

DSODSO

SQL ServerSQL ServerDataData

WarehouseWarehouse

OtherOtherOLE DBOLE DB

ProvidersProviders

OLEDOLEDBB

CLIENTSERVER

Client Access ArchitectureClient Access Architecture

OLAP OLAP ServerServer

Pivot Table Pivot Table ServiceService

(OLEDB for (OLEDB for OLAP)OLAP)

ADOMDADOMDVBVB

VBAVBA

C++C++

ExcelExcel

Office Web ComponentOffice Web Component

Don’t forget to complete the Don’t forget to complete the on-line Session Feedback form on-line Session Feedback form on the Attendee Web siteon the Attendee Web site

https://web.mseventseurope.com/teched/https://web.mseventseurope.com/teched/

© 2002 Microsoft Corporation. All rights reserved.© 2002 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.