Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data...

15
<Insert Picture Here> Best Practices for Data Warehousing

Transcript of Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data...

Page 1: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

<Insert Picture Here>

Best Practices for Data Warehousing

Page 2: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

2

<Insert Picture Here>

Agenda – Best Practices for DW-BI

• Best Practices in Data Modeling• Best Practices in ETL• Best Practices in Reporting

Page 3: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

3

Best Practices in Data Modeling

Page 4: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

4

ProductsProductsDimensionDimension

TimeTimeDimensionDimension

Oracle Order Management & Oracle Order Management & Fulfillment AnalyticsFulfillment Analytics

Q. How many of my top customers bought products from my worst suppliers?

Q. How many of my top customers bought products from my worst suppliers?

Sales Orders Fact Table

Dim TableDim TableDim TableDim TableDimensionDimensionTablesTables

Support for Cross-Application Analysis

Supply Chain AnalyticsSupply Chain Analytics

Purchase Orders

Fact Table

Dim TableDim TableDim TableDim TableDimensionDimensionTablesTables

Fundamental requirement that dimensions be common (conformed)

Page 5: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

5

Features:• Conformed dimensions • Transaction data stored in most

granular fashion• Tracks full history of changes• Prebuilt and extensible• Built for speed

Integrated Enterprise Analytics Data Model

Benefits: • Enterprise-wide business analysis

(across entire value chain)• Access summary metrics or drill to

lowest level of detail• Accurate historical representations

Ser

vice

S

ervi

ce

Customers

Sal

esS

ales

Mar

keti

ng

Mar

keti

ng

Dis

trib

uti

on

Dis

trib

uti

on

Fin

ance

Fin

ance

HR

/ W

ork

forc

eH

R /

Wo

rkfo

rce

Op

erat

ion

sO

per

atio

ns

Pro

cure

men

tP

rocu

rem

ent

Customers

Customers

Suppliers

Suppliers

Suppliers

Page 6: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

6

The Result

From this :

To this :

- Fewer, larger database tables rather than many smaller ones

- Same piece of data appearing in several locations

- Reduces need for join paths

- Structure is denormalized for performance

Page 7: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

7

Best Practices in ETL

Page 8: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

8

Ad

min

istr

atio

n

Me

tad

ata

PresentationDashboards by Role

Reports, Analysis / Analytic Workflows

Metrics / KPIs

Logical Model / Subject Areas

Physical Map

BI Server

Direct Access to

Source Data

Data Warehouse /Data Model

DA

C

Federated Data Sources

SiebelOracle SAP R/3 PSFT EDW

Other

ETL

Load Process

Staging Area

Extraction Process

DA

C

ETL Architecture – Best Practice

Load

Load

Extr

act

Extr

act

SAPSAPPeopleSoftPeopleSoft

Source Independent Layer

Staging TablesStaging Tables

Source Dependent Extract

OtherOtherSiebel Siebel OLTPOLTP OracleOracle

SpecialConnect

SpecialConnect

SQ

L

SQ

L

SQ

L

SQ

LA

pp

Layer

AB

AP

Ap

p L

ayer

DataDataWarehouseWarehouse

Tra

nsfo

rTra

nsfo

rmm

Page 9: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

9

Use of a ETL platform

• Limited programming – GUI interface• Re-usable components• Easy data lineage tracking (where did data come from?)• Pseudo-documentation – fast ramp-up for new resources• Can build, test & implement the data flows more quickly

Page 10: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

10

ETL Framework – Best Practices• Generates surrogate key

• Does lookups for descriptions of code fields

• Does data driven updates – inserts for new rows, updates for old rows

• Reject Capture

• Keep track of effective dates and maintain history as required

• Handles Deletes

Page 11: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

11

Best Practices in Reporting

Page 12: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

12

The Semantic LayerA

dm

inis

trat

ion

Me

tad

ata

BI Presentation

ServicesDashboards by Role

Reports, Analysis / Analytic Workflows

Direct Access to

Source Data

Data Warehouse /Data Model

ETL

Load Process

Staging Area

Extraction Process

DA

C

Federated Data Sources

SiebelOracle SAP R/3 PSFT EDW

Other

• Multi-layered Abstraction• Separation of physical, logical and

presentation layers• Logical modeling builds upon complex

physical data structures• Logical model independent of physical

data sources, i.e. same logical model can be remapped quickly to another data source

• Metrics / KPIs

• Aggregate navigation

• Prebuilt hierarchy drills and cross dimensional drills

Metrics / KPIs

Logical Model / Subject Areas

Physical Map

BI Server

Page 13: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

Object SecurityWhat parts of the application can you see?

• Business Logic Object Security

ObjectSecurity

Presentation LayerPresentation Layer

Physical LayerPhysical Layer

Semantic Object LayerSemantic Object Layer

Controls access to Subject Areas, Tables and Columns in Presentation Layer

Limits access to Dashboards, Reports and Web Folders

• Web Object Security

Page 14: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

14

DW -BI Architecture – Best Practice

Extension of DW Schema for extension columns, additional tables, external sources, aggregates, indices, etc.

Extension of ETL for extension columns, descriptive flexfields, additional tables, external sources, etc.

Additional derived metrics, custom drill paths, exposing extensions in physical, logical and presentation layer, etc.

Additional dashboards and reports, guided and conditional navigations, iBots, etc.

Level ofEffort

Degree of Customization

Easy

Moderate

Intermediate

Involved

Dashboards & Reports

Semantic Metadata

Layer

DW Schema

ETL

Page 15: Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data Modeling Best Practices in ETL Best Practices in Reporting.

15

For any sales queries, please contact  [email protected]