One of the best ETL TESTING online training in India, Canada, Australia
Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data...
Transcript of Best Practices for Data Warehousing. 2 Agenda – Best Practices for DW-BI Best Practices in Data...
<Insert Picture Here>
Best Practices for Data Warehousing
2
<Insert Picture Here>
Agenda – Best Practices for DW-BI
• Best Practices in Data Modeling• Best Practices in ETL• Best Practices in Reporting
3
Best Practices in Data Modeling
4
ProductsProductsDimensionDimension
TimeTimeDimensionDimension
Oracle Order Management & Oracle Order Management & Fulfillment AnalyticsFulfillment Analytics
Q. How many of my top customers bought products from my worst suppliers?
Q. How many of my top customers bought products from my worst suppliers?
Sales Orders Fact Table
Dim TableDim TableDim TableDim TableDimensionDimensionTablesTables
Support for Cross-Application Analysis
Supply Chain AnalyticsSupply Chain Analytics
Purchase Orders
Fact Table
Dim TableDim TableDim TableDim TableDimensionDimensionTablesTables
Fundamental requirement that dimensions be common (conformed)
5
Features:• Conformed dimensions • Transaction data stored in most
granular fashion• Tracks full history of changes• Prebuilt and extensible• Built for speed
Integrated Enterprise Analytics Data Model
Benefits: • Enterprise-wide business analysis
(across entire value chain)• Access summary metrics or drill to
lowest level of detail• Accurate historical representations
Ser
vice
S
ervi
ce
Customers
Sal
esS
ales
Mar
keti
ng
Mar
keti
ng
Dis
trib
uti
on
Dis
trib
uti
on
Fin
ance
Fin
ance
HR
/ W
ork
forc
eH
R /
Wo
rkfo
rce
Op
erat
ion
sO
per
atio
ns
Pro
cure
men
tP
rocu
rem
ent
Customers
Customers
Suppliers
Suppliers
Suppliers
6
The Result
From this :
To this :
- Fewer, larger database tables rather than many smaller ones
- Same piece of data appearing in several locations
- Reduces need for join paths
- Structure is denormalized for performance
7
Best Practices in ETL
8
Ad
min
istr
atio
n
Me
tad
ata
PresentationDashboards by Role
Reports, Analysis / Analytic Workflows
Metrics / KPIs
Logical Model / Subject Areas
Physical Map
BI Server
Direct Access to
Source Data
Data Warehouse /Data Model
DA
C
Federated Data Sources
SiebelOracle SAP R/3 PSFT EDW
Other
ETL
Load Process
Staging Area
Extraction Process
DA
C
ETL Architecture – Best Practice
Load
Load
Extr
act
Extr
act
SAPSAPPeopleSoftPeopleSoft
Source Independent Layer
Staging TablesStaging Tables
Source Dependent Extract
OtherOtherSiebel Siebel OLTPOLTP OracleOracle
SpecialConnect
SpecialConnect
SQ
L
SQ
L
SQ
L
SQ
LA
pp
Layer
AB
AP
Ap
p L
ayer
DataDataWarehouseWarehouse
Tra
nsfo
rTra
nsfo
rmm
9
Use of a ETL platform
• Limited programming – GUI interface• Re-usable components• Easy data lineage tracking (where did data come from?)• Pseudo-documentation – fast ramp-up for new resources• Can build, test & implement the data flows more quickly
10
ETL Framework – Best Practices• Generates surrogate key
• Does lookups for descriptions of code fields
• Does data driven updates – inserts for new rows, updates for old rows
• Reject Capture
• Keep track of effective dates and maintain history as required
• Handles Deletes
11
Best Practices in Reporting
12
The Semantic LayerA
dm
inis
trat
ion
Me
tad
ata
BI Presentation
ServicesDashboards by Role
Reports, Analysis / Analytic Workflows
Direct Access to
Source Data
Data Warehouse /Data Model
ETL
Load Process
Staging Area
Extraction Process
DA
C
Federated Data Sources
SiebelOracle SAP R/3 PSFT EDW
Other
• Multi-layered Abstraction• Separation of physical, logical and
presentation layers• Logical modeling builds upon complex
physical data structures• Logical model independent of physical
data sources, i.e. same logical model can be remapped quickly to another data source
• Metrics / KPIs
• Aggregate navigation
• Prebuilt hierarchy drills and cross dimensional drills
Metrics / KPIs
Logical Model / Subject Areas
Physical Map
BI Server
Object SecurityWhat parts of the application can you see?
• Business Logic Object Security
ObjectSecurity
Presentation LayerPresentation Layer
Physical LayerPhysical Layer
Semantic Object LayerSemantic Object Layer
Controls access to Subject Areas, Tables and Columns in Presentation Layer
Limits access to Dashboards, Reports and Web Folders
• Web Object Security
14
DW -BI Architecture – Best Practice
Extension of DW Schema for extension columns, additional tables, external sources, aggregates, indices, etc.
Extension of ETL for extension columns, descriptive flexfields, additional tables, external sources, etc.
Additional derived metrics, custom drill paths, exposing extensions in physical, logical and presentation layer, etc.
Additional dashboards and reports, guided and conditional navigations, iBots, etc.
Level ofEffort
Degree of Customization
Easy
Moderate
Intermediate
Involved
Dashboards & Reports
Semantic Metadata
Layer
DW Schema
ETL
15
For any sales queries, please contact [email protected]