Dimensional modelingowb11gr2 presentation

2011 © Trivadis

BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

WELCOME Dimensional Modeling

with Oracle Warehouse

Builder 11.2.

Maren Escherman

Date

Title of Presentation Insert Header & Footer

1

2011 © Trivadis

AGENDA

1. Introduction

Dimensional modeling

From the dimensional model to a relational star schema

2. Oracle Warehouse Builder: Dimensional Features

How to build a data mart

3. ETL Accelerators

Loading dimensions and cubes

Slowly Changing Dimensions

Invalid data (Orphan Management)

4. Performance Optimization

5. Summary

Date


2

2011 © Trivadis

Oracle’s Data Warehouse Reference Architecture

Date


3

2011 © Trivadis

Introduction – Dimensional Modeling

Design technique proposed by Ralph Kimball

became the predominant approach to designing

data marts

predictable and cost effective

Can be used for any physical form

relational or multidimensional

Understandability

easier to understand and more intuitive to query

Performance

optimized for data querying

Date


4

2011 © Trivadis

Dimensional Modeling 101

Dimensional modeling always uses the concepts of facts (measures),

and dimensions (context).

Dimensions are

groups of hierarchies and

the foundation of the fact table.

Examples for Dimensions are

Time (when)

Customer (by whom)

Geography (where)

Product (what)

Facts are additive figures such as quantity or cost per unit.

Date


5

2011 © Trivadis

Semantic Loss

Date


6

Zeit

Kalender_Hierarchy

Produkt

Product_Hierarchy

Dimensional Model:

Dimensions, Levels,

Hierarchies, Facts, …

Relational Star Schema:

Tables, Columns

2011 © Trivadis

Avoid Semantic Loss

Date


7

Zeit

Kalender_Hierarchy

Produkt

Product_Hierarchy

Dimensional Model:

Dimensions, Levels,

Hierarchies, Cubes, …

Relational Star Schema:

Tables, Columns

OWB knows both worlds…

automatically created by OWB

2011 © Trivadis

Dimensional Modeling Using OWB

Define, deploy, and load dimensional objects

explicitly separates dimensional design from physical implementation

Support for ROLAP and MOLAP

ETL accelerators: dimension and cube operator

New 11g R2 features:

Automated orphan management policies for

loading dimensional objects

Oracle OLAP Cube-Organized Materialized

Views support

Date


8

2011 © Trivadis

Dimensional Objects

Easy to define by using dimension and cube wizard

Additional OWB Meta-Objects are created

automatically (tables, sequences, bitmap

indexes, constraints, …)

Implementation of best practices

Date


9

2011 © Trivadis

Dimension Tables

Table <DIM_NAME>_TAB

Primary Key on DIMENSION_KEY

For each level

Group of columns

Index on business key

Manual vs. Automatic binding

Automatic binding

- is easy, but …

- … overwrites ALL manual modifications

on the dimension table

Manual Binding allows you to follow

your own naming conventions and design

patterns

Date


10

2011 © Trivadis

«Solved» Dimensions

“Solved “ dimensions contain records for higher level elements (control

rows)

enable you to link fact data to a dimension at any level

Control Rows have

Negative surrogate Keys

NULL attributes for lower levels

Date


11

2011 © Trivadis

Time Dimension

Time dimension is not «solved»

use 01.01.2011 for referencing «January 2011»

Primary Key is a DATE attribute

was a NUMBER in pre-11.2 releases

allows partitioning of fact table by date

Mapping to load the time dimension table is generated by OWB

Different hierarchies are supported

week hierarchy

fiscal year hierarchy

Date


12

2011 © Trivadis

Fact Table

Table <CUBE_NAME>_TAB

One column per

measure

referenced dimension

Bitmap Index and foreign key for each dimension attribute

Unique key for the combination of dimension attributes (optional)

No partitioning! No compression!

Date


13

2011 © Trivadis

Deployment

Date


14

Design Repository Target Schema

Dimension

Cube

Dimension

View

Materialized

Views

Deploy Data Objects

Deploy Aggregation

2011 © Trivadis

Dimension/Cube Operator

Used for loading data into a dimension/cube

can do a lot of tedious work for you

Are actually pluggable mappings

Dimension Operator

populates dimension surrogate key

realizes SCD (Slowly Changing Dimensions) updates

handles invalid dimension records

Cube Operator

looks up surrogate keys based on business key

handles slowly changing dimensions

handles invalid cube records

Date


15

2011 © Trivadis

Dimension Operator

Supports two loading types:

Load

Remove

Allows deduplication of level

elements

Creates OWB$TEMP Tables

when mapping is deployed

needed to provide some of the

hierarchy management and

loading functionality

Date


16

2011 © Trivadis

Slowly Changing Dimension

Support for SCD Types 1, 2 and 3

Type 1: Do not store history

Type 2: Store the complete

change history

Type 3: Store only the previous

value

User can specify which attributes

trigger a new version

only use from 10.2.0.3 upwards

bugs in earlier versions

Date


17

2011 © Trivadis

SCD: «Under The Hood»

Income_Level and Marital_Status trigger a new version

For all other records only the most current record is updated

Date


18

INCOME FIRST LAST MARITAL JOB VONDAT BISDAT

B Constantin Welles single Teacher 05-SEP-07 12-OCT-09

C Constantin Welles single Teacher 12-OCT-09 17-AUG-10

C Constantin Welles married Policeman 17-AUG-10 31-DEC-99

5-Sept-07 – Constantin Welles becomes customer.

12-Okt-09 – Constantin Welles gets a raise.

17-Aug-10– Constantin Welles marries.

??? – Constantin Welles has a new job.

2011 © Trivadis

SCD: «Under The Hood»

Kimball Design Tip #15: Combining SCD Techniques

“For all prior dimension rows for a given customer, the current roll-up attribute

will be overwritten to reflect the current state of the world.”

Kimball’s «Hybrid SCD» is not implemented by dimension operator

Date


19

2011 © Trivadis

Orphan Management

What is an «orphan»?

Dimension level element with no or invalid parent level attribute

Fact with no or invalid dimension reference

Available for dimensions and cubes

Error tables store any records that are detected as anomalous.

Date


20

Specify values used by

the default record

2011 © Trivadis

Dimension Operator: «Behind the Scenes» - 1

Date


21

2011 © Trivadis


Date


22

2011 © Trivadis

Cube Operator

Loading Type

INSERT LOAD

LOAD

- source data is merged into

the cube

REMOVE

Enable Source Aggregation

Handles slowly changing

dimensions

Orphan Managment

Date


24

2011 © Trivadis

Cube Operator: «Behind the Scenes»

Date


25

Dimension Surrogate

Key Lookup Orphan Management

Source Aggregation

2011 © Trivadis

Features, Features, Features, …

Loading dimensions

Physical and logical deletes

Dimensions without surrogate key

History Logging Policy

Support Multiple History Loading

Out of Order History Loading

Standard DML Error Logging

Hierarchy Versioning

- Attribute changes are also recorded for higher levels

Loading cubes

Degenerate dimensions

Different source aggregation functions (SUM, AVG, …)

Different update functions (+=, -=, …)

Date


26

2011 © Trivadis

It’s all about performance!

OWB can create

Relational MViews

Cube Based Mviews

Mviews are created when deploying aggregation of cube

MViews are no meta objects in OWB

Relational Mviews are recreated every time you deploy

MView Features like «fast refreshing» and «partition change tracking» are not

supported

Hint: Select precompute for the leaf level also to avoid OLAP Bug.

Date


27

2011 © Trivadis

Licenses

You need

Data Integrator Enterprise Edition (formerly Warehouse Builder Enterprise ETL)

for

- SCD 2 and 3 support

- Orphan management

OLAP Option for

- Creation of Materialized Views

Date


28

2011 © Trivadis

Be aware of …

You can define default values without orphan management.

Degenerate dimensions are possible (with 11.2)

Each cube must have at least one measure

For factless fact tables use one «dummy» fact with the default value 1

Either all dimensions levels have a surrogate key or none

Dimensional operators can also be used as source operators.

Date


29

2011 © Trivadis

Everything has to fit together

Date


30

Dimension

Settings

Cube

Settings

Dimension

Configuration

Cube

Configuration Operator

Properties

2011 © Trivadis

Open the toolbox!

Take everything that suits your

project-specific requirements

You should have good reasons not to

use dimensional operators

Model dimensions and cubes to avoid

semantic loss

Do prototyping!

Learn how things work

Good understanding of the underlying

technologies (e.g. OLAP) is necessary

With or without dimensional operators:

Use the same design pattern everywhere!!

Date


31

2011 © Trivadis

Summary

Oracle Warehouse Builder offers a broad range of support for

dimensional objects.

When using these features you …

… can efficiently implement your ETL processes

… are «production-ready» in a shorter time span

… avoid semantic loss

… achieve high quality in your implementation

… have a standardized design pattern in your dimension and cube loading

mappings

Date


32

2011 © Trivadis

BASEL BERN LAUSANNE ZÜRICH DÜSSELDORF FRANKFURT A.M. FREIBURG I.BR. HAMBURG MÜNCHEN STUTTGART WIEN

THANK YOU. Trivadis AG

Maren Eschermann

Europa-Strasse 5

8152 Glattbrugg

Tel. +41-44-808 70 20

Fax +41-44-808 70 21

[email protected]

www.trivadis.com

Date


33

Dimensional modelingowb11gr2 presentation

Technology

Transcript of Dimensional modelingowb11gr2 presentation