Data Warehouse/Data Mart Components Concepts Characteristics.

Data Warehouse/Data Mart

Components

Concepts

Characteristics

Overview

• Operational vs Informational Systems

• Data Warehouse components

• Data Marts

Basic Data Warehouse Architecture

Copyright © 1997, Enterprise Group, Ltd.

Source OLTPSource OLTPSystemsSystems

Subset Data MartsSubset Data Marts

EnterpriseData

Warehouse

One VersionOne Versionof the Truthof the Truth

Operational vs. Informational Systems


Information Access TodayInformation Access Today

OperationalOperationalSystemsSystems

OrderOrderEntryEntry Manf.Manf.

Operational vs. InformationalSystems


Information Access TodayInformation Access Today


InformationalInformationalSystemsSystems


• Most of the advances in end-user programming have run into difficulty in actually accessing data that exists in backbone, operational data bases.

• Operational data bases have a very, very long life. Large operational systems are converted from one technology to a more advanced one very infrequently (typically every eight to twenty years).

• Therefore, why not create specific DBs whose role was to make large scale end user access easy to isolate the operational DBs, i.e. a Data Warehouse





InformationInformationDelivery SystemDelivery System





InformationInformationDelivery SystemDelivery SystemDataDataWarehouseWarehouse






Notice that one of the big impacts of Notice that one of the big impacts of Data Warehousing is to eliminate large Data Warehousing is to eliminate large numbers of existing DSS systems!numbers of existing DSS systems!Y2000 will make this essential!!!Y2000 will make this essential!!!






Data Data MartsMarts

Data Mart Layer

Presentation/ Desktop

Access Layer

Meta-data Repository Layer

Warehouse Management Layer

Core DW Layer

Data Staging and Quality Layer

Data Access Layer

Operational Data Layer

External Data Layer

Data Feed/Data Mining/

Indexing Layer

Virtual DW

Coarse DW

Central DW

Distributed DW

Application Messaging (Transport) Layer

Internet/Intranet Layer

direct queries

virtual queries

ad hoc queries

1

2a

2b

2c

3

4 56 7

8

9

10

11

United Statesby Sales

$10,340 to $10,350 (1)$8,730 to $10,340 (2)$4,320 to $8,730 (2)$1,100 to $4,320 (1)

$730 to $1,100 (3)

United States$11,000

Sales

North America

Non-operational

Data Layer

Data Marts vs Data WarehousesData Marts vs Data Warehouses

Data Mart Layer


Access Layer



Core DW Layer


Data Access Layer


External Data Layer


Indexing Layer

Central DW



direct queries

virtual queries

ad hoc queries

1

2a

2b

2c

3

4 56 7

8

9

10

11


$10,340 to $10,350 (1)$8,730 to $10,340 (2)$4,320 to $8,730 (2)$1,100 to $4,320 (1)

$730 to $1,100 (3)


Sales

North America

Non-operational

Data Layer

Central Data WarehouseCentral Data Warehouse

Tracking DBTracking DB

Lawson DBLawson DB

Virtual Date Warehouse

• A Virtual Data Warehouse approach is often chosen when there are infrequent demands for data and management wants to determine if/how users will use operational data.

• One of the weaknesses of a Virtual Data Warehouse approach is that user queries a made against operational DBs.

• One way to minimize this problem is to build a “Query Monitor” to check the performance characteristics of a query before executing it.

• A Coarse Data Warehouse is often chosen when the organization has a relatively clean/new operational system and management wants to make the operational data more easily available for just that system.

• A Central Data Warehouse• is often chosen when the organization has a clear

understanding about it Information Access needs and wants to provide “quality”, “integrated” , information to its knowledge workers

• A Distributed Data Warehouse is similar in most respects to a Central Data Warehouse, except that the data is distributed to separate mini-Data Warehouses (Data Marts )on local or specialized servers

Data Mart Layer


Access Layer



Core DW Layer


Data Access Layer


External Data Layer


Indexing Layer

Virtual DW

Coarse DW

Central DW


Distributed DW


direct queries

virtual queries

ad hoc queries

1

2a

2b

2c

3

4 56 7

8

9

10

11


$10,340 to $10,350 (1)$8,730 to $10,340 (2)$4,320 to $8,730 (2)$1,100 to $4,320 (1)

$730 to $1,100 (3)


Sales

North America

Non-operational

Data Layer

Central Data WarehouseCentral Data Warehouse

Data Mart Layer


Access Layer



Core DW Layer


Data Access Layer


External Data Layer


Indexing Layer

Virtual DW

Coarse DW

Central DW

Distributed DW



direct queries

virtual queries

ad hoc queries

1

2a

2b

2c

3

4 56 7

8

9

10

11


$10,340 to $10,350 (1)$8,730 to $10,340 (2)$4,320 to $8,730 (2)$1,100 to $4,320 (1)

$730 to $1,100 (3)


Sales

North America

Non-operational

Data Layer

Data Marts OnlyData Marts Only

Heterogeneity - The Reality

Oracle Financials

CustomMarketingData Warehouse

PackagedOracle FinancialData Warehouse

PackagedI2 Supply ChainNon- ArchitectedData Mart

SubsetData Marts

i2 Supply Chain Siebel CRM 3rd PartyData

Federated BI Architecture

Real TimeODS

FederatedFinancialData Warehouse

SubsetData Marts

CommonStagingArea

Oracle Financialsi2 Supply Chain Siebel CRM 3rd Party

FederatedPackagedI2 SupplyChainData Marts

AnalyticalApplications

e-commerce

Real TimeData Miningand Analytics

Real TimeSegmentation,Classification, Qualification,Offerings, etc.

FederatedMarketingData Warehouse

Benefits of Data Warehouse Architecture

• Provides organizing framework• Gives flexibility for changes and allows

simplified maintenance• Speeds up future development by aiding

understanding of dw• Communication tool for roles and

requirements• Coordinate data marts

Primary Technical Challenge Axis

EasyEasy HardHard

FastFast

SlowSlowParallelParallelERP DWERP DW

FinanceFinance

CustomCustomERP DWERP DW

TurnkeyTurnkeyERP DWERP DW

VLDBVLDB

NearNearReal Real TimeTime

MarketingMarketing

Mid-Size Co.Mid-Size Co.

Large Co.Large Co.

Single SourceSingle Source

Multi-SourceMulti-Source

MonthlyMonthlyFreqFreq

Small DBSmall DB

Dirty DataDirty Data

Clean DataClean Data

Prerequisites for Success

• Pain driven

• Sponsorship at the highest levels

• Sustainable political will

• Iterative methodology

• Manageable scope

• User driven design

• Service business mindset

• Sustainability

Data Warehouse/Data Mart Components Concepts Characteristics.

Documents

Transcript of Data Warehouse/Data Mart Components Concepts Characteristics.