THE PIONEER INDATA WAREHOUSE

AUTOMATION SOFTWARE

Datavaultautomationconference – Utrecht– 6October2011

Whythispresentation?

COMPANY BACKGROUND• Founded in 2000 in Auckland, New Zealand

by data warehouse experts• 120+ Employees• US – Global Head Office, Portland• UK – European Head Office, Reading• NZ – Development, Auckland

Supported Targets:

USA:Wells Fargo BankNIKEGE AviationIPC SubwayJuniper NetworksNordstromDelta Credit UnionMed Assets

EuropeTescoVodafoneAONXerox HSBCMAN InvestmentsNetwork Rail Sainsbury’sZurich Insurance Volkswagen

Asia PacificMGM MacauAbano InternationalSingtel OptusEricsson VietnamTelstra AUReserve Bank of AustraliaMinistry of DefenceWestfieldQBE InsuranceEnergy AustraliaAstraZenecaAir New ZealandTelecomVodafoneXero

Vodafone Qatar

ABSA Barclays Bank

GLOBAL CUSTOMER BASE700+ WORLDWIDE

SOME OF OUR CUSTOMERS

WHAT WE DO

Profiles, Plans, DesignsRight.

Builds, Documents, Automates Now.

VALUE PROPOSITION

• 1st integrated development environment (IDE) that manages the entire data warehouse lifecycle (Plan & Build)

• Builds data warehouses 10-100 times faster than a traditional approaches

• Data driven approach de-risks projects and automatically applies best practice whilst auto documenting…

DATA WAREHOUSE AUTOMATION

REQUIREMENTS

DW framework

Profile

Logical Model

Physical Model

DB Architecture

Storage Mgmt

Index Mgmt

OLAP Design

ETL Mapping

ETL Dev

Version Control

Workflow

Deployment

Maintenance

Word/Excel

Mainly in-house solutions

Informatica

Microstrategy

IBM Clear Case

Trillium

AbInitioDataStage

TOAD

PowerDesigner

Enterprise Architect

Cognos

JIRA

SSMS

SSIS

SSAS

SVN

IBM Clear Quest

Change tool

Change tool

Change tool

Change tool

Change tool

Change tool

Informatica

ERwin

DB Management Tools

DataStage

Documentation

TRADITIONAL APPROACH

Over complicatedInefficient

DW Framework

Profile

Physical Model

DB Architecture

Storage Mgmt

Index Mgmt

OLAP Design

ETL Mapping

ETL Dev

Version Control

Workflow

Deployment

Maintenance

Logical Model

Documentation

METADATA

WhereScape RED

WhereScape 3D

THE WHERESCAPE WAY

SimplificationAutomation

MANAGING AN EDW/BI ENVIRONMENTIS CHALLENGING

• Too many development tools & skills

• Siloed with a high cost of change

MANAGING AN EDW/BI ENVIRONMENTJUST GOT EASIER

• One tool, one skillset, low cost of change

• Non-proprietary & Non-disruptive

SQL

• I set-based SQL• Scale with your RDBMS or Big Data platform

• Easy migration path to other technologies

• You’re in control, no black boxes

• Don’t we all talk SQL?

DATA WAREHOUSE AUTOMATION

😀

PROCESS

• Different activities warrant different approaches• Model-driven development vs.• Data-driven development

• Bring the agile back in our profession• Break down the barriers between profiles• Don’t limit yourself to “coding only” activities

DATA WAREHOUSE AUTOMATION

“A 3 year project has now been completed in 6 months thanks to WhereScape and agile development.”

Discovery—Rapidlycreateworkingsolutions,whichareputinfrontofbusinessstakeholdersforreviewinsprintsnolongerthan30days.

Production Build—Becomesanassemblyprocess,closelyfollowingstandardsandcommoditizedimplementationpatterns

Redlineshowscurrentdate

THE DATA QUADRANT MODELRONALD D. DAMHOF

DevelopmentStyle

Systematic

Opportunistic

I II

III IV

Research,Innovation&

Design

“ShadowIT,Incubation,

Ad-hoc,Onceoff”

Push/Supply/Source driven Pull/Demand/Productdriven

DataPush/PullPoint

ContextFacts

SCOPE

• End-to-end• Full life cycle -> identify your bottlenecks

• Analysis: source profiling• DevOps / CI• Test Automation• Documentation

• Any modeling style?

DATA WAREHOUSE AUTOMATION

WHERESCAPE ARCHITECTURE

CLASSIC KIMBALLStar Schema

HISTORICAL ODS + CLASSIC KIMBALLData Store Star Schema

NORMALIZED EDW3NF Star Schema

RAW + BUSINESS DATAVAULTData Vault Data Vault

RAW DATAVAULT + STAR SCHEMAData Vault Star Schema

DATA LAKE + STAR SCHEMAStar Schema

COMPETITION

• Very few automation software packages• Traditional ETL suites• Set of templates• But typical: PPP

DATA WAREHOUSE AUTOMATION

• Teradata is an important platform

THANK YOU

Frederik NaessensFrederik.naessens@wherescape.com

+32 473 826080