Oracle for Data Warehousing. OLTP & ODS Systems Data Warehouse Data Mart SAP, Oracle PeopleSoft,...

<Insert Picture Here>

Oracle for Data Warehousing

OLTP & ODS

Systems

DataWarehouse

Data Mart

SAP, Oracle

PeopleSoft, Siebel,

Custom Apps

Files

Excel

XML

Performance Management

Custom Reporting

PackagedApplications

BusinessIntelligence

Analytics

Data Federation

Custom

Data Marts

Data AccessData Silos

SQLBatch Scripts

Data Hubs

OLAP

Data Migration

Java

Data Replication

Data Warehousing

Today’s Reality

Difficult to Manage

Limited Scalability

Inaccessible Information

UntrustworthyData

Fragmented data Silos

Fragmented data Silos• Higher Costs

• Increased Risk

• Poor Decision Making

• Higher Costs

• Increased Risk

• Poor Decisions

What’s the Impact on Your Business?

BU

SIN

ES

S I

NT

EL

LIG

EN

CE

BPEL

PM

EPM

BI

...

CLUSTERS

ETL

APPLICATIONS

ANALYTICS / APPS

SERVERSORCHESTRATION MASTER DATA

MasterData

CRM APPS

SCM APPS

WEBSITES

FINANCIALS

DISTRIBUTION

CUSTOM APPS

OPERATIONS

ERP APPS

ETL

Data Warehouse

ESB

MDM APPLICATIONS

Master

Data

Moving toward a Clean Architecture

5

Data Warehouse Reference Architecture

Data Integration & Management

Data Warehousing

Business Intelligence Foundation

PM and BI Applications

Data Mining

StorageCompression

OLAP Predictive

PM Applications BI Applications

Partitioning Modeling

Data Federation Data Quality

ETL/ELT Data Services

Ad Hoc Analysis

Reporting & Publishing

Proactive Detection

Customer Service

Interactive Dashboards

Performance Management

Database and Middle Tier Servers

Oracle’s Integrated Stack

Source: IDC, August 2010 – “Worldwide Data Warehouse Platform Software 2009 Vendor Shares”

Oracle: #1 in Data Warehousing

Oracle Confidential – Do Not Distribute

Retail Communications Financial Services Manufacturing CPG/HealthcareTransportn/Energy

Oracle Data Warehouse Customers

Manufacturing

http://www.reuters.com/home.jhtml

Oracle Exadata Database MachineExtreme Performance

• Fastest growing new product in Oracle’s history• Server & Storage Integrated Hardware &

Software Platform• Data Warehousing• OLTP• Consolidation

“After carefully testing several data warehouse platforms, we chose the Oracle

Database Machine. Oracle Exadata was able to speed up one of our critical processes

from days to minutes.”

Brian Camp

SVP, Infrastructure Services Knowledge Base Marketing

Oracle for Data Warehousing

• Optimized for strategic warehousing• 25 GB/sec IO bandwidth, with up to 50 GB/sec

with Flash• Optimized for real-world data loading

• Read consistency with the ability to load at 5TB/hr• Optimized for operational warehousing

• Advanced indexing capabilities running at 1M IOPS

• Optimized for advanced analytics• Integrated OLAP, data mining, spatial and

statistics•Optimized for large data sets

• 10x user data compression

10

Exadata Hardware Architecture

Database Grid Intelligent Storage Grid

InfiniBand Network

• Redundant 40Gb/s switches• Unified server & storage network

• 14 High-performance low-cost storage servers

• 8 Dual-processor x64 database servers

OR

• 2 Eight-processor x64 database servers

Scaleable Grid of industry standard servers for Compute and Storage • Eliminates long-standing tradeoff between Scalability, Availability, Cost

•100 TB High Speed disk, or336 TB High Capacity disk

•5.3 TB PCI Flash

•Data mirrored across storage servers

Complete Family Of Database MachinesFor OLTP, Data Warehousing & Consolidated Workloads

• Quarter, Half, Full and Multi-Racks • Full and Multi-Racks

Oracle Exadata X2-8Oracle Exadata X2-2

13

Exadata is Smart Storage

Database Server•Compute and memory intensive data processing executes in database servers

•Fully-parallelized joins and aggregations

Bandwidth Intensive Searches

Compute Intensive Processing

• Database servers and Exadata storage work in conjunction to execute SQL• Exadata cell is smart storage, not a complete database node

Exadata Storage Server• IO-bandwidth intensive database operations executes in storage servers

• Exadata Smart Scans and Exadata Storage Indexes filter out data that is not relevant to a query

Exadata Database Processing in Storage

• Exadata storage servers implement data intensive processing in storage

• Row filtering based on “where” predicate• Column filtering• Join filtering• Incremental backup filtering• Scans on Hybrid Columnar Compressed data• Scans on encrypted data• Data Mining model scoring

• 10x reduction in data sent to DB servers is common

• No application changes needed• Processing is automatic and transparent• Even if cell or disk fails during a query

Exadata Flash Extreme Performance

• Oracle Database Machine has 5 TB of flash storage• 4 high-performance flash cards in every Exadata

Storage Server

• Smart Flash Cache caches hot data• Not just simple LRU

• Knows when to avoid caching to avoid flushing cache

• Allows optimization by application tableOracle is the First Flash Optimized Database

Oracle Database Machine:Optimized for large scans

1 TBwith compression

10 TB of user dataRequires 10 TB of IO

100 GBwith partition pruning

20 GB with Storage Indexes

5 GB with Smart Scans

Subsecond On Database

Machine

2000X less data needs to be processed

What does Extreme Performance mean for your business?

• Massive data volumes• More granular data

• Daily data instead of weekly• Store data instead of account

• More history• 5 years instead of 1 year

• New data sources• Consumer-level data

• Entirely new analytics• Queries that were never possible now run in minutes

• Near-real-time data loading

Oracle Database Machine








18

Fast data loading

• Performant data loading and transformation with in-database ETL• Direct flat file access with external tables• Transformation inside the database with SQL and PL/SQL• Bulk and trickle load

• Up to 5 TB/hr of raw data loading on a Database Machine

Access Transform Load

Transform Data Where Data ResidesIn-database ETL technology

Extract

Change Data Capture

External Tables

SQL*Loader

Data Pump

Transportable Tablespaces

Multi-Table Insert

MERGE

Distributed Queries

Table Functions

Load Transform Insert

Partition Exchange Loading

DML error logging

DBFS - Scalable Shared File System

• Database Machine comes with DBFS shared Linux file system• Shared storage for ETL staging, scripts, reports and other application files

• Files stored as SecureFile LOBs in database tables stored in Exadata • Protected like any DB data – mirroring, DataGuard, Flashback, etc.

• 5 to 7 GB/sec file system I/O throughput

ETL Files in DBFS

Load into database using External Tables

ETL

More File Throughput than High-End NAS Filer

EMP

22

Real Time Data Integration with Oracle GoldenGate

OracleGoldenGate

On-Disk Logs

Source 1

On-Disk Logs

Source 2

OracleGoldenGate

• Real time extracts from transactional systems

• Non-invasive on sources

• Continuous streaming load

into ODS Schema of target• Latency in seconds

EMP DEPT

DEPT

accurate

report

Oracle is uniquely capable of concurrent query and updates

• Concurrent small data loads and queries

• Looks like... OLTP

• Oracle's read consistency• Readers never block writers• Writers never block readers• Queries are always consistent and auditable• No deadlocks• Introduced in Oracle V4 (1982)• Teradata: lock rows for reads and writes to obtain

consistency

report

Rollback Segment

BEFOREIMAGE

Budget table

update

update









24

25


Base data warehouse schemaAtomic-level data, 3nf designSupports general end-user queriesData feeds to all dependent systems

Application-specific performance structuresSummary data / materialized viewsDimensional view of data Supports specific end-users, tools, and applications

26

Random-IO intensive workloadsIO-bandwidth intensive workloads


Oracle delivers complete capabilities for tactical warehouse queries

• Fast lookups• B-Tree indexes

• Dimensional queries• Star query optimizations• Bitmap indexes• Bitmap join indexes

• Aggregate management• Materialized Views• Cube-organized materialized views

Workload Management for DWSetting Up a Workload Management System

WorkloadManagement

Define Workloads

Filter Exceptions

Manage Resources

Monitor Workloads

Adjust Plans

Execute Workloads

Monitor Workloads

Adjust Workload Plans

IORM

RAC OEM

DBRM

Define Workload Plans

Workload Management

Request

Ad-hocWorkload

Each consumer group has:• Resource Allocation (example: 10% of CPU/IO

resources)• Directives (example: 20 active sessions)• Thresholds (example: no jobs longer than 2 min)

RejectDowngrade

Assign

Each request assigned to a consumer group:• OS or DB Username• Application or Module• Action within Module• Administrative

function

Queue

Execute

Each request:• Executes on a RAC Service• Which limits the physical

resources• Allows scalability across racks

Workload Management

Request

Real-TimeETL

Batch ETL

Analytic Reports

OLTP Requests

Ad-hocWorkload

Assign

Reject

Queue

Execute

Downgrade

Execute

Workload Management

Request

Real-TimeETL

Batch ETL

AnalyticReports

OLTP Requests

Ad-hocWorkload

Assign

RejectDowngrade

Queue

Ad-hoc 25%

Analytic Reports

50%

OLTP 5%

Batch 10%

R-T 10%

Queue

Queue

Queue

Queue









32

In-database AnalyticsBring Algorithms to the Data, Not Data to the Algorithms

• Analytic computations done in the database• Dimensional analysis• Statistical analysis• Data Mining

• Scalability• Security• Backup & Recovery• Simplicity

OLAP

Data Mining

Statistics

Deeper insightsfrom Oracle Data Warehouses

Oracle Spatial: GIS data available for analysis and displayed via MapViewer


Oracle OLAP: Hierarchically aware rankings, shares, alerts and time series

calculations are easily defined in the cube and queried by OBIEE using simple and

efficient SQL


Hierarchically aware rankings, shares, alerts and time series calculations are

easily defined in the cube and queried by OBIEE and other tools using simple and

efficient SQL

Oracle Data Mining: Predictions & probabilities are calculated within database

and available for reporting using OBIEE


OLAPData

MiningSpatial

Simple and seamless embedded analytics

Extreme performance and

scalability

Deeper insights for business users

In-Database Analytics

Pervasive benefits

across stack









38

© 2010 Oracle Corporation

39

Exadata Hybrid Columnar CompressionHighest Capacity, Lowest Cost

• Data is organized and compressed by column• Dramatically better compression

• Speed Optimized Query Mode for Data Warehousing• 10X compression typical • Runs faster because of Exadata offload!

• Space Optimized Archival Mode for infrequently accessed data• 15X to 50X compression typical

Qu

ery

Faster and Simpler

Backup, DR, Caching, Reorg,

CloneBenefits Multiply

More Data Capacity

Exadata V210x Compression

Teradata 25801.3x Compression

Netezza TwinFin2x to 4x Compression

EMC VMAX3x Oracle Compression

Systems with Equal User Data

All with Largest Disks,

Best Compression

41

SUMMARY

WAN

Enterprise Availability

• Redundant Hardware• Servers, Storage, Network

• Database Level HA• Tolerate failures and changes

• Real-Time Active Replica


42

Active Data Guard

GoldenGate Replication

RAC

ASM

Flashback

SecureBackup

Online Redefinition

First Secure Database Machine

• Moves decryption from software to hardware• Over 5x faster

• Near zero overhead for fully encrypted database

• Queries decrypt data at hundreds of Gigabytes/second

© 2010 Oracle Corporation 43


44

Standardized and Simple to Deploy

• All Database Machines are the same• Delivered Tested and Ready-to-Run• Highly Optimized• Highly Supportable• No unique configuration issues• Identical to config used by Oracle Engineering

• Runs existing OLTP and DW applications• Full 30 years of Oracle DB capabilities• No Exadata certification required

• Leverages Oracle ecosystem• Skills, knowledge base, people, partners

Deploy in Days, Not Months

• Faster Throughput from Disk

• Much Faster with Flash

• 50 TB of data fits in Flash• Using 10x Query Compression

• Effective Query Throughput on compressed data is even higher• Hundreds of GB/sec

Teradata2580

NetezzaTwinFin 12

ExadataV2

Query Throughput GB/sec Uncompressed Data

Single Rack

Query Throughput GB/sec Uncompressed Data

Single Rack

Flash

Disk

Faster than DW Appliances50 GB/sec!

Oracle Exadata MomentumRapid adoption in all geographies and industries

http://www.huntermanufacturing.org.au/drupal/?q=node/11

47

Oracle Exadata Database Machine:Changes the Game

• Database intelligence and massively parallel scaling in the storage tier• Using state of the art industry standard

hardware• Complete, Integrated Data Warehouse,

OLTP and consolidation solution• High availability• Enterprise Security• Advanced analytics

• Innovative new technologies:• Hybrid Columnar Compression• In-memory parallel execution• FlashFire hardware + flash-optimized

software

Oracle for Data Warehousing. OLTP & ODS Systems Data Warehouse Data Mart SAP, Oracle PeopleSoft,...

Documents

Transcript of Oracle for Data Warehousing. OLTP & ODS Systems Data Warehouse Data Mart SAP, Oracle PeopleSoft,...