MD Anderson: Using Oracle Exadata in Cancer Research ... · Customer Case Study: Using Oracle...

Nicholas Collins, Principal ASA Clinical Analytics and Informatics 03 June 2014

Customer Case Study: Using Oracle Exadata in Cancer Research [E4 2014 Tue 3:30PM]

1. About MD Anderson

2. The Future of Cancer Treatment and Research

3. Oracle Health Sciences at MD Anderson

4. Genomics and NLP Pipelines

5. FIRE Architecture (HDWF Implementation)

6. Oracle HDWF Upgrade to Exadata x4-2

7. Closing/Questions

Topics

2

About MD Anderson

1

3

Non-profit Houston-based cancer hospital and research institution, founded in 1941 as part of The University of Texas System

Named after Monroe Dunaway Anderson (a banker and cotton trader, not an MD)

“Making Cancer History” – our mission is to eradicate cancer

Consistently ranked as the #1 hospital for cancer care

About MD Anderson

4

About MD Anderson

5

About MD Anderson

6

Almost 20,000 employees, majority in the Houston area

Occupying over 20 buildings in the Texas Medical Center

The Texas Medical Center has over 50 member institutions, together over 100,000 employees

The Future of Cancer Treatment and Research

2

7

“The Time is Now. Together we will end cancer.”

Target six forms of cancer

Clear focus on the concept that the answer to curing cancer lies in both clinical and genomic data

MD Anderson Moon Shots Program

8

Breast/Ovarian Leukemia (AML/MDS) Leukemia (CLL) Melanoma Lung Prostate


9 http://www.cancermoonshots.org


10

How do we solve the mysteries of curing cancer?

It’s in the Data!

11

MD Anderson Moon Shots Platforms

12

Massive Data Analytics – An infrastructure for complex analytics and clinical decision support using integrated patient information, including clinical and research data

Big Data – An Information Technology infrastructure/environment that enables centralization, integration and secured access of patient and research data and analytical results

It’s in the Genes!

13

MD Anderson Moon Shots Platforms

14

Clinical Genomics – Clinical gene sequencing infrastructure, including centralized bio-specimen repository and processing

Omics – Bioinformatics – A high-throughput infrastructure for generation and standardization of large-scale “omic” data, including genomics, proteomics and immune profiling

Adaptive Learning in Genomic Medicine – A framework for bringing clinical medicine and genomic research together to enable rapid learning to improve patient management using Clinical Genomics, Omics-Bioinformatics and Massive Data Analytics platforms within the Big Data environment

Genomics in the News

15

Oracle Health Sciences at MD Anderson

3

16

Oracle Healthcare Data Warehouse Foundation (HDWF)

Oracle Healthcare Analytics Data Integration (OHADI)

Oracle TRC (Translational Research Center) Cohort Explorer

Oracle TRC Omics Data Bank (ODB)

Oracle Health Sciences Products at MD Anderson

17

Oracle Database 11gR2

Oracle Exadata (x3 and x4)

Oracle Business Intelligence (OBIEE)

Oracle GoldenGate*

Oracle Technology at MD Anderson

18

*Oracle GoldenGate was used to demonstrate replication capabilities in a significant POC, but has not been purchased or put into production. Informatica is commonly used at MD Anderson for data integration; ODI is not currently in use at the institution.

Oracle Healthcare Data Warehouse Foundation (HDWF)

19

HDI HDM

Oracle Healthcare Analytics Data Integration (OHADI)

20

HDI HDM OHADI

Integration code that maps from the HDWF interface tables (HDI) to the HDWF warehouse tables (HDM)

Available as either Informatica or ODI mappings

Oracle Cohort Explorer

21

CDM

Cohort Explorer

CDM (Cohort Data Model) is the Clinical Data Mart used by Oracle Cohort Explorer


22

Oracle Omics Data Bank (ODB)

23

Review of Oracle Health Sciences Products

25

HDI HDM OHADI CDM

ODB

Cohort Explorer

Genomics and NLP Pipelines

4

26

CDM

ODB


HDM

Genomic Sequencing

Data

Combining Genotypic and Phenotypic Data

Clinical (Phenotypic) Data

Genotypic Data

Reference Data

What kind of data is loaded to ODB?

28

Full Genome Sequence Reference (EMBL) Known Variants (dbSNP/Cosmic) Genes (HUGO) Proteins (SwissProt/UniProt) Pathways (Pathwaycommons) Predictive Phenotyping (Polyphen/SIFT)

Simple Variant (SNP/Indel) Gene Expression Copy Number Variation RNA Sequencing Structural Variants

Result Data

How much data? Three billion bases in human genome

2.8 GB at a byte per base 700 MB at two bits per base

RNA Codon Chart

30

Synonymous Nonsynonymous

Missense Nonsense

Frame Shift (in the case of indels)

Point Mutations

Load reference data into ODB (initialize it for use, no results)

Lab sequences patient specimens to create result files with variant data (vcf & cnv)

Load result files local to Exadata rack in DBFS

Load from DBFS into ODB via java and PL/SQL loaders (stock ODB functionality, though customer loaders can be written)

Genomic data (in ODB) links to clinical data (in CDM) via a specimen ID for use in Oracle Cohort Explorer and other applications

Genomics Pipeline

31

http://www.1000genomes.org/node/101

VCF (Variant Call Format) File Example

CNV (Copy Number Variation) File Example

CDM/ODB Implementation - Exadata x3-2 Equipment Purchased

December 2012

Development/Test Environment

Production Environment

34

Photo shown courtesy of Mr. Robert Jeffries, Project Manager 35

MD Anderson CAI “War Room” Exadata Implementation Team

January 2013

Querying in Cohort Explorer often requires many lengthy and complex conditions in the where clause, pulling based on individual column values (i.e. genes, healthcare terminology codes) without benefit of ranges, smart scans improve performance

HCC compression helps in repetition of common values (just take ‘A’, ‘G’, ‘T’, and ‘C’ for instance), also when using non-EAV tables for EAV-style data, common in clinical data

Overall performance requirements of the most advanced Cohort Explorer functionality would be difficult to back with anything other than Exadata

Gains from CDM and ODB on Exadata

37

Subject: i got to tell you this about exadata!

MD Anderson partnered with IBM to build an Oncology Expert Advisor (OEA) application based on IBM Watson technology

NLP important to make data from clinical notes available to OEA, so the institution began using IBM’s NLP tools, including IBM Content Analytics (ICA)

CAI had a need for NLP to make more clinical data available for Cohort Explorer use - often phenotypic data (i.e. patient diagnosis, comorbidities, family cancer history) is only in the transcribed note

CAI is now collaborating with the IBM Watson team to ensure institutional standards for NLP efforts, and to leverage each others’ work as much as possible

CAI’s NLP team uses Exadata for pre-warehouse staging/processing

Natural Language Processing (NLP)

40

Unstructured source documents created by provider (i.e. patient notes, pathology reports), scanned if not originated electronically or transcribed

Documents pulled from various source systems into a single repository (on Exadata)

Crawler pulls new documents for processing by ICA Server, annotators process documents to distill specific attributes and evidence

Attributes loaded to database in structured relational model as intra-document “preliminary” assertions (on Exadata)

IBM WODM Rules Engine takes preliminary assertions and creates cross-document final assertions, also loaded into relational model (on Exadata)

Final assertions loaded into Oracle HDWF as structured clinical data

NLP Pipeline

41

FIRE Architecture (HDWF Implementation)

5

43

FIRE - Federated Institutional Reporting Environment

A program level initiative, with many projects and products involved, to provide a unified BI/Reporting solution for all of MD Anderson

Managed by the Clinical Analytics and Informatics (CAI) Department, part of Oracle SDP (Strategic Development Parnter) Program

Implmented Oracle HDWF warehouse as the core of the FIRE Program, beginning in 2012

HDWF for The MD Anderson FIRE Program

44

Bring all the data processing together on a single Oracle instance for performance benefits of local movement and transformation

Abstract across all commonalities and patterns to the largest extent possible, avoiding needless one-off solutions, use code generation and automation

Initially implemented on existing AIX hardware, but an ideal candidate for a later “forklift” to Exadata

Architectural Concept

45

The FIRE Architecture

46

SR SI UI UD HDI/HDM

Data Movement

47

There was a desire from our integration team to use Informatica for ETL because of experience base on the team, not much PL/SQL or ODI knowledge

Architecture proposed use of abstracted code generation via Informatica APIs, jointly used with the push-down optimization option for all non-OHADI internal data movement (i.e. SI to HDI, UI to UD)

Data Movement (Planned)

48

Our integration team initially indicated that code generation with Informatica (or other tools) could not be done on account of complexity, and that the push-down optimization option was too expensive

To demonstrate the feasibility, I programmed a PL/SQL-based version of the code generation as proposed in the FIRE Architecture documentation, we used this code in the first release

Data Movement (Actual)

49

Data Movement Code Generation

50

Procedure iv_tv_ip_gen(name_of_sv_view) for SI layer, generates objects for change detection and movement from SR to HDI

Procedure iv_uv_dv_ip_gen(name_of_sv_view) for UI layer, generates objects for change detection and movement from HDM to UD

All that is needed for generation is the SV view, which conforms to the HDI-based structure, data in certain standard HDI columns determine action

A benefit of the generated views is the ability to see what will happen during the next run, without actually running anything

PL/SQL Procedures for Code Generation

51

Had approximately three months to implement, process was difficult, but in the end everything worked and we went to production with the first FIRE release in November 2012

OHADI was somewhat slower than expected but got the job done, Informatica version used, but might be faster with ODI?

Integration team wanted a second chance to get code generation going for Informatica, and wanted more Informatica and less SQL and PL/SQL

CAI committee voted to try Informatica alternatives for the next release

Results/Next Steps

52

Architectural Changes

53

Informatica Code Generation using Java to generate Informatica objects, so not using the PL/SQL code generation with SI and UI views for this release

Integration team wanted an instantiated SI Layer and UI Layer for Informatica-based code generation instead of views in the SI and UI Layer

As a result, hybrid architecture in place with some generated PL/SQL/views and some generated Informatica objects

Architectural Changes

54

Informatica Code Generation with Java

55

HDWF Upgrade – AIX to Exadata x4-2

5

56

HDWF Hardware Upgrade (from AIX) - Exadata x4-2 Equipment Purchased

December 2013

Development/Test Environment

Production Environment

57

Original AIX Hardware: IBM P550, 8 Physical CPUs /1.65 GHz, 64 GB RAM, AIX 5.3/64Bit

Migration Methodology

58

Logical migration using data pump via network (impdp)

Develop a test suite of queries for performance tuning, and automate testing runs

Change configuration using single variable controlled experimentation to hone in on best options, also identify which are the most significant performance boosters (SGA/PGA size, index visibility, tables pinned to cell flash, storage index disabled, compression, Auto DOP, etc.)

Lock configuration and attempt further tuning with test incremental data loads (DML rather than query)

Performance / Times - Indexes

59

Query All Original Indexes Visible All Non-unique Invisible (PKs & UKs visible, FKs invisible) All Indexes Invisble

All Non-unique Invisible except MDACC custom

All Non-unique Invisible except FKs (PKs, UKs, & FKs visible)

Query 1 +00 00:04:15.268206 +00 00:04:18.802606 +00 00:04:17.616623 +00 00:04:15.777681 +00 00:04:15.713001

Query 2 +00 00:00:57.234542 +00 00:00:56.790110 +00 00:00:57.144610 +00 00:00:58.184639 +00 00:00:57.262739

Query 3 +00 00:08:34.878275 +00 00:08:41.121726 never finished +00 00:08:49.401729 +00 00:08:45.867188

Query 4 +00 00:06:41.485783 +00 00:06:34.904931 +00 00:06:29.586784 +00 00:06:42.858168 +00 00:06:42.512104

Query 5 +00 00:24:40.556198 +00 00:24:40.578150 +00 00:26:10.792066 +00 00:24:59.570722 +00 00:24:55.165571

Query 6 +00 00:00:43.029112 +00 00:00:43.410431 +00 00:00:44.056125 +00 00:00:44.123841 +00 00:00:43.635425

Query 7 +00 00:00:01.545078 +00 00:00:01.699091 +00 00:00:01.582558 +00 00:00:01.608624 +00 00:00:01.609783


Query 9 +00 00:00:01.710590 +00 00:00:01.946892 +00 00:00:01.823464 +00 00:00:01.734865 +00 00:00:01.886647

Query 10 +00 00:00:03.919420 +00 00:00:04.223222 +00 01:01:53.262450 +00 00:00:04.191241 +00 00:00:04.139644

Query 11 +00 00:00:57.318168 +00 00:00:07.071767 +00 00:00:06.247229 +00 00:00:06.577124 +00 00:00:06.578497

Query 12 +00 00:00:17.095400 +00 00:00:10.208428 +00 00:00:09.547868 +00 00:00:09.634578 +00 00:00:09.882055


Query 14 +00 00:51:57.673060 never finished never finished +00 03:26:31.477185 +00 00:49:45.230375




Query 18 +00 00:02:40.790018 +00 00:02:39.300987 +00 00:02:39.422138 +00 00:02:40.992558 +00 00:02:41.056058

Query 19 +00 00:00:51.688008 +00 00:00:51.566929 +00 00:00:51.265070 +00 00:00:52.757858 +00 00:00:51.403172

Query 20 +00 00:03:44.814679 +00 00:03:30.898289 +00 00:03:29.446093 +00 00:03:52.102754 +00 00:03:34.554562

Performance / Times - Compression

60

Query No Compression Compressed Compression Type

Query 1 +00 00:04:18.582828 +00 00:04:50.366265 OLTP

Query 2 +00 00:00:57.554905 +00 00:01:15.014118 OLTP

Query 3 +00 00:06:18.164402 +00 00:06:59.956384 OLTP

Query 4 +00 00:07:11.780093 +00 00:08:14.282119 OLTP

Query 5 +00 00:24:58.411945 +00 00:25:23.206456 OLTP

Query 6 +00 00:00:42.437158 +00 00:00:55.025157 OLTP

Query 7 +00 00:00:01.516546 +00 00:00:01.625790 OLTP

Query 8 +00 00:00:09.569546 +00 00:00:06.663720 OLTP

Query 9 +00 00:00:01.744924 +00 00:00:01.272805 OLTP

Query 10 +00 00:00:04.012592 +00 00:00:05.653153 OLTP

Query 11 +00 00:00:06.040799 +00 00:00:07.085242 OLTP

Query 12 +00 00:00:09.923520 +00 00:00:09.957895 OLTP

Query 13 +00 00:00:30.367873 +00 00:03:41.028555 HCC - QH

Query 14 +00 00:49:22.688745 +00 02:23:08.758924 HCC - QH

Query 15 +00 00:04:29.040059 +00 00:14:09.746344 OLTP

Query 16 +00 00:00:26.161868 +00 00:00:28.511322 OLTP

Query 17 +00 00:04:34.196931 +00 00:06:04.440474 OLTP

Query 18 +00 00:02:40.874603 +00 00:02:57.846432 HCC - QH

Query 19 +00 00:00:51.901180 +00 00:01:01.855314 HCC - QH

Query 20 +00 00:02:54.209488 +00 00:03:44.067049 HCC - QH

Compression Ratios

61

Table Size in GB Size in GB at HCC - QH

HCC - QH % of Orig Size

Ratio (Orig : HCC - QH)

Size in GB at HCC - QL

HCC - QL % of Orig Size

Ratio (Orig : HCC - QL)

HCC-QH % of HCC-QL

Table 1 51.97851563 2.487487793 5% 21 4.06964111 8% 13 61% Table 2 41.59350586 2.446899414 6% 17 4.51171875 11% 9 54% Table 3 29.55371094 2.084411621 7% 14 3.78485107 13% 8 55% Table 4 25.5625 2.509277344 10% 10 4.18261719 16% 6 60% Table 5 23.5078125 2.664794922 11% 9 4.40570068 19% 5 60% Table 6 16.61914063 1.192260742 7% 14 2.08892822 13% 8 57% Table 7 15.38574219 1.850524902 12% 8 3.13989258 20% 5 59% Table 8 9.083007813 0.700500488 8% 13 1.18804932 13% 8 59% Table 9 6.918334961 0.898925781 13% 8 1.43572998 21% 5 63% Table 10 6.8359375 0.70690918 10% 10 1.28656006 19% 5 55% Table 11 6.801879883 0.730773926 11% 9 1.22528076 18% 6 60% Table 12 5.343933105 0.930297852 17% 6 1.50109863 28% 4 62% Table 13 4.3828125 0.579589844 13% 8 0.96081543 22% 5 60% Table 14 2.6171875 0.157653809 6% 17 0.24041748 9% 11 66% Table 15 2.5 0.128295898 5% 19 0.22894287 9% 11 56% Table 16 2.088867188 0.115844727 6% 18 0.20172119 10% 10 57% Table 17 1.5 0.181213379 12% 8 0.30102539 20% 5 60% Table 18 1.4375 0.122924805 9% 12 0.24169922 17% 6 51% Table 19 1.1875 0.111816406 9% 11 0.20196533 17% 6 55% Table 20 0.853515625 0.10949707 13% 8 0.14990234 18% 6 73%

Stats – With and Without Histograms

62

Query Baseline (default gather) After stats gathered, forcing histograms (METHOD_OPT => 'FOR ALL COLUMNS SIZE 254‘)

After stats gathered, no histograms (METHOD_OPT => 'FOR ALL COLUMNS SIZE 1‘)

Query 1 +00 00:04:18.069513 +00 00:04:14.479659 +00 00:04:12.718458 Query 2 +00 00:00:56.012300 +00 00:00:55.937563 +00 00:00:56.399873 Query 3 +00 00:12:03.440581 +00 00:06:32.384087 +00 00:08:15.304644 Query 4 +00 00:06:40.072502 +00 00:07:23.185157 +00 00:06:32.371709 Query 5 +00 00:24:59.584111 +00 00:24:52.754278 +00 00:24:43.723337 Query 6 +00 00:00:43.457988 +00 00:00:43.332688 +00 00:00:43.154288 Query 7 +00 00:00:01.825817 +00 00:00:01.735274 +00 00:00:01.684231 Query 8 +00 00:00:08.743574 +00 00:00:09.016219 +00 00:00:06.635865 Query 9 +00 00:00:01.911499 +00 00:00:01.268445 +00 00:00:01.816548 Query 10 +00 00:00:04.326782 +00 00:00:03.879728 +00 00:00:03.953567 Query 11 +00 00:00:07.088747 +00 00:00:09.095229 +00 00:00:07.355436 Query 12 +00 00:00:10.576206 +00 00:00:09.860471 +00 00:00:09.813917 Query 13 +00 00:00:30.217194 +00 00:00:29.858610 +00 00:00:24.543118 Query 14 +00 00:49:28.997130 +00 01:42:28.740533 +00 00:28:13.714565 Query 15 +00 00:03:26.125896 +00 00:03:30.789577 +00 00:03:07.800214 Query 16 +00 00:00:23.819626 +00 00:00:23.579613 +00 00:00:23.572567 Query 17 +00 00:04:23.784166 +00 00:03:49.886384 +00 00:03:49.866113 Query 18 +00 00:02:43.559441 +00 00:03:44.672787 +00 00:02:39.627688 Query 19 +00 00:00:52.545641 +00 00:00:52.185251 +00 00:00:51.524320 Query 20 +00 00:03:23.480463 +00 00:04:35.762286 +00 00:03:21.034797

Performance / Times – Auto DOP

63

Query Baseline Auto, DL 4, Force Local Auto, DL 8, Force Local Auto, DL 16, Force Local

Query 1 +00 00:04:12.718458 +00 00:04:21.658909 +00 00:04:16.253238 +00 00:04:15.601514

Query 2 +00 00:00:56.399873 +00 00:00:56.412281 +00 00:00:55.648165 +00 00:00:56.170821

Query 3 +00 00:08:15.304644 +00 00:08:17.698308 +00 00:08:12.276071 +00 00:08:17.515982

Query 4 +00 00:06:32.371709 +00 00:06:45.082731 +00 00:06:39.860725 +00 00:06:44.585730

Query 5 +00 00:24:43.723337 +00 00:24:57.991367 +00 00:25:09.900648 +00 00:24:50.583881

Query 6 +00 00:00:43.154288 +00 00:00:43.439962 +00 00:00:42.944854 +00 00:00:43.224442

Query 7 +00 00:00:01.684231 +00 00:00:01.691277 +00 00:00:01.610445 +00 00:00:01.715049

Query 8 +00 00:00:06.635865 +00 00:00:06.469042 +00 00:00:06.241034 +00 00:00:06.570801

Query 9 +00 00:00:01.816548 +00 00:00:01.840649 +00 00:00:01.748562 +00 00:00:01.765074

Query 10 +00 00:00:03.953567 +00 00:00:03.933501 +00 00:00:03.754602 +00 00:00:03.810805

Query 11 +00 00:00:07.355436 +00 00:00:09.900409 +00 00:00:09.615272 +00 00:00:09.421509

Query 12 +00 00:00:09.813917 +00 00:00:09.482519 +00 00:00:09.710079 +00 00:00:09.892875

Query 13 +00 00:00:24.543118 +00 00:00:15.723340 +00 00:00:18.721824 +00 00:00:20.151063

Query 14 +00 00:28:13.714565 +00 00:33:37.034347 +00 00:33:25.797660 +00 00:33:33.464341

Query 15 +00 00:03:07.800214 +00 00:03:28.656200 +00 00:03:43.276557 +00 00:03:27.005350

Query 16 +00 00:00:23.572567 +00 00:00:15.264801 +00 00:00:16.036919 +00 00:00:15.330395

Query 17 +00 00:03:49.866113 +00 00:04:04.479679 +00 00:04:11.732845 +00 00:04:08.134070

Query 18 +00 00:02:39.627688 +00 00:02:45.298123 +00 00:02:45.883673 +00 00:02:45.424077

Query 19 +00 00:00:51.524320 +00 00:00:15.818625 +00 00:00:13.755202 +00 00:00:13.746067

Query 20 +00 00:03:21.034797 +00 00:02:57.888175 +00 00:02:57.619376 +00 00:02:57.591926

Performance / Times – DOP Hint

64

Query Initial Run all parallel 24 all parallel 16 all parallel 8 all parallel 6 all parallel 4

Query 1 +00 00:04:16.672367 +00 00:00:25.938898 +00 00:00:26.449299 +00 00:00:34.633897 +00 00:00:43.261151 +00 00:01:01.939329

Query 2 +00 00:00:55.772976 +00 00:00:14.462675 +00 00:00:14.936694 +00 00:00:14.448895 +00 00:00:13.395207 +00 00:00:15.068520

Query 3 +00 00:08:11.134460 +00 00:04:02.388953 +00 00:03:55.116240 +00 00:03:41.027493 +00 00:01:57.306787 +00 00:03:29.615242

Query 4 +00 00:06:41.349366 +00 00:00:39.683390 +00 00:00:44.865423 +00 00:01:26.220494 +00 00:01:37.733822 +00 00:02:08.823786

Query 5 +00 00:24:28.216851 +00 00:01:30.732530 +00 00:02:04.158495 +00 00:02:28.018871 +00 00:04:15.199074 +00 00:05:43.970349

Query 6 +00 00:00:43.078030 +00 00:00:07.565337 +00 00:00:07.655455 +00 00:00:09.543481 +00 00:00:10.952646 +00 00:00:14.283483

Query 7 +00 00:00:01.686269 +00 00:00:00.747075 +00 00:00:00.558309 +00 00:00:00.679023 +00 00:00:00.272988 +00 00:00:00.818093

Query 8 +00 00:00:06.936982 +00 00:00:02.711387 +00 00:00:02.898431 +00 00:00:03.273516 +00 00:00:02.385028 +00 00:00:04.060303

Query 9 +00 00:00:01.827659 +00 00:00:00.686610 +00 00:00:00.460302 +00 00:00:00.474744 +00 00:00:00.377457 +00 00:00:00.681739

Query 10 +00 00:00:03.920712 +00 00:00:02.807461 +00 00:00:02.467666 +00 00:00:02.347095 +00 00:00:01.599735 +00 00:00:02.430706

Query 11 +00 00:00:07.290813 +00 00:00:09.330830 +00 00:00:10.142888 +00 00:00:09.091451 +00 00:00:04.770838 +00 00:00:06.636206

Query 12 +00 00:00:09.591401 +00 00:00:09.964417 +00 00:00:09.468584 +00 00:00:08.968741 +00 00:00:10.821744 +00 00:00:09.175446

Query 13 +00 00:00:24.930160 +00 00:00:23.852529 +00 00:00:17.268112 +00 00:00:17.450359 +00 00:00:15.524086 +00 00:00:15.191239

Query 14 +00 00:27:57.732468 +00 00:09:06.884239 +00 00:09:37.026665 +00 00:12:55.954573 +00 00:20:28.144051 +00 00:20:36.553182

Query 15 +00 00:02:27.918861 +00 00:01:42.524856 +00 00:01:42.352751 +00 00:01:48.366352 +00 00:01:11.173652 +00 00:02:05.916376

Query 16 +00 00:00:14.481427 +00 00:00:10.335290 +00 00:00:10.449666 +00 00:00:12.522837 +00 00:00:08.010176 +00 00:00:11.036114

Query 17 +00 00:03:39.617132 +00 00:01:09.896561 +00 00:01:12.347061 +00 00:01:15.938717 +00 00:01:03.228098 +00 00:01:34.219806

Query 18 +00 00:02:38.285368 +00 00:02:05.803792 +00 00:02:12.261332 +00 00:02:44.436150 +00 00:03:18.629559 +00 00:05:26.549267

Query 19 +00 00:00:52.881317 +00 00:00:13.131290 +00 00:00:13.126071 +00 00:00:13.192107 +00 00:00:14.278196 +00 00:00:15.489964

Query 20 +00 00:03:23.152004 +00 00:02:30.883228 +00 00:02:35.797302 +00 00:03:28.339995 +00 00:04:08.225930 +00 00:05:56.041736

1 hr 27 min 24 min 26 min 32 min 40 min 50 min

alter system set parallel_adaptive_multi_user=FALSE scope=SPFILE; alter system set parallel_degree_limit=16 scope=SPFILE; alter system set parallel_degree_policy=LIMITED scope=SPFILE; alter system set parallel_force_local=TRUE scope=SPFILE; alter system set parallel_min_time_threshold=30 scope=SPFILE; alter system set "_OPTIMIZER_USE_FEEDBACK"=FALSE scope=SPFILE;

truncate table resource_io_calibrate$;

insert into resource_io_calibrate$ values (CURRENT_TIMESTAMP,CURRENT_TIMESTAMP, 0,0,300,0,0);

Parameters at Go-live

65

Most stock indexes were necessary, even FKs were needed (contrary to the HS GBU's recommendation), but a lot of our custom indexes (including bitmaps) unecessary and even impairing performance in some cases

Compression got some fantastic disk space savings, but performance lagged uncompressed in many cases, and was only better in one test case

Gathering stats with histograms caused some performance degradation in our test queries, went live without using histograms

Auto DOP had very little impact on queries (not picking up), but forcing parallelism with hints brought significant results, so the power of parallelism is there, perhaps Auto DOP may be better with this in 12c?

Overall daily incremental load was reduced from 5.5+ hours to 3.5 hours, further improvements down to almost 3 hours and still tuning, have seen problems in Informatica that may be holding us up from better performance

Results

66

Questions?

67 www.mdanderson.org

[email protected]

MD Anderson: Using Oracle Exadata in Cancer Research ... · Customer Case Study: Using Oracle...

Documents

Transcript of MD Anderson: Using Oracle Exadata in Cancer Research ... · Customer Case Study: Using Oracle...