Oracle Database Performance Tunning€¦ · Oracle Database Performance Tunning Kaiyao, ... delete...

<Insert Picture Here>

Oracle Database Performance Tunning

Kaiyao, Huang

Principle Member of Technical Staff

Oracle Real World Performance Group

© 2009 Oracle Corporation – Proprietary and Confidential 2

Agenda

• DW

• OLTP

• SQL

• AWR


Data Warehouse


Data Warehouse Death Spiral

• The Wrong HW pathology

– Too few CPUs to drive multi million row queries 10X

– I/O not balanced with CPUs 10X

– Incorrect Networking strategies 10X

• Queries

– Incorrect Query strategy 10X

• ETL and Loading Degraded

– Slowdowns and serialization

– Database size bloat

• Desire to make the systems look productive

– Overloaded query streams 10X


Data Warehouse Death Spiral

What we see

• HW procurement process wrong

• Serial query execution strategy based upon OLTP

experience

• Database bloat due to indexes and aggregates

• ETL and loads running too slow

• No ability to handle Ad Hoc queries

• Hard to get CPUs busy so high concurrent number of

query streams


Data Warehouse Topics

• Data Loading and Movement

• ETL and DML

• Ad-Hoc Query

• Resource Management

• Partition and Compression Design


ETL and DML


Validation and Transformation

• Apply referential integrity checks and business rules

by the use of SQL statements executed in parallel as

an alternative to declarative constraints

• A simple workflow may be:

– Stage the data into the database

• Basic data integrity, nulls, data types etc.

– Check the data

• Uniqueness, foreign keys, business rules etc.

• Apply constraints with “RELY DISABLE NOVALIDATE”

– Transform the data

• Tax corrections, time zone corrections, etc.

– Always leverage parallel set-based processing


Validation Example

Set based processing vs. row by row

Time

H:MI:SS


Validation and Transformation

Proof Points

• For the two validation processes we can now make

these claims

– Store Validation - Over 200 times faster

– Product Validation - Over 2500 times faster

• Exact same hardware

– This is a case of using the wrong methodology


Set Based Processing

Move away from row by row processing

declare

cur rec_cur;

rec type_table%rowtype;

begin

open cur for select * from table1;

loop

fetch cur into …

if condition(rec)

then

insert into table2 …

else

insert into table3 …

end if;

end loop;

end;



To set based processing

insert /*+ append */ into table2

select *

from table1

where condition …

insert /*+ append */ into table3

select *

from table1

where not condition …



There are usually multiple ways to get the same end

result

insert /*+ append */ first

when condition then

into table2 values …

else

into table3 values …

select *

from table1



• Use SQL to tell the DB what to do

• Let the DB figure out how to do it

• Some SQL operations:

– create table as select

– insert /*+ append */ select

– intersect

– minus

– exists

– not exists

– window functions

– multi-table inserts

– outer joins


Data Validation SQL

• Elimination of duplicates

– Outer Join back to the table

– Window function

– Aggregate with HAVING clause

• Foreign Key References

– Outer Joins between tables

• The choice of techniques will be dependent on the

following

– Good/Bad validation of of the data

– The desire to identify and locate bad rows e.g. find ROWIDS

– The desire to programmatically eliminate bad rows


Data Validation SQL

Uniqueness Check

Simply Check the

Data

Obtain one of the

ROWIDs of duplicates

to investigate

Query the rows you wish to

keep eliminating duplicates

based on the load time

select

pk_id,count(*)

from DIRTY_DATA

group by pk_id

having

count(*)>1;

select

pk_id,

count(*),

max(rowid)

from DIRTY_DATA

group by pk_id

having count(*)>1;

select column_list

from

(

select

a.*,row_number() over

(

partition by pk_id

order by load_time desc

) rowno

from DIRTY_DATA a

)

where rowno=1


Data Validation SQL

Orphaned Row Check

Look For Orphans Look for Parents with no Children

select C.rowid

from CHILD C,PARENT P

where C.pk_fk = P.pk_id(+)

and P.pk_id is null;

select P.rowid

from PARENT P,CHILD C

where P.pk_id = C.pk_fk(+)

and C.pk_fk is null;


Data Transformation SQL

Transformation vs. Modification

Driver Transformation Modification

Compression No Impact Compression may be Lost and severely impact

performance

Fragmentation None Fragmentation, row chaining, and holes will almost

certainly take place

Logging and UNDO Possible to eliminate Will take place and may impact performance and

administration requirements

Indexes Indexes need to be rebuilt if used Indexes will be maintained in place. This may be a

performance overhead and Bit Map indexes may

become fragments and require rebuilding

Meta Data Grants etc will require redefinition No impact

Space Overhead of maintaining 2 copies of the data Overhead of UNDO and Logging

Coding New code required writing and new techniques

need teaching

Old Code runs with performance challenges

3rd Party Issues May not be supported by Tool Vendors


Data Transformation SQL

• Use either

– INSERT /*+APPEND */ INTO … SELECT

– CREATE TABLE … AS SELECT

• It is recommend that you create the destination table

first and then use INSERT

– Constraints such as NOT NULL can be correctly applied

and enforced

– Data type, column lengths and precision can be defined and

preserved


Rewriting DML

DELETE

alter session enable parallel dml

/

delete from tx_log

where

symbol = ‘JAVA’

/

commit

/

create table tx_log_new

nologging

parallel

compress for all operations

as

select * from tx_log

where

symbol != ‘JAVA’

/

alter table tx_log rename to tx_log_old

/

alter table tx_log_new to tx_log

/

The predicate is the

compliment of the

DELETE, it selects

the rows to keep


Rewriting DML

UPDATE

alter session enable parallel

dml

/

update sales_ledger

set tax_rate = 9.9

where tax_rate = 9.3

and sales_date > ‘01-Jan-09’

/

commit

/

create table tx_log_new

nologging

parallel

compress for all operations

as

select

.,

case

sales_date>‘01-Jan-09’

and

tax_rate=9.3

then

9.9

else

tax_rate

end,

.

from sales_ledger

/

The UPDATE predicates are

moved to the SELECT list in

a CASE statement to

transform the rows


Rewriting DML

MERGE

• This combines techniques from UPDATE and

DELETE and may also include an INSERT

• For bulk MERGES join the input and merged table as

a FULL OUTER JOIN and then case/decode the

NULL columns to apply DELETE, INSERT and

UPDATE components


ETL and DML Challenges in the Real World Debate

• The performance gains are often 2-3 orders of

magnitude

• The challenge is getting customers to re-write code

to exploit these techniques

• You are often dealing with an OLTP vs. DW culture

issue


ETL and DML Summary

• Data validation and modification

– Best executed in the database via SQL

– This presents big challenges to users who have committed

to classic ETL tools such as Informatica

– Changes of data are best made via transformation and

redefinition than via classic OLTP DML statements ( delete,

update, merge )

• Allows exploitation of HW and parallelism

• Minimizes fragmentation and maximizes compression

• Minimizes logging and minimizes recovery

– Set based techniques use efficient CPU and IO techniques


OLTP


OLTP Topics

• Basic Computer Science

• Connections and Cursors

• HW and Operating Systems

• Monitoring and Bottleneck Identification

• OLTP Race Conditions


Database Performance Core Principles

• The Oracle database is a process based architecture

and to perform efficiently each process requires:

– To be efficiently scheduled by the O/S until the process

completes the SQL statement, or blocks on an operation

required to complete the SQL statement e.g. Disk I/O

– If the process has to fight to get scheduled, or needs to be

scheduled for an over extended period of time due to SQL

inefficiencies, or any blocking operation takes a long time,

then database performance will be poor

• Database performance engineers spend most of

their time looking for CPU-consuming processes and

eliminating blocking events


Some Important Numbers to Think About

• Best Block Access Speeds

– L2 CPU cache ~ 1 nano sec ( 10-9 )

– Virtual Memory ~ 1 micro sec ( 10-6 )

– Numa Far Memory ~ 10 micro sec ( 10-6 )

– Flash Memory (PCI) ~ 0.01 milli sec ( 10-3 )

– Flash Memory (Networked) ~ 0.1 milli sec ( 10-3 )

– Disk I/O ~ 5-10 milli sec ( 10-3 )


Database Performance Core Principles

• To determine acceptable CPU utilization take a

probabilistic approach to the subject. – If a CPU is 50% busy the chance of getting scheduled is 1 in 2

– If a CPU is 66% busy the chance of getting scheduled is 1 in 3

– If a CPU is 80% busy the chance of getting scheduled is 1 in 5

– If a CPU is 90% busy the chance of getting scheduled is 1 in10

• If the probabilities are used as indicator of the

predictability of user response time, then the

variance in user response time becomes noticeable

at about 60-65%

• This has been observed in production and laboratory

conditions for many years.


Database Core Principles

Impact of too many Connections

0

2000

4000

6000

8000

10000

12000

14000

16000

4 8 12 16 20 24 28 32

1 Proc/Core

10 Proc/Core Avg

50 Proc/Core Avg

10 Proc/Core Max

50 Proc/Core Max

10 Proc/Core Min

50 Proc/Core Min

#of CPUs


Parsing Overheads Sessions & Cursors ( This slide is over 10 years old! But still true today )


Connection Pools and the New Generation Bugs

• Session Leaking

• Cursor Leaking

• The Java Developer Conundrum

• Symptoms of these Happening

– init.ora(open_cursors) > 1000

– increasing connection count over time

– Increase in memory usage over time

– init.ora(processes) > 5000

– Application Server restarts/login


Connection Architecture

Connection Storms

• May be caused by application servers that allow the

size of the pool of database connections to increase

rapidly when the pool is exhausted

• A connection storm may take the number of

connections from hundreds to thousands in a matter

of seconds

• The process creation and logon activity may mask

the real problem of why the connection pool was

exhausted and make debugging a longer process

• A connection storm may render the database server

unstable, unusable


SQL


SQL Topics

• SQL is Everything

• Measurements and Metrics

• Seeding the Optimizer

• Serial and Parallel Execution

• Top 5 SQL Challenges

• Tools for SQL Statement Analysis


Serial and Parallel Execution

• Serial Execution

– The correct solution when:

• the query references a small number of rows

• high concurrency

• efficiency is important

• Parallel Execution

– The correct solution when:

• the query references a large number of rows

• low concurrency

• elapsed time is important


Parallel Execution Primer

• Definitions

– Row Source

• An operation the “produces” rows; eg. scan, join, etc

– Cardinality

• Number of rows from a row source

– Parallel Query (PQ) Slave

• An (OS) process that operates on part of a parallel query

– Query Coordinator (QC)

• The “top level” process for the parallel query

– Distribution

• The method by which data is sent from one set of PQ slaves to

another, eg Hash, Broadcast, None


Parallel Query

• Lets look at 2 parallel execution plans for the same

query:

select count(*)

from yellow y,

green g

where y.deptno = g.deptno


Parallel Query Hash Distribution

PQ1

PQ2

PQ1

PQ2

Scan

PQ3

PQ4

QC

Hash Distribution

Join and

Aggregate

Result Set

Scan

Hash Distribution

Green

Table

Yellow

Table

f(x)

f(x)

f(x)

f(x)


Parallel Query Hash Distribution


Parallel Query Broadcast Distribution

PQ1

PQ2

Scan

PQ3

PQ4

QC

Broadcast

Distribution

Scan, Join and

Aggregate

Result Set

Green

Table

Yellow

Table


Parallel Query Broadcast Distribution


Tools For SQL Analysis

• Static Analysis

– Explain Plan

– SQL*Plus Autotrace

• Dynamic Analysis

– dbms_xplan

– SQL Monitor


Static Analysis SQL*Plus Autotrace

SQL> set linesize 132 tab off

SQL> set autotrace traceonly explain

SQL> SELECT * FROM EMP WHERE DEPTNO = 10;

Execution Plan

----------------------------------------------------------

Plan hash value: 3324114979

--------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |

--------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 3 | 114 | 2 (0)| 00:00:01 |

| 1 | TABLE ACCESS BY INDEX ROWID| EMP | 3 | 114 | 2 (0)| 00:00:01 |

|* 2 | INDEX RANGE SCAN | EMP_N1 | 3 | | 1 (0)| 00:00:01 |

--------------------------------------------------------------------------------------

Predicate Information (identified by operation id):

---------------------------------------------------

2 - access("DEPTNO"=10)


Static Analysis SQL*Plus Autotrace

SQL> set autotrace traceonly explain statistics

SQL> SELECT * FROM EMP WHERE DEPTNO = 10;

Execution Plan

----------------------------------------------------------

…

Statistics

----------------------------------------------------------

0 recursive calls

0 db block gets

4 consistent gets

0 physical reads

0 redo size

1159 bytes sent via SQL*Net to client

525 bytes received via SQL*Net from client

2 SQL*Net roundtrips to/from client

0 sorts (memory)

0 sorts (disk)

3 rows processed


Dynamic Analysis dbms_xplan

SQL> set feedback off linesize 132 pagesize 0 tab off

SQL> SELECT * FROM EMP WHERE DEPTNO = :n;

SQL> select * from table(dbms_xplan.display_cursor);

SQL_ID 5xt3urx81f9th, child number 0

-------------------------------------

SELECT * FROM EMP WHERE DEPTNO = :n

Plan hash value: 3324114979

--------------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |

--------------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | | | 2 (100)| |

| 1 | TABLE ACCESS BY INDEX ROWID| EMP | 3 | 114 | 2 (0)| 00:00:01 |

|* 2 | INDEX RANGE SCAN | EMP_N1 | 3 | | 1 (0)| 00:00:01 |

--------------------------------------------------------------------------------------

Predicate Information (identified by operation id):

---------------------------------------------------

2 - access("DEPTNO"=:N)


Dynamic Analysis SQL Monitor

• Overview

– Feature of SQL Tuning Pack

– Generation

• Oracle Enterprise Manager

• dbms_sqltune.report_sql_monitor

– Text, HTML or Active

• Active should be preferred


SQL Monitor Report Examples

Loading Data with Hash Join

• Before.html

• After.html

• http://www.os2ora.com/use-sql-monitor-report-to-tune-and-diagnose-sql/

http://www.os2ora.com/wp-content/uploads/2010/05/before.html

http://www.os2ora.com/wp-content/uploads/2010/05/after.html

http://www.os2ora.com/wp-content/uploads/2010/05/after.html

http://www.os2ora.com/use-sql-monitor-report-to-tune-and-diagnose-sql/



















HASH JOIN BUFFERED(1)



HASH JOIN BUFFERED(2)

• http://www.os2ora.com/test-sql-monitor-report-example/

http://www.os2ora.com/test-sql-monitor-report-example/










AWR


Performance Improvement Techniques

• Performance Hacker

– Tuning by Google

– Tuning by pattern matching or word association

– Tuning by what worked well on another system

– Tuning by Folklore

• Performance Engineer

– Understands Performance is all about work done in a period

of time and systematically learns where the time is spent

before making any recommendations

– Tuning is seen as a never ending process to systematically

locate where the time is spent and making changes that

reduces where the time is spent most


How to Analyze an AWR Report

• Is the CPU busy?

• Is it a heavy workload?

• Is the IO busy?

• What does the workload look like?

• What other info do we need?


AWR Reports

JSAMIS



• Is the IO busy?




AWR Reports

EHOME



• Is the IO busy?




Questions & Answers

Oracle Database Performance Tunning€¦ · Oracle Database Performance Tunning Kaiyao, ... delete...

Documents

Transcript of Oracle Database Performance Tunning€¦ · Oracle Database Performance Tunning Kaiyao, ... delete...