The Burden of Proof Jonathan Lewis .

The Burden of Proof

Jonathan Lewis

www.jlcomp.demon.co.uk

NoCOUG Keynote The Burden of Proof2Jonathan Lewis

© 2002-2003

Agenda

Who am I ?

Why do we need proof ?

What is proof ?

Constructing proofs

Where next ?

Q & A


© 2002-2003

Who am I ?

Independent Consultant.

18 years experience.

Design, Strategy, Reviews, Briefings, Seminars, Tutorials, Trouble-shooting

www.jlcomp.demon.co.uk


© 2002-2003

Why do we need proof ?

How does an architect design a house ?– Soil Characteristics– Material Strengths– Heating requirements– Illumination

Why are so many Cathedrals standing after 900 years ?

Because they are the second, third or fourth attempt.

Industry Standard Formulae


© 2002-2003

"Proof" by intuition

C = DC = 2r

= 3.1415926 ….. 3

C 6r

To add 36 inches to the circumference, you have to add 6 inches to the radius.


© 2002-2003

"Proof" by intuition

Problem:One transaction takes 0.1 seconds, which is good

The overnight batch takes 3 hours, which is bad.

The "batch" emulated 100,000 transactions !

100,000 * 0.1 = 10,000 seconds.

10,000 / 3,600 = 2 hours 47 minutes

You have to 'fix' the transaction.

The manual says:

"redo log space requests

should be close to zero".


© 2002-2003

Skinner Boxes

Which pig does all the work ?


© 2002-2003

"Proof" by anthropomorphism

• Pigs are not people.

• The Oracle database engine is not a person.

How would you handle this query:

select max(partitioning_col)

frompartitioned_table;


© 2002-2003

"Proof" by anthropomorphism

Quote from Metalink forum:

I would have thought Oracle was smart enough to know that it just has to go to the top partition to get max(pt_col)."

What if the top partition is empty ?

Smart code for max() in indexes only appeared in 8.1.

One day partition code might start from the top down.

How will you discover that this new feature has appeared.


© 2002-2003

"Proof" by self-deception

XThe truth is out there somewhere


© 2002-2003

"Proof" by self-deception

select count(*) from t1;

select count(1) from t1;

select count(pk_column) from t1;

SORT (AGGREGATE)

BITMAP CONVERSION (COUNT)

BITMAP INDEX (FAST FULL SCAN) OF T1_BIT


© 2002-2003

The Dodo argument


© 2002-2003

An Oracle Dodo

I had I/O problems and saw lots of sorts (disk) in v$sysstat. My I/O problems went away when I increased my sort_area_size.

The Dodo says:

If you have I/O problems and lots of sorts to disk, increase your sort_area_size.

The scientist says:

What I/O stopped happening, and why ?


© 2002-2003

Proving or Testing

Why do we need proofs ?

Why do we test ?

What's the difference ?

We test for our own benefit.

A proof is what we pass on to others.

The difference is in the publication.


© 2002-2003

Testing.

Why do we test ?

It avoids expensive mistakes.

How do we test ?

As cost-effectively as possible.


© 2002-2003

Why do we test ?

The Manuals

Here be dragons

To learn anything other than the stuff you find in books, you need to be able to experiment, to make mistakes, to accept feedback and to try again.

Charles Handy


© 2002-2003

Case Study (1)

create table t1 (

id number primary key,

v1 varchar2(20)

)

organization index

partition by range (id)

subpartition by hash (id) (partition p1 values less than (10)

);

ORA-25198: only range partitioning ...


© 2002-2003

How do we test ?

• Don't even try to be realistic to start with - test the concept, not the detailed target.

• If you want to find out if 'feature X' works: imagine the simplest possible case where it is most likely to happen

• If you need 'feature X' to work: don't design a test to see if it works - imagine the simplest possible case that breaks it.


© 2002-2003

Designing tests is hard

Try drawing a 'random' triangle


© 2002-2003

Case study (2)

create view daily_sales_pv as

select s.* from t_1998_01_01 s, dates d

where s.day_code = d.day_code

union all

...

union all

select s.* from t_1998_12_13 s, dates d

where s.day_code = d.day_code

;

The designer thought this was a partition view, and tested it with 1,000 items of data. Response time was good.


© 2002-2003

How do we test ?

• Know how to measure the results– sql_trace

– tkprof

– netstat

– iostat

– v$ snapshots

Not the clock !

• Know how to generate data cheaply– dbms_random

– trunc(), mod(), sequences


© 2002-2003

Proving or Testing

A Test becomes a Proof when:

it has a specification

the results are reproducible

alternative explanations have been eliminated

it is published

it survives peer-group review


© 2002-2003

Example of test/proof

Is memory 14,000 times faster than disk ?

Assume disks can do 100 I/Os per sec

So the hypothesis claims that you could do 1.4M logical I/Os per sec. in memory


© 2002-2003

System used to test

O/S Windows 2000

CPU Speed 700 MHz

Oracle 8.1.7.4

db_block_size 8K

create table t1 (n, v, primary key(n))

organization index as

select rownum n, to_char(rownum) v

from all_objects

where rownum <= 21;


© 2002-2003

Peer Group reviewdeclare

m_v varchar2(32);

begin

for i in 1..140000 loop

select v into m_v from t1 where n = 1;

end loop;

end;

Consistent Gets = 140,000

The hypothesis predicts 0.01 CPU seconds

Actual CPU used by this session = 16.18 seconds

What's wrong with the test ?


© 2002-2003

Second attempt

select count(*) X from t1

connect by n > prior n

start with n = 1;

CR gets 1,000,000

Buffer is pinned 500,000

Total 1,500,000

The hypothesis predicts a little over 1 CPU second

Actual CPU used by this session:15.08 seconds


© 2002-2003

Warning:

Is memory 14,000 times faster than disk ? (No)

Does it matter ?

Was it worth the effort of proving the point ?

Is it a generally applicable observation ?

Be careful about the line between pure research and practical investigation.


© 2002-2003

Relevance

declare

m_v varchar2(32);

begin

for i in 1..140000 loop

select v into m_v from t1 where n = 1;

end loop;

end; Does a small table need an index ?

Should a small table BE an index (IOT) ?

Or a single table hash cluster ?


© 2002-2003

Quality

Designing detailed tests is hard work.

Covering all the options is difficult.

The best results come from group efforts.

Thorough proofs tend to be cumulative– but technology changes push proofs out of date


© 2002-2003

Where next ?

Always ask for some proof– Books and magazines– FAQs– User Group websites

Questions to ask of authors:– How would you demonstrate that ?– Under what circumstances is this true ?– Do you have a sample script ?


© 2002-2003

Where next ?

Document your own tests

Publish your findings - especially problems.– www.jlcomp.demon.co.uk/faq/ind_faq.html– www.dbazine.com

Lobby Oracle Corp. for realistic examples– They are getting much better– There are still too many single table examples


© 2002-2003

The Benefits

• You are not the first !

• You can build on previous examples

• It enhances your skills and understanding

• You avoid problems earlier and earlier


© 2002-2003

Final Thoughts

When you have eliminated the impossible, whatever remains, however improbable, must be the truth.

Sir Arthur Conan Doyle

If I have seen further than others, it is because I was standing on the shoulders of the giants who went before.

Sir Isaac Newton

Please provide a reproducible test case.

Oracle Support

The Burden of Proof Jonathan Lewis .

Documents

Transcript of The Burden of Proof Jonathan Lewis .