Performance tuning using SQL new features

Performance tuning using SQL new features

By

Riyaj Shamsudeen

RMOUG 2008 Riyaj Shamsudeen 2

Who am I?

15 years using Oracle products Over 14 years as Oracle DBA Certified DBA versions 7.0,7.3,8,8i &9i Specializes in performance tuning, Internals and E-business suite Currently consulting for Cingular through Avion systems Adjunct faculty – Northlake College OakTable member Email: rshams4 at yahoo.com


Agenda

Model clause

Analytic functions Lag and lead First_value and last_value Dense_rank

Subquery factoring

Questions


Model

Emulates spreadsheet like functionality

Very powerful, very flexible

Code is very compact


Model Table used: item_Data , Randomly generated data

CREATE TABLE ITEM_DATA( ITEM VARCHAR2(10 BYTE), LOCATION NUMBER, WEEK NUMBER, SALES_QTY NUMBER, RCPT_QTY NUMBER)ITEM LOCATION WEEK SALES_QTY RCPT_QTY---------- ---------- ---------- ---------- ----------Shirt 1 1 623 55Shirt 1 2 250 469Shirt 1 3 882 676Shirt 1 4 614 856…..Shirt 1 8 409 541Shirt 1 9 891 790Shirt 1 10 759 465Shirt 2 1 261 515Shirt 2 2 664 67


Problem #1: Tune inventory report

Formula:Inventory = last_week_inventory +

this week receipts –this week sales

Accesses previous rows


Solution :Original SQL

select item, location, week, (select sum(rcpt_qty)-sum(sales_qty) from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week <= out1.week ) inventory, sales_qty, rcpt_qty from item_data out1 order by item , location,week/

ITEM LOCATION WEEK INVENTORY SALES_QTY RCPT_QTY---------- ---------- ---------- ---------- ---------- ----------Pants 1 1 55 692 747Pants 1 2 188 98 231Pants 1 3 277 235 324Pants 1 4 -158 576 141Pants 1 5 169 123 450Pants 1 6 -544 731 18Pants 1 7 -489 341 396Pants 1 8 -481 185 193Pants 1 9 -505 198 174Pants 1 10 -817 727 415Pants 1 11 -794 275 298

Total logical reads: 3007


Explain plan

call count cpu elapsed disk query current rows------- ------ -------- ---------- ---------- ---------- ---------- ----------Parse 1 0.00 0.00 0 2 0 0Execute 1 0.00 0.00 0 0 0 0Fetch 41 0.11 0.10 0 3005 0 600------- ------ -------- ---------- ---------- ---------- ---------- ----------total 43 0.11 0.10 0 3007 0 600

Misses in library cache during parse: 1Optimizer mode: ALL_ROWSParsing user id: 30

Rows Row Source Operation------- --------------------------------------------------- 600 SORT AGGREGATE (cr=3000 pr=0 pw=0 time=97619 us) 6300 TABLE ACCESS FULL ITEM_DATA (cr=3000 pr=0 pw=0 time=51167 us) 600 SORT ORDER BY (cr=3005 pr=0 pw=0 time=101365 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=32 us)


Model – Excel like..

week ------------------------------------------------------------------------>location 1 2 3 4 5 6 7 8 9 101-sales_qty 199 433 888 165 338 611 858 37 778 1401-rcpt_qty 232 787 714 836 245 699 489 843 787 6741-inventory 33 387 213 884 791 879 510 1316 1325 18592-sales_qty 697 263 …2-rcpt_qty 676 112 …

In Excel, formula would be B5+C4-C3 in Shirts sheet.

1

2

3

4

5

A B C D E


Model

select item, location, week, inventory, sales_qty, rcpt_qty from item_data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty) rules ( inventory [location, week]= nvl(inventory[cv(location), cv(week)-1],0) - sales_qty [cv(location), cv(week) ] +rcpt_qty [cv(location), cv(week) ] ) order by item , location, week /

Script: model_example2.sql


Model

ITEM LOCATION WEEK INVENTORY SALES_QTY RCPT_QTY---------- ---------- ---------- ---------- ---------- ----------Pants 1 1 33 199 232Pants 1 2 387 433 787Pants 1 3 213 888 714Pants 1 4 884 165 836Pants 1 5 791 338 245Pants 1 6 611 699Pants 1 7 510Pants 1 8 1316 37 843Pants 1 9 1325 778 787Pants 1 10 1859 140 674Pants 2 1 -21 697 676Pants 2 2 -172 263 112…Shirts 1 1 23 15 105Shirts 1 2 43 115 25

model return updated rows

partition by (item)

dimension by (location, week)

measures ( 0 inventory , sales_qty, rcpt_qty)

rules ( inventory [location, week]= nvl(inventory[cv(location), cv(week)-1],0) - sales_qty [cv(location), cv(week) ] +rcpt_qty [cv(location), cv(week) ] )

inventory [location, week]= nvl(inventory[cv(location), cv(week)-1],0) - sales_qty [cv(location), cv(week) ] +rcpt_qty [cv(location), cv(week) ]

1 7 7-11

879

858

-+

489

Cv(location) refers to current value of location from left side of rule


Location Week Rcpt Sales Inventory1 1 623 55 0+(623-55)=568

1 2 250 469 568+(250-469)=349

1 3 882 676 349+(882-676)=555

1 4 614 856 555+(614-856)=313

1 5 401 281 313+(401-281)=433

Formula inventory [1,3]= nvl(inventory[1, 2],0) - sales_qty [1,3 ] +rcpt_qty [1, 3) ]

inventory [location, week]= nvl(inventory[cv(location), cv(week)-1],0) - sales_qty [cv(location), cv(week) ] +rcpt_qty [cv(location), cv(week) ]


Model

SQL Output:

ITEM LOCATION WEEK INVENTORY SALES_QTY RCPT_QTY---------- ---------- ---------- ---------- ---------- ----------Pants 1 1 33 199 232Pants 1 2 387 433 787Pants 1 3 213 888 714Pants 1 4 884 165 836Pants 1 5 791 338 245Pants 1 6 879 611 699Pants 1 7 510 858 489Pants 1 8 1316 37 843Pants 1 9 1325 778 787Pants 1 10 1859 140 674Pants 2 1 -21 697 676Pants 2 2 -172 263 112

Notice the value is reset, at the onset of different location.

nvl( inventory[cv(location), cv(week)-1],0)

Accessing non-existent rowsReturns null


Explain plan



Rows Row Source Operation------- --------------------------------------------------- 600 SORT ORDER BY (cr=5 pr=0 pw=0 time=26842 us) 600 SQL MODEL ORDERED (cr=5 pr=0 pw=0 time=17662 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=39 us)

6 logical reads,Compared to 3007 logical reads.

Table accessed just once


More performance issues…

Suppose, if the SQL need to calculate running total ofsales_qty also..

select item, location, week, (select sum(rcpt_qty)-sum(sales_qty) from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week <= out1.week ) inventory, (select sum(sales_qty) from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week <= out1.week ) sales_so_far, sales_qty, rcpt_qty from item_data out1order by item , location,week

This section added


Explain plan



Rows Row Source Operation------- --------------------------------------------------- 600 SORT AGGREGATE (cr=3000 pr=0 pw=0 time=96618 us) 6300 TABLE ACCESS FULL ITEM_DATA (cr=3000 pr=0 pw=0 time=50626 us) 600 SORT AGGREGATE (cr=3000 pr=0 pw=0 time=101683 us) 6300 TABLE ACCESS FULL ITEM_DATA (cr=3000 pr=0 pw=0 time=53670 us) 600 SORT ORDER BY (cr=6005 pr=0 pw=0 time=203392 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=32 us)

Logical reads jumped from 3007 to 6008After adding running_sales_qty metrics


Model

select item, location, week, inventory, sales_qty, rcpt_qty , sales_so_for, rcpt_so_for from item_data model return updated rows partition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty, 0 sales_so_for ) rules ( inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ], sales_so_for [location, week] = sales_qty [cv(location), cv(week)] + nvl(sales_so_for [cv(location), cv(week)-1],0),)order by item , location,week


We can easily add another metrics, withoutany major performance impact, with model.


Model - Rules

sales_so_for [location, week] = sales_qty [cv(location), cv(week)] +

nvl(sales_so_for [cv(location), cv(week)-1],0),

sales_so_for [ 10, 1] = sales_qty [ 10, 1 ] + nvl ( sales_so_for [ 10, 0], 0 )


Explain plan



Rows Row Source Operation------- --------------------------------------------------- 600 SORT ORDER BY (cr=5 pr=0 pw=0 time=37941 us) 600 SQL MODEL ORDERED (cr=5 pr=0 pw=0 time=25158 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=40 us)

No increase in logical reads, even afterAdding a new metric


Model

select item, location, week, inventory, sales_qty, rcpt_qty , sales_so_for, rcpt_so_for from item_data model return updated rows partition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty, 0 sales_so_for, 0 rcpt_so_for ) rules ( inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ], sales_so_for [location, week] = sales_qty [cv(location), cv(week)] + nvl(sales_so_for [cv(location), cv(week)-1],0), rcpt_so_for [location, week] = rcpt_qty [cv(location), cv(week)] + nvl(rcpt_so_for [cv(location), cv(week)-1],0) )order by item , location,week


Added more metrics, still, no increase inLogical reads!


Model

Formula

Moving_sales_avg = Average of ( current week sales and last two week sales )

Problem #2: Tune moving sales average SQL


Old SQL

select item, location, week, ( sales_qty + nvl((select sales_qty from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week = out1.week-1 ), sales_qty) + nvl((select sales_qty from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week = out1.week-2 ),sales_qty) )/3 avg_running_sales, sales_qty, rcpt_qty from item_data out1order by item , location,week/

Current week sales_qty

Last week sales_qty

Week before last week sales_qty


Explain plan



Rows Row Source Operation------- --------------------------------------------------- 600 SORT ORDER BY (cr=2321 pr=2 pw=0 time=26089 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=645 us)

Total logical reads: 2321


Modelselect item, location, week, sales_qty, rcpt_qty , moving_sales_avg from item_data model return updated rows partition by (item)dimension by (location, week)measures ( 0 moving_sales_avg , sales_qty, rcpt_qty) rules ( moving_sales_avg [location, week] = (sales_qty [cv(location), cv(week)]+ nvl( sales_qty [cv(location, cv(week)-1 ], sales_qty [cv(location), cv(week)]

) + nvl(

sales_qty [cv(location), cv(week)-2 ], sales_qty [cv(location), cv(week)]

) ) /3 )order by item , location,week

Script : model_example3.sql

Calculating the moving sales avg using this week and past two weeks..

current week sales_qty

Last week sales_qty

Week before Last week


Model – explain plan

call count cpu elapsed disk query current rows

------- ------ -------- ---------- ---------- ---------- ---------- ----------

Parse 1 0.03 0.02 0 39 208 0

Execute 1 0.00 0.00 0 0 0 0

Fetch 41 0.02 0.02 0 5 0 600

------- ------ -------- ---------- ---------- ---------- ---------- ----------

total 43 0.05 0.04 0 44 208 600

Misses in library cache during parse: 1

Optimizer mode: ALL_ROWS

Parsing user id: 30

Rows Row Source Operation

------- ---------------------------------------------------

600 SQL MODEL ORDERED (cr=5 pr=0 pw=0 time=14086 us)

600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=43 us)

Just 5 logical reads in Fetch step


select item, location, week, inventory, sales_qty, rcpt_qty, max_inventory from item_data model return updated rows partition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory) rules (inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ], max_inventory [location, week] = max(inventory) [cv(location), ANY ] )order by item , location,week

Model & Aggregation Aggregation function also possible. Max value of inventory for a store calculated here. Second rule is referring first rule.

ANY keyword substitutes for all possible Values in that partition.

ITEM LOCATION WEEK INVENTORY SALES_SO_FAR SALES_QTY RCPT_QTY MAX_INVENTORY---------- ---------- ---------- ---------- ------------ ---------- ---------- -------------Pants 1 1 -819 948 948 129 339Pants 1 2 -10 995 47 856 339Pants 1 3 294 1353 358 662 339

Script: model_features.sql


select item, location, week, inventory, sales_qty, rcpt_qty, max_inventory from item_data model return updated rowspartition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory)rules (inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ],max_inventory [location, week] = max(inventory) [cv(location),

week between cv(week)-5 and cv(week)+5 ] )order by item , location,week/

Model & Aggregation More granular boundary conditions can be specified too. Maximum inventory within a 10 weeks boundary.


ITEM LOCATION WEEK INVENTORY SALES_SO_FAR SALES_QTY RCPT_QTY MAX_INVENTORY---------- ---------- ---------- ---------- ------------ ---------- ---------- -------------

Ties 8 1 -127 587 587 460 378Ties 8 2 370 1034 447 944 562Ties 8 3 -376 1898 864 118 963Ties 8 4 -252 1961 63 187 1263Ties 8 5 378 2191 230 860 1263Ties 8 6 272 3017 826 720 1263Ties 8 7 562 3042 25 315 1263Ties 8 8 963 3417 375 776 1263Ties 8 9 1263 4057 640 940 1263Ties 8 10 1226 4912 855 818 1263Ties 8 11 584 5690 778 136 1263Ties 8 12 864 6355 665 945 1263Ties 8 13 564 7030 675 375 1263Ties 8 14 280 7962 932 648 1263Ties 8 15 330 8910 948 998 1226

Model & Aggregation Data from prior SQL for a 10 weeksboundary


Model uses analytic functions under the cover:

Column Projection Information (identified by operation id):1 - "ITEM"[10], "LOCATION"[22], "WEEK"[22], 0[22], "RCPT_QTY"[22], "SALES_QTY"[22], 0[22]2 - "ITEM"[10], "LOCATION"[22], "WEEK"[22], 0[22], 0[22],

"SALES_QTY"[22], "RCPT_QTY"[22]3 - "ITEM"[VARCHAR2,10], "LOCATION"[NUMBER,22],

"WEEK"[NUMBER,22], "SALES_QTY"[NUMBER,22], "RCPT_QTY"[NUMBER,22]

4 - "LOCATION"[22], "WEEK"[22], 0[22], MAX(0) OVER ( PARTITION BY "LOCATION“ ORDER BY "WEEK" RANGE BETWEEN 5 PRECEDING AND 5 FOLLOWING )[22]

Model & AggregationColumn projection for aggregation rule:max_inventory [location, week] = max(inventory) [cv(location), week between cv(week)-5 and cv(week)+5 ]

Uses analytic functions under the cover


explain plan for select * from ( select ….from item_data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty) rules ( inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week)-1 ] + rcpt_qty [cv(location), cv(week)-1 ] ) order by item , location, week) where item='Pants' and location=100 and week =10;

select * from table(dbms_xplan.display(null,'','typical'));

Script: model_pred_push.sql

Model – Predicate push In this SQL, predicates are for all three columns. But, Model rule accesses prior week. Pushing week predicate earlier in the step, will eliminate other weeks, causing logic issues.

Week predicate is not pushed here

------------------------------------------------| Id | Operation | Name | Rows | Bytes |----------------------------------------------------------| 0 | SELECT STATEMENT | | 6 | 432 ||* 1 | VIEW | | 6 | 432 || 2 | SORT ORDER BY | | 6 | 354 || 3 | SQL MODEL ORDERED | | 6 | 354 ||* 4 | TABLE ACCESS FULL| ITEM_DATA | 6 | 354 |----------------------------------------------------------

PLAN_TABLE_OUTPUT---------------------------------------------------------- 1 - filter("WEEK"=10)

4 - filter("ITEM"='Pants' AND "LOCATION"=100)


explain plan for select * from ( select ….from item_data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty) rules ( inventory [location, week] = nvl(inventory [cv(location), cv(week) ] ,0) - sales_qty [cv(location), cv(week) ] + rcpt_qty [cv(location), cv(week) ] ) order by item , location, week) where item='Pants' and location=100 and week =10;


Model – Predicate push In this SQL, it is safe to push all three predicates, without affecting functionality.

All predicates are pushed here

------------------------------------------------| Id | Operation | Name | Rows | Bytes |----------------------------------------------------------| 0 | SELECT STATEMENT | | 6 | 432 ||* 1 | VIEW | | 6 | 432 || 2 | SORT ORDER BY | | 6 | 354 || 3 | SQL MODEL ORDERED | | 6 | 354 ||* 4 | TABLE ACCESS FULL| ITEM_DATA | 6 | 354 |----------------------------------------------------------

PLAN_TABLE_OUTPUT---------------------------------------------------------- 1 - filter("WEEK"=10) 4 - filter("ITEM"='Pants' AND "LOCATION"=100

AND "WEEK"=10)


explain plan for select * from ( select item, location, week, inventory, sales_qty,

rcpt_qty from item_data model return updated rows partition by (item, location) dimension by ( week) measures ( 0 inventory , sales_qty, rcpt_qty) rules ( inventory [ week] = nvl(inventory [cv(week) ] ,0) - sales_qty [cv(location)-1 ] + + rcpt_qty [cv(week) ] ) order by item , location, week) where item='Pants' and location=100 and week =10/


Model – Predicate push

Location predicate is pushed even though Rule accesses prior row, since location is a partitioning column

--------------------------------------------------| Id | Operation | Name | Rows |--------------------------------------------------| 0 | SELECT STATEMENT | | 7 ||* 1 | VIEW | | 7 || 2 | SORT ORDER BY | | 7 || 3 | SQL MODEL ORDERED | | 7 ||* 4 | TABLE ACCESS FULL| ITEM_DATA | 7 |--------------------------------------------------PLAN_TABLE_OUTPUT--------------------------------------------------Predicate Information (identified by operation id):--------------------------------------------------- 1 - filter("WEEK"=10)

4 - filter("ITEM"='Pants' AND "LOCATION"=100)

This predicate push works incorrectly in 10.1.0.4. Throws ORA-7445 in 10.2.0.3,11..1.0.6.


explain plan for select m.* from ( select item, location, week, inventory,

sales_qty, rcpt_qty from item_data model return updated rows partition by (item,location) dimension by ( week) measures ( 0 inventory , sales_qty,

rcpt_qty) rules ( inventory [ week] =

nvl(inventory [cv(week) ] ,0) - sales_qty [cv(week) ] + + rcpt_qty [cv(week) ] ) order by item , location, week) m, item_data i1Where m.item = i1.item and m.week = i1.week and m.location = i1.locationand i1.item='Pants' and i1.location=100

and i1.week =10;

Model – Predicates & transitivity

-----------------------------------------------------| Id | Operation | Name | Rows |-----------------------------------------------------| 0 | SELECT STATEMENT | | 1 || 1 | MERGE JOIN CARTESIAN | | 1 ||* 2 | TABLE ACCESS FULL | ITEM_DATA | 1 || 3 | BUFFER SORT | | 1 || 4 | VIEW | | 1 || 5 | SORT ORDER BY | | 1 || 6 | BUFFER SORT | | 1 || 7 | SQL MODEL ORDERED | | 1 ||* 8 | TABLE ACCESS FULL| ITEM_DATA | 1 |-----------------------------------------------------

Predicate Information (identified by operation id):---------------------------------------------------

2 - filter("I1"."ITEM"='Pants' AND "I1"."LOCATION"=100 AND "I1"."WEEK"=10) 8 - filter("ITEM"='Pants' AND "LOCATION"=100 AND "WEEK"=10)

Predicates are on i1 alias and these predicates are applied to model subquery also transitively.


explain plan forselect item, location, week, inventory, sales_qty,

rcpt_qty, max_inventory from item_data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory) rules ( inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + rcpt_qty [cv(location), cv(week) ] , max_inventory [location, week] =

max(inventory) [cv(location), week between cv(week)-5 and cv(week)+5 ]

)order by item , location,week/Script : model_order.sql

Rule evaluation order

--------------------------------------------------| Id | Operation | Name |--------------------------------------------------| 0 | SELECT STATEMENT | || 1 | SORT ORDER BY | || 2 | SQL MODEL ORDERED | || 3 | TABLE ACCESS FULL | ITEM_DATA || 4 | WINDOW (IN SQL MODEL) SORT| |-------------------------------------------------------------

Rules are evaluated in the orderSpecified, by default. Note ORDERED keyword in the plan.



rcpt_qty, max_inventory from item_data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory) rules sequential order ( inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + rcpt_qty [cv(location), cv(week) ] , max_inventory [location, week] =


)order by item , location,week/


--------------------------------------------------| Id | Operation | Name |--------------------------------------------------| 0 | SELECT STATEMENT | || 1 | SORT ORDER BY | || 2 | SQL MODEL ORDERED | || 3 | TABLE ACCESS FULL | ITEM_DATA || 4 | WINDOW (IN SQL MODEL) SORT| |-------------------------------------------------------------

Sequential order can be explicitly specified



rcpt_qty, max_inventory from item_data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory) rules automatic order ( inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + rcpt_qty [cv(location), cv(week) ] , max_inventory [location, week] =


)order by item , location,week/


--------------------------------------------------| Id | Operation | Name |--------------------------------------------------| 0 | SELECT STATEMENT | || 1 | SORT ORDER BY | || 2 | SQL MODEL CYCLIC | || 3 | TABLE ACCESS FULL | ITEM_DATA || 4 | WINDOW (IN SQL MODEL) SORT| |-------------------------------------------------------------

If rules have many inter dependencies, use automatic order. Oracle will rearrange rule order evaluation optimally.


select item, location, week, inventory, sales_qty, rcpt_qty, max_inventory from item_data model return updated rowspartition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory)rules (inventory [location, for week from 3 to 9 increment 1 ] = nvl(inventory [cv(location),

cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ] )order by item , location,week/

Model & For loop For loop can be specified in the rule, only range of weeks between 3 & 9 printed in this example.


Create table iter (n1 number);

select fib from itermodel return updated rowsdimension by (n1)measures (0 fib, n1) ignore navrules iterate (50) ( fib [-2]=0, fib[-1]=1, fib [iteration_number ]=

fib [iteration_number-1 ] + fib [iteration_number-2] )order by fib/

Model & iterate Model can iterate rule in a loop until maximum number of iteration reached or until condition reached.

0 1 1 2 3 5 8 13 21 34 55

89 144 233 377 610 987 1597 2584 4181 6765 10946


select item, location, week, inventory, sales_qty, rcpt_qty, max_inventory from item_data model return updated rowspartition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory)rules (inventory [location in (10,20) , for week from 3 to 9 increment 1 ] = nvl(inventory

[cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ] )order by item , location,week/

Model & in phrase For loop can be specified in the rule, only range of weeks between 3 & 9 printed in this example.


select item, location, week, inventory, sales_qty, rcpt_qty, max_inventory from item_data model return updated rowspartition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory)rules (inventory [10, for week from 3 to 9 increment 1 ] =

nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ] )order by item , location,week/

Model & positional notationPositional value specified. Rows with location=10.


select item, location, week, inventory, sales_qty, rcpt_qty from item_data model return updated rowspartition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty) ignore navrules (inventory [10 , for week from 3 to 9 increment 1 ] = inventory [cv(location), cv(week)-1 ] - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ] )order by item , location,week/

Model & Ignore NavNot Available Values [ non-existent rows or null values] considered as 0 instead of nulls in arithmetic calculation.


select item, location, week, inventory, sales_qty, rcpt_qty from item_datawhere location=:"SYS_B_0" model return updated rows partition by (item)dimension by (location, week)measures ( :"SYS_B_1" inventory , sales_qty, rcpt_qty) rules ( inventory [location, week] = nvl(inventory [cv(location), cv(week)-:"SYS_B_2" ]

,:"SYS_B_3") - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ] )order by item , location,week


Misses in library cache during parse: 1Misses in library cache during execute: 1Optimizer mode: ALL_ROWS

Works fine with Cursor_sharing set to force too

Model


Of course, if rules are inconsistent, errors thrown:explain plan forselect item, location, week, inventory, sales_qty, rcpt_qty, max_inventory from item_data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty, 0 max_inventory) rules automatic ORDER ( max_inventory [location, week] =

max(inventory) [cv(location), week between cv(week)-5 and cv(week)+5 ], inventory [location, week] = rcpt_qty [cv(location), cv(week) ] - sales_qty [cv(location),

cv(week) ] + max_inventory [location, week] ) order by item , location,week/ERROR at line 11:ORA-32622: illegal multi-cell reference

Gotcha Two rules are cyclically referring to each other. So, illegal multi-cell reference is thrown.


In 10.2.0.1 and 11.1.0.6, model errors out:

select item, location, week, inventory, sales_qty, rcpt_qty from item_data model return updated rows partition by (item)dimension by (location, week)measures ( 0 inventory , sales_qty, rcpt_qty) rules ( inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + + rcpt_qty [cv(location), cv(week) ] )order by item , location,week/select item, location, week, inventory, sales_qty, rcpt_qty*ERROR at line 1:ORA-32637: Self cyclic rule in sequential order MODEL

Gotcha

Just specify rules automatic order To circumvent this bug.


View merging is not considered in SQL with model clause ( at least until 10.2.0.3 )

Few lines from 10053 trace ;

CVM: Considering view merge in query block SEL$1 (#0)CVM: Checking validity of merging SEL$2 (#0)CVM: Considering view merge in query block SEL$2 (#0)CVM: CVM bypassed: MODEL in this view

Gotcha

Analytic functions


Analytic functions

Powerful feature provides ability to access previous and following rows

SQL can be written concisely using these functions

Works great for data warehouse and reporting queries


Running sales total

select item, location, week, (select sum(sales_qty) from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week <= out1.week ) running_sales_total, sales_qty, rcpt_qty from item_data out1order by item , location,week

Script: anfn_example1.sql

Problem: To calculate running sales total for an item and location (store) by week.


Running sales total

ITEM LOCATION WEEK SALES_QTY RCPT_QTY RUNNING_SALES_TOTAL---------- ---------- ---------- ---------- ---------- -------------------Pants 1 1 638 524 638Pants 1 2 709 353 1347Pants 1 3 775 734 2122Pants 1 4 344 879 2466Pants 1 5 990 744 3456Pants 1 6 635 409 4091Pants 1 7 524 801 4615Pants 1 8 14 593 4629Pants 1 9 630 646 5259Pants 1 10 61 241 5320Pants 2 1 315 737 315Pants 2 2 176 282 491Pants 2 3 999 314 1490


Performance – old SQL

all count cpu elapsed disk query current rows------- ------ -------- ---------- ---------- ---------- ---------- ----------Parse 1 0.00 0.00 0 2 0 0Execute 1 0.00 0.00 0 0 0 0Fetch 41 0.04 0.02 2 1232 0 600------- ------ -------- ---------- ---------- ---------- ---------- ----------total 43 0.04 0.02 2 1234 0 600


Rows Row Source Operation------- --------------------------------------------------- 600 SORT AGGREGATE (cr=1227 pr=2 pw=0 time=21528 us) 6300 TABLE ACCESS BY INDEX ROWID ITEM_DATA (cr=1227 pr=2 pw=0 time=19522 us) 6300 INDEX RANGE SCAN ITEM_I1 (cr=611 pr=2 pw=0 time=11008 us)(object id 44482) 600 SORT ORDER BY (cr=1232 pr=2 pw=0 time=24812 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=33 us)


Running sales total – New way

select item, location, week, sales_qty, rcpt_qty, sum (sales_qty) over (partition by item, location order by week rows between unbounded preceding and current row ) running_sales_total from item_data



Running sales total – New way



Rows Row Source Operation------- --------------------------------------------------- 600 WINDOW SORT (cr=5 pr=0 pw=0 time=2606 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=33 us)


Running sales total


sum

sum


Running sales total ITEM LOCATION WEEK SALES_QTY RCPT_QTY---------- ---------- ---------- ---------- ----------Pants 1 1 638 524Pants 1 9 630 646Pants 1 2 709 353Pants 2 1 315 737Pants 1 4 344 879Pants 1 10 61 241Pants 1 5 990 744Pants 1 6 635 409Pants 2 2 176 282Pants 1 3 775 734Pants 1 7 524 801Pants 1 8 14 593Pants 2 3 999 314

rows between unbounded preceding and current row

)

partition by item, locationorder by week

sum (sales_qty) over (


Sorts data by item,location

and week

Calculates sum of prior rows in this partition and current row


Moving sales average

Problem: To calculate moving sales average for an item and location (store).

Moving_sales_avg = Average of (sales_qty for this week and past two weeks).

select item, location, week, sales_qty, rcpt_qty, (select avg(sales_qty) from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week <= out1.week and in1.week >= out1.week-2 ) moving_sales_avg from item_data out1order by item , location,week


Moving sales average



Rows Row Source Operation------- --------------------------------------------------- 600 SORT AGGREGATE (cr=1209 pr=2 pw=0 time=17194 us) 1710 FILTER (cr=1209 pr=2 pw=0 time=12288 us) 1710 TABLE ACCESS BY INDEX ROWID ITEM_DATA (cr=1209 pr=2 pw=0 time=10003 us) 1710 INDEX RANGE SCAN ITEM_I1 (cr=605 pr=2 pw=0 time=5878 us)(object id 46238) 600 SORT ORDER BY (cr=1214 pr=2 pw=0 time=20611 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=35 us)


Moving sales average – New way

select item, location, week, sales_qty, rcpt_qty, avg (sales_qty) over ( partition by item, location order by week rows between 2 preceding and current row ) running_sales_total from item_data /


Specifying a window of current Row and past two rows


Moving sales average – New way

TEM LOCATION WEEK SALES_QTY RCPT_QTY RUNNING_SALES_TOTAL ---------- ---------- ---------- ---------- ---------- ------------------- Ties 9 1 795 312 795 Ties 9 2 957 927 876 Ties 9 3 612 787 788 Ties 9 4 81 415 550 Ties 9 5 161 813 284.666667 Ties 9 6 817 763 353 Ties 9 7 402 112 460 Ties 9 8 15 686 411.333333 Ties 9 9 682 11 366.333333 Ties 9 10 288 949 328.333333 Ties 9 11 407 205 459 Ties 9 12 429 83 374.666667 Ties 9 13 907 776 581 Ties 9 14 10 817 448.666667 Ties 9 15 379 883 432 Ties 9 16 694 665 361 Ties 9 17 581 235 551.333333 Ties 9 18 683 333 652.666667 Ties 9 19 384 872 549.333333 Ties 9 20 276 560 447.666667 Ties 10 1 585 501 585 Ties 10 2 981 455 783

Avg

Avg


Explain plan




5 logical reads compared to 1215 logical reads


Lead & Lag functions

Lead function provides ability to access later rows from that data partition

Lag function provides ability to access prior rows from that data partition

You can even specify the offset and default values.


Lag

Sales change percent formula= (this_week_sales - last_week_sales) /

last_week_sales ) *100

Problem: Tune query for sales_change_percent


Lag – old wayselect item, location, week, sales_qty, rcpt_qty, ((sales_qty - last_week_sales_qty )/last_week_sales_qty ) *100

sales_change_percentfrom (

select item, location, week, sales_qty, rcpt_qty,(select sales_qty from item_data in1

where in1.item=out1.item and in1.location=out1.location and in1.week = out1.week-1 ) last_week_sales_qty

from item_data out1order by item , location,week

)

Script : anfn_lag.sql


Lag – old way



Rows Row Source Operation------- --------------------------------------------------- 600 VIEW (cr=1178 pr=2 pw=0 time=13670 us) 600 SORT ORDER BY (cr=1178 pr=2 pw=0 time=13659 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=632 us)


Lag - tuned

select item, location, week, sales_qty, rcpt_qty, ((sales_qty - last_week_sales_qty )/last_week_sales_qty ) *100

sales_change_percentfrom (

select item, location, week, sales_qty, rcpt_qty,lag(sales_qty,1,sales_qty) over (partition by item, location order by week)

last_week_sales_qtyfrom item_data out1order by item , location,week

)Partitions data,

sorts rows by weekin that data partitions

Lag is to access prior row, in that partition. First argument : column_nameSecond argument : offsetThird argument : default value if out of partition


Sales change percent ITEM LOCATION WEEK SALES_QTY RCPT_QTY ---------- ---------- ---------- ---------- ----------Pants 1 2 239 425Pants 1 3 749 501Pants 1 1 426 331Pants 1 5 559 905Pants 1 7 555 958Pants 1 10 788 488Pants 1 4 702 886Pants 1 6 940 119Pants 1 8 553 662Pants 1 9 277 306Pants 1 13 738 581Pants 1 11 752 804Pants 1 12 988 220

over ( partition by item, location order by week)

lag(sales_qty,1,sales_qty)

ITEM LOCATION WEEK SALES_QTY RCPT_QTY SALES_CHANGE_PERCENT

---------- ---------- ---------- ---------- ---------- --------------------

Pants 1 1 426 331 0

Pants 1 2 239 425 -43.896714

Pants 1 3 749 501 213.389121

Pants 1 4 702 886 -6.2750334

Pants 1 5 559 905 -20.37037

Pants 1 6 940 119 68.157424

Pants 1 7 555 958 -40.957447

Pants 1 8 553 662 -.36036036

Pants 1 9 277 306 -49.909584

Pants 1 10 788 488 184.476534

Pants 1 11 752 804 -4.5685279

Pants 1 12 988 220 31.3829787

Sorts data by item,location

and week

Lag accesses prior row to calculate sales_change_percent

Sales change percent

((749-239)/239 )*100=213.3


Lag – sales_change_percent

ITEM LOCATION WEEK SALES_QTY RCPT_QTY SALES_CHANGE_PERCENT ---------- ---------- ---------- ---------- ---------- -------------------- Pants 1 1 426 331 0 Pants 1 2 239 425 -43.896714 Pants 1 3 749 501 213.389121 Pants 1 4 702 886 -6.2750334 Pants 1 5 559 905 -20.37037 Pants 1 6 940 119 68.157424 Pants 1 7 555 958 -40.957447 Pants 1 8 553 662 -.36036036 Pants 1 9 277 306 -49.909584 Pants 1 10 788 488 184.476534 Pants 1 11 752 804 -4.5685279 Pants 1 12 988 220 31.3829787 Pants 1 13 738 581 -25.303644

((749-239)/239 )*100=213.3


Lag



Rows Row Source Operation------- --------------------------------------------------- 600 VIEW (cr=5 pr=0 pw=0 time=4489 us) 600 WINDOW SORT (cr=5 pr=0 pw=0 time=4486 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=37 us)

6 buffer gets compared to 1180 buffer gets.


Lag and lead

Preceding Preceding This Succeeding SucceedingITEM LOCATION WEEK week 2 week 1 week week 1 week 2---------- ---------- ---------- ---------- ---------- ---------- ---------- ----------Pants 1 1 250 634 747Pants 1 2 634 250 634 747 777Pants 1 3 250 634 747 777 379Pants 1 4 634 747 777 379 309Pants 1 5 747 777 379 309 387Pants 1 6 777 379 309 387 199Pants 1 7 379 309 387 199 465Pants 1 8 309 387 199 465 859Pants 1 9 387 199 465 859 465Pants 1 10 199 465 859 Pants 2 1 453 453 453 281 855Pants 2 2 281 453 281 855 764Pants 2 3 453 281 855 764 864

Problem: Tune SQL that prints past two weeks and next two weeks sales_qty in the same line.


Lag and lead -oldselect item, location, week, rcpt_qty, (select sales_qty from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week = out1.week-2 ) prev_week_2_sales_qty, (select sales_qty from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week = out1.week-1 ) prev_week_1_sales_qty, sales_qty, (select sales_qty from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week = out1.week+1 ) succ_week_1_sales_qty, (select sales_qty from item_data in1 where in1.item=out1.item and in1.location=out1.location and in1.week = out1.week+2 ) succ_week_2_sales_qty from item_data out1order by item , location,week

Script : anfn_lead_n_lag.sql


Lag and leadcall count cpu elapsed disk query current rows

------- ------ -------- ---------- ---------- ---------- ---------- ----------

Parse 1 0.00 0.00 0 5 0 0

Execute 1 0.00 0.00 0 0 0 0

Fetch 41 0.05 0.04 2 4667 0 600

------- ------ -------- ---------- ---------- ---------- ---------- ----------

total 43 0.05 0.05 2 4672 0 600



Parsing user id: 30


------- ---------------------------------------------------

540 TABLE ACCESS BY INDEX ROWID ITEM_DATA (cr=1143 pr=0 pw=0 time=8574 us)

540 INDEX RANGE SCAN ITEM_I1 (cr=603 pr=0 pw=0 time=5512 us)(object id 46444)







600 SORT ORDER BY (cr=4667 pr=2 pw=0 time=43306 us)



Lag and lead - tuned

select item, location , week, rcpt_qty, lag (sales_qty, 2) over ( partition by item, location

order by week asc ) prev_week_2_sales_qty, lag (sales_qty, 1) over ( partition by item, location

order by week asc ) prev_week_1_sales_qty, sales_qty, lead (sales_qty, 1) over ( partition by item, location

order by week asc ) succ_week_1_sales_qty, lead (sales_qty, 2) over ( partition by item, location

order by week asc ) succ_week_2_sales_qty from item_data

Prior weeks

Following weeks


Lag and lead - tuned




6 logical reads, compared to 4667 logical reads


First_value & last_value

These functions provides first or last values in an ordered set of rows

Very useful in finding the top or bottom values and also other columns in that same row.


First_value & last_valueProblem: Tune SQL that finds worst and best seller for an item by week

OLD way:select mi.item, mi.week, ma.max_location top_seller, ma.max_sales_qty top_sales_qty,mi.min_location worst_seller, mi.min_sales_qty worst_sales_qtyfrom(select item, week, sales_qty max_sales_qty, location max_location from item_data where (item, week, sales_qty) in ( select item, week, max(sales_qty) from item_data group by item, week ) ) ma,(select item, week, sales_qty min_sales_qty, location min_location from item_data where (item, week, sales_qty ) in ( select item, week, min(sales_qty) from item_data group by item, week ) ) miwhere mi.item = ma.item andmi.week = ma.weekorder by item, week

Finds maximum sales qty and location

Finds minimum sales qty and location


First_value & last_valuecall count cpu elapsed disk query current rows------- ------ -------- ---------- ---------- ---------- ---------- ----------Parse 1 0.05 0.03 0 42 0 0Execute 1 0.00 0.00 0 0 0 0Fetch 6 0.01 0.01 0 20 0 61------- ------ -------- ---------- ---------- ---------- ---------- ----------total 8 0.06 0.05 0 62 0 61


Rows Row Source Operation------- --------------------------------------------------- 61 SORT ORDER BY (cr=20 pr=0 pw=0 time=15415 us) 61 HASH JOIN RIGHT SEMI (cr=20 pr=0 pw=0 time=14170 us) 60 VIEW (cr=5 pr=0 pw=0 time=1178 us) 60 SORT GROUP BY (cr=5 pr=0 pw=0 time=1174 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=39 us) 610 HASH JOIN (cr=15 pr=0 pw=0 time=9695 us) 61 HASH JOIN RIGHT SEMI (cr=10 pr=0 pw=0 time=5926 us) 60 VIEW (cr=5 pr=0 pw=0 time=1306 us) 60 SORT GROUP BY (cr=5 pr=0 pw=0 time=1186 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=35 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=31 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=27 us)


First_value & last_value (tuned)

select distinct item, week, first_value (location) over (partition by item, week order by sales_qty desc rows between unbounded preceding and unbounded following )

top_seller, first_value (sales_qty) over (partition by item, week order by sales_qty desc rows between unbounded preceding and unbounded following )

top_sales_qty, last_value (location) over (partition by item, week order by sales_qty desc rows between unbounded preceding and unbounded following )

worst_seller, last_value (sales_qty) over (partition by item, week order by sales_qty desc rows between unbounded preceding and unbounded following )

worst_sales_qtyfrom item_dataorder by item, week

Script : anfn_first_value.sql


First_value & last_value (tuned)first_value (location) over (

partition by item, week order by sales_qty desc rows between unbounded preceding and unbounded following )

ITEM WEEK LOCATION SALES_QTY

---------- ---------- ---------- ----------

Shirt 1 1 11

Shirt 2 1 476

..

Shirt 1 2 580

Shirt 2 2 762

...

Shirt 1 5 391

Shirt 2 5 481

..

Shirt 4 1 876

Shirt 4 2 767

ITEM LOCATION WEEK SALES_QTY

---------- ---------- ---------- ----------

Shirt 4 1 876

Shirt 3 1 604

Shirt 2 1 580

Shirt 10 1 579

Shirt 7 1 529...Shirt 1 1 11

Sort by item, week, sales_qty desc

ITEM WEEK TOP_SELLER TOP_SALES_QTY WORST_SELLER WORST_SALES_QTY---------- ---------- ---------- ------------- ------------ ---------------Shirt 1 4 876 1 11Shirt 2 7 969 3 170Shirt 3 10 978 7 32Shirt 4 9 994 5 58

first_value/last_value

First_value and last_valueRetrieves column values from First and last row, in that ordered set


First_value & last_value (tuned)


------- ------ -------- ---------- ---------- ---------- ---------- ----------

Parse 1 0.00 0.00 0 1 0 0

Execute 1 0.00 0.00 0 0 0 0

Fetch 5 0.01 0.00 0 5 0 60

------- ------ -------- ---------- ---------- ---------- ---------- ----------

total 7 0.01 0.00 0 6 0 60



Parsing user id: 30


------- ---------------------------------------------------

60 SORT UNIQUE (cr=5 pr=0 pw=0 time=4478 us)

600 WINDOW SORT (cr=5 pr=0 pw=0 time=3580 us)


6 logical reads, compared to 20 logical reads.Item_data accessed just once, compared to four times.


Dense_rankselect item, week, location, sales_qty , sales_rnk, rcpt_rankfrom (select item, week, location, sales_qty , (select count(*) from item_data i1 where i1.item = iout.item and i1.week = iout.week and i1.sales_qty >= iout.sales_qty ) sales_rnk, (select count(*) from item_data i1 where i1.item = iout.item and i1.week = iout.week and i1.rcpt_qty >= iout.rcpt_qty ) rcpt_rank from item_data iout)where sales_rnk <3order by item, week, sales_qty descScript : anfn_dense_rank.sql

Tune SQL to print top two selling locations and receipt_qty

Finding sales_rnk


Dense_rank



Rows Row Source Operation------- --------------------------------------------------- 120 SORT ORDER BY (cr=2025 pr=2 pw=0 time=78353 us) 120 VIEW (cr=2025 pr=2 pw=0 time=41463 us) 120 FILTER (cr=1687 pr=2 pw=0 time=27869 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=639 us) 600 SORT AGGREGATE (cr=1682 pr=2 pw=0 time=62805 us) 3302 TABLE ACCESS BY INDEX ROWID ITEM_DATA (cr=1682 pr=2 pw=0 time=61347 us) 6000 INDEX RANGE SCAN ITEM_I1 (cr=802 pr=2 pw=0 time=50596 us)(object id 47239)


Dense_rankThis could be accomplished by dense_rank function:

select * from ( select item,week, location , sales_qty, dense_rank () over ( partition by item, week order by sales_qty desc ) sls_rnk, dense_rank () over ( partition by item, week order by rcpt_qty desc ) rcpt_rnk from item_data ) where sls_rnk <3

ITEM WEEK LOCATION SALES_QTY SLS_RNK RCPT_RNK---------- ---------- ---------- ---------- ---------- ----------Pants 1 5 911 1 9Pants 1 10 764 2 3Pants 2 3 838 1 8Pants 2 10 770 2 3Pants 3 3 921 1 4Pants 3 4 871 2 8…

Dense_rank computes rank of rowin an ordered set of rows.

Groups rows with same item ,week value in to a single group. Rows in the group sorted by sales_qty desc, calculating rank


Dense_rank



Rows Row Source Operation------- --------------------------------------------------- 120 VIEW (cr=5 pr=0 pw=0 time=5431 us) 600 WINDOW SORT PUSHED RANK (cr=5 pr=0 pw=0 time=6027 us) 600 WINDOW SORT (cr=5 pr=0 pw=0 time=6573 us) 600 TABLE ACCESS FULL ITEM_DATA (cr=5 pr=0 pw=0 time=35 us)



Predicate pushing & partition by columnsexplain plan forselect * from (Select item, location, week, sales_qty, rcpt_qty, sum (sales_qty) over ( partition by item, location order by week rows between unbounded preceding and current row ) running_sales_total from item_data ) where item='Pants' and week=1

---------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |---------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 20800 | 1462K| 46 (11)| 00:00:01 ||* 1 | VIEW | | 20800 | 1462K| 46 (11)| 00:00:01 || 2 | WINDOW SORT | | 20800 | 385K| 46 (11)| 00:00:01 ||* 3 | TABLE ACCESS FULL| ITEM_DATA | 20800 | 385K| 43 (5)| 00:00:01 |---------------------------------------------------------------------------------

1 - filter("WEEK"=1)

3 - filter("ITEM"='Pants')

Script: anfn_pred_push.sql

Partition columns are item & location

Predicate columns are item & week

Predicates for columns specified in partition clause are pushed.


Predicate pushing & partition by columns

explain plan forselect * from (Select item, location, week, sales_qty, rcpt_qty, sum (sales_qty) over ( partition by item, week order by location rows between unbounded preceding and current row ) running_sales_total from item_data ) where item='Pants' and week=1

-------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |-------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 100 | 7200 | 3 (34)| 00:00:01 || 1 | VIEW | | 100 | 7200 | 3 (34)| 00:00:01 || 2 | WINDOW SORT | | 100 | 1900 | 3 (34)| 00:00:01 || 3 | TABLE ACCESS BY INDEX ROWID| ITEM_DATA | 100 | 1900 | 2 (0)| 00:00:01 ||* 4 | INDEX RANGE SCAN | ITEM_I1 | 100 | | 1 (0)| 00:00:01 |

4 - access("ITEM"='Pants' AND "WEEK"=1)

Partition columns are item & week

Predicate columns are item & week

Both predicates are pushed.


Predicate pushing & functionscreate index item_i1 on item_data ( upper(item), week );explain plan forselect * from (Select item, location, week, sales_qty, rcpt_qty, sum (sales_qty) over ( partition by upper(item), week order by location rows between unbounded preceding and current row ) running_sales_total from item_data ) where upper(item)='PANTS' and week=1

---------------------------------------------------------------------------------

| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |

---------------------------------------------------------------------------------

| 0 | SELECT STATEMENT | | 344 | 24768 | 44 (7)| 00:00:01 |

|* 1 | VIEW | | 344 | 24768 | 44 (7)| 00:00:01 |

| 2 | WINDOW SORT | | 344 | 20296 | 44 (7)| 00:00:01 |

|* 3 | TABLE ACCESS FULL| ITEM_DATA | 344 | 20296 | 43 (5)| 00:00:01 |

---------------------------------------------------------------------------------

Predicate Information (identified by operation id):

---------------------------------------------------

1 - filter(UPPER("ITEM")='Pants')

3 - filter("WEEK"=1)

Applying a function to partitioning column, disallows predicate pushing for that column.


Predicate pushing & transitivityexplain plan for Select idt.*, idt2.item from (select item, location, week, sales_qty, rcpt_qty, sum (sales_qty) over ( partition by item, week order by location

rows between unbounded preceding and current row ) running_sales_total

from item_data) idt, item_data idt2where idt2.item ='Pants' and idt2.week=1 and idt2.item = idt.item and idt2.week = idt.week and idt2.location = idt.location--------------------------------------------------------------------------------------------| Id | Operation | Name | Rows | Bytes | Cost (%CPU)| Time |--------------------------------------------------------------------------------------------| 0 | SELECT STATEMENT | | 156 | 16380 | 6 (34)| 00:00:01 ||* 1 | HASH JOIN | | 156 | 16380 | 6 (34)| 00:00:01 || 2 | TABLE ACCESS BY INDEX ROWID | ITEM_DATA | 100 | 3300 | 2 (0)| 00:00:01 ||* 3 | INDEX RANGE SCAN | ITEM_I1 | 100 | | 1 (0)| 00:00:01 || 4 | VIEW | | 100 | 7200 | 3 (34)| 00:00:01 || 5 | WINDOW SORT | | 100 | 5900 | 3 (34)| 00:00:01 || 6 | TABLE ACCESS BY INDEX ROWID| ITEM_DATA | 100 | 5900 | 2 (0)| 00:00:01 ||* 7 | INDEX RANGE SCAN | ITEM_I1 | 100 | | 1 (0)| 00:00:01 |--------------------------------------------------------------------------------------------Predicate Information (identified by operation id):--------------------------------------------------- 1 - access("IDT2"."LOCATION"="IDT"."LOCATION") 3 - access("IDT2"."ITEM"='Pants' AND "IDT2"."WEEK"=1) 7 - access("ITEM"='Pants' AND "WEEK"=1)

idt subquery use item, week for partition by clause

idt2 literal predicates can be applied to idt subquery.

Transitive predicates are pushed correctly.


Predicate pushing

Also beware of bug 3582445

Fixed in 9.2.0.6/10.1.0.4/10.2.0.1

“CBO misses some obvious cases where it can push predicates through query blocks (views) with window functions…”

Sub query factoring

RMOUG 2008 Riyaj Shamsudeen

Sub query factoring

Results of a subquery can be accessed as if it is another row source.

Even nesting of subqueries supported.

Improved performance as aggregation can be calculated once and reused.


Sub query factoring -old

select i1.item, i1.location, (i1.sum_sales_qty/i2.tot_sales_qty)*100 sales_percent

from ( select item, location, sum(sales_qty) sum_sales_qty, from item_data group by item,location ) i1, ( select item, sum(sales_qty) tot_sales_qty, sum(rcpt_qty) tot_rcpt_qty from item_data group by item ) i2where i1.item=i2.itemorder by 1,3 desc

Problem: To tune SQL for sales_percent

Calculates sum at item, location level

Calculates sum at item level

Script: sqf_example1.sql RMOUG 2008 Riyaj Shamsudeen

Sub query factoring



Rows Row Source Operation------- --------------------------------------------------- 3000 SORT ORDER BY (cr=1100 pr=0 pw=0 time=544421 us) 3000 HASH JOIN (cr=1100 pr=0 pw=0 time=531605 us) 3 VIEW (cr=550 pr=0 pw=0 time=189459 us) 3 SORT GROUP BY (cr=550 pr=0 pw=0 time=189453 us) 156000 TABLE ACCESS FULL ITEM_DATA (cr=550 pr=0 pw=0 time=44 us) 3000 VIEW (cr=550 pr=0 pw=0 time=337442 us) 3000 SORT GROUP BY (cr=550 pr=0 pw=0 time=334440 us) 156000 TABLE ACCESS FULL ITEM_DATA (cr=550 pr=0 pw=0 time=39 us)


Sub query factoring - tuned

with itm_loc_sum as ( select item, location, sum(sales_qty) sum_sales_qty, sum(rcpt_qty) sum_rcpt_qty from item_data group by item,location), itm_sum as ( select item, sum(sum_sales_qty) tot_sales_qty, sum(sum_rcpt_qty) tot_rcpt_qty from itm_loc_sum group by item)select i1.item, i1.location, (i1.sum_sales_qty/i2.tot_sales_qty)*100 sales_percentfrom itm_loc_sum i1, itm_sum i2where i1.item=i2.itemorder by 1,3 desc

Defining a row source itm_loc_sum

Defining a row source itm_sum



with itm_loc_sum as ( select item, location, sum(sales_qty)

sum_sales_qty, sum(rcpt_qty) sum_rcpt_qty from item_data group by item,location), itm_sum as ( select item, sum(sum_sales_qty) tot_sales_qty, sum(sum_rcpt_qty) tot_rcpt_qty from itm_loc_sum group by item)select i1.item, i1.location,

(i1.sum_sales_qty/i2.tot_sales_qty)*100 sales_percent

from itm_loc_sum i1, itm_sum i2where i1.item=i2.itemorder by 1,3 desc

Rows Row Source Operation------- --------------------------------------------------- 3000 TEMP TABLE TRANSFORMATION 1 LOAD AS SELECT 3000 SORT GROUP BY 156000 TABLE ACCESS FULL OBJ#(47918) 3000 SORT ORDER BY 3000 HASH JOIN 3 VIEW 3 SORT GROUP BY 3000 VIEW 3000 TABLE ACCESS FULL SYS_TEMP_4254950932 3000 VIEW 3000 TABLE ACCESS FULL SYS_TEMP_4254950932




------- ------ -------- ---------- ---------- ---------- ---------- ----------

Parse 1 0.02 0.00 0 2 1 0

Execute 1 0.35 0.34 0 550 17 0

Fetch 201 0.04 0.03 11 28 1 3000

------- ------ -------- ---------- ---------- ---------- ---------- ----------

total 203 0.41 0.38 11 580 19 3000



Subquery factoring with dblink

Works even with database links.

You can materialize the remote table locally and then join

If the tables are remote and accessed numerous times, avoids extra network overhead.

Script: sqf_example2.sqlRMOUG 2008

Riyaj Shamsudeen

Subquery factoring with dblink

------------------------------------------------------------------| Id | Operation | Name |------------------------------------------------------------------| 0 | SELECT STATEMENT | || 1 | TEMP TABLE TRANSFORMATION | || 2 | LOAD AS SELECT | || 3 | SORT GROUP BY | || 4 | REMOTE | || 5 | SORT ORDER BY | ||* 6 | HASH JOIN | || 7 | VIEW | || 8 | SORT GROUP BY | || 9 | VIEW | || 10 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6617_816F0B7F || 11 | VIEW | || 12 | TABLE ACCESS FULL | SYS_TEMP_0FD9D6617_816F0B7F |------------------------------------------------------------------

Remote SQL Information (identified by operation id):----------------------------------------------------

4 - SELECT "ITEM","LOCATION","SALES_QTY","RCPT_QTY" FROM "ITEM_DATA" "ITEM_DATA" (accessing 'LOOPBACK' )

with itm_loc_sum as ( select item, location, sum(sales_qty) sum_sales_qty, from item_data@loopback group by item,location), itm_sum as ( select item, sum(sum_sales_qty) tot_sales_qty, sum(sum_rcpt_qty) tot_rcpt_qty from itm_loc_sum group by item)select i1.item, i1.location, (i1.sum_sales_qty/i2.tot_sales_qty)*100 sales_percentfrom itm_loc_sum i1, itm_sum i2where i1.item=i2.itemorder by 1,3 desc


Subquery factoring

explain plan forwith itm_loc_sum as ( select item, location, sum(sales_qty)

sum_sales_qty, sum(rcpt_qty) sum_rcpt_qty from item_data group by item,location), itm_sum as (select item, sum(sales_qty) tot_sales_qty, sum(rcpt_qty) tot_rcpt_qty from item_data group by item)select i1.item, i1.location,

(i1.sum_sales_qty/i2.tot_sales_qty)*100 sales_percent

from itm_loc_sum i1, itm_sum i2where i1.item=i2.itemorder by 1,3 desc

-------------------------------------------------------------

| Id | Operation | Name | Rows |---------------------------------------------------| 0 | SELECT STATEMENT | | 129K|| 1 | SORT ORDER BY | | 129K||* 2 | HASH JOIN | | 129K|| 3 | VIEW | | 129K|| 4 | SORT GROUP BY | | 129K|| 5 | TABLE ACCESS FULL| ITEM_DATA | 129K|| 6 | VIEW | | 129K|| 7 | SORT GROUP BY | | 129K|| 8 | TABLE ACCESS FULL| ITEM_DATA | 129K|---------------------------------------------------

Note temp table transformation step is missing.

My guess is that only if there are more than one reference to a subquery, it is materialized and few of my test case proves that. (except read only databases).


Subquery factoring & CBO

Explain plan forwith itm_loc_sum as ( select item, location, sum(sales_qty) sum_sales_qty, from item_data group by item,location), itm_sum as ( select item, sum(sum_sales_qty) tot_sales_qty, sum(sum_rcpt_qty) tot_rcpt_qty from itm_loc_sum group by item)select i1.item, i1.location, (i1.sum_sales_qty/i2.tot_sales_qty)*100 sales_percentfrom itm_loc_sum i1, itm_sum i2where i1.item=i2.itemorder by 1,3 desc

Parse step is creating a temporary table, if the subquery is to be materialized.

CREATE GLOBAL TEMPORARY TABLE "SYS"."SYS_TEMP_0FD9D6622_816F0B7F" ("C0" VARCHAR2(10),"C1" NUMBER,"C2" NUMBER,"C3" NUMBER ) IN_MEMORY_METADATA CURSOR_SPECIFIC_SEGMENT STORAGE (OBJNO 4254950946 )END OF STMT

SINGLE TABLE ACCESS PATH TABLE: ITEM_DATA Alias: ITEM_DATA Original Card: 156000 Rounded: 156000 Computed: 156000.00 Non Adjusted: 156000.00 Access Path: table-scan Resc: 104 Resp: 104 BEST_CST: 103.56 PATH: 2 Degree: 1Grouping column cardinality [ ITEM] 3GROUP BY cardinality: 2122.00, TABLE cardinality: 156000.00...SINGLE TABLE ACCESS PATH TABLE: SYS_TEMP_0FD9D6626_816F0B7F Alias: T1 Original Card: 2122 Rounded: 2122 Computed: 2122.00 Non Adjusted: 2122.00 Access Path: table-scan Resc: 3 Resp: 3 BEST_CST: 3.05 PATH: 2 Degree: 1

Then statistics for that temporary table are used in CBO calculations


Subquery factoring & RO databases

In read only databases, subquery is not materialized..


------- ---------------------------------------------------

2 HASH JOIN (cr=10 pr=0 pw=0 time=3483 us)

2 NESTED LOOPS (cr=6 pr=0 pw=0 time=1005 us)

1 VIEW (cr=3 pr=0 pw=0 time=552 us)

1 SORT AGGREGATE (cr=3 pr=0 pw=0 time=543 us)

14 TABLE ACCESS FULL OBJ#(20979) (cr=3 pr=0 pw=0 time=365 us)

2 VIEW (cr=3 pr=0 pw=0 time=451 us)

3 SORT GROUP BY (cr=3 pr=0 pw=0 time=445 us)





All together.with idata as ( select item, location, week, max(sales_qty ) over ( partition by item, location,week ) max_sales_qty, sales_so_far,sales_qty, rcpt_qty, max_inventory from ( select item, location, week, inventory, sales_so_far,sales_qty,

rcpt_qty, max_inventory from item_Data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty, 0 sales_so_far, 0 max_inventory) rules ( sales_so_far [location, week] = sales_qty [cv(location), cv(week)] + nvl(sales_so_far [cv(location), cv(week)-1],0) + inventory [cv(location), cv(week)], inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + rcpt_qty [cv(location), cv(week) ], max_inventory [location, week] = max(inventory) [cv(location), week between cv(week)-5 and cv(week )+5 ]

) order by item, location, week ) )select * from idata

Throws ORA-00600: internal error code, arguments: [qkaMarkQkn2], [], [], [], [],

[], in 10.1.0.4


All together..with idata as ( select item, location, week, inventory, sales_so_far,sales_qty,

rcpt_qty, max_inventory from item_Data model return updated rows partition by (item) dimension by (location, week) measures ( 0 inventory , sales_qty, rcpt_qty, 0 sales_so_far, 0 max_inventory) rules ( sales_so_far [location, week] = sales_qty [cv(location), cv(week)] + nvl(sales_so_far [cv(location), cv(week)-1],0) + inventory [cv(location), cv(week)], inventory [location, week] = nvl(inventory [cv(location), cv(week)-1 ] ,0) - sales_qty [cv(location), cv(week) ] + rcpt_qty [cv(location), cv(week) ], max_inventory [location, week] = max(inventory) [cv(location), week between cv(week)-5 and cv(week )+5 ]

) order by item, location, week ) select item, location, week, max(sales_qty ) over ( partition by item, location,week ) max_sales_qty, sales_so_far,sales_qty, rcpt_qty, max_inventory from idata

Slightly rewritten query works fine Though.


Thank you for listening.

QUESTIONS ?

Performance tuning using SQL new features

Documents

Transcript of Performance tuning using SQL new features