CPS216: Advanced Database Systems Notes 08:Query Optimization (Plan Space, Query Rewrites)

38
CPS216: Advanced Database Systems Notes 08:Query Optimization (Plan Space, Query Rewrites) Shivnath Babu

description

CPS216: Advanced Database Systems Notes 08:Query Optimization (Plan Space, Query Rewrites). Shivnath Babu. SQL query. parse. parse tree. Query rewriting. statistics. logical query plan. Physical plan generation. physical query plan. execute. result. - PowerPoint PPT Presentation

Transcript of CPS216: Advanced Database Systems Notes 08:Query Optimization (Plan Space, Query Rewrites)

CPS216: Advanced Database Systems

Notes 08:Query Optimization (Plan Space, Query Rewrites)

Shivnath Babu

parse

Query rewriting

Physical plan generation

execute

result

SQL query

parse tree

logical query planstatistics

physical query plan

Query Processing - In class order

2; 16.1

3; 16.2,16.3

1; 13, 15

4; 16.4—16.7

Roadmap• Query optimization: problem definition• Space of physical plans

– Counting exercise

• Approaches for query optimization– Heuristic-based (Oracle calls them rule-based)– Cost-based– Hybrid

• Heuristics for query optimization (Query rewrites)

Query Optimization Problem

Pick the best plan from the space of physical plans

The Space of Physical Plans is Very Large

• Algebraic equivalences

• Different physical operators for the same logical operator– nested loop join, hash join, sort-merge join– index-scan, table-scan

• Different plumbing details - pipelining vs. materialization

• Different tree shapes

A Plan Counting Exercise

• Work on blackboard

Approaches for Query Optimization

• Approach 1: Pick some plan– Bad plans can be really bad!

• Approach 2: Heuristics– Ex: maximize use of indexes (MySQL)

• Approach 3: Cost-based– “Enumerate”, find cost, pick best– Be smart about how you iterate through the

plans. Why?

• Hybrid

Query Optimization in Practice

• Hybrid

• Use heuristics, called query rewrite rules– eliminate many of the really bad plans– avoid eliminating good plans

• Cost-based– Be smart about how you iterate through plans– Ex: dynamic programming, genetic search

parse

Query rewriting

Physical plan generation

execute

result

SQL query

parse tree

logical query planstatistics

physical query plan

Initial logical plan

“Best” logical plan

Logical plan

Rewrite rules

Why do we need Query Rewriting?

• Pruning the HUGE space of physical plans– Eliminating redundant conditions/operators– Rules that will improve performance with very

high probability

• Preprocessing– Getting queries into a form that we know how

to handle best

Reduces optimization time drastically without noticeably affecting quality

Query Rewrite Rules

• Transform one logical plan into another– Do not use statistics

• Equivalences in relational algebra• Push-down predicates• Do projects early• Avoid cross-products if possible• Use left-deep trees• Use of constraints, e.g., uniqueness

Example Query

Select B,D

From R,S

Where R.A = “c” R.C=S.C

Example: Parse Tree<Query>

<SFW>

SELECT <SelList> FROM <FromList> WHERE <Cond>

<Attribute> <SelList> <RelName> <FromList> <Cond> AND <Cond>

B <Attribute> R <RelName>

S<Attr> <Op> <Const>

<Attr> <Op> <Attr>

R.A = “c”

R.C S.C=

D

Select B,DFrom R,SWhere R.A = “c” R.C=S.C

Along with Parsing …

• Semantic checks– Do the projected attributes exist in the

relations in the From clause?– Ambiguous attributes?– Type checking, ex: R.A > 17.5

• Expand views

Initial Logical Plan

Relational Algebra: B,D [ R.A=“c” R.C = S.C (RXS)]

Select B,DFrom R,SWhere R.A = “c” R.C=S.C

B,D

R.A = “c” Λ R.C = S.C

X

R S

Apply Rewrite Rule (1)

B,D [ R.C=S.C [R.A=“c”(R X S)]]

B,D

R.A = “c” Λ R.C = S.C

X

R S

B,D

R.A = “c”

X

R S

R.C = S.C

Apply Rewrite Rule (2)

B,D [ R.C=S.C [R.A=“c”(R)] X S]

B,D

R.A = “c”

X

R

S

R.C = S.C

B,D

R.A = “c”

X

R S

R.C = S.C

Apply Rewrite Rule (3)

B,D [[R.A=“c”(R)] S]

B,D

R.A = “c”

R

S

B,D

R.A = “c”

X

R

S

R.C = S.CNatural join

Equivalences in Relational Algebra

R S = S R Commutativity

(R S) T = R (S T) Associativity

Also holds for: Cross Products, Union, Intersection

R x S = S x R

(R x S) x T = R x (S x T)

R U S = S U R

R U (S U T) = (R U S) U T

Rules: Project

Let: X = set of attributes

Y = set of attributes

XY = X U Y

xy (R) = x [y (R)]

Let p = predicate with only R attribs

q = predicate with only S attribs

m = predicate with only R,S attribs

p (R S) =

q (R S) =

Rules: combined

[p (R)] S

R [q (S)]

Rules: combined (continued)

pq (R S) = [p (R)] [q (S)]

pqm (R S) =

m [(p R) (q S)]pvq (R S) =

[(p R) S] U [R (q S)]

p1p2 (R) p1 [p2 (R)]

p (R S) [p (R)] S

R S S R

x [p (R)] x {p [xz (R)]}

Which are “good” transformations?

Conventional wisdom: do projects early

Example: R(A,B,C,D,E) x={E} P: (A=3) (B=“cat”)

x {p (R)} vs. E {p{ABE(R)}}

But: What if we have A, B indexes?

B = “cat” A=3

Intersect pointers to get

pointers to matching tuples

Bottom line:

• No transformation is always good

• Some are usually good: – Push selections down– Avoid cross-products if possible– Subqueries Joins

More Query Rewrite Rules

• Transform one logical plan into another– Do not use statistics

• Equivalences in relational algebra• Push-down predicates• Do projects early• Avoid cross-products if possible• Use left-deep trees• Subqueries Joins• Use of constraints, e.g., uniqueness

Avoid Cross Products (if possible)

• Which join trees avoid cross-products?• If you can't avoid cross products, perform

them as late as possible

Select B,DFrom R,S,T,UWhere R.A = S.B R.C=T.C R.D = U.D

Use Left Deep Plans

• What are some left-deep, right-deep, and bushy plans for this query?

• Why is this heuristic useful?

– Reason #1: We maximize the possibility of using indexes

– Reason #2: Better for nested-loop join

• What about hash joins?

• Homework: Construct examples where (i) right-deep plan is best, (ii) where bushy is best

Select B,DFrom R,S,T,UWhere R.A = S.A R.A=T.A R.A = U.A

More Query Rewrite Rules

• Transform one logical plan into another– Do not use statistics

• Equivalences in relational algebra• Push-down predicates• Do projects early• Avoid cross-products if possible• Use left-deep trees• Subqueries Joins• Use of constraints, e.g., uniqueness

SQL Query with an Uncorrelated SubqueryFind the movies with stars born in 1960

MovieStar(name, address, gender, birthdate) StarsIn(title, year, starName)

SELECT titleFROM StarsInWHERE starName IN (

SELECT nameFROM MovieStarWHERE birthdate LIKE ‘%1960’

);

Parse Tree<Query>

<SFW>

SELECT <SelList> FROM <FromList> WHERE <Condition>

<Attribute> <RelName> <Tuple> IN <Query>

title StarsIn <Attribute> ( <Query> )

starName <SFW>

SELECT <SelList> FROM <FromList> WHERE <Condition>

<Attribute> <RelName> <Attribute> LIKE <Pattern>

name MovieStar birthDate ‘%1960’

Generating Relational Algebra

title

StarsIn <condition>

<tuple> IN name

<attribute> birthdate LIKE ‘%1960’

starName MovieStar

Two-argument selection

Rewrite Rule for Two-argument Selection with Conditions Involving IN

Lexp <condition>

<tuple> IN Rexp

Two-argument selection

Lexp

Rexp

δ

X

<condition>

Applying the Rewrite Rule

title

StarsIn <condition>

<tuple> IN name

<attribute> birthdate LIKE ‘%1960’

starName MovieStar

title

starName=name

StarsIn δ

birthdate LIKE ‘%1960’

MovieStar

name

Improving the Logical Query Plan

title

starName=name

StarsIn name

birthdate LIKE ‘%1960’

MovieStar

title

starName=name

StarsIn δ

birthdate LIKE ‘%1960’

MovieStar

name

SQL Query with an Correlated Subquery

MovieStar(name, address, gender, birthdate) StarsIn(title, year, starName)

SELECT titleFROM StarsInWHERE starName IN (

SELECT nameFROM MovieStarWHERE name LIKE ‘Tom%’ and year = birthdate + 30

);

parse

Query rewriting

Physical plan generation

execute

result

SQL query

parse tree

logical query planstatistics

physical query plan

Query Processing - In class order

2; 16.1

3; 16.2,16.3

1; 13, 15

4; 16.4—16.7