Post on 05-Jan-2016
description
1
VView iew MaintenanceMaintenance
Based on several papers in view maintenance, most notably:Based on several papers in view maintenance, most notably:
A.Gupta and I.S. Mumick. A.Gupta and I.S. Mumick.
Maintenance of Materialized Views: Problems, Techniques, and ApplicationMaintenance of Materialized Views: Problems, Techniques, and Application . . In Bulletin of Technical Committee on Data Engineering 1995In Bulletin of Technical Committee on Data Engineering 1995
2
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. Applications of materialized viewsApplications of materialized views
8.8. Open problemsOpen problems
3
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. Applications of incremental view maintenanceApplications of incremental view maintenance
8.8. Open problemsOpen problems
4
What is a view? What is a view?
A view is a derived relation defined in terms of A view is a derived relation defined in terms of base (stored) relations.base (stored) relations.
Example: Example: Flight is table of available direct flights. Flight is table of available direct flights. We need a view of flights with one intermediate stop.We need a view of flights with one intermediate stop.
1. Introduction to views
CREATE TABLE Flight ( from CHAR(20), to CHAR(20),);
CREATE VIEW Conn(src, dest) AS SELECT F1.from, F2.to FROM Flight F1, Flight F2 WHERE F1.to = F2.from;
5
Views are treated as base tables in regard of querying Views are treated as base tables in regard of querying
1. Introduction to views
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester PhiladelphiaNew York Las VegasPhiladelphia Seattle
From ToWorcester Boston
Boston New YorkBoston SeattleWorcester PhiladelphiaNew York Las VegasPhiladelphia
Seattle
Src Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
The base relationThe base relationFlightFlight
The viewThe viewConnConn
6
Motivation: Why views? Motivation: Why views?
Logical data independence Logical data independence If the conceptual schema changes, the changes can be “masked” If the conceptual schema changes, the changes can be “masked”
through the views in the external schemathrough the views in the external schema
Security Security Not everybody may see everythingNot everybody may see everything
““Relations” tailored to user’s needs Relations” tailored to user’s needs
And many more reasons and applications … discussed laterAnd many more reasons and applications … discussed later
1. Introduction to views
7
How will the following query be answered? How will the following query be answered?
Two options:Two options: Computing on demand Computing on demand
• query modification, composing of the user query and the view query query modification, composing of the user query and the view query
View materialization View materialization • The view Conn would be materialized, its content would be stored in The view Conn would be materialized, its content would be stored in
the databasethe database
View materialization vs. computing on demandView materialization vs. computing on demand
1. Introduction to views
SELECT * FROM Conn C WHERE C.src = “Worcester”SELECT * FROM Conn C WHERE C.src = “Worcester”
SELECT * FROM (SELECT F1.from, F2.to FROM Flight F1, Flight F2 WHERE F1.to = F2.from ) AS C
WHERE C.src = “Worcester”
8
How will the following query be answered? How will the following query be answered?
Two options:Two options: Computing on demand Computing on demand
• query modification, composing of the user query and the view query query modification, composing of the user query and the view query
View materialization View materialization • The view Conn would be materialized, its content would be stored in The view Conn would be materialized, its content would be stored in
the databasethe database
View materialization vs. computing on demandView materialization vs. computing on demand
1. Introduction to views
SELECT * FROM Conn C WHERE C.src = “Worcester”SELECT * FROM Conn C WHERE C.src = “Worcester”
SELECT * FROM (SELECT F1.from, F2.to FROM Flight F1, Flight F2 WHERE F1.to = F2.from ) AS C
WHERE C.src = “Worcester”
9
View materialization vs. computing on demandView materialization vs. computing on demand
Trade-offs? Trade-offs?
1. Introduction to views
Queries can be answered fasterQueries can be answered faster Indexes can be build over a materialized view to even more Indexes can be build over a materialized view to even more
speed up the processing of the queries defined over the viewspeed up the processing of the queries defined over the view
The view requires additional storage spaceThe view requires additional storage space The consistency of the view has to be maintainedThe consistency of the view has to be maintained
10
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. ApplicationsApplications
8.8. Open problemsOpen problems
11
Definition of the problem of materialized view Definition of the problem of materialized view maintenancemaintenance
What is materialized view maintenance ?What is materialized view maintenance ? When the base relations are modified, When the base relations are modified, the view (often) becomes inconsistent. the view (often) becomes inconsistent. Updating the view to make it consistent Updating the view to make it consistent is called view maintenance (refreshing).is called view maintenance (refreshing).
2. The problem of view maintenance2. The problem of view maintenance
12
View maintenance policies ( When ? )View maintenance policies ( When ? )
Immediate view maintenanceImmediate view maintenance The view is refreshed within the same transaction that updates The view is refreshed within the same transaction that updates
the underlying tablesthe underlying tables+ the view is always up to date+ the view is always up to date
- slows down the transaction - slows down the transaction
Deferred view maintenanceDeferred view maintenance LazyLazy – the view is refreshed when query over it has to be – the view is refreshed when query over it has to be
evaluatedevaluated- slows down the queries- slows down the queries
PeriodicPeriodic - the view is refreshed periodically, e.g., once a day - the view is refreshed periodically, e.g., once a day• Such views are called snapshotsSuch views are called snapshots
ForcedForced – the view is refreshed after a certain number of changes – the view is refreshed after a certain number of changes have been made to the underlying tableshave been made to the underlying tables
2. The problem of view maintenance2. The problem of view maintenance
13
Methods of view maintenance (How ?)Methods of view maintenance (How ?)
Recomputation Recomputation recompute to view from scratchrecompute to view from scratch
Incremental view maintenanceIncremental view maintenance compute the changes to the view in response to the changes to compute the changes to the view in response to the changes to
the base relationthe base relation add/delete some tuples in the existing materialized viewadd/delete some tuples in the existing materialized view
HeuristicsHeuristics: Incremental view maintenance is usually : Incremental view maintenance is usually cheaper then recomputationcheaper then recomputation
2. The problem of view maintenance2. The problem of view maintenance
14
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. ApplicationsApplications
8.8. Open problemsOpen problems
15
The idea behind incremental view maintenance The idea behind incremental view maintenance
Example: Flight is table of available direct flights. We need a Example: Flight is table of available direct flights. We need a view of flights with one intermediate stopview of flights with one intermediate stop
)2F1F( : Dest) Conn(Src,
)Flight,2F(
)Flight,1F(
string) : tostring, :(fromFlight
from.2Fto.1Fto.2F,from.1F
16
)2F1F( : Dest) Conn(Src,
)Flight,2F(
)Flight,1F(
string) : tostring, :(fromFlight
from.2Fto.1Fto.2F,from.1F
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Src Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
F1
F2 The view ConnThe view Conn
3. The idea behind view maintenance3. The idea behind view maintenance
The base relationThe base relation
17
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New YorkSrc Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
Seattle Las Vegas
Boston New York
Philadelphia New York
F1
F2 The view ConnThe view Conn
The base relationThe base relation
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New York
)2F1F( : Dest) Conn(Src,
)Flight,2F(
)Flight,1F(
string) : tostring, :(fromFlight
from.2Fto.1Fto.2F,from.1F
Inserted Inserted tupletuple
New tuples New tuples in the viewin the view
3. The idea behind view maintenance3. The idea behind view maintenance
18
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New YorkSrc Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
Seattle Las Vegas
Boston New York
Philadelphia New York
F1
F2 The view ConnThe view Conn
The base relationThe base relation
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New York
)2F1F( : Dest) Conn(Src,
)Flight,2F(
)Flight,1F(
string) : tostring, :(fromFlight
from.2Fto.1Fto.2F,from.1F
Inserted Inserted tupletuple
New tuples New tuples in the viewin the view
3. The idea behind view maintenance3. The idea behind view maintenance
19
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New YorkSrc Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
Seattle Las Vegas
Boston New York
Philadelphia New York
F1
F2 The view ConnThe view Conn
The base relationThe base relation
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New York
)2F1F( : Dest) Conn(Src,
)Flight,2F(
)Flight,1F(
string) : tostring, :(fromFlight
from.2Fto.1Fto.2F,from.1F
F1F1
F1F1
F1F1=F1+=F1+ F1 F1
3. The idea behind view maintenance3. The idea behind view maintenance
20
))2F1F()2F1F()2F1F((Conn
)2F1F(Conn
))2F2F()1F1F((ConnConn
)2F1F(Conn2F2F2F
1F1F1FConnConnConn
from.2Fto.1Ffrom.2Fto.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
The differentiation equation The differentiation equation
3. The idea behind view maintenance3. The idea behind view maintenance
21
))2F1F()2F1F()2F1F((Conn
)2F1F(Conn))2F2F()1F1F((ConnConn
from.2Fto.1Ffrom.2Fto.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New YorkSrc Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
Seattle Las Vegas
Boston New York
Philadelphia New York
F1
F2 The view ConnThe view Conn
The base relationThe base relation
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New York
3. The idea behind view maintenance3. The idea behind view maintenance
22
))2F1F()2F1F()2F1F((Conn
)2F1F(Conn))2F2F()1F1F((ConnConn
from.2Fto.1Ffrom.2Fto.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New YorkSrc Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
Seattle Las Vegas
Boston New York
Philadelphia New York
F1
F2 The view ConnThe view Conn
The base relationThe base relation
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New York
Join
3. The idea behind view maintenance3. The idea behind view maintenance
23
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New YorkSrc Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
Seattle Las Vegas
Boston New York
Philadelphia New York
F1
F2 The view ConnThe view Conn
The base relationThe base relation
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New York
Join
))2F1F()2F1F()2F1F((Conn
)2F1F(Conn))2F2F()1F1F((ConnConn
from.2Fto.1Ffrom.2Fto.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
3. The idea behind view maintenance3. The idea behind view maintenance
24
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New YorkSrc Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
Seattle Las Vegas
Boston New York
Philadelphia New York
F1
F2 The view ConnThe view Conn
The base relationThe base relation
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New York
Join=
))2F1F()2F1F()2F1F((Conn
)2F1F(Conn))2F2F()1F1F((ConnConn
from.2Fto.1Ffrom.2Fto.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
3. The idea behind view maintenance3. The idea behind view maintenance
25
))2F1F()2F1F()2F1F((Conn
)2F1F(Conn))2F2F()1F1F((ConnConn
from.2Fto.1Ffrom.2Fto.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
from.2Fto.1Fto.2F,from.1F
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New YorkSrc Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
Seattle Las Vegas
Boston New York
Philadelphia New York
F1
F2 The view ConnThe view Conn
The base relationThe base relation
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Seattle New York
Savings!
3. The idea behind view maintenance3. The idea behind view maintenance
26
Insertions Insertions The observed example was for insertions into the base relationThe observed example was for insertions into the base relation
DeletionsDeletions If tuples are deleted from a base relation, the tuples that need to If tuples are deleted from a base relation, the tuples that need to
be deleted from the view are computed similarly using delta’sbe deleted from the view are computed similarly using delta’s
Updates Updates May be treated separately or may be modeled as deletions May be treated separately or may be modeled as deletions
followed by insertsfollowed by inserts
Multiple relations involvedMultiple relations involved The deltas are similarly computedThe deltas are similarly computed
3. The idea behind view maintenance3. The idea behind view maintenance
27
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. ApplicationsApplications
8.8. Open problemsOpen problems
28
DimensionsDimensions
Amount of Information
Type of Modification
Expressiveness of View
Definition Language
Insertions
Other Views
BaseRelations
IntegrityConstraints
MaterializedView
.
.
.
DeletionsUpdates Sets of each
Group UpdatesChange view definition
RecursionDifference
Outer-JoinsChronicle Algebra
Conjunctive
Queries
DuplicatesArithmetic
AggregationSubqueries
Union
….
4. Dimensions4. Dimensions
+ Instance dimension
29
Information Dimension Information Dimension
Information dimension Information dimension Information available for view maintenance Information available for view maintenance (other than (other than view definition and the modificationview definition and the modification, , which which are always assumed availableare always assumed available))
E.g., E.g., • base relations,base relations,• the materialized view, the materialized view, • other views, other views, • integrity constraints…integrity constraints…
Don’t we have all these information?Don’t we have all these information?
• Sometimes we don’t, Sometimes we don’t, • Sometimes it is expensive to access them Sometimes it is expensive to access them (base relations when views are physically not close to data) (base relations when views are physically not close to data)
30
Dimensions – information & modificationDimensions – information & modification
Example:Example:
4. Dimensions4. Dimensions
Information
Modification
Materialized view Conn only
Plus, base relation Flight
Key info only:
(from, to) is key in Flight
Insert
Delete
Update
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) );
CREATE VIEW CheapF(src, dest) AS SELECT DISTINCT F.from, F.to FROM Flight F WHERE F.price < 400;
31
Dimensions – information & modificationDimensions – information & modification
Example:Example:
4. Dimensions4. Dimensions
Information
Modification
The materialized
view Conn only
The base relation Flight
only
Key info only:
(from, to) is key in Flight
Insert
Delete
Update
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) );
CREATE VIEW CheapF(src, dest) AS SELECT DISTINCT F.from, F.to FROM Flight F WHERE F.price < 400;
Can we maintain the view when tuples are inserted into base relation if only information available is the materialized view (its content)
Reminder: The view definition and the modification are always available
32
Dimensions – information & modificationDimensions – information & modification
Example:Example:
Information
Modification
The materialized
view Conn only
The base relation Flight
only
Key info only:
(from, to) is key in Flight
Insert Delete Update
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) );
4. Dimensions4. Dimensions
CREATE VIEW CheapF(src, dest) AS SELECT DISTINCT F.from, F.to FROM Flight F WHERE F.price < 400;
33
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. ApplicationsApplications
8.8. Open problemsOpen problems
34
View maintenance using full informationView maintenance using full information
‘ ‘Classical’ view maintenance algorithms assume : Classical’ view maintenance algorithms assume : Full Information: the base relations, the materialized view, keys,…Full Information: the base relations, the materialized view, keys,… All database and modification instancesAll database and modification instances Modification: inserts, deletes, updates ( as insert-delete )Modification: inserts, deletes, updates ( as insert-delete ) Language: Focus on efficient techniques for maintaining PSJ views Language: Focus on efficient techniques for maintaining PSJ views
expressed in subset of view definition language (SQL)expressed in subset of view definition language (SQL)
Classification along language dimension:Classification along language dimension: Nonrecursive viewsNonrecursive views
• The counting algorithmThe counting algorithm Outer-join viewsOuter-join views Recursive viewsRecursive views ……
35
The Counting Algorithm - MotivationThe Counting Algorithm - Motivation
5. View Main. using full information5. View Main. using full information
))2F1F()2F1F()2F1F((
Conn
from.2Fto.1F
from.2Fto.1F
from.2Fto.1F
to.2F,from.1F
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Src Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
F1
F2The view ConnThe view Conn
The base relationThe base relation
36
5. View Main. using full information5. View Main. using full information
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
Src Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
F1
F2The view ConnThe view Conn
The base relationThe base relation
Shall we delete theShall we delete the (Worcester, Seattle) (Worcester, Seattle) tuple from the view?tuple from the view?
The Counting Algorithm - MotivationThe Counting Algorithm - Motivation
37
5. View Main. using full information5. View Main. using full information
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
F1
F2
The base relationThe base relation
No, becauseNo, becauseit can still be derived fromit can still be derived from
the remaining tuplesthe remaining tuples
Src Dest
Worcester New York
Worcester Seattle
Boston Las Vegas
The view ConnThe view Conn
The Counting Algorithm - MotivationThe Counting Algorithm - Motivation
38
The Counting AlgorithmThe Counting Algorithm
We need to know We need to know
if there are more derivations of one tuple in the viewif there are more derivations of one tuple in the view
Main idea: Main idea: Keep count of number of derivations for each tuple in viewKeep count of number of derivations for each tuple in view A tuple is removed from view only if its count is zeroA tuple is removed from view only if its count is zero
5. View Main. using full information5. View Main. using full information
39
5. View Main. using full information5. View Main. using full information
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
F1
F2
The base relationThe base relation
Src Dest Count
Worcester New York 1
Worcester Seattle 2
Boston Las Vegas 1
The view ConnThe view Conn
The Counting AlgorithmThe Counting Algorithm
40
5. View Main. using full information5. View Main. using full information
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
From To
Worcester Boston
Boston New York
Boston Seattle
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
F1
F2
The base relationThe base relation
Src Dest Count
Worcester New York 1
Worcester Seattle 2 1
Boston Las Vegas 1
The view ConnThe view Conn
The Counting AlgorithmThe Counting Algorithm
41
5. View Main. using full information5. View Main. using full information
From To
Worcester Boston
Boston New York
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
From To
Worcester Boston
Boston New York
Worcester Philadelphia
New York Las Vegas
Philadelphia Seattle
F1
F2
The base relationThe base relation
Src Dest Count
Worcester New York 1
Worcester Seattle 1 0
Boston Las Vegas 1
The view ConnThe view Conn
The Counting AlgorithmThe Counting Algorithm
42
The Counting AlgorithmThe Counting Algorithm
What if we have aggregation? What if we have aggregation?
5. View Main. using full information5. View Main. using full information
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL );
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL );
CREATE VIEW CheapFlight(src, dest, minPrice) AS SELECT F.from, F.to, MIN (F.price) FROM Flight F GROUP BY F.from, F.to;
CREATE VIEW CheapFlight(src, dest, minPrice) AS SELECT F.from, F.to, MIN (F.price) FROM Flight F GROUP BY F.from, F.to;
43
5. View Main. using full information5. View Main. using full information
From To Price
Boston New York 200
Chicago Philadelphia 350
Boston New York 300
New York Las Vegas 330
Chicago Philadelphia 490
Boston New York 250
The base relationThe base relation
Src Dest MinPrice Count
Boston New York 200 1
Chicago Philadelphia 350 1
New York Las Vegas 330 1
The view CheapFlightThe view CheapFlight
The Counting Algorithm - AggregationThe Counting Algorithm - Aggregation
Inserted Inserted tupletuple
The inserted tuple does not affect the view
The inserted tuple does not affect the view
Potentially affected tuplePotentially affected tuple
44
5. View Main. using full information5. View Main. using full information
From To Price
Boston New York 200
Chicago Philadelphia 350
Boston New York 300
New York Las Vegas 330
Chicago Philadelphia 490
Boston New York 180
The base relationThe base relation
Src Dest MinPrice Count
Boston New York 200 180 1
Chicago Philadelphia 350 1
New York Las Vegas 330 1
The view CheapFlightThe view CheapFlight
The Counting Algorithm - AggregationThe Counting Algorithm - Aggregation
One tuple in the viewhas to be updated
One tuple in the viewhas to be updated
Inserted Inserted tupletuple
Potentially affected tuplePotentially affected tuple
45
5. View Main. using full information5. View Main. using full information
From To Price
Boston New York 200
Chicago Philadelphia 350
Boston New York 300
New York Las Vegas 330
Chicago Philadelphia 490
Boston New York 180
The base relationThe base relation
Src Dest MinPrice Count
Boston New York 180 200 1
Chicago Philadelphia 350 1
New York Las Vegas 330 1
The view CheapFlightThe view CheapFlight
The Counting Algorithm - AggregationThe Counting Algorithm - Aggregation
One tuple in the view has to be recomputed
One tuple in the view has to be recomputed
DeletedDeletedtupletuple
Potentially affected tuplePotentially affected tuple
46
The Counting Algorithm - AggregationThe Counting Algorithm - Aggregation
When a change to the base relation occurs:When a change to the base relation occurs:
Identifies the tuples that may be affectedIdentifies the tuples that may be affected
Whenever possible incrementally computes new values of Whenever possible incrementally computes new values of affected tuples by only looking at materialized view and affected tuples by only looking at materialized view and modification. modification.
Other aggregation functions that may be computed this way: Other aggregation functions that may be computed this way: COUNT, SUM, MIN, MAXCOUNT, SUM, MIN, MAX
Some other aggregation functions like AVERAGE and VARIANCE Some other aggregation functions like AVERAGE and VARIANCE can be decomposed into incrementally computable functionscan be decomposed into incrementally computable functions
5. View Main. using full information5. View Main. using full information
47
The Counting Algorithm – Multiple relations and views over viewsThe Counting Algorithm – Multiple relations and views over views
Handles views over multiple relations, handles views over views Handles views over multiple relations, handles views over views (by first updating the views lower in the hierarchy)(by first updating the views lower in the hierarchy)
5. View Main. using full information5. View Main. using full information
Base relation 1Base relation 1
Materialized view 1Materialized view 1
Base relation 2Base relation 2 Base relation 3Base relation 3
Materialized view 2Materialized view 2
II
I II I
48
The Counting Algorithm - SummaryThe Counting Algorithm - Summary
Keeps track of the number of derivation of each tuple – tuples with Keeps track of the number of derivation of each tuple – tuples with count zero are deleted from the viewcount zero are deleted from the view
Handles updates as difference of positive and negative countsHandles updates as difference of positive and negative counts
Handles views over multiple relations, handles views over viewsHandles views over multiple relations, handles views over views
Language limitations: SPJ views, UNION, negation, aggregationLanguage limitations: SPJ views, UNION, negation, aggregation
Works for both set and duplicate semanticsWorks for both set and duplicate semantics
5. View Main. using full information5. View Main. using full information
49
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. ApplicationsApplications
8.8. Open problemsOpen problems
50
View maintenance using partial informationView maintenance using partial information
Views Views maymay be maintainable using partial information be maintainable using partial information May depend on the modification: insert, delete or updateMay depend on the modification: insert, delete or update
Goals:Goals: Check whether the view can be maintained Check whether the view can be maintained How to maintain the viewHow to maintain the view
51
Using no Information: Query Independent of UpdateUsing no Information: Query Independent of Update
Some modifications to the base tables may be irrelevant to Some modifications to the base tables may be irrelevant to the view - leave it unchangedthe view - leave it unchanged
Determine if the modification is irrelevant using:Determine if the modification is irrelevant using: The view definitionThe view definition The modificationThe modification
Recognizing irrelevant modifications prevents unnecessary Recognizing irrelevant modifications prevents unnecessary delta-computationsdelta-computations
6. View Main. using partial information6. View Main. using partial information
52
Query Independent of Update - ExampleQuery Independent of Update - Example
6. View Main. using partial information6. View Main. using partial information
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) )
CREATE TABLE RCompany( name CHAR(20), rating INTEGER )
CREATE VIEW GoodFlights(src, dest) AS SELECT F.from, F.to FROM Flight F, RCompany CWHERE F.company = C.name
AND C.rating > 5 AND F.price < 400
Which of these modifications would be irrelevant? Which of these modifications would be irrelevant?
INSERT INTO RCompany VALUES (“Swissair”, 10)INSERT INTO RCompany VALUES (“Swissair”, 10)
INSERT INTO RCompany VALUES (“EnronAir”, 2)INSERT INTO RCompany VALUES (“EnronAir”, 2)
DELETE FROM Flight F WHERE F.from=“Boston” AND PRICE< 350DELETE FROM Flight F WHERE F.from=“Boston” AND PRICE< 350
53
Query Independent of Update - ExampleQuery Independent of Update - Example
6. View Main. using partial information6. View Main. using partial information
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) )
CREATE TABLE RCompany( name CHAR(20), rating INTEGER )
Which of these modifications would be irrelevant? Which of these modifications would be irrelevant?
INSERT INTO RCompany VALUES (“Swissair”, 10)INSERT INTO RCompany VALUES (“Swissair”, 10)
INSERT INTO RCompany VALUES (“EnronAir”, 2)INSERT INTO RCompany VALUES (“EnronAir”, 2)
DELETE FROM Flight F WHERE F.from=“Boston” AND PRICE< 350DELETE FROM Flight F WHERE F.from=“Boston” AND PRICE< 350
CREATE VIEW GoodFlights(src, dest) AS SELECT F.from, F.to FROM Flight F, RCompany CWHERE F.company = C.name
AND C.rating > 5 AND F.price < 400
54
Self-MaintenanceSelf-Maintenance
Self-maintainable views are views that can be maintained Self-maintainable views are views that can be maintained using only materialized view (self) and key constraints using only materialized view (self) and key constraints
A view may be self-maintainable in respect to some A view may be self-maintainable in respect to some modification types (insert, delete, update) modification types (insert, delete, update) but not in respect to all of thembut not in respect to all of them
6. View Main. using partial information6. View Main. using partial information
55
6. View Main. using partial information6. View Main. using partial information
CREATE VIEW GoodFlights(src, dest) AS SELECT F.from, F.to FROM Flight F, RCompany CWHERE F.company = C.name
AND C.rating > 5 AND F.price < 400
Self-MaintenanceSelf-Maintenance
Distinguished Attribute – Appears in the SELECT clause in Distinguished Attribute – Appears in the SELECT clause in the view definitionthe view definition
Exposed Attribute – Used in a predicate in the view definitionExposed Attribute – Used in a predicate in the view definition
DistinguishedDistinguishedExposedExposed
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20), duration REAL, PRIMARY KEY (from, to) )
CREATE TABLE RCompany( name CHAR(20), rating INTEGER PRIMARY KEY (name) )
56
6. View Main. using partial information6. View Main. using partial information
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20), duration REAL, PRIMARY KEY (from, to) )
CREATE TABLE RCompany( name CHAR(20), rating INTEGER PRIMARY KEY (name) )
CREATE VIEW GoodFlights(src, dest) AS SELECT F.from, F.to FROM Flight F, RCompany CWHERE F.company = C.name
AND C.rating > 5 AND F.price < 400
Self-Maintenance - insertSelf-Maintenance - insert
SPJ view that takes join of two or more distinct relations is SPJ view that takes join of two or more distinct relations is notnot self maintainable in respect to insertions self maintainable in respect to insertions
DistinguishedDistinguishedExposedExposed
57
6. View Main. using partial information6. View Main. using partial information
CREATE VIEW GoodFlights(src, dest) AS SELECT F.from, F.to FROM Flight F, RCompany CWHERE F.company = C.name
AND C.rating > 5 AND F.price < 400
Self-Maintenance - updateSelf-Maintenance - update
SPJ view is self-maintainable in respect to updates on non-SPJ view is self-maintainable in respect to updates on non-exposed attributes when the key attributes are distinguished. exposed attributes when the key attributes are distinguished.
DistinguishedDistinguishedExposedExposed
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20), duration REAL, PRIMARY KEY (from, to) )
CREATE TABLE RCompany( name CHAR(20), rating INTEGER PRIMARY KEY (name) )
58
6. View Main. using partial information6. View Main. using partial information
CREATE VIEW GoodFlights(src, dest) AS SELECT F.from, F.to FROM Flight F, RCompany CWHERE F.company = C.name
AND C.rating > 5 AND F.price < 400
Self-Maintenance - updateSelf-Maintenance - update
DistinguishedDistinguishedExposedExposed
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20), duration REAL, PRIMARY KEY (from, to) )
CREATE TABLE RCompany( name CHAR(20), rating INTEGER PRIMARY KEY (name) )
Update of Update of duration does duration does not affect the not affect the
viewview
60
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. ApplicationsApplications
8.8. Open problemsOpen problems
61
Applications of materialized viewsApplications of materialized views
Query speed-upQuery speed-up
Integrity constraint checkingIntegrity constraint checking
Query optimizationQuery optimization
Data warehousingData warehousing
Chronicle systems (~ continuous queries)Chronicle systems (~ continuous queries)
Mobile systemsMobile systems
Data visualization Data visualization
………………
62
Query speed-upQuery speed-up
Queries are answered faster if their answers are Queries are answered faster if their answers are precomputed (materialized)precomputed (materialized)
How does the keeping of precomputed query result How does the keeping of precomputed query result (materialized view) correlate to the: (materialized view) correlate to the: Frequency of the query ?Frequency of the query ? Frequency of the updates to the base relations?Frequency of the updates to the base relations?
7. Applications7. Applications
63
Integrity constraint checkingIntegrity constraint checking
Static integrity constraints can be modeled as Static integrity constraints can be modeled as materialized views that are required to be empty materialized views that are required to be empty
Example: Example:
7. Applications7. Applications
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) )
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT F.company
FROM Flight FGROUP BY F.companyHAVING COUNT(*) > 100 ) )
64
Integrity constraint checkingIntegrity constraint checking
Example: Example:
7. Applications7. Applications
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) )
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT F.company
FROM Flight FGROUP BY F.companyHAVING COUNT(*) > 100 ) )
CREATE VIEW Monopolies AS SELECT F.company FROM Flight F GROUP BY F.company HAVING COUNT(*) > 100
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT *
FROM Monopolies ) )
Can be Can be modeled asmodeled as
65
Integrity constraint checkingIntegrity constraint checking
Example: Example:
7. Applications7. Applications
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) )
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT F.company
FROM Flight FGROUP BY F.companyHAVING COUNT(*) > 100 ) )
CREATE VIEW Monopolies AS SELECT F.company FROM Flight F GROUP BY F.company HAVING COUNT(*) > 100
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT *
FROM Monopolies ) )
What happens What happens on updates on updates
to the base relation?to the base relation?
66
Integrity constraint checkingIntegrity constraint checking
Example: Example:
7. Applications7. Applications
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) )
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT F.company
FROM Flight FGROUP BY F.companyHAVING COUNT(*) > 100 ) )
CREATE VIEW Monopolies AS SELECT F.company FROM Flight F GROUP BY F.company HAVING COUNT(*) > 100
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT *
FROM Monopolies ) )
How can we do How can we do better?better?
67
Integrity constraint checkingIntegrity constraint checking
Example: Example:
7. Applications7. Applications
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) )
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT F.company
FROM Flight FGROUP BY F.companyHAVING COUNT(*) > 100 ) )
CREATE VIEW Monopolies AS SELECT F.company FROM Flight F GROUP BY F.company HAVING COUNT(*) > 100
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT *
FROM Monopolies ) )
Which additional view Which additional view shall we materialize? shall we materialize?
68
Integrity constraint checkingIntegrity constraint checking
Materialize an additional view Materialize an additional view for maintaining Monopolies:for maintaining Monopolies:
7. Applications7. Applications
CREATE TABLE Flight ( from CHAR(20), to CHAR(20), price REAL, company CHAR(20) )
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT F.company
FROM Flight FGROUP BY F.companyHAVING COUNT(*) > 100 ) )
CREATE ASSERTION NoMonopolyCHECK ( NOT EXISTS ( SELECT *
FROM Monopolies ) )
CREATE VIEW Helper (name, total) AS SELECT F.company, COUNT(*) FROM Flight F GROUP BY F.company
CREATE VIEW Monopolies (name) AS SELECT F.company FROM Flight F GROUP BY F.company HAVING COUNT(*) > 100
CREATE VIEW Monopolies(name) AS SELECT H.company FROM Helper H WHERE H.total > 100
69
Query optimizationQuery optimization
Materialized views can be used even for answering Materialized views can be used even for answering queries that do not explicitly contain the views in their queries that do not explicitly contain the views in their definition definition
Cashed results can be seen as temporarily materialized Cashed results can be seen as temporarily materialized views and can also by used for faster query views and can also by used for faster query optimizationoptimization
Problems:Problems: Recognizing which views may be utilized for processing the Recognizing which views may be utilized for processing the
queryquery Finding which views and/or relations should be used for Finding which views and/or relations should be used for
achieving lowest cost of the query evaluation achieving lowest cost of the query evaluation
7. Applications7. Applications
70
Query optimization – the Microsoft SQL Server approachQuery optimization – the Microsoft SQL Server approach
Approach:Approach: Generate all possible rewritings of the queryGenerate all possible rewritings of the query View matching: Determine subexpressions that may be View matching: Determine subexpressions that may be
computed from materialized views computed from materialized views Estimate their costsEstimate their costs Choose the one with the lowest costChoose the one with the lowest cost
Indexes the view definitions using a special index to Indexes the view definitions using a special index to speed up the view matchingspeed up the view matching
Language limitation: Select-Project-Join-GroupByLanguage limitation: Select-Project-Join-GroupBy
7. Applications7. Applications
71
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. ApplicationsApplications
8.8. Open problemsOpen problems
72
Open problemsOpen problems
7. Open problems7. Open problems
Amount of Information
Type of Modification
Expressiveness of View
Definition Language
Insertions
Other Views
BaseRelations
IntegrityConstraints
MaterializedView
.
.
.
DeletionsUpdates Sets of each
Group UpdatesChange view definition
RecursionDifference
Outer-JoinsChronicle Algebra
Conjunctive
Queries
DuplicatesArithmetic
AggregationSubqueries
Union
….
+ Instance dimension
73
Open problemsOpen problems
7. Open problems7. Open problems
Great portion of the problem space is still unsolved or Great portion of the problem space is still unsolved or may be solved more efficiently , so many open may be solved more efficiently , so many open questions remainquestions remain
Other issues:Other issues: When to perform the maintenance?When to perform the maintenance? How to efficiently select additional views to be maintained?How to efficiently select additional views to be maintained? …………....
74
OutlineOutline
1.1. Introduction to viewsIntroduction to views
2.2. The problem of materialized view maintenanceThe problem of materialized view maintenance
3.3. The idea behind incremental view maintenanceThe idea behind incremental view maintenance
4.4. Dimensions – the problem spaceDimensions – the problem space
5.5. View maintenance using full informationView maintenance using full information The counting algorithmThe counting algorithm
6.6. View maintenance using partial informationView maintenance using partial information Self-maintenanceSelf-maintenance
7.7. ApplicationsApplications
8.8. Open problemsOpen problems
75
ReferencesReferences A.Gupta, I.S.Mumick. A.Gupta, I.S.Mumick. Maintenance of Materialized Views: Problems, Techniques, Maintenance of Materialized Views: Problems, Techniques,
and Applicationand Application. In Bulletin of the Technical Committee on Data engineering 1995. In Bulletin of the Technical Committee on Data engineering 1995 Most of this presentationMost of this presentation
R.Ramakrishnan, J.Gehrke. R.Ramakrishnan, J.Gehrke. Database Management SystemsDatabase Management Systems, McGraw-Hill 2000, McGraw-Hill 2000 Introduction to viewsIntroduction to views
A.Gupta, I.S. Mumick, V.S. Subrahmanian. A.Gupta, I.S. Mumick, V.S. Subrahmanian. Maintaining Views IncrementallyMaintaining Views Incrementally. In . In SIGMOD 1995SIGMOD 1995 The counting algorithmThe counting algorithm
J.A. Blakeley, P.Larson, and F.Tompa. J.A. Blakeley, P.Larson, and F.Tompa. Efficiently Updating Materialized ViewsEfficiently Updating Materialized Views. In . In SIGMOD 1986SIGMOD 1986 Query independent of updateQuery independent of update
K.Ross, D.Srivastava, S.Sudarshan. K.Ross, D.Srivastava, S.Sudarshan. Materialized View Maintenance and Integrity Materialized View Maintenance and Integrity Constraint Checking: Trading Space for TimeConstraint Checking: Trading Space for Time. In SIGMOD 96.. In SIGMOD 96. Applications: Integrity constraint checkingApplications: Integrity constraint checking
J Goldstein, P. Larson. J Goldstein, P. Larson. Optimizing Queries Using Materialized Views: A practical Optimizing Queries Using Materialized Views: A practical Scalable SolutionScalable Solution. In SIGMOD 2001. In SIGMOD 2001 Applications: Query optimization – the Microsoft SQL Server approachApplications: Query optimization – the Microsoft SQL Server approach