SMILE: A Data Sharing Platform for Mobile Apps in the Cloud
-
Upload
kylie-bird -
Category
Documents
-
view
30 -
download
2
description
Transcript of SMILE: A Data Sharing Platform for Mobile Apps in the Cloud
![Page 1: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/1.jpg)
SMILE: A Data Sharing Platform for Mobile Apps in the Cloud
Jagan SankaranaryananHakan Hacıgümüs NEC Labs America
Haopeng ZhangUMass, Amherst
Mohamed SarwatUMN
![Page 2: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/2.jpg)
Motivation For Sharing in Cloud• Mobile apps run their databases in the cloud
– Often small databases– Often hosted in the same cloud infrastructure– Often need “fresh” data from other apps
• e.g., Calendar app wants the itinerary from airline booking app
• Need a declarative way for apps to share data
Database As a ServiceApp 1 DB App 2 DB App n DBM
ultit
enan
t D
atab
ase
Sharing MiddLe warE (SMILE)
![Page 3: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/3.jpg)
Declarative Sharing
CloudDB
D3
D1
D2
Sharing (S1):
D1 D2 D3Datasets
Transform
Transformation: (SPJ)
Sharing (S2): …
Sharing (Sn): …
Staleness SLA
![Page 4: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/4.jpg)
Three ways of Enabling Sharing
AppAliceData
AppBobData
SQL
DirectSharing
AppAliceData
AppBobData
Web
Ser
vice
API
Sharingvia API
Sharing using a Materialized Shared Space (i.e., view)
SMIL
E AppAliceData
AppBobData
SQL
Mat
eria
lized
Shar
ed S
pace
SQL
Service provider’s cost in keeping shared space consistent
What requirements materialization satisfies?
![Page 5: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/5.jpg)
Sharing ExampleSimple Sharing Scenario
ps(
SP ✖
UP)
SP
SP = Stock Price
UP
UP = User Portfolio
SP ✖ UP
Sharing (S1): 1. Sources: SP, UP2. Transformation: ps(SP ✖ UP)3. Staleness: <= 5 Seconds
< 5
seco
nds
![Page 6: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/6.jpg)
Sharing Example (Contd.)
SP UPSP
COPY
JOIN
SP ✖ UP
COPY
SP UP
COPY
UPSPSP ✖ UP
JOIN
SP SP
COPY
UP
DISTRIBUTEDJOIN
SP ✖ UP
COPY
$$, 3 second staleness $$$, 1 secondstaleness
$, 10 second staleness
![Page 7: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/7.jpg)
Problem Formulation
• Given n sharings S:– S = {S1 Sn}– Each sharing specifies a staleness requirement in seconds
• e.g., 5 seconds
• Datasets are relations in RDBMS– Updated asynchronously (i.e., independently)
• Goal: Enable all sharing such that– Using MVs that are always consistent– All MVs under the staleness SLA– At the cheapest cost for service provider
![Page 8: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/8.jpg)
Postgres 2
Postgres 3 Postgres 4
Postgres 1
Pos
tgre
sql
Dat
abas
e
R
SMILE
Sharing Planoptimizer
InputSharings
Sharing PlanG
atew
ay
Upd
ates
Copy
Delta
¢R
RLOG
¢R
CaptureDelta
SMILE System Architecture
![Page 9: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/9.jpg)
Sharing Plan Optimizer
• R*-style optimizer– Varies join ordering and operator placement– Using a dynamic programming formulation
• Uses four operators to express SPJ transformations in sharings– DeltaToRel, Join, Union, CopyDelta
• Two cost models: – Dollar Cost of a plan– Time Cost of a plan
![Page 10: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/10.jpg)
COPYDELTA
DETATOREL DETATOREL
COPYDELTA
Machine m1Machine m2
ΔA
ΔAA ΔBB
ΔB
Δ(A⋈ΔB) Δ(ΔA⋈B)
Δ(ΔA⋈B)
Δ(A⋈B) A⋈B
JOINJOIN
UNION
DETATOREL
COPYDELTA
Machine m3
Δ(A⋈ΔB)
COPYDELTA
Sharing Plan
![Page 11: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/11.jpg)
![Page 12: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/12.jpg)
Cost Models: Dollar and Time
Dollar cost is expense to provider to execute the sharing plan, in $/second– Use Amazon EC2 pricing
Time cost is critical data path time in seconds– Using a synthetic time model for
each operator type
staleness$
![Page 13: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/13.jpg)
Time Cost Model
CopyDelta DeltaToRel
Join Union
We use a simple linear cost model to estimate the time taken by each operator
![Page 14: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/14.jpg)
Generating Global Sharing Plan• Input: Set of n sharings• Step 1: For each sharing generate a sharing plan so that:
– Plan is admissible • Means that its critical time path is less than the Staleness SLA
– Generate two plans• DPD: Cheapest Dollar Cost Plan• DPT: Smallest Critical Time Path Plan
– Discard if not admissible but choose DPD is both admissible
• Step 2: Make cheaper by merging commonalities with other sharing plans in the style of Multi-query optimization– We call merging operation as ``plumbing’’
![Page 15: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/15.jpg)
Rem
ove
Rem
ove
Plumbing Operation
SRC(pi)DST(pi)COPYDELTA
piDST(pi)
JOIN
pi
SRC (pi)
Plumbing increases the critical time path of the left plan, so valid as long as left plan is still under its staleness SLA
Perform plumbing in a greedy fashion one at a time starting with the one resulting in most cost savings
![Page 16: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/16.jpg)
Postgres 2
Postgres 3 Postgres 4
Postgres 1
Pos
tgre
sql
Dat
abas
e
R
LOG
¢R
SMILE
Sharing Planoptimizer
Sharings SharingExecutor
Sharing Plan
Pub/Sub
Push
HeartbeatAgent
Agent Agent
Agent
Gat
eway
Upd
ates
Copy
Delta
¢R
R
CaptureDelta
SMILE System Architecture
![Page 17: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/17.jpg)
Sharing Executor• Accounts for runtime variations in the system
– Change in the input update rate– Machine or resource contention or unavailability
• Basically obtains current timestamp of vertices and issues “push” operation– Push operation specifies how much to “synchronously” advance the
timestamp of each vertex in the sharing plan– Tries to combine work as much as possible
• Uses a feedback loop to automatically account for runtime variations
![Page 18: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/18.jpg)
Staleness and Push
• Current STALENESS = MAX_TS(SRCS) - TS(DEST)• PUSH: How much to advance TS(DEST)?
• Cannot be more than MIN_TS(SRCS) – TS(DEST)• Look at the paper for a sharing executor that is lazy by
design and refreshes MVs just as it is about miss the staleness SLA
![Page 19: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/19.jpg)
Experiments
• Twitter GardenHose Stream• 6 machines
– One machine generates updates and hosts base relations
– 5 machines for hosting sharing plan operators• Rate: 50—10k tweets/sec• Sharings: 5—50 sharings• SLA: 10—60 seconds
![Page 20: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/20.jpg)
Base relations
• Unpack incoming Tweets into 9 base relations
![Page 21: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/21.jpg)
Sharing Arrangements
• 25 sharing arrangements as SPJ transformations on base relations
![Page 22: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/22.jpg)
Sharing Plan
25 Sharings, 6 machines
![Page 23: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/23.jpg)
Staleness: 10k tweets/sec, 25 Sharings
![Page 24: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/24.jpg)
Why Some Sharings have a large gap?
![Page 25: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/25.jpg)
Tuples Moved across Sharing Plan
![Page 26: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/26.jpg)
Staleness before vs. after push
![Page 27: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/27.jpg)
For Varying Update Rates
• SLA violation is low even for large update rates
![Page 28: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/28.jpg)
Actual Running Cost
![Page 29: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/29.jpg)
Related Work
• View Maintenance• View Selection• Cache Placement• Data Quality/Staleness• Data Integration• Distributed Databases• Multi-query optimization• Other data sharing effort
![Page 30: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/30.jpg)
View Maintenance
• When sources not always at a consistent snapshot– Need to use compensation [Zhuge et al., SIGMOD 1995]– Rolling join [Salem et al., SIGMOD 2000]
• Shows how to compose n-way asynchronous propagation queries• Sharing plan is based on this work
• How to reduce maintenance cost?– Merge common sub-expressions in the update mechanism of different
MV’s to reduce cost [Ross et al., SIGMOD 1996][Mistry et al., SIGMOD 2001]
• Staleness in data warehouse setup:– Labrinidis et al., UMD CS TR, 1998]
![Page 31: SMILE: A Data Sharing Platform for Mobile Apps in the Cloud](https://reader036.fdocuments.us/reader036/viewer/2022062321/56812d03550346895d91dbbe/html5/thumbnails/31.jpg)
Summary
• SMILE is a declarative data sharing platform in the cloud
• Sharings can specify a transformation and a staleness SLA
• SMILE uses both static and runtime optimizations
• Experimental results show that it can handle high update rates and large number of sharings