StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar...
Transcript of StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar...
![Page 1: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/1.jpg)
StatusQuo: Making Familiar Abstractions Perform Using Program Analysis
Alvin Cheung Samuel Madden
Armando Solar-Lezama MIT
Owen Arden Andrew C. Myers
Cornell
![Page 2: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/2.jpg)
Developing Database Applications
1/8/2013 CIDR '13 2
Application Server SQL Database
Java Application Logic
SQL Query
![Page 3: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/3.jpg)
Developing Database Applications
1/8/2013 CIDR '13 3
SQL Database
Java Application Logic
PL/SQL Stored Procedures
SQL Query
Application Server
![Page 4: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/4.jpg)
Developing Database Applications
1/8/2013 CIDR '13 4
SQL Database
Java Application Logic
PL/SQL Stored Procedures
SQL Query
Application Server
Language Choice for Application Logic
Application Distribution
Program analysis to the rescue!
![Page 5: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/5.jpg)
StatusQuo
• Express application logic in ways that programmers are comfortable with
• Job of compiler & runtime to determine the most efficient implementation
1/8/2013 CIDR '13 5
![Page 6: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/6.jpg)
Two Key Technologies
• Infer queries from imperative code
• Migrate computation between servers for optimal performance
1/8/2013 CIDR '13 6
![Page 7: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/7.jpg)
Relational Operations in Imperative Code
1/8/2013 CIDR '13 7
List getUsersWithRoles () { List users = getUsersFromDB(); List roles = getRolesFromDB();
List results = new ArrayList();
for (User u : users) {
for (Role r : roles) {
if (u.roleId == r.id)
results.add(u); }}
return results; }
SELECT * FROM user
SELECT * FROM role
List getUsersWithRoles () {
return executeQuery(
“SELECT u FROM users u, roles r
WHERE u.roleId == r.id
ORDER BY u.roleId, r.id”; }
convert to
![Page 8: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/8.jpg)
Relational Operations in Imperative Code
1/8/2013 CIDR '13 8
List getUsersWithRoles () { List users = getUsersFromDB(); List roles = getRolesFromDB();
List results = new ArrayList();
for (User u : users) {
for (Role r : roles) {
if (u.roleId == r.id)
results.add(u); }}
return results; }
List getUsersWithRoles () {
return executeQuery(
“SELECT u FROM users u, roles r
WHERE u.roleId == r.id
ORDER BY u.roleId, r.id”; }
convert to
Goal
Find a variable that we can rewrite into a
SQL expression
post-condition variable results
![Page 9: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/9.jpg)
Query By Synthesis (QBS)
• Identify potential code fragments – i.e., regions of code that fetches persistent
data and return values
• Find SQL expressions for post-condition variables
• Try to prove that those expressions preserve program semantics – if so, convert the code!
1/8/2013 CIDR '13 9
![Page 10: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/10.jpg)
Initial Code Fragments Identification
• Find program points that retrieve
persistent data
• Run an inter-procedural analysis that: – determine where persistent data are used – delimit code fragment to analyze
1/8/2013 CIDR '13 10
![Page 11: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/11.jpg)
Search for Post-Condition Expressions
1/8/2013 CIDR '13 11
List getUsersWithRoles () { List users = query(select * from users); List roles = query(select * from roles);
List results = [];
for (User u : users) {
for (Role r : roles) {
if (u.roleId == r.id)
results = results : [] }}
return results; }
Relations involved: users, roles Possible expressions to consider for results: σf(users) topf(users) πf(users ⨝g roles) πf(σg(users) ⨝h roles) other expressions involving users, roles
Infinite search space size!
users roles
results
![Page 12: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/12.jpg)
Constraints for Post-Condition Expressions
1/8/2013 CIDR '13 12
List getUsersWithRoles () { List users = query(select * from users); List roles = query(select * from roles);
List results = [];
for (User u : users) {
for (Role r : roles) {
if (u.roleId == r.id)
results = results : [] }}
return results; }
users roles
results
Hoare-style program verification
If
post-condition expression
outer loop invariant
outer loop invariant outer loop terminates
then post-condition expression
is true and
is true
Still need a smarter way to search
results = πuser( users ⨝roleId = id roles )
results = πuser( users[0 .. i] ⨝roleId = id roles )
![Page 13: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/13.jpg)
Search for Post-Condition Expressions and Invariants
• Use program synthesis as search engine
1/8/2013 CIDR '13 13
Program synthesizer
Symbolic desc. of search space
Solution constraints
Expression that satisfies all the constraints
Symbolic manipulation
Counter-example driven search
![Page 14: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/14.jpg)
Experiments
1/8/2013 CIDR '13 14
![Page 15: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/15.jpg)
Real-world Evaluation
1/8/2013 CIDR '13 15
Wilos (project management application) – 62k LOC
Operation type # Fragments found
# Fragments converted
Projection 1 1
Selection 13 10
Join 7 7
Aggregation 11 10
Total 33 28
![Page 16: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/16.jpg)
100
1K
10K
100K
1000K
0 20K 40K 60K 80K 100K
Exec
uti
on
tim
e (m
s)
Number of roles / users in DB
original (lazy)
inferred (lazy)
Performance Evaluation: Join Query
1/8/2013 CIDR '13 16
Nested-loop join Hash join! O(n2) O(n)
![Page 17: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/17.jpg)
Developing Database Applications
1/8/2013 CIDR '13 17
SQL Database
Java Application Logic
PL/SQL Stored Procedures
SQL Query
Application Server
Application Distribution
![Page 18: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/18.jpg)
Running Example
discount = executeQuery("select discount from customers where id = " + cid);
totalAmount = orderTotal * (1 – discount);
credit = executeQuery("select credit from customers where id = " + cid);
if (credit < totalAmount)
printToConsole("Only " + credit + " in account!");
else
executeUpdate("update customer set credit = " + (credit – totalAmount) + " where id = " + cid);
1/8/2013 18 CIDR '13
![Page 19: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/19.jpg)
Actual Execution
discount = executeQuery("select discount from customers where id = " + cid);
totalAmount = orderTotal * (1 – discount);
credit = executeQuery("select credit from customers where id = " + cid);
if (credit < totalAmount)
printToConsole("Only " + credit + " in account!");
else
executeUpdate("update customer set credit = " + (credit – totalAmount) + " where id = " + cid);
1/8/2013 19 CIDR '13
DB
APP
APP
DB
DB
![Page 20: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/20.jpg)
Actual Execution
discount = executeQuery("select discount from customers where id = " + cid);
totalAmount = orderTotal * (1 – discount);
credit = executeQuery("select credit from customers where id = " + cid);
if (credit < totalAmount)
printToConsole("Only " + credit + " in account!");
else
executeUpdate("update customer set credit = " + (credit – totalAmount) + " where id = " + cid);
1/8/2013 20 CIDR '13
network communication
network communication
network communication
network communication
DB
APP
APP
DB
DB
![Page 21: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/21.jpg)
Speeding up Execution
discount = executeQuery("select discount from customers where id = " + cid);
totalAmount = orderTotal * (1 – discount);
credit = executeQuery("select credit from customers where id = " + cid);
if (credit < totalAmount)
printToConsole("Only " + credit + " in account!");
else
executeUpdate("update customer set credit = " + (credit – totalAmount) + " where id = " + cid);
1/8/2013 21 CIDR '13
DB
APP
DB
![Page 22: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/22.jpg)
Speeding up Execution
discount = executeQuery("select discount from customers where id = " + cid);
totalAmount = orderTotal * (1 – discount);
credit = executeQuery("select credit from customers where id = " + cid);
if (credit < totalAmount)
printToConsole("Only " + credit + " in account!");
else
executeUpdate("update customer set credit = " + (credit – totalAmount) + " where id = " + cid);
1/8/2013 22 CIDR '13
data dependency DB
APP
DB
control dependency
![Page 23: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/23.jpg)
Speeding up Execution
discount = executeQuery("select discount from customers where id = " + cid);
totalAmount = orderTotal * (1 – discount);
credit = executeQuery("select credit from customers where id = " + cid);
if (credit < totalAmount)
printToConsole("Only " + credit + " in account!");
else
executeUpdate("update customer set credit = " + (credit – totalAmount) + " where id = " + cid);
1/8/2013 23 CIDR '13
DB Server
DB
APP
DB
control dependency
data dependency
![Page 24: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/24.jpg)
Introducing Pyxis
• “Store-procedurizes” DB apps and pushes computation to the DB
• Adaptively controls the amount of computation pushed to DB for optimal performance
• No programmer intervention required
1/8/2013 24 CIDR '13
![Page 25: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/25.jpg)
Using Pyxis
1/8/2013 CIDR '13 25
![Page 26: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/26.jpg)
How Pyxis Works
1/8/2013 CIDR '13 26
Instrument
Partition
Monitor
App Server DB Server
Deploy
Java SQL Java SQL Java
Java
Java
SQL Java SQL
SQL Java
Java SQL Java SQL Java SQL Java
SQL
Java
control transfer
![Page 27: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/27.jpg)
How Pyxis Works
1/8/2013 CIDR '13 27
Monitor
App Server DB Server
Deploy Java
Java
SQL Java
SQL
SQL
Java
control transfer
Instrument
Partition
Java SQL Java SQL Java SQL Java
Java SQL Java SQL Java SQL Java
![Page 28: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/28.jpg)
Generating Program Partitions
• Deploy and profile application as-is • Construct a dependence graph of
program statements – captures both control and data flow
• Formulate linear program from profile data and dependence graph – solution gives a partitioning of the source
code
1/8/2013 CIDR '13 28
![Page 29: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/29.jpg)
Executing Partitioned Programs
• Pyxis compiler translates partitioned code into standard Java code
• Pyxis runtime executes compiled Java code – runtime is just another Java program
running on a standard JVM – includes monitoring component to
determine partition switching
1/8/2013 CIDR '13 29
![Page 30: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/30.jpg)
Experiments
1/8/2013 CIDR '13 30
![Page 31: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/31.jpg)
Experiment Setup
• TPC-C Java implementation – 20 terminals issuing new order transactions – DB server has 16 cores total
– Compared against two implementations:
• JDBC: everything on app server except for JDBC stmts • Manual: custom “store procedurized”
implementation where everything is on the DB server
1/8/2013 31 CIDR '13
![Page 32: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/32.jpg)
5
10
15
20
25
100 300 500 700 900 1100 1300
Ave
rage
Lat
ency
(ms)
Average Thruput (xact / s)
JDBC
Manual
Pyxis
All Cores Available
1/8/2013 32 CIDR '13
Pyxis generated implementation: 3x latency reduction 1.7x thruput increase
![Page 33: StatusQuo: Making Familiar Abstractions Perform Using Program Analysis StatusQuo: Making Familiar Abstractions Perform Using Program Analysis Alvin Cheung Samuel Madden Armando Solar-Lezama](https://reader031.fdocuments.us/reader031/viewer/2022022610/5b968bad09d3f25a5a8ba25c/html5/thumbnails/33.jpg)
StatusQuo
Ease DB application development
Convert imperative program statements into declarative SQL
Fully automatic code partitioning using
application and server characteristics
db.csail.mit.edu/statusquo
1/8/2013 CIDR '13 33