MedleySolver: Online SMT Algorithm Selection
Transcript of MedleySolver: Online SMT Algorithm Selection
![Page 1: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/1.jpg)
MedleySolver: Online SMT Algorithm Selection
Nikhil Pimpalkhare1, Federico Mora1, Elizabeth Polgreen1, 2, and Sanjit A. Seshia1
Sunday, June 27, 2021 24th International Conference on Theory and Applications of Satisfiability Testing
1 University of California, Berkeley2 University of Edinburgh
![Page 2: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/2.jpg)
Where Are SMT Solvers Used?Example Applications
⢠Verification engines, ⢠check many verification conditions.
⢠Symbolic execution engines,⢠check many path conditions.
⢠Program synthesis engines, ⢠check many candidate programs.
Example Tools
2
![Page 3: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/3.jpg)
What is SMT Algorithm Selection?
⢠For a given query, can we predict the solver that will perform best?
âđâ â đ = đ â âđâ?
Solver B Solver A Solver C
đđđđ´đ
âđâ â đ = đ â âđâ?
Solver B Solver A Solver C
Step Not TakenStep Taken
[1] J. R.Rice: The algorithm selection problem. Advances in Computers (1976).
3
![Page 4: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/4.jpg)
Desired Features for Algorithm Selection
ApproachFeature
Expert Encoded Decision Rule
Existing Methods(Offline Learning)
MedleySolver(Online Learning)
No Manual Input
Minimal Upfront Costs
No Data Requirements
No Solver Requirements
No Need to Repeat Upfront Costs
Fine Grained Decisions
(End-User Perspective)
4
Strong
Medium
Weak
Feature Support
![Page 5: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/5.jpg)
Existing ApproachesExpert Encoded
E.g., Z3âs quantifier-free bit-vector solver tactic
Machine Learned
E.g., SatZilla [2], MachSMT [3], FastSMT [4], âŚ
Dataset of Formulas
Learning
Policy
[2] Xu, L., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Satzilla: Portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. (2008)[3] Scott, J., Niemetz, A., Preiner, M., Nejati, S., Ganesh, V.: MachSMT: A machine learning-based algorithm selector for SMT solvers. TACAS (2021)[4] Balunovic, M., Bielik, P., Vechev, M.T.: Learning to solve SMT formulas. NeurIPS (2018)
5
![Page 6: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/6.jpg)
Existing ApproachesExpert Encoded
E.g., Z3âs quantifier-free bit-vector solver tactic
Machine Learned
E.g., SatZilla [2], MachSMT [3], FastSMT [4], âŚ
Dataset of Formulas
Learning
Policy
[2] Xu, L., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Satzilla: Portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. (2008)[3] Scott, J., Niemetz, A., Preiner, M., Nejati, S., Ganesh, V.: MachSMT: A machine learning-based algorithm selector for SMT solvers. TACAS (2021)[4] Balunovic, M., Bielik, P., Vechev, M.T.: Learning to solve SMT formulas. NeurIPS (2018)
SMT Enabled Tools
6
![Page 7: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/7.jpg)
Existing ApproachesExpert Encoded
E.g., Z3âs quantifier-free bit-vector solver tactic
Machine Learned
E.g., SatZilla [2], MachSMT [3], FastSMT [4], âŚ
Dataset of Formulas
Learning
Policy
[2] Xu, L., Hutter, F., Hoos, H.H., Leyton-Brown, K.: Satzilla: Portfolio-based algorithm selection for SAT. J. Artif. Intell. Res. (2008)[3] Scott, J., Niemetz, A., Preiner, M., Nejati, S., Ganesh, V.: MachSMT: A machine learning-based algorithm selector for SMT solvers. TACAS (2021)[4] Balunovic, M., Bielik, P., Vechev, M.T.: Learning to solve SMT formulas. NeurIPS (2018)
1. Big upfront costs2. Requires a lot of data3. Must re-train for change in
solvers/hardware/domain
1. Tailored to specific solvers2. Not fine grained
(takes time to engineer)
SMT Enabled Tools
7
![Page 8: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/8.jpg)
Desired Features for Algorithm Selection
ApproachFeature
Expert Encoded Decision Rule
Existing Methods(Offline Learning)
MedleySolver(Online Learning)
No Manual Input
Minimal Upfront Costs
No Data Requirements
No Solver Requirements
No Need to Repeat Upfront Costs
Fine Grained Decisions
(End-User Perspective)
8
Strong
Medium
Weak
Feature Support
![Page 9: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/9.jpg)
Desired Features for Algorithm Selection
ApproachFeature
Expert Encoded Decision Rule
Existing Methods(Offline Learning)
MedleySolver(Online Learning)
No Manual Input
Minimal Upfront Costs
No Data Requirements
No Solver Requirements
No Need to Repeat Upfront Costs
Fine Grained Decisions
(End-User Perspective)
9
Strong
Medium
Weak
Feature Support
![Page 10: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/10.jpg)
Our Proposal: Use Online Learning
⢠Learn as we go with what we have locally!
SMT Enabled Tools. E.g.,
âŚMedleySolver
đđ´đ/đđđđ´đ
InternalInput/Output
LEARN!
đđ´đ/đđđđ´đâŚ
10
![Page 11: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/11.jpg)
MedleySolver Overview
11
![Page 12: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/12.jpg)
MedleySolver Overview
Featurize
Input/OutputInternal
12
![Page 13: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/13.jpg)
MedleySolver Overview
Featurize
Input/OutputInternal
Feature Values
Feature Values
13
![Page 14: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/14.jpg)
MedleySolver Overview
MAB Solver Selector
FeaturizeâŚ
Input/OutputInternal
Feature Values
Feature Values
Ranking
14
![Page 15: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/15.jpg)
MedleySolver Overview
Featurize
Time Predictor
âŚ
Input/OutputInternal
Feature Values
Feature Values
Timeouts
15
![Page 16: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/16.jpg)
MedleySolver Overview
MAB Solver Selector
Featurize
Time Predictor
âŚ
âŚ
Input/OutputInternal
Feature Values
Feature Values
Ranking
Timeouts
16
![Page 17: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/17.jpg)
MedleySolver Overview
MAB Solver Selector
Featurize
Time Predictor
Deploy SolversâŚ
âŚ
Input/OutputInternal
Feature Values
Feature Values
Ranking
Timeouts
17
![Page 18: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/18.jpg)
MedleySolver Overview
MAB Solver Selector
Featurize
Time Predictor
Deploy Solvers
đđ´đ/đđđđ´đ
âŚ
âŚ
Input/OutputInternal
Feature Values
Feature Values
Ranking
Timeouts
Results
Results
18
![Page 19: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/19.jpg)
Computing Features
Featurize
![Page 20: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/20.jpg)
Feature Selection
⢠Used a set of context-free features⢠E.g., ânumber of integer variablesâ
⢠Identified relevant features using Pearson R coefficient⢠Compute correlation of each feature against solving time, for each solver⢠Use 10 features with the highest average coefficient
20
BAD
GOOD
Hypothetical feature correlation graphs (actual graphs are messier..)
![Page 21: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/21.jpg)
Selecting Solver Order
MAB Solver Selector
![Page 22: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/22.jpg)
Selecting Solver Order
Multi-Armed Bandit Intuition
⢠Arms are slot machines⢠Reward is payout⢠Pick what slot machine to use
Our Multi-Armed Bandit Use
⢠Arms are solvers⢠Reward is negative time taken⢠Pick order of solvers to run
22
MAB Solver SelectorImplemented Algorithms
⢠Thompson Sampling
⢠k-Nearest-Neighbour Bandit
⢠Exp3
⢠Neural network bandit
⢠LinUCB
![Page 23: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/23.jpg)
Thompson SamplingNon-Contextual Algorithm 1
⢠Model rewards using probability distributions ⢠(beta distribution)
⢠One distribution per solver⢠Sample from each beta distribution
⢠execute solvers in order of sample value
⢠Update using Bayesâ rule⢠Arms which are not thoroughly explored will have
higher variance, making them more likely to be explored in the future
23
![Page 24: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/24.jpg)
k-Nearest NeighborContextual Algorithm 1
⢠Given a query, k-NN looks at the k closest past solved queries and orders solvers by their number of appearances in these k, breaking ties randomly⢠Update: if query k was solved, we put it
into our list of past solved queries with the corresponding solver label
24
3-Nearest Neighbors
A
?
B
A
B
B
B
![Page 25: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/25.jpg)
k-Nearest NeighborContextual Algorithm 1
⢠Given a query, k-NN looks at the k closest past solved queries and orders solvers by their number of appearances in these k, breaking ties randomly⢠Update: if query k was solved, we put it
into our list of past solved queries with the corresponding solver label
25
3-Nearest Neighbors
A
?
B
A
B
B
B
![Page 26: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/26.jpg)
k-Nearest NeighborContextual Algorithm 1
⢠Given a query, k-NN looks at the k closest past solved queries and orders solvers by their number of appearances in these k, breaking ties randomly⢠Update: if query k was solved, we put it
into our list of past solved queries with the corresponding solver label
26
3-Nearest Neighbors
A
?
B
A
B
B
B
![Page 27: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/27.jpg)
k-Nearest NeighborContextual Algorithm 1
⢠Given a query, k-NN looks at the k closest past solved queries and orders solvers by their number of appearances in these k, breaking ties randomly⢠Update: if query k was solved, we put it
into our list of past solved queries with the corresponding solver label
27
3-Nearest Neighbors
A
?
B
A
B
B
B
![Page 28: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/28.jpg)
k-Nearest NeighborContextual Algorithm 1
⢠Given a query, k-NN looks at the k closest past solved queries and orders solvers by their number of appearances in these k, breaking ties randomly⢠Update: if query k was solved, we put it
into our list of past solved queries with the corresponding solver label
28
3-Nearest Neighbors
A
?
B
A
B
B
B
Run A, then B
![Page 29: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/29.jpg)
k-Nearest NeighborContextual Algorithm 1
⢠Given a query, k-NN looks at the k closest past solved queries and orders solvers by their number of appearances in these k, breaking ties randomly⢠Update: if query k was solved, we put it
into our list of past solved queries with the corresponding solver label
29
3-Nearest Neighbors
A
?
B
A
B
B
B
Suppose A fails, B succeeds
![Page 30: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/30.jpg)
B
k-Nearest NeighborContextual Algorithm 1
⢠Given a query, k-NN looks at the k closest past solved queries and orders solvers by their number of appearances in these k, breaking ties randomly⢠Update: if query k was solved, we put it
into our list of past solved queries with the corresponding solver label
30
3-Nearest Neighbors
A
B
A
B
B
B
Suppose A fails, B succeeds
![Page 31: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/31.jpg)
B
k-Nearest NeighborContextual Algorithm 1
⢠Given a query, k-NN looks at the k closest past solved queries and orders solvers by their number of appearances in these k, breaking ties randomly⢠Update: if query k was solved, we put it
into our list of past solved queries with the corresponding solver label
31
3-Nearest Neighbors
A
B
A
B
B
B
?
Repeat for new query
![Page 32: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/32.jpg)
Allocating Timeouts
Time Predictor
![Page 33: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/33.jpg)
Allocating Timeouts
⢠Task: Split overall timeout đ⢠allocate time to each solver in order.
⢠Cut off solver once we are confident it is unlikely to terminate
33
![Page 34: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/34.jpg)
Modeling Execution Time as an Exponential
⢠Solver runtimes look like samples from an exponential distribution⢠Estimate parameter đ using maximum likelihood estimation
⢠For a hyper-param đż, find đĄ such that solver is đż% likely to terminate
⢠After đĄ seconds try the next solver!
34
đâ =đ
â" đ"
t =âln(đż + đ#$%)
đâ
![Page 35: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/35.jpg)
Contextual Time Allocation
⢠k-Nearest Neighbor Exponential:⢠Use the k-nearest queries to estimate đ
⢠Linear Regression:⢠Train a linear model which
⢠takes in a queryâs feature vector as input,⢠outputs the expected runtime
35
7s
3-Nearest Neighbors
1s
1s
?1s
2s 8s 5s
![Page 36: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/36.jpg)
Deploying Solvers and Recap
![Page 37: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/37.jpg)
MedleySolver Overview
MAB Solver Selector
Featurize
Time Predictor
Deploy Solvers
đđ´đ/đđđđ´đ
âŚ
âŚ
Input/OutputInternal
Feature Values
Feature Values
Ranking
Timeouts
Results
Results
37
![Page 38: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/38.jpg)
Desired Features for Algorithm Selection
ApproachFeature
Expert Encoded Decision Rule
Existing Methods(Offline Learning)
MedleySolver(Online Learning)
No Manual Input
Minimal Upfront Costs
No Data Requirements
No Solver Requirements
No Need to Repeat Upfront Costs
Fine Grained Decisions
(End-User Perspective)
38
Strong
Medium
Weak
Feature Support
![Page 39: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/39.jpg)
Desired Features for Algorithm Selection
ApproachFeature
Expert Encoded Decision Rule
Existing Methods(Offline Learning)
MedleySolver(Online Learning)
No Manual Input
Minimal Upfront Costs No pre-training
No Data Requirements
No pre-training
No Solver Requirements
No Need to Repeat Upfront Costs
Fine Grained Decisions
(End-User Perspective)
39
Strong
Medium
Weak
Feature Support
![Page 40: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/40.jpg)
Desired Features for Algorithm Selection
ApproachFeature
Expert Encoded Decision Rule
Existing Methods(Offline Learning)
MedleySolver(Online Learning)
No Manual Input
Minimal Upfront Costs No pre-training
No Data Requirements
No pre-training
No Solver Requirements
No assumptions
No Need to Repeat Upfront Costs
Fine Grained Decisions
(End-User Perspective)
40
Strong
Medium
Weak
Feature Support
![Page 41: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/41.jpg)
Desired Features for Algorithm Selection
ApproachFeature
Expert Encoded Decision Rule
Existing Methods(Offline Learning)
MedleySolver(Online Learning)
No Manual Input
Minimal Upfront Costs No pre-training
No Data Requirements
No pre-training
No Solver Requirements
No assumptions
No Need to Repeat Upfront Costs
Adapt to changes as they come.
Fine Grained Decisions
(End-User Perspective)
41
Strong
Medium
Weak
Feature Support
![Page 42: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/42.jpg)
Desired Features for Algorithm Selection
ApproachFeature
Expert Encoded Decision Rule
Existing Methods(Offline Learning)
MedleySolver(Online Learning)
No Manual Input
Minimal Upfront Costs No pre-training
No Data Requirements
No pre-training
No Solver Requirements
No assumptions
No Need to Repeat Upfront Costs
Adapt to changes as they come.
Fine Grained Decisions
Complex, flexible decision rules
(End-User Perspective)
42
Strong
Medium
Weak
Feature Support
![Page 43: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/43.jpg)
Evaluation and Future Work
![Page 44: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/44.jpg)
Experimental Setup
⢠6 individual SMT solvers⢠CVC4, MathSAT, Z3, Boolector, Bitwuzla, and Yices
⢠4 individual benchmark sets⢠BV, QF_ABV, Sage2, and Uclid5 (first 3 sets are from SMTCOMP)⢠500 queries each (randomly sampled without replacement)
⢠6 MedleySolver configurations in paper, 1 for presentation⢠10-Nearest Neighbor for order selector⢠10-Nearest Neighbor exponential distribution estimation for time predictor
⢠Dell PowerEdge C6220 server, 60 second timeout per query.
44
![Page 45: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/45.jpg)
Exp. 1: Comparison to Individual Solvers
~ 5 hours better!
Best Individual Solver
MedleySolver
Par-2
Sco
re
45
![Page 46: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/46.jpg)
Exp. 2: Comparison to Pre-Trained Tools
⢠Compare against MachSMT⢠State-of-the-art
⢠Report par-2 score for test⢠Report time in seconds for training
⢠40% of queries for training⢠Only used by MachSMT
⢠60% of queries for testing⢠Used by both
46
BenchmarkTool
BV QF_ABV Sage2 Uclid5 Combined
MedleySolver Test: 1638.7 Test: 310.5 Test: 9245.3 Test: 4248.0 Test: 18565.5
MachSMT Test: 1458.3Train: 33895.5
Test: 919.2Train: 4498.9
Test: 8516.1Train: 55115.5
Test: 2430.9Train: 276419.8
Test: 12539.1Train: 300072.8
Virtual Best Test: 801.7 Test: 184.3 Test: 5204.2 Test: 1464.7 Test: 6746.0
![Page 47: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/47.jpg)
Future Work
⢠Whitebox monitoring techniques⢠instead of black box timeout estimation
⢠What if one of the solvers is a distributed solver in the cloud?⢠Front end to decide when to solve locally and when to query server
47
![Page 48: MedleySolver: Online SMT Algorithm Selection](https://reader034.fdocuments.us/reader034/viewer/2022050214/626e40e0f30622345d3b42a7/html5/thumbnails/48.jpg)
âŚMedleySolver
LEARN!
đđ´đ/đđđđ´đ
đđ´đ/đđđđ´đâŚ
MedleySolver: Online SMT Algorithm Selection
Paper PDF
Code & Data