Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from...
Transcript of Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from...
![Page 1: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/1.jpg)
Precise Piecewise Affine Models from Input-Output Data
Rajeev Alur Nimit Singhania University of Pennsylvania
EXCAPE PI Meeting, June 2015
![Page 2: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/2.jpg)
2
Motivation
Real World Systems
Complex
Models
Easy to Analyze
![Page 3: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/3.jpg)
3
Motivation
Real World Systems Models
Input-Output Data
Data Driven
![Page 4: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/4.jpg)
4
Motivation
Real World Systems
Data Driven
Input-Output Data
Piecewise Affine Models
![Page 5: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/5.jpg)
5
1D Piecewise Affine Model
if (x < 2.5) 5 - x else if (x < 7.5) x else 15 - x
![Page 6: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/6.jpg)
6
2D Piecewise Affine Model
l1: 0
l2: y - x
l3: (y - x - 16)/5
Linear Functions
![Page 7: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/7.jpg)
6
2D Piecewise Affine Model
l1: 0
l2: y - x
l3: (y - x - 16)/5
Linear Functions
y < x - 4(x < -1 and y < -1) or (x > 1 and y > 1)
Guard Predicates
![Page 8: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/8.jpg)
7
Problem
Input-Output Data D Piecewise Affine Model f
f closely approximates D f is simple
![Page 9: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/9.jpg)
Problem
8
• Given input-output data D: Rd x R and error bound δ, learn a piecewise affine model f: Rd → R
• | f(a) - b | ≤ δ for all points (a, b) ∈ D
• Minimize size of model
• Minimize number of linear functions and size of predicates
![Page 10: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/10.jpg)
Problem
8
• Given input-output data D: Rd x R and error bound δ, learn a piecewise affine model f: Rd → R
• | f(a) - b | ≤ δ for all points (a, b) ∈ D
• Minimize size of model
• Minimize number of linear functions and size of predicates
• Two problems
• Learn linear functions
• Learn guard predicates for associated regions
![Page 11: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/11.jpg)
Problem
8
• Given input-output data D: Rd x R and error bound δ, learn a piecewise affine model f: Rd → R
• | f(a) - b | ≤ δ for all points (a, b) ∈ D
• Minimize size of model
• Minimize number of linear functions and size of predicates
• Two problems
• Learn linear functions
• Learn guard predicates for associated regions
• Minimizing the model size - NP-Hard
• Best effort solution
![Page 12: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/12.jpg)
• Select random point p
• Fit a linear function that covers points in neighborhood of p
• Remove the covered points and repeat
9
Learning Linear Functions
![Page 13: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/13.jpg)
10
Learning Linear Functions
l3: (y - x + 16)/5
l1: 0
l2: y - x
![Page 14: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/14.jpg)
10
Learning Linear Functions
l3: (y - x + 16)/5
l1: 0
l2: y - x
![Page 15: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/15.jpg)
10
Learning Linear Functions
l3: (y - x + 16)/5
l1: 0
l2: y - x
![Page 16: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/16.jpg)
10
Learning Linear Functions
l3: (y - x + 16)/5
l1: 0
l2: y - x
![Page 17: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/17.jpg)
11
Learning Guard Predicates
l3: (y - x + 16)/5
l1: 0
l2: y - x
![Page 18: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/18.jpg)
11
Learning Guard Predicates
l3: (y - x + 16)/5
l1: 0
l2: y - x
![Page 19: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/19.jpg)
11
Learning Guard Predicates
Positive
Negative
l3: (y - x + 16)/5
l1: 0
l2: y - x
![Page 20: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/20.jpg)
12
Learning Guard Predicates
• Problem of classification
![Page 21: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/21.jpg)
12
Learning Guard Predicates
• Problem of classification
• SVM, Logistic Regression
• Small model
• Imprecise
![Page 22: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/22.jpg)
12
Learning Guard Predicates
• Problem of classification
• SVM, Logistic Regression
• Small model
• Imprecise
• Decision Trees, Nearest Neighbor
• Precise
• Huge model
![Page 23: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/23.jpg)
Learning Guard Predicates
• Propose a counterexample guided approach
• Learns precise classifier
• Learns small predicate
• Inspired by “Beautiful Interpolants” by Albarghouthi and McMillan
![Page 24: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/24.jpg)
14
Learning Guard Predicates
![Page 25: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/25.jpg)
14
Learning Guard Predicates
2y + 1 < 0
![Page 26: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/26.jpg)
14
Learning Guard Predicates
2y + 1 < 0
![Page 27: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/27.jpg)
14
Learning Guard Predicates
2y + 1 < 0
![Page 28: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/28.jpg)
15
Learning Guard Predicates
3y - 4x - 10 < 0
![Page 29: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/29.jpg)
15
Learning Guard Predicates
3y - 4x - 10 < 0
![Page 30: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/30.jpg)
15
Learning Guard Predicates
3y - 4x - 10 < 0
![Page 31: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/31.jpg)
16
Learning Guard Predicates
4y - 3x - 16 < 0
![Page 32: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/32.jpg)
16
Learning Guard Predicates
4y - 3x - 16 < 0
![Page 33: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/33.jpg)
16
Learning Guard Predicates
4y - 3x - 16 < 0
![Page 34: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/34.jpg)
17
Learning Guard Predicates
4y - 3x - 16 < 0 and 2y - 3x + 6 > 0
![Page 35: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/35.jpg)
18
Learning Guard Predicates
x < -1.5 and y < -1.5 or
x > 1.5 and y > 1.5
![Page 36: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/36.jpg)
Evaluation
19
Test Root Mean Square Error (x 0.1)
T1 T2 T3 T4
Mosaic 0.516 0.61 0.969 2.248
LinReg + DTree 0.551 0.642 1.142 3.114
LinReg + SVM 0.571 0.949 1.159 3.585
Ferrari-Trecate 0.731 1.122 1.534 3.058
Data from Pick and Place Machines (Averaged over 15 runs)
![Page 37: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/37.jpg)
Conclusion
• Novel application of Verification to Machine Learning
• Scalable techniques for precise reasoning
• Counter-example guided strategies
• Abstraction-refinement
• Compositional reasoning
• Useful when mistakes in data are costly
20
![Page 38: Precise Piecewise Affine Models from Input-Output Data · Precise Piecewise Affine Models from Input-Output Data Rajeev Alur Nimit Singhania University of Pennsylvania EXCAPE PI Meeting,](https://reader030.fdocuments.us/reader030/viewer/2022040821/5e6b16c158799a3308332705/html5/thumbnails/38.jpg)
• Project Webpage: http://www.seas.upenn.edu/~nimits/mosaic/
• Source Code and Benchmarks: https://github.com/nimit-singhania/mosaic
21
Mosaic