Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas,...
Transcript of Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas,...
![Page 1: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/1.jpg)
Formal Methods in the Real World: Specification, Verification &
Synthesis
Sanjit A. SeshiaProfessor
UC Berkeley
EECS 219CMay 1, 2019
![Page 2: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/2.jpg)
Synthesis of Controllers/Plans for Cyber‐Physical Systems
S. A. Seshia 2
[Shoukry et al., HSCC 2017, CDC 2017, Proc. IEEE 2018;Desai et al. ICCPS 2017; Saha et al., IROS 2014]
Joint work with:Ankush Desai, Shromona Ghosh, Pierluigi Nuzzo, Indranil Saha, Yasser Shoukry, Vijay Kumar, George Pappas, Shaz Qadeer,
Alberto Sangiovanni‐Vincentelli, Paulo Tabuada
![Page 3: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/3.jpg)
TerraSwarm Research Center & NSF ExCAPE project
Goal: Correct‐by‐Construction Motion Planning for Robotics
3
Declarative Task Specification (Temporal Logic)[+ Examples]
Safe, Correct Executable Software
ComponentLibrary
CompilerDrone executing plan in ROS/Gazebo Simulator
![Page 4: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/4.jpg)
Challenges
• Scalable synthesis of motion plans blending high‐level (discrete) and low‐level (continuous) control
• Implementation in software on top of networked robotics platforms
• Dealing with untrusted components, uncertainty in environment, and ML‐based perception
S. A. Seshia 4
![Page 5: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/5.jpg)
Motion Planning Problem
5
• Problem: Compute an obstacle‐free trajectory that is feasible with the given robot dynamics within the target workspace.
• Core problem for autonomous vehicles
• Challenges:
– Handling high‐level tasks (captured by formalisms such as Linear Temporal Logic)
– Handling distributed team of robots
– Handling uncertainty
[Shoukry et al., HSCC 2017, CDC 2017, Proc. IEEE 2018;Saha et al., IROS 2014]
![Page 6: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/6.jpg)
Related Work: Abstraction Based Techniques
6
Discrete Continuous
X
Scales poorly as the number of continuousstates increases
• P. Tabuada and G. J. Pappas, “Linear time logic control of discrete-time linear systems,” TAC 2006.
• M. Kloetzer and C. Belta, “Temporal Logic Planning and Control of Robotic Swarms by Hierarchical Abstractions,” TAC 2007.
• G. E. Fainekos, A. Girard, H. Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009.
• …
![Page 7: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/7.jpg)
Related Work: Optimization Based Techniques
7
Discrete Continuous• Alberto Bemporad and Manfred
Morari, “Control of systems integrating logic, dynamics, and constraints,” Automatica, 35(3), 1999.
• E. M. Wolff, U. Topcu, and R. M. Murray, “Optimization-based trajectory generation with linear temporal logic specifications,” ICRA 2014.
• V. Raman, A. Donze, D. Sadigh, R. Murray, S. Seshia, “Reactive synthesis from signal temporal logic specifications,” HSCC 2015.
• …
Scales poorly as the number of discretestates increases
![Page 8: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/8.jpg)
Two Extremes
8
Discrete Continuous(Abstraction based)
Discrete Continuous(Optimization based)
X
Scales poorly as the number of continuousstates increases
Scales poorly as the number of discretestates increases
![Page 9: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/9.jpg)
Boolean Constraints
Convex Constraints
Satisfiability Modulo Convex Programming
9
• SAT Solvers: central tools in formal methods to reason about discrete dynamics.
• Convex Optimization: central tools in control theory to reason about continuous dynamics.
• Approach for hybrid dynamics: SAT Solving + Convex Programming
Convex Optimization
Mixed IntegerProgramming
SAT + ConvexSAT Solvers SMT
Solvers
[Shoukry et al., HSCC 2017]
![Page 10: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/10.jpg)
Motivating Example: Obstacle Avoidance
10
• Given:• Robot dynamics (linear)• Input constraints• Initial and final states
• Generate the input sequence (controller)
![Page 11: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/11.jpg)
Motivating Example: Obstacle Avoidance
11
• Given:• Robot dynamics (linear)• Input constraints• Initial and final states
• Generate the input sequence (controller)
![Page 12: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/12.jpg)
Motivating Example: Obstacle Avoidance
12
![Page 13: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/13.jpg)
Motivating Example: Obstacle Avoidance
13
![Page 14: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/14.jpg)
Motivating Example: Obstacle Avoidance
14
SMC FormulaMonotoneDefinition:
![Page 15: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/15.jpg)
Applications: Controller Synthesis
15
Obstacle Avoidance
LTL Motion Planning
Multi-robotMotion Planning
![Page 16: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/16.jpg)
Secure Traffic Routing
Applications: CPS Security
16
Secure Localization
Secure State Estimation
![Page 17: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/17.jpg)
Satisfiability Modulo Convex Programming
17
Monotone SMC Formula
The satisfiability of the monotone SMC formula can always be cast as a feasibility problem for a finite disjunction of convex constraints.
Any feasibility problem for a finite disjunction of convex programs can be posed as a satisfiability problem for a monotone SMC formula.
Theorem:
![Page 18: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/18.jpg)
Complexity of Solving SMC
18
Reduce the number of iterations?
Monotone SMC Formula
![Page 19: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/19.jpg)
Solving SMC
19
Key idea: abstraction-based search
Monotone SMC Formula
![Page 20: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/20.jpg)
• Step 0:• Given a monotone SMC formula , construct its
Boolean expansion:
where is obtained from by replacing each convex constraint with a Boolean variable
Boolean Expansion
20
Lemma:and are equi-satisfiable.
![Page 21: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/21.jpg)
Solver Operation
21
• Step I: Solve the “Booleanabstraction” of the problem.
• Step II: Extract which convexconstraints are active.
• Step III: Check the satisfiability of:
• Step IV: Generate UNSAT certificate:
![Page 22: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/22.jpg)
Solver Operation
22
Performance?
![Page 23: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/23.jpg)
Minimal UNSAT Certificate
23
• Finding a minimal UNSAT certificate is equivalent to finding an Irreducible Infeasible Set (IIS) of:
• NP-hard in general.
• However, our problem has more structure than general IIS problems.
• Can we exploit the structure of the Boolean constraintsto help find the IIS of the convex program?
![Page 24: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/24.jpg)
Summary of UNSAT certificates
24
UNSAT Certificate Minimal
Complexity(number of convex
problems)
Trivial No Constant
IIS Yes Exponential
Sum of Slacks Yes* Linear
Minimum Prefix Yes* Constant
* under reasonable technical assumptions
Monotone SMC Formula
![Page 25: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/25.jpg)
Minimum Prefix Certificates
25
Example: switched linear/convex systems, motion planning
![Page 26: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/26.jpg)
Minimum Prefix Certificates
26
Example: switched linear/convex systems, motion planning
![Page 27: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/27.jpg)
Minimum Prefix Certificates
27
Example: switched linear/convex systems, motion planning
![Page 28: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/28.jpg)
Minimum Prefix Certificates
28
Example: switched linear/convex systems, motion planning
![Page 29: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/29.jpg)
Minimum Prefix Certificates
29
Key idea: Find the “shortest” UNSAT certificate (e.g., the shortest prefix witness of a safety property violation).
Example: switched linear/convex systems, motion planning
![Page 30: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/30.jpg)
Minimum Prefix Certificates
30
Key idea: Find the “shortest” UNSAT certificate (e.g., the shortest prefix witness of a safety property violation).
Example: switched linear/convex systems, motion planning
![Page 31: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/31.jpg)
Minimum Prefix Certificates
31
Definition: Positively Ordered Unate (POU) FunctionA function is said to be POU with respect to an ordering of its variables if for all values of bi, we have
Theorem:Consider a monotone SMC formula and its Boolean abstraction . If: (1) is positively ordered unate(2) the domain of the real variables is boundedthen minimal UNSAT certificates exist and can be computed in constant time.
![Page 32: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/32.jpg)
Theorem:Let s* be the solution of the above optimization problem. If k* is the minimal index such that |s*|> 0, then the certificate: is the minimum prefix certificate.
Minimum Prefix Certificates
32
![Page 33: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/33.jpg)
Summary of UNSAT certificates
33
UNSAT Certificate Minimal
Complexity(number of convex
problems)
Trivial No Constant
IIS Yes Exponential
Sum of Slacks Yes* Linear
Minimum Prefix Yes* Constant
* under reasonable technical assumptions
Monotone SMC Formula
![Page 34: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/34.jpg)
Scalability Results
34
Increase the number of Boolean constraints#Boolean variables = 4800#Real variables = 100
Increase the number of Real variables#Boolean variables = 4800#Boolean constraints = 7000
10000 x
http://yshoukry.bitbucket.io/SatEX
![Page 35: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/35.jpg)
Results(1): Single Robot, Reach-Avoid
10000 x
Syclop (Synergistic Combination of Layers Of Planning):- High level planner + low level RRT/EST- outperform traditional sampling-based algorithms by orders of magnitude
Z3 and MILP (LTL OPT) times outtime out =1 hour
SMC generated trajectories (blue) aresmoother than Syclop generated traje
![Page 36: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/36.jpg)
Result (2): LTL Motion Planning
![Page 37: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/37.jpg)
Result (3): Multi-Robot, LTL
![Page 38: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/38.jpg)
Result (4): Secure CPS
38
Under attack - no protection Under attack - with protection
![Page 39: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/39.jpg)
Specification & Verification of Industrial CPS via Formal Inductive Synthesis
S. A. Seshia 39
[Jin, Donze, Deshmukh, Seshia, HSCC 2013, TCAD 2015;Yamaguchi et al. FMCAD 2016;Jha & Seshia, Acta Inf. 2017]
Joint work with:Jyotirmoy Deshmukh, Alex Donze, Susmit Jha, Xiaoqing Jin,
Tomoyuki Kaga, Tomoya Yamaguchi
![Page 40: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/40.jpg)
Typical Closed‐Loop CPS Model/Verification
S. A. Seshia 40
Controller Plant
Environment
u
ego
adv
![Page 41: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/41.jpg)
Challenges for Verification of Closed‐Loop Automotive Control Systems
Closed‐loop setting very complex software + physical artifacts nonlinear dynamics large look‐up tables large amounts of switching
Requirements Incomplete/Informal Specifications often created concurrently with the design!
Designers often only have informal intuition about what is “good behavior” “shape recognition”
Mining Requirements from Closed‐Loop Models 41
Experimental Engine Control Model
![Page 42: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/42.jpg)
Industry Problem: Applying Formal Methods to Legacy Systems
It’s working, but I don’t understand why!
Our Solution: Requirement Mining
Value added by mining:
Mined Requirements become useful
documentation
Use for code maintenance and revision
Use during tuning and testing
Mining Requirements from Closed‐Loop Models 42
![Page 43: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/43.jpg)
Designer reviews mined requirements “Settling time is 6.25 ms” “Overshoot is 100 units” Expressed in Signal Temporal Logic
[Maler & Nickovic, ‘04]
Tool extracts properties of closed‐loop design using a Simulator
Designer’s View of Our Solution
6.25ms
100
Mining Requirements from Closed‐Loop Models 43
![Page 44: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/44.jpg)
CounterExample Guided Inductive Synthesis (CEGIS)
Find “Tightest” Properties
Settling Time is ??Overshoot is ??Upper Bound on x is ??
Are there behaviors that do NOT satisfy theserequirements?
Settling Time is 5 msOvershoot is 5 KPaUpper Bound on x is 3.6
1.
Mining Requirements from Closed‐Loop Models 44
Experimental Engine Control Model
[Jin, Donze, Deshmukh, Seshia, HSCC’13; TCAD’15]
Idea of CEGIS goes back to ASPLOS 2006 paperby Solar‐Lezama et al.
![Page 45: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/45.jpg)
Settling Time is 5.3 msOvershoot is 5.1 KPaUpper Bound on x is 3.8
Settling Time is … msOvershoot is … KPaUpper Bound on x is …
CounterExample Guided Inductive Synthesis (CEGIS)
Find “Tightest” Properties
Settling Time is ??Overshoot is ??Upper Bound on x is ??
Are there behaviors that do NOT satisfy theserequirements?
Counterexamples
1.
Mining Requirements from Closed‐Loop Models 45
Experimental Engine Control Model
![Page 46: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/46.jpg)
CounterExample Guided Inductive Synthesis
Find "Tightest" Properties
Settling Time is ??Overshoot is ??Upper Bound on x is ??
Are there behaviors that do NOT satisfy theserequirements?
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
NO
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
Mined Requirement
Counterexamples
1.
Mining Requirements from Closed‐Loop Models 46
Experimental Engine Control Model
![Page 47: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/47.jpg)
CounterExample Guided Inductive Synthesis
Find "Tightest" Properties
Settling Time is ??Overshoot is ??Upper Bound on x is ??
Are there behaviors that do NOT satisfy theserequirements?
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
NO
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
Mined Requirement
Counterexamples
1.
Mining Requirements from Closed‐Loop Models 47
Experimental Engine Control Model
Parametric Signal Temporal Logic
(PSTL)
Parameter Synthesis (exploits monotonicity)
Optimization‐based Falsification
![Page 48: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/48.jpg)
CounterExample Guided Inductive Synthesis
Find "Tightest" Properties
Settling Time is ??Overshoot is ??Upper Bound on x is ??
Are there behaviors that do NOT satisfy theserequirements?
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
NO
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
Mined Requirement
Counterexamples
1.
Mining Requirements from Closed‐Loop Models 48
Experimental Engine Control Model
Parametric Signal Temporal Logic
(PSTL)
Parameter Synthesis (exploits monotonicity)
Optimization‐based Falsification
![Page 49: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/49.jpg)
Signal Temporal Logic (STL)• Extension of Linear Temporal Logic (LTL) and Metric Temporal Logic (MTL)– Quantitative semantics: satisfaction of a property over a trace given real‐valued interpretation
– Greater value more easily satisfied – Non‐negative satisfaction value Boolean satisfaction
• Example: “For all time points between 60 and 100, the absolute value of x is below 0.1”
S. A. Seshia 49
0 100
1
-0.1 +0.1
60
x
t
[Maler & Nickovic, 2004]
![Page 50: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/50.jpg)
Quantitative Satisfaction Function for STL
• Function that maps STL formula and a given trace (valuation of signals) to a numeric value
• Example: is inf[60,100] (0.1 - |x|)
• Quantifies “how much” a trace satisfies a property– Large positive value: trace easily satisfies – Small positive value: trace close to violating – Negative value: trace does not satisfy
Mining Requirements from Closed‐Loop Models 50
![Page 51: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/51.jpg)
Parametric Signal Temporal Logic (PSTL)
• Constants in STL formula replaced with parameters– Scale parameters– Time parameters
• Examples:
S. A. Seshia 51
Between some time and 10 seconds, x remains greater than some value
After transmissionshifts to gear 2, itremains in gear 2 for at least secs
![Page 52: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/52.jpg)
CounterExample Guided Inductive Synthesis
Find "Tightest" Properties
Settling Time is ??Overshoot is ??Upper Bound on x is ??
Are there behaviors that do NOT satisfy theserequirements?
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
NO
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
Mined Requirement
Counterexamples
1.
Mining Requirements from Closed‐Loop Models 52
Experimental Engine Control Model
Parametric Signal Temporal Logic
(PSTL)
Parameter Synthesis (exploits monotonicity)
Optimization‐based Falsification
![Page 53: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/53.jpg)
Parameter Synthesis = Find ‐tight values of params (for suitably small )
0 100
3
2.9
1 000 000
Find "Tightest" Properties
Mining Requirements from Closed‐Loop Models 53
Too loose
Want the value of corresponding to the “tightest” satisfaction over a set of traces
x
![Page 54: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/54.jpg)
Parameter Synthesis
• Non‐linear optimization problem
– Satisfaction function for STL is non‐linear in general
• Naïve (“strawman”) approach:
– grid parameter space to precision
– evaluate satisfaction value at each point
– pick valuation with smallest satisfaction value
• Problem: Exponential number of grid points (in
#parameters)
Find "Tightest" Properties
Mining Requirements from Closed‐Loop Models 54
![Page 55: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/55.jpg)
If upper bound of all signals is 3, any number > 3 is also an upper bound
• Satisfaction function monotonic in parameter value• Example:
• (, x) = inft ( ‐ x(t) )• For all x, (, x) is a monotonic function of • Advantage: If monotonic, use binary search over parameter space, otherwise exhaustive search
Satisfaction Monotonicity
0 10050
34
Find "Tightest" Properties
Mining Requirements from Closed‐Loop Models 55
![Page 56: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/56.jpg)
• Need to decide whether:For all x, (, x) is a monotonic function of
• Theorem: Deciding monotonicity of a PSTL formula is undecidable
• Use an encoding to satisfiability modulo theories (SMT) solving – Quantified formulas involving uninterpreted functions, and arithmetic over reals linear arithmetic if PSTL predicates are linear
– Solved easily with Z3
Deciding Satisfaction MonotonicityFind "Tightest" Properties
Mining Requirements from Closed‐Loop Models 56
![Page 57: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/57.jpg)
CounterExample Guided Inductive Synthesis
Find "Tightest" Properties
Settling Time is ??Overshoot is ??Upper Bound on x is ??
Are there behaviors that do NOT satisfy theserequirements?
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
NO
Settling Time is 6.3 msOvershoot is 5.6 KPaUpper Bound on x is 4.1
Mined Requirement
Counterexamples
1.
Mining Requirements from Closed‐Loop Models 57
Experimental Engine Control Model
Parametric Signal Temporal Logic
(PSTL)
Parameter Synthesis (exploits monotonicity)
Optimization‐based Falsification
![Page 58: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/58.jpg)
Black‐Box Falsification Procedure
u S(u)
Falsification Tool
\
\
Mining Requirements from Closed‐Loop Models
Are there behaviors that do NOT satisfy theserequirements?
58
![Page 59: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/59.jpg)
Falsification as Optimization
• Solve * = minu (, S(u))– Leverages quantitative semantics of STL– Relies on standard numerical optimization methods (e.g.
Nelder‐Mead)
• If * < 0, found falsifying trace!
Nonlinear Optimization Problem, No exact solution, Limited theoretical guarantees
Are there behaviors that do NOT satisfy theserequirements?
Mining Requirements from Closed‐Loop Models 59
![Page 60: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/60.jpg)
Experimental Evaluation Summary [details in TCAD’15 paper]• Defined Templates for Common Requirements in Automotive Control – all monotonic PSTL!!– Dwell‐Time requirements– Timed/Untimed Safety properties– Timed Inevitability (bounded liveness)– Input Profiles: assumptions on shape of input signals– Control‐theoretic requirements on output signals (bounded overshoot/undershoot, settling time, error from reference signal, etc.)
• Three Benchmarks– Simple Simulink Automatic Transmission Model– Toyota HSCC’14 Challenge – Air‐Fuel Ratio controller– Toyota Experimental Diesel Engine Airpath controller
S. A. Seshia 60
![Page 61: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/61.jpg)
Results on Industrial Airpath Controller
• Found max overshoot with 7000+ simulations in 13 hours• Attempt to mine maximum observed settling time:
– stops after 4 iterations– gives answer tsettle = simulation time horizon (shown in trace below)
Experimental Engine Control Model
Mining Requirements from Closed‐Loop Models 61
[Jin, Donze, Deshmukh, Seshia, HSCC 2013]
![Page 62: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/62.jpg)
Mining can expose deep bugs
• Uncovered a tricky bug– Discussion with control designer revealed it to be a real bug
– Root cause identified as wrong value in a look‐up table, bug was fixed
• Why mining could be useful for bug‐finding:– Can uncover subtle relations that should not hold– Looking for bugs Mine for negation of bug
Experimental Engine Control Model
Mining Requirements from Closed‐Loop Models 62
![Page 63: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/63.jpg)
Bug fixed Settling time successfully mined
S. A. Seshia 63
OLD
NEW
![Page 64: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/64.jpg)
Industrial Case Studies with Toyota
• Work with group @ Toyota Japan on enabling software verification by mining specifications on the closed‐loop system
• Useful in a production setting:– Finds “issues” where previous methods fell short!– Reduced 70% of human effort
S. A. Seshia 64
[Yamaguchi et al., FMCAD’16]
![Page 65: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/65.jpg)
Toyota Unit’s Experience with Model Checking
65
Total Work hour
560min
1 trial
70min
40min
7 trials
Making modelMaking property
Revising property
Revising model
Executingmodel checking
Mappingcounterexample
Making/revising property: 110 minMapping counterexample: 280 min for just 1 module
[Yamaguchi et al., FMCAD’16]
![Page 66: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/66.jpg)
66
Overview of Methodology
in out
1. Pre-condition mining
Pre-condition forsoftware module
2. Softwaremodel checking
Module levelcounterexample
System levelcounterexample
3. Simulation-BasedVerification
Breach
Breach
SLDV/CBMC
[Yamaguchi et al., FMCAD’16]
![Page 67: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/67.jpg)
Requirement Mining In Toyota Case Study
67
▼ Founded false case
Violation area of post-condition
▼
▼▼
Find system level violationactuator output < 150
[Yamaguchi et al., FMCAD’16]
![Page 68: Formal Methods in the Real World: Specification ...sseshia/219c/spr...Kress-Gazit, and G. J. Pappas, “Temporal logic motion planning for dynamic robots,” Automatica 2009. ... •Initial](https://reader033.fdocuments.us/reader033/viewer/2022051920/600dad8ccfdef4070517e8c1/html5/thumbnails/68.jpg)
Summary: Specification & Verification of Industrial CPS
• Formal Specification often not available• Inductive Synthesis from Models and Data can generate useful specifications and find bugs– Counterexample‐Guided Inductive Synthesis (CEGIS)
• Simulation‐based Temporal Logic Falsification is a scalable, industrially‐applicable method
• Requirement Mining + Falsification can make Software Verification tools more effective
S. A. Seshia 68