1 Verification in the Model-Based Design Flow Bruce H. Krogh CMACS Review March 4, 2010 TexPoint fonts use in EMF. Read the TexPoint manual before you delete this box.: AA A AAAAA A A AA A A AA 2 Overview VVIACS Model reduction Heterogeneous verification Verification of numerical code Current directions Vincent W. Crum Air Vehicles Directorate Air Force Research Laboratory | AFRL Cover 4 FCS Design flow (LM VVIACS Project) Model-based Environments Formal Specification Techniques Advanced V&V-aware Designs Control Analysis Software Implementation Formal V&V Automated Test Process-Based Certification 5 VMS: System Requirement s Hardware Requirements Software Requirements Other Subsystems Requirements Control Law Requirements Hardware Design & Analysis Other Subsystem Design Control Law Design & Analysis Software Design & Analysis Software Unit and Component Testing Hardware & Software Integration Testing Subsystem Unit and Component Testing System Level Testing System Level Certification Stability and Control Simulation Test Tool Development Hardware Qual & Acceptance Testing flows & blocks modified from current process Current Process 6 FCS Verification via Testing 7 FCS Cost of Testing Future Military Program Testing Hours Are Forecast to Triple 8 FCS Possible applications of HSV 9 VVIACS - Impact Analysis Results Significant Cost/Schedule Increase Projected Due to Complexity Single-Vehicle ECS Increases Development Costs ~ 50%, V&V Costs ~ 100%, and Critical Path Length ~ 50% Multiple-Vehicle ECS Increases Development Costs ~ 100%, V&V Costs ~ 150%, and Critical Path Length ~ 125% Software: Single-Vehicle 100% Increase and Multiple-Vehicle 200% Increase in V&V Costs Test: Single-Vehicle 150% Increase and Multiple-Vehicle 250% Increase in V&V Costs 10 Hardware Design & Analysis Other Subsystem Design Algorithm Design, Analysis and Functional Test Software Analysis, Design, Integration Subsystem Unit and Component Testing Integration & System Level Testing System Level Certification Stability and Control Simulation Test Tool Development Hardware Qual & Acceptance Testing Rapid Proto Type Hardware and Test Environmen t Hardware Requirements Software Requirements Other Subsystems Requirements Algorithm Requirements Vehicle: System Requirement s Formal Req/Spec V&V Aware Far-Term (7-9 Yrs) Process 11 Some Goals Guarantee correct behavior of the complete system especially with respect to timing constraints in the implementation performance specifications associated with mission-level objectives Develop a comprehensive approach integrating verification, validation and test procedures throughout the complete development cycle, from requirements capture to deployment. Achieve confidence levels that exceed those achievable by current practice and current technologies for systems that incorporate emerging adaptive and intelligent control laws. 12 Model Reduction for Scalability of Hybrid System Verification Use simplified models and/or set representations to perform the reach set computations high-order model weakly-coupled model nonlinear model reduced-order model -coupled subsystems trajectory piecewise -linear model Model Model Order Reduction decomposition Piecewise Linearization Set representation Full-dimensional polytopes Low-dimensional polytopes and their neighborhoods 13 Heterogeneous Verification Motivation: verifying properties of complete systems is beyond the reach of any one tool or modeling formalism Objective: reason about verification information collected from multiple sources to achieve system-level verification Verif icatio n repor t.. Existing docs Hybrid Analysis Discrete Analysis Simulation System Unstructured, semi- structured and structured information from various analyses on components /models. 14 Heterogeneous Verification ontology specialization knowledge assimilation knowledge composition, deduction specialized ontology inferences + knowledge gaps queries information embedded system ontology (base domain description) knowledge base (Protg) database + epistemic rule base update docs hybrid analysis simulation heterogeneous information sources discrete analysis targeted knowledge acquisition static ontology + epistemic ontology entities, relationships, rules domain experts verification manager requirements developers model development & verification activities inference engine Managing knowledge for model-based development 15 Verification of numerical programs Problem definition Polyhedral domains Control flow automata (CFA) CFA reachability Widening based on coefficient limiting CFA reduction Kahan summation example Conclusions 16 Design and implementation of numerical programs design model code generation target processor compiler model-based development source code implementation platform implementation 17 Verification of numerical programs design model code generation target processor compiler Need to verify how numerical code will execute on the target processor 18 design model code generation target processor compiler disassembler CFA generator control flow graph target processor error model PHAVer reachability results CFA numerical program verifier Verification of numerical programs 19 design model code generation target processor compiler disassembler CFA generator control flow graph target processor error model PHAVer error bounds CFA todays presentation Verification of numerical programs reachable sets 20 Scope of this work instructions of the form real constants and variables linear arithmetic floating point error bounds 21 Polyhedral domains linear predicates convex polyhedron: conjunction of linear predicates polyhedron: disjunction of convex polyhedra Parma Polyhedra Library (PPL): performs exact computations with non-convex polyhedra PHAVer: performs reachability for LHA exact and robust arithmetic with unlimited precision (PPL) bit-constrained over-approximations for termination heuristics on-the-fly over-approximation of piecewise affine dynamics support for compositional and assume-guarantee reasoning. 22 Exact Arithmetic in PHAVer Finite resources require over approximation Semi-bounded exact arithmetic exact computations that result in finite precision * Managing the complexity by over-approximation 1.generate time- elapse polyhedron initial set 2.* compute conservative over- approximation derivative 109 x 121 y y x0 6 x 6 y 6 1.limit the number of bits of coefficients 2.limit the number of constraints 23 Control flow automata (CFA) same node/transition structure as the control flow graph instructions replaced by action predicates on the transitions representing the operation error bounds 24 CFA example 25 CFA reachability CFA state: (q,x) q discrete state, x valuation of variables x Reachable states: smallest fixed point of where All sets are polyhedra In general, the reachability iteration will not terminate 26 Widening for iterative computations Accelerates convergence to a fixed point. P. Cousot and R. Cousot. Abstract interpretation: A unified lattice model for static analysis of programs by construction or approximation of fixpoints. In Proceedings of the Fourth Annual ACM Symposium on Principles of Programming Languages, pp , New York, ACM Press. 27 Standard widening defined for convex polyhedra retains constraints of P 1 also satisfied by P 2 and constraints of P 2 with equivalent constraints in P 1 28 Standard widening: example exact (nonterimintating) std. widening terminates at iteration 5 large over approximation 29 Widening based on coefficient limiting (NEW) Preliminaries: C P : set of linear predicates defining polyhedron P assume integer coefficients with common divisor 1 max_coeff(C P ) : maximum coefficient in C P coeff_limit(P,k) : polyhedron P such that 1) P P 2) max_coeff(C P ) k NOTE: Such a P is computed by PHAVer 30 Widening based on coefficient limiting (NEW) Proposition 1. is a widening operator. follows from 31 Example 1: Application to Program 1 std. widening coefficient-limiting widening 32 Example 2: Non-convex polyhedra (w/o convex hull) 33 CFA Reduction Objective: Given a set of variables W, reduce the number of transitions and variables in the CFA without affecting the reachable set for the variables in W. 34 Merging transitions Transition condition for merging applied only to the first transition in a pair of transitions 35 Eliminating irrelevant variables transition merging increases the number of globally irrelevant variables retains variables that influence error bounds on variables of interest 36 Precision vs. Efficiency using Value of k introduces a tradeoff between - accuracy of polyhedral approximations and - complexity of the computations Smaller k increases the over approximation but doesnt necessarily make termination faster 37 Example: Kahan summation algorithm adding N numbers: error = N Kahan algorithm introduces intermediate variables to mitigate the effects of repeated summations: error = 2 +O(N 2 ) From Wikipedia: function kahanSum(input, n) var sum = input[1] var c = 0.0 // A running compensation for lost low-order bits. for i = 2 to n y = input[i] - c // So far, so good: c is zero. t = sum + y // Alas, sum is big, y small, so low-order digits of y are lost. c = (t - sum) - y //(t - sum) recovers the high-order part of y; // subtracting y recovers -(low part of y) sum = t // Algebraically, c should always be zero. // Beware eagerly optimising compilers! next i // Next time around, the lost low part will // be added to y in a fresh attempt. return sum 38 Our implementation = e-7 (i386) x 0 = 1 x i = N = 8 reduced: 3 locations, 3 transitions, vars y,t eliminated 39 Current Directions Integration through architecture Assume-guarantee approach to controller-plant decomposition Innovative uses of reachability