Models for Control and Verification• Optimal control works with a single optimal trajectory •...

1

Models for Control and Verification

Ian MitchellDepartment of Computer ScienceThe University of British Columbia

research supported byNational Science and Engineering Research Council of Canada

January 2009 Ian Mitchell (UBC Computer Science) 2

Outline• Classes of models

– Well-posed models

– Difference Equations

– Nonlinear Ordinary Differential Equations

– Syntax vs semantics– Visualization

• Optimal Control– Objective and value functions

• Verification: Reachability– Forward & backward, tubes & sets, maximal & minimal


Control & Verification Require Modeling• Dynamic systems change with time

• We wish to reason about that change– Control: We seek to guide the evolution to achieve a desired

objective

– Verification: We seek to confirm the evolution will achieve a desired objective

• Control and verification require prediction of future evolution– Prediction is achieved by mathematical models

• System is described by state and time


Discrete vs Continuous• Discrete variable

– Drawn from a countable domain, typically finite

– Often no useful metric other than the discrete metric

– Often no consistent ordering

– Examples: names of students in this room, rooms in this building, natural numbers, grid of Rd, …

• Continuous variable– Drawn from an uncountable domain, but may be bounded– Usually has a continuous metric

– Often no consistent ordering– Examples: Real numbers [ 0, 1 ], Rd, SO(3), …

2


Classes of Models for Dynamic Systems• Discrete time and state

• Continuous time / discrete state– Discrete event systems

• Discrete time / continuous state• Continuous time and state

• Markovian assumption– All information relevant to future

evolution is captured in the state variable

• Deterministic assumption– Future evolution completely

determined by initial conditions

• Not the only classes of models


Well-Posed Models• Mathematical models may not behave nicely

– May describe impossible evolutions

– May not be easy to apply formal reasoning

• We want to forbid such (eg ignore) models

• Common desirable traits– There exists a solution for all (or some) time

– The solution is unique

– The solution depends continuously on the data (initial conditions, dynamics)


Difference Equations

• Existence: for all tttt and xxxx, ffff(tttt, xxxx) ≠ ∅• Uniqueness: for all tttt and xxxx, | ffff(tttt, xxxx) | = 1• Continuous dependence on the data: for all tttt, xxxx, yyyy

there exists constant λ such that

– Only makes sense if state space has a continuous metric

– Sufficient but not necessary– Might also want to handle mistakes in ffff


Lipschitz Continuity

• Called “Lipschitz continuity” with respect to xxxx (or yyyy)

• Constant λ is the “Lipschitz constant”

• Relationship with continuity and differentiability?

Continuity

Differentiability

Differentiabilitywith bounded derivative Lipschitz continuity

Lipschitz continuity

Lipschitz continuity

norelation

3


Lipschitz Continuous Functions

• Which of these functions is Lipschitz continuous?

A

D

B

C


Ordinary Differential Equations (ODEs)

• What about second order ODE?– Newton’s second law: force = (mass)(acceleration)

• Need to reformulate into first order form– Define new variable zzzz(tttt) ∈ R2*dddd

• May also be useful to remove dependence on tttt– Define new variable y(t) ∈ Rdddd+1

– Called “autonomous system” in mathematics


Well-Posed ODEs

• Consider initial value problem (IVP): xxxx(ttttiiii) = xxxxiiii• If ffff is Lipschitz continuous in xxxx for all tttt ∈ [ ttttiiii, ttttffff ]

– There exists a unique solution xxxx(tttt) for tttt ∈ [ ttttiiii, ttttffff ] for each xxxxiiiisuch that dxdxdxdx/dtdtdtdt exists and dxdxdxdx/dtdtdtdt = ffff(tttt,xxxx)

– For perturbed initial data yyyyiiii yielding yyyy(tttt)

– For perturbed dynamics

• Sufficient but not necessary conditions


Ill-Posed ODEs• Why do we care that the ODE is well-posed?

– Theory: much depends on the existence of a unique solution

– Numerics: approximate solution may not be desired solution, and may not even be near a true solution

4


Hybrid Systems• Some systems are not purely continuous or discrete

– Computer control of continuous systems– Vastly varying timescales for continuous systems, in which only the

equilibrium value of the faster timescale components matters– Simplify some components of a system by discretizing a continuous

state or taking continuum limit of discrete state

seven mode collision avoidance protocol


Hybrid Automata• Discrete modes and transitions• Continuous evolution within each mode

• No simple conditions for well-posed HA


Checking a Model• Well-posed conditions are examples of static checks:

tests applied directly to the model– Model does not itself evolve, but is a static entity

– Complexity of check depends only on the complexity of the model

• Alternative: Dynamic checks– Requires understanding the evolving solution

– Complexity of check depends on the complexity of the solution trajectory


Restricted Classes of Model• Many results in control and verification assume a

restricted class of models– Permits more checks to be static

– May simplify checks of dynamic

– Example: FA and ODEs both have easy static checks for well-posedness; hybrid systems do not

• Our nonlinear ODE and DI models are very general– Most of what we discuss (beyond well-posedness) will be

semantic / dynamic checks

5


Visualization• Most of the visualization of

system evolution will be done in the phase or state space (ignore time)– Pendulum states angle and

angular velocity

phas

e S

pace

stat

e vs

time

pend

ulum

wor

kspa

ce









Dimitri Bertsekas, Dynamic Programming & Optimal Control,

Athena Scientific (3rd edition 2005)


Achieving Desired Behaviours• We can attempt to control a system when there is a parameter uuuu

of the dynamics (the “control input”) which we can influence

– Time dependent dynamics are possible, but we will mostly deal with time invariant systems

• Without a control signal specification, system is nondeterministic– Current state cannot predict unique future evolution

• Control signal may be specified– Open-loop uuuu(tttt) or uuuu: R →

UUUU

– Feedback, closed-loop uuuu(xxxx(tttt)) or uuuu: S →UUUU

– Either choice makes the system deterministic again


Visualization: Vector Fields• Introduction of a free control input changes the vector field plot

in the phase space into a field of cones (nondeterministic)

• Feedback control law changes it back into a (static) vector field

• Open loop control law does not

no inputs(“autonomous” for control engineers)

unspecifiedinput signal

feedbackinput signal

6


Objective Function• We distinguish quality of control by an objective / payoff / cost

function, which comes in many different variations– eg: discrete time discounted with fixed finite horizon ttttffff

– eg: continuous time no discount with target set TTTT


Value Function• Choose input signal to optimize the objective

– Optimize: “cost” is usually minimized, “payoff” is usually maximized and “objective” may be either

• Value function is the optimal value of the objective function– May not be achieved for any signal (eg: min should be inf)

• Set of signals U is contentious– For implementation purposes, we desire restricted classes:

bounded, continuous, piecewise constant– Unfortunately, theory applies to (and thus can only guarantee

optimality with) very general classes: measurable


Example: LQR for Linear Systems• Much of the “optimal control” literature and most classes focus

(without mentioning it) on linear systems

• Corresponding objective functions are usually quadratic

where AAAA, BBBB, QQQQ, RRRR, QQQQffff are all matrices of appropriate size

• Successful but restricted class of problems– Not rigorously part of the results to follow (due to a technicality)









Ian Mitchell, “Comparing Forward and Backward Reachability

as Tools for Safety Analysis,”Hybrid Systems Computation and Control,

LNCS 4416, Springer-Verlag (2007).

7


Verification: Safety Analysis• Does there exist a trajectory of system H leading from

a state in initial set IIII to a state in terminal set TTTT ? (under some policy for input uuuu(·))


Typical Systems: ODEs• Common model for continuous state spaces

• Standard existence and uniqueness


Typical Systems: Hybrid Automata• Adapted from [Gao, Lygeros & Quincampoix 2006]

• Challenging existence and uniqueness– eg: [Broucke & Arapostathis, Sys. & Con. Letters 2002] or

[Lygeros, Johansson, Simic, Zhang & Sastry, TAC 2003]

– requires at least non-Zeno and non-blocking– all non-determinism must be expressed through input uuuu(·)


Working with Sets• Optimal control works with a single optimal trajectory

• Verification works with sets of trajectories– Takes a nondeterministic (but not probabilistic) viewpoint

• Basic construct is reachability– Many versions: forward and backward, sets or tubes– What should the input do?

• Many related concepts in control theory– Invariant sets, controlled invariant sets, stability

• Safety is not the only verification goal– Liveness is a common goal, but often harder to verify

8


Forward Reachability• Start at initial conditions and compute forward


Backward Reachability• Start at terminal set and compute backwards


Exchanging Algorithms• Algorithms are (mathematically) interchangeable if

system dynamics can be reversed in time

• For example:• Then


Maximal Reachability• Input signal uuuu(·) maximizes size of the set or tube

9


Maximal Reachability Definition


Maximal Reachability Results• Reach sets and tubes provide similar information

• The following properties are equivalent

• Any maximal reachability operator can be used to demonstrate safety for all possible inputs


Maximal Reachability Demonstration

System Dynamics

Forward Reach Set Results

Initial and Terminal Sets



System Dynamics

Forward Reach Tube Results


10



System Dynamics

Backward Reach Set Results




System Dynamics

Backward Reach Tube Results



Minimal Reachability• Input signal uuuu(·) minimizes size of the set or tube


Minimal Reachability Definition

11


Minimal Reachability Results• Reach tubes provide more information

– Choice of trajectory length tttt is quantified first for sets but last for tubes


Minimal Reachability Results• Backward reach tubes are the only minimal

reachability operator that can prove that there exists an input uuuu(·) which keeps the system safe

– Basic problem with minimal forward reachability: the state lying in the terminal set is chosen before the input, while the state lying in the initial set is chosen after


Minimal Reachability Demonstration

System Dynamics

(Correct) Backward Reach Tube Results



Minimal Reachability Demonstration

System Dynamics

(Incorrect) Forward Reach Tube Results


Models for Control and Verification• Optimal control works with a single optimal trajectory •...

Documents

Transcript of Models for Control and Verification• Optimal control works with a single optimal trajectory •...