Complex Differential Games of Pursuit-Evasion Type with ... · Since this time, applications of...

JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: VoL 78, No, 3, SEPTEMBER 1993

Complex Differential Games of Pursuit-Evasion Type with State Constraints, Part 1: Necessary Conditions

for Optimal Open-Loop Strategies 1' 2

M . H . BREITNER, 3 H . J . PESCH, 4 AND W.. GRIMM 5

Communicated by R. Bulirsch

Abstract. Complex pursuit-evasion games with state variable inequality constraints are investigated. Necessary conditions of the first and the second order for optimal trajectories are developed, which enable the calculation of optimal open-loop strategies. The necessary conditions on singular surfaces induced by state constraints and nonsmooth data are discussed in detail. These conditions lead to multipoint boundary-value problems which can be solved very efficiently and very accurately by the multiple shooting method. A realistically modelled pursuit-evasion problem for one air-to-air missile versus one high performance aircraft in a vertical plane serves as an example. For this pursuit-evasion game, the barrier surface is investigated, which determines the firing range of the missile. The numerical method for solving this problem and extensive numerical results will be presented and discussed in Part 2 of this paper; see Ref. 1.

Key Words. Differential games, pursuit-evasion games, singular surfaces, barrier surface, state constraints, open-loop strategies, muttipoint boundary-value problems, flight mechanics, missile firing range,

1This paper is dedicated to the memory of Professor John V. Breakwell. 2The authors would like to express their sincere and gratefuI appreciation to Professors R. Bulirsch and K. H. Well for their encouraging interest in this work.

3Assistant Professor, Department of Mathematics, University of Technology, Munich, Germany.

4privatdozent ffir Mathematik, Department of Mathematics, University of Technology, Munich, Germany.

5Assistant Professor, Department of Flight Mechanics and Control, University of Stuttgart, Stuttgart, Germany.

419

0022-3239/93/0900~041950%00/0 © 1993 Plenum Publishing Corporation

420 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

1. Introduction

Differential game theory was initiated by the pioneering work of Rufus Isaacs (Refs. 2-6) in the early fifties and has found wide interest since the publication of his book in 1965 (Ref. 7). Since this time, applications of differential game theory have influenced such contrasting fields as warfare and collision avoidance, economy, and ecology. Recently, optimal control against unknown disturbances turned out to be another very important field for applications of differential games.

This paper is concerned with so-called two-person pursuit-evasion games. One player of the game, the pursuer P, tries to force the state toward a terminal manifold against any action of the opposite player, the evader E, who exactly tries to avoid this. For a wide class of pursuit- evasion problems, a natural approach to value the outcome of this conflict is to take the capture time as payoff. If the player P can enforce capture at all, P wants to minimize the capture time, whereas the player E wants to maximize it. In terms of differential game theory, the paper deals with two-person zero-sum differential games with variable terminal time and complete information.

As is known from the literature, a comprehensive analysis of pursuit- evasion games can only be carried through for some problems with rather simple dynamics. In particular, this is caused by two severe obstacles in solving such problems. First, in contrast to optimal control problems, the computation of optimal feedback strategies is required, since open-loop controls need not be optimal in pursuit-evasion games. In general, these feedback strategies cannot be obtained in closed form for realistically modelled problems. The second problem occurring in the solution of differential games is associated with singular surfaces of various types. These surfaces divide the state space into different parts, in which the feedback strategies must be calculated separately. A good example showing the difficulties mentioned is the classical homicidal chauffeur game; see Refs. 8 and 9.

A wide gap can be seen comparing the complexity of solved pursuit- evasion game problems with the complexity of optimal control problems which today can be solved by sophisticated numerical methods. Because of the close relationship between differential game theory and optimal control theory, it is obvious to try to transfer methods for the solution of optimal control problems to the solution of pursuit-evasion games. An approved and reliable method for the solution of optimal control problems is the multiple shooting method; see, e.g., Refs. 10 and 11. This method solves the multipoint boundary-value problems ~rising from the necessary conditions of optimality.

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 42l

In the present paper, necessary conditions of optimality of the first and the second order are developed for a complex pursuit-evasion game with a state-variable inequality constraint. In particular, the singular surfaces induced by the state constraint and by nonsmooth data are investigated in detail. These necessary conditions give rise to multipoint boundary-value problems similar to those based on the necessary conditions for optimal control problems. By this approach, optimal trajectories within the capture zone and the barrier can be computed numerically. The respective optimal open-loop strategies provide an open-loop representation of the optimal feedback strategies.

A special air-combat scenario serves as a real-life example, in which a medium-range air-to-air missile pursues a high-performance aircraft in a vertical plane. The state-variable inequality constraint is introduced by a dynamic pressure limit for the aircraft. Special emphasis is placed on realistic approximations of the lift, drag, and thrust of both vehicles and the atmospheric data.

The numerical method for solving complex pursuit-evasion games of that type and extensive numerical results for the aforementioned problem will be described and discussed in Part 2 of this paper; see Ref. 1.

Both parts of the paper are based on Ref. 12.

2. Mathematical Model

For motivation purposes and in order to provide the subsequent theory with an illustrating example, we first describe the complete mathematical model of the two players involved in the pursuit-evasion game. The player P is represented by a missile, the player E by an aircraft.

The equations of motion for such flight vehicles are well known from the literature; see, e.g., Ref. 13. Here, the following simplifying assumptions are made: the flight is assumed to take place in a vertical plane; the air- plane reference line is assumed to be parallel to the velocity vector with the thrust also parallel to the velocity vector and the drag opposite to the thrust; the lift is assumed to be orthogonal to the velocity vector and point- ing upward; a flat Earth and a constant gravitational acceleration conclude these simplifications. Then, the equations of motion for both the evading aircraft and the pursuing missile can be written as follows:

= v COS 7,

= v sin 7,

i~ = g ( q T m ~ x ( t , h , v ) - D ( t , h, v, n ) ) / W ( t ) - g sin 7,

~) = ( g / v ) ( n - - cos 7).

(la)

(lb)

(~c)

(ld)


Here, x, h, v, ~ denote the horizontal position, the altitude, the velocity, and the flight path angle, respectively. The thrust T := r/Tm~x is controlled by the power setting q, and ;~ by the load factor n. As mentioned above, the gravitational acceleration g is taken to be constant.

For the weight W, lift L, and drag D, we assume that there hold

W ( t ) = m ( t ) g ,

L( t, n) =nW( t ) = CL( t, h, v, n) Sq(h, v),

D(t, h, v, n) = C o ( t , h, v, n) Sq(h, v),

(2a)

(2b)

(2c)

where q, C L, CD denote the dynamic pressure, lift coefficient, and drag coefficient, respectively. The reference wing area is denoted by S. Further- more, we assume a quadratic polar, i.e.,

Co(t, h, v, n) = Coo(h, v) + CL(t, h, v, n) 2 Cm(h, v), (3)

with zero-lift drag coefficient Coo and induced drag coefficient Cm. Then, the drag is described by

D(t, h, v, n)-=Do(h, v) + n2Dt(t, h, v),

where

Do(h, v) = Coo(h, v) Sq(h, v),

Dx(t, h, v) = Cm(h, v) W(t)2/(Sq(h, v)).

(4a)

(4b)

(4c)

Here, D O and D~ are the zero-lift drag and induced drag. The dynamic pressure q is defined by

q(h,v)=Q(h)v2/2, (5)

where 0 denotes the air density. According to Ref. 14, the following analytic approximation of the air density is used:

Q(h)=exp( r r+rsh+exp( r4ha+r3h3+r2h2+r lh+ro) ) . (6)

Since the drag coefficients depend on the Mach number M,

M(h, v) = via(h), (7)

an analytic expression of the speed of sound a(h) must also be provided,

a(h) = ~ a 3 h3 + a2h 2 + alh + ao. (8)

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 423

The approximation of the speed of sound is also taken from Ref. 14. The relative error of the air density compared to the US standard atmosphere of 1976 varies between - 6 % and +2%. The relative error for the speed of sound used here compared to the speed of sound of the standard atmosphere varies between - 1% and + 6 %. The above analytic approximations are more convenient for the later numerical treatment.

Next, the approximations of the aerodynamic drag coefficients and of the thrust based on realistic data for a F15E-Strike Eagle fighter aircraft and for an advanced medium-range air-to-air missile of the type AMRAAM AIM-120A are summarized. To avoid confusion between the evading aircraft and the pursuing missile, subscripts E and P are thoroughly used in the following. In the aircraft model, CooE, Cme, and TEma~ are analytic functions carefully adapted to realistic tabular data; see Ref. 15. The following model functions are used:

b4M4 + b3 M3 + bzM~ + blME+ bo _ , ( 9 a )

CDOe= - 4 +~3M3 +~2M~+[~lMe+bo baME

c4M + c3M . + c2M + + Co _ , ( 9 b )

CD~e = - s +?,,Mae+O3M3E+?2MZe+i~ME+C ° c s M e

5 5

Te=rteTemax :=r/e E E d~jh~M~. (9c) i=0 j = 0

The denominator of the rational approximation of CDOE has no real zeros, and the only real zero of the denominator of the approximation of Co. w has no influence for Mach numbers M < 5.491.

In a realistic aircraft model, a dynamic pressure constraint has to be taken into account,

Q(he, re) := qe(he, re) - qEmax ~< 0, (10)

where qEma~ denotes the dynamic pressure boundary of the evader. This state variable inequality constraint will be discussed in more detail in the differential game approach.

Since we only consider short flight times, fuel consumption is neglec- ted. Thus, W E is constant, and CLE, C~e, and De do not depend explicitly on t. Therefore, we omit the argument t in these variables.

In the following, conditions are considered to characterize the set of all meaningful initial conditions for the altitude and the velocity of the aircraft. This set is bounded by the evader's flight envelope. The flight envelope is determined by the minimum and maximum altitude (hEroin and hemax ) and by the dynamic pressure constraint (10). In addition, there are conditions

424 JOTA: VOL 78, NO. 3, SEPTEMBER 1993

related to the capability of horizontal flight with constant velocity, i.e., /~E=0, bE=O, implying n~:= 1 and t/e= 1. To satisfy /~E=0, the lift coefficient is to be bounded by

C 1 LE(hE, rE) := Cz, e(he, rE, ne = 1)4 CLEmax(hE, rE). (11)

An approximation for CLEma x c a n be also found in Ref. 15. This condition indicates that there exists a minimum velocity for each altitude to produce enough lift for a horizontal flight. To satisfy additionally be=0, an additional inequality has to be taken into account,

Te(hE, v e ) - De(he, re, ne = 1)/>0. (12)

Clearly, the velocity cannot be held constant in a horizontal flight if the drag DE is greater than the available thrust Te. These two inequalities complete the description of the evader's flight envelope; see Fig. 1. All initial conditions for the aircraft should be chosen within this flight envelope,

Instead of specifying all constants of the model (cf. Ref. 12), we show only the difference between maximum thrust with afterburner and drag for the horizontal flight in the domain of the flight envelope of the F15E-Strike Eagle; see Fig. 2. This figure describes the acceleration capacity of the aircraft.

The generic model of the AMRAAM AIM-120A missile is now given. For the missile, the subscript P is used. The maximum thrust is assumed

20.

15

10,

S 5

0 ......... / ,

hE < h:m~ ° v~ (m/s)

h E > hE max TE ~ D 0

_ _ / O,~¢

75o

Fig, I. Flight envelope of the aircraft,

JOTA: VOL. 78, NO. 3, SEPTEMBER t993 425

Fig. 2. Thrust excess for the aircraft.

to be a decreasing step function of time describing the boost, march, and coasting phases,

t Tb, 0~<t<3 ,

Temax(t ) = T,,,, 3 ~< t < 15,

(0 , 15~<t.

(13)

In accordance with the thrust profile, the mass is a continuous piecewise linearly decreasing function of time, which remains constant in the coasting phase. Thus, the system (1) depends explicitly on the time.

The drag coefficients are modelled as follows:

CDOl. = e 3 l n ( 1 + e x p ( ( e 2 / e 3 ) ( 3 - - M p ) ) ) + el,

C D I P ~ CI "

(14a)

(14b)

For the precise model data, see also Ref. 12. In order to facilitate a later synthesis of the open-loop strategies for

the construction of feedback strategies, a model reduction in the sense of singular perturbations is made. Under the assumption that I3~1 is small, the equations of motion (1) for both vehicles can be reduced, without significant toss of accuracy, to

2 = v cos ~, (15a)

/t = v sin 7, (15b)

= (t/Tmax(t, h, v) - Do(h, v) - cos 2 7Dz(t , h, v ) ) /m - g sin ~. (15c)

Herewith, the dimension of the state space is reduced to 6, and 7E and 7e now play the roles of control functions. It should already be mentioned


here that the above assumption was always satisfied for the numerical solutions obtained. In addition, one also has to pay attention that the optimal flight path angles must be verified to be continuous for physical reasons, which in general is not quaranteed by the theory of differential games. Moreover for the aircraft, it is assumed that maximum thrust is optimal (i.e., t/E= 1). This also has to be verified a posteriori. For the missile, maximum thrust (i.e., t /e= 1) must be chosen since the rocket boosters are not controllable.

3. Differential Game

In modern air-combat scenarios, fighter aircraft generally attack hostile aircraft with medium-range air-to-air missiles. Therefore, it is important to estimate the firing range of these missiles and to gain an insight into the best possible quidance laws. Because aircraft and missiles will be guided in the future with improved guidance schemes using high- performance onboard computers, this will bring a bipartite optimality aspect into the setting. A subproblem of the above scenario is the encounter of one aircraft versus one missile. A standard situation for such an encounter is the so-called follow-up shot, where the velocity vectors of the two vehicles are i na common vertical plane and enclose an angle less than 90 ° with the positive x-axis. This situation serves as an example in the present paper.

The classical pursuit-evasion concept of Isaacs (Ref. 7, see also Refs. 16 and 17) provides the suitable mathematical framework for this scenario. In terms of differential game theory, the above pursuit-evasion problem is a two-person zero-sum differential game with immediate and complete information of the actual state and the model data of the vehicles E and P. Information about the actual and future controls of the opposite player is not available. Both players are assumed to be able to choose their control functions instantaneously and independently from each other.

With the models of Section 2, the dynamics of the two vehicles are described by the following system of ordinary differential equations:

3~=[ )~e] =f(t, y, ~;p, YE) [fe(t, y~, 7e)~, (16) LYEJ := L fE(Ye 7e) J

where Ye := (xp, he, VF) T and YE := (xe, he, re) T denote the state variables, and ;~e and 7e denote the control variables of the two players. The time t is the time elapsed since the start of the game. The right-hand s ide f of (16) is given by (15).

J O T A : VOL. 78, NO. 3, S E P T E M B E R 1993 427

The motion of the player E is subject to the state constraint Q ( y e ) <<. 0; see Eq. (10).

The game begins with the initial conditions

y ( 0 ) = Yo (17)

at time t = 0 and terminates as soon as the capture condition,

F(y(•)) : = ( x . ( 9 ) - xp(tj)) 2 + (h~(t s) - h p ( t ~ ) ) 2 - d 2 = 0, ( 1 8 )

is fulfilled. This determines the capture time ty. The constant d denotes the capture radius. If capture is not possible in finite time, we define t f := o0.

The objective function of the game is defined by

J(7P, VE; t = 0, y = Yo) := tf. ( 191

The pursuer P tries to drive the state y from the initial state Yo to the terminal manifold (18) in minimum time. The evader E tries to avoid capture or, if escape is impossible, E tries to maximize capture time,

max min J - max rain tf, 7E ?P 7E ~'P

for all 7p, 7e~ C°[0, oe[, (20)

where C°[0, oe[ denotes the set of all piecewise continuous functions defined on the interval [0, oe [. In Eq. (20), it is tacitly assumed that the operators min and max commute, i.e.,

max rain J = rain max J. (2t) ~/E YP ~P ~E

4. Necessary Conditions

Because of technical reasons, it is obvious that the pursuit-evasion game of Section 3 possesses both a capture zone for the pursuer and an escape zone for the evader. The time-dependent separating hypermanifolds between the states of the capture zone and those of the escape zone form the barrier. In the following, necessary conditions of the first and the second order are developed to characterize in the sense of Eq. (20) optimal trajectories within the capture zone of P. Note that the capture zone includes the barrier. It should be mentioned that the formulation of Section 3 can be applied only to initial conditions (17) where Yo lies in the capture zone of P at time t = 0. Then, optimal trajectories y( t ) starting in Y0 will always stay within the capture zone because of the semipermeability of the barrier; see Ref. 7, p. 204. Outside this region, optimal trajectories cannot


be computed by the approach of Section 3. Another objective function has to be introduced in order to compute optimal trajectories within the escape zone.

4.1. Open-Loop Representation of Optimal Feedback Strategies. For pursuit-evasion problems, only admissible feedback strategies,

yp(t) = Fe(t, y(t)), 7E(t) = Fe(t, y(t)), (22)

have to be considered, where Fp and FE are assumed to be piecewise continuous in t and y. A pair of feedback strategies is called admissible if and only if the state y(t) is driven from any point y of the capture zone at any time t to the terminal manifold (18)via (16) and (22) while obeying the state constraint (10).

An admissible pair of feedback strategies (F*, F*) is said to be in saddle-point equilibrium if

J(F*, Fe; t, y)~< J(F*, F~; t, y) ~< J(Fp, F*; t, y), (23)

for all feedback strategies Fp and F~. such that (F*, FE) and (Fe, F*) are admissible, and for all t ~ I-0, o0 E and all y of the capture zone. This condition can be used only for some rather simple problems to verify that a pair of feedback strategies is optimal, i.e., is in a saddle-point equilibrium.

To pick up the end of Section 3, the value V of the objective function of the game can be considered as a function of the actual position y and the time t using prescribed feedback strategies,

v(t, y) := J(Fp(t, y), rE(t, y); t, y). (24)

The function describing the minimax value of the objective function when the game is started from a position (t, y) has historically been called the value 6 of the game and will be called the saddle-point value V* in the following. There holds

V*(t, y):= min max J(Fe(t, y), FE(t , y); t, y) Up FE

= J(F*(t, y), F*(t, y); t, y). (25)

Note that the minimax value equals the maximin value because of the minimax assumption (21). If such a function V* exists and is continuously

6Isaacs stated: " , . . the best value of the payoff, its minimax, will be termed the Value;" see Ref. 7, p. 4.


differentiable in t and y, it satisfies the partial differential equation, the so-called Isaacs equation, 7' 8

rain max (c?V*/@ ) f ( t, y, 7e, '/ E) + (OV*/Ot) = 0; (26) 7P 7E

see, e.g., Ref. 7, p. 67, and Ref. 16, p. 346. Note that, in case of an integral payoff, Eq. (26) is not valid in this form. A rigorous proof of Eq. (26) can be found in Ref. 20.

The essential transition from Eq. (25) to Eq. (26) provides the basis for the realization of the optimal feedback strategies F* and F* by the optimal open-loop strategies 7~ and 7E*. In detail, we first obtain

* t v y * , v ~ * ) 7*='?*(t, y, V* , V*) and ~.;*=~/E(, Y,

by minimaximizing Eq. (26). Substituting these expressions into (26), the solution V*(t, y) of the partial differential equation yields

V * = 7 * ( t , Y ) and " /*= '* t y~(, y).

Herewith, we have

* t F*(t, y ) = 7e(, y) and F*(t, y ) = 7~(t, y).

Trajectories obtained by substituting 7* and 7* into the dynamic system will be called saddle-point trajectories. Finally, we end up with the relations (22).

7The counterpart in optimal control theory is known as the Hamilton-Jacobi-Bellman equation. However, Constantin Carath6odory seems to be the first who discovered this equation in •935: " . . . mfissen wir insbesondere danach trachten, die Funktionen ~i(t, xj) und S(t, xj) so zu bestimmen, dab der Ausdruck

r *(t, xj, x'j) = L(t, xj, x}) -- S~-- Sx, x;,

als Funktion der x~ aufgefaBt, ftir x} = Oi(t, xj) ein Minimum besitze, das fiberdies den Wert Null habe." Carath6odory also pointed out the importance of this equation in the calculus of variations. This equation leads him directly to the so-called necessary conditions of Carath6odory. Carath6odory: " . . . sie sollen die FundamentaIgleichungen der Variations- rechnung genannt werden." See Ref. 18, p. 201.

Sin the context of the calculus of variations, Eugenio Beltrami found, in 1868, the fundamental relation between variational problems and first-order partial differential equations: "Infatti, eliminando dalle due precedenti equazioni la derivata y', si ottiene un risultato della forma

• (~, y, (aF/ax), (aF/ay))= O,

ciod un'equazione a derivate parziali del primo ordine non Iineare, cui deve soddisfare Ia funzione F e che pu6 quindi servire a determinarla'; see Ref. 19, p. 368.


In Eq. (26), it again has to be assumed that the minimax value equals the maximin value. For the problem considered in this paper, this assumption is satisfied, since the dynamic system (16) is separable. Later, we will see that also the first derivative of the constraint (10) has to be separable. Separability means that both the right-hand side and the first derivative of the constraint can be written as a sum of a function independent of ?e and a function independent of ?e.

Since V* is not known in advance, Eq. (26) cannot be used directly in the derivation of saddle-point strategies. However, if V* and V* are known in (t, y), the optimal open-loop strategies can be computed via Eq. (26). As known (see, e.g., Ref. 7), the optimal open-loop strategies exist, not necessarily uniquely, if they are chosen within a convex set and if llfll is bounded. Whether the optimal open-loop strategies are unique or not, the saddle-point value V* is uniquely determined in the entire state space for all t.

In Section 4.2, we discuss the necessary conditions of optimality for regular subarcs of the trajectories, i.e., V* is a C2-function in the neighborhood of any interior point of the subarc. Regular subarcs are limited, for example, by discontinuities of the right-hand side or by junc- tions with state constrained subarcs. The necessary optimality conditions associated with the latter cases are discussed in Section 4.3. Section 4.4 deals with the optimality conditions on the terminal manifold. In Section 4.5, necessary sign conditions for optimality are developed. Section 4.6 presents the resulting multipoint boundary-value problems, which will provide the starting point for the numerical computations presented in Part 2 of this paper.

4.2. Regular Subarcs. Necessary conditions for regular subarcs are known from the literature. In the main, we follow Ref. 16, Theorem 8.2, p. 349.

For convenience, we first rewrite the system (16) of ordinary differential equations in autonomous form,

(dido:) =tf fe(Y;,Te) , ~ e [ 0 , 1 ] . (27)

In addition to Eq. (17), we have the initial condition

t(0)=0. (28)


To simplify the following notations, we introduce the adjoint variables

2p := (2x~, '~h,, 2v~) T := tisV*, T,

2e := (2x~, ;the, 2~) T := tssV*~ T,

2, := tlsV,*, ,~:= (,~;, • 2E) •

(29a)

(29b)

(29c)

(29d)

The scaling factor s e N + will later be used to normalize one of the adjoint variables. Then, defining the Hamiltonian by

g( t , y, 2t, 2, ye, 7E) := 2vpfP( t, Ye, 71") V +2EfE(YE, 7E)+2, , (30)

Eq. (26) yields

min max H( t, y, 2 t ,2 ,7e , Te ) -O, for all r E [0, 1]. (31) 7P ?E

From this equation, the necessary conditions of the first and the second order for the optimal open-loop strategies ?* and 7* follow directly,

H~e = 0, H~e~e/> 0, (32a)

H,~= 0, H,~,~< 0. (32b)

Because of the separability, the second-order optimality conditions are easy to handle.

For the problem of Section 3, we have

H~e = -2xeVp sin 7* + (l~hl, Vp- •vt, g) cos 7*

+ 2ve(Dze/me) sin(27*) = 0, (33a)

H~e~e = - - ( ~ . h p V p - - 2 v p g) sin V* -- 2x, vp cos 7"

+ 22~e(Dze/rne) cos(27") ~> 0. (33b)

Similar relations hold for 7*. The first-order conditions lead to equations of fourth degree in sin 7" and sin 7}, respectively. It can be shown that each of these equations has at least two real solutions. By means of the second- order conditions, it can be shown that there exist at most two candidates for sin 7~ and sin * 7E, one of which can be singled out by Eq. (31) if there are two candidates. Because of the initial conditions considered here, optimal open-loop strategies satisfy [7"1, 17"1 < z/2. Therefore, we end up with a unique pair

7" = 7*(t, Ye, 2e), 7* = 7*(YE, 2E). (34)


For details and an algorithm to compute the optimal open-loop strategies, see Part 2 of the paper and Ref. 12.

A characteristic curve of the first-order partial differential equation (31) can now be constructed if V* and its gradient are known at some point (t, y); see, e.g., Ref. 21, pp, 97-131. For this purpose, we have to combine the integration of the system (27) for the state variables with the integration fo the so-called adjoint system for the scaled gradient ()t,, 2 T) over the interval 0 ~< ~ ~< 1,

(d/dz)2, = - tf(O/~t) H(t , y, 2,, 2, ~*(t, Ye, 2p), 7*(YE, 2e)), (35a)

y~, ~), 7E(YE, 2D) T. (35b) (d/dz)2 = - tf(~/Oy) H(t, y, 2 . 2 , 7*(t, 2 *

For example, Eq. (35b) yields, for the components with subscript E,

(d /dz )2e = - tf(C3/(~yE) f e ( Y E , 7") T 2e + (~?Hfi37e)((?)'*/(?Ye)V) • (36)

Note that the second term of the sum vanishes here because of Eq. (32b). This is not true on constrained subarcs; see Section 4.3. For the derivation of Eqs. (35), it is sufficient to suppose that the saddle-point value V* and its derivatives up to the second order are only one-sided contiimous if saddle-point trajectories are considered on the barrier.

4.3. Constrained Subarcs and Singular Surfaces. The first singular surface that we have to deal with is induced by the state constraint (10). The constraint is called active at z e [0, 1] if

Q(hE, re) = 0, (37a)

Q(1)(hE, rE, y*.) := (d/dz) Q(hE, re) > 0, (37b)

where the control 7" is chosen according to Eq. (32b). In this case, the optimal control 7 ' has to satisfy

Q(1)(hE, re, 7*) ~< 0. (38)

Because of the separability of both the right-hand side f and the first derivative of Q, ?* can be obtained from Eq. (31) as

T . 7*= a rg max 2 e f e ( Y e , Te), (39) ~'E s.t. Q(! ) ~< 0

while the control variable 7* is still to be chosen according to Eq. (32a). The maximum problem in Eq. (39) leads to at most two candidates for 7*. The first candidate satisfies the equation

Q~l)(he, re, 7~) = 0, (40)


and a second candidate appears only if the Hamiltonian has a second local maximum with respect to 7E. Should the second candidate become maximal, severe theoretical and numerical difficulties would occur. Since this never happened in the numerical examples, we assume that the first candidate is maximal. Then, the so-called optimal enforced control 7 b must E fulfill Eq. (40), and the state constraint remains active. Solving Eq. (40) for sin 7e and substituting v e via Eq. (37a), one obtains

7~(hE) = arcsin [( -- K 2 -- x j K ~ - 4 K~ K-3)/( 2K~ )], (41)

where

K1 := D I E ~ m E , K2 := 1/2v~ 0 In ~/Oh E - g ,

K 3 : = ( T E - - DOE -- D l £ ) / m E.

~b On a constrained subarc, where 7* := YE, the second term of Eq. (36) does not vanish. For abbreviation, we define

# := - (0/07e) H(t, y, 2 t, 2, 7", 7*)/(0/~77E) Q(1)(he, rE, 7"). (42)

Then, Eq. (36) becomes

7E) ]" (43) (d/d~)2E= - tt[(O/@E) fE(YE, ;b)V 2E + #(O/OyE) Q(~)(YE, h V

In addition, one obtains the important sign condition

=0 <=~ constraint (10) is inactive, (44) # <0~=~constraint (10) is active.

This can be proved as follows. From

Q(I} {>0, for 7E<7~, ~<0, for 7E>~7~,

one has

(Q/#7E) Q(I)(hE, re, 7~,) < 0.

If Eq. (37a) holds, one can show that

{~> 0 ~ 7* ~> 7~ ~;~ constraint (10) is inactive, (3/D7E) H( t, y, ).t, )o, 7*, 7~) <0~=~7*<7~=~constraint (10) is active.

From Eq. (32b) and the last two equations, the result (44) follows immediately. In view of Eq. (44), we may replace Eq. (36) by Eq. (43),


whether the constraint is active or not. Thus, the result obtained here fits in the same pattern as known from optimal control theory, and/~ plays the role of a multiplier; see, e.g., Ref. 22. Since 7E appears explicitly in Q(~), the state constraint is said to be of the first order; compare Ref. 22, too.

The necessary conditions are to be discussed when saddle-point trajectories enter or leave the singular surface induced by the state constraint. Moreover, additional singular surfaces are caused by the discontinuities of the right-hand side f due to the thrust model (13). On entering singular surfaces of these types, the gradient of the saddle-point value V* may have discontinuities. If the singular surface is defined by an equation of the form

A(t, y ) = 0 , (45)

the gradient of the saddle-point value immediately before and after reaching this singular surface must satisfy, according to Ref. 23, the following jump condition:

12,(3;- = + ~,(~/~(t, y)) A(t(3s), y(3,)) ~, )] ?,(3: )~

4(32)_I L 4(3 +) J e R. (46)

In addition, Eq. (31) yields

H(t, y, 4 2 , 2 - , 7* - , 7 " - ) = 0,

H(t, y, 4 7 , 2 +, 7*% 7 *+ ) =0 ,

where

(47a)

(47b)

2~-:=2~(~r ), 2 + :=2 , ( z+) , etc.,

7 * - : = 7 ~ ( t , ye, 2e) , 7 *+ :=7*(t, yp, 2;,), etc.

Equations (31) and (47) are based on, what was named by Isaacs, the tenet of transition; see Refs. 2 and 7, p. 67. For the problem of Section 2, the jump condition (46) leads to the following jump conditions for the adjoint variables 2he and 2rE entering the singular surface Q(h e, r e )= 0 at 3 = z a :

),~ = 2h~ + tfp + (~/~he) Q(hE(3a), vE(za)), (48a)

).;~_ = 2 + + tf# + (~?/~v~) Q(hE(za), v~(za)). (48b)

Here, #+ is defined according to Eq. (42). This can be shown as follows. Because of Eq. (46), only the adjoint variables 2he and 2re can be discontinuous in 3~. The control 7* is continuous because of the continuity of 2p. However, 7* may be discontinuous in z,. Note that 7 *+ is determined by


Eq. (41). Since 7*- must satisfy Eq. (32b), the jump condition (46) leads to

2{r(O/C~7~.)fe(yE, 7~- )+c~(~?/OyE)Q(ye) (O/OTe) fe (yE,~ ,*- )=O. (49)

From the continuity of the Hamiltonian (47), we obtain a second condition to determine 7 " - and c~,

~ j~ty~, )- ~ /~(y~,7~* )=o.

This equation leads, using (46), to

).yTA.(yE, r .+ ) +T -- 2e f E ( Y e , )'* - ) -- ,~(O/~ye) Q(YE) f e ( Y e , ~* ) = O.

(50)

The nonlinear equations (49) and (50) can be fulfilled by choosing ~*- and e as

7 " - =7 *+, (51a)

~= - t f 2 ~ T ( 3 / O T e ) f ~ ( y e , v*+) / (~ /Ove)Q(U(ye , v * + ) = t f # +. (51b)

These solutions also satisfy Eq. (47). Thus, the proof of Eq. (48) is completed.

Since # is continuous, constrained subarcs become nonoptimal at z = rb if there holds

Y(zb) = 0, (d/dz) y(%) ~> 0. (52)

The entry point ~ and exit point re of a constrained subarc are determined by

Q(he(za), vE(za)) = 0, #(%) = 0. (53)

It should be mentioned that the role of the entry and the exit points can be interchanged; i.e., the jumps of the adjoint variables can be placed at the exit point ~ = ~b instead of the entry point.

In summary, the singular surface due to the state constraint consists of two parts. The first part is of universal (afferent) type, and the second part is of dispersal (efferent) type. Both parts lose their validity at the separating submanifold induced by the exit condition (52). This type of (say) universal-dispersal singular surface is sketched in Fig. 3.

In the same way, the discontinuities due to the thrust model also cause two singular surfaces of the form

A, ( t ) := t - t~ = O, i = 1, lI, (54)


where ti indicates the switching times between the different thrust phases of the missile. Only 2, has discontinuities at re := t i / t l ,

2,(z7) = ).,(z+) + [ 2 ~ ( z 3 / m e ( ~ j ] E T p ( z + ) - Tp(z,:)] , i = I , II. (55)

Note that all derivatives of the state and control variables are continuous when saddle-point trajectories penetrate these singular surfaces, although 4, is discontinuous.

xe(1) :=Pl ,

he( l ) :=P2,

v~(1) :=p3,

t(1) :=P6.

4.4. Conditions on the Terminal Manifold. The first condition at z = 1 is given by the capture condition (18), which specifies the terminal manifold. In order to obtain further terminal conditions for the adjoint variables, the terminal manifold is parametrized,

XE(1) :=P~ ~ \/d2- (P4 - -P2 ) 2,

he( l ) :=P4,

rE(1) :---Ps,

Due to symmetry, it is sufficient to consider initial conditions with

0 = x A O ) < xe(O) .

The negative sign in front of the square root cannot occur, since optimal trajectories cannot terminate with xp(1 )>~ x E(1 ). On the terminal manifold, the capture time t s is zero, i.e.,

V * ( t ( p ) , y ( p ) ) = ty = - O,

where p := (Px . . . . . p6) v. By differentiating V* =P6 = 0 with respect to Pi, for i = 1 , . . . , 6, one obtains, together with the Isaacs equation (26), a system of seven linear equations for V* and V*. By means of the scaling factor s [see Eq. (29)], the degree of freedom in the definition of the adjoint variables is used to define

2xe(1) = t fsV*~(t(1) , y(1)) := - t. (56)

From the aforementioned linear system for V* and Vy*, the terminal conditions for the adjoint variables can now be easily calculated,

2x~(1) = - 1, (57a)

2h~(1) = (he ( l ) - -hL . (1 ) ) / (xe (1) - xe(1)), (57b)

I


Fig. 3. Singular surface induced by the state constraint.

The

2~(1) = 0, (57c)

2xE(1 ) = 1, (57d)

2hE(l) = (hE(l) -- he(1))/(xe(1) - xe(1)), (57e)

2~E(1)=0 , (57f)

2,(1) = -2ve fe ( t, Ye, ~*)1~=1- 2vefE(Ye, Y*)I~= ~. (57g)

usable part of the terminal manifold (18) is determined by

max rain (d/&) F(t(1), y(1))~<0; (58) YE ])P

cf. Ref. 7, p. 83. The boundary of the usable part is determined by the equality sign in Eq. (58). Its importance is based on the fact that a saddle- point trajectory lies in the barrier if and only if the trajectory terminates in the boundary of the usable part. A straightforward calculation yields for the boundary of the usable part

max min (H(t, y, 2~, 2, ~e, ~e)-2,)[~=1 =0. (59) YE "~'P

Note that ( F , Fy) is parallel to (V*, V*) at r = t, caused by V*-~0 at z = 1. The resulting optimal open-loop strategies are then given by

V*(1) = arctan[(he (1) - he(1 ))/(x~ (1) - xe(1 ))], (60a)

(arctan [ (hE (1) -- he(1 ))/(xe(1 ) - xp(1 ))] ~=~ constraint (10) is inactive,

|

7"(1) = ]Tb(hE(1)) (60b) I - t. ~=~ constraint (10) is active.


By means of the terminal conditions (57) and (60), the boundary-of-the- usable-part condition (59) leads to

re(1) = ~vE(1) ~ constraint (t0) is inactive, ( r e ( 1 ) cos(~'~ ( 1 ) -- ~,*( 1 )) ,¢~ constraint (10) is active.

(6t)

This equation defines a continuous hypermanifold of the terminal manifold. Therefore, the usable part is simply connected. Note that this does not imply that the barrier is simply connected, too. The barrier may have gaps. However, because of the higher maneuverability of the missible, gaps are unlikely to appear. The complexity of the problem prevents a detailed theoretical investigation.

4.5. Sign Conditions. The first sign condition was derived above; see Eq. (44). This sign condition must be fulfilled by the multiplier /~ on constrained subarcs. In addition, we have to require

2vp< 0, 2rE> 0, for all r e [0, 1[. (62)

Indeed, considering t/e and t/E as additional control variables in Eq. (15), one obtains the switching functions

$1 :=2,,pTemax/me and S2:=)%~TEmax/m E.

Because of Eq. (31),

S , < 0 and $2>0

imply

q * = l and q * = l ,

respectively. Note that the sign condition (62) for the evader is valid on unconstrained subarcs only, since 7b depends on t/e if the constraint (10) is active.

4.6. Muitipoint Boundary-Value Problem. The above necessary conditions can now be combined in a multipoint boundary-value problem for na ordinary differential equations of the following form:

(Go(z(~)),

~(~)=a(z(~))=tal(z!~))' (~.,(z(~)),

T1 ~'C <'C2,

z,,~<r<~ 1,

(63a)


z(r;- ) = ak('Ck, Z(:r~- )), 1 ~< k ~< n~, (63b)

re(z(O), z(1)) =0, 1 ~i<~nb, (63C)

ri(rk~, z(r~i )) = 0, nb+l<.i<~nt, ki~{1 . . . . . n,}. (63d)

In order to obtain a well-defined boundary-value problem, it is generally required that there holds na+ns=n t. The differential equations (63a) include the equations of motion (27), the system for the adjoint variables (35), and some trivial differential equations, for example, (d/dz)ts=-O for the unknown terminal time. The right-hand side of (63a) is only piecewise defined. This is a consequence of the different control laws for constrained and unconstrained subarcs; see Eqs. (34) and (41). Note that the differential equations for the adjoint variables are also piecewise defined; see Eqs. (43) and (44). Moreover, nonsmooth data must be taken into account, too; see Eq. (13). Equations (63b) are the so-called jump conditions, which describe possible discontinuities of the unknown functions z; see Eqs. (48) and (55). Equations (63c) contain the prescribed boundary conditions for the state variables (t, y), i.e., Eqs. (28), (17), (18), and the terminal conditions for the adjoint variables (57) as well. Equations (63d) are the conditions at interior points; see Eqs. (53) and (54).

The dimension of the multipoint boundary-value problem can be reduced by omitting the unknowns t, 2xp, ~.xE, )~t. The differential equations for the first three variables can be solved analytically. The variable 2, is decoupled and does not contribute any information necessary to compute the saddle-point trajectories and the optimal open-loop strategies. In summary, we have nd= 11 unknown functions xe, he, vp, xE, hE, v~, 2hp, 2~p, 2h~, 2~, tf. In addition, we have two unknown switching points za and % for each interior constrained subarc and only one unknown switching point ra in the case of a constrained subarc which ends on the terminal manifold. Moreover, we have two switching points zz and zH due to the nonsmooth thrust model. For the example of one interior constrained subarc, we have ns = 4 and, thus, n, = 15. The corresponding 15 multipoint boundary conditions are given by Eqs. (17), (18), (57b), (57c), (57e), (57f), (53), (54). Note that the jump parameter # + need not be considered as an additional unknown in Eqs. (48) if the integration is performed backward. Otherwise, a trivial differential equation for # + has to be taken into account together with an interior condition at ~ = za according to Eq. (42).

The boundary-value problem as formulated above is used to compute saddle-point trajectories within the interior of the capture zone. For saddle- point trajectories within the barrier, we only have to replace one arbitrary initial condition out of Eq. (17) by the boundary-of-the-usable-part condition (61).


The aforementioned sign conditions must be verified a posteriori. The conditions can be used to single out nonoptimal solutions of the boundary- value problem.

5. Conclusions

Necessary conditions for differential games of pursuit-evasion type lead to multipoint boundary-value problems with jump conditions when state variable inequality constraints or nonsmooth data are involved. By this approach, optimal open-loop strategies and their associated saddle- point trajectories can be computed for the entire capture zone of the game. This also includes optimal open-loop strategies and saddle-point trajectories on the barrier of the game. These open-loop strategies provide an open-loop representation of the optimal feedback strategies. The paper will be continued with a second part in which the numerical method for the solution of the boundary-value problems is described. In addition, the results obtained for the special air-combat scenario between one medium- range air-to-air missile and one high-performance aircraft will be presented; see Ref. t.

References

1. BREITNER, M. H., PESCH, H. J., and GRIMM, W., Complex Differential Games of Pursuit-Evasion Type with State Constraints, Part 2: Numerical Computation of Optimal Open-loop Strategies, Journal of Optimization Theory and Applica- tions, Vol. 78, pp. 443463, 1993.

2. ISAACS, R., Games of Pursuit, Paper No. P-257, RAND Corporation, Santa Monica, California, 1951.

3. [SAACS, R., Differential Games, I: Introduction, Research Memorandum RM- 1391, RAND Corporation, Santa Monica, California, 1954.

4. ISAACS, R., Differential Games, II: The Definition and Formulation, Research Memorandum RM-1399, RAND Corporation, Santa Monica, California, 1954.

5. IsA~;CS, R., Differential Games, III: The Basic Principles of the Solution Process, Research Memorandum RM-1411, RAND Corporation, Santa Monica, Califor- nia, 1954.

6. [SAACS, R., Differential Games, IV: Mainly Examples, Research Memorandum RM-1486, RAND Corporation, Santa Monica, California, 1955.

7. ISAACS, R., Differential Games, John Wiley and Sons, New York, New York, 1965; see also Krieger, New York, New York, 3rd Printing, 1975.

8. BREAKWELL, J. V., and MERZ, A. W., Toward a Complete Solution of the Homicidal Chauffeur Game, Proceedings of the 1st International Conference on Theory and Applications of Differential Games, Vol. 3, pp. 1-5, 1969.


9. MERZ, A. W., The Homicidal Chauffeur: A Differential Game, PhD Thesis, Stanford University, Stanford, California, 1971.

10. BULmSCH, R., Die Mehrzielmethode zur numerischen Lfsung yon nichtlinearen Randwertproblemen und Aufgaben der optimalen Steuerung, Report, Carl-Cranz Gesellschaft, Oberpfaffenhofen, Germany, 1971.

11. STOER, J., and BULmSCH, R., Introduction to Numerical Analysis, Springer- Verlag, New York, New York, 1993.

12. BREn'NZR, M. H., Numerische Berechnung der Barriere eines realistischen Dif- ferentialspiels, Diploma Thesis, Department of Mathematics, Munich University of Technology, Munich, Germany, 1990.

13. MIELE, A., Flight Mechanics, I: Theory of Flight Paths, Addison-Wesley, Read- ing, Massachusetts, 1962.

14. DIEKHOFF, H. J., Praktische Berechnung optimaler Steuerungen .[fir ein t)berschallfiugzeug, PhD Thesis, Department of Mathematics, Munich Univer- sity of Technology, Munich, Germany, 1977.

15. SEYWALD, H., Reiehweitenoptimale Flugbahnen fiir ein t)berschallflugzeug, Diploma Thesis, Department of Mathematics, Munich University of Technol- ogy, Munich, Germany, 1984.

16. BA~AR, T., and OLSDER, G. J,, Dynamic Noncooperative Game Theory, Academic Press, London, England, 1982.

17. Ho, Y. C., and OLSDZR, G. J., Differential Games: Concepts and Applications, Mathematics of Conflict, Edited by M. Shubik, Elsevier Science Publishers, Amsterdam, The Netherlands, 1983.

18. CARATHfODORY, C., Variationsrechnung und partielle Differentialgleichungen erster Ordnung, Teubner, Leipzig, Germany, 1935.

19. BELTRAMI, E., Sulla Teoria delle Linee Geodetiehe, Rendiconti del Reale Istituto Lombardo, Serie II, Vol. 1, pp. 708-718, I868.

20. BERKOVlTZ, L. D., Necessary Conditions for Optimal Strategies in a Class of Differential Games and Control Problems, Journal SIAM on Control, Vol. 5, pp. 1-24, 1967.

21. COURANT, R., and HmB~RT, D., Methods of Mathematical Physics, Vol. 2, Interscience, New York, New York, 1962.

22. BRYSON, A. E., and Ho, Y. C., Applied Optimal Control, Hemisphere, Washington, DC, 1987.

23. BLAQUIERE, A., GI~RARD, F., and LEITMANN, G., Quantitative and Qualitative Games, Academic Press, New York, New York, 1969.

Complex Differential Games of Pursuit-Evasion Type with ... · Since this time, applications of...

Documents

Transcript of Complex Differential Games of Pursuit-Evasion Type with ... · Since this time, applications of...