JOURNAL OF OPTIMIZATION THEORY AND APPLICATIONS: Vol, 78, No, 3, SEPTEMBER 1993

Complex Differential Games of Pursuit-Evasion Type with State Constraints, Part 2: Numerical Computation

of Optimal Open-Loop Strategies 1'2

M . H . BREITNER, 3 H . J . PESCH, 4 AND W . GRIMM 5

Communicated by R. Bulirsch

Abstract. In Part 1 of this paper (Ref. 1), necessary conditions for optimal open-loop strategies in differential games of pursuit-evasion type have been developed for problems which involve state variable inequality constraints and nonsmooth data. These necessary conditions lead to multipoint boundary-value problems with jump conditions. These problems can be solved very efficiently and accurately by the well-known multiple-shooting method. By this approach, optimal open-loop strategies and their associated saddle-point trajectories can be computed for the entire capture zone of the game. This also includes the computation of optimal open-loop strategies and saddle-point tra- jectories on the barrier of the pursuit-evasion game. The open-loop strategies provide an open-loop representation of the optimal feedback strategies. Numerical results are obtained for a special air combat scenario between one medium-range air-to-air missile and one high- performance aircraft in a vertical plane. A dynamic pressure limit for the aircraft imposes a state variable inequality constraint of the first order. Special emphasis is laid on realistic approximations of the lift, drag, and thrust of both vehicles and the atmospheric data. In par- ticular, saddle-point trajectories on the barrier are computed and dis- cussed. Submanifolds of the barrier which separate the initial values of the capture zone from those of the escape zone are computed for two representative launch positions of the missible. By this way, the firing range of the pursuing missile is determined and visualized.

lThis paper is dedicated to the memory of Professor John V. Breakwell. 2The authors would like to express their sincere and grateful appreciation to Professors R. Bulirsch and K. H. Well for their encouraging interest in this work.

3Assistant Professor, Department of Mathematics, University of Technology, Munich, Germany.

4privatdozent f/ir Mathematik, Department of Mathematics, University of Technology, Munich, Germany.

5Assistant Professor, Department of Flight Mechanics and Control, University of Stuttgart, Stuttgart, Germany.

443

0022-3239/93/0900-0443507.00/0 ~ 1993 Plenum Publishing Corporation

444 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

Key Words. Differential games, pursuit-evasion games, open-loop strategies, multiple shooting, multipoint boundary-value problems, saddle-point trajectories, barrier trajectories, missile firing range.

1. Introduction

A wide gap can be seen comparing the complexity of pursuit-evasion game problems which have been solved in the literature with the com- plexity of optimal control problems which today can be solved by sophisticated numerical methods. Because of the close relationship between differential game theory and optimal control theory, it is obvious to try to transfer methods for the solution of optimal control problems to the solution of pursuit-evasion games. Unfortunately, direct methods based on a parametrization of the state and/or the control variables, which are satisfactory and very convenient techniques in the numerical treatment of optimal control problems, cannot be applied to pursuit-evasion games. Indeed, the discretized finite-dimensional problem does not represent the information structure of the game and the freedom of the player's decision to choose their controls instantaneously and independently. It seems that only dynamic programming methods and indirect methods are appropriate for the numerical solution of pursuit-evasion games. Among these methods, we find for example differential dynamic programming (Ref. 2), the gradient-restoration algorithm of Ref. 3 applied to a certain class of differential games (Ref. 4), gradient methods (Refs. 5 and 6), the min-H method (Ref. 7), and the multiple shooting method of Refs. 8 and 9 for the solution of two-point boundary-value problems arising from differential game problems (Ref. 10).

Considering the complexity of solved problems in optimal control, the multiple shooting method has turned out to be reliable, efficient, and very accurate. Moreover, the method is applicable to reaMife problems includ- ing problems with many different types of constraints; see, for example, Refs. 11 and 12. In general, the boundary-value problems arising from the necessary conditions of optimality contain multipoint boundary conditions and jump conditions, in particular, if state variable inequality constraints are involved. The advanced multiple shooting method of Ref. 13 is especially well suited for this class of boundary-value problems.

In Part t of this paper (Ref. 1), necessary conditions for optimal open- loop strategies have been developed for differential games of pursuit- evasion type with state variable inequality constraints. These necessary conditions lead also to the aforementioned type of multipoint boundary- value problems with jump conditions.

JOTA: VOL. 78, NO. 3, SEPTEMBER I993 445

For the special air-combat scenario between one aircraft and one air- to-air missile as described in Part 1 of the paper, saddle-point trajectories on the barrier are computed and discussed in the present Part 2. Further- more, the barrier surface is investigated by computing three-dimensional submanifolds of the six-dimensional barrier to exemplify the domain of the state space in which the theoretical approach of Part 1 holds. Here, the barrier determines the so-called firing range of the pursuing missile.

Hitherto, games of pursuit-evasion type have been investigated only for models simplified by tinearization or singular perturbation techniques; see, e.g., Refs. 14-17. Therefore, the two parts of the paper may serve as an example how to treat more realistic differential games, both theoretically and numerically.

2. Multiple Shooting Method

The multiple shooting method is an efficient and reliable method for the numerical solution of multipoint boundary-value problems of the following form:

( Go(z('c ) ),

2(z)=G(z(r))=tGt(z(:z))'

t.Q(z(z)),

r,(z(O), z(1))=0,

ri(rk~, z(r~)) = O,

0 ~<7:<'Cl,

"C l ~ "C < "C2, (la)

I <~k<~n,, (lb)

1 <~i<~nb, (tc)

nb+l<~i<~n,,kf~{1 . . . . . ns}. ( td)

The boundary-value problem consists of a system including the piecewise- defined differential equations (la), the so-catted jump conditions (lb), the two-point boundary conditions (lc), and the conditions at interior points (ld). The possible discontinuities of the problem are placed at the so-called switching points ~k, k = 1 . . . . . n~.

The multiple shooting method is based on a fixed subdivision

0 = : f l < f 2 < . . . < f ~ m : = 1 (2)

of the interval [0, 1 ] with f j ¢ vk, j = 1 . . . . . nm, and k = 1 . . . . . ns. Initial data for the variables zi at the nodes fj have to be guessed as well as the switching points vk- Let Z i denote an initial guess for the vector z(f~), for j = 1 . . . . . nm, and f := (?I . . . . . f,,)v for the switching points.

446 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

The multiple shooting method is based on reducing the solution of the boundary-value problem to the solution of a series of initial-value problems: For j = 1 . . . . , n m - 1 , find the numerical solutions of the initial-value problems

where

~(~) = [ c (~ ) ) ] , ¢j~<~<%,, with e(~j) = Zj := [ZJ] , (3)

R(G, Z.o_l):= [r,(2~, z(i°m; % _ 1 , 2 .~_~)] ;=1 . . . . . . b,

S ( Z ' I , • • • , Z n m - I ) : = [Fi (~ki, 2(ekii "~ ~ci, Z~¢i )) ] i= n o + 1 . . . . . . . •

The index ~c i is defined by ~, < fk~ < ¢~+ ~-

with

~(¢) := [=(¢)L ~]¢.

During the numerical integration, the jump conditions (Ib) have to be carried out at the switching points G e ]~j, ~j+l[. Moreover, the integra- tion must be stopped also at switching points where no jumps have to be performed, since higher derivatives of some variables may have discontinuities here. These discontinuities must be taken into account to preserve the order of convergence of the integration method. Note that the right-hand side G changes with k.

Let

e(T; ~, 2~) = (z(~; G 2 f , ~)r

denote the solution of the initial-value problem (3) in the interval [fj, fj+l]- Then, a trajectory z(r) and the associated switching points rk are a solution of the above multipoint boundary-value problem if and only if the vector 2 := ( Z , , . . . , Z,,,_,) -r is a zero of

N(2) = 0. (4)

Here, the components of N include the continuity or matching conditions

~ ( 2 , . . . . . Z.m_,):= ~ ( f j + , ; ~ j , G ) - 2 ' j + , , l<j<~nm-2, (5)

and the boundary and switching conditions

R(ZI' Znm-l) ] (6) ( f f n m - l ( Z " " " " Z n m - 1 ) : = Ls(2, . . . . . 2.m__,) '

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 447

A zero of the above system of nonlinear equations is determined by a modified Newton method. The multiple shooting algorithm as briefly described above is implemented in the FORTRAN code BNDSCO; see Ref. 13. A user manual and a listing of the program can be found in Ref. 18. The code BNDSCO is a development of predecessor codes published in Refs. 8 and 19. The integration method used here for the numerical solution of the initial-value problems (3) is the well-known Gragg-Bulirsch-Stoer extrapolation method; see, e.g., Ref. 9, pp. 458.

3. Numerical Computation of the Optimal Open-Loop Strategies

Along a saddle-point trajectory, one has to compute the optimal open- loop strategies at every z e [0, 1 ], where the right-hand side of the differen- tial equations (la) is to be evaluated for the numerical integration. On con- strained subarcs, the optimal open-loop strategy 7" is to be determined by the equation for the enforced optimal control; see Section 4.3 of Part 1. On unconstrained subarcs, the optimal open-loop strategy 7* is implicitly defined by an equation of the form

N(w) := ClW+x/1-~(C2+C3w)=O, with tw[<l , (7)

where w := sin 7e. The same result with w := sin 7e is obtained for the optimal open-loop strategy 7* along the entire trajectory; see Section 4.2 of Part 1. The coefficients Ci are defined differently for 7p and 7E. By multiplying Eq. (7) with the polynomial

~(w) := C l W - , / 1 - w 2 (C2 + Cgw),

the solutions of Eq. (7) are in the set of the zeros of the resulting polyno- mial NN of the fourth order. The real zeros of this fourth-order polynomial can be computed with an appropriate library subroutine, for example, the modified Laguerre method (NAG, Mark 13, routine C02AGF). Here, advantage is taken of the fact that the real zeros of the derivative of a poly- nomial separate the possible zeros of the polynomial itself. For the pursuit- evasion game considered, one obtains at least two real zeros. The global minimizer w* and the global maximizer w* of the Hamiltonian can then be directly obtained. Note that it is very important for the convergence of the numerical integration method to obtain a solution w* of Eq. (7) with the highest possible accuracy. If Iw*[ ~<0.99, one obtains 7* = arcsin w* from w* with the loss of at most one digit. For more details, see Ref. 20.

By the reduction of the model (see Section 2 of Part 1), a boundary layer for 7" at • = 0 is introduced: 7* increases from up to - 8 0 °, in eases

448 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

of low initial velocities of the missile, to about + 50 ° during the first three seconds. This is due to the fact that large values of 17el are not appropriately taken into account by the reduced model. Therefore, a constant suboptimal control 7~, is chosen in the interval [0, zz] with the additional interior condition

~5(z , ) = ~*(z,) , (8)

where zz is the z-value corresponding to tl = 3 s. Jumps of the adjoint variables do not occur, nor is it necessary to modify the differential equa- tions for the adjoint variables. The choice of this suboptimal control for the pursuer does not cause a significant deviation from the saddle-point trajec- tory. Indeed, the missile is relatively slow at launch, and compared to the entire flight time, the suboptimal control is chosen only in a small time interval. Otherwise, the assumption that I~el and I~e[ are small is always satisfied in the solutions presented in this paper. The falsifying effects of the model reduction are insignificant for optimal midcourse guidance; see Ref. 21.

4. Initiation of the Multiple Shooting Method

One of the severe obstacles that must be overcome on applying the multiple shooting method is the construction of a so-called starting trajec- tory, i.e., an initial guess for the Newton iteration. A good starting trajec- tory for the example presented can be obtained as follows. Because of the separability of the equations of motion and of the system for the adjoint variables, trajectories for both vehicles can be computed independently from each other. The systems are coupled via only the terminal conditions. For the aircraft E, reasonable initial conditions for the state variables xe, he, ve can be chosen easily. The initial conditions for the adjoint variables, except for 2xE and for the terminal time tf, must be guessed properly. Then, a forward integration of the system of the evader is carried through. By means of the terminal values of xe, he, rE, 2hE, one obtains terminal values for the missile P via the capture condition and the terminal conditions of the adjoint variables. Finally, a backward integration of the system of the pursuer is performed. In general, the state constraint will be violated as well as the terminal condition for 2rE. Nevertheless, these two trajectories provide a suitable initial guess for starting the multiple shooting iteration to solve a boundary-value problem with boundary conditions as used above. Here, the state constraint cannot be taken into account at once; it has to be introduced step-by-step using homotopy techniques. See for example Ref. 12.

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 449

For the computation of saddle-point trajectories in the capture zone, multipoint boundary-value problems must be solved for various boundary conditions. For this purpose, the solution of one problem can be used as a good initial guess for a problem with neighboring boundary conditions. The convergence of the multiple shooting method along a homotopy path can be speeded up be extrapolation of previously computed solutions along the homotopy path.

5. Numerical Results

The point of issue in pursuit-evasion games is the determination of the barrier, which separates the capture zone from the escape zone. The practi- cal importance of the barrier, for the example considered in Sections 2 and 3 of Part 1, is given by the fact that the cross section of the barrier with the hyperplane t = 0 provides the so-called firing envelope of the missile. To gain an insight into the structure of the barrier, two three-dimensional submanifolds of the six-dimensional barrier are computed.

In Section 5.1, we present some typical saddle-point trajectories on the barrier, which have been published in part by the authors in Ref. 22 and are summarized here briefly for the benefit of the reader. Their calculated optimal open-loop strategies show general properties of the optimal feed- back strategies of the entire capture zone. In Section 5.2, two computed two-dimensional submanifolds of the barrier are presented to visualize the firing envelope.

All numerical computations were performed on an IBM 3090 in FORTRAN 77 double precision with a 52-bit mantissa. 6 The relative machine precision is 0.22 × 10 -15.

5.1. Saddle-Point Trajectories on the Barrier. Three different sets of initial conditions for the vehicles are considered. The first two examples arise from standard air-combat situations. The first set of initial conditions is given by

xe(0) = 0 kin, he(O) = 5 kin, re(O) = 250 m/s, (9a)

xe(0) free, he(O ) = 8 km, rE(0) = 500 m/s. (9b)

The unspecified initial value xE(0) is determined such that the initial point (t(0), y(0)) lies on the barrier; compare the formulation of the multipoint

6This computer belongs to the German Aerospace Research Establishment DLR in Oberpfaffenhofen, Germany.

450 JOTA: VOL, 78, NO. 3, SEPTEMBER 1993

~. 2O E

~ 1 0 .2Z

Z 0

Fig. 1.

, f -i

0 2o 4 o 6o

xp , x E (km)

Saddle-point trajectories in the vertical plane, First example.

8O

A Cm

@ 4o

o

- 4 0 ...... ,

0 30 60 90

t (s) Fig. 2. Optimal open-loop strategies. First example.

boundary-value problem at the end of Section 4.6 of Part 1. This first example describe a situation where the launch position of the pursuer is strategically rather unfavorable, whereas the position of the evader is advantageous.

In Fig. 1, the influence of the dynamic pressure constraint is shown] The dashed line shows a saddle-point trajectory with the state constraint omitted, The solid line shows an example of a saddle-point trajectory where the state constraint becomes active and remains active until capture. The svdtching structure for the state constraint is of type inactive/active. The difference of 3.8 km between the initial horizontal positions of the evader aircraft indicates the advantage of the missile when the aircraft is restricted by the dynamic pressure limit. In the constrained case, the pursuer uses the longer duration ( i f ~ 86 s) compared to the unconstrained c a s e ( t f ~ 33 S) for a climb within the first 35 s. This phenomenon generally occurs if the flight time is not too small. The smaller drag at higher altitudes enables a distinct gain of range for the pursuing missile.

Figure 2 shows the time histories of the optimal open-loop strategies.

7The authors would like to thank M. Paus for providing us with his graphic software packages SOGAP and XGRAMI and for his assistance.

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 451

t60

r-i E

Z _~ 80

tu c r

dynamic pressure l lmi t

/

0 30 60

t (s)

Fig. 3, Dynamic pressure. First example.

90

2O

E

~10

2

o

f l i g h t envelope of E .Z

0 400 800 1000

vp ,v E (m/s)

Fig. 4. Altitude/velocity diagram. First example.

In the unconstrained case, one can see the typical final tail chase. A tail chase generally occurs if the dynamic pressure constraint is inactive in the final phase. Contrary to this case, the constrained trajectory shows an example where a slower pursuer can cut off the way of the faster evader; see the end of Section 4.4 of Part 1. From the histories of the dynamic pressure in Fig. 3, one can clearly see that the dynamic pressure constraint is essential. No aircraft would stand a dynamic pressure of about 150 kN/m 2.

The altitude/velocity diagram for the first set of initial conditions is given in Fig. 4. The diagram shows the three thrust phases of the missile. The transitions between the different thrust phases are related to the corners in the curve of the pursuer. The missile accelerates to more than Mach 3 in the first 15 s. The aircraft dives permanently if the dynamic pressure limit is omitted, whereas the aircraft avoids the dynamic pressure boundary in the constrained case for nearly 80% of the flight time. Note that, as mentioned before, the hit occurs at different terminal velocities if the state constraint is taken into account.

452 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

%

7E Fig. 5. Definition of the look angles.

For practical purposes, it is important to consider the look angle co, since the radio-location is generally restricted to a cone; see Fig. 5 for the definition of co. Figure 6 shows the histories of the look angles for the two vehicles in the unconstrained and the constrained problem. In the latter case, difficulties obviously arise on the observation of the opponent due to the rapid change of the look angles. This is a general experience of engineers who are involved in the construction of missile guidance schemes for the last seconds before a possible hit.

The second set of initial conditions,

xe(0 ) = 0 km, he(O) = 5 km, re(O) = 250 m/s, (10a)

xe(O ) free, hE(0 ) = 12 km, re(0 ) = 650 m/s, (10b)

270

om

RD

L~ 90 3

[2-

3 - 9 0

Fig. 6.

0 30 60 90

t (s)

Histories of the look angles. First example.

JOTA: VOL 78, NO. 3, SEPTEMBER 1993 453

15

"~ '12

w

~: 6

0

0 10 2O 30 Xp ,x E (kin)

Fig. 7. Saddle-point trajectories in the vertical plane. Second example.

100

~-. 80 e.l

z

"~ 40 w

cJ-

dynomic pressu_[e l im i t

10 20 30 35

t(~)

Fig. 8. Dynamic pressure. Second example.

describes a situation where the launch position of the pursuer is unchanged, whereas the position of the aircraft is significantly more advantageous compared to the first example.

Figure 7 shows the saddle-point trajectories in the vertical plane. The influence of the dynamic pressure constraint is considerably decreased. The firing range is very small, less than 3 kin. In both cases, we have tail chase situations before the hit. The flight time is rather short, less than 35 s for both cases: The constrained trajectory of this second example belongs to a class with one interior constrained subarc, i.e., switching structure inac- tive/active/inactive. Entry and exit points can be seen in Fig. 8. The figure also gives an insight on how to introduce the state constraint via the homotopy parameter qE max" The constraint is to be tightened step-by-step by decreasing the homotopy parameter. Then, the switching structure can be guessed easily.

454 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

f ~

30~, ~_

60 120 180

x P ,x E (km)

Fig. 9. Saddle-point trajectories in the vertical plane. Third example.

o,3 -(23

80

-80

P

0 60 120 180 210

t (s )

Fig. 10. Optimal open-loop strategies. Third example.

Finally, the third set of initial conditions is given by

x e ( 0 ) = 0 km, hp(0) = 12 kin, ve(O)=4OOm/s, ( l l a )

xE(0) free, hE(0 ) = 2 km, rE(0 ) = 100 m/s. (1 lb)

This third example describes a situation where a starting aircraft is attacked by a missile launched from a strategically superior position. The aircraft is in an extremely unfavorable position because of its low altitude and velocity.

This set gives an example where saddle-point trajectories with omitted state constraint cannot be computed without introducing a constraint hE > 0 for the altitude of the evader. Figure 9 again shows the pursuit-eva- sion maneuvers in the vertical plane. The flight time here is extremely long and exceeds 200 s. The missile ascends up to about 27 km. Its firing range increases to about 87 kin, which is extraordinarily large for a medium- range air-to-air missile. As can be seen from the histories of the optimal open-loop strategies in Fig. 10, the evader aircraft dives during the first seconds to use the higher thrust excess at low altitude; see Fig. 2 of Part 1. Moreover, Fig. 10 shows that there might be some falsifying effects caused

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 455

1 O0

-~, 80

> 7 2 ~

"~ 40 w

0 60 120 180 210

Fig. 11. Dynamic pressure. Third examp!e.

50

E -~ 20 t J

~ z

~ 10 ~ z

0 r

0 500 1000 1250

, v

Fig. 12. Altitude/velocity diagram. Third example.

by the model reduction since the values of ]~e] are no longer very small. The pursuit-evasion maneuver again ends in a cutoff of the way of the slower missile. The switching structure is of type inactive/active where the constraint is inactive for more than 80% of the flight time; see Fig. 11. The local maximum of the dynamic pressure at about 28 s indicates the possible appearance of a second constrained subarc if neighboring initial conditions are chosen or if the dynamic pressure limit is tightened.

Figure 12 presents the altitude/velocity diagram for the third example. The missile achieves a maximum velocity of more than Mach4. The terminal velocity of the missile is lower than the terminal velocity of the aircraft. In this long-lasting maneuver, the evasive maneuver of the aircraft is similar to a maximum range trajectory for fixed time.

In this third example, the following phenomenon occurs. During the last few seconds, the relation x p < x e ( w h i c h normally holds) is violated. We have a first point with x p = x E . Here, a dispersal surface (see, e.g.,

456 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

Ref. 23, pp. 132) is situated. The evader has now the freedom to choose either ~* or n - ~* as his optimal control. The pursuer must follow the evader immediately. In both cases, we have, after an arbitrarily small interlude, x e < x e or x p < x e , respectively. However, these relations immediately force anew a switch of the optimal controls until again x p = x e

is achieved. Hence, a situation arises similar to the so-called perpetual dilemma (see, e.g., Ref. 23, pp. 137 and pp. 149). The dilemma with the chattering controls can only be overcome by mixed strategies for the evader. This dilemma is obviously caused by the reduction of the model as described in Section 2 of Part 1, since it is assumed that the flight path angle can be chosen discontinuously in contrast to reality. To comply with reality, we stick to the flight path angles [~e], [7el ~< re~2, even if there holds x p >/x E. Obviously, this leads to nonoptimal open-loop strategies for the reduced models, which however correspond to the optimal open-loop strategies for the nonreduced models. Before capture occurs, we have again the relation x p < xE as usual.

For the launch position (9a) of the pursuer, the pursuit-evasion problem is now solved for all pairs (hE(0), rE(0)) within the flight envelope of the evader. Figure 13 summarizes the saddle-point trajectories on the barrier according to their switching structures. Thereby, regions are omit- ted where either h e < 0 (see region I in Fig. 13) or 2 ~ < 0 (see region II in Fig. 13) is encountered along a saddle-point trajectory on the barrier. The first situation requires an addittionat first-order state variable inequality constraint to be imposed, whereas the latter situation indicates that maxi- mum thrust is not optimal for the evader, which points to the appearance of one or more switching surfaces or a universal surface. The remaining part of the flight envelope gives the validity domains of the three different

Fig. 13. Validity domains of the switching structures of the evader.

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 457

switching structures. In the largest region A, we have the switching struc- ture inactive/active (compare the first example above); in region B, the switching structure inactive/active/inactive is optimal (compare the second example above); and finally in region C, the state constraint is inactive dur- ing the entire maneuver. Note that the formulation of the necessary condi- tions as a multipoint boundary-value problem also allows one to compute the boundaries between those regions very accurately. We only have to add additional unknown parameters and boundary conditions. For example, the boundary between region I and region A can be determined as follows. Let he(0) be unspecified, and let the switching time, where he takes its minimum, be denoted by Th. Then add two interior boundary conditions hE(~h)=0 and 7E(Zh)=0. The boundary-value problem remains well defined, and the resulting initial value he(0) constitutes a point on the boundary between region I and region A. In a similar way, the boundaries between the other regions can be computed. In any case, he(O) is to be released. To determine the boundary between regions A and B, a terminal condition #(1)= 0 is added. The boundary of the region C in the interior of the flight envelope is determined by taking into account two additional interior conditions,

qE('Cq)=qE, max and (d/d'c) qe('Cq)=O.

In the same way, we have to add the interior condition 2~E(z~-)= 0, where za denotes the entry point of the constrained subarc, to compute the boundary between regions A and II. Note that the jump of 2re at z = rc, is positive.

5.2. Submanifolds of the Barrier. For the two launch positions (9a) and (1 la) of the air-to-air missile, all pairs of initial values (he(0), re(0)) within the flight envelope of the aircraft are considered. For the lower launch position (9a), initial values in regions I and II are omitted; see Fig. 13. For the upper launch position (1 la), only saddle-point trajectories with switching structure inactive/active as in region A occur. Trajectories with h E < 0 as in region I of the lower launch position also occur.

For the following parameter studies, about 400 multipoint boundary- value problems had to be solved where each boundary-value problem needed between 5 and 30 s of computing time.

Figures 14 and 15 show the firing range of the missile for both launch positions. The curves indicate initial positions of the aircraft with constant re(0 ). The numbers above the curves are the values of re(O). If he(0) and re(0) are given, the firing range xE(O)=xe(O)-xp(O) can be easily obtained. The influence of the dynamic pressure limit for the aircraft on the

458 JOTA: VOL. 78, NO. 3, SEPTEMBER t993

20

E

v

t - . t 0 0 v

_(z b ' l

- - , i i m - - T

0 lO 20 30 x (O) (km)

Fig. 14. Firing range. Lower launch position.

i

35

20

---10 O

0 - { z

o 20 40 60 8'o 9,0

Fig. 15.

XE (0) (krn)

Firing range. Upper launch position.

firing range xe(O) can be seen. The closer the aircraft starts at the dynamic pressure limit and the longer the flight lasts, the stronger the firing range is increased by the state constraint. Note that the lower endpoints of the curves are located on the constraint, i.e., qE(0)=qema X. Comparing Figs. 14 and 15, the importance of a strategically superior launch position of a missile for its firing range is evident in the encounter considered. Note that a considerably inferior launch position compared to (9a) causes a shift of the curves to the left; i.e., some curves due to high values of rE(0) lose their validity because of xe(0)<x~(0) . In this case, the missile cannot enforce capture although the aircraft is positioned right above the missile. A dispersal surface is present, since the aircraft can start in both, the positive and the negative x-direction.

For the lower launch position of the missile, Figs. 16-19 show the total flight time t I, the relative duration z~ of the first unconstrained subarc, the optimal initial flight path angle y~(0) for the aircraft, and the optimal initial flight path angle 7p(0) for the missile, respectively. The limits of the model, because of the constant mass assumed for the aircraft, are reached

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 459

Fig. 16. Total flight time. Lower launch position.

Fig. 17. Relative duration of the first unconstrained subarc. Lower launch position.

Fig. 18. Optimal initial flight path angle of the aircraft. Lower launch position.

460 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

Fig. 19. Optimal initial flight path angle of the missile. Lower launch position.

for long-lasting maneuvers of more than 3 rain, since the mass would decrease up to 15%. For relevant starting positions of the aircraft of medium velocities and altitudes, the dynamic pressure limit is avoided for about 80% of the total flight time; see Fig. 17. The same result holds for the upper launch position. The aircraft dives if the initial velocity and/or the initial altitude is small; see Fig. 18. Figure 19 shows that a launch with flight path angles ~e(0) between 35 ° and 70 ° is optimal. For the upper launch position, the optimal initial flight path angles for the missile are almost constant, 7e(0) ~ 60°.

6. Conclusions

Complex differential games of pursuit-evasion type can be solved numerically when the multiple shooting method is applied to the bound- ary-value problems based on the necessary conditions for optimal open- loop strategies. These boundary-value problems contain multipoint bound- ary and jump conditions if state variable inequality constraints or non- smooth data are involved in the problem. By this approach, saddle-point trajectories and their associated open-loop strategies can be computed for the entire capture zone of the game. Different boundary-value problem for- mulations correspond to the interior and the boundary of the capture zone. The resulting optimal open-loop strategies realize a so-called open-loop representation of the optimal feedback strategies if the latter exist. The optimal open-loop strategies computed so as to satisfy the necessary condi- tions of optimality can then serve to construct those feedback strategies: One computes the optimal open-loop strategies for each point of a multi- dimensional grid, appropriately discretizing the capture zone, and stores

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 461

the information to solve the underlying multipoint boundary-value problem by the multiple shooting method as efficiently as possible. Then, neighboring problems can be solved due to every point of the capture zone, maybe in real time, when using the inherently parallel structure of the multiple shooting algorithm.

The performance of the multiple shooting method is demonstrated for the pursuit-evasion problem between one aircraft and one air-to-air missile in the vertical plane, where realistic data are used to model the aerodynamic properties and thrust of the vehicles and the atmosphere. A dynamic pressure constraint for the aircraft imposes a first-order state constraint on the model.

A reduction of the models in the sense of singular perturbations is carried out to facilitate a subsequent synthesis of the open-loop strategies and to gain an insight into the nature of the optimal trajectories. The model reduction is valid for midcourse guidance. Note that the considera- tion of the full model would not impose severe difficulties for the numerical treatment by the multiple shooting method. On the contrary, the optimal open-loop strategies could be expressed in closed form for a parabolic drag polar. Moreover, a dynamic pressure constraint would remain of the first order, whereas an altitude constraint would then be of the second order.

References

1. BREITNER, M. H., PESCH, H. J., and GRIMM, W., Complex Differential Games of Pursuit-Evasion Type with State Constraints, Part 1: Necessary Conditions for Optimal Open-Loop Strategies, Journal of Optimization Theory and Applica- tions, Vol. 78, pp. 419-441, 1993.

2. J)~RMARK, B., Convergence Control in Differential Dynamic Programming Applied to Air-to-Air Combat, AIAA Journal, Vol. 14, pp. 118-121, 1976.

3. MIELE, A., DAMOULAKIS, J. N., CLOUTIER, J. R., and TmTZ~, J. U, Sequential Gradient-Restoration Algorithm for Optimal Control Problems with Nondifferen- tial Constraints, Journal of Optimization Theory and Applications, Vot. 13, pp. 218-255, 1974.

4. EICHBERGER, H., Ein sequentieller Gradienten-Restaurations-Algorithmus zur Behandlung einer Klasse yon differentiellen Spielen mit vorgegebener Spieldauer, Diploma Thesis, Department of Mathematics, Berlin University of Technology, Berlin, Germany, 1977.

5. DOLEZaL, I. A Gradient-Type Algorithm for the Numerical Solution of Two-Player Zero-Sum Differential Game Problems, Kybernetika, Vol. 14, pp. 429-446, 1978.

6. QUINTANA, V. H., and DAVISON, E. J., Two Numerical Techniques to Solve Dif- ferential Game Problems, International Journal of Control, Vol. 16, pp. 465-474, 1972.

462 JOTA: VOL. 78, NO. 3, SEPTEMBER 1993

7. TOLWlNSKI, B., Numerical Solution of n-Person Nonzero-Sum Differential Games, Control and Cybernetics, Vol. 7, pp. 37-50, 1978.

8. BuLmscn, R., Die Mehrzielmethode zur numerischen L6sung yon niehtlinearen Randwertproblemen und Aufgaben der optimalen Steuerung, Report of the Carl- Cranz Gesellschaft, Carl-Cranz Gesellschaft, Oberpfaffenhofen, Germany, 1971.

9. STOER, J., and BULmSCH, R., Introduction to Numerical Analysis, Springer- Verlag, New York, New York, 1993.

10. BAYEN, H., Numerische L6sung yon differentiellen Spielen mit der Mehrzielmethode, Diploma Thesis, Department of Mathematics, University of Cologne, Cologne, Germany, 1974.

11. BULIRSCH, R., MONTRONE, F., and PESCH, H. J., Abort Landing in the Presence of a Windshear as a Minimax Optimal Control Problem, Partl; Necessary Con- ditions, Journal of Optimization Theory and Applications, Vol. 70, pp. 1-23, 1991.

12. BULIRSCH, R., MONTRONE, F., and PESCH, H. J., Abort Landing in the Presence of a Windshear as a Minimax Optimal Control Problem, Part2; Multiple Shooting and Homotopy, Journal of Optimization Theory and Applications, Vol. 70, pp. 221-252, 1991.

13. OBERLE, H. J., Numerisehe Bereehnung optimaler Steuerungen yon Heizung und Kiihlung fiir ein realistisehes Sonnenhausmodell, Habilitationsschrift, Munich University of Technology, Munich, Germany, 1982.

t4. SHINAR, J., and GUTMAN, S., Three-Dimensional Optimal Pursuit and Evasion with Bounded Control, IEEE Transactions on Automatic Control, Vol. 25, pp. 492-496, 1980.

15. GUTMAN, S., and KATZ, D., On Guaranteed-Cost, Closed-Form Guidance via Simple Linear Differential Games, Proceedings of the 27th IEEE Conference on Decision and Control, Austin, Texas, pp. 1421-1424, 1988.

16. GUELMAN, M., SHINAR, J., and GREEN, A., Qualitative Study qf a Planar Pursuit-Evasion Game in the Atmosphere, Paper No. 88-4158-CP, Proceedings of the AIAA Guidance, Navigation, and Control Conference, Minneapolis, Minnesota, 1988.

17. SHINAR, J. and GAZIT, S., Optimal No-Escape Envelopes of Guided Missiles, Paper No. 85-1960, Proceedings of the AIAA Guidance, Navigation, and Control Conference, Snowmass, Colorado, 1985.

18. OBERLE, H. J., and GRIMM, W., BNDSCO: A Program for the Numerical Solution of Optimal Control Problems, Internal Report No. 515-89/22, Institute for Flight Systems Dynamics, DLR, Oberpfaffenhofen, Germany, 1989.

19. DEUFLnARD, P., Ein Newton-Verfahren bei fastsinguliir Funktionalmatrix zur L6sung yon nichtlinearen Randwertaufgaben mit der Mehrzielmethode, Disserta- tion, Department of Mathematics, University of Cologne, Cologne, Germany, 1972.

20. BREITNER, M, H., Numerische Berechnung der Barriere eines realistischen Dif- ferentialspiels, Diploma Thesis, Department of Mathematics, Munich University of Technology, Munich, Germany, 1990.

21. KATZlR, S., CLIFF, E. M., and LUTZE, F. H., A Comparison of Dyncanic Models

JOTA: VOL. 78, NO. 3, SEPTEMBER 1993 463

for Optimal Midcourse Guifance, Proceedings of the AIAA Guidance, Naviga- tion and Control Conference, Boston, Massachusetts, 1989.

22. BREITN~R, M. H., GRIMM, W., and PESCH, H. J., Barrier Trajectories of a Realistic Missile/Target Pursuit-Evasion Game, Differential Games: Develop- ments in Modelling and Computation, Edited by R. P. H~im~it~inen and H. K. Ehtamo, Springer-Vertag, Berlin, Germany, pp. 48-57, 1991.

23. ISAACS, R., Differential Games, John Wiley and Sons, New York, New York, 1965; see also Krieger, New York, New York, 3rd Printing, 1975.