7/27/2019 Isaacs. Differential games

1/13

JOU RNA L OF O PTIM IZA TION "FHEORY AND APPLICATIONS: Vol. 3, No. 5, 1969

Differential Games: Their Scope, Nature, and Future 1RUF US ISAACS 2

Communicated by J. V. Breakwell

A b st r ac t. Th ere is a profo und distinction between classical mathematicalanalysis and game theory which comes into especial prominence with theadve nt of differential games. T he re is a hierarchy of theories of appliedmathematics in which the classical theo ry is the b otto m row. Thu s, it isimportant in the inevitable pending developments of higher forms of gametheory to be prepared for ideas and concepts which break with tradition.The se general thoughts, not at present widely understood , are expoundedhere with some simple examples which already illustrate the novelties offuture research.

1. I n t r oduc t i onSince the appearan ce of m y bo ok (Ref . 1), I have fe l t inc reas ing concern

over a ce r ta in preva lent m isund ers tand ing of the rea l na ture o f d i f fe rentia lgames . My own re f lec t ions , f rom the perspec t ive tha t t ra i l s the se t t ing downof a l l the complex technica l de ta i l s , conversa t ions wi th o thers and the i rwri t ings, and react ions to my lectures, a l l bolster this view. At present , thereappear to be compara t ive ly few people who have grappled in depth wi th theproblems of d i f fe rent ia l games . Hence , the re i s a ra ther widespread fa i lureto grasp the dist inct ion from classical analysis .

Th is d is t inc t ion i s due to the presence of two players wi th con t ra rypurposes . When each makes a dec is ion, he must take in to account h isoppone n t ' s po r t e nd ing a c t ion tow a rd the oppos i t e e nd , h i s oppone n t ' s s imi l a rwarine ss of the fi rs t playe r 's act ions, and so forth. T his s i tuat ion is basical lydifferent from that of m uc h of c lassical math ema tics, which, as I shal l arguebe low, cons is t s of one-player games .1 P a p e r r e c e i v e d N o v e m b e r 2 5 , 1 9 6 8."~P r o f es s o r , D e p a r t m e n t s o f O p e r a t io n s R e s e a r c h a n d E l ec t ri c al E n g i n e e r i n g , J o h n s H o p k i n s

U n i v e r s i t y , B a l t i m o r e , M a r y l a n d .283

1969 Plen um Publishi ng Corporation, New York, N.Y.

8o9/3/s-~

7/27/2019 Isaacs. Differential games

2/13

284 JO TA : V O L. 3 , N O . 5 , 1969Thus, my main thesis is that different ia l games l ie in a s t ra tum dist inc-

t ive ly above the mathemat ics to which we a re accus tomed; accordingly , ourth inking in th i s f ie ld must t ranscend the habi t s inculca ted by t radi t iona lt ra in ing. Beyond a re fur ther s t ra ta which can be expec ted to harbor fur thernove l t ies . I sha l l shor t ly a t tempt to a rrange them in a rough hie ra rchy,a l though the emphasis wi l l be on the d is t inc t ion a l ready ment ioned.

2 . Class ica l Appl ied Mathemat icsWe sha l l be c onc e rne d w i th t hose p rob le ms w hose a nsw e rs a re numbe rsor se ts of num bers . Inc lu ded in the la t te r a re func t ions , so tha t d i f ferent ia l -

equa t ion prob lems or prob lems of var ia t iona l ca lculus are among the legionsof e xa mple s . The sc i e nc e o f suc h p rob le ms c a n be t e rme d classical appliedmathematics.

Th ere a re fami lia r ways to f rame ma ny of these prob lems as maximiz ingor minimiz ing problems. Th us , the dom ain o f c lass ical mechanics reduces tominimiz ing the Ham il tonian; Ferm at ' s pr inc iple of leas t t ime appl ies to opt ics ;f inding the root of a polynom ia l can be done by minimiz ing i ts modulu s . I f wetolerate a bi t of arti f ice , there is no do ub t that a ll such p rob lem s can b e soframed. Such problems a re one-player games . The i r mat r ix has a s inglec o lumn or row .

This v iewpo int provides perspec t ive on the re la t ion of game theo ry togenera l appl ied mathem at ics and on th e re lat ion of d i f fe rentia l games to genera lgame theory . Thus , we can apprehend a bas ic and ca rdina l fac t : genera l ly ,a problem of game th eory i s not mere ly a b i t ha rder than i ts t radi t iona lone -p l a ye r c oun te rpa r t bu t ma ny t ime s a s ha rd . In t he s imple s t t e rms , t heformer has a ful l two-dimensional matrix; the la t ter , a s ingle-row matrix.

We can al so perce ive the l imi ta tions of t radi tiona l game theory , wher e theapproach i s through f i rs t wri t ing the mat r ix . Should we adopt the same ideafor c lass ica l problems, such as those ment ioned above , the one-row mat r ixwo uld s imply be a l i s t of candida tes for the solut ion a wi th the ensuing payoffe n te re d fo r e a ch . The ga me theor i s t' s d i c tum " f ind the ma x im um " w ould beas use less for obta in ing the solut ion as the order "win the ba t t le" would befor obtaining a mil i tary vic tory.

A grea t dea l of game-theore t ic exposi t ion s tops wi th f in i te mat r ices ,which can be solved by wel l -known methods . But le t us not forge t how nearthey a re to the threshold of puer i le th inking, a t leas t in pr inc iple . Th e i r3 Of course , th i s l i s t would genera l ly be inf in i te and could be w ri t ten i n pr inc ip le only .

7/27/2019 Isaacs. Differential games

3/13

7/27/2019 Isaacs. Differential games

4/13

286 JOTA: V OL . 3, NO. 5, 1969of the gam e, the so lu t ion should inc lude a t least one opt imal s t ra tegy for eachplayer . The bas ic re la t ion be tween these three ent i t i e s , which i s tantamountto their defini t ions, is as fol lows: If e i ther player plays an opt imal s t ra tegy,he i s ce r ta in to reap a payoff equa l to or b e t te r than the V alue . I f he p lays astra tegy that is not opt imal , there is a strategy fo r his opponent leading to apayoff worse than the Value (for the fi rs t player).

P laying opt imal ly , in the game-th eore t ic sense , thus guarantees a p layer acer ta in outcome which he cannot expec t to improve , provided his opponentacts rationally. As this outcome, the Value, is the same for both players , i t isuniq ue and has wo n acceptance as a cons t i tuent of the mathem at ica l ly def inedsolut ion.The solu t ion so def ined appears conserva t ive and here in l ie s a poss ib lereason wh y i t migh t not a lways be used. N ote th e i ta lic ized c lauses above . Forins tance , i f a mi l i ta ry com ma nde r has reason to be l ieve tha t h i s opp one ntwil l not act ra t ional ly (that is , wil l not play opt imally), he may be able to dobe t te r than the V alue i f h i s be l ie f turns out to be jus t if ied . But , to reap th i sga in , our commander must devia te f rom his opt imal s t ra tegy; and there wi l lbe a way for the o ppos i t ion to exploi t th i s de fec t ion b y caus ing a payoff toour c omma nde r w orse t ha n the V a lue .

Exam ples of th i s k ind of r i sk taking a re not unkn ow n in h is tory . T o acom ma nde r ben t on su ch a r isk , the game theor i s t can offe r l it tl e quant i ta t ivesound advice , unless (and how l ikely is i t in pract ice ?) he can re l iably assignnumerica l probabi l i t i e s to the opponent ' s devia t ion f rom opt imal i ty .

Vir tua l ly a l l of my o wn w ork on d i f fe rent ia l games and most of tha t ofothers has bee n on th i s rung. W e can no w perce ive the p lace of these resul t sin the b roa d math ema tical schem e. T his level of differentia l games is in thesame re la t ion to the usua l mat r ix presenta t ion of game theo ry as our classicalapplied mathematics is to maximizing over a small , f ini te se t of numbers.

I f one p layer in the preceding type of d i f fe rent ia l game i s depr ived of a l lvol it ion , wha t rem ains i s a pa r t icula r ty pe of maximiz ing pro blem (one-playergame) . Th e s tud y of these prob lems i s ca lled cont ro l theory or , more p roper ly ,the theory of opt imal cont ro l . I f control theory replaces the phrase i ta l ic izedin the preceding paragraph, the re lat ionship g iven there becom es met iculo us lyexact.

F ina l ly , a pragmat ic word must be g iven to cont ro l theor i s t s about api t fa l l which bese ts them when they f i rs t embark on d i f fe rent ia l games . Theyare accus tomed to open-loop control, which means tha t the cont ro l va r iablesa re func t ions of the t im e ra ther than of the s ta te var iables , a s i s the case wi tha s t ra tegy. B ut the form er do es not suff ice for two-playe r games . Th e in te r imac t ions of the o ppo nen t canno t be ignored. Imagine how a chess p layer would

7/27/2019 Isaacs. Differential games

5/13

JOT A: VOL. 3, NO. 5, 1969 287

fare i f , before a match, he were to decide his f i rs t , second, third, . . . , moveswi thout regard to those of h i s opponent .

5 . C l a s s ic a l A p p l i e d M a t h e m a t i c s a n d D i f f er e n t ia l G a m e sWe re turn to the important point made ea r l ie r : d i f fe rent ia l games involve

an essent ia l ly higher and novel level of difficul ty than classical appliedmathematics (aga in , the reader may pre fe r to subs t i tu te the words controltheory, thus ga in ing prec is ion but los ing scope) . We employ three examplesdiverse ly typica l of nove l phenomena which a r i se only in the two-player case .

E x a m p l e 1: T h e H o m i c i d a l C h a u f f e u r G a m e a n d A R e n d e z v o u sP r o b l e m . As the form er game i s presented a t l ength in Ref. 1, I suppose tha tthe reade r is famil iar a t least with i ts defini t ion. On page 11 of Ref. 1 is descr ibed,in in tu i t ive ly p laus ib le te rms, the sw erve maneu ver : a t an ea r ly s tage of ce r ta inpar t ies , the evader E commences by chas ing the pursuer P ; these ac t ions ofcourse are la ter interchanged. Note that , in the diagram on page 13 of Ref. 1,the p ath of E t ies on tw o stra ight l ines, while the pa th of P consists of severalsegments , a l i of which a re e i the r s t ra ight or of sharpes t poss ib le r ight or le f tcurva ture . La te r (pages 298 and 300) , we lea rn tha t th i s need not be so: forpar t of the p lay , each p layer can opt imal ly t raverse curv ed pa ths m ore comp lexin na ture and presumably t ranscendenta l , corresponding to the equivoca lcurve in the reduced space .For s ta r t ing points in the region labe led wi th a ques t ion mark on page301, the re appears to be s ti ll a th i rd phase of opt imal p lay . Th is i s not t rea tedin Ref. 1, nor, as far as I know, has i t been quanti ta t ively analyzed; but we candraw some reasonably ce r ta in heur i s t ic conc lus ions . Should E t ry the swervefrom such a s ta r t ing pos i t ion , he w ould be in jeop ardy of an a lmost immed ia tec a p tu re w e re P t o t u rn r i gh t a t t he ou t se t . Fur the rmore , i t w ou ld beadvantageous for P in i t ia l ly to force E away so as to enhance the separa t ionof P and E. T hen, P wo uld ga in in the swerve to fo l low, for , whe n E p ursuesP, he must do so a t a grea te r d i s tance ; and so P a t ta ins the sooner room for aturnabout leading to the f ina l k i l t . Thus , we can expec t the f ina l opt imal p layto enjoy three phases ; f irs t, P swings to the r ight and E f lees f rom him; second,P v eers le f t and E, swi tching d i rec t ion , pursu es h im; and th i rd , P , no w havingadequ a te ma neuv er ing room, turns r ight aga in , E f lees again , and captureensues.Professor J . BreakweU and his assoc ia tes a t S tanford Univers i ty (Ref . 3)offer an a l ternat ive con jectur e as to the st i ll open p hase o f the solut ion.

7/27/2019 Isaacs. Differential games

6/13

288 JOTA: VOL. 3, NO. 5, 1969Star t ing a t a reduced pos i t ion in the region labe led wi th a ques t ion mark,P swings to the le f t wi th E in pursui t but taking ca re not to come too c loseto P . The pa th in the reduced space a rr ives tangent to the capture c i rc le a tsom e point , fol lows this c irc le for a while , and leaves tang ent to this c irc leunti l E is direct ly behind P; and only la ter does P swing to the right .

I t is not easy to ascerta in which, i f e i ther, of these tw o con cepts is correct .He re i s a typica l instance of a main tene t of th i s paper : an u t te r ly newphenomenon occurs unl ike anything in c lass ica l , one-player ana lys is . But , inany case , there is s t i l l an addit ional phase in the solut ion with a t least threesegments in the evader ' s opt imal pa th .

N ote how th i s c omple x i ty w ould e va pora t e , shou ld w e tu rn t he p rob le minto a one-player game. Of the severa l ways to do so , the most appropr ia tei s to leave the k inemat ics unchanged and le t both p layers be minimiz ing.The pursu i t ga me be c ome s a re nde z vous p rob le m; bo th p l a ye rs now s t r i veto min imiz e t he t ime un t il " c a p tu re " . Th e ga me is now of the one -p l a ye r t ypein the sense tha t we can envisage a s ingle mind cont rol l ing both c ra f t wi ththe i r mutua l desideratum.

I f S i s the s ta r t ing point of E and, und er opt imal p lay , he te rmina tes a t T,i t is c lear that his opt imal path is the stra ight segment S T . For obv ious lysuch pa th minimizes the t ime to reach T. Any mot iva t ion, such as above , foranother pa th i s now void , for ce r ta in ly E has no reason to fea r a prematuremeet ing w i th P (such w ould lead to a te rmina t ion t ime smal le r than the Valueend, hence , be cont radic tory) .

No te tha t, in each of these two games , the k inem at ic equa t ions , theplaying space , and the te rmina l surface a re ident ica l . The main equa t ion ofth e l a t te r d i f fers f rom tha t of the fo rmer only in tha t th e or ig ina l m ax-r a in(over the con t rol va riables) is replaced by ra in-ra in . Th us , the solut ions inthe smal l ( those having to do wi th the in tegra t ion of d i f fe rent ia l equa t ions)are very much al ike . The dist inct ion rests ent i re ly on singular surfaces.

E x a m p l e 2: T h e L a d y i n the Lake? Th e lady E is swimm ing a tspeed w 1 in a c i rcula r lake of radius R and cente r O. T he purs uer P , a lasciv iousgent leman, remains on the shore l ine on which he runs a t speed w2 cons iderablygreater than w 1 . As E wishes to re ach th e s hore as far from P as possible , thepayoff is the d is tance PE, ei ther metrical ly or arcwise , a t the instant the ladyleaves the water.

Le t us suppose tha t we se t up the usua l formal appara tus to solve th i s5 This game does not appear in Ref. 1 but has been inserted in both the French a nd Russian

translations.

7/27/2019 Isaacs. Differential games

7/13

J O T A : V O L . 3 , N O . 5 , 1 96 9 289game. The s tandard procedure leads a t f i rs t to wel l -behaved opt imal pathscon s truc ted re t rogress ively f rom the shore (or, ra ther , i t s equivalent in asui table red uc ed space) : E t ravels on cer ta in s t ra ight l ines ; P follows the shorein the expected direct ion; there is an obvious dispersal surface, wi th anins tantane ous mix ed s t ra tegy, wh en P and E s tar t f rom diame tr ical ly opposi tepoints .But , myster iou s ly , th is solut ion soon breaks dow n. F ormally , i t involvesthe square root of a quant i ty which becomes negat ive as soon as the dis tanceOE becomes tess than (wl/w2)R. W h at h a s h ap p en ed ?The answer is c lear i f we drop the formal analys is and cons ider thefol lowing pol icy open to th e lady. Let K be a disk conce ntr ic with the lakeand o f r ad ius (wl/w2)R. When E i s wi th in K, the lady can t ravel a t a greaterangu la r speed abou t O than P . Therefore , the l ady can a t any t ime swim toO and the n reach the rim of K, always rem aining diametr ical ly opposi te to p .6From this ins tant on, she ut i l izes the opt imal s t ra ight paths found ear l ier .

Observe tha t, w hen E is in K , ne i ther E n or P is und er any com puls ionat any ins tant , for re levance of the termi nat ion t ime to the payoff has not b eenpos tu la ted (bu t u nb ou nd ed e ndura nce o f the sw imm er tac it ly has) . Bo thplayers m ay loi ter before E co mpletes h er p lay; a ll st ra tegies here are opt imal .Th is i s t rue even beyo nd K. L e t V0 be the Value wh en E ut i l izes thepreceding ploy. Clear ly , she can do so f rom any s tar t ing point . Thus , ourear l ier formal solut ion is val id only in that region of the playing space in w hic hthe co mp uted Value is l es s than V0 . Outs ide th is region, the V alue is V0 andall s trategies are optimal.I t i s d if f icul t to conceive any one-play er vers ion of th is gam e for whic hthe p reced ing s i tua t ion would ho ld .

E x a m p l e 3 : B a n g - B a n g - B a n g S u r f a c e s . I n t he e arly d ay s o fdif ferent ia l games , the s imples t s ingular surface appeared to me to be thetrans i t ion surface. Later , I learned that control theor is ts used the same ent i tyb u t u n d e r t h e m o r e o n o m a to p o e ic t it le o f bang-bang control.The phenomenon occurs when the k inemat ic equa t ions a re l inear in acon trol va riable 4, w hic h is constrain ed, say, - -1 ~< ~ ~< 1. Th en , almosteverywh ere in the p lay ing space , the op t imal ~ as sumes one o f i ts ex t remevalues, ex cept wh en th ere is a region in wh ich al l adm iss ible are opt imal .

6 S t r i c tl y , s h e is n o t h e r e u s i n g a s t r a t e g y , f o r h e r c o n t r o l v a r i a b l e d e p e n d s o n t h a t o f P r a t h e rt h a n o n t h e s t a t e v a r i a b l e s . B u t t h i s i s ea s i l y r e m e d i e d i f w e a r e w i l l i n g t o s e t t l e f o r l o c a t i o n sa r b i t r a ri l y c l o s e t o d i a m e t r i c a l l y o p p o s i t e w h e n E r e a c h e s t h e e d g e o f K .

7/27/2019 Isaacs. Differential games

8/13

290 JOTA: VOL. 3, NO. 5, 1969A transition surface i s one c rossed by opt imal pa ths and a t whichswi tches f rom one ext reme to the o ther . T hes e can usua l ly be de tec ted throughthe switching function A, a quant i ty which can be computed a long eachoptimal path as i ts different ia l equat ions are integrated. When A = 0, theswitching surface occurs. The detai ls are wellknown to control theorists ; i t isf rom the m tha t I bo r row the na me fo r A .

Afte r Ref . 1 was wri t ten , I foun d, to m y surpr i se , th a t i t i s poss ib le tohave t rans it ion surfaces at whic h A does not vanish . I ca lled th i s pheno men on,tenta t ive ly and jocula r ly , bang-bang-bang control. At such a surface, a controlvar iable of the opponent , which appears nonlinearly in the kinematic equations,a lso abru pt ly sh i ft s it s opt imal va lue. Th us , I do not be l ieve th i s phen om eno nis poss ib le in one-player cases . An example or two i s a l l tha t i s now knownof t h is sub j e c t (s ee t he A ppe nd ix ) bu t a s tude n t o f mine , Po-L un g Y u , ha sselected th ese surfaces as his disserta t ion topic (Ref. 4). ~

A ship P , w i th the curva tur e of i t s t ra jec tory boun ded , endeav ors tomainta in surve i l lance over a s lower submarine E which moves wi th s implemot ion. Thu s , the k inemat ics a re as "in the homic ida l chauffeur game. Bu tthe payoff i s the m axim um of the d is tance P E occurr ing dur ing a pa t t ie ,wh ich P seeks to minimize. Ou r n ew surfaces arise here . At a s ta te on one, P,who has a l inear vec togram, shi f t s f rom sharp r ight to sharp le f t rud der und eroptimal play. At the same instant , E, who has a c ircular vectogram, a lsodiscont inuous ly changes h is t rave l d i rec t ion .

I m p l i c a t i o n s o f t h e E x a m p l e s . I n all t h r ee e x am p l es , t h e l oc al o rdifferent ia l equ at io n aspec t of the so lut ion entai led no essent ia l novel t y overone-player gam es o ther than the tw o se ts of cont ro l va r iables , o ne maximiz ingand one minimiz ing. Thus , wha t i s new can be sa id to res ide in s ingula rsurfaces . The i r ro le has a l ready been d iscussed in Example 1 , and Example 3is abo ut a new ty pe of surface. O ur s ta teme nt a lso holds for Exam ple 2 ,p rov ide d w e g ra n t t he bounda ry o f t he re g ion w he re V = V0 the s ta tus ofs ingula r surface . Whether th i s i s va l id or not depends on the def in i t ion ofsingular surface.

I have not a t tem pted a sharp def in it ion of s ingula r surface , because as ye tthere i s no u ni f ied theory of these surfaces , nor d o we know w hat s t range newtypes a re ye t to appear . Could the re be , for example , a bang-bang-bangcoun te rpar t of the universa l surface ? Th is i s a surface which i s universa l , ye t

His term for the loci of this new phenomenon is double transition surface. He has by nowadmirably clarified the who le question.

7/27/2019 Isaacs. Differential games

9/13

JOTA: VOL. 3, NO. 5, 1969 291fa il s to sa t i s fy the ana lytic cond i t ions . for such, due to the opp one nt havingsome kind of d i scont inuous opt imal behavior the re .M y dictum i s tha t the em phas is for tw o-play er d i f ferentia l games wi th fu l Iinformat ion should be on s ingula r surfaces . Through them wi l l the theorybe c omple t e d .

6+ Th e Hi e r a r c h y Re s u m e dThe tw o-p l a ye r z e ro -sum ga me w i th i nc omple t e i n fo rma t ion i s t he ne x t

leve l. Here , e i ther or both p layers may possess some essentia l knowledge of thecurrent s ta te not ava i lable to h is opponent . Any book on bas ic game theoryexpla ins tha t now the opt imal s t ra tegies usua l ly a re mixed or randomized.To dece ive h is opponent , to concea l h i s own in tent , a p layer genera l ly shouldma ke his decisions probabil is t ical ly. Solving the gam e consists of f indingthe bes t probab i l i ty d is t r ibut ion of dec is ions for each p layer , h i s opt imal m ixeds t ra tegy, and the Value of the game, which now mean s the minim ax of theexpec ted va lue of the payoff . The d iscovery tha t , for f in i te games , so lu t ionsin th i s sense a lways exist i s a major achievem ent of Von Ne um ann ; me tho dsfor f inding the solu t ions a re in the textbooks .

W e can fee l ce r ta in tha t mix ed s t ra tegies wi l l domina te d i f fe rentia l gamesof incomple te informat ion a l so . But very l i t t l e i s known of so lu t ion methods .All that I know is wri t ten in Chapter 12 of Ref. 1.Let us note that , a t this level , we have something radical ly different fromanything on a lower leve l . We a re confronted wi th a new potent ia l domain ofideas . I suspec t tha t i t i s i l l unders tood by many working in a l l ied f ie lds suchas cont ro l theory .S tochas t ic proble ms a re wel l know n in th i s subjec t . B ut they a re concer nedwi th one-player a ffa i rs in which the da ta or current s ta te var iables a re knownonly probabil is t ical ly. If the cri terion is the expected value of the payoff, i t isof ten poss ib le to f ind opt imal s tochas t ic s t ra tegies . But i t is not necessary toemploy them. Pure s t ra tegies do as wel l . There i s no opponent here f rom whichto concea l our in tent ions .I am convinced, an d have been for many years, tha t d i ffe rent ia l games wi llu l t imate ly be a v i ta l mi l i ta ry took Already, in te res t i s keen in the USSR.W he n Am erican author i t ie s a re suffic ient ly convinced o f th i s mi l i ta ry va lue ,the re wi l l inevi tably ensue a concent ra ted and f rene t ic e f for t to c lose thegap. Th e incom ple te informat ion case wi ll be of pa ra mo unt in te res t . I urgea sea rch n ow for the break throu ghs ~and new ideas tha t wi ll render poss ib lea use ful theory .

7/27/2019 Isaacs. Differential games

10/13

292 JOTA: VOL. 3, NO. 5, 1969To the next leve l of the h ie ra rchy, w e ass ign two-player gam es which

are not ze ro-sum. No t only do the losses of one p layer not necessar i ly have toba lance the ga ins of hi s opp onen t , bu t bo th p layers may lose or both m ay ga in .Consequent ly , we a re no longer faced wi th pure conf l ic t ; these games enta i lcooperat ion as welt .

At each leve l , we advance to an essent ia l ly new type of mathemat ica lproblem. We a re now a t a s tage where the very meaning of a so lu t ion i s fa rf rom c lea r. To formu la te the prob lem ful ly , we mu st s ta te (because coopera t ionis an e lement ) whe ther or not the p layers a re permi t ted to negot ia te . Also , dothey do so expl ic i t ly or tac i t ly ? Are the re mean s of enforc ing agreem ents ,i f any, be tw een players ? Is p layer A a l lowed to make a side paym ent to p layerB in order to induce B to play a s t ra tegy great ly benefic ia l to A but , accordingto the rules, only sl ight ly so to B ?

Diffe rent answers to these ques t ions engender d i f fe rent problems.I know o f no unequivoca l , unco ntes ted def in it ion of so lu t ion to any suchproblems. Var ious candida tes , such as the equi l ibr ium points of J . Nash ( foran account of his ideas and l is t of references, see Ref. 5) have been proposedand m any s eem to have jus t i f iable meri t in ma ny cases . But counte rexam pleshave been offe red in which na tura l expec ta t ions of reasonableness seem toevanesce , wi th a lmost an aura of paradox.

Th e n ext rung of the h ie ra rchy might b e n-perso n games . Here , J . Caseis breaking new ground (Ref . 6) . We have reached ra re f ied a i r now and I wi l lno longer a t tempt too sharp de l inea t ions .

M uc h of the research in game th eory for the pas t several years has bee ndevoted to n-player games . Many promis ing and deep resul t s have beenobta ined, bu t they a re by ma ny authors and sca t te red in many papers . I am noexper t and shal l not a t temp t a sum ma ry or an eva lua t ion , except to say tha there , too, there is no single general ly accepted concept of a solut ion.

7 . C o n c l u s i o n sI have t r ied to show that different ia l games, a t their broadest , open to us

a h ie ra rchy of new types of appl ied m athemat ics ; among these, the c lassica l i sa t the bot tom leve l . Each rung demands essent ia l ly new concepts . I advoca tee ffor t s to c l imb th is d i f ficul t l adder ra ther than to descen d b y adher ing tooclosely to t radi t ional thinking.

7/27/2019 Isaacs. Differential games

11/13

JO TA : VOL . 3 , NO . 5 , 1969 293

A P P E N D I XBang-Bang-Bang S u r f a c e s i n D i f fe r e n t i a l G a m e s

Th e t rans i tion surfaces appear ea r ly in d if fe rent ia l games as an apparent lyvery s imple type of s ingula r surface . The idea has been extens ive ly s tudiedin the one-player case (cont rol theory) under the name bang-bang control.

If the kinem atic equ at ions are l inear in a contro l variable ~b, wh ich isbo und ed, then i t i s we l l kno wn tha t the opt im al ~ genera l ly assumes one orthe o ther of i t s ext reme va lues according to the s ign of a ca lculable quant i ty ,the swi tching func t ion A.W e de m ons t ra t e he re w ha t w e be li e ve is a ne w ph e nome n on . In a p rope r ,two-player different ia l game, i t is possible to have a t ransi t ion surface, a twhich each player ' s opt imal s t ra tegy i s d i scont inuous even though A doesnot change sign. This t rans i t ion surface does not appear poss ib le in the one-player case .Consider the d i f fe rent ia l game wi th p laying space in the ha l f -p lanexy w i t h y /> - -1 . T he t e rmina l su r fac e cg i s a t y = - -1 . T he pa yof f is t e rmina lwi th H, the Value on c~, be ing given as - -x . T he kinemat ic equa t ions a re thefol lowing: - 4( - V~y) + V~(2 - ly ) + 2 V ~ co~

2) = (3 y) @ (y - - I) q - 2 ~,/3 sin ~bwi th --1 ~ } ~< 1. He re, as usua l, ~ is mi nim izing while ~b is max imizin g.

I f we solve th i s game in the usua l way, we f ind tha t

and the op t imal ~ is ~ = -- sgn A. O n an d a bov e cg, we find that

so tha t A = - -3 ~ /3 is nega t ive and cons tant . Th e opt imal pa ths a re s t ra ightl ines of incl inat ion --rr/6. On th em , y =-- 1 -- 2 ex p( -- r) , so that t he p aths

ap pr oa ch the l ine y = 1. T he opt im al ~b is ~ = --2rr/3, also constant .Ye t , despi te the cons tant nega t ive A, there is a t ransi t ion surface a t

y = 0. For, in the upper half-plane, vert ical l ines are semipermeable withthe proper or ienta t ion. This i s seen formal ly by not ing tha t

max rain (--~) = 0o 4

7/27/2019 Isaacs. Differential games

12/13

294Y

y = O - k

y - - ~ l -

J

JOTA: VOL. 3, NO. 5, 1969

f

e_Fig. 1

wh en y > / 0 . I t i s not h a rd to see tha t these ver t ical l ines a re opt imal pa ths ,and the original --~r/6 l ines mus t be discar ded a bov e the x-axis , a l thoughthey fu lf il l the formal requ i remen ts of a so lu tion . T he correc t pa ths appearin Fig. 1. T he op t imal s t ra tegies in the up pe r half-plane are ~ = --1 , ~ -- --zr ,so tha t both have abrup t ly changed a t the bang -ban g-ban g surface y = 0 .

Note tha t we must adjo in the ukase to the above game tha t , shouldi t not te rmina te , both p layers would suffe r a dras t ic pena l ty . The submarinesurve i l lance game shows tha t these surfaces can be meaningful in rea l i s t icp rob le ms .

W e ha ve spe ci fi ca ll y p rove d tha t t he a bove phe nom e non c a nno t oc c urin one-play er games of the same format ; tha t i s , whe n the k inemat ic equ a t ionsare of the form

2 = a 6 1 + u + c o s 6 2 , 3~ = b 6 1 - t - v + s i n 6 2wh ere a , b, u, v are given func t ions of x, y and b oth control variab les arem l m m l z m g .

7/27/2019 Isaacs. Differential games

13/13

JOTA: VOL. 3, NO. 5, 1969 295References1, ISAACS, R. , Differential Games, John Wiley and Sons, New York, t965.2. DANS~_I~, J., The Theory of Max-~lin, Springer-Verlag New York, New York,

1967.3. BREAKWELL, . V ., MITCHELL,A., and MERZ, T., Personal Com mun icat ion, 1967.4. Yu, P. L., Transition Surfaces o f a Class of Differential Games, Johns HopkinsUniversity, Department of Operations Research and Industrial Engineering,

Ph.D. Thesis, 1969.5. LUCE,R. D., AND RAIFFA,H., Games and Decisions, John Wiley and Sons, New York,

1957.6. CASE, J. H., Equilibrium Points in N-Person Differential Games', University ofMichigan, Department of Industrial Engineering, Technical Report No. 1967-1,1967.