Research Note 86-86

ARTIFICIAL INTELLIGENCE THROUGH EVOLUTIONARY PROGRAMMING:PREDICTION AND IDENTIFICATION

In Lawrence J. Fogel and David FogelTitan Systems, Inc.

Contracting Officer's RepresentativeGeorge H. Lawrence

DTICELECTE

BASIC RESEARCHMilton S. Katz, Director

U.S. Army

Research Institute for the Behavioral and Social Sciences

AUGUST 1986

Appro.vd for publiC relese; distrobution unlimited.

U. S. ARMY RESEARCH INSTITUTE

FOR THE BEHAVIORAL AND SOCIAL SCIENCES

A Field Operating Agency under the Jurisdiction of the

Deputy Chief of Staff for Personnel

WM. DARRYL HENDERSON

EDGAR M. JOHNSON COL, IN g

Tcchnical Dircctor Commanding

Research accomplished under contract for

the Department of the Army .4

Titan Systems, Inc.

Accesion For

Technical review by DTIC TAB

Steve Kronheim Urannou:ced 0lJ:a.tfication

By ....Di. .i..............-....

Availability Codes

AVsSpecdial or

(DTIC) to comply with regulatory requirements. It has been given no primary distribution other than to DTICaind will be available only through DTIC or other reference services such as the National Technic~al Information .

Service (NTIS). The views. opinions, eand/or findings contained in this report are those of the authorls| endshould not be consiruej as an officia; Ioepatmeni of the Army position, policy, or decision, unless so designatedby other official documentation.

UNCLASSIFIEDSECURITY CLASSIFICATION OF THIS PAGE (Vlther Data Entered)

REPORT DOCUMENTATION PAGE READ INSTRUCTIONSBEFORE COMPLETING FORM

-0 1. REPORT NUMBER 2. GOVT ACCESSION NO. 3. RECIPIENT'S CATALOG NUMBER

ARI Research Note 86-86

4. TITLE (md Subtitle) 5. TYPE OF REPORT & PERIOD COVEREDARTIFICIAL INTELLIGENCE THROUGH EVOLUTIONARY InterimPROGRAMMING: PREDICTION AND IDENTIFICATION February 1986 - April 1986

6. PERFORMING ORG. REPORT NUMBER

7. AUTHOR(s) S. CONTRACT OR GRANT NUMBER(e)

Lawrence J. Fogel and David Fogel PO-9-X56-1102C-l

9. PERFORMING ORGANIZATION NAME AND ADDRESS t0. PROGRAM ELEMENT, PROJECT. TASKAREA & WORK UNIT NUMBERSTitan Systems, Inc.

P.O. Box 12139 2Q161102B74FLa Jolla, California 92037

11. CONTROLLING OFFICE NAME AND ADDRESS 12. REPORT OATEU.S. Army Research Institute for the Behavioral August 1986and Social Sciences. 5001 Eisenhower Avenue 13. NUMBER OF PAGES

Alexandria, Virginia 22333-5600 19114. MONITORING AGENCY NAME & ADDRESS(It different from Controlling Office) IS. SECURITY CLASS. (of this report)

UNCLASSIFIED

lie. DECLASSI FICATION/DOWNGRADINGSCHEDULE

.1%., 16. DISTRIBUTION STATEMENT (of thia Report)

Approved for public release; distribution unlimited.

I-*,

17. DISTRIBUTION STATEMENT (of the abstrt en tered In Block 20, if different from Report)

IS. SUPPLEMENTARY NOTES

Contracting Officer's Representative, George H. Lawrence.

9I. KEY WORDS (Continue on revers. aide If n.ces.sry mid Identify by block number)

evolutionary programming prediction and controlsingle state machines artificial intelligencenoisy environments forecasting

24. ASTRAC" rCmto as roesa ffng'eei.fy mod Identify by block numbow)'This report examines the manner by which evolutionary programming treats anarbitrary prediction problem. Additional experiments were conducted to clarifyuncertainties given increased machine size and the cost/benefit of retainingI"offspring" programs, the impact of noise on the predictive capability of theevolutionary process, and the efficacy of crossover as a mechanism for improvingsimulated evolution. It was also found that some difficult combinatorialproblems such as the classic Traveling Salesman Problem can be addressed throughless complex logics.

DD A 6 1473 EDITION OF I NOV OIS OBSOLETE UNCLASSIFIEDSECU9"TY CLASSIFICATION OF ThIS PAGE (Whm Defa Entered)

'l " / - '- .. ' .. , ; -.' '.-V -'. . -.,'. ,.' . *.," " ' .. / . ..- """" ": -

- .rnvuw n 7W -. -W

I

III

Evolution, it is often said, is the most patientof engineers, redesigning organismspiece-by-piece in the tiniest of increments,using the grand and gradual sweep of geologic

I time as a drafting table.*

*Omni Magazine, March 1986, page 20

Ix

TABLE OF CONTENTS

Page

FOREWORD. .. ..... ........ ........ ... iv

INTRODUCTION .. .. ......... ........ .... 1

DISCUSSION .. .. ........ ........ ....... 2

CONCLUSION. .. ..... ........ ........ .. 136

APPENDIX: Addressing the Traveling Salesman ProblemThrough Adaptation .. ...... ........ 137

jr%

FOREWORD

Military decisions hinge on an ability to forecast the developing

situation. A sufficient understanding of the enemy's response behavior

sets the stage for causing the desired response. Clearly, prediction

and control are essential aspects of the decision process. And yet,

classical techniques treat only a limited portion of such real world

situations. Least mean squared error reduction is rarely the appropriate

criterion. The "plant" or transducer is neither linear nor passive.

The usual approach is to extend the classical techniques through

Iquasi-linear approximations and higher ordered differential functions.

But, such progress leads to ever more complex formulations that are

increasingly unsuitable to primitive computation, and furthermore

requires an in-depth understanding of the physical process which is

not always available.

What is needed is an alternate formulation.., a new beginning

based on a different view of the logical process required for prediction,

control, and other aspects of intelligence. Evolutionary programing

constitutes such an approach... and simple high speed parallel processing

will be notably suitable for such fast time simulation of the evolutionary

process. The task at hand is to devise such programming in support of

military prediction control and decision processes while, at the samer time, to extend our intellectual capability.

I

I, v

1p7

II

i INTRODUCTION

The first Quarterly Progress Report described experiments in theprediction of diverse time series against various payoff functions through

the use of Evolutionary Programming. It is now worthwhile to resolve some

uncertainty concerning the structure of this program and to dimensionalize

the use of this predictive capability. Given an arbitrary prediction

Iproblem, how can one determine the manner in which it should be treatedthrough Evolutionary Programming?I

Some additional experiments were performed to clarify uncertainties

and to prepare for conducting a significant number of larger scale

experiments to be performed on the NASA Ames Cray computer.

I Various authors have suggested the use of crossover as a mechanism

for improving simulated evolution. Some experiments were performed to

explore the worth of this concept. Another series of experiments were

conducted to demonstrate Evolutionary Programming within the context of

I identifying an arbitrary unknown transducer. Some difficult combinatorial

problems such as the classic Traveling Salesman Problem can be addressed

I through the evolution of less complex logics (for example, single state

machines). A demonstration in this regard is contained in Appendix.

I

I

11I%

*.. pa %.~ 'a p ~'.%..t. ~. -*:v-aK~.: K. :...- n

... .' # - - - . ..'.' v -'-" ""'_ -' " '"- -~ ."- ,'- a.'- -' ". -"- " . .

DISCUSSION

Additional Experiments on Prediction

Increased Machine Size and the Saving of Equal-Worth Offspring

In view of the previous experiments, it remains unclear as to

whether or not it is worthwhile to save offspring that are of the same

worth as their parent. It is also of interest to enquire as to the worth

of allowing the evolution of larger finite state machines. The previously

used evolutionary program was altered to save equally worthwhile offspring

and to permit finite state machines of up to fifty states, this in the

hope that an increase in size will improve the predictive capability. In

addition, offspring of equal worth to the parent were also saved.

The first experiment required prediction of the same binary cyclic

environment used in the original experiments (101110011101). As expected,

in the first experiment, a perfect predictor was found, but this logic

was not discovered any faster. Figure I shows the predictive fit, while

Figure 2 indicates the cumulative percent correct prediction. Note that

perfect predictions were always made after the 170th prediction. Figure 3

indicates that adding a state is a beneficial form of mutation. The rate

of increase in the size of the machines follows the probability of adding

a state. Note that as the evolutionary process proceeds, the percentage

of such beneficial mutations (as compared with all possible mutations)Igets smaller, and therefore the evolutionary process "slows down." This

suggests that after a given complexity is reached, it might be appropriate

to include a penalty for that complexity or an increase in the probability

of deleting a state.IThe Prediction of Noisy Environments

Ten experiments were conducted to examine the impact of noise on the

predictive capability of the evolutionary process. The same binary cyclic

environment was corrupted by having each symbol change, this with a

2

QC) 0CN

Co 0

E CD

0 C

U)

0 U)

00

=30

0~ C)x C-

oE W

-0 ) Go

C.)

r 0 04-

CN

-~ or

41

Figure 1

3

"°6co~

E 0

0 0nb4- Co

0

10 U-

4-..

E~u

00

0 C)i V6

0 0

L....'-I. Goo

0

.00

Figure 2

CC,0

0

-j

40

00

-4-

0 0 U

0 C)

o Q)Do

cnq

>1

LO 't w4 oN0 N

Figure 3

&I kk5

probability of one-sixth. Figure 4 shows the best of these experiments

with the predictive fit score being very close to the expected value

for a machine that perfectly fits the underlying cycle. Note that the

excursion of the trace above the dashed line indicates that the evolving

finite state machines were fitting the experienced noise. As shown in

Figure 5, the evolution achieves about 65 percent correct predictions.

The size of the machines grew rapidly to an upper limit to twenty-five

states because of the noise, reference Figure 6.

I Figure 7 shows the worst of the ten experiments. Here the predictive

fit score was about ten percent below what could be expected. After 196

II predictions, there was no evidence that the evolutionary process could

predict better than fifty percent, see Figure 8. Figure 9 again reveals

a rapid increase in the size of the evolving finite state machines.

Figures 10 and 11 indicate the mean and two sigma limits for the ten

il experiments. The mean shows a steady but slow increase (probably to 83 1/3

percent, the highest possible percent correct for this noisy environment).

The two sigma "confidence limits" converge as expected. The mean worth

(average predictive fit) was very close to the maximum expected value

and had relatively narrow confidence limits, see Figures 12 and 13. Note

that if the environment is noisy and an all-or-none payoff function is

imposed, the evolutionary process constructs machines that fit both signal

and noise with equal weight. This is particularly apparent if the

environment is binary.

A brief experiment required the prediction of a noisy four-symbol

environment. Here the noise was quasi-Gaussian... a 67 percent chance

of altering each symbol ± 1 and a 33 percent chance of altering each

symbol ± 2. See Figure 14. A linear payoff function was used to provide

some "incentive" for predicting close to the correct true symbol. This

encourages discovering the signal as opposed to the noise. Figures 15

and 16 indicate the predictive fit and prediction capability generated

through use of this less severe payoff function. Since the four-symbol

environment was known to be simple (a two-state machine would perfectly

predict the cycle), a 0.01 penalty per state was imposed.

6L

00I*

C Ir 0

I C C::)

E c. C) 0 -

C71~

> r,

In tm C).

(9 1 0L E)~0 ) ) 0

00 qOw

4-*

M- 00 0,-W U)

I Figure 4

7

0

0)0

too0 I0 2 C

6~ 0-

0)

q~c 0 *0*

00.0 -o

0

0

a))

Fiur2

0

.4-' 4-

Ecoo K0

00

I C)

too

Figre

CL ) o f

0 C

0

a U) 0L IN

0 (

10 .

I N

I

00

0

•I N N

0 0

! °4- '1J-

I (00 -

CDC0

0 a

W -0

000

0-

C N

Fiur 8

co

o °Lr) 0,--

i0

"-00

_o' Z

coD

Go

0 z

0

0 C) 0 O to4--

Figure 9 "•°

12

I~ 'p

II °

C0N

c 0

1 o~

I o

(00 L

C!)0-

I -4- _ooU

. J

IZ0

IC 0 Q)z

r0- 0

I LOZ

I.-q

C,,

Figure 10

13

00

Cl) N

0 c V0

(S. 0IO-

"s--00

CfCl

O 0 0-~

+Figure 114.

o

0

I!D00

I -CN0

l °D

0

I I.' o .0

I U)0~

I 7 ; .'4--'' " ' "00

I Figure

I C1

. ..I" "" " " , ' " -,'o0., " .'-,- , . " * 0 0. . . ' ' " - - " ••" • "

-I , . - . _ ,. ,,, .. " . :¢ . .., , ,, . , . -. . . . . . . ... , . . . . . .I igr 12-

_00

T- -4-

~~Q)

C)

~~0o L - Qs

0S

oG

00

C'4'

FigureL1

C) E 0 GJLI16

.~*. uUV-W -~to

I0

CC

11

-~ ~ FLY~-X M.? 'W7 .. v~ 77

LC

U) )'

L) to

-D 04o~ cu -0E 0 I

L~~ 0)4

'4--

~> E

U))

UUo m

0~ LO)

to

Figure 15 .

18 .'

a LI)

E5

1 a az0

-4-4J

:E .0EO~ Z

0 0 0

L. w c -,L) Eo Lo 0!E -

V)V

() 0

C)0 0 1 0S- ) C7 c) P D.O '

C., '

Fiur 1-

(\J ~10

b

Saving Equal Offspring with a Penalty for Complexity

Nine experiments were performed on the above-referenced binary cyclic

environment wherein offspring equal in predictive fit to the parent were

saved, and a 0.01 penalty per state was imposed. In one experiment, an

eight-state perfect predictor machine was found on the 63rd prediction,

see Figure 17. 12,603 offspring were evaluated in the 196 predictions.

Figure 18 shows that the machines greatly increase in complexity as

required to "solve this problem." Figures 19 and 20 show that the mean and

I two sigma limits were slightly better than those previously obtained,

reference the first Quarterly Progress Report.

Predictin the Fibonacci Series, Modulo-t0

Having determined that saving offspring of equal worth to the parent

can be of benefit, an experiment was performed to reveal the predictive

capability of the evolutionary program with respect to an environment

generated by the Fibonacci Series, Modulo-10. Figures 21 and 22 indicate

significant improvement over the previously reported results. The

evolutionary process was now able to "learn" more of the 60-symbol long

cycle. After 3,020 predictions, a 42 percent cumulative predictive score

was achieved. In principle, this environment would eventually be perfectly

predicted by a ten-state machine.

Predicting the "I.Q. Test" Environment

Seven experiments were performed on the typical "I.Q. test"

environment 101001000... again saving offspring of equal worth; all other

conditions being the same as those indicated in the first Quarterly Progress

Report. Figures 23 through 29 show these experiments in terms of percent

correct and demonstrate the adaptation with respect to an ever-changing

environment. In the first experiment, the "discovery" that it is better

to predict the zeros was not yet demonstrated because it had successfully

predicted the early one's. The second experiment shows a distinct

I20

-'.~ -r vF~'. r

I °~ 4-j

o) 0

E 0

-0

C) 0 6 0

L--

CD L) ._

0 -"I0 0

0,. ,

0 4-'

(JOD

Figue 17

I 21

• , -, -, - . . 4 -.. . . .', . .. . _ _.. . . .. . . . . .. .. '. :

.o__ _____ N

.- cn

o C-ci 0

000 0- 1 C)

-00p 0

• .- E02 ;0 C : IQ)

a, 0)

-~( D

I CL

II

V' .-.

I Figure 18

1 22

.. .......... ......

o Q)

C:/

4-0-I L0E Q0I XC)4-%4 ..-'0

%0I C: D-c )1 001

I C)

C CN

CD G I- (D t 1- to C

Figure 1I i'-c~) I--, 1 23

CC4-j4

c U) C) 00c (0

0 (D U)x 0 UoW 0 C

_c E *0~ 0) C) 0rt

E L 40~> C)

C: to _

c CL 0

0-i0

0) C 'C'

C 24

crr)

0

E 1

0))I C

0 4!

LI)CCI~

0 0

'.4- 0 0 Q -

>--

A- L-,c

00I

0 0z J

o00)cLL.

Figure 21

01

0D4O z-4

0r E0)

0I

_ _ _ _ ___ V -

oG

-E oi

Figure 22

26

0 0

N 0 )

rZ0 .4 4

co

000

> 0 LO~D *.2

L. ..0C

0 a,

0L_ CDcn M %- w n .0

Fiur 23J I..27

- f' V u WV W V 4-'V'VVV--' 7 - :- -lt K I -T - T TV - WTT pi TT -- .

U)0CL)

0 _ __ _

N) C

'0 0)o 0>

0 0.

'4- U)0000 U') C

00ILL0

I CD

EIr- 0

E 0

(L) cc

CE

ci)00

I-O

ci)

Figure 24

28

-; - W- -Ta -W.L -W

ClC0 Co LO ~

W0

0)jL0 Cl)

CC

0 0 . LO0- 0 ) 0(1

Q) 00) BE

CD 0 C) -4-J

0 0L)

o 0- cQ) C4)

ca E 04- 00

0o0

-4-' 0

0)0

0 co r% c 0 '- I

Figure 2

029

o 0 -

CI) =-4-

00

4- 0

9 1.0

0

CD 0 4) 0

-) 0

E 0

L -

=

L i V

0

ccn 0I~ c 0 );CU).

:3 0

Fi~gure -

03

- ~~~~ 7. -- P -

IL C')___

0)0

0y)

0 v0

-CLL.

CC

0 0 Q)- Lo~~~ C 'i--.0a-

C)C

00 CO

.6 --1- 4 - 1C

0) U'c 0

E. 00 -

, 0

0 0~ 04-J E - O I) q ' ~ -

C- EI (/) :I~L Fiue2

X-31

I~ ~ i&§ ~ J.-'..- 0

0 CC

ci))CL 0

0 C 0 0

4- 0

o 0O i-0 I

E4-0' 0 ,Q) 00

- .-

0 0~ E-

0)0

- 0

LL

oo r o 0 w 0

Figure 2o 02

AveI

Ul) 6. %Ib

0i

0 CL

Cf) -0U)*

0 -5O

0

r) -ch

4-q

4- E o

0 0 UVE

~ 0

Q) &-05 L

:3 aC 0CL) CCU)-)t)

Q-)

Figure 29

33 P

improvement after 220 predictions. Clearly, the evolving finite state

machines now primarily predict the zeros. Figures 30 through 36 show

these seven experiments in terms of the size of the machines. In every

case, there was a rapid build-up to the imposed limit. Conceivably, a

higher limit would have allowed for a greater predictive fit score;

however, in the limit, the results would be the same. As the machines

increase in size, each state would correspond to a given symbol in the

environment. Eventually, there would be a steady state that predicts

only zeros. Figures 37 and 38 show the mean and two sigma limits for

these seven experiments.

Altered Mutation Variation

Ten experiments were performed to examine the worth of altering the

probabilities of mutation in order to improve the effectiveness of the

evolutionary process. Here the same binary cyclic environment was used

with a thirty percent chance of adding a state and ten percent chance of

deleting a state. Figures 39 through 44 indicate the cumulative percent

correct. In these ten experiments, the evolutionary process generated

five perfect predictors. As stated in the previous Progress Report, no

perfect predictors were found in thirty such experiments. Figure 45 -'.

indicates the manner in which the states bulid up in these experiments.

Figures 46 and 47 indicate the mean cumulative percent correct scores and

two sigma limits, respectively. The average number of evaluated offspring

was 3,238.8 with only a slight variance.

Predicting Cycles within Cycles

A "double obverse" environment was constructed consisting of ten

repetitions of the above binary cycle followed by ten repetitions

of 223445, this being followed by another ten cycles of the original

binary sequence. Offspring of equal worth were saved. Five experiments

were conducted demonstrating early learning. The prediction score then

.34 'p£

. . a---a

Ul)

Q) 0 4l

0~ 4.

00 CUCD - i*0. IQ) Co- Ca) 0-

0a 0 0

oc 0) ) C eCO~L c-0 C

0 4-- (1

c

0~ E

E1 - 0 M

Cfl

0~1 4.,

0

U')

E

ciO 0 O t 0Qr) to *

Figure 30

35

51)

0 0

o CLC0

01 00Q0 (1

0-

0 o CL;COcio 0

cn EoC

.I-J 00Q)

C 0

L 0O

0 E:3:

to) to 00

IFigure 31

36

UC)ciC)

00

0 0

N 0- ) CCJ) C) U/

C)C

c Ca)

CoL

4- o )-

0 C C) C .

0 -1

0 C

CLO

ot

-37

- 0 Z

o CfciO)

C

0 o-0-

LO0 r)4 0

0,QQ3 0'

0 0)0 4D

-) c cE- C.,

~ %

4-4-

0 0

r 0

zl 0

0~~~ to to o o

Figure 33

38

.N.

0CLC)C0

N to)

D ) 0

00

U))

0 a)4

U)) 00DL.O

V)o

0'0

c 0 %0 0J U). CN.. 0 -0..

-E o E%r- 0 =3

0"

4-j > E flp4-' -4- V

0' ID 0-4-

0) E

:3:0 LO 0 to' 0 LO)0) to C*4 T

Figure 34

39

'17 -9 7- Id- VVO T -- w -z W... I

es

ci.)(D 0o C

Cr ---o-- U-)

C .0

T- >

0 L

0 0O4--

0 0Ca lCD Q) a)) .2

o~ 0.

0)) EL 0 1-- C

C 0 ch

4-

4 -1 4 - E r.

0 0 =

0 L0

ci.) E

ci I) LO a t

V,

Figure 35

40

II

U')

41 NC CI--

00

I> 0 00

00(

C L at Cla• -5I

.a 4a

0 C)

I 0 0, -U . , )

S .

(Ic)0 0) o O 0 LO 0o

Figure 36

41

-7 -W pIW MV1W -I - %IIMTlWfPtP

I) U'0

N

0 c0

E

I -6 00

00-

X IC)

0 0

Q) 0 0U 0

Q c) 0

IU~)

1 ) 00 r-, to O 4t to 4 0-

I Figure 37I 42

Cl)ci)

Q)E

N LO0)0

NNC.) 0

0

1 0 U)

Q) L

CC\

Cfa)

'443

I0

00

o0

0IEE-_o.2 Wd- C:-

1Q) r") O4 " --Cl) Q~-J (D~ 4 0L

*C, 0 +. 0 0 .

C:) _no-- o G

*- 0 a) C:L*M oQ) 0

C 0o) c:-)oc

C.))

0 ~~~0 0~~'

00I0

II

0-4-' 0o 0I (N

o 0c -:

-40 0 K

E 0 U)E c

Joo

0- L-4

I 0)

00 0

to

CY, 0 d

Figue -

0 05

00

-4-' 0

o 0

:3:0 v1 I*- 40 b

L o'

E~ 0 -z

a)U Q)~vb

0 (9 0 a)Q ) J-f- (1) u a

0)- (1 -.. > 00

E4- 0)c )Q

-4. - - () 0

C C)

o 0NL.U

0 C)nN C

Figure 41

46

000V-

C14

c0

1,, 0

I l 5-

0 0 0

I Qc) V

I " 0

I 0""

I "- '.

Figure 42.'

47 -

.C.)

4I.

0.4-j

0C 0N

o Lrw

->- 0

o0(

-44-j

D C)C4-

0o 0a)W

oC 0 0 Q~*

0 0

CC

-448

oC

4-a4

,4

0~ tz'~*- c

0o 00

C:)Go .. =

E~

oo Z

oq CRLo2

Figure 44

49

0

0 .1

o0

o0(

00

LO ) c -0.4 0

Q) (L) .- n

L-- -4jV ~ :

CD0 - 5 0 -,00 (nOZ)o

c. E0.

0 V2C 3 C__

.jI (1 0 ) O

0 I.) 0If) 0 If) o

0))V

Fiur 45

05

0

> °0

0 C

0

EII

CC0 0-

CN-SL_

I0 z :CI 0

0.

L-QL- . .,0 E

00

0 C

Figure 46 4I .. a51

O

40

0 0

CC'

0

23

•0 Ei o

.4J4a

o V

Co

0 00

l Figure 47

52

NUN IA4

I

Ireflected difficulty in learning the second cyclic component of theenvironment, see Figures 48 through 52. Saving equally worthwhile off-

Ispring leads to a rapid increase in machine complexity, see Figures 53through 57. It is believed that the added complexity that results from

saving offspring of equal worth acts as a buffer to the coding that

predicts the original environment. As expected, therefore, when the

original environment was re-introduced, the evolutionary process quickly

redemonstrated its previous ability. Again, note that an increase in size

can markedly slow down the evolutionary process when it encounteres a newI-environment. Also note that the initial learning of the first sequence

was slower than previously demonstrated. This was due to the fact that the

evolutionary process "believed" it was dealing with a six-symbol envir-

onment instead of merely a two-symbol environment (the initial machine

I was experssed in a six-symbol language).

Similar experiments were also conducted wherein the cycles were allwithin the same four-symbol language. Here the sequence 0132 was repeated

fifteen times followed by 331022 repeated fifteen times with a return to

the original sequence repeated fifteen times. In the first experiment,

only offspring that were superior to their parents in predictive fit were

saved. Here the evolutionary process discovered a perfect predictor

of the first cyclic sequence at the 20th prediction, see Figure 58. 348

offspring were evolved to yield a four-state machine shown in Figure 59.

Note that only a single state machine is necessary to predict this firstsequence. After the environment changed at the 51 t prediction, the

evolutionary process showed a sharp drop in success but regained to a

- prediction capability of five correct out of the six symbols. At the 14 1st

prediction, the original environment returned, and a perfect predictor

was still evident. 200 predictions evaluated 6,585 offspring. Throughout

I the experiment, the machines remained of small size (three to four states)

because here only offspring superior to the parent were saved.

IIn the second experiment, the offspring were saved if they scored

equal to or better than the parent. Here again, the evolution indicated

53

Im

E~ 0

0 0- - - - - - - - - - C

> 0

lo

0 jC%JU> t C

10 0 .0

00 C:)

0 1% c EU') 0

0 cl0

C4J

N4. -4 a

~~0

J 0

Figure 48

54

r 0

- _ 0

0CI II"4T N

CC

0 N

o C

0 :-

C)

CD 0

C) S-00

N (1

co 0

:3 C)

I Figure 49

I5

I U)

1 C:

000CDIN

01 00 igoo

I U) K00 W.

0(

>) ..-

.0

0''

IQ0 a (01 '

Figure 50

56 .

1ini

_ _ 0

c C4

C)~

4-'

> loo of-4 Cl)

C-4-

*00

C C)0 - CL

ICC

00t

ClC0

Figure 51

57 5

IU

0 0D

IL qJ N

0> C))

0~' NL.

LC C)

cn0 C:)

x 0 _

C,4,

0n o 0 I)

I Figre 5

x5>I0-% 0Gq~ '% : % . I '~.~ . .~ * *. . * ~ **0.

0.

U) 0

_ 0

0D IIq0 I N

Occeq 0-:

(01)00

0 >8010

-00-

C) i.~-.C 0

C)C

L. J

0 o 00t oC,

Figure 52

59'4

C/i) 0

0o4- CojCIN

CfllC)

0 CC)

04

,-.. IC)C)

C~4CO

C _)

/3;

0 LO inLC 0 0OCN 0 "

Figure 54

60

U) 0

.0

cL0 C'4

Oo0- ~0

0 0N00(1) 19

0I V) C:)v- 1 0

o

_0

-A

U) 0 .

>20

0 0 o0

o

Figure 55

61

4--J

CL, Cl

0 10

OO6 0

0

L) 0 C

Q) x 2:

U) #- - C

0 T- o+)

E 1N

** CO 0 Lto jo CN

II

Co 0

Io 10~OGo

C)M

'0

j) C4- 04

0 C

~-0

0f 0) 10 L_0

00 z0 1

000

T-' 10%

o - 0 .. 0 .00LC0 -

u~~j Go

>iur 57o -W 63

U)1

L-

OLo

0-

0o

~LO (D

o 5'

0A

00

LL-I

leFiue5

E6

C4,

.%

Figure 5

65A

1 superior learning ability perfectly predicting the cyclic environment on

only the eighth prediction (having seen only four cycles) see Figure 60.

However, the evolutionary process had great difficulty in learning the

second environment (due to the added complexity gained while learning

II the first environment). When the initial environment returned, a perfect

predictor was still in evidence. The size of the finite state machines

quickly grew to the upper limit of twenty-five states. This experiment

tends to confirm the notion that large machines slow down the evolutionary

process. It is useful to compare the above results to the classical

zeroth order prediction of the same environment, reference Figure 61. Note

its extremely poor performance.

Crossover Experiments

J.H. Holland, in his book, Adaptation in Natural and Artificial

j Systems, 1975, proposed the use of genetic.operators. He believed that

simulated evolution could be improved by drawing an anology to sexual

reproduction. Specifically, he suggested that two machines be "crossed-

over" through a substitution of states. A large number of experiments have

been reported by Holland and others on the use of this technique...

without their being any reference to finite state machines as being the

embodiment of the evolving behavior.

Several experiments were therefore conducted to investigate the

worth of this proposition. The first fifteen experiments involved the

same binary cyclic environment and used a two-state crossover, that is,

I: two arbitrary states of the best machine was substituted for two states

in another machine of similar size. The probability of this crossover

was chosen to be fifty percent with the remaining probabilities uniformly

distributed over the five modes of mutation. Figures 62 through 67

show that the best experiment discovered a perfect predictor at the 63rd

i prediction. 2,976 offspring were evaluted, and the size of the evolving

machines rose steadily. The worst experiment did not discover a perfect

predictor in 196 predictions. 3,018 offspring were evaluated. The

I 66

I

'- - -U ME~M ff -.- ~ VRW~ m z~mw TWx w- 'RI -KV Wj - 7r -i V.Zu,. -S KV %v ;,,w-.-

Zx C0c\4

Q)CL c

'4-0 C)00

o 0

(1 0

V) 0J 04--

C- 440 0

T> N 6co 0

Qo 10 ojjU U) K

0C 1

fn 4 0

CC)U)C)

Figure 60

67

4-j C

Ix

EI 0

0-

0 10 CL0 0.o U

1x -o

00

0 L~.

I E.

D U3

(I) gur 61n~j& 18

CNN

00

o CNI 06C&n0 to>

= -W 0 DC

CI L- 0

00

'3)C)L4-

0 -C

04- 0 D)

al)

C,,

0

E) C)

E C

000 0 0 0 0 0 0 0C'0 OY) 00 rN* C0 U) 14- 10 cq

Figure 6269

-4- .__ ___0_

0

00

00 0x

'4-E= 0

0

0N 0

F i u r 63 LA -

700ca,

Ic

0Aa,* t% %~. ~ ~ -~ " ~w . .~ .E

a

0) 0,

0 0

>1 04

C..) CLie%

00

0 CDo Q.0

00

c 0 0 00000000in- w 0-

Fiue6

71E

C) 0CU

0 0

0- oo 0CU -4-

0 0)0 E -W

Q) 0J 0L1 0 I 0 VQ) >I 0 .

o ) 0 9-

c 0

- D E cn.c n = 0

0

0) L)

> E(U) CD Zq-i

00

0 0

.. .. .. .. 00 0 0 0 0 0 0 0 0 0 0

Figure 65

72

c*Iq

0

Q) 0

E VU 0o U L. C') CD

x ~~i .0 .oL o 0- 4) ) V

x x r

U - r 0 0

E mQ~z- 0 CU L

00

Q) 0

0- 0 L-

0)Q

>) E

o0

C) 00 1*a)LO q, n C

Fiur 6

E 073

U 00%1IU

0~0

0)

o10 0~0 :0

0 0

Fiur 67M 74

increase in machine size was similar to that previously demonstrated.

While the mean worth (predictive fit) of the process consistently increased

(Figures 68 and 69) the mean percent correct of future predictions was

almost identical to that without crossover; see Figures 70 and 71. On

the average, 3,001.2 offspring were evaluated in 196 predictions.

Crossover was also examined within the context of the I.Q. test

environment. Twenty experiments were conducted, each performing 85

predictions with an all-or-none payoff function. Figures 72 and 73 indiate

the mean cumulative percent correct and the two sigma limits. After the 40th

prediction, the process properly predicts all the zeros, although with a

significant variance. However, Figures 74 and 75 show that the predictive

fit variation around the mean is very narrow. All machines tend to fit

the previous history in the same manner. There was little difference

in the results with and without crossover. ,

The environment was then predicted using the asymmetric payoff function.

Ipredicted0 iiI ____

actual 0 3 0

1 0 10

Figures 76 through 79 show the result of eighteen experiments. The mean

worth of the abilities to fit previous history was slightly worse than that

without any crossover. The poorest results asymptotically fell toward

the expected value of the environment. The experiment with the best

results showed some ability to predict the early one's.

I75

L

C,,)

00

4-j4

00 x

UJ~ 0 Co (0

o 9-4-a4-, V2

0

L.0

100.40 U4

Q)

E(fl . .- D(

CDo0: mn 0 ,, 0 O ", t

Figue 0(~) 070

U.). %,

*%Y -wVIWVqM WI

C-) -j.

4- C

C~ I-

0 0

CL1 CN

V) D

0q ......... 1

:Fiur 69

Q) 1..77

.1>1

otx 0

0 D0)L

> 00 UD

E C.)

Cl) C- C)0 0 0

n C) m0 -0DC

Fiu) 7_ C78

0

00

00

0~

-4-, 0

00 E3

0.. 0U

U) E)0 00 m6o 0 S

4I -4'

L CLo) C C

Figure 71

79

0

o D0

o C0(I)

_ (0o--0

LL.

0 0 Qg

L- - 0 )0 4- .

1) 0

L- 0C) L)J

-~80

Elot

'./

0-

"on

oL 0

0

En .E_o 8 ...

0 0

C-C

CoL

Figure 73 S ,

01%L.81

0

r- c-,

C!):

0

(L) x U)

0 JU) Lfl

U) 0)U

0~0

C)00

CL)

ci TO

aFiur 74-c80

I

IGo

0

I- o _ ______00

LO,

oO

00

CLo

I 110

I. 0

I,

64.

I

Figure 75

83

I "2

0-0 (

n c - U

.4-" CL4-S-

4- a -

0~ 0 :Ecf) I ) 0 '4

(n

4I Cm

E 0

o) al~C ~'4 )~~

I Figure 76

84

.I0p

o

-00.

0)0r-

-%

0 c

o 0

Ile.

Figure L

1.~.85

o.

00C) 4

0.

U)

Q))Qo C

o p

0 (D

0

00

oo w*C.

3r.-

4-4 c

0. . .. .0.. .... .

C) c 00 -, t Ln t n.

Fiur 7

0(386

(f)(I)

o CEC44-

C) (1 '4-0

(a)D

0 VI)

0.- ) o

0(

oLn0

0

00

0

0 0

(0 N C0CL)0 0 M No W- co ut n N 0-

Figure 79

87

As pointed out by Dr. Wirt Atmar, Holland's crossover technique

does not resemble the manner in which recombination occurs in nature.

Sexuality in natural organisms results in the recombination of alleles.

Alleles are defined as one of two or more alternative genes that control

the same characteristic and occupy the same place on similar chromosomes.

For Holland's technique to be effective, it must be assumed that a finite

state machine is analogous to a chromosome and that each state output

and state transition is analogous to a gene. But, this is not correct.

Genes contain the instructions for generating specific physical functions.

They are much like subroutines. Here a finite state machine could be

thought of as a gene. However, the structure of a finite state machine

could never be thought of as a gene.

As pointed out in Elements of Biology by Charles K. Levy, Addison

Wesley Publisher, Reading, Massachusetts, 1982, changes by crossover are

not true mutations "since neither the amount nor the function of genetic

material is altered." Under Holland's crossover, while the amount of coding

is not altered, the function of the coding is greatly altered. The result

of this is a near-random search throughout the entire space of possible

solutions. As the size of the machines grow larger, the number of

I states used in corssover increases. As shown in Figures 80 through 85,

when predicting the cyclic environment 012345676532210, without loss of

generality, the evolutionary processes which do not use crossover perform

significantly better than those using two-state and ten-state crossover.

Holland's technique effectively destroys the link between parent and

offspring that is necessary for an evolving process to succeed. In the

words of Dr. Atmar, "While his original thought was in the right direction,

the technique he promotes is at too severe a scale. True sexuality does

not destroy (nor create) information; it only shuffles contending subroutines,

eventually trying almost every probable combination of those in existence,

retaining only those combinations found to be most beneficial.

Even at this far milder level of informational reorganization, evaluating

I the "costs" of sexual recombination in producing inappropriately behaving

1 88 5

.5

o 0

0-

V') 1 O

OD. r- L...0

Fiue8891

VV

C,)

I 0

0 U -

CLCx~1

G) cc

CD E

(.0 0NE

(0 a%C) V001

2 cn- 00t- D LO 1 n

090

* l-

U0

Go

(00

ro

(00

01

9~ (n

I

0 CL

Q)I .4,re-8

'I I'-~ * ~~91

05 0

r-I-

0OW In

c% 0

(.0

1 0

0~ (3)

(D 1,0C-

5 0) 0 W - 0M

FigureL8

v92

to C:

toc EnC) q6*

I- LI) 0IJ-

0 0

% ol4) 0

IE E

U) )

cn U) )1

0- 0

I C-L

CY))

0) OD N -O L)r 4 0755 *ID

Figure 84

93 4

0

LO)(D

1O c' 00 10

o~% ofoI -

4) co40 .

0 0

0 z

0)0I0U 0 --

C 0

070 0 N a O 10 0I 0 V0

I Figure 8594a.

organisms has been a topic of substantial debate which has continued on

in biology for some time now. The debate is currently being resolved

in the following manner: the costs of sexual recombination are no longer

considered severe because (1) the contending alleles (subroutines) have

generally been found to be quite similar, and (2) there are not nearly

as manny alleles contending for each site as was originally thought or

predicted.

Sexuality is not a mutational phenomenon. It is, rather, a method

of rapidly sorting through the effects of mutated subroutines. The

advantages of sexuality in an evolutionary optimization program are not

to be underestimated. However, a very specific structure must be in

place before these advantages are obtainable."

Identification Experiments

As previously indicated, prediction is key to the process of

identification. Suppose an unknown entity is responding to known stimuli

in a clearly observable manner. The task is to develop an explicit repre-

sentation for that transduction, this on the basis of the stimulus/response

experienced to date. The predictor observes the sequence of stimulus/

response pairs, the most recent stimulus, then predicts the next response...

this process being repeated as each new stimulus is observed. The

predictor must develop a most suitable logic in terms of the specific

payoff function and span of prediction. If this span is the single time

unit, the discovered predictive regularity should correspond with the

underlying logic of the transducer. Evolutionary programming provides a

basis for such prediction and identification.

To demonstrate this hypothesis, a series of pilot experiments were

conducted. Success in this regard might justify more formidable experi-

ments on the Cray computer. As a first step, the transducer or plant is

95

.4".'I

deterministic and fully controllable, that is, each of the possible

response symbols is generated by each of the states. Plants of increasing

complexity in terms of alphabet size and number of states were used. Once

a searching prediction score has been obtained, the predictive logic is

compared to the actual transduction as a test of the identification. In

these experiments, an all-or-none payoff function was used in that all

symbols are equally important, and no credit was to be given for nearness.

To ensure adequate exploration, random input sequences were used to drive

the plant.

The first experiment required the identification of a two-symbol,

two-state finite state machine, see Figure 86. The evolutionary process

properly identified this finite state machine at the second prediction,

this after evaluating only thirty offspring.

The next experiment required identification of a two-symbol, five-state

machine, see Figure 87. Here, in 254 predictions, the evolutionary process

correctly predicted 80 percent of the responses and had a predictive fit

score of 90 percent, see Figures 88 and 89. In all, 32,376 offspring were

evaluated. Figure 90 indicates the final evolved machine consisting of

six states. It is tempting to compare this machine with the one shown in

Figure 87, but this is dangerous. The addition of even a single state

1changes the meaning of all other states. There can be no simplistic state-

to-state comparison. Note that a fully controllable finite state machine

driven by random input in a binary alphabet has a fifty percent chance of

responding with either of the alphabet symbols. The predictive score

should be fifty percent. Clearly, the evolutionary process does

significantly better.

In the next experiment, a two-symbol, ten-state machine was the

unknown transducer, see Figure 91. Here, the evolutionary process was able

to achieve a high predictive fit score, see Figure 92, and predicted

extremely well, see Figure 93. The final machine was of fifteen states,I

I 96-V'

.. qf ly 't € ". . ... ' . " . . .. , . .... -,-.€ € ,- . -. .

JR

o co

~~Figure 86 "

I 97

I

IIiI

I I.r

an)

IIIi

I-o

Figure 87

98

*,j

Cq.

0

0 41

10

Go3

1 04-

1 .99

100

0

0

co 04) cN

0 0Cf

LU

I 0D

1.2 - OL) * 0

Fiur 89010ol

TI-Tv gIlrprp-.M rwt

ICIcI w

I0I0

C)C

Fiur 90

10

.

0#'0

o

0 CD

0

N

{ ..-.

f,-'-.

I5- V.,

o 5**.L,

-S-

- k ;..-.

.-,...,," ,4' • . .. 4' * " ',, '.- ,,.'- -',i '.. ..

"*. °-" ,, " ., . .'. ... -,.,-.'w,-. " " " "''' ''" " J' . ' "" " " "1 .'"" .

-0'

cI4.J 0

U C)

"' 0 N ,

o 4-. 4 0

Q)0 L)

1.-

4... . L

00.4-- 0.. 9-

0o~

0 0

r- -- 0

Figure 92

103

Uf)

0

Cr)

0'44 0 0.r

U) %o o C0

0L _o LL. v- : ::

2 < 0 o

0 0 0 0 0 0 0

wx

Figure 93 ,104

" 00 0

while evaluating 8,040 offspring in the 290 predictions. Note that

increased complexity of behavior may not be simply determined by the number

of states. Here the plant has a relatively simple structure which was

evidently discovered by the evolutionary programming. Presumably, further

evolution would finally discover the computer logic of this machine.

Next, a four-symbol alphabet was used driving a five-state machine

that represented the unknown transducer, see Figure 94. Four experiments

were conducted; the percentage correct prediction ranges from 36 to 50, but

in every case, this is better than the expected 25 percent prediction byzeroth order statistics. Figures 95 to 101 show the predictive fit score

and percent correct for each of these experiments.

An eight state machine was then used as the plant, see Figure 102.

Four experiments were conducted. Even though the size of the machine had

been increased, the evolutionary process was still able to predict 30 to 40

percent correct, see Figures 103 to 106. It is also of interest to

examine the predictive fit score for these experiments, see Figures 107 to 110.

The 60 to 70 percent worth indicates that if the evolutionary process had

been given more time between predictions, superior response prediction

could hve been achieved. In each of these experiments, there was a rapid

increase in the size of the machines, see Figures 111 to 114, even though

equally valued offspring were not being saved. An average of 11,422.5

offspring were evaluated in making the 290 predictions. The final predic-

tive machine of the first experiment consisted of ten states, see Figure 115.J

Finally, two experiments were conducted using an eight-symbol,

five-state machine as the plant, see Figure 116. Figures 117 and 118

indicate the precent correct predictions to be 20 to 25 percent, this being

twice the level that would be expected on the basis of a zeroth order

prediction (12.5%). Figures 119 and 120 indicate the predictive fit score

for these two experiments. The results range from 45 to 50 percent worth.

Evidently, five generations per prediction was insufficient. Greaterexploration before predictions would have increased the predictive fit

105

I

ICIL

I%

NO

C%

Figure 94

I N10-

0 ~0

Ct) 0NL. d

UO c0 4

(A) N0 gel

IIN 0,-L 0

000CC

0 c In0~ E4

00

0 0 50

0

t00

0 Q0

04 0

Cf) ULr) 0

0- Z-

1 0 U,

0N

(fc0 asC * t n W

~10

-4C ) 01

00

CLC

U- N 0

-~ 0

W C:

.00

Qo r- D n v 9

10

:4,

: %

L-V

V)Q)

00

UL) C N4

01 00

I.f)

00

-CCq 0ogo

0 CLJ

0 .p.

0 0

'4-1

Coco' I 000

(7) 00 P..a n V

Figure 98

110 p.

. 1,

00

oI o -

00

) .

og

r,'.

8 ,

ro

7-30)

LLLa

. ....N-EN

S-O) 90 t*. (D ,) 1 .-

Figure gg99;

';.

i ) _

i0

I 0

C CDU)C0

I CL0

ul)_ S)-

U) N

00.Q

0

L..

, , , " , , R"- , - , , " . . - . . . •, ... , W .

Ig e1

1.112

4. 0

(0 0E~

C-C

0N

0 Q O 00.

:11

C14

cliv

V4 S-

044

Cl LA-

CIj %--a

114k

PS

!

i CT)

.000 U, 0

,) . IN

¢N

4--

I Ia0 (n

q, c

I C-

"o b

00'

0>0 )I: LU)

pE

I3 0 .0)

*LI

0

o 00

00

0 LO n N

Figure 103

115

(j 0

0 e0

0 03

C) 0

I N

Fiue 0

11

Q) _____

-4--i 0

00

00 U

4- NL!- -

q)):w4z

L

EE

0 >

0 03

a.

Figure 105

117

, s . 6. J ' LL- Ic W Wn~~ V JWLW ~ W.qjW ~

Q)-

0

i2 0

I N > _

C/)C. 00

I -- 0

0..gI!

0 DL Q

0 5,f

C Ll-

I)

118'

07 . 1:-U) 0

4-'

o b_ -

-0-

00

Q11N

0I D in qt W, 04

Figur 107119 P

00) 0

C,

0 C0

N 0

WOCC)

00

C)f

0 000

QI I c.. ..

I ) IX) Cl o t) 1* ?) N - 0

Figure 108120

C)

4-) 0

SC0 CN

0 C)

C) oL --

In -C -) 0

fL- 0 ON

01 LO c

0 1"4

Cl) -0.

I EC ~z

0 000

I22

Q)0-4--J'

0 00 1*IU) -0

00 00C

0oc

N 0

I - i

I-Co

-4--'4-

(1) 0

0 -0

I Figure 110I 122

Q!)

-0-- 0

0l 0 n

o 0o

-5

0)

0LLO-00I :3

0 >%

4-L

o 0o

@12

U)

0 Jo:3

CL)rVW C 1

-0A W 0

4- E,

o00

4-j-

0n

4U-nEE

0~U)

in L.

TJ N

Figure 112 '

124 '

00

O0 3

oo

U) L

00 ,0 -1"0>2:

L- n

00 n - z%

-4--'a z0 "0

00

0 vE*0 0 )-

Figure 113

125

;..~*, p.' ~ . -55*~* I0

V ~ ~ o• l . ,. : .,.¢ . , . ., '. .. ., . 'W .'r . . . a.. % . . t ,.j'. ... . .*.

, ,- ,,.., . ."

ci) 0

00

(f) 2 04c

nl0 oL C>: 0

CI- UI O-C

-00

I U) EC L. :3

0 WU--

0000

FiueI1I 126

w - '. -- I. - . . -v -

I-II4 I.

-CI

00

C144

IL-

12

- I

1C 4

%D -7N

C-4I

IIIr4

I Figure 116

[I128

4-Q) c

) C)

0! ccCLLUL

0~~ 5QC* a

C- E.

-

FC) 'U(L) Jo

I0

0 ~ ~ 0 r0 0 u0 0 r)c

Figure 117

ii129

U) 4LC) ___ _0

0

Co to

01-L4. A.s'0L-& 1 "A&

0 0 0 0 0 -0

C/Do 0 w -)

1130

C) _ _

Co*

Cfl)

Lu--0

LO 4-

U))

( -) C 0

3+J

a Figure 119131

1O 0 _ __

LL.7

IZ~ CO 500LC) L

I- LA--

E 0'

(IDCT

UJ 0J

I Cl) o

-o ~ ~ 0 Go r- Coi 0~ q, n N 0

I Figure 120

1 132

worth and the percent correct. Once again, there was a steady increase in

the size of the evolving machine, see Figures 121 and 122. The final

machines range from 16 to 18 states.

These experiments indicate successful identification through

evolutionary programming. The logic of the unknown transducer was

represented by finite state machines of greater size than required. This

is especially true with a larger alphabet. However, no penalty for

complexity was invoked. It should be possible to evolve machines of

similar size to the unknown plant, but there might be some cost for this

decreased specificity.

On the basis of these results, it seems worthwhile to explore the

identification of time varying and noisy plants through pilot experiments,then design of experiments for predictive identification using the

Cray computer.

III "

I,

133

Ii

133 e 'S.

U) C,

021

C!)D

L0 0 '-(n

-0>0E 10U

E

0440

4-3

0 -0In 0

C-a _

0 lo CL

~00.

0: JN 0$

II

I 0

1 -o0 10 0 O

I Figure 122

135

CONCLUSION

Evolutionary Programming has now been demonstrated as a versatile

means for the prediction of nonstationary time series and the identification

of an unknown plant. It is fully recognized that much remains to be done

to dimensionalize the program as a function of the presented problem in

both prediction and identification. Arrangements are underway to use the

NASA Ames Cray computer for such work. In the mean time, further pilot

experiments are planned to guide these larger-scale demonstrations.

It is now considered appropriate to explore the application of

prediction and identification to real world problems in a preliminary

manner. Medical, financial and other data sources are being reviewed in

this regard. It is of particular interest to focus attention on situa-

tions wherein the least mean squared payoff function is notably inappropriate.

This is surely the case for epidemiologic and economic time series where

equally correct predictions are not of equal worth, and the cost of a"cry wolf" error is clearly different from the cost of a missed catastrophe.

Plans are being made to move from identification to control through a

series of pilot experiments. Success in this regard might well influence

the design of weapon systems.

II:

I 136-9

I.

II%

I.

I

e[ AppendixI!

AddressingThe Traveling Salesman Problem

Through Adaptation

13

[II

iI1= 137

I ,

138

ADDRESSING

THE TRAVELING SALESMAN PROBLEM

THROUGH ADAPTATION

By David Fogel

INTRODUCTION

The traveling salesman problem has received a great deal of attention

In recent years. The task Is to arrange a tour or n cities such that each

city is visited only once and the length of the tour (or some other cost

function) Is minimized. Here Is a simple yet recalcitrant combinatorial

optimization problem. For an exact solution the only known algorithms

require the number or steps to grow at least exponentially with the

number or elements in the problem. Brute force finding of the shortest

path by which a traveling salesman can complete a tour or n cities

requires compiling a list of (n- )1/2 alternative tours...a number that

grows faster than any finite power of n. The task quickly becomes

unmanageable.

Two recent papers1 addressed the traveling salesman problem

1- "Proedlngs of an International Conference on Genetic Algorithms end Their ,Applications," John J. Grefenstette, Editor, Carnegie-Mellon University, July 1985.

I A e

I

- ,;' , ,- < . , . , , ,, , , ... . < ., . ,...... , , . ,.. -,.,. ., .. , ...- .... ... '

139 "

through use of the genetic algorithm as proposed by J.H. Holland In 1975.2 ,

This algorithm Is an offshoot of the evolutionary programming concept

offered by L.J. Fogel In 1962,3 then demonstrated In his doctoral

dissertation4 and described In the book Artificial Intelligence through

Simulated Evolution.5 Here, Intelligent behavior was viewed as requiring

prediction of an environment then the use of such predictions for the sake

of Its control (at least to the extent possible). The logical process of

iterative mutation and selection of behavior Is simulated In fast time to

eventually evolve a logic most suitable for resolving the given problem.

The behavior of each artificial organism Is portrayed by a finite state

machine.., a mathematical function that does not constrain the represented

transduction. It need not be linear, passive, or without hysteresis. The

original 'machine" (an arbitrary logic or a 'hint") Is measured In Its ability

to predict each next event In Its "experience' with respect to whatever

payoff function has been prescribed. An offspring is now created through

random mutation of this "parent" machine. It Is scored in a similar manner

to the parent In predictive ability. If the parent Is better than Its

offspring, the parent Is used to generate another offspring. If, however,

the offspring Is better than the parent, the offspring becomes the new

parent. This assures non-regressive evolution.

2- Afttntion in Natural und Arttficial Systems, J.H. Holland, University of MichiganPress.Ann Arbor, 1975.

p.'.

3- "Autoomos AutomrAa," L.J. Fogel, Indusfrlal Rewrch, February 1962.

4- 00n t Organlztion of Intellect," L.J. Fogel, Ph.D. Dissertation, U.C.LA. 1964.

5- Wtlfi&Sa IntellwnM thro Smulaed Evolution,, LA Fogel, J. Owens, M.J. Walsh, JohnWiley& Sons, Now York, 1966.

2

140

An actual prediction Is made when the predictive fit score demon-

strates that a sufficient level of credibility has been achieved. The

surviving machine generates a prediction, Indicates the logic of this

prediction and becomes the progenitor for the next sequence of progeny,

this In preparation for the next prediction. In this way, randomness is

selectively incorporated Into the surviving logic. The sequence of

predictor machines constitutes phyletic learning... idcv

generalization.., generation of new hypotheses concerning the relevant

regularities found within the experienced environment, this In the light of

the given payoff function. Prediction can be used for the purpose of

Identifying an unknown transducer. Feedback of the evolved model then

forms a basis for control.

Rather than describe each organism only in terms of Its behavior,

Holland chose to evolve the code which generates such organisms.

According to D.H. Ackley,6 Holland's genetic algorithms search a parameter

space where "any point in the parameter space can be represented as an n

bit vector. The technique manipulates a set of such vectors to record

Information" about the parameter space. "There are two primary operations

applied to the population by a genetic algorithm. Bacrduclion changes the

contents of the population by adding copies of genotypes with

above-average figures of merit. In addition to this...It Is necessary to

generate new, untested genotypes and add them to the population, else the

population will simply converge on the best one it started with. Crsoe

Is the primary mean of generating plausible new genotypes for addition to

the population."

6- 6A DornectionstAlgorithm for Gerstic Seerch," D. Ackley, in Procemo/ngs of wnInternmionol Confaranoc an ntic Aprfthms and Ther Appl/cutions ,

J.J. Orefenstett, Editw, Crnegle-Mellon University, Juy 1985.

3

" - ,- . . ..'.. .'...--. ... - . .

141 'n

As defined by Holland, crossover takes two structures,

Al=al lal2...aln and A2 = a2 1a2 2...a2n, and at a random point x between

I and n, exchanges the set of attributes to the right of this position

yielding offspring of the form: A* -a I I a 12...a I xa2(x* 1 )..a2 n Ackley

continues to state: "This 'offspring' is added to the population, displacing

some other genotype according to various criteria where it has the

opportunity to flourish or perish depending on its fitness." "Iutatil

provides a chance for any allele to be changed to another randomly chosen

value.* "if the mutation rate is too low, possibly critical alleles missing

from the initial population will have only a small change of getting...into

the population. However, if the probability of a mutation is not low enough,

Information... will be steadily lost to random noise.' $,

Holland likened the actual code being mutated to that of the genetic

code that defines a given organism. While Fogel, et al. only used small

degrees of "background" mutation, Holland examined the operations of gene

crossover" and "inversion" among other actual biologic genetic

recombinations. Although Holland's work went largely unnoticed for some -

time, today a great deal of attention is being given to genetic algorithms.

At the above referenced conference D. Goldberg and R. Lingle, Jr,7

offered several observations:

1) Simple genetic algorithms work well in problems which can be

7- "Alleles, Loci, and the Traveling Selesmm Problem," D.E. Ooldberg, R. Lingle, Jr., InProVnW ofan I,ntrnah'onl Conforsnc on Onwfi Algorithms and heir

Applicotons ,. J. Orefwtette, Editor, Carnagie-Mellon University, July 1985.

AI,

142

coded so the underlying building blocks (highly fit, short defining

length schemata) lead to Improved performance.

2) There are problems (more properly codings for problems) that are

GA-hard -- difficult for the normal reproduction + crossover +

mutation processes of the simple genetic algorithm.

3) Inversion is the conventional answer when genetic algorithmists

are asked how they Intend to find good string ordering, but Inversion

has never done much in empirical studies to date.

4) Despite numerous rumored attempts, the traveling salesman

problem has not succumbed to genetic algorithm-like solution."

The authors then suggested a new type of crossover operator,

partially-mapped crossover (PM)," that would lead to a more efficient

solution of the traveling salesman problem.

Specifically, consider two possible codings of a tour of eight cities,

A, and A2 , a return to the Initial city being Implicit:

A,:3 5 1 2 7 6 8 4

A2:I 8 5 4 3 6 2 7

PMX would proceed as follows: Two positions are determined randomly

along the AI coding. The actual cities located between these positions

along A1 are exchanged with the cities located between the same

C ,-(.9-..9*9I. n.. ,(VV % .~% %'V.1. V* .. , ,. .. ' .. '"

143 '.

positions along A2. For example, if the positions three and five are chosen,

the sub-coding along A1 Is 1-2-7, and the sub-coding along A2 is 5-4-3.

Each of these cities Is then exchanged, leading to the new tours, A* I and 4A*2: ';

A*I:7 1543682 82 i.

A*2:5 8 1 2 7 6 4 3.

They reported two experiments on ten cities where the PMX operator

enabled the search to efficiently find either the absolute or near optimal :.

solution. Goldberg and Lingle stated that this operator was more complex

than "simple crossover,' as proposed by Holland. As will be shown, In fact,

It Is not.

In another paper by J.J. Grefenstette, R. Gopal, B. Rosmaita and

D. Van Gucht,8 Hollands "simple crossover* was utilized. This required the

formation of a special coding structure. Clearly, using this operator on

two valid tours could result In an offspring' that was not a valid tour. As

A.K Dewdney 9 related, the authors' method for devising the appropriate

coding was Ingenious.

8- "Genetic Algorithms for the Traveling Salesman Probler," J.J. Grefenstette, R. opal, B.

Rosmalta, D. Ya Oucht, In Prxeings of n Inernatoal Conference on Ofnetic

Algorithms end Their AppIhwtions, J.J. Orefenstette, Editor, Carnegie-Mellon

University, July 1985.

9- 'Computer Recreations: Exploring the field of genetic algorithms In a primorilial computer

se full of flibs, AK. Dewdney, 5cientific American, November 1985.

6

144

"The representation for a five-city tour such as % c, e, o b turns out

to be 12321. To obtain such a numerical string reference Is made to

some standard order for the cities, say, a , c, d, e Given a tour such

as a, c, e, d, b, systematically remove cities from the standard list

in the order of the given tour-, remove a, then c, e and so on. As each

city Is removed from the special list, note Its position just before

removal, ais first, cIs second, els third, d Is second and, finally 0

is first. Hence the chromosome 12321 emerges. Interestingly, when

two such chromosomes are crossed over, the result Is always a tour."

Dewdney continued further to report that unfortunately the experiments

with this representation were "not very encouraging." The authors

conducted larger experiments than those of Goldberg and Lingle, Including

50, 100 and 200 cities. In the three reported experiments, after a large

number of trials (approximately 14000. 20000 and 25000, respectively),

the best tours were still far away from the expected optimal solutions.

At this point it Is natural to ask "why?'. After all, the traveling

salesman problem only requires discovery of a logical pattern. This seems

completely analogous to what occurs in nature. If the crossover of genes

works in natural evolution, why shouldn't It work here? The answer Is, in

fact, as noted In observation '2 by Goldberg and Lingle: The traveling

salesman problem is GA-hard -- difficult for the normal reproduction +

crsor+ mutation. This is due to the fact that Holland's crossover

operation d nl mimic the biologic crossover of genes.

As defined by C.K. Levy, 10 crossover Is the phenomenon where "old

10 - Elements Bof l R }, C.K. Levy, third edition, Addison-Wesley Publishing Company In., "'

Reading M ssehusettes, 1982.7

145

linkages between genes on homologous chromosomes are broken, and new

linkages are established. Genes that reside on the same chromosome and

move together are said to be 'finked.' A linkage group is any group of genes

physically linked on one chromosome." Levy goes on to state: "Cangesn

linkage groups are not truly mutations, however, since neither the amount

nor the function of genetic material is altered "'

Crossover allows for different combinations of alleles. Alleles, by

definition, control the same characteristic and occupy the same place on

similar chromosomes. Holland's crossover treats the entire tour as a

chromosome and each city In a tour as a gene. While Holland's crossover

does not change the amount of coding, It greatly alters the function of the

coding! A more appropriate biologic interpretation of a tour would be that

it is itself a gene. Crossover inside a gene is a nonsequetor. The tour Is

absolutely nmL analogous to a chromosome and each city In a tour is nt

analogous to a gene. These relations are in fact anomolous.

The result of Holland's crossover, therefore, is a near random search

throughout the entire space of possible tours. Perhaps Dewdney said it

best when he stated that by using crossover "there is so much juggling of

genes and cracking of chromosomes that...(a parent)...is hard put to

recognize its own grandchildren." This is, of course, the very essence of

the difficulty! Adaptive plans must retain already made advances In order

to ensure that an optimal solution will be found. As the number of cities

grows larger, Holland's crossover effectively destroys the link between

each parent and its offspring. The results can even be worse than a

*-Underline addd by this "uhor.

8

146

complete enumeration of all possible tours (reference the Addendum).

AN ALTERNATI VE APPROACH

As an alternative solution, consider the Adaptive Algorithm, so named

because it does not include any of the pseudo-genetic operators that

Holland has suggested. In this algorithm, which is equivalent to

evolutionary programming restricted to single state machines, only slight

mutations are made to an existing tour by removing just = city from a

given list and replacing It in a different randomly chosen position. This

mutation is only slightly more complicated than the simplest possible

mutation...swapping adjacent cities. It is clearly less complex than both

the PIX operator and Holland's crossover. Through multiple mutation, this

single alteration can be made equivalent to either of these crossover

operators.

According to Holland: "If successive populations are produced by

mutation alone (without (genetic) reproduction), the result is a random

sequence of structures drawn from (all possible structures)." I1 This is

only partly correct. The Adaptive Algorithm does result in a random

search, but only in a portion of the space relatively close to the parent

that generates the offspring. This Increases the effectiveness of the

algorithm dramatically as It allows for the retention of advances.

I I- aiptatin In Natural mdArtlfcial Systems,.J.H. HOWlln, University of Michigan Press,

Ann Arbor, 1975.

9.....................~ ~ - ,t

147

But more is still required. Not only must advances be retained but

"dead-ends" must be circumvented. Because there is a finite number of

of fspring that can be generated through mutation, there might well be

stagnation on a local optimum. To prevent this it is useful to randomly

alter the adaptive topography that is being searched. This can be

accomplished in various ways. One of these Is to occasionally allow for

the survival of offspring that are slightly worse than their parents. In

effect, the scoring function is made "noisy." What results is analogous to

the searching of a maze; when a dead-end is reached some backtracking Is

allowed and the overall search Is reinitiated.

Unfortunately, the topography is much like an upside-down bed of

nails, with some nails being longer (better) than others. From any given

nail, it is possible to travel to any of n(n- I) other nails In a single

mutation. Thus, unlike a maze, when the evolving phyletic line reaches a

non-optimal nail from which no single mutation results in a better tour, it

zeis Impossible to determine the "direction" in which to backtrack. A given

nail can be reached In a multitude of ways; therefore, bactracking is

allowed in any direction. To further aid In the circumvention of stagnation,. . the degree of "poorer quality" offsping that are occasionally accepted

should gradually be reduced as the adaptive process proceeds.

EXPERIMENTAL FINDINGS

0Experiments were performed to demonstrate the effectiveness of the

- "Adaptiv Toporphy" refers to the wing function on the rypersps of poiblei "pims."

4" .

°.'.. 1n

148

Adaptive Algorithm. Initially, 128 independent trials were performed on a

24 city traveling salesman problem where the cities were positioned on I

the periphery of a rectangle. Clearly, the minimum length tour is equal to

the perimeter of the rectangle, here, 250. Of the 128 trials, 90.625X found

the optimum solution in an average of 5297.48 Iterations, see Figure 1, the

maximum number of iterations being arbitrarily set at 14,000. Figure 2

Indicates the results of the remaining 9.375% of the trials. Here, at least

temporarily, the evolving tours were trapped on a local optimum. Despite

the seemingly non-complex arrangement of cities, the numerous local

optima make this traveling salesman problem difficult.

The cities were then distributed at random. First, 19 experiments

were conducted requiring a tour of 50 cities. The cities were redistributed

for each experiment. In each, no optimum tours were discovered in 20,000

iterations, but It was clear that the evolutionary process was "solving the

problem." Figures 3 through 21 indicate the results of each experiment

Figure 22 Indicates a typical example of the evolutionary process

discovering more and more suitable tours as offspring are evaluated. Note

that "backtracking" plays an Integral part of the search.

Experiments were then performed requiring a tour of 100 cities under

similar conditions. Again, while none of the eight experiments found a

perfect tour, the evolutionary process performed well. Figures 23 through

30 indicate the results of the eight experiments while Figure 31 indicates

the reduction in tour length as offspring are evaluated.

%a.

IIII

149 1

c 0

00

NN

oco

o00

0~ c;

.E 0.

o "* . .%

to_

o 0 DoCL-

o w-nv

Fiir

*'r--i-. .L-- -

150

0/) 0

L. 0

C-)

0o 0

0 x

re))

lb.-0

E E " o

0.0

0 -

xC

0I

00

C- o o o o o 0 0 0 0 0 0 :

xn 0

F-

Figure 2

-* - ~V ~*~F *t*~U~1 - - * ~ * 7 77 .-34-r 07 ~wJWe~ ~

151

U)C

0

00

l CCE E0ci) CL.x0

0 0 -

_0

c 1.0 '0 C)It) -4-'

E DD p~U)

o -o

Q) E

C- C) M) P- (DO It W1 ) cq '

Figure 3 .

W--4-L ar- 16F. -. - -

152

0cN-

II0)

>41%

0~0

C

E cE

C- m

Ec0

L...~~ 1 0 co I,-- c~o 0f ' '

Figure 4

153

0 - 0

(0

ICC)

0

0)0

E 00CL) -p

00L-..

o U~cc

o 0)C % DL *K *

CFigur 5

154

0

'0E co

0o

cic

L.C) 0

oL00

C-,

otC.)

0

C. ( 0 0 0 0 0 0 0 0

0 h o ~ to Lo qt cq

Figure 6

- -.- ~ ~ .p F W sP . .y ~ ~ W r~r~w - grrx -V W -w

155

ICC

L-,J

0-

ot

or

UZ)

0 C)

Fiur

156

IC

C'))

00

LOLLJ

CC

0l

m 0

0.EoO

ol ILF)n0

E Figure 8

- ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~~~~- W7917.- Ti~w~1U'~~~ V ~T%.i~~~* '~~ L - ~~

157

'I-,

4 J 0

0 0

Li

00

cc0

C)'0)0

E

C' 0 0 00 O cOCO~- 0 C) 00 -~ ED U*) 1q t

Figure 9

158 LI

Cl)

LC).0

-.

C) CO

00

0) 0

E C)

cc

0 0 0 0 0 c 0 0 00oC 1

Fiur 10

159

00

0

aC)

Q))

U) c

00

I:' C!0

E co 0

o 00 00 000 00 0000 ) CO r*- CO LO '1 to 04 ~

Figure I1I

- - - -. ~ ~ *FJ'~ VINP'~~ '~W~* ~ h 1- V ~l ~ -~7~~

160

Coo

- 0

LC) 1'

E0o to

CC

Co C)0 0

0tr 12

161

U

LC)0

00

E 0o

00

o -0

0

CCE -'-a

0*

ci)

Qo c

0 CE 0

o 0 M

Figure 13

162

ClC)

0

C)

o ) 0

Q0)

LLJ 0

0 -

L... 0-C"- C)

0

E C)

o) C.

00

0

O 0 00000000 00 0

Figure 14

163

U)

00

0 L.J

QL UL. 0

coC

W 0)) 00

Fiur 1

--- ------ A5 AMU-WT1Wqtwtflt ~ P~ , MflTnt . r- 5.= -2~~' r pU ~ I 72

164

Cl)0C'~i3

09i0

LC)

IQCL -- W

0) 00 "0x

CLdL.n o,

(D =:

o>>4'00

o" o) coC L

Fi(g) 0 e

165

0)0

LCJ)C:C

Qi)

00LD0

Cl- Ja0

0

o 0C ~ b- '.

E U

cn

~1) C0

CC

Cf) 0

0~ 00 0 0 0 00 00 0'- 0 0 co r- to i C) ' to N) C

Figure 17

* -3-.. - a~ -F U~! . -~~ ~ ~ '~r~N ~ Mk ~ MW".q~VL L

166

4-4'

Cl))ci)

Q) Q)0-0

LCD0~

a))L.O

00

0 p

0 0

0) 0 0 )000C D0C)C0) OCr)( n

Fiue1

167

C'c)

00

C0~~~~- (7 'p, C U) 't t N

Figure 1

168

E 0

E- 0

0'

0

O0O

0o

0-00

0 m0 0

Fiur 20C, CN1

E "

> ", 'E

Fiue2II _______

169

4-j4

ClC)

ESCL L .0-

U-)

E C)-Cl)

00

E

0 C C O 0 0 0 COO

Figure 21

*~ZW7 -w .. .- 7~ 7 - . . -

170

U)

C.) E.

LO >0

Eg 0 U)- 0 4 -

0,*0

0Qbi

0

C!))Lo -'-

C /6

CDJ

0 C 0 0 0

Figure 22

171

Cl))00

CC

0)

C/)00

£~ L C

EEQ) i

>1 .. . W4U .. '-

0~1 0

- 0 0 - O.q

Figure 23

.7 V~sJ w:-j.rr

172

C) 4-0

1"*0Q) U)x )

0-

LO

0 >2CC)

ci)n00

0 0000c0 00 00 0001~ 0 0) CO r- LC ) -.0, n cN

Figure 24

- - -- - - - - - -' a - .,tru ~wL~tr'.lr'. fl.~. WSt 09~t-. *. - t, -u ,W

173

Cl)CL

0

A0

to

ci)

Q) -(0 0 0 0 0 0

00

Fiue2

.~ i. -u~ ~ .~ L ~ i~ ~w - ~u -. it J w r i-L ~WT~t U . i. t.Ir ~k ~~ZVI~.V

174

ci)

- C)

o UoxI

a_ 0

E

0(

Q)Q

F.mI.- I1 A IA0 0 0 0

CFiur 26

175

C 'C

E QQ) C)

CL 0L0o0

0 Q) 0C) to

C!);

E

w- toN- l

Figure 27

1 176

I Cl)

0 c

QI 0Q

-4-' 0

I I 0

CD~~ 0

LL.

Ir- 0

Eci) 0n

C!):

'I0 000 0 00 0 00 00

II Figure 28

T-M .- -A - --l 47 -j W

177

U)

00 ,0

oJ

N C

LU

-E co

0- 0* (D

0 L 0

cc(1) 0

0 0

* C) 0 ~

00

I-0 '-t

U)p

Q).

Figure 29

178

IkU)

000

CL

I CO0 d a) 1

00

0 0)1to

CL 0Q

E: 000

00

00

Cf)4

(D C

1 000 0 00 000 00L.. '- 0 M r%, 0 0 1* n' N '

I Figure 30

179

0> -o

cE0

0 L

C*4 0

0 c0 4J

0C 00 _

004C00

0' 0

-0

U0 000 0 qt tn n NV0T

Figure 31E

180

Another experiment required a tour of 90 cities. Here, ten groups of

nine cities were randomly placed on the coordinate grid. The process was

allowed to evolve 32,000 offspring. While the optimum solution remained

undiscovered, it is of interest to note that the problem was evidently

addressed at two distinct levels. The evolutionary process initially solved1

the problem at a gross level, discovering the minimum tour btwen the

groU. of cities, see Figure 32. Insufficient time was allowed to sort out

the problem at a finer level of detail.

Finally, an extremely large traveling salesman problem was analyzed.

Here, 256 cities were randomly distributed. The previous results Indicated

that the Adaptive Algorithm would not discover the optimum solution;

however, in only 10,000 iterations It reduced the initial tour length by

roughly 50%. Figure 33 Indicates the surviving tour after evaluating

10,000 offspring while Figure 34 Indicates the success of the evolu-

I ltionary process in discovering better and better tours. The available

computation time limited the analysis, however the results were certainly

I encouraging.

Clearly, the Adaptive Algorithm is an effective method for addressing

Sthe traveling salesman problem. Several Important conclusions can be

drawn from the previous experiments:I" Sophisticated" mutation operations are not only unnecessary,

but are derimetal The experiments point up the necessity for

BiI,,,

%' . . A ..o.A S ~ ~ ~ *S

r~~vr ~r, -, f V fl~~ -. -. - - -' - -W -Tv LK u -f S W*Wf s -tw, .Ct..LL ~

181

Q)* -)

-4-

-o0

CD <D

<0Q)

Eo0Q) _

000 0

CC-

E_ LU00

E

LO) 0 tp 0O

Figure 32

-~ ~ ~ ''~W~' - V~ ~V ~W VVI.W YL~ V~.~ 2~ i W1Ju U l 'Wh ' r- kr JW U 1WV rVI.U W I . L-u.I LVL' .

182

U)0

000

0.(0)

LOO

(N o

EE

0)C C 0 -

0 0 -. w L t

Figure 3

183

ci)J

* 0 )

4_ __ _ __j__ __ _ __ _ C :

UU(00

0 0

0 o

o- 0-I

00E0

0>

0'

0CL4 000 w

F--

Figure 34

1 184IJ

maintaining a substantial link between parent and off spring. The

more sophisticated and complex mutation operations destroy this

link. The PMX operation may perform generally superior to

Holland's crossover operation because it tends to retain moreInformation during each generation.

. There is a beneficial effect of using a noisy payoff function.

The concept of a noisy payoff function is similar to that suggested

by S. Kirkpatrick, C.D. Gelatt and M.P. Vecchi, 12 for optimizing

simulated annealing, but it is not necessary to resort to such

specific analogies. In a constantly changing environment, the

rewards and penalties for different behaviors vary. The search for

I better and better solutions Is never ending. Evolution is acontinuous process with no truly optimum solution.

.. . Further, it appears unlikely that any specific noisy payoff

function exists that will allow discovery of the optimum solution

In every traveling salesman problem. Each such problem offers a

different adaptive topography; therefore the appropriate

distribution and amount of noise cannot be determined a priorl

Although no single Adaptive Algorithm can optimally solve e .:

traveling salesman problem, a unique Adaptive Algorithm can be developed

to address eaj= traveling salesman problem In a very efficient manner.

12 - "OpUmzUon by Similatd Anmaling." S. Kirkpatrick. CJ). 6eutt Jr., lP. Vecchi. Sdenee.

hMy 1953.

I ,.' . , , . .. ,, .., " '' , . ' ,,, . , , . ., • ,. ' ' ,'. . , . - - - -. , , ," .." . -..

185

References:

1. D. Ackley, 0A Connectionist, Agoithm for Geneic Search," In Procaigs of en7Interna&tional Con7ference w7 C&nvtc Algorithms &7d Their Applications , .J

Grefenstette, Editor, Carnegie-Mellon University, July 1985. -

2. A.K. Dewckwv, "Computer Recreatons. Exploring the field of genetic algorithmns In a .

primordial computer see full of flubs," Scientific American, November 198S.

3. L.J. Fogel, "Autonomous Automata," IndustrIei Aerch, February 1962.

4. L.J. Fogel, "On the Organization of Intellect." Ph.D. dissertation, U.C.LA 1964.

5. L.J. Fogel, A.J Owens, M.J. Wash, Artiffic/al /nleligelnce Through SlmulatWEvolultn John Wiley& Sonis, New York, 1966.

6. D.E. Goldberg, R. Lingle, Jr., *Alleles, Loci, and the Traveling Salesman Problem,0 In

Pr xeedfngs of an7International Ctenca on 08netfc Algor~ims and' Thir/A4pplications , J.J. Grefenstette, Editor, Carnegie-Mellon University, July 1985.IR

7. J.J Greustette, R. (3opal, B. Rosmaita, D. Van Oucht. "Genetic Algorithms for teTraveling Salesman Problem," In Prxings of an Internetional Corence on60notIc Alarihms and1 Their Applications , J.J. OeasteEditor, Carnegie-MellonUniversity, July 1985.

8. J.H. Holland. A~woalon? i97 Natural and Artificial Systems, University ofMichigan Press, Ann Arbor, 1975.

9. S. Kirkpatrick, C.D. Oelatt, Jr., M.P. Yecchi, 00ptimization tby Simulated Annealing,"

Sciuncs, May 1983.

10. C.K. Levy, Elemins of Siolcy, third edition, Addison-Wesley Publishing Company

Inc., Reading, Mp~ad ichust, 1982. .

F = ~~~~~~~~~~~~~~~~~~~~~~~ .. . .-~ ... ..j ..... T .., . ,,. ;7 : ., , - , . , .

186

Addendum!

A completely random search (with replacement) will take roughly

twice as long to find the optimum solution as an enumerative search

(without replacement). To show this, consider the following two theorems:

I Theorem 1: It there are B possible solutions and only one

optimum solution, the expected number of trials that must be

made before the optimum solution Is found, using an enumerative

Isearch, assuming one trial Is made at a time. Is equal to

(B+ 1)/2.IProof- In an enumerative search, sampling Is made without replacement

I The probability, therefore, of discovering the optimum solution on any

I given trial Is equal to the product of the probabilities of not discovering

the optimum solution on any prior trial multiplied by the reciprical of the

I number of untried solutions. The expected number of trials that would have

to be examined before finding the optimum solution would therefore be:

I -f W 1 B- 1 + 2'[(B- I )/B]'(B- I )- + 3'[(3- I )/13][(-2)/(B- 1 )](13-2) - 1

+ (B- 1 )[(13- 1 )/8[2/31[1/2] + B1[(B- I )/1]".[2/3]-[1/21 1

1 " I13- + 2'13- + 3"-3 + "'" 4 (B-I)'B"1 + 13-

B"1 '(I + 2+ 3 +"" + (B-1)+ B)

I 3-1[1303+1)/2]j (B+ I1)/2. Q.E.D.

I " , " . " , r . " . " . . ' " . " • . " . " 4 " . . " . " . " . . " - . " . " . ' . r ." . " .

Theorem 2: If there are B possible solutions and only one

optimum solution, the expected number of trials that must be

made before the optimum solution Is found. In a completely

random search, assuming one trial Is made at a time, Is equal to

B.

Proof" In a completely random search, sampling Is made with

Il replacement. The probability, therefore, of discovering the optimum

solution on any given trial Is equal to the product of the probabilities of

I not discovering the optimum solution on any previous trial multiplied by

1 the reciprical of the total number of possible solutions. The expected

number of trials that would have to be examined before finding the optimal

solution would therefore be:

I I xt(x) - .1 #-' 2.1(3-1)/3)-1 # 3.[(0-1)/I 2.f-1 .1 +

I1 '1 * 2"1(0-I)/B] 3)(3O-)/J)2. 013?

-. O.E.D.

!

I.R

" 'r ' ':+Y":¥" ' , "' . .'%''""' """ 'x +; ,":, ' ,' : ' '; ' 'g ' ' ' ; . '';+ ,%