Bio-inspired Computing

Jeff McGough

Department of Computer ScienceRobotics and Intelligent Autonomous SystemsSouth Dakota School of Mines and Technology

February 25, 2011

Bio-Inspired Computing 1

SDSMT RIAS Master’s Program

http://rias.sdsmt.edu/

Bio-Inspired Computing 2

SDSMT RIAS Master’s Program

The M.S. in Robotics and Intelligent Autonomous Systems (RIAS)

An interdisciplinary, research-oriented degree in an emerging technical area.

Students in the program take courses in Computer Science, ComputerEngineering, Electrical Engineering and Mechanical Engineering.

The primary objective of the RIAS program is to give students a basicunderstanding of the mechanical, electrical and computing systems neededto participate in advanced mobile intelligent robotics applications.

Bio-Inspired Computing 3

Challenging Problems

I Pattern RecognitionI Machine VisionI Speech RecognitionI Text Recognition

I Optimization (continuous-discrete, multivariate, multiobjective)I Aircraft Wing designI Truss designI Circuit designI Antenna designI Controller design

And, of course, Robotics

Bio-Inspired Computing 4

Biological Approaches in Computer Science

How does a machine exhibithuman-like problem solving skills?

For example, how can we build a robot that can play checkers?

I Need to see (recognize) the pieces.

I Need to know how to move the pieces.

I Need to know the rules of the game.

I Need to know how to win.

Bio-Inspired Computing 5

Pattern Recognition

Say you want a computer to read text. How about the letter “A”?

Just chop up the font into little pixels and record which ones are part ofthe letter:

→ {(5,2), (6,2), (4,3), .... }

To tell if a letter is an “A”, one just compares the list to a known list.

Bio-Inspired Computing 6

Pattern Recognition

So what about

There are a million versions of “A”, and you can read them all ....

but the computer (more accurately, the algorithm) cannot.

What should we do?

Bio-Inspired Computing 7

Copy from Nature

If you want a computer to read text, we borrow from nature.

Since objects can be easily recognized by our Neural Networks, we emulatea network of neurons.

A single neuron ..

Bio-Inspired Computing 8

Artificial Neural Networks

This is modeled as

y = f

(n∑

i=1

wixi

)

and multiple ones in a network:

Layer 1:

yj = fj

(n∑

i=1

wi ,1xi

)@@

@@@R

Layer 2:

zi = fi

m∑j=1

wj ,2yj

Bio-Inspired Computing 9

Training ANNs

The weights wp,q in the ANN need to be determined. There is not aformula for A, the weights need to be “learned”. Using training data

→

z1 = 1, for A

zi = 0 for i 6= 1 for A

The algorithm is known as Back-Propagation. It will iteratively determinethe weights wp,q.

Once trained, we can use the ANN to determine letters from inputpatterns.

Bio-Inspired Computing 10

Checkers

How can a computer play a board game?

What is the basic idea?

Bio-Inspired Computing 11

Artificial Intelligence

From the initial opening board, red (or black) moves first.

There are multiple choices:

Then the opponent has choices:

then you have choices:

Bio-Inspired Computing 12

Machine Intelligence

This would continue until each choice lead to a win/loss:

Best move done by a decision process known as Minimax.

Bio-Inspired Computing 13

Machine Intelligence?

The game tree complexity is roughly 1031 which is

100000000000000000000000000000000

(roughly here means give or take a few hundred trillion).

We have a huge problem.

If your computer can explore 109 nodes per second

then the computation will take 1031/109 = 1022 seconds

= 317,097,919,837,646 years

Bio-Inspired Computing 14

Faster ....

We need to steamline this process.

The standard approach is to expand out the game tree a certain numberof levels:

?

Four levels.

But then how do we evaluate the bottom nodes?(since we have not pushed all the way down to a win or loss?)

Bio-Inspired Computing 15

Heuristics

We will use a function called a heuristic to look at the board and decidehow “good” the board is - or how close to a win.

Store the board in an array Bi ,j : 1 −1 1 . . .1 0 −1 . . .. . . . . . . . . . . .

⇔Bi ,j = 1 if cell (i,j) has a dark checker,Bi ,j = −1 if cell (i,j) has a light checker,Bi ,j = 0 if cell (i,j) is empty.

We need to find

f (Bi ,j)→ approximately how close to a win

but this needs to be fast and not just evaluate the tree.

Bio-Inspired Computing 16

ANNs to the rescue

How about we feed the board into an Artificial Neural Network and let theANN rank the board position?

Win

Loss

Bio-Inspired Computing 17

ANNs?

All we need to do is find the weights in the network, i.e. train it.

Well, IF we had the training data the Supervised Learning approach wouldsucceed.

Without training data, how do we find the weights.

Evolution

We try simulated or artificial evolution.

A brief aside on simulated evolution ...

Bio-Inspired Computing 18

Evolutionary Computing

Evolutionary Algorithms:

I Genetic AlgorithmsFrazer (1957), Bremermann, Reed, Holland

I Evolutionary ProgrammingFogel (1964), Koza

I Genetic ProgrammingKoza (1990)

I Evolutionary StrategiesSchwefel (1965), Rechenberg, Fogel

Bio-Inspired Computing 19

Evolutionary Computing Origin

I Evolution and Natural SelectionI Alfred Russel WallaceI Charles Darwin (Origin of the Species, 1859)

I One of the most significant and influential ideas in science

I Evolution = change in allele frequency

I Evolution by natural selectionI Selection acts on the individualI Group of individuals is a populationI A population has variationI Differential rates of reproduction existI Population growth is exponential

Bio-Inspired Computing 20

Prototypical Evolutionary Algorithm

I Stochastic search method

I Population: P(t), t = 0, 1, 2, ...

I IndividualI encoding: encoded version of possible solution

I Fitness associated with each individualI fitness function

I Reproduction operatorsI MutationI Recombination

I Selection operators (stochastic or deterministic)I FitnessI Survival

Bio-Inspired Computing 21

Evolutionary Algorithm

I t = 0

I initialize P(t) and evaluate P(t)

I while (not terminated) do

I P ′(t) = variation[P ′(t)]

I evaluate [P ′(t)]

I P(t) = select[P ′(t),P(t)]

I t = t + 1

I end while

EAs are simple algorithmsThe difficulty is in the encoding, selection and reproduction operators

Bio-Inspired Computing 22

Simple Example

Assume someone asks you to find the maximum for the function:

f (x) =(ex − e−x/2

) cos(10x) sin(x)

x, − 2 < x < 2.

Let y be the 16 bit binary number

y = {b15b14b13b12 . . . b0}

Example y = 1010101001000110.

Sixteen digit binary numbers convert to numbers between 0 and 65535.

So, we can convert x = 4y/65535− 2.

This means we can work with the binary number.

Bio-Inspired Computing 23

Simple Example Cont

An individual would be one of these binary numbers:

101010001010010

A population would be a bunch of these binary numbers:

0001001010110101,

1011111100011010,

1000001000110101,

0011101010111101,

...

(we can build an initial population by generating them randomly)

Bio-Inspired Computing 24

Simple Example Cont

Mutation is easy, one only needs to flip a bit:

?1010101001000110→ 1010101001010110

Crossover is very straightforward as well:

101|0101001000110 + 011|001001001000 → 011|0101001000110

6Random cut point

Use crossover and mutation to produce new individuals.

Bio-Inspired Computing 25

Simple Example Cont

We then use the function to optimize as the fitness function: f (x)

I Plug the individual into f (x) and get the fitness.

I Rank the population according to fitness.

I Keep the top half, use them to produce (via mutation and crossover)new individuals.

Repeat the steps above...

And the population convergesto 1111100001010001 whichmakes f (x) largest: x ≈ 1.88.

Bio-Inspired Computing 26

Back to Checkers ...

So how does this work for the artificial neural network?

Load the weights of the network into the genome:

x = {w1,1,w2,1, . . . }

Define mutation as a random change of one of the weights.

Define crossover as done before.

The only thing left is the fitness function - but if we knew that, then wecould just use it for the heuristic and skip the ANN.

Now what should we do?

Bio-Inspired Computing 27

Implicit fitness - Tournament Selection

It is unlikely that biologists can tell you what the fitness is for anorganism, at best relative fitness.

We have a checkers program, with an untrained ANN. How about playingtwo against each other. The winner has the better ANN and so we can usethis as our implicit fitness.

Tournament selection ...

Bio-Inspired Computing 28

David Fogel (1999)

David Fogel applied a variation of the evolutionary algorithm:

I Population = 15

I Random weights and biases to start

I Each parent generates an offspring (mutation only)

I Each would play against randomly selected opponents (5)

I A score would be assigned

I 150 games per generation

I Top 15 survive to next generation

He created a user, Blondie24, on a popular checkers site. The profile was a24 year old female mathematics graduate student. He hired a model forthe profile picture.

Bio-Inspired Computing 29

Blondie24 - Results

He played his evolved game against challengers on the site for severalmonths. He continued to evolve the Neural Network.

I Neural net was on the par of players rated in 1800-1900 USCF level

I Max rating = 1975.8

I Mean rating = 1901.98 ⇒ Class A player

A success!

Long before IBM Watson and Jeopardy

Bio-Inspired Computing 30

Issues

Problems with traditional genetic algorithms:

I EAs don’t always converge.

I They can be very slow.

I They can be memory and cpu hogs.

I They get stuck in local extremals.

I They have fixed genotypes and phenotypes.

I No Free Lunch: The average performance of all optimizers is identicalif the distribution of functions is average.

Solutions ...Apply concepts from outside computer science

Bio-Inspired Computing 31

Biological Insight

Traditional Evolutionary Algorithms directly code structure:

−→

New approaches do not code structure, but code for dynamics:

−→ −→ −→

Bio-Inspired Computing 32

No Free Lunch - but a box of chocolates...

NFL Theorem does not apply to co-evolutionary systems.

Predator-Prey Models

I Predators and prey co-evolve.

I Predator and prey represented by different genomes with different EArules.

I Rules for predator prey interaction, for example:I If a predator is stronger, the prey is eaten (removed) and the predator

lives.I If a prey is stronger, the prey lives and the predator starves (possible

removal).

I Both reproduce normally and prey has adequate resources.

I Often Lotka-Volterra dynamics

Evolved quadratic formula’s with this approach.

Bio-Inspired Computing 33

Environment

Distributed (dEA)

I Island model

I Multi-population with migration

I Coarse grained

Cellular (cEA)

I Neighborhood/grid model

I Single population with local selection

I Fine grained

Bio-Inspired Computing 34

Thank You

Questions?

Bio-Inspired Computing 35