Module 3

Genetic Algorithm

International conference paper on genetic algorithms

1.Evolutionary Simulation-Based Validation

(F. Corno, M. Sonza Reorda, G. Squillero International Journal on Arti�cialIntelligence Tools (IJAIT), Vol. 14, 1-2, Dec. 2004)

Scope:

This paper describes evolutionary simulation-based validation, a new pointin the spectrum of design validation techniques, besides pseudo-random simu-lation, designer-generated patterns and formal veri�cation. The proposed ap-proach is based on coupling an evolutionary algorithm with a hardware simu-lator, and it is able to �t painlessly in an existing industrial �ow. Prototypicaltools were used to validate gate-level designs, comparing them against both theirRT-level speci�cations and di�erent gate-level implementations. Experimentalresults show that the proposed method is e�ectively able to deal with realisticdesigns, discovering potential problems, and, although approximate in nature,it is able to provide a high degree of con�dence in the results and it exhibits anatural robustness even when used starting from incomplete information

2.Exploiting Symbolic Techniques within Genetic Algorithms for Power Op-timization

(S. Chiusano, F. Corno, P. Prinetto, M. Rebaudengo, M. Sonza Reorda IC-TAI97: 9th IEEE International Conference on Tools with Arti�cial Intelligence,Newport Beach, CA (USA), November 1997 )CV. Ramamoorthy Best PaperAward

Scope:

This paper proposes an optimization algorithm for reducing the power dis-sipation in a sequential circuit. The encoding of the di�erent states in a FiniteState Machine is modi�ed to obtain a functionally equivalent circuit that ex-hibits a reduced power dissipation. The algorithm is based on a newly proposedpower estimation function, that is able to quickly give an accurate estimate ofthe dissipated power without actually synthesizing the circuit. Given this esti-mate, a Genetic Algorithm provides a state re-encoding for the circuit. The esti-mation function is computed in a very e�cient way by exploiting some symboliccomputations with Binary Decision Diagrams. The algorithm is experimentallyshown to provide good results from the power optimization point of view, at alimited cost in terms of area increase, when compared with similar approaches.

1

3.Hybrid Genetic Algorithms for the Traveling Salesman Problem

(P. Prinetto, M. Rebaudengo, M. Sonza Reorda International Conference onNeural Networks and Genetic Algorithms, Innsbruck (A), Aprile 1993)

Scope:

A comparative analysis is performed on an experimental basis among fourdi�erent cross-over operators. In order to exploit the bene�ts of the di�erent op-erators, a new one (called Mixed Cross-over) is introduced, trading-o� the CPUtime requirements and the obtained results. A new operator is then proposed,whose goal is to include in the genetic mechanism some heuristic knowledgedrawn from the already proposed local-optimization techniques. The perfor-mance of the new operator is discussed.

1. 4.Thomas Bäck (Ed.): Proceedings of the 7th International Conferenceon Genetic Algorithms, East Lansing, MI, USA, July 19-23, 1997.

Scope:

The Seventh International Conference on Genetic Algorithms (ICGA-97) willbe held on July 19-23, 1997, at the Kellogg Center, Michigan State University,East Lansing, MI. This meeting will bring together an international communityfrom academia, government and industry interested in the �eld of EvolutionaryComputation, i.e., algorithms gleaned from models of natural evolution. Exam-ples of such algorithms are: Evolutionary Programming, Evolution Strategies,Genetic Algorithms, Genetic Programming, Learning Classi�er Systems,

and similar paradigms as used in evolving e.g., Arti�cial Life, Cellular Au-tomata, Computational Ecosystems, Cultural Algorithms, Fuzzy Systems, Im-mune Networks, Machine Learning, Multiagent Systems, Neural Networks, Sim-ulated Annealing and others.

Genetic Algorithm De�nitions

Grefenstette

"A genetic Algorithm is an iterative procedure maintaining a population ofstructures that are candidate solutions to speci�c domain challenges. Duringeach temporal increment (called a generation), the structures in the currentpopulation are rated for their e�ectiveness as domain solutions, and on thebasis of these evaluations, a new population of candidate solutions is formedusing speci�c genetic operators such as reproduction, crossover, and mutation."

Goldberg

"They combine survival of the �ttest among string structures with a struc-tured yet randomized information exchange to form a search algorithm with

2

some of the innovative �air of human search. In every generation, a new setof arti�cial creatures (strings) is created using bits and pieces of the �ttest ofthe old; an occasional new part is tried for good measure. While randomized,genetic algorithms are no simple random walk. They e�ciently exploit his-torical information to speculate on new search points with expected improvedperformance."

Genetic Algorithms Overview

Developed by John Holland in 1975 .Genetic Algorithms (GAs) are searchalgorithms based on the mechanics of the natural selection process (biologicalevolution). The most basic concept is that the strong tend to adapt and survivewhile the weak tend to die out. That is, optimization is based on evolution, andthe "Survival of the �ttest" concept. GAs have the ability to create an initialpopulation of feasible solutions, and then recombine them in a way to guidetheir search to only the most promising areas of the state space.

Each feasible solution is encoded as a chromosome (string) also called agenotype, and each chromosome is given a measure of �tness via a �tness (eval-uation or objective) function.The �tness of a chromosome determines its abilityto survive and produce o�spring. A �nite population of chromosomes is main-tained.GAs use probabilistic rules to evolve a population from one generationto the next. The generations of the new solutions are developed by genetic re-combination operators.GAs are ways of solving problems by mimicking processnature uses ;i.e Selection, Cross over, Mutation, and Accepting to evolve a solu-ton to a problem. GAs are adaptive heuristic search based on the evolutionaryideas of natural selection and genetics.

Optimization Algorithm & Genetic Algorithm

Optimization is the process that �nds the best,Optimal, solution for theproblem.The Optimization problem are centered around three factors:

1. An Objective Function which is to be minimized or maximized.;

2. A set of unknows or Variables that a�ect the objective function.

3. A Set of constraints that allows unknown to take on certain values butexclude others.

An Optimization problem is de�ned as �nding the value of variables that mini-mize or maximize the objective function while satisfying the constraints.

3

Search Optimization Algorithm

The Evolutionary Algorithms include:

• Genetic Algorithms

• Genetic Programming.

Evolutionary Algorithms are subset of Evolutionary Computation. Which is asub�eld of Arti�cial Intelligence(AI). Genetic Algorithms (GAs) represent themain paradigm of Evolutionary Computation.

-GAs simulate natural evolution.mimicking process the nature uses:Selection,Cross over,Mutation, and Accepting.

-GAs simulate the survival of the �ttest among individuals overconsecutive generation for solving a problem.

Development History

4

Basic genetics

All living organism consists of cells

• Each cell of a living thing contains chromosomes - strings of DNA

• Each chromosome contains a set of genes - blocks of DNA

• Each gene determines some aspect of the organism (like eye colour)

• A collection of genes is sometimes called a genotype

• A collection of aspects (like eye colour) is sometimes called a phenotype

General scheme of Evolutionary process

Biological Terminology

Chromosome : A Set of gene. Strings of DNA�that serve as a "blueprint"for the organism. A Chromosome contains a solution in form of genes.

Gene : A part of chromosome ; a gene contains a part of solution. Itdetermines the solution.The "genes" are either single bits or short blocksof adjacent bits that encode a particular element of the candidate solution.

Individual : Same as chromosome.

Population : Number of individuals present with the same length ofchromosome.

5

Fitness : The value assigned to an individual based on how far or closeinsividual is from the solution;greater the �tness value better the solution itcontains.

Fitness Function: The function that assigned to Fitness value to theindividual.It is problem speci�c.

Breeding: Taking two �t individual and then intermingling therechromosome to create new two individual.

Mutation : Changing a random gene in an individual.

Selection : Selecting individuals for creating the next generation.

Working principle

Genetic algorithm begins with a set of solution(represented by chromosome)called the population.Solution from one population are taken and used to forma new population.This motivated by the possibility that the new population willbetter than the old one. Solutions are selected according to their �tness to newsolution (O�spring); more suitable they are ,more chance they have to reduce.

Procedure Of Genetic Algorithms

Fig shows the �ow chart for GAs. A GA for a particular problem must havethe following �ve components.

6

1. A genetic representation for the potential solutions to the problem.

2. A way to create an initial population of potential solution.

3. An evaluvation function that plays the role of the environment,rating so-lution in terms of their 'Fitness'.

4. Genetic operators that alter the composition of the o�spring.

5. Values for the various parameters that the genetic algorithm uses.

Outline of genetic algorithm

1. [Start] Generate random population of 'n' chromosome (i.e. sutable forsolution for the problem)

2. [Fitness] Evaluvate the �tness f(x) for each chromosome x in the popu-lation.

3. [New Population] Create new population by repeating the following stepsuntill new population is complete.

(a) [Selection] Select two parent chromosomes from a populationaccording to their �tness(better the �tness Bigger the chance tobe selected).

(b) [Crossover] with a crossover probability,cross over the parentsto form new o�spring(children). If no crossover was performed,o�spring is the exact copy of parents.

(c) [Mutation] with a mutation probability,mutate new o�springat each locus (position in chromosome)

(d) [Accepting] Place new o�spring in the new population.

4. [Replace] use new generated population for furthur run of the algorithm.

5. [Test] If the end condition is satis�ed ,stop,and run the best solution inthe current population.

6. [Loop] Go to step 2

Genetic Representation

Representation or Encoding the problem in hand when applying aGA is a vital task. Encoding can be de�ned as the chromosomal represnta-tion ofthe problem.When GA was initially introduce the binary string encodingtechnique was used. when it came to industrial & other scienti�c applications

7

applying GA directly using binary string became a problem. Because it was nota natural coding.

Chromosomes could be: Bit strings (0101 ... 1100)

Real numbers (43.2 -33.1 ... 0.0 89.2)

Permutations of element (E11 E3 E7 ... E1 E15)

Lists of rules (R1 R2 R3 ... R22 R23)

Program elements (genetic programming) ... any data structure ...

Binary Encoding

Binary representation: Here encoding is done using sequence of 1's and 0's. Thelength of the string is determined by the precision desired for the solution. Anyinteger can be converted in to binary by dividing it by 2.

Examle:

A Gene represnet some data (Eye color,hair color,..)A chromosome is an array of genes. In binary form

Gene (11100010)

Chromosome Gene 1 (11000010)

Gene2 (00001110)

Gene3 (001111010)

Gene 4 (10100011)

A chromosome should in some way contains information about solutionwhich it represents; It thus requires encoding. The most popular way of en-coding is binary string like

Chromosome 1 : 1101100100110110

Chromosome 2 : 1101111000011110

Each bit in the string represent some characterestics of the solution. Binaryencoding gives many possible chromosomes even with a small number of allelesie possible settings for a trait. This encoding is often not natural for many prob-lems and sometimes corrections must be made afte crossover and/or mutation

Two variable function represented by 4 bit string for each variable.

Let two variables x1 , x2 as (1011 011). Every variable will have both upperand lower limits as Xi

L≤ Xi ≤ XiU .Because 4-bit string can represent integers

from 0 to 15, So (0000 0000) and (1111 1111) represent the points for x1 , x2as (X1

L ,X2L) and (X1

U ,X2U ) respectively.

Thus, an n-bit string can represent integers from 0 to 2n − 1 , i.e.2n integers

8

Consider a 4-bit string (0111)The decoded value is equal to

23 ∗ 3+22 ∗ 1+21 ∗ 1+20 ∗ 1=7Xi

L and XiU correspond to (0000) and (1111) the equivalent value for any

4-bit string can be obtained as

Xi=XLi u

(XUi −XL

i )2ni−1 * (decodevalueofstring)

For example variable Xi.Let XLi =2 and X

Ui =17 �nd what value the 4-bit

string Xi=(1010) would represent.

Decoded value Si=1010=23 ∗ 1+22 ∗ 0+21 ∗ 1+20 ∗ 8 = 10 then

Xi = 2+ (17−2)(24−1) ∗ 10 = 12

Permutation encoding

Permutation encoding can be used in ordering problems, such as TSP ortask ordering problem.

1. In permutation encoding, every chromosome is a string of numbers thatrepresent a position in a sequence.

Chromosome A 1 5 3 2 6 4 7 9 8Chromosome B 8 5 6 7 2 3 1 4 9

2. Permutation encoding is useful for ordering problems. For some problems,crossover and mutation corrections must be made to leave the chromosomeconsistent.

Example

TSP are cities and given distance between them. Traveling salesman has tovisit all of them, but the does not want to travel more than necessary. Finda sequence of cities with a minimal traveled distance. Here, encoded chrom-osome describe the order of cities the salesman visits.

9

Tree Encoding

In tree encoding, every chromosome is a tree of some objects, such as func-tions or commands in programming language. Tree encoding is useful forevolving programs or any other structures that can be encoded in trees. Thecrossover and mutation can be done relatively easy way.

Genetic operators

Genetic operators used in genetic algorithm to maintain genetic diversity.Genetic diversity or validation is a necessity for the process of evaluvat-ion.Genetic operators are analogous to those which occur in the naturalworld.There is mainly three Genetic operators.

• Reproduction (or Selection)

• Crossover (or Recombination)

• Mutation

SELECTION

Selection will allow selection rules and random behavior to select next population.In selection the parents must be selected based on their �tness.Theindividuals with a higher �tness must have a higher probability of havingo�spring.Fitness value F is calculated The probability of selection of ith

10

chromosome is done

Pi=Fi∑

j

popsize

−Fj

The cumulative frequency qi=∑i

j=iPj

Generate a random number r from the range [0, z]If r < q1, select the �rst chromosome, otherwise select chromosome from 2to pop_size

There are several methods for selection.

• Roulette-wheel selection.

• Tournament selection.

• Rank selection.

• Steady-state selection.

• Boltzmann selection.

• Scaling selection

Roulette-Wheel Selection

Roulette: the classical selection operator for generational GA as describedby Goldberg. Each member of the pool is assigned space on a roulette wheelproportional to its �tness. The members with the greatest �tness have thehighest probability of selection. This selection technique works only for aGA which maximizes its objective function.

Concept : the chance of an individual's being selected is proportional to its�tness, greater or less than its competitors' tness.

The Probability of parenthood is proportional to �tness.The wheel is spununtil two parents are selected.The two parents create one o�spring.The process is repeated to create a new population for the next generation.Roulette wheel selection has problems if the �tness changes by orders ofmagnitude.If two individuals have a much higher �tness, they could be theparents for every child in the next generation.

11

Reason Not to Use the Roulette Wheel

• If the �tness value for all individuals is very close, the parents will bechosen with equal probability, and the function will cease to optimize.

• Roulette selection is very sensitive to the form of the �tness function andgenerally requires modi�cations to work at all.

Boltzmann Selection

Simulated annealing is a method used to minimize or maximize a function.This method simulates the process of slow cooling of molten metal to achieve theminimum function value in a minimum function value in a minimization prob-lem.The cooling phenomena is simulated by controlling a temprature like pa-rameter introduced with the concept of Boltzmann probability distribution.Thesystem in thermal equilibrium at a temperature T has its energy distributionbased on the probability de�ned by

P(E) =exp(-E/KT) where K is Boltzzmann constant

This expression suggest that a system at a higher temperature has almostuniform probability at any energy state, but at lower temperature it has asmall probability of being at ahigher energy state.

Crossover

Two parents produce two o�spring .There is a chance that the chromosomesof the two parents are copied unmodi�ed as o�spring .There is a chance that thechromosomes of the two parents are randomly recombined (crossover) to formo�spring .Generally the chance of crossover is between 0.6 and 1.0 .Crossoveris usually the primary operator with mutation serving only as a mechanism to

12

introduce diversity in the population. However, there are a number of crossoveroperators that have been used on binary and real-coded GAs.There are variouscrossover

1.Single-point Crossover2.Double-point Crossover3.Multiple-point Crossover4.Uniform Crossover5.Matrix Crossover6.Random Crossover7.Permutation-based Crossover8.Partial mapped crossover9.Ordered crossover(OX)10.Position based crossover(OX)11.Ordered based crossover12.Cycle crossover(CX)13.Sub-tour exchange crossover14.Heuristic crossover

1.Single-point Crossover

Given two parents, single-point crossover will generate a cut-point and re-combines the �rst part of �rst parent with the second part of the second parentto create one o�spring. Single-point crossover then recombines the second partof the �rst parent with the �rst part of the second parent to create a secondo�spring.

+ Parents 0 1 1 1 1

1 1 1 1 1

Children 1 1 1 1 1

0 1 1 1 1

2.Double-point Crossover

Two-Point crossover is very similar to single-point crossover except that twocut-points are generated instead of one.

13

Parents 0 1 1 0 1

1 1 1 1 1

Children 1 1 1 1 1

0 1 1 0 1

3.Multiple-point Crossover

In the multiple point crossover operation,the crossover opertion takes placeateven and odd numbered sites.In the case of even numbered crossover sites thechromosomal string is viewed as a ring with no begning and end and crossoversite are selected around the circle at random.In case of odd numbered crossoversite,the crossover point is choosen at the begining of the string.

Parents 1 1 1 1 1 1 1 1 1 1

0 0 0 0 0 0 0 0 0 0

Children 0 0 1 0 0 1 1 1 0 0

1 1 0 1 1 0 0 0 1 1

4.Uniform Crossover

In Uniform Crossover, a value of the �rst parent's gene is assigned to the�rst o�spring and the value of the second parent's gene is to the second o�springwith probability 0.5. With probability 0.5 the value of the �rst parent's geneis assigned to the second o�spring and the value of the second parent's gene isassigned to the �rst o�spring.

Parents 1 1 1 0 0 1 0 0

1 1 0 1 0 1 1 1

Children 1 1 0 0 0 1 0 1

1 1 1 1 0 1 1 0

5.Matrix Crossover

It is used in two dimensional array .Here the rows &columns of the crossoversites select randomly.Thus two crossover sites forms a three layer matrix.Thenwe can select any region.

Parents

1 0 11 0 10 1 1

0 0 11 0 01 1 0

14

Children

0 0 11 0 10 1 0

1 0 11 0 01 1 1

6..Random Crossover

Random Crossover creates o�spring randomly within a hyper-rectangle de�ned by the parent points. There are two type :i).Flat crossover: An o�spring is produced by uniformly picking a value foreach gene from a range of values correspond to the parent node .

ii).Blend crossover: which incorporates more variance by picking values thatlies between 2 points containing the 2 parents .

7.Permutation-based Crossover

Permutation-based Crossover is developed mainly for combinatorial optimization problem such as the travelling salesman problem,machine scheduling,resource allocation etc.These operators function based on two approaches:

i).Canonical approach :It is the extention of Double-point Crossover &Multiple point Crossover of binary string .it is based on blind randommechanism.There is no guarantee that an o�spring produced by this method is betterthan its parents .

ii).Heuristic approach :Application of Heuristic in crossover tends to generateimproved o�spring .

8.Partial mapped crossover

Goldberg and Lingle proposed this method. Adopts a repairing procedureknown as relationship mapping in order to prevent illegal duplication of genes.Extension to double point crossover. Figure shows the two substrings are se-lected and exchanged between the parent chromosomes.

Parents 1 2 4 6 5 3 9 8 7

3 8 5 1 2 9 6 4 7

Mapping Relationship9�>32�>5�>41�>6

Children 6 4 5 1 2 9 3 8 7

9 8 4 6 5 3 1 2 7

15

9.Ordered crossover(OX)

Davis proposed this method,which is an extension of partial mapped crosoverwith a di�erent repairing procedure.Asubstring is selected reandomly from oneof the parents and copied exactly to the same positions in the child string.

Parents 4 5 2 1 3 7 8 6 9

5 7 3 2 1 4 6 8 9

Children 5 2 4 1 3 7 8 6 9

5 3 7 2 1 4 6 8 9

12.Cycle crossover(CX)

Oliver,Smith and Holland proposed this crossover. The nodes to be selectedfrom parent are de�ned by a cycle according to the corresponding position be-tween the parents.

Parents 1 Parent 2 Child1 Parent 2 Parent 1 Child2

452138769

573214689

453218769

573214689

452138769

572134689

Cycle=4�>5�>7�>6�>8�>4 Cycle=5�>4�>8�>6�>7�>5

14.Heuristic crossover

A random node is picked at the start then its shortest edge that does notlesd to a cycle is selected. If two edges lead to a cycle,then a random node ispicked up again and the above process is continued till all the nodes are checkedand a new o�spring is produced. This approavh is very e�cient compared toother permutation operator.

Mutation

Mutation is a genetic operator used to maintain genetic diversity from onegeneration of a population of genetic algorithm chromosomes to the next. Thereare three type of mutation:

1.Boundary Mutation2.Uniform Mutation3.Non-uniform Mutation

16

1.Uniform Mutation

A mutation operator that replaces the value of the chosen gene with a uni-form random value selected between the user-speci�ed upper and lower bounds .This mutation operator can only be used for integer and �oat genes. Letconsider a chromosome xt=[x1, x2, ......xm].A random number is selected such

that ,k [1,n] then o�spring xt+1=[x1, x

′

k, ..............xm

]is produced. X

′

k is

the random value generated from range[xlk,x

um]

2.Boundary Mutation

This mutation operator replaces the genome with either lower or upperbound randomly. This can be used for integer and �oat genes. The replacementof gene X'k by either xlk,xum

3.Non-uniform Mutation

The probability that amount of mutation will go to 0 with the next gener-ation is increased by using non-uniform mutation operator. It keeps the popu-lation from stagnating in the early stages of the evolution. It tunes solution inlater stages of evolution. This mutation operator can only be used for integerand �oat genes.

X'L={XL +4(t,Xu

L −Xk ),if the random digit is 0

X'L={Xt −4(t,Xt −XL

k ),if the random digit is 1

One's Complement Operator

This Operator is Known as unary operator(~). It causes the bits of itsoperand to be inverted.

1 0 1 1 0 0 1 1output0 1 0 0 1 1 0 0

Logical bitwise operator

This operator can furthur classi�ed in to

1. bitwise AND

2. bitwise exclusive(X)-OR

3. bitwise OR.

17

Two Bitwise Operator usedX Y AND XOR OR

0 0 0 0 00 1 0 1 11 0 0 1 11 1 1 0 1

AND Operator OR OperatorPx 0 1 0 1 1 1 0 PX 0 1 0 1 1 1 0

qx 1 1 1 1 0 1 1 qx 0 1 1 1 0 1 1

Cxy 0 1 0 1 0 1 0 Cxy 0 1 1 1 1 1 1

Shift Operator

Shift operator 2 types.

1. Shift Left(<�<) operator.

2. Shift Right(>�>) operator.

Shift Left Operator1 0 0 1 1 1

0 0 1 1 1 0Shift Right Operator1 0 0 1 1 1

0 1 0 0 1 1Masking OperatorTransform a given bit pattern in to another form with the help of Logical

bitwse operation. There is a mask operand is used.

px 1 0 1 0 1 1 1 0

masking table 0 1 1 1 0 0 1 1o/p 0 0 1 0 0 0 1 0

Mutation schemas are:

1. Inversion.

2. Insertion.

3. Displacement.

4. Reciprocal Exchange Mutation.

Insertion.Node selected random and inserted random.px 2 5 4 9 6 8 1 3

cx 2 6 4 9 5 8 1 3DisplacementSelect substring random & insert it in random position.

18

px 2 5 4 9 6 8 1 3

cx 2 6 4 5 9 8 1 8Inversion.Two position randomly selected. Substring between these two position is

inverted.px 2 5 4 9 6 8 1 3

cx 2 5 8 6 9 4 1 3Reciprocal Exchange Mutation.Select two position at random then swap the nodes in these position.px 2 5 4 9 6 8 1 3

cx 2 6 4 9 5 8 1 3

Applications of Genetic Algorithms

1. Traveling Salesman Problem

2. Decoding A Secret Message

3. Robot Trajectory Planning

4. Optimizing Arti�cial Neural Nets

1. 1.Traveling Salesman Problem

The goal is to �nd the shortest route for a salesperson to take in visiting Ncities The cost function for the simplest form of the problem is just the distancetraveled by the salesperson for the given ordering (xn, yn), n = 1,...,

where (xn, yn) are the coordinates of the nth city visited

cost=∑N

n=0

√(xn − xn+1)

2+(yn − yn+1)2

Let N = 13 cities and put the starting and ending point at the origin, so(x0, y0) = (xn+1, yn+1) = (0, 0)= starting and ending point .There are a totalof 13!/2 = 3.1135 x 109 possible combinations to check. Assumes that all thecities lie in a rectangle and the minimum distance is 14 .The GA parametersNpop = 400 Nkeep = 200 Mutation Rate = 0.04

19

Here we use Permutation EncodingEg : [ 2 5 3 1 4 6 ]Crossover Operator is a variation of Cyclic Crossoverparent 1 : [ 1 5 3 6 4 2 ]parent 2 : [ 4 2 1 3 6 5 ]Randomly select a location to exchangeo�spring 1 : [ 4 5 3 6 4 2 ]o�spring 2 : [ 1 2 1 3 6 5 ]Next Stepo�spring 1 : [ 4 5 3 6 4 2 ]o�spring 2 : [ 1 2 1 3 6 5 ]

o�spring 1 : [ 4 5 3 6 6 2 ]o�spring 2 : [ 1 2 1 3 4 5 ]o�spring 1 : [ 4 5 3 3 6 2 ]o�spring 2 : [ 1 2 1 6 4 5 ]o�spring 1 : [ 4 5 1 3 6 2 ]o�spring 2 : [ 1 2 3 6 4 5 ] .The process iterates until we return to the �rst exchanged site .The mutation

operator randomly chooses a string, selecting two random sites within thatstring, and exchanges the integers at those sites Convergence of the geneticalgorithm .

If we doesn't have an obvious minimum path, then The optimal solution willhave no crossing paths So plot the solution and check

20

Decoding A Secret MessageUses a GA to break a secret code. A message consisting of letters and spaces

is encoded by randomly changing one letter to another letter. If the messageuses every letter in the alphabet plus a space, then there are a total of 27!possible codes, with only one being correct .If the message uses S symbols, thenthere are 27! - S! possible encodings that work. A chromosome consists of 27genes with unique values from 1 to 27 A 1 corresponds to a space and 2 through27 correspond to the letters of the alphabet. Letters and spaces in the messagereceive the appropriate numeric values.

The cost is calculated by subtracting the guess of the message from theknown message, taking the absolute value, and summing:

cost=∑N

n=t[message(n)− guess(n)]

Optimizing Arti�cial Neural NetsIn training process of Neural Networks, the weights and bias values are

optimized to produce the desired output. GA to compute the optimum weightsand biases

To do this, we used the two-layer neural network with log-sigmoid transferfunctions .The goal is to compute the optimum weights and biases of the ANN

f(x)=12/x2cos(x)+1/x for 1<=x<=5using GA

21

The GA chromosome is made up of potential weights and biases .The GAcost function computes the mean square di�erence between the current guessof the function and the exact function evaluated at speci�c points in x.Thefunction computed from the neural network with GA hybrid training matchesthe known curve quite well.

Other Applications

• Locating An Emergency Response

• Unit Stealth Design Building

• Dynamic Inverse Models

• Combining Gas With Simulations�air Pollution Receptor Modeling

• Antenna Array Design

22

Support Vector machine

one of the most well studied and widely used learning algorithms for binaryclassi�cation Extensions of SVMs exist for a variety of other learning problems,including regression, multiclass classi�cation, ordinal regression, ranking, struc-tured prediction, and many others. Similar to perceptrons they aim to �nd ahyper plane that linearly separates data points belong to di�erent classes Inaddition SVMs aim to �nd the hyper plane that is least likely to over�t thetraining data.

Support Vector Machines are based on the concept of decision planes thatde�ne decision boundaries. A decision plane is one that separates between a setof objects having di�erent class memberships. A schematic example is shown inthe illustration below.

In this example, the objects belong either to class GREEN or RED. Theseparating line de�nes a boundary on the right side of which all objects areGREEN and to the left of which all objects are RED. Any new object (whitecircle) falling to the right is labeled, i.e., classi�ed, as GREEN (or classi�ed asRED should it fall to the left of the separating line).

We are given a set of n points (vectors) : such that is a vector of length m ,and each belong to one of two classes we label them by �+1� and �-1�. -So ourtraining set is: (x1,y1),(x2,y2),.........(xn,yn)∀ixi ε Rm, yiε{+1,−1}We want to�nd a separating hyperplane

w.x+ b = 0

that separates these points into the two classes. �The positives� (class �+1�)and �The negatives� (class �-1�). (Assuming that they are linearly separable)

23

Separating Hyperplane

Suppose we choose the hypreplane (seen below) that is close to some samplexi.Now suppose we have a new point x

′that should be in class �-1� and is close

to xi . Using our classi�cation function f (x) this point is misclassi�ed or Poorgeneralization.

So Hyperplane should be as far as possible from any sample point.As shownin the below �gure.This way a new data that is close to the old samples will beclassi�ed correctly.Which perform good generalization.

24

Linearly Separable Data - Hard Margin SVM

It is the SVM idea is to maximize the distance between The hyperplane andthe closest sample point.Shown in below �gure. In the optimal hyper- plane:

�Distance to the closest negative point = Distance to the closest

positive point.�

SVM's goal is to maximize the Margin which is twice the distance �d�between the separating hyperplane and the closest sample.

Why it is the best?

• Robust to outliners as we saw and thus strong generalization ability.

• It proved itself to have better performance on test data in both practiceand in theory.

Support vectors are the samples closest to the separating hyperplane.

25

A training sample S = ((x1; y1); : : : ; (xm; ym)) 2 (R { -1; 1}) is said tobe linearly separable if there exists a linear classi�er

h(x) = sign(w.x+ b)

which classi�es all examples in S correctly, i.e. for which yi(w.xi + b) > 0 {1;: : : ;m}g. For example, Figure (left) shows a training sample in R2 that islinearly separable, together with two possible linear classiers that separate thedata correctly.

Fig : Left: A linearly separable data set, with two possible linear classiersthat separate the data. Blue circles represent class label 1 and red crosses 1; thearrow represents the direction of positive classication. Right: The same dataset and classiers, with margin of separation shown.

Although both classiers separate the data, the distance or margin with whichthe separation is achieved is dierent; this is shown in Figure 1 (right). TheSVM algorithm selects the maximum margin classier, i.e. the linear classierthat separates the training data with the largest margin. More precisely, de�nethe (geometric) margin of a linear classierh (x) = sign (w.x+ b) on an example

(xi, yi) εRnX {−1, 1} εRnX{−1, 1} as γi=yi(w.Xi+b)‖W‖

where ||w|| denotes the Euclidean norm of w. (Note that the distance of xifrom the hyperplane [w.x+ b = 0]

Let us look at our decision boundary :This separating hyperplane equationis : wtx + b = 0. where wεRm,xεRm,bεR.Note that w

‖w‖ is orthogonal to the

26

separating hyperplane and its length is 1. Let γibe the distance between thehyperplane and Some training example xi . So γi is the length of the segmentfrom p to xi.

Learning linear SVM

It is convenient to represent classes by +1 and -1 using y = 1; if wx+b >0 , -1; if wx+b < 0 w can be rescaled such that for all points x lying on therespective boundaries it holds that wx+b = 1 or wx+b = -1 These points arecalled the support vectors

The task of learning a linear SVM consists of estimating the parameters w

and b The �rst criterion is that all points in the training data must be classi�ed

correctly: w.xi + b ≥ 1 if yi = 1 w.xi+b ≤ -1 if yi = -1 This can be re-written

as: yi(w.xi+b) ≥ 1 for 1≤ i ≤ N

27