1

GENETIC PROGRAMMING

Ranga RodrigoMarch 17, 2014

2

INTRODUCTION• Viewed by many researchers as a specialization of

GA.• Where GAs use string (or vector) representations,

GP uses a tree representation.• Originally, GP was developed by Koza to evolve

computer programs.• For each generation, each evolved program

(individual) is executed to measure its performance within the problem domain.

• The result obtained from the evolved computer program quantifies the fitness of that program.

3

IMPLICATIONS OF TREE-BASED REPRESENTATION

• Adaptive individuals: – In Gas the size of individuals are fixed, – A GP population will usually have individuals of different size,

shape and complexity. – Size: tree depth– Shape: branching factor of nodes in the tree. – The size and shape of a specific individual too may change

due to application of the reproduction operators.• Domain-specific grammar– A grammar needs to be defined that accurately reflects the

problem to be solved. – It should be possible to represent any possible solution using

the defined grammar.

4

GRAMMAR FOR CHROMOSOME REPRESENTATION

• A grammar forms an important part of chromosome representation.

• As part of the grammar, a terminal set, function set, and semantic rules need to be defined. – The terminal set specifies all the variables and constants– The function set contains all the functions that can be

applied to the elements of the terminal set. – These functions may include mathematical, arithmetic

and/or Boolean functions. Decision structures such as if-then-else and loops can also be included in the function set.

5

GRAMMAR IN TREE TERMINOLOGY• Using tree terminology, elements of the terminal

set form the leaf• nodes of the evolved tree, and elements of the

function set form the non-leaf nodes.• For a specific problem, the search space consists of

the set of all possible trees that• can be constructed using the defined grammar.

6

EXAMPLE: XORx1 x2 Target OP

0 0 0

0 1 1

1 0 1

1 1 0

2121 xxxxy

(x1 AND NOT x2) OR (NOT x1 AND x2)

OR

AND AND

x2

NOT NOTx1 x2

x1

Function set: {AND, OR, NOT}Terminal set: {x1, x2}

7

EXAMPLE: MATHEMATICAL EXPRESSION

5.3)1exp(

)sin()ln(

zaxy

Y = x*ln(a)+sin(z)/exp(-x) – 3.5

+

* -

a

ln /x 3.5

z

sin

x

exp

-

Function set: {-, +, *, /, sin, exp, ln}Terminal set: {a, x, z, 3.5}

8

INITIAL POPULATION• The initial population is generated randomly within

the restrictions of – a maximum depth and – semantics as expressed by the given grammar.

• The branching factor of the root, and each non-terminal node are determined by the arity of the selected function.

• For each non-root node, the initialization algorithm randomly selects an element either from the terminal set or the function set.

• When an element from the terminal set is selected, the particular node becomes a leaf node.

9

FITNESS FUNCTION• Because individuals usually represent a program,

calculation of fitness requires the program to be evaluated against a number of test cases.

• E.g.: – For the Boolean expression, fitness is calculated as the

number of correctly predicted target outputs. – For the mathematical expression a data set of sample

input patterns and associated target output is needed.

10

CROSSOVER OPERATORS• Generating one offspring: – A random node is selected within each of the parents.

Crossover then proceeds by replacing the corresponding subtree in the one parent by that of the other parent.

• Generating two offspring: – A random node is selected in each of the two parents. – The corresponding subtrees are swapped to create the

two offspring

11

ONE OFFSPRING

12

TWO OFFSPRING

13

MUTATION OPERATORS• Function node mutation: – A non-terminal node, or function node, is randomly

selected and replaced with a node of the same arity, randomly selected from the function set.

• Terminal node mutation: – A leaf node, or terminal node, is randomly selected and

replaced with a new terminal node, also randomly selected from the terminal set.

• Swap mutation: – A function node is randomly selected and the arguments

of that node are swapped

14

MUTATION OPERATORS• Grow mutation: – With grow mutation a node is randomly selected and

replaced by a randomly generated subtree. The new subtree is restricted by a predetermined depth.

• Gaussian mutation: – A terminal node that represents a constant is randomly

selected and mutated by adding a Gaussian random value to that constant.

• Trunc mutation: – A function node is randomly selected and replaced by a

random terminal node. This mutation operator performs a pruning of the tree.

15

FUNCTIONAL NODE MUTATION

16

TERMINAL NODE MUTATION

17

SWAP MUTATION

18

GROW MUTATION

19

GAUSSIAN MUTATION

20

TRUNC MUTATION

ADVANTAGES OF GP• GP does not impose any fixed length of the solution.

In principle, the maximal length can be extended up to the hardware limits.

• GP does not require as much knowledge about the problem and the possible solutions as do GAs.

• GPs can theoretically evolve any series of actions a computer can possibly do, provided that we give the GP algorithm a set of commands to choose from that can describe all possible actions.

Slide from Erdem KOÇ & Ömer UZEL

ADVANTAGES OF GA• They efficiently search the model space, so they are

more likely to converge toward a global minima.• There is no need of linearization of the problem.• There is no need to compute partial derivatives.• More probable models are sampled more

frequently than less probable ones.

Slide from Erdem KOÇ & Ömer UZEL

APPLİCATİON FİELDS OF GP• Symbolic Regression and

Function Synthesis (Performing symbolic regression)

• Grammar Induction• Data Mining and Data Analysis

(Data mining and classification)• Electrical Engineering and

Circuit Design (Designing analog circuits)

• Medicine • Economics and Finance • Geometry and Physics • Cellular Automata and Finite

State Machines

• Automated Programming• Robotics• Networking and

Communication• Evolving Behaviors for Agents or

Game Players• Pattern Recognition• Biochemistry• Machine Learning• Discovery of quantum

algorithms• Image processing• Creating security protocols

Slide from Erdem KOÇ & Ömer UZEL