Second Order Heuristics in ACGP

Second Order Heuristics in

ACGP

John AleshunasUniversity of Missouri –

Saint [email protected]

Cezary JanikowUniversity of Missouri –


Mark W HauschildUniversity of Missouri –


OutlineThe Genetic Programming (GP) ModelIssues with the GP ModelConstrained Genetic Programming (CGP)Adaptable Constrained GP (ACGP)Adaptable Constrained GP (ACGP)

HeuristicsACGP Learning ResultsConclusions

2

The Genetic Programming (GP) Model

Evolve a solution program from a population of candidates

Represents individual solutions as variable size trees

Employs standard genetic operators Selection Crossover Mutation

3

Issues with the GP Model

The Function set and Terminal set (labels) define the representation space of a GP implementation

Often the Function set and Terminal set are over specified to ensure that they cover the solution space

Closure Sufficiency

This over specification increases the representation space and increases search time

4

Issues with the GP Model

Search is not completely biased Mutation is always random Crossover is random given the current population

(driven by the current population but random there) Selection is the only bias

Selection drives the population and thus affects crossover

How to improve the bias? Building “models” and using them to drive the search

5

Constrained Genetic Programming (CGP)

An Option to Reduce Complexity

Developed by Dr. Cezary Janikow in 1996It uses user-specified “model” 1st order heuristics parent-child

Global at the root Local anywhere

Demonstrated to dramatically improve search time and complexity for problems such as SantaFe and 11-multiplexer

But the user has to give the “right” heuristics – discovery of the right heuristics can be a long process

6

Adaptable Constrained GP (ACGP)Automating the CGP Methodology

Developed by Dr. Cezary Janikow in 2003ACGP discovers 1st order heuristics Assumes that best trees are the “correct” heuristics Adjusts the heuristics at generational intervals Regrows the population using the new heuristics Mutation and crossover are now biased using the

heuristics Uses gradual adjustments of the heuristics to control

convergence

7

GP Building BlocksA Digression

8

The building block hypothesis states:Genetic processes work by combining relatively fit, short schema to form complete solutions.

a

b

Adaptable Constrained GP (ACGP)1st Order Heuristics

Controls one argument to a functionDevelops a set of heuristicsLooks at parent node (function) and one

child node (function or terminal)Treats those schema that are frequent in

the fittest population members as desirable

Gradual heuristic adjustment prevents premature convergence

9

Adaptable Constrained GP (ACGP)2nd Order Heuristics

Controls all arguments to a functionDevelops a set of heuristicsLooks at parent node (function) and all

child nodes (function or terminal)Treats those schema that are frequent in

the fittest population members as desirable

Gradual heuristic adjustment prevents premature convergence

10

Adaptable Constrained GP (ACGP)2nd Order Heuristics

Consider x*x 1st order can only speak of * and one argument at a

time Suppose that x* is 10% and that *x is 10% In the absence of heuristics about the entire subtree

x*x, the probability of x*x will be joint as 1% But the true probability of x*x in the problem can be

up to 10% Processing 1st order will always process it as 1% We need 2nd order (subtree) to process this as 10%

We need Mechanism to discover 1st and 2nd order structures Validate performance gains of 2nd order vs. 1st order

11

Where is the Advantage in ACGP?

Discovery of fit building blocks

Promotes the use of fit building blocks in crossover, mutation and regrow

Creates a population of highly fit compact solutions

ACGP discovers heuristics that improve fitness primarily but also reduce tree sizes

12

How ACGP Develops Its Heuristics

Tabulate the frequency of building blocks in the population

Tabulate the frequency of building blocks in the fittest individuals in the population

Increase the probability of fit heuristicsSuppress the probability of less fit

heuristicsAccess to heuristics is constant because of

the tabulation Disregarding paging

13

Representation of Heuristics in ACGP1st Order Heuristics

The general form of the 1st order combinations is

Using 4 binary functions, 14 terminals, this creates 162 combinations

14

Representation of Heuristics in ACGP2nd Order Heuristics

The general form of the 2nd order combinations is

Using 4 binary functions, 14 terminals, this creates 2592 combinations

15

Representation of Heuristics in ACGPHigher Order Heuristics

The general form of the 3rd order combinations is

Using 4 binary functions, 14 terminals, this creates 13,728,800 combinations

4th OH 3.76 * 1014 combinations 5th OH 2.84 * 1029 combinations 6th OH 1.62 * 1059 combinations

16

ACGP 2nd order Implementation was painfully validated

But experiments with Santafe and 11-multiplexer using 1st order and 2nd order gave the same results

Means that these problems do not have 2nd order structure not implied by joint probabilities

For validation, needed a function with obvious difference between 1st and 2nd order probabilities

17

Experimental Setup Target Equation: x*x + y*y + z*z

Has very explicit 2nd order structure different from that implied by 1st order

Function set: {+, -, *, /} (protected divide) Terminal set: {x, y, z, -5, -4, -3, -2, -1, 0, 1, 2, 3, 4, 5} Population size: 500 Generations: 500 Operators: crossover 85%, mutation 10%, selection 5%,

regrow 100% at each iteration Number of independent runs: 30 Fitness: sum of square errors on 100 random data points

in the range -10 to 10 Iteration length: 20 generations

18

ACGP Learning Results

19

Find the equation

Comparison of GP-Base, ACGP with 1st order and 2nd order heuristics.

Training slope-off with 50% update


20

Find the equation


Training slope-on


21

Find the equation


Training slope-on, first five iterations


22

Comparison of GP-Base, Strong 1st Order and Strong 2nd Order Heuristics on 200 generation, no iterations

Find the equation


23

Find the equation

Comparison of Base, 1st Order and 2nd Order Heuristics on a Time Scale


24

Average

Best Tree Size

Best Tree Depth

Execution Time

Base 728.40 19.43 347.6

1st OH 123.67 10.37 44.70

2nd OH 123.87 11.40 65.67

Find the equation

Average tree structure for Base, 1st Order and 2nd Order Heuristics

ACGP Learning Limits

25

Find the equation

Comparison of Base, 1st Order and 2nd Order Heuristics with Training Slope on for an extended Bowl3 equation

ACGP Learning ResultsDesirable 1st order heuristics

Implied 2nd order heuristics

Desirable 2nd order heuristics

26

{*1 , x} {*2 , x}{*1 , z} {*2 , z}{*2 , y}{*1 , y}{+1 , +} {+2 , +} {+2 , *}{+1 , *}

{*, x, x} {*, x, z} {*, y, x} {*, y, z}{*, x, y} {*, y, y}

{*, z, x} {*, z, z}{*, z, y}

{+, +, +} {+, *, +} {+, *, *}{+, +, *}

{*, x, x} {*, y, y} {*, z, z}{+, *, +} {+, *, *}{+, +, *}


27

1st order heuristics

2nd order heuristics

{*1 , x} {*2 , x}{*1 , z} {*2 , z}{*2 , y}{*1 , y}

0.2289 0.2426 0.2403 0.2323 0.2489 0.2087ACGP 1

{*, x, x} {*, x, z} {*, y, y} {*, z, z}{*, x, y} {*, y, z}

0.0532 0.0570 0.0478 0.0604 0.0506 0.0502ImpliedACGP 1

0.2545 0.0001 0.0001 0.2368 0.0001 0.2436ActualACGP 2


28

1st Order HeuristicsHeuristic Initial Final

‘*’

Left argX 0.056 0.2289Y 0.056 0.2426Z 0.056 0.2403

Right argX 0.056 0.2323Y 0.056 0.2489Z 0.056 0.2087

‘+’Left arg

‘*’ 0.056 0.4796‘+’ 0.056 0.2171

Right arg‘*’ 0.056 0.4168‘+’ 0.056 0.2410

Root ‘+’ 0.056 0.7669Average of all other heuristics 0.056 0.0371

First-order heuristics discovered. Root’s heuristics are zero-order for

Bowl3 equation


29

Second-order heuristics summary for ‘*’ for Bowl3 equation

Multiply Heuristics

Heuristic Initial Final

‘*’

X X 0.0031 0.2545

Y Y 0.0031 0.2368

Z Z 0.0031 0.2436

Average of all other heuristics

0.0031 0.0008


30

Second-order heuristics summary for ‘+’ for Bowl3 equation

Addition HeuristicsHeuristic Initial Final‘+’ ‘*’ ‘*’ 0.0031 0.3110‘*’ ‘+’ 0.0031 0.0688‘+’ ‘*’ 0.0031 0.1289Average of all other heuristics 0.0031 0.0015

Conclusions and Further Work

31

The 1st and 2nd order heuristics of ACGP provide a significant performance improvement over the base GP

If very strong second-order heuristics are present, ACGP is able to process them and also to discover them

Running 2nd order doesn’t hurt if there is no explicit 2nd order not implied by 1st

Overhead is minimal If a problem does not have explicit second-order

structure, ACGP running in first or second-order mode is the same

Going higher than 2nd order has drawbacks.The next step will be to move on to standard test

or real world problems

SummaryThe Genetic Programming (GP) ModelIssues with the GP ModelConstrained Genetic Programming (CGP)Adaptable Constrained GP (ACGP)Adaptable Constrained GP (ACGP)

HeuristicsACGP Learning ResultsConclusions

32

References Banzhaf, W., Nordin, P., Keller, R., Francone, F., Genetic Programming – An

Introduction, Morgan Kaufmann, 1998

Janikow, Cezary Z. A Methodology for Processing Problem Constraints in Genetic Programming, Computers and Mathematics with Applications. 32(8):97-113, 1996.

Janikow, Cezary Z., Deshpande, Rahul, Adaptation of Representation in GP. AMS 20032

Janikow, Cezary Z. ACGP: Adaptable Constrained Genetic Programming. In O’Reilly, Una-May, Yu, Tina, and Riolo, Rick L., editors. Genetic Programming Theory and Practice (II). Springer,

New York, NY, 2005, 191-206

Janikow, Cezary Z., and Mann, Christopher J. CGP Visits the Santa Fe Trail – Effects of Heuristics on GP. GECCO’05, June 25-29, 2005

Koza, John R. Genetic Programming. The MIT Press. 1992.

Koza, John R. Genetic Programming II. The MIT Press. 1994.

Looks, Moshe, Competent Program Evolution, Sever Institute of Washington University, December 2006 33

References Looks, Moshe, Competent Program Evolution, Sever Institute of Washington University,

December 2006

McKay, Robert I., Hoai, Nguyen X., Whigham, Peter A., Shan, Yin, O’Neill, Michael, Grammar-based Genetic Programming: a survey, Genetic Programming and Evolvable Machines, Springer Science + Business Media, September 2010

Poli, Riccardo, Langdon, William, B., Schema Theory for Genetic Programming with One-point Crossover and Point Mutation, Evolutionary Computation, MIT Press, Fall 1998

Sastry, Kumara, O’Reilly, Una-May, Goldberg, David, Hill, David, Building-Block Supply in Genetic Programming, IlliGAL Report No. 2003012, April 2003

Shan, Yin, McKay, Robert, Essam, Daryl, Abbass, Hussein, A Survey of Probabilistic Model Building Genetic Programming, The Artificial Life and Adaptive Robotics Laboratory, School of Information Technology and Electrical Engineering, University of New South Wales, Australia, 2005

Burke, Edmund, Gustafson, Steven, and Kendall, Graham. A survey and analysis of diversity measures in genetic programming. In Langdon, W., Cantu-Paz, E. Mathias, K., Roy, R., Davis, D., Poli, R., Balakrishnan, K., Honavar, V., Rudolph, G., Wegener, J., Bull, L., Potter, M., Schultz, A., Miller, J., Burke, E. and Jonoska, N., editors. GECCO2002: Proceedings of the Genetic and Evolutionary Computation Conference, 716-723, New York. Morgan Kaufmann.

34

References Daida, Jason, Hills, Adam, Ward, David, and Long, Stephen. Visualizing tree

structures in genetic programming. In Cantu-Paz, E., Foster, J., Deb, K., Davis, D., Roy, R., O’Reilly, U., Beyer, H., Standish, R., Kendall, G., Wilson, S., Harman, M., Wegener, J., Dasgupta, D., Potter, M., Schultz, A., Dowsland, K., Jonoska, N., and Miller, J., editors, Genetic and Evolutionary Computation – GECCO-2003, volume 2724 of LNCS, 1652-1664, Chicago. Springer Verlag.

Hall, John M. and Soule, Terence. Does Genetic Programming Inherently Adopt Structured Design Techniques? In O’Reilly, Una-May, Yu, Tina, and Riolo, Rick L., editors. Genetic Programming Theory and Practice (II). Springer, New York, NY, 2005, 159-174.

Langon, William. Quadratic bloat in genetic programming. In Whitley, D., Goldberg, D., Cantu-Paz, E., Spector, L., Parmee, I., and Beyer, H-G., editors, Proceedings of the Genetic and Evolutionary Conference GECCO 2000, 451-458, Las Vegas. Morgan Kaufmann.

McPhee, Nicholas F. and Hopper, Nicholas J. Analysis of genetic diversity through population history. In Banzhaf, W., Daida, J., Eiben, A. Garzon, M. Honavar, V., Jakiela, M. and Smith, R., editors Proceedings of the Genetic and Evolutionary Computation Conference, volume 2, pages 1112-1120, Orlando, Florida, USA. Morgan Kaufmann.

Franz Rothlauf. Representations for Genetic and Evolutionary Algorithm. Springer, 2010.

35

Second Order Heuristics in ACGP

Technology

Transcript of Second Order Heuristics in ACGP