Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit...

17
Genetic Programming: Finding Approximate Analytic Solution to First-Order Ordinary Differential Equations

TAGS:

Transcript of Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit...

Page 1: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Genetic Programming:Finding Approximate Analytic Solution to First-Order Ordinary Differential Equations

Page 2: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Differential Equations

Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b

(1)Closed-form solution: explicit formula -y’ = y at y(0)=1

(Separable) Ans: y = e^x

(2) Numerical solution: y’ = (x*y)/(x+y) + y^2 at y(0)=1

Can’t solve explicitly… So we’ll use numerical methods: (a) Dormand-Prince (b) ode45: returns [x,y] values

Page 3: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Regression / Curve-fitting Given data points, find best fitting curve

• Normal regression finds coefficients of some predefined function (ie linear regression)

• Symbolic Regression finds function and coefficients Since normal requires human intuition, we’ll use symbolic

regression using genetic programming

Page 4: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Genetic Programming Basics

Tree Structure Function Nodes from a Function Set Terminal Nodes (numbers, variables) from

a Terminal Set Create Random

Function Trees

Making the function: y = x*sin(3.63/(2+x))

Page 5: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Function Tree = Individual Use Polish Notation

• Operator comes first. Ex: 5 + x – (x * 8)) + 5 – x * x 8 Quiz: + 3 * 6 – 2 1 = ?

Syntax-preserving: binary or unary functions?• Binary: takes two nodes• Unary: takes one node

Page 6: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Fitness Evaluation Evaluate error at each point Fitness= -Total Sum of Error

If we want greater accuracy, we will want to evaluate more

points in domain

We may also want to limit the maximum error between any sample point and the function

Page 7: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Reproduction Random tournament to select who

mates• Randomly select two individuals in

population• The one with the best fit is parent 1• Randomly select crossover or mutation• If mutate, mutate parent 1• Else find parent 2 and crossover• Repeat reproduction until

Population size is filled

Page 8: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Reproductive Operator: Crossover

Crossover (GP) Crossover (DNA analogy)

Parent 1: Parent 2:

(a+b*sqrt(b)) / sqrt(b+b) a*(((b/b)/a) * sqrt(b+b))

Child: a+(b*(b/b)/a) / sqrt(b+b) ( Throw away second child)

Page 9: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Reproductive Operator: Mutation Randomly select node in tree to

mutate Randomly select a node of the same

type Mutate node

Mutation gets us out of pitfalls

Page 10: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Parameters Determine Number of Data Points in Domain

(Matlab)• Step size: 0.001

Determine Function Set• F = {sin, cos, e^, /, *, +, -}

Determine Terminal Set• T = {x, pi/2, 100 random numbers between -5 and 5}

Create a Random Population of Function Trees• 10,000 indiv. w/ initial depth of 5

Set Minimum Tolerance (error)• Tolerance: 0.01

Set Maximum Num of Generations• Max num of generations: 600

Set Crossover and Mutation rate• Crossover: 80% Mutation: 1-Crossover = 20%

Page 11: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Results: Finding a Closed-form Solution

y’=-y*cos(x) Solvable Ans: y=1 / e^(sin(x)) Using Genetic Programming

• Pop size: 1000• Domain [0 , 5]

Problem• Protected division

Page 12: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Program Evaluation: Finding: y=1 / e^(sin(x))

Page 13: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Results: Finding an Approximation

y’=cos(2*x*y^2)• No closed form solution• Polyfit fails in matlab

Page 14: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Results: y’ = cos(2*x*y^2)

…the explicit solution is…

Page 15: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Program Evaluation

…the explicit solution is…

Page 16: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

The Approximate Solution y = e^( ((((x / cos( sin( ((cos( -1.3977331148909333) / (x * ((x - (sin( e^( 0.5839073910575703)) / ((0.5839073910575703 + (((cos( -1.0768808474282068) / (((x

- e^( (sin( (x + (-2.1183334622298835 + cos( -1.3977331148909333)))) * (sin( (x + 4.404029716618364)) / (x + x ))))) / -3.9294994016950726) * cos( cos( cos( ((x + -2.1183334622298835) / (((-3.488290785681106 / 4.429086745576107) - (4.64613256685335 + (2.032700600098037 / (cos( x ) / (3.951688988841374 / cos( sin( (((-1.6691938170386078 / cos( ((2.032700600098037 / -1.3977331148909333) / cos( cos( sin( (0.5839073910575703 + (0.5839073910575703 + (cos( 0.21588984462299265) / cos( 0.5839073910575703)))))))))) + 0.5839073910575703) + (0.5839073910575703 + (cos( 0.21588984462299265) / cos( 0.5839073910575703))))))))))) + -2.1183334622298835))))))) / -1.3977331148909333) + (cos( ((0.5839073910575703 / (0.5839073910575703 * 0.5839073910575703)) * (sin( (x + e^( 0.5839073910575703))) / (x - cos( e^( 0.5839073910575703)))))) / cos( 0.5839073910575703)))) / (3.951688988841374 / cos( sin( (0.5839073910575703 + (0.5839073910575703 + (cos( 0.21588984462299265) / cos( 0.5839073910575703)))))))))) * -1.3977331148909333))) / (sin( -4.09932580082895) / -1.3977331148909333))))) / cos( sin( ((cos( -1.3977331148909333) / (x * ((x - (sin( e^( 0.5839073910575703)) / ((0.5839073910575703 + (((cos( -1.0768808474282068) / (((0.5839073910575703 - e^( (sin( (x + -2.1183334622298835)) * (sin( (x + 4.404029716618364)) / (x + x ))))) / -3.9294994016950726) * cos( cos( cos( 0.5839073910575703))))) / -1.3977331148909333) + (cos( (e^( 0.5839073910575703) * (sin( (x + e^( 0.5839073910575703))) / (x - cos( e^( 0.5839073910575703)))))) / cos( 0.5839073910575703)))) / (3.951688988841374 / cos( sin( (0.5839073910575703 + (0.5839073910575703 + (x / cos( 0.5839073910575703)))))))))) * -1.3977331148909333))) / (sin( -4.09932580082895) / -1.3977331148909333))))) - e^( (sin( (cos( (cos( (((0.5839073910575703 / (0.5839073910575703 * 0.5839073910575703)) * (sin( (x + sin( (((2.032700600098037 / -1.3977331148909333) / 0.5839073910575703) + -2.1183334622298835)))) / (x - cos( e^( 0.5839073910575703))))) + -1.6691938170386078)) / (((e^( x ) + sin( e^( (x / cos( cos( (x / e^( 0.5839073910575703)))))))) - ((x / cos( sin( (((0.5839073910575703 - cos( -1.3977331148909333)) / (x * (3.141592653589793 * -1.3977331148909333))) / (x / -1.3977331148909333))))) / -1.3977331148909333)) + ((0.5839073910575703 / ((x / cos( cos( x ))) - (0.5839073910575703 * cos( cos( (0.5839073910575703 / (((-3.9294994016950726 * sin( x )) - cos( (0.5839073910575703 - (0.5839073910575703 * cos( (x / 0.5839073910575703)))))) + -2.1183334622298835))))))) - cos( -1.3977331148909333))))) + (x + sin( (((2.032700600098037 / -1.3977331148909333) / sin( 0.5839073910575703)) + -2.1183334622298835))))) * sin( (cos( (x / (cos( (x - (((cos( -1.0768808474282068) / (((x - e^( (sin( (x + (x - ((x - (sin( e^( 3.9169117826293363)) / ((0.5839073910575703 + (((cos( -1.0768808474282068) / (((3.141592653589793 - e^( (sin( (x - (sin( (x - 0.5839073910575703)) + 0.5839073910575703))) * (sin( (x + 4.404029716618364)) / (x / 4.5366847258566025))))) - -3.9294994016950726) + cos( cos( cos( ((x - -2.1183334622298835) / (((-3.488290785681106 / -3.423767380013757) / (4.496363738665307 + (2.032700600098037 + (cos( -2.9602495684565455) / (3.951688988841374 / (x - cos( e^( 0.5839073910575703)))))))) + -2.1183334622298835))))))) * -1.3977331148909333) * (cos( ((0.5839073910575703 / (0.5839073910575703 * 0.5839073910575703)) * (sin( (-1.7908824571411053 + e^( 0.5839073910575703))) / (x - cos( e^( x )))))) + e^( 0.5839073910575703)))) / (3.951688988841374 / cos( sin( (0.5839073910575703 + (-2.8354922153845505 + (cos( 0.21588984462299265) / cos( 0.5839073910575703)))))))))) * -1.3977331148909333)))) * sin( (cos( cos( 0.5839073910575703)) + (sin( 0.5839073910575703) / (x - ((((2.032700600098037 / -1.3977331148909333) / sin( ((sin( (cos( (x - (x + e^( 0.5839073910575703)))) + sin( (x + cos( sin( e^( (0.5839073910575703 * (sin( (x + 4.404029716618364)) / (x + x )))))))))) / cos( x )) / x ))) + -2.1183334622298835) + 0.5839073910575703)))))))) / -3.9294994016950726) * cos( 0.5839073910575703))) / -1.3977331148909333) / ((x - (cos( 0.5839073910575703) + (sin( e^( 0.5839073910575703)) / (((x - (sin( e^( (((0.5839073910575703 / ((x / cos( cos( x ))) - (0.5839073910575703 * cos( (cos( 4.404029716618364) / (x + cos( (-1.6691938170386078 / x )))))))) - cos( (sin( (x - cos( ((0.5839073910575703 / (0.5839073910575703 * (sin( 0.5839073910575703) * cos( 0.5839073910575703)))) * (0.5839073910575703 / (0.5839073910575703 * 0.5839073910575703)))))) * -1.3977331148909333))) / -4.09932580082895))) / (4.5366847258566025 / (3.951688988841374 / cos( -1.3977331148909333))))) * -1.3977331148909333) - cos( e^( 0.5839073910575703)))))) / cos( cos( (x - cos( e^( 0.5839073910575703))))))))) + 3.951688988841374))) + (sin( e^( 0.5839073910575703)) / (x - cos( e^( 0.5839073910575703))))))))) / -3.9294994016950726))

Not simplified (Maxima nor Matlab could simplify) Fitness penalty for length

Page 17: Initial Value Problem: Find y=f(x) if y’ = f(x,y) at y(a)=b (1) Closed-form solution: explicit formula -y’ = y at y(0)=1 (Separable) Ans: y = e^x (2)

Sources Adapted Tiny_GP Java code Riccardo Poli. Found at http://www.gp-field-guide.org.uk/

Burgess, Glenn. “Finding Approximate Analytic Solutions to Differential Equations” commissioned by Department of Defense (Australia) in 1999

Koza, John. Genetic Programming

Riccardo Poli

http://www.gp-field-guide.org.uk/

ö - Echo360€¦ · - /$)"x )x #*÷ûôx1$ *x ..$")( )/x$)x/# x ** ' x x + " xõx*!xö yyyyy y # y1$ *y ..$")( )/y! /0- y ''*2.y./0 )/.y/*y - / y $/y( ) " y ) y.0 ($/y1$ *y *)/ )/y!*-

ö - Echo360€¦ · - /$)"x )x #*÷ûôx1$ *x ..$")( )/x$)x/# x ** ' x x + " xõx*!xö yyyyy y # y1$ *y ..$")( )/y! /0- y ''*2.y./0 )/.y/*y - / y $/y( ) " y ) y.0 ($/y1$ *y *)/ )/y!*-

1 Elementary Functions - Elsevier.com · 2013. 12. 20. · 3. cos2 x−sin2 y =cos(x+y)cos(x−y) = cos2 y −sin2 x 4. sinh2 x+cosh2 y =cosh(x+y)cosh(x−y)=cosh2 x+sinh2 y 1.316

1 Elementary Functions - Elsevier.com · 2013. 12. 20. · 3. cos2 x−sin2 y =cos(x+y)cos(x−y) = cos2 y −sin2 x 4. sinh2 x+cosh2 y =cosh(x+y)cosh(x−y)=cosh2 x+sinh2 y 1.316

R Z Y X Wx X Y X R X Z Y X Y X Y X Y X Y X Y X Y X ΩY ΩY X ...pages.cs.wisc.edu/~jerryzhu/pub/NIPS10_MC_poster.pdf · Transduction with Matrix Completion: Three Birds with One Stone

R Z Y X Wx X Y X R X Z Y X Y X Y X Y X Y X Y X Y X ΩY ΩY X ...pages.cs.wisc.edu/~jerryzhu/pub/NIPS10_MC_poster.pdf · Transduction with Matrix Completion: Three Birds with One Stone

Separable Equations Including the Logistic Equation...6.5 Separable Equations Including the Logistic Equation 259 61 For any power n, Problem 6.2.59 proved ex> xnfor large x. Then

Separable Equations Including the Logistic Equation...6.5 Separable Equations Including the Logistic Equation 259 61 For any power n, Problem 6.2.59 proved ex> xnfor large x. Then

Separable Space

Separable Space

3D Forward and Back-Projection for X-Ray CT Using Separable Footprints

3D Forward and Back-Projection for X-Ray CT Using Separable Footprints

Separable Monte Carlo

Separable Monte Carlo

Separable Preference Orders

Separable Preference Orders

Algebra Days 1-10 · a2 a x a y³ ÷ y 4 = y³ = y x y x y = 1 = y −1 y4 y x y x y x y y 1 (y³)² = (y x y x y )² = y x y x y x y x y x y = y 6 y0 = 1 Recognising and collecting

Algebra Days 1-10 · a2 a x a y³ ÷ y 4 = y³ = y x y x y = 1 = y −1 y4 y x y x y x y y 1 (y³)² = (y x y x y )² = y x y x y x y x y x y = y 6 y0 = 1 Recognising and collecting

Separable Verbs

Separable Verbs

Ordinary Differential Equations·¥程數學(一... · 2016-09-23 · O. D. E. Chapter2_9 . Solving this for 1 x. m y, and multiplying by . x. m 1, we have . is. separable. ( ) (

Ordinary Differential Equations·¥程數學(一... · 2016-09-23 · O. D. E. Chapter2_9 . Solving this for 1 x. m y, and multiplying by . x. m 1, we have . is. separable. ( ) (

NMHC Internship Manual 2019-2020 Final NMHC Psychology... · ï 7deoh ri &rqwhqwv :hofrph dqg 2xu 0lvvlrq y y y y y y y y y y y y x x x x x x x x x x x x x x x x x x x x x x x x x

NMHC Internship Manual 2019-2020 Final NMHC Psychology... · ï 7deoh ri &rqwhqwv :hofrph dqg 2xu 0lvvlrq y y y y y y y y y y y y x x x x x x x x x x x x x x x x x x x x x x x x x

AP Calculus Ms. Battaglia. Equations are separable if all x terms can be collected with dx and all y terms with dy. The solution procedure is called separation.

AP Calculus Ms. Battaglia. Equations are separable if all x terms can be collected with dx and all y terms with dy. The solution procedure is called separation.

Summer Work Packet For Students Entering Algebra 1 Honors€¦ · x x y 8) x x y y x x y 10) y x x 11) y x x y 12) y x x y ©Y i2j0 Z1j2 3-2-xKauIt ya 1 uSqozf xtSw 3a or 1ee wLhLqCL.0

Summer Work Packet For Students Entering Algebra 1 Honors€¦ · x x y 8) x x y y x x y 10) y x x 11) y x x y 12) y x x y ©Y i2j0 Z1j2 3-2-xKauIt ya 1 uSqozf xtSw 3a or 1ee wLhLqCL.0

Long-Time Behaviour of Nonlinear Fokker-Planck Equations · 2018-07-03 · formulae, we consider two separable metric space Xand Y and denote by P(X), P(Y) the set of all probability

Long-Time Behaviour of Nonlinear Fokker-Planck Equations · 2018-07-03 · formulae, we consider two separable metric space Xand Y and denote by P(X), P(Y) the set of all probability

Town of MinocquaTRL Voting Wards · 2018. 1. 3. · xy xy xy xy xy xy xy xy xy xy xy xy xy xy xy xy xy xy x y x y x y x y x y x y x x x x y x y x y x y x y x y x y x y x y x y x y

Town of MinocquaTRL Voting Wards · 2018. 1. 3. · xy xy xy xy xy xy xy xy xy xy xy xy xy xy xy xy xy xy x y x y x y x y x y x y x x x x y x y x y x y x y x y x y x y x y x y x y

Separable Connectors - ElekNet

Separable Connectors - ElekNet

DSP Slide 1 Graph theory x y y = x x y a y = a x x z y y = x and z = x xz y z = x + y z = x - y xz y - y = z -1 x x y z -1 DSP graphs are made up of points.

DSP Slide 1 Graph theory x y y = x x y a y = a x x z y y = x and z = x xz y z = x + y z = x - y xz y - y = z -1 x x y z -1 DSP graphs are made up of points.

K x x y x - WordPress.com · 2018. 6. 10. · 6 Q | x | y a x { x y | | y | | : ^ x x x | x { x x x x x | | x x y | x , x { x x x ~

K x x y x - WordPress.com · 2018. 6. 10. · 6 Q | x | y a x { x y | | y | | : ^ x x x | x { x x x x x | | x x y | x , x { x x x ~

Basic Sets - Mathematics | Michigan State University · MTH299 - Examples Scratch Work: x x 1 = y x= y(x 1) x= yx y x xy= y x(1 y) = y x= y 1 y and let x= y 1 y then we can see that:

Basic Sets - Mathematics | Michigan State University · MTH299 - Examples Scratch Work: x x 1 = y x= y(x 1) x= yx y x xy= y x(1 y) = y x= y 1 y and let x= y 1 y then we can see that: