Review of elements of Calculus (functions in more than one ...
Transcript of Review of elements of Calculus (functions in more than one ...
Review of elements of Calculus(functions in more than one variable)
Partly adapted from the lectures of prof Piero Fariselli(University of Bologna)
yx
z 2 2, 100z f x y x y
x
y
10
10
10
10
100
sketch of graph level curves
Level curves are drawn by holding the z value constant (similar to contour lines on a topographic map.)
Functions of two variables
Partial derivatives are defined as derivatives of a function of multiple variables when all but the variable of interest are held fixed during the differentiation.
Definition of Partial Derivatives of a Function of Two VariablesIf z = f(x,y), the the first partial derivatives of f with respect to x and y are the functions fx and fy defined by
0
0
, ( , ), lim
, ( , ), lim
x x
y y
f x x y f x yf x y
x
f x y y f x yf x y
y
Provided the limits exist.
Partial Derivatives
To find the partial derivatives, hold one variable constant and differentiate with respect to the other.
Example 1: Find the partial derivatives fx and fy for the function
4 2 2 3( , ) 5 2f x y x x y x y
To find the partial derivatives, hold one variable constant and differentiate with respect to the other.
Example 1: Find the partial derivatives fx and fy for the function
4 2 2 3( , ) 5 2f x y x x y x y
Solution:
4 2 2 3
3 2 2
2 3
( , ) 5 2
( , ) 20 2 6
( , ) 2 2
x
y
f x y x x y x y
f x y x y x yx
f x y x y x
Notation for First Partial Derivative
For z = f(x,y), the partial derivatives fx and fy are denoted by
( , ) ,
( , ) ,
x x
y y
zf x y f x y z
x x
and
zf x y f x y z
y y
The first partials evaluated at the point (a,b) are denoted by
( , ) ( , ), ,a b x a b y
z zf a b and f a b
x y
Example 2: Find the partials fx and fy and evaluate them at the indicated point for the function
( , ) (2, 2)xy
f x y atx y
Example 2: Find the partials fx and fy and evaluate them at the indicated point for the function
( , ) (2, 2)xy
f x y atx y
Solution:
2 2
2 2 2
2
2
2 2
2 2 2
2
2
( , ) (2, 2)
,( ) ( ) ( )
2 4 12, 2
16 4(2 2 )
,( ) ( ) ( )
4 12, 2
16 4( )
x
x
y
y
xyf x y at
x y
x y y xy xy y xy yf x y
x y x y x y
f
x y x xy x xy xy xf x y
x y x y x y
xf
x y
The following slide shows the geometric interpretation of the partial derivative. For a fixed x, z = f(x0,y) represents the curve formed by intersecting the surface z = f(x,y) with the plane x = x0.
0 0,xf x y represents the slope of this curve at the point (x0,y0,f(x0,y0))
Thanks to http://astro.temple.edu/~dhill001/partial-demo/For the animation.
Definition of Partial Derivatives of a Function of Three or More VariablesIf w = f(x,y,z), then there are three partial derivatives each of which is formed by holding two of the variables
0
0
0
, , ( , , ), , lim
, , ( , , ), , lim
, , ( , , ), , lim
x x
y y
z z
f x x y z f x y zwf x y z
x x
f x y y z f x y zwf x y z
y y
f x y z z f x y zwf x y z
z z
In general, if
1 2
1 2
( , ,... )
, ,... , 1,2,...k
n
x nk
w f x x x there are n partial derivatives
wf x x x k n
x
where all but the kth variable is held constant
Notation for Higher Order Partial Derivatives
Below are the different 2nd order partial derivatives:
yx
xy
yy
xx
fyx
f
y
f
y
fxy
f
x
f
y
fy
f
y
f
y
fx
f
x
f
x
2
2
2
2
2
2
Differentiate twice with respect to x
Differentiate twice with respect to y
Differentiate first with respect to x and then with respect to y
Differentiate first with respect to y and then with respect to x
Theorem
If f is a function of x and y such that fxy and fyx are continuous on an open disk R, then, for every (x,y) in R,
fxy(x,y)= fyx(x,y)
Example 3:
Find all of the second partial derivatives of yxyxyyxf 22 523),(
Work the problem first then check.
Example 3:
Find all of the second partial derivatives of yxyxyyxf 22 523),(
xyyxf
xxyyxf
yxyxyyxf
xyyxf
xyyyxf
yxyxyyxf
xyxf
xxyyxf
yxyxyyxf
yyxf
xyyyxf
yxyxyyxf
yx
y
xy
x
yy
y
xx
x
106),(
526),(
523),(
106),(
103),(
523),(
6),(
526),(
523),(
10),(
103),(
523),(
2
22
2
22
2
22
2
22
Notice that fxy = fyx
Example 4: Find the following partial derivatives for the function
zxyezyxf x ln),,(
a. xzf
b. zxf
c. xzzf
d. zxzf
e. zzxf
Work it out then go to the next slide.
Example 4: Find the following partial derivatives for the function
zxyezyxf x ln),,(
a. xzf
b. zxf
zzyxf
zyezyxf
zxyezyxf
xz
x
x
x
1),,(
ln),,(
ln),,(
zzyxf
z
xzyxf
zxyezyxf
zx
z
x
1),,(
),,(
ln),,(
Again, notice that the 2nd
partials fxz = fzx
c. xzzf
d. zxzf
e. zzxf
2
1),,(
1),,(
ln),,(
ln),,(
zzyxf
zzyxf
zyezyxf
zxyezyxf
xzz
xz
x
x
x
2
1),,(
1),,(
),,(
ln),,(
zzyxf
zzyxf
z
xzyxf
zxyezyxf
zxz
zx
z
x
2
2
1),,(
),,(
),,(
ln),,(
zzyxf
z
xzyxf
z
xzyxf
zxyezyxf
zzx
zz
z
x
NoticeAll
Are Equal
f: RnR. If f(x) is of class C2, objective function
Gradient of f
Is a vector containing all the partial derivatives of the first order
Gradient
Given a function f(xy) and a level curve f(x,y) = c
the gradient of f is:
Consider 2 points of the curve: (x,y); (x+εx, x+εy), for small ε
The Gradient is locally perpendicular to level curves
y
f
x
ff ,
(x,y)(x+εx, y+εy)
),(
),(),(
,
,,
yx
T
yx
y
yx
xyx
gyxf
y
f
x
fyxfyxf
ε
Since both : (x,y); (x+εx, x+εy), points satisfy the curve equation:
The gradient is perpendicular to ε.For small ε, ε is parallel to the curve and,by consequence, thegradient is perpendicular to the curve.
The gradient points towards the direction of maximumincrease of f
The local perpendicular to a curve: Gradient
(x,y)(x+εx, x+εy)
0),(
),(
yx
T
yx
T
f
fcc
ε
ε
ε
grad (f)
f: RnR. If f(x) is of class C2, objective function
Hessian of f
Is a square matrix of order n containing all the partial derivatives of the second order
H=
Since the mixed partial derivatives are equal irrespectively of the order of derivation, the matrix is symmetric
Hessian
Taylor expansion
If the variables are represented with a n-valued vector x, the Taylor expansion around a point x* is
.....*)(*)(2
1*)(*)()(
* xxHxxxxxfxfxf T
x
Local optima (without constraints)
The condition to have a local optimum in x* is
i.e.: all the partial derivatives must be null
In order to study whether the point is a minimum or a maximum, the matrix H must be considered
H is a nxn symmetric real matrix: it has n real eigenvalues
The point x* isa maximum if all eigenvectors are positive, a minimum if all the eigenvectors are negative
)(
*0 n
xf
Saddle points
When eigenvalues of the Hessian matrix have differentsign , the critical poin is a Saddle point
Constrained optimization problems: Lagrange Multipliers
10/12/2015 28
We want to maximise the function z = f(x,y)subject to the constraint g(x,y) = c (curve in the x,y plane)
Aim
Solve the constraint g(x,y) = c and express, for example, y=h(x)
The substitute in function f and find the maximum in x of
f(x, h(x))
Analytical solution of the constraint can be very difficult
Simple solution
Contour lines of a function
Contour lines of f and constraint
Suppose we walk along the constraint line g (x,y)= c.
In general the contour lines of f are distinct from the constraint g (x,y)= c.
While moving along the constraint line g (x,y)= c the value of fvary (that is, different contour levels for f are intersected).
Only when the constraint line g (x,y)= c touches the contour lines of f in a tangential way, we do not increase or decrease the value of f: the function f is at its local max or min along the constraint.
Lagrange Multipliers
Geometrical interpretation
Contour line and constraint are tangential: their localperpendicular to are parallel
Normal to a curve
Lagrange Multipliers
On the point of g(x,y)=c that
Max-min-imize f(x,y), the gradient
of f is perpendicular to the curve
g(x,y) =c, otherwise we should increase or decrease f by
moving locally on the curve.
So, the two gradients are parallel
for some scalar λ (where is the gradient).
Thus we want points (x,y) where g(x,y) = c and
,
To incorporate these conditions into one equation, we introduce
an auxiliary function (Lagrangian)
and solve
.
Lagrange Multipliers
cyxgyxfyxF ),(),(),,(
Recap of Constrained Optimization
Suppose we want to: minimize/maximize f(x) subject to g(x) = 0
A necessary condition for x0 to be a solution:
a: the Lagrange multiplier
For multiple constraints gi(x) = 0, i=1, …, m, we need a Lagrange multiplier ai for each of the constraints
-
-