Review of elements of Calculus(functions in more than one variable)
Partly adapted from the lectures of prof Piero Fariselli(University of Bologna)
yx
z 2 2, 100z f x y x y
x
y
10
10
10
10
100
sketch of graph level curves
Level curves are drawn by holding the z value constant (similar to contour lines on a topographic map.)
Functions of two variables
Partial derivatives are defined as derivatives of a function of multiple variables when all but the variable of interest are held fixed during the differentiation.
Definition of Partial Derivatives of a Function of Two VariablesIf z = f(x,y), the the first partial derivatives of f with respect to x and y are the functions fx and fy defined by
0
0
, ( , ), lim
, ( , ), lim
x x
y y
f x x y f x yf x y
x
f x y y f x yf x y
y
Provided the limits exist.
Partial Derivatives
To find the partial derivatives, hold one variable constant and differentiate with respect to the other.
Example 1: Find the partial derivatives fx and fy for the function
4 2 2 3( , ) 5 2f x y x x y x y
To find the partial derivatives, hold one variable constant and differentiate with respect to the other.
Example 1: Find the partial derivatives fx and fy for the function
4 2 2 3( , ) 5 2f x y x x y x y
Solution:
4 2 2 3
3 2 2
2 3
( , ) 5 2
( , ) 20 2 6
( , ) 2 2
x
y
f x y x x y x y
f x y x y x yx
f x y x y x
Notation for First Partial Derivative
For z = f(x,y), the partial derivatives fx and fy are denoted by
( , ) ,
( , ) ,
x x
y y
zf x y f x y z
x x
and
zf x y f x y z
y y
The first partials evaluated at the point (a,b) are denoted by
( , ) ( , ), ,a b x a b y
z zf a b and f a b
x y
Example 2: Find the partials fx and fy and evaluate them at the indicated point for the function
( , ) (2, 2)xy
f x y atx y
Example 2: Find the partials fx and fy and evaluate them at the indicated point for the function
( , ) (2, 2)xy
f x y atx y
Solution:
2 2
2 2 2
2
2
2 2
2 2 2
2
2
( , ) (2, 2)
,( ) ( ) ( )
2 4 12, 2
16 4(2 2 )
,( ) ( ) ( )
4 12, 2
16 4( )
x
x
y
y
xyf x y at
x y
x y y xy xy y xy yf x y
x y x y x y
f
x y x xy x xy xy xf x y
x y x y x y
xf
x y
The following slide shows the geometric interpretation of the partial derivative. For a fixed x, z = f(x0,y) represents the curve formed by intersecting the surface z = f(x,y) with the plane x = x0.
0 0,xf x y represents the slope of this curve at the point (x0,y0,f(x0,y0))
Thanks to http://astro.temple.edu/~dhill001/partial-demo/For the animation.
Definition of Partial Derivatives of a Function of Three or More VariablesIf w = f(x,y,z), then there are three partial derivatives each of which is formed by holding two of the variables
0
0
0
, , ( , , ), , lim
, , ( , , ), , lim
, , ( , , ), , lim
x x
y y
z z
f x x y z f x y zwf x y z
x x
f x y y z f x y zwf x y z
y y
f x y z z f x y zwf x y z
z z
In general, if
1 2
1 2
( , ,... )
, ,... , 1,2,...k
n
x nk
w f x x x there are n partial derivatives
wf x x x k n
x
where all but the kth variable is held constant
Notation for Higher Order Partial Derivatives
Below are the different 2nd order partial derivatives:
yx
xy
yy
xx
fyx
f
y
f
y
fxy
f
x
f
y
fy
f
y
f
y
fx
f
x
f
x
2
2
2
2
2
2
Differentiate twice with respect to x
Differentiate twice with respect to y
Differentiate first with respect to x and then with respect to y
Differentiate first with respect to y and then with respect to x
Theorem
If f is a function of x and y such that fxy and fyx are continuous on an open disk R, then, for every (x,y) in R,
fxy(x,y)= fyx(x,y)
Example 3:
Find all of the second partial derivatives of yxyxyyxf 22 523),(
Work the problem first then check.
Example 3:
Find all of the second partial derivatives of yxyxyyxf 22 523),(
xyyxf
xxyyxf
yxyxyyxf
xyyxf
xyyyxf
yxyxyyxf
xyxf
xxyyxf
yxyxyyxf
yyxf
xyyyxf
yxyxyyxf
yx
y
xy
x
yy
y
xx
x
106),(
526),(
523),(
106),(
103),(
523),(
6),(
526),(
523),(
10),(
103),(
523),(
2
22
2
22
2
22
2
22
Notice that fxy = fyx
Example 4: Find the following partial derivatives for the function
zxyezyxf x ln),,(
a. xzf
b. zxf
c. xzzf
d. zxzf
e. zzxf
Work it out then go to the next slide.
Example 4: Find the following partial derivatives for the function
zxyezyxf x ln),,(
a. xzf
b. zxf
zzyxf
zyezyxf
zxyezyxf
xz
x
x
x
1),,(
ln),,(
ln),,(
zzyxf
z
xzyxf
zxyezyxf
zx
z
x
1),,(
),,(
ln),,(
Again, notice that the 2nd
partials fxz = fzx
c. xzzf
d. zxzf
e. zzxf
2
1),,(
1),,(
ln),,(
ln),,(
zzyxf
zzyxf
zyezyxf
zxyezyxf
xzz
xz
x
x
x
2
1),,(
1),,(
),,(
ln),,(
zzyxf
zzyxf
z
xzyxf
zxyezyxf
zxz
zx
z
x
2
2
1),,(
),,(
),,(
ln),,(
zzyxf
z
xzyxf
z
xzyxf
zxyezyxf
zzx
zz
z
x
NoticeAll
Are Equal
f: RnR. If f(x) is of class C2, objective function
Gradient of f
Is a vector containing all the partial derivatives of the first order
Gradient
Given a function f(xy) and a level curve f(x,y) = c
the gradient of f is:
Consider 2 points of the curve: (x,y); (x+εx, x+εy), for small ε
The Gradient is locally perpendicular to level curves
y
f
x
ff ,
(x,y)(x+εx, y+εy)
),(
),(),(
,
,,
yx
T
yx
y
yx
xyx
gyxf
y
f
x
fyxfyxf
ε
Since both : (x,y); (x+εx, x+εy), points satisfy the curve equation:
The gradient is perpendicular to ε.For small ε, ε is parallel to the curve and,by consequence, thegradient is perpendicular to the curve.
The gradient points towards the direction of maximumincrease of f
The local perpendicular to a curve: Gradient
(x,y)(x+εx, x+εy)
0),(
),(
yx
T
yx
T
f
fcc
ε
ε
ε
grad (f)
f: RnR. If f(x) is of class C2, objective function
Hessian of f
Is a square matrix of order n containing all the partial derivatives of the second order
H=
Since the mixed partial derivatives are equal irrespectively of the order of derivation, the matrix is symmetric
Hessian
Taylor expansion
If the variables are represented with a n-valued vector x, the Taylor expansion around a point x* is
.....*)(*)(2
1*)(*)()(
* xxHxxxxxfxfxf T
x
Local optima (without constraints)
The condition to have a local optimum in x* is
i.e.: all the partial derivatives must be null
In order to study whether the point is a minimum or a maximum, the matrix H must be considered
H is a nxn symmetric real matrix: it has n real eigenvalues
The point x* isa maximum if all eigenvectors are positive, a minimum if all the eigenvectors are negative
)(
*0 n
xf
Saddle points
When eigenvalues of the Hessian matrix have differentsign , the critical poin is a Saddle point
Constrained optimization problems: Lagrange Multipliers
10/12/2015 28
We want to maximise the function z = f(x,y)subject to the constraint g(x,y) = c (curve in the x,y plane)
Aim
Solve the constraint g(x,y) = c and express, for example, y=h(x)
The substitute in function f and find the maximum in x of
f(x, h(x))
Analytical solution of the constraint can be very difficult
Simple solution
Contour lines of a function
Contour lines of f and constraint
Suppose we walk along the constraint line g (x,y)= c.
In general the contour lines of f are distinct from the constraint g (x,y)= c.
While moving along the constraint line g (x,y)= c the value of fvary (that is, different contour levels for f are intersected).
Only when the constraint line g (x,y)= c touches the contour lines of f in a tangential way, we do not increase or decrease the value of f: the function f is at its local max or min along the constraint.
Lagrange Multipliers
Geometrical interpretation
Contour line and constraint are tangential: their localperpendicular to are parallel
Normal to a curve
Lagrange Multipliers
On the point of g(x,y)=c that
Max-min-imize f(x,y), the gradient
of f is perpendicular to the curve
g(x,y) =c, otherwise we should increase or decrease f by
moving locally on the curve.
So, the two gradients are parallel
for some scalar λ (where is the gradient).
Thus we want points (x,y) where g(x,y) = c and
,
To incorporate these conditions into one equation, we introduce
an auxiliary function (Lagrangian)
and solve
.
Lagrange Multipliers
cyxgyxfyxF ),(),(),,(
Recap of Constrained Optimization
Suppose we want to: minimize/maximize f(x) subject to g(x) = 0
A necessary condition for x0 to be a solution:
a: the Lagrange multiplier
For multiple constraints gi(x) = 0, i=1, …, m, we need a Lagrange multiplier ai for each of the constraints
-
-
Top Related