Convex Analysis
-
Upload
khoirun-nisa -
Category
Documents
-
view
31 -
download
2
description
Transcript of Convex Analysis
-
Convex Analysis
Jose De Dona
September 2004
Centre of Complex DynamicSystems and Control
-
Outline1 Convex Sets
Definition of a Convex SetExamples of Convex SetsConvex ConesSupporting HyperplaneSeparation of Disjoint Convex Sets
2 Convex FunctionsDefinition of a Convex FunctionProperties of Convex FunctionsConvexity of Level SetsContinuity of Convex Functions
3 Generalisations of Convex FunctionsQuasiconvex FunctionsPseudoconvex FunctionsRelationship Among Various Types of ConvexityConvexity at a Point
Centre of Complex DynamicSystems and Control
-
Convex Analysis
The notion of convexity is crucial to the solution of many realworld problems. Fortunately, many problems encountered inconstrained control and estimation are convex.
Convex problems have many important properties foroptimisation problems. For example, any local minimum of aconvex function over a convex set is also a global minimum.
Centre of Complex DynamicSystems and Control
-
Convex Sets
We have the following definition of a convex set.
Definition (Convex Set)A set S Rn is convex if the line segment joining any two points ofthe set also belongs to the set. In other words, if x1, x2 S thenx1 + (1 )x2 must also belong to S for each [0, 1].
The Figure illustrates thenotions of convex andnonconvex sets. Note that,in case (b), the linesegment joining x1 and x2does not lie entirely in theset.
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
x1x1
x2x2
(a) Convex (b) Nonconvex
Centre of Complex DynamicSystems and Control
-
Examples of Convex Sets
The following are some examples of convex sets:
(i) Hyperplane. S = {x : ptx = }, where p is a nonzero vectorin Rn, called the normal to the hyperplane, and is a scalar.
(ii) Half-space. S = {x : ptx }, where p is a nonzero vector inR
n, and is a scalar.
(iii) Open half-space. S = {x : ptx < }, where p is a nonzerovector in Rn and is a scalar.
(iv) Polyhedral set. S = {x : Ax b}, where A is an m n matrix,and b is an m vector. (Here, and in the remainder of thesenotes, the inequality should be interpreted elementwise.)
Centre of Complex DynamicSystems and Control
-
Examples of Convex Sets
(v) Polyhedral cone. S = {x : Ax 0}, where A is an m nmatrix.
(vi) Cone spanned by a finite number of vectors.S = {x : x = mj=1 jaj , j 0, for j = 1, . . . ,m}, wherea1, . . . , am are given vectors in Rn.
(vii) Neighbourhood. N(x) = {x Rn : ||x x || < }, where x is afixed vector in Rn and > 0.
Centre of Complex DynamicSystems and Control
-
Convex ConesSome of the geometric optimality conditions that we will study useconvex cones.
Definition (Convex Cone)A nonempty set C in Rn is called a cone with vertex zero if x Cimplies that x C for all 0. If, in addition, C is convex, then Cis called a convex cone.
00 xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
xxxxxxxxxxxxx
Convex cone Nonconvex cone
Centre of Complex DynamicSystems and Control
-
Separation and Support of Convex SetsA well-known geometric fact is that, given a closed convex set Sand a point y S, there exists a unique point x S with minimumdistance from y.Theorem (Closest Point Theorem)Let S be a nonempty, closed convex set in Rn and y S. Then,there exists a unique point x S with minimum distance from y.Furthermore, x is the minimising point, or closest point to y, if andonly if (y x)t(x x) 0 for all x S.
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xx
xx
xx
xx
xx
xx
xx
xx
xx
xx
xx
xx
xx
xx
xx
xxx
SS
y
y
Convex set Nonconvex set
Centre of Complex DynamicSystems and Control
-
Separation of Convex SetsAlmost all optimality conditions and duality relationships use somesort of separation or support of convex sets.Definition (Separation of Sets)Let S1 and S2 be nonempty sets in Rn. A hyperplaneH = {x : ptx = } separates S1 and S2 if ptx for each x S1and ptx for each x S2. If, in addition, ptx + for eachx S1 and ptx for each x S2, where is a positive scalar,then the hyperplane H is said to strongly separate the sets S1 andS2. (Notice that strong separation implies separation of sets.)
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
Separation Strong separation
S1S1
S2S2 H H
Centre of Complex DynamicSystems and Control
-
Separation of Convex SetsThe following is the most fundamental separation theorem.
Theorem (Separation Theorem)Let S be a nonempty closed convex set in Rn and y S. Then,there exists a nonzero vector p and a scalar such that pty > and ptx for each x S.
Outline of the Proof:
By the Closest Point Theorem, thereexists a unique minimising point x S suchthat: (y x
p0
)t(x x) 0, x S.
Letting = ptx, we have:ptx , x S, andpty = pty ptx = (y x)t(y x)= y x2 > 0 pty > .
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xx
Sy p
Centre of Complex DynamicSystems and Control
-
Supporting Hyperplane
Definition (Supporting Hyperplane at a Boundary Point)Let S be a nonempty set in Rn, and let x S (the boundary of S).A hyperplane H = {x : pt(x x) = 0} is called a supportinghyperplane of S at x if either pt(x x) 0 for each x S, or else,pt(x x) 0 for each x S.
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
xx
p
H
S
Centre of Complex DynamicSystems and Control
-
Supporting Hyperplane
A convex set has a supporting hyperplane at each boundary point.
Theorem (Supporting Hyperplane)Let S be a nonempty convex set in Rn, and let x S. Then thereexists a hyperplane that supports S at x; that is, there exists anonzero vector p such that pt(x x) 0 for each x cl S.
As a corollary, we have a result similar to the SeparationTheorem, where S is not required to be closed.
CorollaryLet S be a nonempty convex set in Rn and x int S. Then there isa nonzero vector p such that pt(x x) 0 for each x cl S.
Centre of Complex DynamicSystems and Control
-
Separation of Disjoint Convex SetsIf two convex sets are disjoint, then they can be separated by ahyperplane.Theorem (Separation of Two Disjoint Convex Sets)Let S1 and S2 be nonempty convex sets in Rn and suppose thatS1 S2 is empty. Then there exists a hyperplane that separates S1and S2; that is, there exists a nonzero vector p in Rn such that
inf{ptx : x S1} sup{ptx : x S2}.
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
S1
S2 H
Centre of Complex DynamicSystems and Control
-
Separation of Disjoint Convex SetsThe previous result (Separation of Two Disjoint Convex Sets) holdstrue even if the two sets have some points in common, as long astheir interiors are disjoint.CorollaryLet S1 and S2 be nonempty convex sets in Rn. Suppose that int S2is not empty and that S1 int S2 is empty. Then, there exists ahyperplane that separates S1 and S2; that is, there exists anonzero p such that
inf{ptx : x S1} sup{ptx : x S2}.
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
xxxxxxxxxxxxxxxxxxx
S1
S2 HCentre of Complex DynamicSystems and Control
-
Convex Functions
We will present some properties of convex functions, beginningwith their definition.Definition (Convex Function)Let f : S R, where S is a nonempty convex set in Rn. Thefunction f is convex on S if
f (x1 + (1 )x2) f (x1) + (1 )f (x2)for each x1, x2 S and for each (0, 1).The function f is strictly convex on S if the above inequality is trueas a strict inequality for each distinct x1, x2 S and for each (0, 1).The function f is (strictly) concave on S if f is (strictly) convex onS.
Centre of Complex DynamicSystems and Control
-
Convex Functions
The geometric interpretation of a convex function is that the valueof f at the point x1 + (1 )x2 is less than the height of the chordjoining the points [x1, f (x1)] and [x2, f (x2)]. For a concave function,the chord is below the function itself.
Convex function Concave function Neither convexnor concave
x1x1x1 x2x2x2
ff
f
x1+(1)x2x1+(1)x2
Centre of Complex DynamicSystems and Control
-
Properties of Convex Functions
The following are useful properties of convex functions.
(i) Let f1, f2, . . . , fk : Rn R be convex functions. Thenf (x) = kj=1 j fj(x), where j > 0 for j = 1, 2, . . . k , is a convexfunction;f (x) = max{f1(x), f2(x), . . . , fk (x)} is a convex function.
(ii) Suppose that g : Rn R is a concave function. LetS = {x : g(x) > 0}, and define f : S R as f (x) = 1/g(x).Then f is convex over S.
Centre of Complex DynamicSystems and Control
-
Properties of Convex Functions
(iii) Let g : R R be a nondecreasing, univariate, convexfunction, and let h : Rn R be a convex function. Then thecomposite function f : Rn R defined as f (x) = g(h(x)) is aconvex function.
(iv) Let g : Rm R be a convex function, and let h : Rn Rm bean affine function of the form h(x) = Ax + b, where A is anm n matrix, and b is an m 1 vector. Then the compositefunction f : Rn R defined as f (x) = g(h(x)) is a convexfunction.
Centre of Complex DynamicSystems and Control
-
Convexity of Level Sets
Associated with a convex function f is the level set S defined as:S = {x S : f (x) }, R. We then have:Lemma (Convexity of Level Sets)Let S be a nonempty convex set in Rn and let f : S R be aconvex function. Then the level set S = {x S : f (x) }, where R, is a convex set.
Proof.Let x1, x2 S S . Thus, f (x1) and f (x2) . Let (0, 1)and x = x1 + (1 )x2 S (S convex). By convexity of f ,
f (x) f (x1) + (1 )f (x2) + (1 ) = .Hence, x S, and we conclude that S is convex.
Centre of Complex DynamicSystems and Control
-
Continuity of Convex Functions
An important property of convex functions is that they arecontinuous on the interior of their domain.Theorem (Continuity of Convex Functions)Let S be a nonempty convex set in Rn and let f : S R be aconvex function. Then f is continuous on the interior of S.
Consider, for example, the followingfunction defined onS = {x : 1 x 1}:
f (x) =
x2 for | x |< 12 for | x |= 1
Obviously, f is convex over S, but iscontinuous only on the interior of S. -1 1
1
2
x
Centre of Complex DynamicSystems and Control
-
Generalisations of Convex Functions
We present various types of functions that are similar to convex orconcave functions but share only some of their desirableproperties.
Definition (Quasiconvex Function)Let f : S R, where S is a nonempty convex set in Rn. Thefunction f is quasiconvex if, for each x1, x2 S, the followinginequality is true:
f (x1 + (1 )x2) max {f (x1), f (x2)} for each (0, 1).The function f is quasiconcave if f is quasiconvex.
Note, from the definition, that a convex function is quasiconvex.
Centre of Complex DynamicSystems and Control
-
Quasiconvex FunctionsFrom the above definition, a function f is quasiconvex if, wheneverf (x2) f (x1), f (x2) is greater than or equal to f at all convexcombinations of x1 and x2. Hence, if f increases locally from itsvalue at a point along any direction, it must remain nondecreasingin that direction. The figure shows some examples of quasiconvexand quasiconcave functions.
(a) (b) (c)
(a) Quasiconvex function. (b) Quasiconcave function. (c) Neitherquasiconvex nor quasiconcave.
Centre of Complex DynamicSystems and Control
-
Level sets of Quasiconvex Functions
The following result states that a quasiconvex function ischaracterised by the convexity of its level sets.
Theorem (Level Sets of a Quasiconvex Function)Let f : S R, where S is a nonempty convex set in Rn. Thefunction f is quasiconvex if and only if S = {x S : f (x) } isconvex for each real number .
Centre of Complex DynamicSystems and Control
-
Strictly Quasiconvex Functions
Definition (Strictly Quasiconvex Function)Let f : S R, where S is a nonempty convex set in Rn. Thefunction f is strictly quasiconvex if, for each x1, x2 S withf (x1) f (x2), the following inequality is true
f (x1 + (1 )x2) < max {f (x1), f (x2)} for each (0, 1).The function f is strictly quasiconcave if f is strictly quasiconvex.
Note from the above definition that a convex function is also strictlyquasiconvex.
Centre of Complex DynamicSystems and Control
-
Strictly Quasiconvex FunctionsNotice that the definition precludes any flat spots from occurringanywhere except at extremising points. This, in turn, implies that alocal optimal solution of a strictly quasiconvex function over aconvex set is also a global optimal solution.The figure shows some examples of quasiconvex and strictlyquasiconvex functions.
(a) (b) (c)
(a) Strictly quasiconvex function. (b) Strictly quasiconvex function.(c) Quasiconvex function but not strictly quasiconvex.
Centre of Complex DynamicSystems and Control
-
Pseudoconvex Functions
We will next introduce another type of function that generalises theconcept of a convex function, called a pseudoconvex function.Pseudoconvex functions share the property of convex functionsthat, if f (x) = 0 at some point x, then x is a global minimum of f .Definition (Pseudoconvex Function)Let S be a nonempty open set in Rn, and let f : S R bedifferentiable on S. The function f is pseudoconvex if, for eachx1, x2 S with f (x1)(x2 x1) 0, then f (x2) f (x1); or,equivalently, if f (x2) < f (x1), then f (x1)(x2 x1) < 0.The function f is pseudoconcave if f is pseudoconvex.The function f is strictly pseudoconvex if, for each distinct x1, x2 Swith f (x1)(x2 x1) 0, then f (x2) > f (x1); or, equivalently, if foreach distinct x1, x2 S, f (x2) f (x1), then f (x1)(x2 x1) < 0.
Centre of Complex DynamicSystems and Control
-
Pseudoconvex FunctionsNote that the definition asserts that if the directional derivative of apseudoconvex function at any point x1 in the direction x2 x1 isnonnegative, then the function values are nondecreasing in thatdirection.
The figure shows examples of pseudoconvex and pseudoconcavefunctions.
(a) (b) (c)
Inflection point
(a) Pseudoconvex function. (b) Both pseudoconvex andpseudoconcave. (c) Neither pseudoconvex nor pseudoconcave.
Centre of Complex DynamicSystems and Control
-
Relationship Among Various Types of ConvexityThe arrows mean implications. In general, the converses do nothold.
Strictly convex
Convex
Pseudoconvex
Strictlyquasiconvex
Quasiconvex
Strictlypseudoconvex
Under differentiability
Under differentiability
Under lowersemicontinuity
Centre of Complex DynamicSystems and Control
-
Convexity at a Point
In some optimisation problems, the requirement of convexity maybe too strong and not essential, and convexity at a point may be allthat is needed. Hence, we present several types of convexity at apoint that are relaxations of the various forms of convexitypresented so far.
Centre of Complex DynamicSystems and Control
-
Convexity at a Point
Definition (Various Types of Convexity at a Point)Let S be a nonempty convex set in Rn, and f : S R. We thenhave the following definitions:Convexity at a point. The function f is said to be convex at x S if
f (x + (1 )x) f (x) + (1 )f (x) for each (0, 1) and each x S.
Strict convexity at a point. The function f is strictly convex at x Sif f (x + (1 )x) < f (x) + (1 )f (x) for each (0, 1) and each x S, x x.
Quasiconvexity at a point. The function f is quasiconvex at x S iff (x + (1 )x) max {f (x), f (x)} for each (0, 1)and each x S.
Centre of Complex DynamicSystems and Control
-
Convexity at a Point
Definition (Various Types of Convexity at a Point)Strict quasiconvexity at a point. The function f is strictly
quasiconvex at x S iff (x + (1 )x) < max {f (x), f (x)} for each (0, 1)and each x S such that f (x) f (x).
Pseudoconvexity at a point. Suppose f is differentiable atx int S. Then f is pseudoconvex at x iff (x)(x x) 0 for x S implies that f (x) f (x).
Strict pseudoconvexity at a point. Suppose f is differentiable atx int S. Then f is strictly pseudoconvex at x iff (x)(x x) 0 for x S , x x, implies thatf (x) > f (x).
Centre of Complex DynamicSystems and Control
-
Convexity at a Point
The figure illustrates some types of convexity at a point.
(a) f is quasiconvex but not strictlyquasiconvex at x1; f is both quasiconvexand strictly quasiconvex at x2.
(b) f is both pseudoconvex and strictlypseudoconvex at x1; f is pseudoconvexbut not strictly pseudoconvex at x2.
(a)
(b)
f
f
x1
x1
x2
x2
Centre of Complex DynamicSystems and Control
OutlineConvex SetsDefinition of a Convex SetExamples of Convex SetsConvex ConesSupporting HyperplaneSeparation of Disjoint Convex Sets
Convex FunctionsDefinition of a Convex FunctionProperties of Convex FunctionsConvexity of Level SetsContinuity of Convex Functions
Generalisations of Convex FunctionsQuasiconvex FunctionsPseudoconvex FunctionsRelationship Among Various Types of ConvexityConvexity at a Point