ENE 2XX: Renewable Energy Systems and Control...Powers. xa is convex on the set of all positive x,...

ENE 2XX: Renewable Energy Systems and Control

LEC 01 : Convex Sets, Functions, & Minimizers

Professor Scott MouraUniversity of California, Berkeley

Summer 2017

Prof. Moura | Tsinghua-Berkeley Shenzhen Institute ENE 2XX | LEC 01 - Convex Sets,Fcns,Min Slide 1

Mathematical Preliminaries

Convex Sets

Convex Functions

Minimizers


Outline

1 Convex Sets

2 Convex Functions

3 Minimizers


Convex Sets

A set D is convex if the line segment connecting any two points in Dcompletely lies in D. We formalize this concept into the following definition.

Definition (Convex Set)Let D be a subset of Rn. Also, consider scalar parameter λ ∈ [0,1] and twopoints a,b ∈ D. The set D is convex if

λa + (1− λ)b ∈ D (1)

for all points a,b ∈ D.


Test your knowledge

Figure: Some simple convex and nonconvex sets. [Left] The hexagon, which includesits boundary (shown darker), is convex. [Middle] The kidney shaped set is notconvex, since the line segment between the two points in the set shown as dots isnot contained in the set. [Right] The square contains some boundary points but notothers, and is not convex.


Important Examples of Convex Sets - I

The empty set , any single point (i.e. a singleton), {x0}, and the wholespace Rn are convex.

Any line in Rn is convex.

Any line segment in Rn is convex.

A ray, which has the form {x0 + θv | θ ≥ 0, v 6= 0} is convex.

A hyperplane, which has the form{

x | aTx = b}

, where a ∈ Rn, a 6= 0,b ∈ R.


Important Examples of Convex Sets - II

A halfspace, which has the form{

x | aTx ≤ b}

, where a ∈ Rn, a 6= 0,b ∈ R. See Fig. 2.A Euclidean ball in Rn, which is centered at xc and has radius r. Think ofthe Euclidean ball as a sphere in n-dimensions (See Fig. 3.).Mathematically, the Euclidean ball is

B(xc, r) = {x | ‖x− xc‖2 ≤ r} ={

x | (x− xc)T(x− xc) ≤ r2}

(2)

x0

a

Figure: The shaded set is thehalfspace given by{x | aT(x− x0) ≤ 0} Vector a points inthe outward normal direction of thehalfspace.

xc

r

Figure: The Euclidean Ball B(xc, r) iscentered at xc and has radius r. Atwo-dimensional example is shown.


Important Examples of Convex Sets - Ellipsoid

An ellipsoid in Rn, which is centered at xc. Think of an ellipsoid as an ellipsein n-dimensions. Mathematically,

E ={

x | (x− xc)TA−1(x− xc) ≤ 1}

(3)

where A = AT � 0, i.e. A is symmetric and positive definite. The matrix Aencodes how the ellipse extends in each direction of Rn. In particular, thelength of the semi-axes for the ellipse are given by

√λi(A), where λi is the

ith eigenvalue of A. See Fig. on next slide. Another common representationof an ellipsoid, which we use later in this chapter, is:

E = {xc + Pu | ‖u‖2 ≤ 1} (4)

where P is square and positive semi-definite. The semi-axis lengths aregiven by λi(P) in this case. If we define P = A2, then this representation isequivalent to (3).


Important Examples of Convex Sets - Ellipsoid

xc

Figure: An ellipsoid E = {xc + Pu | ‖u‖2 ≤ 1} in two-dimensions with center xc. Thesemi-axes have length given by λi where λi =eig(P).


Important Examples of Convex Sets - Polyhedron

A polyhedron is defined by the values x ∈ Rn that satisfy a finite set of linearinequalities and linear equalities:

P ={

x | aTi x ≤ bi, i = 1, · · · ,m, aT

eq,jx = beq,j, j = 1, · · · , l}

(3)

See Fig. on next slide. By definition, a polyhedra is the intersection of afinite number of hyperplanes and halfspaces. Thus, all halfspaces,hyperplanes, lines, rays, and line segments are polyhedra. It is convenientto use the compact vector notation

P = {x | Ax ≤ b, Aeqx = beq} (4)


Important Examples of Convex Sets - Polyhedron

P

a1

a2

a3

a4

a5

Figure: The polyhedron P is the intersection of five halfspaces, each with outwardpointing normal vector ai.


Exercise - Square and Disk

Define the square and disk in R2 respectively as

S ={

x ∈ R2| 0 ≤ xi ≤ 1, i = 1,2}, D =

{x ∈ R2| ‖x‖2 ≤ 1

}(3)

Are the following statements TRUE or FALSE:

(a) S ∪ D is convex, i.e. the union of sets S and D is convex

(b) S ∩ D is convex, i.e. the intersection of sets S and D is convex

(c) S D is convex, i.e. set difference of D from S is convex


Outline

1 Convex Sets

2 Convex Functions

3 Minimizers


Convex Functions

Definition (Convex Function)Let D be a convex set. Also, consider scalar parameter λ ∈ [0,1] and twopoints a,b ∈ D. Then the function f(x) is convex on D if

f(x) = f(λa + (1− λ)b) ≤ λf(a) + (1− λ)f(b) (4)


Figure 1: Convex Function

Figure 2: Concave Function

b a

f(x)

f(b)

f(a)

f(x)

0

1

2

3

4

5

6

7

8

9

10

11

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

0≤ λ ≤ 1

X = λa+(1-λ)b

f(a)

f(b)

f(x)

a b

f(x)

0

1

2

3

4

5

6

7

8

9

10

11

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0≤ λ ≤ 1

X = λa+(1-λ)b


Concave Functions

Definition (Concave Function)Let D be a convex set. Also, consider scalar parameter λ ∈ [0,1] and twopoints a,b ∈ D. Then the function f(x) is concave on D if

f(x) = f(λa + (1− λ)b) ≥ λf(a) + (1− λ)f(b) (5)


Figure 1: Convex Function

Figure 2: Concave Function

b a

f(x)

f(b)

f(a)

f(x)

0

1

2

3

4

5

6

7

8

9

10

11

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14

0≤ λ ≤ 1

X = λa+(1-λ)b

f(a)

f(b)

f(x)

a b

f(x)

0

1

2

3

4

5

6

7

8

9

10

11

0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15

0≤ λ ≤ 1

X = λa+(1-λ)b


STRICTLY Convex/Concave Fcns

Definition (Strictly Convex Function)

Let D be a convex set. Also, consider scalar parameter λ ∈ [0,1] and twopoints a,b ∈ D. Then the function f(x) is strictly convex on D if

f(x) = f(λa + (1− λ)b) < λf(a) + (1− λ)f(b) (6)


Definition (Strictly Concave Function)

Let D be a convex set. Also, consider scalar parameter λ ∈ [0,1] and twopoints a,b ∈ D. Then the function f(x) is strictly concave on D if

f(x) = f(λa + (1− λ)b) > λf(a) + (1− λ)f(b) (7)



Exercises

Which of the following functions are convex, concave, neither, or both, overthe set D = [−10,10]? You may use graphical arguments or the definitionsto prove your claim.

(a) f(x) = 0

(b) f(x) = x

(c) f(x) = x2

(d) f(x) = −x2

(e) f(x) = x3

(f) f(x) = sin(x)

(g) f(x) = e−x2

(h) f(x) = |x|


Convex/Concave Function Properties

Consider a function f(x) : Rn → R and compact set D.

1. If f(x) is convex on D, then −f(x) is concave on D.

2. If f(x) is concave on D, then −f(x) is convex on D.

3. f(x) is a convex function on D ⇐⇒ d2fdx2 (x) is positive semi-definite a.e.

∀ x ∈ D.

4. f(x) is a concave function on D ⇐⇒ d2fdx2 (x) is negative semi-definite

a.e. ∀ x ∈ D.


Examples: Scalar Convex/Concave Functions

Consider functions f(x) where x ∈ R is scalar.

Quadratic. 12ax2 + bx + c is convex on R, for any a ≥ 0. It is concave on

R for any a ≤ 0.

Exponential. eax is convex on R, for any a ∈ R.

Powers. xa is convex on the set of all positive x, when a ≥ 1 or a ≤ 0. Itis concave for 0 ≤ a ≤ 1.

Powers of absolute value. |x|p, for p ≥ 1 is convex on R.

Logarithm. log x is concave on the set of all positive x.

Negative entropy. x log x is convex on the set of all positive x.


Examples: Multivariable Convex/Concave Functions

Consider functions f(x) where x ∈ Rn is multivariable.

Norms. Every norm in Rn is convex.

Max function. f(x) = max{x1, · · · , xn} is convex on Rn.

Quadratic-over-linear function. The function f(x, y) = x2/y is convexover all positive x, y.

Log-sum-exp. The function f(x) = log (expx1 + · · ·+ expxn ) is convex onRn. This function can be interpreted as a differentiable (in fact, analytic)approximation of the max function. Consequently, it is extraordinarilyuseful for gradient-based algorithms.

Geometric mean. The geometric mean f(x) = (Πni=1xi)

1/n is concave forall elements of x positive, i.e. {x ∈ Rn | xi > 0 ∀ i = 1, · · · ,n}.

Log determinant. The function f(X) = log det(X) is concave w.r.t. X forall X positive definite. This property is useful in many applications,including optimal experiment design.


Operations that conserve convexity

Linear combinations. Consider αi ≥ 0 for all i. A non-negative weightedsum of convex functions is also convex

f(x) = α1f1(x) + · · ·+ αmfm(x) (8)

Pointwise maximum. If f1(x) and f2(x) are convex functions on D, thentheir point-wise maximum f defined below is convex on D.

f(x) = max{f1(x), f2(x)} (9)

Composition with Affine Mapping. Consider f(·) : Rn → R withparameters A ∈ Rn×m and b ∈ Rn. Define function g(·) : Rm → R as

g(x) = f(Ax + b) (10)

If f is convex, then g is convex. If f is concave, then g is concave.


Outline

1 Convex Sets

2 Convex Functions

3 Minimizers


Definition of Minimizers

Definition (Global Minimizer)x? ∈ D is a global minimizer of f(x) on D if

f(x?) ≤ f(x), ∀ x ∈ D (11)

In words, this means x? minimizes f(x) everywhere in D. In contrast, wehave a local minimizer.

Definition (Local Minimizer)x? ∈ D is a local minimizer of f(x) on D if

∃ ε > 0 s.t. f(x?) ≤ f(x), ∀ x ∈ D ∩ {x ∈ R | ‖x− x?‖ < ε} (12)

In words, this means x? minimizes f(x) locally in D. That is, there existssome neighborhood whose size is characterized by ε where x? minimizesf(x). Examples of global and local minimizers are provided in Fig. on nextslide.


Definition of Minimizers

Figure: The LEFT figure contains two local minimizers, but only one global minimizer.The RIGHT figure contains a local minimizer, which is also the global minimizer.


ENE 2XX: Renewable Energy Systems and Control...Powers. xa is convex on the set of all positive x,...

Documents

Transcript of ENE 2XX: Renewable Energy Systems and Control...Powers. xa is convex on the set of all positive x,...