Lecture 8

Linear programming is about problems of the form

maximize 〈c,x〉subject to Ax ≤ b,

where A ∈ Rm×n, x ∈ Rn, c ∈ Rn, and b ∈ Rm, and the inequality sign meansinequality in each row. The feasible set is the set of all possible candidates,

F = {x ∈ Rn : Ax ≤ b}.

This set can be empty (example: x ≤ 1, −x ≤ −2), unbounded (example: x ≤ 1)or bounded (example: x ≤ 1, −x ≤ 0). In any case, it is a convex set. The set offeasible solutions is the set over which we will search for solutions, and it is thereforeimportant to understand the geometry of this set. The set of the feasible sets of linearprogramming is called a polyhedron.

8.1 Linear Programming Duality: a first glance

Suppose we are faced with a linear programming problem. A first important task is tofind out if there exist any feasible solutions at all. That is, we would like to know ifthe feasible set F is empty or not, i.e., ifAx ≤ b has a solution. If it is not empty, wecan certify that by producing a vector from F . To verify that the feasible set is emptyis more tricky: we are asked to show that no vector lives in F . What we can try todo, however, is to show that the assumption of a solution toAx ≤ b would lead to acontradiction. Denote by a>i the rows of A, so that the entries of Ax are given bya>i x for 1 ≤ i ≤ m. Assuming x ∈ F , then given a vector 0 6= λ = (λ1, . . . , λm)>

with λi ≥ 0, the linear combination satisfies

〈λ,Ax〉 =m∑i=1

λia>i x ≤

m∑i=1

λibi = 〈λ, b〉. (8.1)

If we can find parameters λ such that the left-hand side of (8.1) is identically 0 andthe right-hand side is strictly negative, then we have found a contradition and can

1

2

conclude that no x satisfiesAx ≤ b. A condition that ensures this ism∑i=1

λiai = 0, 〈λ, b〉 < 0. (8.2)

In matrix form,∃λ ≥ 0, A>λ = 0, 〈λ, b〉 < 0.

This condition will still be satisfied if we normalise the vector λ such that∑m

i=1 λi =1, so the statement says that 0 is a convex combination of the vectors defining theequations.

Example 8.1. Consider the system

x1 + x2 ≤ 2

−x1 ≤ −1−x2 ≤ −1.5.

(8.3)

The transpose matrixA> and the vector b are

A> =

(1 −1 01 0 −1

), b =

2−1−1.5.

Drawing the columns ofA> we get

a1

a2

a3

We can get the origin as a convex combination of the vectors ai (drawing arope around them encloses the origin), and such a combination is given by λ =(1/3, 1/3, 1/3). Taking the scalar product with the vector b we get

〈λ, b〉 = 1

3(2− 1− 1.5) = −1

6< 0.

This shows that the system (8.3) does not have a solution (a fact that in this simpleexample can also be seen by drawing a picture).

It turns out that Condition (8.2) is not only sufficient but also necessary, and theseparating hyperplane theorem is an essential part of this. We first make a detour inorder to better understand the feasible sets, the polyhedra.

8.2. POLYHEDRA 3

8.2 Polyhedra

Definition 8.2. A polyhedron (plural: polyhedra) is a set defined as the solution oflinear equalities and inequalities,

P = {x ∈ Rn : Ax ≤ b}, (8.1)

whereA ∈ Rm×n, b ∈ Rm.

More classically, we can write out the equations.

a11x1 + · · ·+ a1nxd ≤ b1,· · ·

am1x1 + · · ·+ amnxd ≤ bm.(8.2)

We now introduce some useful terminology and concepts associated to polyhedra,and illustrate them with a few examples. A supporting hyperplane H of a polyhedronP is a hyperplane such that P ⊆ H−, where H− is a halfspace associated to H . IfH is a supporting hyperplane, then a set of the form F = H ∩ P is called a face ofP . In particular, the polyhedron P is a face. Each of the inequalities 〈ai,x〉 ≤ biin (8.1) defines a supporting hyperplane, and therefore a face. The dimension of aface F , dimF , is the smallest dimension of an affine space containing F . Faces ofdimension dimF = dimP − 1 are called facets, faces of dimension 0 are vertices,and of dimension 1 edges. A vertex can equivalently be characterised as a point x ∈ Pthat can not be written as a convex combination of two other points in P .

Example 8.3. Polyhedra in one dimension are the sets [a, b], [a,∞), (−∞, b], R or∅, where a ≤ b. Each of them is clearly convex.

Example 8.4. The set

P = {x ∈ R2 : x1 + x2 ≤ 1, x1 ≥ 0, x2 ≥ 0}.

is the polyhedron shown in Figure 8.1. We can write the defining inequalities instandard formAx ≤ b by setting

A =

1 1−1 00 −1

, b =

100

.

This polyhedron has one face of dimension 2 (itself), three facets of dimension 1 (thesides, corresponding to the three equations), and three vertices of dimension 0 (thecorners, corresponding any two of the defining equations).

Example 8.5. The polyhedron in Albrecht Dürer’s famous “Melencolia I”, aka the“truncated triangular trapezohedron”, can be described using eight inequalities, seeFigure 8.2.

4

a1

a2

a3

Figure 8.1: A two-dimensional polyhedron and defining equations.

0.7071x1 − 0.4082x2 + 0.3773x3 ≤ 1

−0.7071x1 + 0.4082x2 − 0.3773x3 ≤ 1

0.7071x1 + 0.4082x2 − 0.3773x3 ≤ 1

−0.7071x1 − 0.4082x2 + 0.3773x3 ≤ 1

0.8165x2 + 0.3773x3 ≤ 1

−0.8165x2 − 0.3773x3 ≤ 1

0.6313x3 ≤ 1

−0.6313x3 ≤ 1

Figure 8.2: Dürer’s Melencolia and equations defining the polyhedron

We now move to a different characterization of bounded polyhedra. The mainresult of this lecture is that bounded polytopes can be described completely fromknowing their vertices. A polyhedron P is called bounded, if there exists a ballB(0, r) with r > 0 such that P ⊂ B(0, r). For example, halfspaces are not bounded,but the polytope from Examples 8.4 and 8.5 are.

We first observe the nontrivial fact that a polyhedron has only fin

Definition 8.6. A polytope is the convex hull of finitely many points,

P = conv({x1, . . . , xk}) = {k∑

i=1

λixi, λi ≥ 0,∑i

λi = 1}.

Theorem 8.7. A bounded polyhedron P is the convex hull of its vertices.

Example 8.8. The triangle in Example 8.4 is the convex hull of the points (0, 0)>,(0, 1)>, and (1, 0)>.

8.2. POLYHEDRA 5

Example 8.9. The Dürer polytope is the convex hull of the following 12 vertices:

v1 =

−1.4142−0.8165−0.8835

,v2 =

−1.41420.81650.8835

,v3 =

−0.8536−0.4928−1.5840

,v4 =

−0.85360.49281.5840

,

v5 =

−0.0000−1.63300.8835

,v6 =

0.0000−0.98561.5840

,v7 =

−0.00000.9856−1.5840

,v8 =

0.00001.6330−0.8835

,

v9 =

0.8536−0.4928−1.5840

,v10 =

0.85360.49281.5840

,v11 =

1.4142−0.8165−0.8835

,v12 =

1.41420.81650.8835

.

The converse of Theorem 8.7 is also true.

Theorem 8.10. A polytope is a bounded polyhedron.

The equivalence between polytopes and bounded polyhedra gives a first glimpseinto linear programming duality theory, a topic of central importance in both modelingand algorithm design.