Basic Feasible Solutions

John E. Mitchell

Department of Mathematical Sciences

RPI, Troy, NY 12180 USA

January 2016

Mitchell Basic Feasible Solutions 1 / 22

Standard form

We work with the standard form linear program

minx2IRn cT xsubject to Ax = b (P)

x � 0

where c 2 IRn, b 2 IRm, and A 2 IRm⇥n. Let K denote the feasible region

of (P),K := {x 2 IRn : Ax = b, x � 0}.

The aim of the simplex algorithm is to move from one extreme point of

K to a neighboring extreme point, while improving the objective

function value.

In order to implement this efficiently, a nice algebraic representation of

“extreme points” is useful. Such a representation takes the form of a

basic feasible solution.

Mitchell Basic Feasible Solutions 2 / 22

i :O→ o o

Support of a feasible solution

Definition

Let x̄ 2 K . The support of x̄ is the index set of the columns used by x̄,

Jx̄ := {j : x̄j 6= 0}.

Definition

Let x̄ 2 K with support Jx̄ . The point x̄ is a basic feasible solution(BFS) to (P) if and only if the set of columns of A used by x̄ is alinearly independent set.

Mitchell Basic Feasible Solutions 3 / 22

Support of a feasible solution

Definition

Let x̄ 2 K . The support of x̄ is the index set of the columns used by x̄,

Jx̄ := {j : x̄j 6= 0}.

Definition

Let x̄ 2 K with support Jx̄ . The point x̄ is a basic feasible solution(BFS) to (P) if and only if the set of columns of A used by x̄ is alinearly independent set.

Mitchell Basic Feasible Solutions 3 / 22

× , - 3 x , +2×4=224-x. + 7x , -x4=z A:[{I ,}})I-10,511,0] f - ={2,3}Columns usedby5 : f.',},f } ]linearly independent, s o I i s a BES

X , t 2 × 2+ Tx ,-Xu,= 6

3×1-x, t 2×3+5×4=4- X ,t X , +2×4=0

I : ( l , l ,1,0) J j : {1,2 ,3}

Column, a n d by5 :

Att i la ,( inearly dependent:

(1)t.fi/f:fE=1.So-xnotaBFS

Geometry and linear algebra

The definition of a basic feasible solution is based on linear algebra.

The definition of an extreme point comes from geometry.

For a polyhedron, these two definitions coincide.

Theorem

Let x̄ 2 K . The point x̄ is an extreme point of K if and only if it is a BFS.

Mitchell Basic Feasible Solutions 4 / 22

K:{xc.IR?Ax--b,xx}

K i {xc.IR?Ax-.b.x20}A x : b : give, a plane. Eg,the s c r e e n .

X 7 0 : give, halfspace,+ 3 = 0

€i÷÷o÷÷.+4=0 /

x

X i a o •X 5=0

Ty : { I ,3,4,6,7}

Eg: x , t 3 x - = 6, x 2 0

K

4¥ . .

Eg: x . +2×13×3=6, x j3O t i

i t

¥¥÷÷:A "(x ,

Proof, part 1

We prove both directions by contradiction.

First, assume x̄ is not a BFS.

Then the set of columns used by x̄ is linearly dependent, so there

exists y 2 IRn, y 6= 0, with Ay = 0 and yj = 0 if x̄j = 0.

We can add carefully chosen multiples of y to x̄ and still remain

feasible, because this only modifies the positive components of x̄ .

Let ✓ = minj{x̄j : x̄j > 0} and ↵ = maxj{|yj |}.

The points x̄ ±�✓↵

�y are both in K . Then

x̄ = 0.5

✓x̄ +

✓✓

↵

◆y◆

+ 0.5

✓x̄ �

✓✓

↵

◆y◆

so x̄ is not an extreme point of K .

Mitchell Basic Feasible Solutions 5 / 22

Eg: AYE,}!'d * (G)T f - y

F - {3 1 1 03 × ¥ ¥ ,y - [2 i - i 051*17=7Age(E) y i n nullspaugff:

Aleta,)=§o¥.se/II.o?o..o,

Proof, part 2

Conversely, assume x̄ is not an extreme point of K .

Then x̄ = �w + (1� �)z for some w , z 2 K with w 6= z and with

0 < � < 1.

If x̄j = 0 then we must have wj = zj = 0 since

0 = x̄j = �wj + (1� �)zj ,

so if wj > 0 then zj < 0 so z 62 K and if zj > 0 then wj < 0 so w 62 K .

Let y = w � z 6= 0. Then Jy ✓ Jx̄ and Ay = A(w � z) = b � b = 0.

So the columns of A used by x̄ are linearly dependent, so x̄ is not a

BFS. ⇤

Mitchell Basic Feasible Solutions 6 / 22

Jw E JgJ z E F x

Plan. i t Axels.

ftp.?i::iiii:.i.o:i.a.I z - I , W - E i n

•w..- •" -oz nu l lspace ofi t .

I = X - + (I-1)z , O - Y - l .

Number of extreme points

The number of possible choices for the support is finite, so the number

of basic feasible solutions of (P) is finite, although it may be

exponentially large in m and n.

Consequently, we have the following theorem.

Theorem

The number of extreme points of K is finite.

Mitchell Basic Feasible Solutions 7 / 22

Existence of a BFS

In our standard form (P), the feasible region is contained in the

nonnegative orthant. One consequence of this is that if (P) is feasible

then it must have a BFS.

Theorem

If (P) has a feasible solution then it has a BFS.

Mitchell Basic Feasible Solutions 8 / 22

¥

A polyhedron with n o extreme points:

K : { x , + xutxs-6,4=13x , < 0 .

÷E÷÷÷tXg<O

I + Od/ d÷:÷÷€E

E i

Proof of existence

We use a constructive proof. We start from a feasible point x̄ and show

how to modify it iteratively in order to obtain a BFS.

Let x̄ 2 K . If x̄ is a BFS then we are done. Otherwise, the support Jx̄of x̄ gives a linearly dependent set of columns of A, so there exists a

direction d 2 IRn with

d 6= 0, Ad = 0, and dj = 0 if x̄j = 0.

If d � 0 then we redefine d �d , so we can assume without loss of

generality that d has at least one negative component.

We define the minimum ratio

↵ = minj

⇢x̄j

�dj: dj < 0

�> 0 so x̄j + ↵dj � 0 8 j .

In addition, A(x̄ + ↵d) = b, so x̄ + ↵d 2 K . Further, for at least one

component j 2 Jx̄ , we have (x̄ + ↵d)j = 0, namely the component that

achieved the minimum ratio.

Mitchell Basic Feasible Solutions 9 / 22

Proof of existence, part 2

So we have a new feasible point x̄ + ↵d whose support is a strict

subset of the support of the original point x̄ .

If this new point is a BFS then we are done.

Otherwise, we apply the same procedure to the new point, and keep

iterating. ⇤

Mitchell Basic Feasible Solutions 10 / 22

Existence of optimal BFS

This proof technique can be extended to prove the following result:

Theorem

If (P) has an optimal solution then it has an optimal BFS.

This result is the justification for the simplex algorithm: it is enough to

look only at extreme points.

There may be additional optimal solutions, but there is always an

optimal solution that is an extreme point (provided an optimal solution

exists).

Mitchell Basic Feasible Solutions 11 / 22

Need I d = D .

Set of optimal solutions

The following theorem characterizes the set of optimal solutions.

Theorem

The set of optimal solutions to (P) is a face of K .

Proof.

First, we consider the case where (P) does not have an optimal

solution. In this case, the set of optimal solutions is the empty set,

which by definition is a face of K .

Now assume (P) has an optimal solution, with optimal value z⇤. Define

the hyperplane

H = {x 2 IRn : cT x = z⇤}.

The set of optimal solutions is then K \ H, which by definition is a face

of K .

Mitchell Basic Feasible Solutions 12 / 22

(' " ' ' ' i .

H e { x : cTx=z}z = optimal value.

Set of optimal solutions

The following theorem characterizes the set of optimal solutions.

Theorem

The set of optimal solutions to (P) is a face of K .

Proof.

First, we consider the case where (P) does not have an optimal

solution. In this case, the set of optimal solutions is the empty set,

which by definition is a face of K .

Now assume (P) has an optimal solution, with optimal value z⇤. Define

the hyperplane

H = {x 2 IRn : cT x = z⇤}.

The set of optimal solutions is then K \ H, which by definition is a face

of K .

Mitchell Basic Feasible Solutions 12 / 22

Constructing basic feasible solutions

So far, we’ve looked at a point x̄ and worked out whether it is a BFS.

Now we look to construct basic feasible solutions. In what follows, weassume that the rows of A are linearly independent, so it has full

row rank, so rank(A) = m.

The columns used by a BFS must be linearly independent, so we

consider the largest possible sets of columns that could be linearly

independent, namely those corresponding to bases of IRm.

Every BFS corresponds to (at least) one such basis.

Mitchell Basic Feasible Solutions 13 / 22

Constructing basic feasible solutions, 2

Let B be an invertible m ⇥m matrix whose columns are columns of A.

Without loss of generality, we can reorder the columns of A so that the

columns of B are written first.

We denote the remaining columns by N, so we write

A = [ B|{z}m columns

, N|{z}n � m columns

].

We similarly order the entries in c and x , so

c =

cBcN

�and x =

xBxN

�

with cB, xB 2 IRm and cN , xN 2 IRn�m.

Mitchell Basic Feasible Solutions 14 / 22

The (B,N) formulation

We can then write the problem (P) as

min cTB xB + cT

NxN

subject to BxB + NxN = b

xB, xN � 0.

The matrix B is invertible, so we can premultiply the equality

constraints by B�1 without changing the problem. This gives the

equivalent problem

min cTB xB + cT

NxN

subject to xB + B�1NxN = B�1b

xB, xN � 0.

Mitchell Basic Feasible Solutions 15 / 22

The (B,N) formulation and a BFS

It is clear from the equality constraints that we must have

xB = B�1b � B�1NxN ,

whatever the value of xN .

Taking xN = 0 and xB = B�1b is the basic solution corresponding to

the basis B. It is a BFS if B�1b � 0.

We can exploit the equality to rewrite our LP one more time:

min cTB B�1b +

�cT

N � cTB B�1N

�xN

subject to xB + B�1NxN = B�1b

xB, xN � 0.

Mitchell Basic Feasible Solutions 16 / 22

CBTxis= CBTB -'b-c,jT5'N×,

Degeneracy

Different choices of B may lead to the same BFS. This can only

happen if the BFS is degenerate.

Definition

Assume A has full row rank. Let x̄ be a BFS of (P). If x̄ has fewer thanm positive components then it is degenerate. Otherwise it isnondegenerate.

Mitchell Basic Feasible Solutions 17 / 22

Degeneracy

Example

Let

A =

�1 �3 3 2

2 1 2 �1

�and b =

18

12

�.

The point x̄ = (0, 0, 6, 0)T is a degenerate BFS. It is equal to B�1bwhen B consists of the third column of A and any one other column.

There are also nondegenerate BFS’s for this problem, for example

x = (14, 0, 0, 16)T .

Mitchell Basic Feasible Solutions 18 / 22

A:[I ? I } ) . Get same-x v i a

Pick column, 1 23 for B :¥124831

B --f i % B -'=÷fE i fB -'be.fi?1fYI=-tsfE.sf--181

I f x , = {¥3:{8) then↳= B -'biff;1=18).

Sr * (&)

Degeneracy and dimension

Nondegeneracy implies something about the dimension of K :

Theorem

If x̄ is a nondegenerate BFS for (P) then the dimension of K is n �m.

Mitchell Basic Feasible Solutions 19 / 22

Proof, part 1

First show dim(K ) n �m:

Since x̄ is a nondegenerate BFS, it has m positive components, so

there are m linearly independent columns of A, and rank(A) = m.

Thus,

dim(K ) dim{x 2 IRn : Ax = b} = n � rank(A) = n �m.

Mitchell Basic Feasible Solutions 20 / 22

X , +2×2+3×3=6, x 2 0 .

÷:*.i÷÷÷÷÷i÷÷ linearly independent

x , + 24+3×3=6 µ

"×;>o .= " §,

i \(1,0.FI. .

.7

, \

→÷ " .i t - x ,i

- I .o )

x .i f I

increase × 3 'adjust × , >X i :d=(0,-2,1)

Proof, part 2

Now show dim(K ) � n �m:

Rearrange the columns of A so that A = [B,N], with the columns Bcorresponding to the basic variables in x̄ .

Let aj be the j th column of N. Since B is invertible, there exists a

vector dj := �B�1aj 2 IRm, so Bdj + aj = 0 2 IRm.

Let ej 2 IRn�m be the j th unit vector. Define the direction d̂ j 2 IRn by

d̂ j =

dj

ej

� in IRm

in IRn�m

By construction, Ad̂ j = 0. Thus, the point x̄ + ✓d̂ j 2 K for small positive

step lengths ✓, since x̄ is nondegenerate.

By using the directions for all the columns of N, we get n �m linearly

independent directions in K originating from x̄ , so K has dimension at

least n �m. ⇤Mitchell Basic Feasible Solutions 21 / 22

i÷in÷a÷.

X ,T X - t x , = ?

- X ,-12×2 -1×4=4 ×-A

x-530 i t ;

iii." ¥ : " .f¥"a:&:T

/ X ,

I - (2,0,0,61Tn o ndegenerate.

Rays

Definition

A vector d 2 IRn is a homogeneous solution corresponding to (P) ifAd = 0 and d � 0.

If x̄ is feasible for (P) and d is a homogeneous solution then

x̄ + ✓d 2 K for all ✓ � 0.

In the Example, the vector d = (1, 3, 0, 5)T is a homogeneous solution.

Definition

Let d 2 IRn with d 6= 0. The ray generated by d is the set

{x 2 IRn : x = �d , � � 0}.

If x̄ 2 IRn then the half-line through x̄ parallel to to the ray generatedby d is

{x 2 IRn : x = x̄ + �d , � � 0}.

Mitchell Basic Feasible Solutions 22 / 22

A - I I ? ! ? )D = [ i z o s ,t}Ad:[§.

Rays

Definition

A vector d 2 IRn is a homogeneous solution corresponding to (P) ifAd = 0 and d � 0.

If x̄ is feasible for (P) and d is a homogeneous solution then

x̄ + ✓d 2 K for all ✓ � 0.

In the Example, the vector d = (1, 3, 0, 5)T is a homogeneous solution.

Definition

Let d 2 IRn with d 6= 0. The ray generated by d is the set

{x 2 IRn : x = �d , � � 0}.

If x̄ 2 IRn then the half-line through x̄ parallel to to the ray generatedby d is

{x 2 IRn : x = x̄ + �d , � � 0}.

Mitchell Basic Feasible Solutions 22 / 22