LINEAR PROGRAMMING

Introduction

Development of linear programming was among themost important scientific advances of mid-20th cent.Most common type of applications: allocate limitedresources to competing activities in an optimal way.Very large and wide range of applications from local tovery global ones.Linear programming uses a mathematical model.

Linear because it requires linear functions.Programming as synonymous of planning.

31

Introduction

Besides allocate resources to activities, any problemdescribed by a mathematical model can be solved.Very efficient solution procedure: simplex method.This method will be described next as well as theinterior-point algorithm for larger problems.Besides these algorithms, we will also discusssensitivity analysis.

32

Prototype example

Wyndor Glass Co. produces glass products, includingwindows and glass doors.Plant 1 produces aluminum frames, Plant 2 produceswood frames and Plant 3 produces glass andassembles.Two new products:

Glass door with aluminum framing (Plant 1 and 3)New wood-framed glass window (Plant 2 and 3)

As the products compete for Plant 3, the mostprofitable mix of the two products is needed.

33

Data for Wyndor Glass Co.

34

Wyndor Glass Co. Product-Mix Problem

Doors WindowsProfit Per Batch $3.000 $5.000(Batches of 20)

Hours availableper week

Plant 1 1 0 4Plant 2 0 2 12Plant 3 3 2 18

Hours Used Per Batch Produced

Formulation of the LP problem

x1 = number of batches of product 1 produced per weekx2 = number of batches of product 2 produced per weekZ = total profit per week (in thousands)

35

1 2maximize 3 5Z x x

1

2

1 2

subject to 42 12

3 2 18

xx

x x

1 2and 0, 0.x x

Graphical solution

Trial-and-error: Z = 10, 20, 36Slope of the line is –3/5:

36

1 2and 0, 0.x x

2 13 15 5

x x Z

Feasible region:satisfies allconstraints

Solution (using Excel)

37

Wyndor Glass Co. Product-Mix Problem

Doors Windows Range Name CellsProfit Per Batch $3.000 $5.000 BatchesProduced C12:D12

Hours Hours HoursAvailable G7:G9Used Available HoursUsed E7:E9

Plant 1 1 0 2 <= 4 HoursUsedPerBatchProducedC7:D9Plant 2 0 2 12 <= 12 ProfitPerBatch C4:D4Plant 3 3 2 18 <= 18 TotalProfit G12

Doors Windows Total ProfitBatches Produced 2 6 $36.000

Hours Used Per Batch Produced

Generalizing example

The most common type of LP application involvesallocating resources to activities.

38

Prototype example General problem

Production capacities of plants Resources

3 plants m resources

Production of products Activities

2 products n activities

Production rate of product j, xj Level of activity j, xj

Profit Z Overall measure of performance Z

Linear programming model

Z = value of overall performance measurexj = level of activity j, for j = 1, 2, …, n.cj = parameters of Z related to xj.bj = amount of resource i that is available for allocationof activities, for i = 1, 2, …, m.aij = amount of resource i consumed by each unity ofactivity j.

39

Standard form

40

1 1 2 2maximize n nZ c x c x c x

11 1 12 2 1 1

21 1 22 2 2 2

1 1 2 2

subject to

n n

n n

m m mn n m

a x a x a x ba x a x a x b

a x a x a x b

1 2and 0, 0, , 0.nx x x

Other possible forms

Minimizing rather then maximizing

Some functional constraints are greater-than-or-equal

Some functional constraints in equation form

Deleting the non-negativity constraints for some vars

41

1 1 2 2minimize n nZ c x c x c x

1 1 2 2 , for some values ofi i in n ia x a x a x b i

1 1 2 2 , for some values ofi i in n ia x a x a x b i

unestricted in sign for some values ofjx i

Allocation of resources to activities

42

Resource

Resource Usage per Unit of ActivityAmount ofResourceavailable

Activity1 2 … n

1 a11 a12 … a1n b1

2 a21 a22 … a2n b2

… … … … … …m am1 am2 … amn bm

Contributionto Z per unit

of activity

c1 c2 … cn

Terminology for solutions

Solution: any specification of values for the decisionvariables (x1, x2,…, xn).Feasible solution: solution for which all the constraintsare satisfied.Infeasible solution: solution for which at least oneconstraint is violated.Feasible regionNo feasible solution: no solution satisfies allconstraints.

43

Example: no feasible solution

If in the Wyndor Glass Co. problem the profit shouldbe above $50000 per week:

44

Terminology for solutions

Optimal solution: solution with the most favorablevalue.Most favorable value: largest value if maximizing orsmallest value if minimizing.No optimal solutions: (1) no feasible solutions or,(2) unbounded Z (see next slide).More than one solution: see next slide.

45

Examples of solutions

More than one solution Unbounded cost function

46

Corner-point feasible solution

Lies in a corner. Corners must have at least oneoptimal solution.

47

Assumptions of linear programming

Proportionality: value of objective function Z isproportional to level of activity xj.Violations: 1) Setup costs

48

Violations of proportionality

2) Increase of marginal return due to economy ofscale (longer production runs, quantity discounts,learning-curve effect, etc.)

49

Violations of proportionality

3) Decrease of marginal return due to e.g. marketingcosts: to sell more the advertisement growsexponentially.

50

Assumptions of linear programming

Additivity: every function in a linear programmingmodel is the sum of the individual contributions.Divisibility: decision variables can have any values.

If values are limited to integer values, the problemneeds an integer programming model.

Certainty: parameters of the model cj, aij and bi areassumed to be known constants.

In real applications parameters have always somedegree of uncertainty, and sensitivity analysis must beconducted.

51

Conclusions

Linear programming models can be solved using:Spreadsheet as Excel Solver for relatively simpleproblemsProblems with thousands (or even millions!) of decisionvariables and/or functional constraints must usemodeling languages such as: CPLEX/MDL orLINGO/LINDOMatlab Optimization Toolbox

Linear programming solves many problems.However, when one or more assumptions are violatedseriously, other programming techniques are needed.

52

SIMPLEX METHOD

Simplex method

Algebraic procedure with underlying geometricconcepts.Definitions:

Constraint boundary: line of boundary of theconstraint.Corner-point solutionCorner-point feasible solution (CFP solution)CFP solutions are adjacent if they share n – 1 constraintboundaries.The line segment between two CFP solutions is an edge.

54

Example revisited

Wyndor Glass Co.

55

Optimality test

Optimality test: consider LP with at least one optimalsolution. If a CFP solution has no adjacent CFPsolutions that are better, it must be an optimalsolution.Adjacent CFP solutions for each CFP solution ofWyndor Glass Co. problem:

56

CFP solution Its adjacent CFP solutions(0, 0) (0, 6) and (4, 0)(0, 6) (2, 6) and (0, 0)(2, 6) (4, 3) and (0, 6)(4, 3) (4, 0) and (2, 6)(4, 0) (0, 0) and (4, 3)

Solving the example

Initialization: at CPF solution (0, 0).Optimality test: conclude that (0, 0) is not optimalIteration 1:

1. Choose to move along the edge with faster rate (x2)2. Stop at new constraint boundary (2x2 = 12)3. Determine point of intersection of new set boundaries

(0,6)Optimality test: conclude that (0, 6) is not optimalIteration 2: repeat steps of iteration 1. CFP sol. (2,6)Optimality test: conclude that (2, 6) is optimal. Stop.

57

Steps of simplex algorithm

Sequence of CFP solutions:

58

Setting up the simplex method

Original form Augmented form

59

1 2maximize 3 5Z x x

1

2

1 2

subject to 42 12

3 2 18

xx

x x

1 2and 0, 0.x x

1 2maximize 3 5Z x x

3

4

5

1

2

1 2

subject to(1) 4(2) 2 12(3) 3 2 18

xx

x

xxx

x

and 0, 1,2,3,4,5.jx j

Convert functional inequality constraints to equivalentequality constraints using slack variables.

Setting up the simplex method

Augmented solution: solution with original variables(decision variables) and slack variables.

Example: (3, 2, 1, 8, 5)Basic solution: augmented corner-point solution.

Example (infeasible solution): (4, 6, 0, 0, –6)Basic feasible (BF) solution: augmented corner-pointfeasible solution.

60

Basic definitions

Example problem has 5 variables and 3 non redundantequations, and 2 (5 – 3) degrees of freedom.Two variables have an arbitrary value. Simplex usesthe zero value for these nonbasic variables.The solution for the other variables (basic variables) isa basic solution.Basic solutions have thus a number of properties.

61

Properties of a basic solution

Each variable is basic or nonbasic.Number of basic variables is equal to number offunctional constraints (equations).The nonbasic variables are set to zero.Values of basic variables are solving system ofequations (functional constraints in augmentedform).Set of basic variables are called the basis.If basic variables satisfy the nonnegativity constraintsbasic solution is a BF solution.

62

Final form

63

maximize Z

1 2

1 3

2 4

1 2 5

subject to(0) 3 5 0(1) 4(2) 2 12(3) 3 2 18

Z x xx xx x

x x x

and 0, 1, ,5.jx j

Tabular form (simplex tableau)

64

(a) Algebraic form (b) Tabular Form

Basicvariable Eq.

Coefficient of: RightsideZ x1 x2 x3 x4 x5

(0) Z – 3x1 – 5x2 = 0 Z (0) 1 -3 -5 0 0 0 0

(1) x1 + x3 = 4 x3 (1) 0 1 0 1 0 0 4

(2) 2x2 + x4 = 12 x4 (2) 0 0 2 0 1 0 12

(3) 3x1 + 2x2 + x5 = 18 x5 (3) 0 3 2 0 0 1 18

Simplex method in tabular form

Initialization. Introduce slack variables. Select decisionvariables to be initial nonbasic variables and slackvariables to be initial basic variables.

Initial solution of example: (0, 0, 4, 12, 18)Optimality test. Current BF solution is optimal if everycoefficient in row zero is nonnegative. If yes, stop.

Example: coefficients of x1 and x2 are –3 and –5.

65

Simplex method in tabular form

Iteration.Step 1: Determine entering basic variable, which is thenonbasic variable with the largest negative coefficient.

66

Basicvariable Eq.

Coefficient of: RightsideZ x1 x2 x3 x4 x5

Z (0) 1 -3 -5 0 0 0 0

x3 (1) 0 1 0 1 0 0 4

x4 (2) 0 0 2 0 1 0 12

x5 (3) 0 3 2 0 0 1 18

entering basic variablepivot column

Iteration

Step 2: Determine leaving basic variable, by applyingminimum ratio test:

67

Basicvariable Eq.

Coefficient of:Right side Ratio

Z x1 x2 x3 x4 x5

Z (0) 1 -3 -5 0 0 0 0

x3 (1) 0 1 0 1 0 0 4

x4 (2) 0 0 2 0 1 0 12 12/2 = 6 minimum

x5 (3) 0 3 2 0 0 1 18 18/2 = 9

pivot row

pivot number

Iteration

Step 3: solve for the new BF solution by usingelementary row operations.

68

Iteration Basicvariable Eq.

Coefficient of: RightsideZ x1 x2 x3 x4 x5

Z (0) 1 -3 -5 0 0 0 0

0 x3 (1) 0 1 0 1 0 0 4

x4 (2) 0 0 2 0 1 0 12

x5 (3) 0 3 2 0 0 1 18

Z (0) 1 -3 0 0 2.5 0 30

1 x3 (1) 0 1 0 1 0 0 4

x2 (2) 0 0 1 0 0.5 0 6

x5 (3) 0 3 0 0 -1 1 6

Iteration 2: Steps 1 and 2

69

Iteration Basicvariable Eq.

Coefficient of: Rightside Ratio

Z x1 x2 x3 x4 x5

Z (0) 1 -3 0 0 2.5 0 30

1 x3 (1) 0 1 0 1 0 0 4 4/1 = 4

x2 (2) 0 0 1 0 0.5 0 6

x5 (3) 0 3 0 0 -1 1 6 6/3 = 2

minimum

Complete set of simplex tableau

70

Iteration Basic variable Eq.Coefficient of:

Right sideZ x1 x2 x3 x4 x5

Z (0) 1 -3 -5 0 0 0 0

0 x3 (1) 0 1 0 1 0 0 4

x4 (2) 0 0 2 0 1 0 12

x5 (3) 0 3 2 0 0 1 18

Z (0) 1 -3 0 0 2.5 0 30

1 x3 (1) 0 1 0 1 0 0 4

x2 (2) 0 0 1 0 0.5 0 6

x5 (3) 0 3 0 0 -1 1 6

Z (0) 1 0 0 0 1.5 1 36

2 x3 (1) 0 0 0 1 1/3 -1/3 2

x2 (2) 0 0 1 0 0.5 0 6

x1 (3) 0 1 0 0 -1/3 1/3 2

Other model forms

Equality constraintsIn this case, artificial variables are used. The Big Mmethod is used.

Functional constraints in formSurplus variables are used.

MinimizationMaximize –Z.

Big M method can have two phases: 1) all artificialvariables driven to zero; 2) compute BF solutions.

71

Postoptimality analysis

Reoptimization: when the problem changes slightly,new solution is derived from the final simplex tableau.Shadow price for resource i (denoted yi

*) is themarginal value of i.

i.e. the rate at which Z can be increased by slightlyincreasing the amount of this resource bi.

Example: recall bi values of Wyndor Glass problem.The final tableau gives:y1

* = 0, y2* = 1.5, y3

* = 1.

72

Graphical check

Increasing b2 in 1 unit, the new profit is 37.5

73

Shadow prices

Increasing b2 in 1 unit increases the profit in $1500.Should this be done? Depends on the marginalprofitability of other products using that hour time.Shadow prices are part of the sensitivity analysis.Constraint on resource 1 is not binding the optimalsolution. Resources as this are free goods.Constraints on resources 2 and 3 are bindingconstraints. Resources are scarce goods.

74

Sensitivity analysis

Identify sensitive parameters (those that cannot bechanged without changing optimal solution).

Parameters bi can be analyzed using shadow prices.Parameters ci for Wyndor example (see figure in nextslide).Parameters aij can also be analyzed graphically.

For problems with a large number of variables usuallyonly bi and ci are analyzed, because aij are determinedby the technology being used.

75

Sensitivity analysis of parameters ci

Graphical analysis ofWyndor example:c1 = 3 can have values in[0, 7.5] without changingoptimalc2 = 5 can have any valuegreater than 2 withoutchanging optimal

76

Interior-point approach

Discovery made in 1984 by Narendra Karmarkar.Used in huge linear programming problems, beyondthe scope of the simplex method.Starts by identifying a feasible trial solution, moving abetter one until it reaches the optimal trial solution.Trial solutions are interior points.Interior-point or barrier algorithms (each constraintboundary is treated as a barrier).

77

Example

The Wyndor Glass Co. example reaches the solution in15 iterations for changes smaller than 0.0001

78

Theory of simplex method

Recall the standard form:

79

1 1 2 2maximize n nZ c x c x c x

11 1 12 2 1 1

21 1 22 2 2 2

1 1 2 2

subject to

n n

n n

m m mn n m

a x a x a x ba x a x a x b

a x a x a x b

1 2and 0, 0, , 0.nx x x

Augmented form

80

1 1 1 1maximize n n n n n m n mZ c x c x c x c x

11 1 12 2 1 1 1

21 1 22 2 2 2 2

1 1 2 2

subject to

n n n

n n n

m m mn n n m m

a x a x a x x ba x a x a x x b

a x a x a x x b

and 0, 1,2, , .jx j m

Matrix form (standard form)

81

maximize Z cxsubject to

andAx b x 0

1 2[ , , , ],nc c cc

1

2 ,

n

xx

x

x

1

2 ,

m

bb

b

b

00

,

0

0

11 12 1

21 22 2

1 2

.

n

n

m m mn

a a aa a a

a a a

A

Matrix augmented form

Introduce the column vector of slack variables:

The constraints are (with I (m m) and 0 (1 (n+m)) ):

82

1

2

n

ns

n m

xx

x

x

[ , ] ands s

x xA I b 0

x x

Solving for a BF solution

One iteration of simplex:The n nonbasic variables are set to zero.The m basic variables is a column vector denoted as xB.The basic matrix B is obtained from [A, I], by eliminatingthe columns with coefficients of nonbasic variables.The problem becomes: BxB = b.The solution for basic variables and the value of theobjective function for this basic solution are:

83

1 1andB B B BZx B b c x c B b

Revised simplex method

1. Initialization: (as original) Introduce slack variables xs.Decision variables are initial nonbasic variables.

2. Iteration:Step 1: (as original) Determine entering basic variable(nonbasic variable with the largest negativecoefficient).Step 2: Determine leaving basic variable. As original,but computations are simplified.Step 3: Determine new BF solution: set xB = B–1b.

3. Optimality test: solution is optimal if everycoefficient in Z is 0. Computations are simplified.

84

DUALITY THEORY ANDSENSITIVITY ANALYSIS

Duality theory

Every linear programming problem (primal) has a dualproblem.Most problem parameters are estimates, others (asresource amounts) represent managerial decisions.The choice of parameter values is made based on asensitivity analysis.Interpretation and implementation of sensitivityanalysis are key uses of duality theory.

86

Primal and dual problems

87

1

maximizen

j jj

Z c x

1

subject to

, for 1,2, ,n

ij j ij

a x b i m

and 0 for 1,2, , .jx j n

Primal problem

1

minimizem

i ii

W b y

1

subject to

, for 1, 2, ,m

ij i ji

a y c j n

and 0 for 1,2, , .iy i m

Dual problem

Duality theory

Relations between parameters:The coefficients of the objective functions of the primalare the right-hand sides of the functional constraints inthe dual.The right-hand side of the functional constraints in theprimal are coefficients of the objective functions in thedual.The coefficients of a variable in the functionalconstraints of the primal are the coefficients in thefunctional constraints of the dual.

88

Primal-dual table

89

Primal problem

Coefficient of: Rightsidex1 x2 … xn

Coefficient of:

y1 a11 a12 … a1n b1

Coefficients ofobjective function

(minim

ize)

Dual problem

y2 a21 a22 … a2n b2

… … … … … …

ym an1 an2 … amn bmRightside …

c1 c2 … cn

Coefficients of objective function(maximize)

Example: Wyndor Glass Co.

90

1

2

3 5 ,x

Maximize Zx

1

2

subject to1 0 4

0 2 123 2 18

xx

1

2

0and .

0xx

Primal problem Dual problem

1 2 3

4Minimize 12 ,

18Z y y y

1 2 3

subject to1 0

0 2 3 53 2

y y y

1 2 3and 0 0 0 .y y y

Example: Wyndor Glass Co.

Tableau representation

91

x1 x2

y1 1 0 4

y2 0 2 12

y3 3 2 18

3 5

Primal-dual relationships

Weak duality property: If x is a feasible solution forthe primal problem and y is a feasible solution for thedual problem, then

cx ybStrong duality property: If x* is an optimal solution forthe primal problem and y* is an optimal solution forthe dual problem, then

cx* = y*b

92

Primal-dual relationships

Complementary solutions property: At each iteration,the simplex method identifies a CFP solution x for theprimal problem and a complementary solution for thedual problem, where

cx = ybIf x is not optimal for the primal problem, then y is notfeasible for the dual problem.

93

Primal-dual relationships

Complementary optimal solutions property: At thefinal iteration, the simplex method identifies anoptimal solution x* for the primal problem and acomplementary optimal solution y* for the dualproblem, where

cx* = y*bThe yi* are the shadow prices for the primal problem.

94

Primal-dual relationships

Symmetry property: For any primal problem and itsdual problem, all relationships between them must besymmetric because the dual of this dual problem isthis primal problem.Duality theorem: The following are the only possiblerelationships between the primal and dual problems:

95

Primal-dual relationships

1. If one problem has feasible solutions and a boundedobjective function, then so does the other problem,so both the weak and strong duality properties areapplicable.

2. If one problem has feasible solutions and anunbounded objective function, then the otherproblem has no feasible solution.

3. If one problem has no feasible solutions, then theother problem has either no feasible solutions or anunbounded objective function.

96

Primal-dual relationships

Relationships between complementary basic solutions

97

Primal basicsolution

Complementarydual basicsolution

Both basic solutions

Primal feasible? Dual feasible?

Suboptimal Superoptimal Yes No

Optimal Optimal Yes Yes

Superoptimal Suboptimal No Yes

Neither feasiblenor superoptimal

Neither feasiblenor superoptimal

No No

Primal-dual relationships

98

Applications of duality theory

If the number of functional constraints m is bigger thanthe number of variables n, applying simplex directly tothe dual problem will achieve a substantial reductionin computational effort.Evaluating a proposed solution for the primalproblem. i) if cx = yb, x and y must be optimal evenwithout applying simplex; ii) if cx < yb, yb provides anupper bound on the optimal value of Z.Economic interpretation as before

99

Duality theory in sensitivity analysis

Sensitivity analysis investigate the effect of changingparameters aij, bi, cj in the optimal solution.Can be easier to study these effects in the dualproblem

Optimal solution of the dual problem are the shadowprices of the primal problem.Changes in coefficients of a nonbasic variable. Onlychanges one constraint in the dual problem. If thisconstraint is satisfied the solution is still optimal.Introducing a new variable in the objective function.Only introduces a new constraint in the dual problem. Ifthe dual problem is still feasible, solution is still optimal.

100

Essence of sensitivity analysis

LP assumes that all the parameters of the model (aij, biand cj) are known constants.Actually:

The parameters values used are just estimates based ona prediction of future conditions;The data may represent deliberate overestimates orunderestimates to protect the interests of theestimators.

An “optimal” solution is optimal only with respect tothe specific model being used to represent the realproblem ® sensitivity analysis is crucial!

101

Simplex tableau with parameter changes

What changes in the simplex tableau if changes aremade in the model parameters, namely b® , c® ,A® ?

102

Ab c

Coefficient of:

Right sideEq. Z Original variables Slack variables

New initialtableau

(0) 1 0 0(1,2,…,m) 0 I

Revisedfinal tableau

(0) 1 y*(1,2,…,m) 0 S*

cA b

* *Z y b

* *b S b* *z c y A c

* *A S A

Applying sensitivity analysis

Changes in b:Allowable range: range of values for which the currentoptimal BF solution remains feasible (find range of bisuch that , assuming this is the only changein the model). The shadow price for bi remains valid if bistays within this interval.The 100% rule for simultaneous changes in right-handside: the shadow prices remain valid as long as thechanges are not too large. If the sum of the % changesdoes not exceed 100%, the shadow prices will still bevalid.

103

* *b S b 0

Applying sensitivity analysis

Changes in coefficients of nonbasic variable:Allowable range to stay optimal: range of values overwhich the current optimal solution remains optimal(find , assuming this is the only change in themodel).The 100% rule for simultaneous changes in objectivefunction coefficients: if the sum of the % changes doesnot exceed 100%, the original optimal solution will stillbe valid.

Introduction of a new variable:Same as above.

104

*j jc y A

Other simplex algorithms

Dual simplex methodModification useful for sensitivity analysis

Parametric linear programmingExtension for systematic sensitivity analysis

Upper bound techniqueA simplex version for dealing with decision variableshaving upper bounds: xj uj, where uj is the maximumfeasible value of xj.

105