EEM2046_-_1314_T2_-_OR_Lecture_Notes Operation Research notes

EEM2046 Engineering Mathematics IV Operations Research

T2, 2013/14 of 59

Operations Research

1.0 Introduction to Operations Research - Operations research (commonly refer to as OR) uses scientific methods to

investigate the problems that concern how to conduct and coordinate the operations within an organization.

- Various definitions have been given to describe operations research, among them are:

Definition Operations research is concerned with scientifically deciding how best to design and operate man-machine system usually under conditions requiring the allocation of scarce resources. – OR Society of America –

Definition Operations Research is a scientific approach to problem solving for executive management. – HM Wagner –

- The features of the definitions are:-

(a) OR is a decision making process by some scientific problem-solving techniques

(b) It deals with optimizing the results (c) The resources are limited (d) Concerned with the practical management of the organization

- OR has given an impressive impact on improving the efficiency of numerous organizations around the world. Some important applications are in the following areas: (a) Manufacturing/Production

Production planning and control, inventory management. (b) Facilities Planning

Design of logistic system, factory/building location and size decisions, transportation loading and unloading, planning warehouse locations, inventory management.

(c) Accounting Credit policy decisions, cash flow and fund flow planning. (d) Construction Management Allocation of resources to different projects in hand, workforce / labour

planning, project management (scheduling, monitoring and control). (e) Financial management Investment decision, portfolio management.


T2, 2013/14 of 59

(f) Marketing management Product-mix decisions, advertisement/promotion budget decisions, launching

new product decision. (e) Purchasing decision

Inventory management (optimal level of purchase), optimal re-ordering. (f) Personnel management

Recruitment and selection of employees, designing training and development programs, human resources planning.

1.1 Methodology of OR - In general, the major phases of a typical OR study involves the following:

(a) Define the problem of interest and gather relevant data - The process of problem definition is a crucial one because it greatly

affects how relevant the conclusions of the study will be. - Data are needed both to gain an accurate understanding of the problem

and to provide the needed input for the mathematical model being formulated in the next phase of study.

- If the data is not available when the study begins, a management information system is required to collect the necessary data on an ongoing basis and in the needed form.

- If there is too much data, locating the particularly relevant data and identifying the interesting patterns in these data can become a difficult task. One of the OR tools is a technique called data mining that addresses this problem. Data mining methods search large databases for interesting patterns that may lead to useful decisions.

(b) Formulate a mathematical model to represent the problem

- Mathematical model describes a problem in terms of variables and their relationship. It helps to reveal important cause-and-effect relationship.

- Approximations and simplifying assumptions generally are required if the model is to be tractable. Care must be taken to ensure that the model remains a valid representation of the problem.

(c) Develop a computer-based procedure for deriving solutions to the problem

from the model - A common practice in OR is the search for an optimal solution. These

solutions are optimal only with respect to the model being used, and are valid only under these assumptions. An optimal solution for the original model may be far from ideal for the real problem, so additional analysis is needed. Therefore, postoptimality analysis (analysis done after finding an optimal solution) is a very important part of most OR studies. It involves addressing some questions about what would happen to the optimal solution if different assumptions are made about future conditions.


T2, 2013/14 of 59

(d) Test the model and refine it as needed. - The first version of a large mathematical model inevitably contains

many flaws. Some relevant factors might not been incorporated into the model. Before we use the model, it must be thoroughly tested to try to identify and correct as many flaws as possible. This process of testing and improving a model to increase its validity is commonly referred to as model validation

- A systematic approach to test the model is to use a retrospective test. This test involves using historical data to reconstruct the past and then determining how well the model and the resulting solution would have performed if they had been used.

(e) Implementing the solution

- Throughout the entire period during which the system is being used, it is important to continue to obtain feedback on how well the system is working and whether the assumptions of the model continue to be satisfied.

- The application involves many behavioral aspects. Any gap between the perception of the management and the approach of OR team must be removed.

1.2 Tools of OR - OR uses many tools/techniques for problem solving. Some of the tools of OR are

as discussed below:

(a) Linear Porgramming (LP) In LP, the objective function and constraints must be linear. It is a mathematical technique used to allocate limited resources among various activities in an optimal manner.

(b) Queuing Theory Queuing theory aims to minimize the overall cost due to servicing and waiting. It concerns with the phenomenon of waiting in real life situations, such as the queue in waiting to buy goods in a shop, patients waiting outside an OPD, vehicles waiting to be serviced in a garage.

(c) Network Analysis Technique Network is used to present the activities necessary to complete a project, such as planning, scheduling, monitoring and control of large and complex projects. The questions interested are in knowing the total project completion time, probability that a project can be completed by a particular time, and the least cost method of reducing the total project completion time.


T2, 2013/14 of 59

(d) Replacement Theory Model It deals with the optimal time to replace the equipment or machinery, either because there is deterioration in their efficiency or because new and better equipment is available and the old one has become obsolete.

(e) Inventory control How much of raw material should be purchased, when should it be purchased and how much should be kept in stock. These are the common questions to be answered in inventory control. Overstocking will result in locked capital not available for other purposes, whereas understocking will mean stock-out and idle manpower and machine resulting in reduced output. It is desirable to have just the right amount of inventory at the right time.

(f) Integer programming Integer programming deals with certain situations in which the variable assumes non-negative integer (complete or whole number) values only

(g) Transportation problem The transportation problem is basically a LP model. A typical transportation problem is to find the minimum transportation cost for transporting the single commodity from a number of sources to a number of destinations.

(i) Decision theory and games theory Games theory is concerned with decision-making in a conflict situation where two or more intelligent opponents try to optimize their own decision. The games theory helps the decision maker to analyze the course of action available to his opponent. In decision theory, we use decision tree which can be graphically represented to solve the decision–making problems.

(j) Assignment problems Assignment problems are problems of assigning a number of tasks to a number of persons so that the cost is least.

(k) Markov Analysis Markov analysis is used to predict future conditions based on the probability theory. It assumes that the occurrence of a future state depends upon the immediately preceding state and only on it.

(l) Simulation Techniques When real life situations cannot be represented mathematically and cannot be solved mathematically, certain assumptions are made and the models which simulate the natural system are developed. In simulation model, the behavior of the system is initiated over a period of time.


T2, 2013/14 of 59

2.0 Formulation of Mathematical Model - The standard statement of a mathematical model in OR has the form

Minimize/Maximize ( )xf (objective function) subject to: Ω∈x

with (a) ( ) 0xgx ≥=≤=Ω or: , , where ( )xg are given functions (constrained

optimization problem), or (b) nR=Ω (unconstrained optimization problem). The vector ( )T

nx,,x,x K21=x consists of the independent variables nx,,x,x K21 which are called as the decision variables.

- Construction of mathematical model:

Step 1: Identify the decision variables. Step 2: Identify the objective function. Step 3: Identify the constraints.

Variable-type constraints - nonnegativity constraints: xi ≥ 0

(when xi are nonnegative real numbers) . - unrestricted in sign constraint: xi = .urs (when xi takes all real values).

Main constraints The constraints that specify the restrictions and interactions, other than variable type constraints.


T2, 2013/14 of 59

Example (Optimization in Operations Research) Suppose that we wish to enclose a rectangular equipment yard by at most 80 meters of fencing. Formulate an optimization model to find the design of maximum area. Solution:

Set l = length of the equipment yard (in meters) w = width of the equipment yard (in meters)

maximize A = lw (objective function) subject to: 2l + 2w ≤ 80 (main constraints)

l, w ≥ 0 (variable-type constraints)

Example (Principles of Operations Research for Management) A refuse incinerator is being planned to service four metropolitan areas. The relative locations of the metropolitan areas are shown as follows, where coordinates are expressed in kilometers. Environmental management officials wish to determine the optimal location of the incinerator location that minimizes the sum of the squares of the distances from the incinerator to each metropolitan area.

Solution:

Set x = the x-coordinate for the location of refuse incinerator y = the y-coordinate for the location of refuse incinerator.

( ) ( ) ( ) ( )

( )( ) ( )( ) ( )( ) ( )2222

2222

20302020

20303010 Minimize

−+−−+−−+−−+

−+−+−+−=

yxyx

yxyxs

subject to: x, y .urs


T2, 2013/14 of 59

Example (Optimization in Operations Research) The state Department of Labor is considering the establishment of area job training centers at up to 4 sites. The following table shows the land cost (in thousands of dollars) of the 4 sites and indicates with an X the sites that could provide adequate service to each of the five regions of the state.

Site Region 1 2 3 4

Northwest X X - - Southwest X X X -

Capital X - X X Northeast X X - - Southeast X - - X

Cost 200 40 55 75 The Department seeks a minimum total cost collection of sites that together could service all five regions. Formulate an optimization model to decide what sites to build using decision variables 1=iy if site i is selected and 0=iy otherwise. Solution: Set 1=iy if site i is selected and 0=iy otherwise, i = 1,2,3,4. Minimize z = 200y1 + 40y2 + 55y3 + 75y4 subject to: y1 + y2 ≥ 1 y1 + y2 + y3 ≥ 1 y1 + y3 + y4 ≥ 1 y1 + y4 ≥ 1 y1, y2, y3, y4 = 0, 1. Example The Chemical Problem (Lecture Notes Series: Engineering Mathematics Volume 2) A company produces two high quality chemical substances: I and II. Two raw materials, A and B, are used to manufacture the chemical substances. The maximum availability of raw material A is 6 tons a day, and that of raw material B is 8 tons a day. Each product has different input requirements as shown below. Tons of raw material per ton of chemical

substance Maximum

availability I II (tons) Raw material A 1 2 6

Raw material B 2 1 8 The daily demand for chemical substance II won’t exceed that of chemical

substance I by more than 1 ton. The maximum demand for chemical substance II is limited to 2 tons daily. The price per ton is RM3000 for chemical substance I, and RM2000 for chemical substance II. How much chemical substance I and II should the


T2, 2013/14 of 59

company produce daily to maximize the gross income? Model the above decision problem. Solution:

Set x1 = tons of chemical substance I to be produced, and

x2 = tons of chemical substance II to be produced Example (Schaum’s series) Universal Mines Inc. operates three mines in West Virginia. The ore from each mine is separated into two grades before it is shipped; the daily production capacities of the mines, as well as their daily operating costs, are as follows:

High-Grade Ore, Tons/day

Low-Grade Ore, Tons/day

Operating Cost $1000/day

Mine I Mine II Mine III

4 6 1

4 4 6

20 22 18

Universal has committed itself to deliver 54 tons of high-grade ore and 65 tons of low-grade ore by the end of the week. It also has labor contracts that guarantee employees in each mine a full day’s pay for each day or fraction of a day the mine is open. Determine the number of days each mine should be operated during the upcoming week if Universal Mines is to fulfill its commitment at minimum cost. Solution: Let x1, x2 and x3, respectively, denote the numbers of days that mines I, II and III will be operated during the upcoming weeks. The linear programming problem can then be written as follows. Minimize z = 20x1 + 22x2 + 18x3

subject to: 4x1 + 6x2 + x3 ≥ 54 4x1 + 4x2 + 6x3 ≥ 65

x1 ≤ 7 x2 ≤ 7

x3 ≤ 7 x1, x2 , x3 ≥ 0


T2, 2013/14 of 59

Example (Schaum’s series) A 24-hour supermarket has the following minimal requirements for cashiers:

Period 1 2 3 4 5 6 Time of the day (24-h clock)

3-7 7-11 11-15 15-19 19-23 23-3

Minimum No. 7 20 14 20 10 5 Period 1 follows immediately after period 6. A cashier works eight consecutive hours, starting at the beginning of one of the six periods. Determine a daily employee worksheet that satisfies the requirements with the least number of personnel. Solution: Let xi (i=1,2,..,6) denote the number of cashiers beginning work at the start of period i.

Minimize z = x1 + x2 + x3 + x4 + x5+ x6 subject to: x1 + x6 ≥ 7 x1 + x2 ≥ 20

x2 + x3 ≥ 14 x3 + x4 ≥ 20 x4 + x5 ≥ 10 x5 + x6 ≥ 5

with all variables nonnegative and integral. Example (Introduction to Operations Research) The DISTRIBUTION UNLIMITED CO. will be producing the same new product at two different factories, and then the product must be shipped to two warehouses, where either factory can supply either warehouse. The distribution network available for shipping this product is shown below, where F1 and F2 are the two factories, W1 and W2 are the two warehouses, and DC is a distribution center. The amounts to be shipped from F1 and F2 are shown to their left, and the amounts to be received at W1 and W2 are shown to their right. Each arrow represents a feasible shipping lane. Thus F1 can ship directly to W1 and has three possible routes(F1 → DC → W2, F1→ F2 → DC → W2, and F1 → W1 → W2) for shipping to W2. Factory F2 has just one route to W2 (F2 → DC → W2) and one to W1 (F2 → DC → W2 → W1). The cost per unit shipped through each shipping lane is shown next to the arrow. Also shown next to F1 → F2 and DC → W2 are the maximum amounts that can be shipped through these lanes. The other lanes have sufficient shipping capacity to handle everything these factories can send. The decision to be made concerns how much to ship through each shipping lane. The objective is to minimize the total shipping cost. Formulate the above problem as a linear programming problem.


T2, 2013/14 of 59

Solution:

- The remainder of the discussion focuses primarily on solving mathematical models. Anyway, this constitutes only a portion of the overall process involved in conducting a typical OR study. The other phases described above also are very important to the success of the study.

- Our discussion would focus on: - Linear Programming - Dynamic Programming - Combinatorial Optimization

W1 F1

DC

F2 W2

$900/unit 30 units needed

60 units needed

$300/unit

$200/unit

50 units produced

40 units produced

80 units max.

$400/unit

$100/unit

$300/unit

$200/unit 10 units max.


T2, 2013/14 of 59

3.0 Linear Programming (LP) - General LP model Minimize/Maximize z = f(x1, x2, … , xn) *objective function subject to: g1 (x1, x2, … , xn) ≤ , ≥ or = b1 * constraints g2 (x1, x2, … , xn) ≤ , ≥ or = b2 M gm (x1, x2, … , xn) ≤ , ≥ or = bm

with f(x1, x2, … , xn) and each gi (x1, x2, … , xn), i = 1,2,…,m must be linear functions.

- Majority of optimization problems actually encountered in engineering and management practice cannot be modeled straightforwardly as linear programs. They involve discrete, nonlinear and multiobjective elements that have to be confronted.

3.1 Solving LP using Graphical Method Definition (i) A feasible point is any point that satisfies all constraints. The feasible region/

feasible set is the set of all feasible points. (ii) The optimal solution is a point in the feasible region that gives the largest

(smallest) objective function value in the maximization (minimization) problem. (iii) The optimal value is the objective function value of any optimal solution. The Graphical Method Step 1: Find the feasible region Step 2: Graph the objective function Step 3: Identify the optimal solution & the optimal value


T2, 2013/14 of 59

Example: Solve the Chemical problem by using graphical method. (LNS*) Solution:

*Lecture Notes Series: Engineering Mathematics Volume 2, Prentice Hall.

3.1.1 Special Cases Ø Unique solution vs. alternative optimal solutions Unique optimal solution: when the optimal value contour intersects the feasible set at exactly one point. Otherwise, the model has alternative optimal solutions. Example:

x1

x2

Alternative optimal

x1

x2

x1

x2

Unique optimal solution


T2, 2013/14 of 59

Ø Infeasible model - When the constraints cannot be satisfied simultaneously.

Ø Unbounded Solutions - If the objective function can be improved without limit in an unbounded feasible

region. 3.2 Solving LP using Simplex Method - The problem must be put in the standard linear programming form. - Standard LP form has the following properties:

x1

x2

x1

(i) All the main constraints are equations with the right-hand sides must be positive.

(ii) All variables are nonnegative.


T2, 2013/14 of 59

(i) All the main constraints are equations with the right-hand sides must be positive.

Constraint of the type ≤ - A constraint of the type ≤ can be converted to an equation by adding a

slack variable to the left-hand sides of the constraint. - If the right-hand sides of the constraint represent the limit on the

availability of a resource, then the slack variable gives the amount of the unused resource in the constraint.

Example: Constraint x1 + 4x2 ≤ 4

can be converted to equation x1 + 4x2 + s1 = 4, s1 ≥ 0.

Constraint of the type ≥ - A constraint of the type ≥ can be converted to an equation by subtracting a

surplus variable to the left-hand sides of the constraint. - If the constraint of the type ≥ set the minimum specification requirements,

then surplus represents the excess of the left-hand sides over the minimum requirement.

Example: Constraint 3x1 − 2x2 ≥ 6

can be converted to the equation 3x1 − 2x2 − s2 = 6, s2 ≥ 0.

(ii) All variables are nonnegative. - An unrestricted variable (.urs) yi can be expressed in terms of two

nonnegative variables by using the substitution .0,, ≥−= −+−+

iiiii yyyyy

- A purely negative variable yi can be expressed in standard form with the substitution

.0, ≥−= −−iii yyy

The above substitutions affect all constraints and the objective function.


T2, 2013/14 of 59

Example: Construct the standard LP form for the following LP. Maximize z = x1 + 2x2

subject to: x1 + x2 ≥ 6 3x1 + 2x2 ≤ −8 −x1 + 4x2 = 1

x1, x2 ≥ 0 Solution: Example: Model the following problem into the LP standard form

Minimize z = 2x1 + 3x2 subject to: x1 + 2x2 = 10

−2x1 + 3x2 ≤ 5 −7x1 + 4x2 ≤ −6

x1 .urs, x2 ≥ 0 Solution: Set −+ −= 111 xxx . By adding the slack and surplus variables, we have:

Minimize z = 211 322 xxx +− −+ subject to:

.s,s,x,x,xsxxx

sxxxxxx

0

6 477

5 322

10 2

32211

3211

2211

211

≥

=−−−

=+++−

=+−

−+

−+

−+

−+


T2, 2013/14 of 59

The Background Theory of Simplex Method Consider a LP in its standard form with n the no. of the variables, m the no. of main constraints (m < n) and Px = b the set of main constraints.

Minimize/Maximize z = cTx subject to: Px = b x ≥ 0,

where c, x nR∈ , b mR∈ , P nm×∈ R , m < n and rank P = m. Assume that b ≥ 0 and vector x inclusive of the decision variables and any newly introduced variables.

Minimize/Maximize z = cTx subject to: Px = b x ≥ 0.

Set x = ( ) TTT

B ,0x (rearrange x so that the variables that

we are interested appeared first.)

m variables that are set as TBx ,

are known as basic variables.

(n – m) variables that are set as T0 , are known as non basic variables.

Solve B xB = b, P = (B, D), B and D correspond to the basic variables and the nonbasic variables, respectively.

Basic solution: xB = B−1b with the nonbasic variables equal zeros.

If the basic solution ≥ 0, it is called basic feasible solution.


T2, 2013/14 of 59

Example: Consider the following LP problem: Minimize z

subject to : x1 + 2x2 ≤ 10 −x1 + 5x2/4 ≤ 5 x1, x2 ≥ 0 The standard LP form is:

Minimize z subject to : x1 + 2x2 + s1 = 10 −x1 + 5x2/4 + s2 = 5 x1, x2, s1, s2 ≥ 0 Thus, we have m = 2, n = 4 with Px = b is

x1 + 2x2 + s1 = 10 −x1 + 5x2/4 + s2 = 5

The graph of the feasible region is

Case 1: basic feasible solution Nonbasic variables: s1, s2 = 0.

Equations: x1 + 2x2 = 10 −x1 + 5x2/4 = 5 Solution: x1 = 10/13, x2 = 4 8/13 (unique solution). Status: basic feasible solutions as all variables > 0. Vertex: Point A (extreme point of the feasible region). Case 2: basic infeasible solution Nonbasic variables: x2, s2 = 0.

Equations: x1 + s1 = 10 −x1 = 5 Solution: x1 = −5, s1 = 15 (unique solution). Status: basic infeasible solutions as the variable x1 < 0. Vertex: Point B (infeasible region).

x1

x2

A

B


T2, 2013/14 of 59

Case 3: Nonbasic solution Consider the solution x1 = 5, x2 = 0, s1 = 5 and s2 = 10. Although it satisfies Px = b, but it is a nonbasic solution as we only have one variable set to 0. Note that it does not refer to any extreme point of the feasible region.

Definition Consider a LP in its standard form with n the number of variables, m the no. of main constraints (m < n) and Px = b the set of main constraints. (a) A basic solution to Px = b is obtained by setting n – m variables equal to 0 and then

solving the m equations for the m variables, provided that the resulting solution exists and is unique.

(b) If all the variables of the basic solution are nonnegative, it is then known as basic feasible solution. Otherwise, it is basic infeasible solution.

(c) The variables fixed at zero are called nonbasic and the ones obtained by solving the equations are called basic.

(d) The leaving variable is the variable that is going to change its status from basic to nonbasic.

(e) The entering variable is the variable that is going to change its status from nonbasic to basic.

Theorem The optimal solution to the linear programming problem Maximize z = cTx, subject to 0xbxx ≥==Ω ,P , when finite, must occur at an extreme point of its feasible set Ω.

Theorem Optimal solution A point x is an extreme point of the convex set Ω if and only if x is a basic feasible solution to Px = b, x ≥ 0. - The theorem guarantees that an optimal solution can be found by searching through

the basic feasible solutions. Since the standard LP form includes m linear equations with n unknown, we have to divide the n variables into two groups:

(1) n − m variables are nonbasic variables which we assign zero values; and (2) the remaining m variables are basic variables, its unique values are obtained by

solving the equations simultaneously. The resulting values give the basic solution. If the basic solution satisfies the nonnegativity constraints, then it is a basic feasible solution (correspond to one of the extreme point of the feasible region). In the attempt to improve the objective value, we identify one entering variable and one leaving variable so that a new basic feasible solution can be obtained (move to an adjacent basic feasible solution with better objective value). This process is repeated until the optimum reached.


T2, 2013/14 of 59

Simplex Algorithm Step 1: Convert the problem into the standard LP form Step 2: Construct the simplex tableau Step 3: Identify the entering and the leaving variables Step 4: Perform Gauss-Jordan elimination to obtain the new tableau. Step 5: If tableau is optimal, then output optimal solution and optimal value. Otherwise, repeat steps 3 and 4 until optimal stage reached. - The process of selecting the leaving variable and entering variable is stated in the

optimality condition and feasibility condition.

Optimality condition The entering variable in maximization (minimization) is the nonbasic variable with the most negative (positive) coefficient in the objective z-equation. The optimal stage is reached when all the nonbasic coefficients in the z-equation are nonnegative (nonpositive). Feasibility condition For both maximization and minimization problems, the leaving variable is the current basic variable associate with the smallest nonnegative ratio (element in Solution column ÷ corresponding element in entering column). Example: Solve the following Chemical problem by using the simplex algorithm. (Lecture Notes Series: Engineering Mathematics Volume .)

Maximize z = 3x1 + 2x2 (in $1000 unit)

subject to : x1 + 2x2 ≤ 6 (1) 2x1 + x2 ≤ 8 (2)

− x1 + x2 ≤ 1 (3) x2 ≤ 2 (4) x1 ≥ 0, x2 ≥ 0 Solution: Step 1: Convert the problem into the standard LP form


T2, 2013/14 of 59

Step 2: Construct the simplex tableau Iteration 0:

Basic z x1 x2 s1 s2 s3 s4 Solution Ratio z 1 −3 −2 0 0 0 0 0 s1 0 1 2 1 0 0 0 6 s2 0 2 1 0 1 0 0 8 s3 0 −1 1 0 0 1 0 1 s4 0 0 1 0 0 0 1 2

The basic feasible solution is:

Step 3: Identify the entering and the leaving variables The entering variable is

The leaving variable is § The row of the leaving variable is known as the pivot equation. § The column for the entering variable is known as the entering column. § The intersection element of the entering column and the pivot equation is

known as the pivot element. Step 4: Perform the Gauss-Jordan elimination to obtain the new tableau.

We need to re-express the simplex tableau such that the entering column has all zero entries, except for the pivot element, which must be unity. This can be accomplished through a series elementary row operations, generalized as follows:

1. For the pivot equation:

new pivot equation = old pivot equation ÷ pivot element 2. For all other equations:

new equation = old equation – (its entering column coefficient) × (new pivot equation)

To transform the Iteration 0 tableau, the relevant row operations are therefore: 1. For pivot equation, r3 : r3 = r3 /2 2. For z-equation, r1: r1 = r1 – (–3) r3 3. For s1-equation, r2: r2 = r2 – (1) r3 4. For s3-equation, r4: r4 = r4 – (–1) r3 5. For s4-equation, r5: r5 = r5 Iteration 1:

Basic z x1 x2 s1 s2 s3 s4 Solution Ratio z 1 0 –1/2 0 3/2 0 0 12 s1 0 0 3/2 1 –1/2 0 0 2 x1 0 1 1/2 0 1/2 0 0 4 s3 0 0 3/2 0 1/2 1 0 5 s4 0 0 1 0 0 0 1 2

The basic feasible solution is


T2, 2013/14 of 59

Step 5: The tableau is not yet optimal since the z-equation still has negative coefficients. Repeat Step 3 and Step 4.

From the tableau above, the entering variable is and the leaving variable is The elementary row operations involved are as follows: 1. For pivot equation, r2 : r2 = r2 /(3/2) 2. For z-equation, r1: r1 = r1 – (–1/2) r2 3. For x1-equation, r2: r3 = r3 – (1/2) r2 4. For s3-equation, r4: r4 = r4 – (3/2) r2 5. For s4-equation, r5: r5 = r5 – r2

Iteration 2:

Basic z x1 x2 s1 s2 s3 s4 Solution z 1 0 0 1/3

311 0 0

3212

x2 0 0 1 2/3 –1/3 0 0 311

x1 0 1 0 –1/3 2/3 0 0 313

s3 0 0 0 −1 1 1 0 3 s4 0 0 0 –2/3 1/3 0 1 2/3 The last tableau is optimal because none of the nonbasic variables have a negative coefficient in the z-equation. Hence the optimal solution is _________________ and the optimal value is ____.

Example: Solve the following LP problem. Maximize z = x1 + 4x2 + 5x3

subject to : 3x1 + 6x2 + 3x3 ≤ 22 x1 + 2x2 + 3x3 ≤ 14

3x1 + 2x2 ≤ 14 x1, x2, x3 ≥ 0

Solution:

The standard LP form is Maximize z = x1 + 4x2 + 5x3

subject to : 3x1 + 6x2 + 3x3 + s1 = 22 x1 + 2x2 + 3x3 + s2 = 14

3x1 + 2x2 + s3 = 14 x1, x2, x3, s1, s2, s3 ≥ 0

Basic z x1 x2 x3 s1 s2 s3 Solution Intercept ratio

z 1 −1 −4 −5 0 0 0 0 s1 0 3 6 3 1 0 0 22 22/3 s2 0 1 2 3 0 1 0 14 14/3 (min) s3 0 3 2 0 0 0 1 14 -


T2, 2013/14 of 59

The elementary row operations involved are as follows: 1. For pivot equation, r3 : r3 = r3 /3 2. For z-equation, r1: r1 = r1 – (–5) r3 3. For s1-equation, r2: r2 = r2 – (3) r3 4. For s3-equation, r4: r4 = r4

Basic z x1 x2 x3 s1 s2 s3 Solution Intercept ratio

z 1 2/3 −2/3 0 0 5/3 0 70/3 s1 0 2 4 0 1 −1 0 8 8/4 = 2 (min) x3 0 1/3 2/3 1 0 1/3 0 14/3 (14/3)/(2/3)=7 s3 0 3 2 0 0 0 1 14 14/2=7

The elementary row operations involved are as follows: 1. For pivot equation, r2 : r2 = r2 /4 2. For z-equation, r1: r1 = r1 – (–2/3) r2 3. For x3-equation, r3: r3 = r3 – (2/3) r2 4. For s3-equation, r4: r4 = r4 – 2 r

Basic z x1 x2 x3 s1 s2 s3 Solution z 1 1 0 0 1/6 3/2 0 74/3 x2 0 1/2 1 0 1/4 −1/4 0 2 x3 0 0 0 1 −1/6 1/2 0 10/3 s3 0 2 0 0 −1/2 1/2 1 10

Since all the objective coefficients are nonnegative, the optimal reached. The optimal solution is: x1 = 0, x2 = 2, x3 = 10/3. The optimal value is z = 74/3.

3.2.1 Special cases 1. When there is no leaving variable :- Unbounded solutions

The situation occurs when the coefficient for the denominator of the intercept ratios are either zero or negative.

Example: Consider a maximization problem. Iteration 0:

Basic z x1 x2 s1 s2 Solution Intercept ratio z 1 −4 −2 0 0 0 s1 0 2 −2 1 0 14 14/2 = 7(min) s2 0 1 −3 0 1 9 9 / 1 = 9


T2, 2013/14 of 59

Iteration 1: Basic z x1 x2 s1 s2 Solution Intercept ratio

z 1 0 −6 2 0 28 x1 0 1 −1 1/2 0 7 Negative value s1 0 0 −2 −1/2 1 2 Negative value

x2 can be taken as the entering variable, but the intercept ratios all carry negative values; the feasible region is unbounded and the optimal value can be improved infinitely.

2. When zero appears in the column of a nonbasic variable in the z-row of the optimal

tableau :- Multiple optimal solution / Alternative optimal solutions Example: Consider a maximization problem Iteration 0:

Basic z x1 x2 s1 s2 s3 s4 Solution Intercept ratio z 1 −8 −4 0 0 0 0 0 s1 0 2 1 1 0 0 0 180 180 / 2 = 90 s2 0 1 2 0 1 0 0 160 160 / 1 = 160 s3 0 5/11 5/11 0 0 1 0 50 50 / (5/11) = 550/5 s4 0 1/4 1/4 0 0 0 1 50 50 / (1/4) = 200

Iteration 1:

Basic z x1 x2 s1 s2 s3 s4 Solution Intercept ratio z 1 0 0 4 0 0 0 720 x1 0 1 1/2 ½ 0 0 0 90 90 / (1/2) = 180 s2 0 0 3/2 −1/2 1 0 0 70 70 / (3/2) = 140/3 s3 0 0 5/22 −5/22 0 1 0 100/11 (100/11) / (5/22) = 40 s4 0 0 1/8 −1/8 0 0 1 55/2 (55/2) / (1/8) = 220

- The above tableau is the optimal one, with the optimal solution as x1=90, x2 = 0. - The z-row coefficient of the nonbasic variable x2 is zero, indicating that x2 can enter

as the basic solution without changing the value of z. Iteration 2:

Basic z x1 x2 s1 s2 s3 s4 Solution z 1 0 0 4 0 0 0 720 x1 0 1 0 1 0 −11/5 0 70 s2 0 0 0 2 1 −33/5 0 10 x2 0 0 1 −1 0 22/5 0 40 s4 0 0 0 0 0 −11/20 1 45/2


T2, 2013/14 of 59

- This is again another optimal tableau with different optimal solution, x1 = 70, x2 = 40. Thus any point on the line segment joining these two extreme points will also be optimal:

x1 = 90 − 20t and x2 = 40t with 0 ≤ t ≤ 1.

3. Tie for the leaving basic variable:– Degeneracy - When there occurs a tie in choosing the leaving variable, then one or more basic

variables may become zero in the next iteration. A basic variable with value zero is called degenerate.

- Degeneracy implies that there is a possibility that the same BFS is encountered more than once. This may reveal that the LP model has cycling problem or redundant constraints.

Cycling - In cycling, we will loop among a set of basic feasible solutions without going to the

optimal solution. Example: Consider a maximization problem

Iteration 0: Basic z x1 x2 s1 s2 s3 Solution ratio

z 1 −3 −1 0 0 0 0 s1 0 −1 3

1 1 0 0 0 0 (tie) s2 0 −1 1 0 1 0 0 0 (tie) s3 0 −1 3 0 0 1 3

Iteration 1:

Basic z x1 x2 s1 s2 s3 Solution ratio z 1 0 −2 −3 0 0 0 x1 0 1

31− −1 0 0 0 0 (tie)

s2 0 0 32 −1 1 0 0 0 (tie)

s3 0 0 322 −1 0 1 3

Iteration 2:

Basic z x1 x2 s1 s2 s3 Solution z 1 −3 −1 0 0 0 0 s1 0 −1 3

1 1 0 0 0 s2 0 −1 1 0 1 0 0 s3 0 −1 3 0 0 1 3

The tableau obtained is exactly the same as the initial tableau. The simplex method does not lead to other extreme point with a better objective return.


T2, 2013/14 of 59

Redundant Constraints - At the extreme point of the redundant constraint, there would be more than one set

of nonbasic variables correspond to it. The simplex method would visit all these sets of nonbasic variables before it moves to other point or before it finds that the point is the optimal one.

Example: Consider a maximization problem. Iteration 0:

Basic z x1 x2 s1 s2 s3 Solution Intercept ratio z 1 −4 −12 0 0 0 0 s1 0 1 4 1 0 0 16 16 / 6 = 4(tie) s2 0 1 2 0 1 0 8 8/ 2 = 4 (tie) s3 0 2 1 0 0 1 6 6 / 1 = 6

Iteration 1:

Basic z x1 x2 s1 s2 s3 Solution Intercept ratio z 1 −1 0 3 0 0 48 x2 0 1/4 1 1/4 0 0 4 s2 0 1/2 0 −1/2 1 0 0 0 (min) s3 0 7/4 0 −1/4 0 1 2

Iteration 2:

Basic z x1 x2 s1 s2 s3 Solution z 1 0 0 2 2 0 48 x2 0 0 1 1/2 −1/2 0 4 x1 0 1 0 −1 2 0 0 s3 0 0 0 3/2 −7/2 1 2

(*Note: The basic feasible solutions at iterations 1 and 2 refer to the same point,

which is the extreme point for the redundant constraint)


T2, 2013/14 of 59

3.2.2 Matrix Representation of the Simplex Tableau - In contrast to the step-by-step algorithm of Sect 3.2.1, an entire Simplex tableau

can actually be generated directly using a set of matrix formulae. - Consider a representative LP problem:

Maximize z = cTx subject to: Ax ≤ b

x ≥ 0 - At any iteration, the matrix representation of the simplex tableau is given by:

Basic z x slack variables Solution z 1 cB

TB−1A − cT cBTB−1 cB

TB−1b xB 0 B−1A B−1 B−1b

where cT is the row vector of original objective function coefficients, A is the matrix of original LHS coefficients of the main constraints, b is the column vector of the RHS of the main constraints, xB is the column vector of basic variables for the current iteration,

cBT is the row vector of original objective coefficients corresponding to xB,

B is the matrix of original LHS constraint coefficients corresponding to xB. Example: Refer to the following LP problem (the same problem as in p. 21) which has been solved by simplex method. Maximize z = x1 + 4x2 + 5x3


3x1 + 2x2 ≤ 14 x1, x2, x3 ≥ 0

The standard LP form is

Maximize z = x1 + 4x2 + 5x3 subject to : 3x1 + 6x2 + 3x3 + s1 = 22

x1 + 2x2 + 3x3 + s2 = 14 3x1 + 2x2 + s3 = 14

x1, x2, x3, s1, s2, s3 ≥ 0

Applying simplex method, we have the followings:


T2, 2013/14 of 59

Iteration 0:

Basic z x1 x2 x3 s1 s2 s3 Solution Intercept ratio z 1 −1 −4 −5 0 0 0 0 s1 0 3 6 3 1 0 0 22 22/3 s2 0 1 2 3 0 1 0 14 14/3 (min) s3 0 3 2 0 0 0 1 14 -

Iteration 1:

Basic z x1 x2 x3 s1 s2 s3 Solution Intercept ratio z 1 2/3 −2/3 0 0 5/3 0 70/3 s1 0 2 4 0 1 −1 0 8 8/4 = 2 (min) x3 0 1/3 2/3 1 0 1/3 0 14/3 (14/3)/(2/3)=7 s3 0 3 2 0 0 0 1 14 14/2=7

Iteration 2: Basic z x1 x2 x3 s1 s2 s3 Solution

z 1 1 0 0 1/6 3/2 0 74/3 x2 0 1/2 1 0 1/4 −1/4 0 2 x3 0 0 0 1 −1/6 1/2 0 10/3 s3 0 2 0 0 −1/2 1/2 1 10

The optimal solution is : x1 = 0, x2 = 2, x3 = 10/3. The optimal value is z = 74/3.

We could regenerate the table at iteration 1 as follows: Since [ ] T

B sxs 331=x (the basic variables at iteration 1 must be known in advance) , we have

[ ],050=TBc

−=⇒

= −

10000011

100030031

311BB

Thus, the z-row coefficients for the slack variables are:

[ ] [ ]0010000011

550 35

311 =

−=−BT

Bc


T2, 2013/14 of 59

And the z-row coefficients for the decision variables x are:

[ ] [ ] .032

32541

023321363

00 351

−

=−

=−− TT

B AB cc

The right-hand side values (i.e. entries under the Solution column) are:

[ ]3

70

141422

00 351 =

=− bc BT

B ,

=

−== −

14

8

141422

10000011

314

311bx BB

The left-hand side coefficients (i.e. entries below the z-row) for x and the slack variables are respectively:

=

−= −−

0231042

023321363

10000011

32

31

311 AB and

−= −−

10000011

311B

3.3 The Primal and the Dual Problem Ø Every LP has associated with it another LP which is called as the dual. Ø When finding the dual of a given LP problem, we refer to the given LP as the primal

problem. Ø To find the dual for a primal, we assume that the primal is in the standard form, then

(i) A dual variable is defined for each of the m primal constraint equations. (ii) A dual constraint is defined for each of the n primal variables. (iii) The LHS coefficients of the i-th dual constraint equal the constraint

coefficients of the i-th primal variable. The right hand side of the dual constraint equal the objective coefficient of the same primal variable.

(iv) The objective coefficients of the dual equal the right hand sides of the primal constraint equations.

(v) Finally, the objective function type, the constraint type and the variable type of the dual LP problem may be determined from the table below:

Primal problem Dual problem

Objective function Objective function Constraints type Variable sign Maximization Minimization ≥ .urs Minimization Maximization ≤ .urs


T2, 2013/14 of 59

Example: Find the dual of the following primal problem. Maximize w = 5x1 + 4x2 subject to x1 + 3x2 ≤ 10

2x1 + 6x2 ≤ 12 5x1 − 3x2 ≥ 10 2x1 − 3x2 ≥ 15

x1, x2 ≥ 0. Solution: Put into the standard LP form.

Maximize w = 5x1 + 4x2 subject to x1 + 3x2 + s1 = 10

2x1 + 6x2 + s2 = 12 5x1 − 3x2 − s3 = 10 2x1 − 3x2 − s4 = 15

x1, x2, s1, s2, s3, s4 ≥ 0. The dual is

Minimize z = 10y1 + 12y2 + 10y3 + 15y4 subject to y1 + 2y2 + 5y3 + 2y4 ≥ 5

3y1 + 6y2 − 3y3 – 3y4 ≥ 4 y1 ≥ 0 y2 ≥ 0

− y3 ≥ 0 − y4 ≥ 0

y1, y2, y3, y4 .urs. Simplify it, we have the dual problem as

Minimize z = 10y1 + 12y2 + 10y3 + 15y4 subject to y1 + 2y2 + 5y3 + 2y4 ≥ 5

3y1 + 6y2 − 3y3 – 3y4 ≥ 4 y1, y2 ≥ 0, y3, y4 ≤ 0

Example: Find the dual for the following primal LP problem,

Minimize w = 2x1 + x2 subject to x1 + x2 = 2

2x1 − x2 ≥ 3 x1 − x2 ≤ 1

x1 ≥ 0, x2 .urs


T2, 2013/14 of 59

Solution: Set −+ −= 222 xxx . The standard LP form is Minimize w = −+ −+ 2212 xxx subject to

.0,,,,

1

3 2

2

32221

3221

2221

221

≥

=++−

=−+−

=−+

−+

−+

−+

−+

ssxxxsxxx

sxxxxxx

The dual is

Maximize z = 2y1 + 3y2 + y3 subject to y1 + 2y2 + y3 ≤ 2

y1 − y2 − y3 ≤ 1 −y1 + y2 + y3 ≤ −1

− y2 ≤ 0 y3 ≤ 0

y1, y2, y3 .urs.

Simplify the above, the dual problem is Maximize z = 2y1 + 3y2 + y3 subject to y1 + 2y2 + y3 ≤ 2 y1 − y2 − y3 = 1

y1 .urs, y2 ≥ 0, y3 ≤ 0. 3.3.1 Relationship Between Primal and Dual Problem Property I: For any pair of feasible primal and dual solutions:

≤

problem onminimizati

thein valueObjectiveproblem onmaximizati

thein valueObjective

At the optimal stage:

=

problem onminimizati

thein valueObjectiveproblem onmaximizati

thein valueObjective


T2, 2013/14 of 59

Example: Primal Minimize w = 5x1 + 2x2 subject to x1 – x2 ≥ 3 2 x1 + 3x2 ≥ 5 x1, x2 ≥ 0

Dual Maximize z = 3 y1 + 5y2 subject to y1 + 2 y2 ≤ 5 – y1 + 3 y2 ≤ 2 y1, y2 ≥ 0

Primal Dual

Points Feasibility w Points Feasibility z Estimation for optimal value

(4,1) Yes 22 (1,0) Yes 3 3 ≤ optimal value ≤ 22 (3,0) Yes 15 (3,1) Yes 14 14 ≤ optimal value ≤ 15 (3,1) No (4,1) No (3,0) Yes 15 (5,0) Yes 15 Optimal value = 15 Note: Points are chosen arbitrarily for illustration purpose only. Feasibility can be tested by checking if the chosen points satisfy the given constraints. Property II: At the optimal stage, the dual variables will have the solution

yT = TBc B−1.

- T

Bc and B−1 are elements from the matrix representation of the Simplex tableau, their meanings explained in Sect. 3.2.2 (see p. 26).

- The expression TBc B−1 corresponds exactly to the z-row entries below the slack

variables in the Simplex tableau. In other words, the optimal solution of the dual variables is simply the slack variable coefficients of the primal LP at the optimal stage.

Example: Consider the following problem: Primal Maximize z = x1 + 4x2 + 5x3


3x1 + 2x2 ≤ 14

x1, x2, x3 ≥ 0

Dual Minimize w = 22y1 + 14y2 + 14y3 subject to : 3y1 + y2 + 3y3 ≥ 1 6y1+ 2y2 + 2y3 ≥ 4 3y1+ 3y2 ≥ 5 y1, y2, y3 ≥ 0.


T2, 2013/14 of 59

The optimal tableau for the primal problem is:

Basic z x1 x2 x3 s1 s2 s3 Solution z 1 1 0 0 1/6 3/2 0 74/3 x2 0 1/2 1 0 1/4 −1/4 0 2 x3 0 0 0 1 −1/6 1/2 0 10/3 s3 0 2 0 0 −1/2 1/2 1 10

By Property II, the optimal solution of the dual variables can be easily obtained:

yT = TBc B−1 = [ ]02/36/1 ,

thus y1 = 1/6, y2 = 3/2 and y3 = 0. Furthermore, by Property I, we may conclude that the optimal value of the dual is w = 74/3. Remarks Ø Because of the dual properties, it may be easier and simpler to solve a LP problem

vial its dual problem instead of solving its primal problem directly. Generally, the number of the main constraints affects the computation effort of the simplex method far more than the number of variables does. A dual problem with fewer constraints than its primal can usually be solved in less simplex iterations compared to the primal.

Economic interpretation of dual problem Ø Consider the primal and dual problem as follows:

Primal

0xbx

xc

≥≤

=

:subject to Maximize

Az T

where c, x ∈ Rn, b ∈ Rm , A ∈ Rm × n.

Dual

0ycy

by

≥≥

=

:subject to

MinimizeTT

T

Aw

where y, c ∈ Rn, b ∈ Rm , A ∈ Rm × n.

Ø Suppose the objective of the primal problem is to maximize revenue, subject to the

availability of limited resources. Thus, we can think of z as representing profit dollars ( $ ).

Ø By Property I, at the optimal stage we have

z = w = yT b = ∑=

m

iii yb

1

Keeping in mind that bi represents the availability of resource i, the above can be expressed dimensionally as

$ = Σ (units of resource i) × ($ / unit of resource i) Ø Thus, yi represents the $ amount that each unit of resource i contributes to the

profit z, also known as the dual price or shadow price. More formally, yi refers to


T2, 2013/14 of 59

the $ amount by which the optimal z-value can be improved if the right hand side of the ith constraint, bi, is increased by 1 unit.

Example: Refer to the Chemical problem in p. 7 and its optimal simplex tableau in p. 21. By Property II, the optimal solution for its dual variables y1 and y2 are 3

1 and 311 ,

respectively. This means that if the availability of raw material A is increased by 1 unit, then the total profit will increase by RM( 3

1 ×1000). Similarly, incrementing the availability of raw material B by 1 unit will contribute RM( 3

11 ×1000) to the total profit. So if faced with a choice, raw material B should be given the priority for increase. 3.4 Sensitivity Analysis Ø Sensitivity analysis is a process to explore how changes in any parameter of a

problem would affect the optimal solution. In this section, we are going to investigate the following cases: (a) Changes in the coefficients of the objective function (marginal profit/cost) (b) Changes in the right hand sides (resource availability) (c) Addition of a new decision variable (new activity)

Ø In general, sensitivity analysis utilizes the matrix representation of simplex tableau

presented in Sect. 3.2.2: Basic z x slack variables Solution

z 1 yTA − cT yT = cBTB−1 yTb

xB 0 B−1A B−1 B−1b

For each change, we just need to re-compute the affected parts in the tableau. Then we check if the new values satisfy the optimality condition (coefficients in the z-row are all non-negative) and the feasibility condition (solution of basic variables are all non-negative).

Ø Throughout the whole section 3.4, the following problem (first appeared in p. 21) would be used to demonstrate the idea of the sensitivity analysis. The original mathematics model and the respective optimal table are restated here for reference. Example: The Product Problem.

Let x1, x2 and x3 be the units of Product 1, 2 and 3 produced daily in tons, and let z be the total daily income obtained by selling all these products. The LP is given as:

Maximize z = x1 + 4x2 + 5x3

subject to : 3x1 + 6x2 + 3x3 ≤ 22 (raw material A) x1 + 2x2 + 3x3 ≤ 14 (raw material B)

3x1 + 2x2 ≤ 14 (raw material C) x1, x2, x3 ≥ 0


T2, 2013/14 of 59

By simplex method, the optimal tableau is:

Basic z x1 x2 x3 s1 s2 s3 Solution

z 1 1 0 0 1/6 3/2 0 74/3 x2 0 1/2 1 0 1/4 −1/4 0 2 x3 0 0 0 1 −1/6 1/2 0 10/3 s3 0 2 0 0 −1/2 1/2 1 10

(a) Changes in the Coefficients of the Objective Function Ø The changes just affect the entries in the z-row of the simplex tableau. Thus, may

alter the optimality of the problem.

If the objective coefficient of the basic variable xj is changed, then the vectors Tc and TBc would change. Let T*c be the new coefficients for the objective function, and T

B*c

be the new objective coefficients for the optimal xB . Step 1: Compute the new coefficients in the z-row:

y*T = TB*c B-1, TT A ** cy − .

Basic z x slack variables Solution

z 1 TT A ** cy − y*T = TB*c B−1 y*Tb

xB 0 B−1A B−1 B−1b Note, however, that if the variable xj is not a basic variable, then the vector

TBc and yT remain, we just need to calculate the values of TT A *cy − .

Step 2: Two cases will result:

(i) The optimality condition is satisfied, the current optimal solution remains with the new optimal value y*Tb.

(ii) The optimality condition is not satisfied, simplex method must be applied to recover optimality.

Example: Identify the new optimal solution for the Product Problem if: (a) the prices per ton for product 2 and product 3 are changed to 5 and 6 respectively. (b) the price per ton for product 1 is changed to 2.5.


T2, 2013/14 of 59

Solution:

(a) From the original optimal tableau, [ ] TB sxx 332=x . Thus [ ]065* =T

Bc , and we have

[ ] [ ]01100

065 43

41

21

21

21

61

41

41

1** =

==

−

−

−

−BTB

T cy .

As for the remaining z-row entries represented by y*TA − c*T, we have

[ ] [ ] [ ].002/3651023321363

01 43

41** =−

=− TT A cy

Since all the new z-row coefficients are nonnegative, the optimality condition is still satisfied. The current optimal solution remains unchanged, but the new optimal value is given by

[ ] .30141422

01 43

41* =

=by T

The new optimal simplex tableau is

Basic z x1 x2 x3 s1 s2 s3 Solution z 1 3/2 0 0 1/4 7/4 0 30 x2 0 1/2 1 0 1/4 −1/4 0 2 x3 0 0 0 1 −1/6 1/2 0 10/3 s3 0 2 0 0 −1/2 1/2 1 10

Note the changes compared to the original optimal tableau.

(b) Here, the price per ton is changed for a non-basic variable x1. Thus yT = cBTB−1 is

unaffected, leaving only yTA − c*T to be re-calculated. The z-row coefficients for x2 and x3 will remain as 0, so we may focus solely on the

coefficient of x1,

[ ] .212313

0 21

23

61*

11 −=−

=− cATy

Since 021*

11 <−=− cATy , the optimality condition is not satisfied. We have to perform another iteration of the simplex algorithm. The simplex tableau is:


T2, 2013/14 of 59

Basic z x1 x2 x3 s1 s2 s3 Solution

z 1 −1/2 0 0 1/6 3/2 0 74/3 x2 0 1/2 1 0 1/4 −1/4 0 2 x3 0 0 0 1 −1/6 1/2 0 10/3 s3 0 2 0 0 −1/2 1/2 1 10

Apply simplex method to recover the optimality, we have

Basic z x1 x2 x3 s1 s2 s3 Solution z 1 0 1 0 5/12 1 1/4 0 26 2/3 x1 0 1 2 0 1/2 −1/2 0 4 x3 0 0 0 1 −1/6 1/2 0 3 1/3 s3 0 0 −4 0 −1 1/2 1 1/2 1 2

The new optimal solutions are x1 = 4, x2 = 0 and x3 = 3 1/3 with the new optimal value is z = 26 2/3.

(b) Changes in the Right Hand Side of the Constraints Ø Changes in the right hand sides would affect the feasibility of the LP problem.

Given that the new RHS as b*. Step 1: Compute the new solution of the basic variables: B-1b*

Basic z x slack variables Solution z 1 TT A cy − yT = T

Bc B−1 yTb*

xB 0 B−1A B−1 B−1b*

Step 2: Two cases will result: (i) If B-1b*≥ 0, the resulting solution remains feasible with the new

optimal value yTb*. (ii) If one of the values in B-1b* ≤ 0, the resulting solution is infeasible.

To obtain the optimal solutions, dual simplex method must be applied.

Example: Find the new optimal solution and optimal value of the Product Problem if: (a) the daily availability of raw material A has been changed to 18. (b) both the daily availability of raw material B and C have been changed to 24.


T2, 2013/14 of 59

Solution:

(a) Set

=

141418

*b .

The new solution of the basic variables are:

=

=

−

−

−

−

1241

141418

100

21

21

21

61

41

41

*1bB

Since B-1b*≥ 0, the solution is optimal and feasible. The new optimal value is

[ ] 24141418

023

61* =

=byT .

The new optimal tableau is:

Basic z x1 x2 x3 s1 s2 s3 Solution z 1 1 0 0 1/6 3/2 0 24 x2 0 ½ 1 0 1/4 −1/4 0 1 x3 0 0 0 1 −1/6 1/2 0 4 s3 0 2 0 0 −1/2 1/2 1 12

The new optimal solution is x1 = 0, x2 = 1 and x3 = 4 with the optimal value is z = 24. Note the changes compared to the original optimal tableau.

(b) Set

=

242422

*b .

The affected entries are:

=

=

−

−

−

−

−

258

242422

100

31

21

21

21

21

61

41

41

*1bB

Since one of the entries in 0x <*B , the resulting solution becomes infeasible. Dual

simplex method has to be applied to recover the feasibility.


T2, 2013/14 of 59

(c) Addition of A New Decision Variable Ø Addition of a new decision variable is similar to adding a new activity. The addition

of this new variable would only affect the optimality of a problem. The new decision variable would become one of the basic solutions if and only if the new variable is able to improve the objective function value. If not, this new variable would act as the nonbasic variable.

Step 1: Add the new decision variable, xi, to the LP model. xi is assumed to be

nonbasic first. Step 2: Calculate the z-row coefficient for xi : yTAj − cj .

Step 3: Two cases will result:

(i) If the optimality condition is satisfied, the optimal solution remains. (ii) If the optimality condition is not satisfied, the objective value can be

improved by taking xi as basic variable. The entries of xi in the tableau is:

z-row coefficient: yTAj − cj coefficients below z-row: B−1Aj

Basic z x xi slack variables Solution

z 1 TT A cy − yTAj − cj yT = TBc B−1 yTb

xB 0 B−1A B−1Aj B−1 B−1b

Then, perform simplex iteration until the optimal stage is reached.

Example: Suppose that a new product, Product 4, has been introduced into the Product problem. The new LP model is as follows: Maximize z = x1 + 4x2 + 5x3 + 4x4

subject to: 3x1 + 6x2 + 3x3 + 3x4 ≤ 22 (raw material A) x1 + 2x2 + 3x3 + x4 ≤ 14 (raw material B)

3x1 + 2x2 + x4 ≤ 14 (raw material C) x1, x2, x3, x4 ≥ 0

What will the new optimal value be? Solution: First, we assume x4 = 0 (nonbasic variable). Its corresponding z-row coefficient is

[ ] .24113

023

61

44 −=−

=− cATy


T2, 2013/14 of 59

Since 0244 <−=− cATy , the optimality condition is not satisfied. x4 must enter as a basic variable in the next simplex iteration. But first, we complete the column for x4 in the simplex tableau by computing

=

=

−

−

−

−

00

113

100 2

1

21

21

21

61

41

41

41AB ,

and the simplex tableau with x4 inserted is as follows.

Basic z x1 x2 x3 x4 s1 s2 s3 Solution z 1 1 0 0 −2 1/6 3/2 0 74/3 x2 0 1/2 1 0 1/2 1/4 −1/4 0 2 x3 0 0 0 1 0 −1/6 1/2 0 10/3 s3 0 2 0 0 0 −1/2 1/2 1 10

By performing another simplex iteration, the tableau becomes

Basic z x1 x2 x3 x4 s1 s2 s3 Solution z 1 1 1/2 4 0 0 1 1/6 1/2 0 32 2/3 x4 0 1 2 0 1 1/2 − 1/2 0 4 x3 0 0 0 1 0 − 1/6 1/2 0 3 1/3 s3 0 2 0 0 0 − 1/2 1/2 1 10

The new optimum value is 32 2/3 with the optimal solution is x1 = 0, x2 = 0, x3 = 3 1/3 and x4 = 4.


T2, 2013/14 of 59

4.0 Dynamic Programming Ø Dynamic programming (DP) is a recursive approach to solve an n-variable

optimization problem by breaking up the problem into n-stage and at each stage computing a single variable.

Ø DP problems may be solved using forward recursion – in which the computations

proceed from the first stage to the last stage – or using backward recursion – in which the computations start from the last stage and end at the first stage. Both forward and backward recursions yield the same solution. However, most DP literature employs backward recursion as it is generally more efficient computationally.

Ø The basic steps in solving DP problems using backward recursion are:

Step 1: Decomposition (break the problem into stages based on the no. of decisions to be made)

Step 2: Define the state variables, recursive formula and decision variables Step 3: Carry out the computation stage by stage, beginning from the last stage Step 4: Composition (trace the decisions from the first stage to the last to form the

optimal policy) Definition: (a) The state variable, u, describes the possible "system conditions" at a particular stage-

j. The state variable is the information that links the stages together so that the optimal decisions for the upcoming stages can be made without having to re-examine the decision made at stage j.

(b) The recursive formula, ( )uf j , is the objective function used to rate all the alternatives when the system is in state u at stage j. ( )uf j gives the optimal cost/benefit beginning stage j till the last stage.

(c) The decision variable is the decision made at stage j to achieve ( )uf j . (d) The optimal policy is the sequence of decisions made at each stage. 4.1 Solution of Routing Problem by Stages (Backward recursion) Example: The following network provides the possible routes and the distances from City 1 to City 10. Determine the shortest route from City 1 to City 10.

City 1

City 2

City 3 City 5

City 6

City 7

City 8 City 10 5

6

6

2

3

4

10

12

6 6

4

4

City 4

7

9 4

City 9

12

8

3


T2, 2013/14 of 59

Solution: Step 1: Decomposition (break the problem into stages) - The problem can be divided to 4 stages. At each stage, the individual needs to decide

which city to visit next. Step 2: Define the state variables, recursive formula and decision variables Let the state variable, u be our possible location at the start of a particular stage, the recursive formula, fj(u) be the minimum distance to complete the tour beginning city u till city 10, and the decision variable, dj(u) be the next city visited in order to achieve fj(u). Step 3: Carry out the computation Stage 4 Computation Assume that Stages 1, 2 and 3 have been completed. At the start of Stage 4, we must be either in city 7, 8 or 9. Thus, u = 7, 8 or 9. Since this is final stage of the tour, the obvious decision is to simply go from the current position to city 10.

f4(7) = 4 d4(7) = 10 (from city 7 to city 10) f4(8) = 3 d4(8) = 10 (from city 8 to city 10) f4(9) = 4 d4(9) = 10 (from city 9 to city 10)

Stage 3 Computation In this stage of the tour, we start out in either city 5 or 6, i.e. u = 5 or 6. From u, we must decide whether to proceed to city 7, 8 or 9. The immediate distance from u to city 7, 8 and 9 is given in the network, but since our goal is to reach city 10, we have to factor in the distance calculated in Stage 4. Set mut = the distance between cities u and t, the recursive formula is

( ) ( ) .6,5,min 49,8,73 =+==

utfmuf utt

f3(5) = minm57 + f4(7), m58 + f4(8), m59 + f4(9)

= min12+4, 12+3, 10+4 = min16, 15, 14 = 14 d3(5) = 9 (from city 5 to city 9) f3(6) = minm67 + f4(7), m68 + f4(8), m69 + f4(9) = min8+4, 6+3, 6+4 = min12, 9, 10 = 9 d3(6) = 8 (from city 6 to city 8)

city 1 1st city

2nd city 3rd city

city 10 Stage 1

Stage 2

Stage 3

Stage 4


T2, 2013/14 of 59

Stage 2 Computation We start from u=2, 3 or 4, and at each u must decide whether to proceed to city 5 or 6. The recursive formula is:

( ) ( ) .4,3,2,min 36,52 =+==

utfmuf utt

f2(2) = min m25 + f3(5), m26 + f3(6) = min 6 +14, 4 + 9 = min20, 13 = 13 d2(2) = 6 (from city 2 to city 6) f2(3) = min m35 + f3(5), m36 + f3(6) = min 2 + 14, 3 + 9 = min16, 12 = 12 d2(3) = 6 (from city 3 to city 6) f2(4) = min m45 + f3(5), m46 + f3(6) = min 4 + 14, 9 + 9 = min18, 18 = 18 d2(4) = 5, 6 (from city 3 to city 5 or city 6)

Stage 1 Computation u = 1. The recursive formula is:

f1(1) = min m12 + f2(2), m13 + f2(3), m14 + f2(4) = 5 + 13, 7 + 12, 6 + 18 = 18, 19, 24 = 18 d1(1) = 2 (from city 1 to city 2)

The above computation result can be represented in tableau form:

u 1 2 3 4 5 6 7 8 9

Stage 4 f4(u) 4 3 4 d4(u) 10 10 10

Stage 3 f3(u) 14 9 d3(u) 9 8

Stage 2 f2(u) 13 12 18 d2(u) 6 6 5,6

Stage 1 f1(u) 18 d1(u) 2

Step 4 : Composition: determine the optimal policy The minimum distance from city 1 to city 10 is f1(1) = 18. The optimal policy is

d1(1) = 2 : the individual chooses to go to city 2 from city 1. d2(2) = 6 : he next visits city 6.

( ) ( ) .tfmminf t,,t 214321 1 +==


T2, 2013/14 of 59

d3(6) = 8 : from city 6, he chooses to go to city 9. d4(9) = 10 : he arrives at city 10.

In conclusion, the optimal route is 1-2-6-8-10 with a minimum distance of 18.

4.2 Review and implication of computational process Ø DP is an approach to optimization rather than a technique, there is no general

algorithm. The computation details at each stage are different and are problem dependent.

Ø The process of optimizing the multistage decisions problem is based on the principle

of optimality. Definition: Principle of optimality An optimal policy has the property that, regardless of the decision taken to enter a particular state in a particular stage, the remaining decisions must constitute an optimal policy for leaving that state.

4.3 Resource Allocation Problem Ø Resource allocation problem is the problem in which limited resources must be

optimally allocated among several activities. Example: An individual has $4000 to invest and three opportunities available to him. Each opportunity requires deposits in $1000 increments. The expected returns are tabulated as follows:

$ Invested 0 1000 2000 3000 4000

Return from opportunity 1 0 2000 5000 6000 7000 Return from opportunity 2 0 1000 3000 6000 7000 Return from opportunity 3 0 1000 4000 5000 8000 Determine an optimal policy on how the investor can distribute the money among different opportunity to achieve the maximum total returns. Solution: Set xi (i=1,2,3) = the amount of money (in thousands of $) invested in opportunity i, mi(xi) (i=1,2,3) = the return (in thousand-$ units) from opportunity i when xi units of

money are invested in it.


T2, 2013/14 of 59

The problem can be formulated as: Maximize z = m1(x1) + m2(x2) + m3(x3) subject to : x1 + x2 + x3 ≤ 4 with xi nonnegative and integral, i = 1,2,3.

Step 1: Decomposition (break the problem into stages)

The problem can be divided to 3 stages, j = 1, 2, 3. At stage j the decision has to be made as to how much money to allocate to opportunity j. Step 2: Define the state variables, recursive formula and decision variables Let

u be the money available for investment in thousands of dollars (u=0,1,2,3,4), fj(u) be the maximum return from stage j till stage 3 with u thousand dollars

available at the start of stage j, dj(u) be the amount of money out of u thousand dollars that should be invested in

opportunity j to achieve fj(u). Step 3: Carry out the computation. The following network shows the possible amount of money available at each stage:

Stage 3 Computation: Assume that the individual has already invested in opportunity 1 and 2. The amount of money available for opportunity 3 is possibly u = 0, 1, 2, 3 or 4 (in units of thousands).

4

1

2

0

1

0

3

Stage 1 Invest in opportunity I

4

2

3

4

Stage 2 Invest in opportunity II

Stage 3 Invest in opportunity III

0

1

2

4

1 2

3 4

0 1

2 3 4 0

1 2 3

0 1

2 0 1

0

3

0


T2, 2013/14 of 59

No matter what value of u at this stage, it is clear that we can maximize return by allocating all remaining funds to opportunity 3. We thus have,

f3(4) = 8 d3(4) = 4 (4 units to be invested) f3(3) = 5 d3(3) = 3 (3 units to be invested) f3(2) = 4 d3(2) = 2 (2 units to be invested) f3(1) = 1 d3(1) = 1 (1 unit to be invested) f3(0) = 0 d3(0) = 0 (0 unit to be invested)

Stage 2 Computation Here, we wish to compute how much to invest in opportunity 2. According to the network above, the amount of funds available is possibly u = 0,1,2,3 or 4. With u amount of money available, we can decide to invest anywhere between $0 and $u thousand into opportunity 2. The returns from this stage will be aggregated with the returns from Stage 3. The recursive formula is therefore:

)()(max)( 3202 xufxmufux

−+=≤≤

We have, f2(4) = max m2(0) + f3(4−0), m2(1) + f3(4−1), m2(2) + f3(4−2), m2(3) + f3(4−3), m2(4) + f3(4−4) = max0+8, 1+5, 3+4, 6+1, 7+0 = 8 d2(4) = 0 (0 unit to be invested) f2(3) = max m2(0) + f3(3−0), m2(1) + f3(3−1), m2(2) + f3(3−2), m2(3) + f3(3−3) = max0+5, 1+4, 3+1, 6+0 = 6 d2(3) = 3 (3 units to be invested) f2(2) = max m2(0) + f3(2−0), m2(1) + f3(2−1), m2(2) + f3(2−2) = max0+4, 1+1, 3+0 = 4 d2(2) = 0 (0 unit to be invested) f2(1) = max m2(0) + f3(1−0), m2(1) + f3(1−1) = max0+1, 1+0 = 1 d2(1) = 0,1 (0,1 unit to be invested) f2(0) = max m2(0) + f3(0−0) = max0+0 = 0 d2(0) = 0 (0 unit to be invested) Stage 1 computation Since Stage 1 corresponds to investing in opportunity 1, we all the money available on hand, so u = 4. The recursive formula is:

)4()(max)4( 21401 xfxmfx

−+=≤≤

f1(4) = maxm1(0)+f2(4−0), m1(1)+f2(4−1), m1(2)+f2(4−2), m1(3)+f2(4−3), m1(4) + f2(4−4) = max0+8, 2+6, 5+4, 6+1, 7+0 = 9 d1(4) = 2 (2 units to be invested)


T2, 2013/14 of 59

Summarizing the results in tableau form:

u 0 1 2 3 4

f3(u) 0 1 4 5 8 d3(u) 0 1 2 3 4 f2(u) 0 1 4 6 8 d2(u) 0 0,1 0 3 0 f1(u) - - - - 9 d1(u) - - - - 2

Step 4 : Composition: determine the optimal policy by working forward The optimal return is given by f1(4) = 9 units. To achieve this return, the decisions to be made are d1(4) = 2 d2(4−2) = d2(2) = 0 d3(2) = 2 The optimal policy is to allocate $2000 to investment 1, $0 to investment 2, and $2000 to investment 3. 4.4 Equipment Replacement Problems - Equipment replacement problem is the problem of determining how long an

equipment should be utilized before it should be traded in for a new one. The alternatives at each stage are either to replace or keep using the equipment.

Example: A company needs to determine the optimal replacement policy that would maximize the total profit for a current 3-year-old machine over the next 4 years – that is, until the start of year 5. The machine would be sold at the end of the problem. The following table gives the data of the problem. The company requires that a 6-year old machine be replaced. The cost a new machine is $100, 000.

Age, t (yr)

Revenue, r(t) ($)

Operating cost, c(t) ($)

Salvage value, s(t) ($)

0 20, 000 200 - 1 19, 000 600 80, 000 2 18, 500 1, 200 60, 000 3 17, 200 1, 500 50, 000 4 15, 500 1, 700 30, 000 5 14, 000 1, 800 10, 000 6 12, 200 2, 200 5, 000

(Taha, Operations Research: An Intoduction)


T2, 2013/14 of 59

Solution: Step 1: Decomposition (break the problem into stages) - The problem is a five-stage process, j = 1, 2, 3, 4, 5, with stage j representing the j-th

year of the policy. The decision at each stage is whether to keep or replace the current machine on hand.

Step 2: Define the state variables, recursive formula and decision variables Set

u = the possible ages of the machine at the start of the j-th year.

fj(u) = the maximum profit (in $1000) obtained from stage j till stage 5 when the machine is u years old at the start of stage j.

dj(u) = the decision of whether to KEEP or REPLACE a u year old machine at the start of stage j in order to realize fj(u). We will SELL the machine at the end of the problem.

Step 3: Carry out the computation. The following network summarizes the feasible age of the machine for each stage:

The recursive formulae are as follows: Case 1: for stage j = 5 Since the problem is over at time 5, we sell the machine at time 5 and receive the salvage value su.

f5(u) = su.and d5(u) = SELL. Case 2 : for stages j = 2, 3 and 4 When u = 1, 2, 3, 4 or 5, the recursive formula is:

fj(u) = max s(u)– 100 + r(0) – c(0) + fj+1(1), (if REPLACE) r(u) – c(u) + fj+1(u+1) (if KEEP)

3

4

5

K

1

2

1

2

1

2

1

3 3

K

K

K

K

K

K R

R

R R R

R R

R R

S

S

S

1st Year Stage 1

K – “Keep” R – “Replace” S – “Sell”

6

K

R 4

S

2nd Year Stage 2

3rd Year Stage 3

4th Year Stage 4

5th Year Stage 5

K


T2, 2013/14 of 59

When the machine is 6 years old (i.e. u = 6), it has to be replaced.

fj(6) = s(6)– 100 + r(0) – c(0) + fj+1(1) and dj(6) = REPLACE Case 3 : for stage 1

f1(3) = max s(3) – 100 + r(0) – c(0) + f2(1), (if REPLACE) r(3) – c(3) + f2(4) (if KEEP) Stage 5 computation: u = 1, 2, 3, 4

f5(1) = 80 d5(1) = SELL f5(2) = 60 d5(2) = SELL f5(3) = 50 d5(3) = SELL f5(4) = 30 d5(4) = SELL

Stage 4 computation: u = 1, 2, 3, 6

f4(6) = 5 – 100 + 20 – 0.2 + f5(1) = 4.8 d4(6) = REPLACE

f4(3) = max 50 – 100 + 20 – 0.2 + f5(1) , 17.2 – 1.5 + f5(4) = max 49.8, 45.7 = 49.8 d4(3) = REPLACE

f4(2) = max 60 – 100 + 20 – 0.2 + f5(1), 18.5 – 1.2 + f5(3) = max 59.8, 67.3 = 67.3 d4(2) = KEEP f4(1) = max 80 – 100 + 20 – 0.2 + f5(1), 19 – 0.6 + f5(2) = max 79.8, 78.4 = 79.8 d4(1) = REPLACE

Stage 3 computation: u = 1, 2, 5.

f3(5) = max 10 – 100 + 20 – 0.2 + f4(1), 14 – 1.8 + f4(6) = max 9.6, 17 = 17 d3(5) = KEEP f3(2) = max 60 – 100 + 20 – 0.2 + f4(1), 18.5 – 1.2 + f4(3) = max 59.6, 67.1 = 67.1 d3(2) = KEEP f3(1) = max 80 – 100 + 20 – 0.2 + f4(1) , 19 – 0.6 + f4(2) = max 79.6, 85.7 = 85.7 d3(1) = KEEP


T2, 2013/14 of 59

Stage 2 computation: u = 1, 4.

f2(4) = max 30 – 100 + 20 – 0.2 + f3(1), 15.5 – 1.7 + f3(5) = max 35.5, 30.8 = 35.5 d2(4) = REPLACE f2(1) = max 80 – 100 + 20 – 0.2 + f3(1), 19 – 0.6 + f3(2) = max 85.5, 85.5 = 85.5 d2(1) = KEEP, REPLACE

Stage 1 computation: u = 3.

f1(3) = max 50 – 100 + 20 – 0.2 + f2(1) , 17.2 – 1.5 + f2(4) = max 55.3, 51.2 = 55.3 d1(3) = REPLACE

Summarizing the results in tableau form:

u 1 2 3 4 5 6 Stage 5 f5(u) 80 60 50 30 d5(u) SELL SELL SELL SELL Stage 4 f4(u) 79.8 67.3 49.8 4.8 d4(u) REPLACE KEEP REPLACE REPLACE Stage 3 f3(u) 85.7 67.1 17 d3(u) KEEP KEEP KEEP Stage 2 f2(u) 85.5 35.5 d2(u) REPLACE,

KEEP REPLACE

Stage 1 f1(u) 55.3 d1(u) REPLACE

Step 4 : Composition: determine the optimal policy by working forward In conclusion, the maximal total profit that can be obtained is $55 300 and the corresponding optimal policy is (i) REPLACE – REPLACE – KEEP – KEEP – SELL or (ii) REPLACE – KEEP – KEEP – REPLACE – SELL Remarks: - The state variables can take multi-dimension, or continuous form. But the increase in

the dimension of the state variable increases the computations at each stage.


T2, 2013/14 of 59

5.0 Combinatorial Optimization Ø Combinatorial optimization combines techniques from combinatorial, linear

programming and the theory of algorithms to solve optimization problems over discrete structure.

Ø Combinatorial problem can be modeled in terms of graph or digraphs and be solved

by the algorithms in graph theory. Definition: Graph & Digraph (a) A graph G = (V, E) consists of a finite set V of points, called vertices/nodes, and a

set E of connecting lines, called edges/links/branches, such that each edge connects two vertices.

(b) A digraph is a graph in which each edge has a direction (indicated by an arrowhead).

Example: A graph A digraph

G = (V, E) G = (V, E) V = 1,2,3,4 V = 1,2,3,4 E = e1, e2, e3, e4 E = e12, e13, e32, e34

Ø Some examples of combinatorial problems:

(i) The traveling salesman problem (iii) The maximal network flow problem (ii) The shortest paths problem (iv) The matching problem

5.1 Minimal Spanning Trees An important subclass of graph is called tree. To properly describe what a tree is, we require additional terminology below: Definitions (i) A cycle is a path that begins and ends at the same vertex/node. (ii) A tree is a graph that contains no cycles. (iii) Suppose G is a graph with n vertices. A tree T is a spanning tree of graph G if T

connects all vertices of G with exactly n – 1 edges of G. (iv) A minimal spanning tree of G is a spanning tree of G with minimum total

length/weight of surviving edges.

1

2

3 4

e1

e2 e4

e3 1

2

3 4

e12

e13 e34

e32


T2, 2013/14 of 59

Example: Consider the following graph G:

Figure 1 to Figure 5 below are several sub-graphs than can be constructed from G:

1 24

4

13 5 7

3

4

5

6

7

8

8

6

9

3

2

12

5

6

6

12

1 24

4 3

4

5

6

7

8

6

3

2

5 6

Figure 5

1 24

4

13

5

6

7 6 2

6

12

1 24

4 3

4

5

6

6

3

5

6

Figure 1 Figure 2

Figure 3 Figure 4

1 24

4

7

3

5 7

8

6 2

5 6

1 24

4

13 7

3

5

6

7

8 3

2

6

6


T2, 2013/14 of 59

(a) Graphs in Figure 1 and Figure 2 above are not trees since we can construct a path which is a cycle. For example, (1,2,7,1) is a cycle for the graph in Figure 1. Similarly, (2,4,3,1,2) is again a cycle for the graph in Figure 2. However, graphs in Figure 3, Figure 4 and Figure 5 are trees as they do not contain paths that begin and end with the same node, i.e. no cycles.

(b) The tree in Figure 3 above is not a spanning tree of G as vertex 6 is not included, but

the trees in Figure 4 and Figure 5 are spanning trees of G since they consist of all vertices in G. We see here that a spanning tree of G is not necessarily unique .

(c) Figure 4 above gives the spanning tree of weight 41 and Figure 5 gives the spanning

tree of weight 30. It would be shown later on that Figure 5 is the minimal spanning tree of G.

- Minimal spanning trees are useful in constructing networks in computer science,

communication, distribution or transportation network. Two most popular algorithms for constructing the minimal spanning trees are Kruskal’s algorithm and Prim’s algorithm.

5.1.1 Kruskal’s Algorithm Input : number of vertices n, edges ( )j,i and their lengths lij

1. Arrange the edges in ascending order of lengths. 2. Beginning from the shortest edge, do: à Reject an edge if it forms a cycle, otherwise add it to T. à If n – 1 edges have been chosen, then STOP.

Output : minimal spanning tree T Example: Apply Kruskal’s algorithm to obtain the minimum spanning tree from the graph G in page 51: Solution First, we arrange the edges in ascending order of lengths.

Edge Length (1,5) 6 (2,7) 2 (3,8) 6 (4,6) 3 (7,8) 7 (1,2) 4 (4,5) 8 (2,3) 4 (3,7) 9 (1,3) 5 (1,7) 12 (3,4) 5 (4,8) 12 (1,4) 6 (5,6) 13


T2, 2013/14 of 59

Next, we add the edge one by one for each iteration according to the above sorted order. Reject the edge if it would form a cycle in the tree.

Iteration 1 Add (2,7) with length 2




Iteration 5 Reject edge (1,3) with length 5 since it forms a cycle. Iteration 6 Add (3,4) with length 5

Iteration 7 Reject edge (1,4) with length 6 since it forms a cycle. Iteration 8 Add (1,5) with length 6

Iteration 9 Add (3,8) with length 6 Since 7 edges have been added to the spanning tree T, iterations terminate.

5.1.2 Prim’s Algorithm - Kruskal’s algorithm does not necessarily gives a tree during each intermediate step.

Prim's algorithm does. Prim’s Algorithm Input : number of vertices n, edges ( )j,i and their lengths lij 1. Initialization: Initialize the tree with vertex 1 and set F = v1 NF = v2, v3,…, vn L = 0 L = nn lLlLlL 1133122 ,...,, === Note: If edge ( )j,1 does not exist, then let jl1 be ∞ .

1 2 4

4 3

4

6

7

3

2 1 2 4

4 6

7

3

2 2

4 6

7

3

2 2

7

2

1 2 4

4 3

4

5

6

7

8

6

3

2

5 6

1 2 4

4 3

4

5

6

7 6

3

2

5

1 2 4

4 3

4

6

7

3

2

5


T2, 2013/14 of 59

2. Addition of a vertex/edge to the tree: Choose a vertex in NF (say vk) corresponding to the shortest length in L (say kL ). Connect vk to the tree with its edge of length kL . Transfer vk to set F and kL to set L. 3. Update L : Update the elements in L for the new NF according to

( ) ,min oldkjjj lLL =

Repeat Step 2 and Step 3 until NF is empty. Output: F and and L, where F is the structure of the minimum spanning tree and L are the lengths of the surviving edges. Example: Apply Prim’s algorithm to obtain the minimum spanning tree from the graph G in page 51: Solution

Step 1: Initialization F = v1 NF = v2,v3,v4,v5,v6,v7,v8 L = 0 L = 4, 5, 6, 6, ∞ , 12, ∞

Step 2: 42 =L has the smallest length. Add vertex 2 to

the tree. Transfer v2 to F and 42 =L to L.

Step 3: Updating. ∞==∞==== 876543 ,2,,6,6,4 LLLLLL F = v1,v2 NF = v3,v4,v5,v6,v7,v8 L = 0, 4 L = 4, 6, 6, ∞ , 2, ∞ Step 2:

27 =L has the smallest length. Add vertex 7 to the tree. Transfer v7 to F and 27 =L to L.

Step 3: Updating. 7,,6,6,4 86543 =∞==== LLLLL F = v1,v2,v7 NF = v3,v4,v5,v6,v8 L = 0, 4, 2 L = 4, 6, 6, ∞ , 7 Step 2:


1

1 2 4

3

4

7

2

1 2 4

1 2 4

7

2


T2, 2013/14 of 59

Step 3: Updating. 6,,6,5 8654 =∞=== LLLL F = v1,v2,v7,v3 NF = v4,v5,v6,v8 L = 0, 4, 2, 4 L = 5, 6, ∞ , 6 Step 2:


Step 3: Updating. 6,3,6 865 === LLL F = v1,v2,v7,v3, v4 NF = v5,v6,v8 L = 0, 4, 2, 4, 5 L = 6, 3, 6 Step 2:

36 =L has the smallest length. Add vertex 6.to the tree. Transfer v6 to F and 36 =L to L.

Step 3: Updating. 6,6 85 == LL F = v1,v2,v7,v3, v4,v6 NF = v5,v8 L = 0, 4, 2, 4, 5, 3 L = 6, 6 Step 2: Both 5L and 6L have the same length. Add vertex 5 to the tree. Transfer v5 to F and 65 =L to L.

Step 3: Updating. 68 =L F = v1,v2,v7,v3, v4,v6,v5 NF = v8 L = 0, 4, 2, 4, 5, 3, 6 L = 6 Step 2: Only one more vertex left in NF. Add vertex 8 to the tree. Transfer v8 to F and 68 =L to L. Iterations terminate as NF is empty. Output: F = v1,v2,v7,v3, v4,v6,v5,v8 L = 0, 4, 2, 4, 5, 3, 6, 6

1 2 4

3

4

7

2

5 4

1 2 4

4 3

4

5

6

7

8

6

3

2

5

6

1 2 4

4 3

4

6

7

3

2

5

1 2 4

4 3

4

5

6

7 6

3

2

5


T2, 2013/14 of 59

Exercise: Construct the minimal spanning tree from the following graph by using Kruskal’s and Prim’s algorithm.

5.2. Dijkstra’s Algorithm for Shortest Paths Ø Dijkstra’s Algorithm is used in finding the shortest paths from a given vertex, usually

denoted by 1 and called the origin, to all other vertices 2,3,…,n in a given graph, G. Theorem: Bellman’s optimality principle: If P:1 → j is a shortest path from 1 to j in G and eij is the last edge of P, then Pi:1 → i (obtained by dropping eij from P) is a shortest path 1 → i.

Ø Based on Bellman’s principle, the following two Bellman equations can be derived: Let Lj denote the length of a shortest path 1 → j in G and lij be the length of the edge eij.

L1 = 0

Ø Let F be the set of vertices whose shortest paths have been found, and NF be the set

of vertices that whose shortest paths have not been found. Each vertex in F associates with Lv, the shortest path 1 → v, and each vertex in NF associates withLv, the minimal path 1 → v found so far. At each iteration, the shortest path of one of the vertices from NF would be determined, and that vertex would then be deleted from NF and added to F. This process is continued until NF is empty.

9

3

2

4

3 17

4 2

8

1

5 5

njlLL ijijij ,...,2)(min =+=≠

1

2 i

j

eij

… …

P: 1 → i P: 1 → j


T2, 2013/14 of 59

Dijkstra’s Algorithm Input : number of vertices n, edges eij and edge lengths lij 1. Initialization:

Set F = v1 NF = v2, v3,…, vn L = 0 L = nn lLlLlL 1133122 ,...,, === Note: If edge je1 does not exist, then let jl1 be ∞ .

2. Finalize the minimum distance to a vertex: Choose a vertex in NF (say vk) corresponding to the shortest length in L (say kL ). Transfer vk to set F and kL to set L. Note: After the transfer, kL is now denoted as kL .

3. Update L : Update the elements in L for the new NF according to

( ) ,min oldkjkjj lLLL +=

Repeat Step 2 and Step 3 until NF is empty. Output : F and L, where L contains the lengths of shortest paths from vertex 1 to the corresponding vertices in F. Example: Find the shortest path from vertex 1 to vertices 2,3,4 by using the Dijkstra’s algorithm.

Step 1. Initialization F = v1 NF = v2,v3,v4,v5 L = 0 L = 2, 6, 8, ∞

Step 2.L2 = 2 is minimum. Transfer v2 to F and L2 = 2 to L.

Step 3. Update L L3 = min6, 2L + l23 = min 6, 2 + 5 = 6 L4 = min8, 2L + l24 = min 8, 2 + 3 = 5 L5 = min∞, 2L + l25 = min ∞, 2 + ∞ = ∞

F = v1,v2 NF = v3,v4,v5 L = 0, 2 L = 6, 5, ∞


5 4

3

2

5

5

20

8

2

1 6

3


T2, 2013/14 of 59

Step 3. Update L L3 = min6, 4L + l43= min 6, 5+5 = 6 L5 = min∞, 4L + l45 =min ∞, 5 + 20 = 25

F = v1,v2,v4 NF = v3,v5 L = 0, 2, 5 L = 6, 25


Step 3. Update L L5 = min25, 3L + l35= min 25, 6+∞ = 25

F = v1,v2,v4,v3 NF = v5 L = 0, 2, 5, 6 L = 25

Step 2. Only one more vertex left in NF. Transfer v5 to PL and L5 = 25 to L.

Iterations terminate since NF is empty. Output : PL = v1,v2,v4,v3,v5 L = 0, 2, 5, 6, 25 The shortest path from vertex 1 to vertex 2 is 2 units in length. The shortest path from vertex 1 to vertex 4 is 5 units in length. The shortest path from vertex 1 to vertex 3 is 6 units in length. The shortest path from vertex 1 to vertex 5 is 25 units in length.

Exercise: Find the shortest paths from vertex 1 to all others vertices.

[ L2 = 7, L3= 3, L4=5, L5=5, L6=7]

3 5

4

3

2 8

4 4 1 6

3

2

1

5

2

2


T2, 2013/14 of 59

Operations Research

Linear Programming

Solved by: (1) Graphical Method (2) Simplex method

Duality and Sensitivity Analysis

Dynamic Programming

Solved by : Dynamic Programming with Backward Recursion

Combinatorial Problems

Dijkstra’s Algorithm for Shortest Paths

Kruskal’s Algorithm for minimal spanning tree

Prim’s Algorithm for minimal spanning tree

EEM2046_-_1314_T2_-_OR_Lecture_Notes Operation Research notes

Documents

Transcript of EEM2046_-_1314_T2_-_OR_Lecture_Notes Operation Research notes