Post on 11-Jan-2016
1
Teaching a Traditional Optimization Course in a non-Traditional Manner
James B. OrlinMike Metzger
MIT
November 6, 2006
2
Part I: Introduction
Quote of the day.
If you treat individuals as they are they will remain as they are, but if you treat them as if they were what they ought to be and could be, they will become what they ought to be and could be. – Johann Wolfgang von Goethe.
3
Optimization Methods in Management Science
Models, applications, and algorithms for optimization within Management Science. – finance, manufacturing,
transportation, timetabling, games, operations, e-business, and more.
Linear programming, integer programming, network optimization, and decision analysis. Mike Trick
I’ve scheduled league gamesfor a major basketball
conference. There are an incredible number ofconstraints, and it is very hard to solve.
4
Topic
Introduction
Applications of Linear and Non-Linear Programming
Geometry of LP
Simplex Method 1
Simplex Method 2
Sensitivity Analysis
LP Duality 1
LP Duality 2
Game Theory
Introduction to Networks
Maximum Flows.
Min Cost Flows
The Network Simplex Algorithm
IP Models
More IP Models
Branch and Bound
Cutting Planes
Decision Trees 1
Decision Trees 2.
Syllabus
5
Our Goals Today
Lots of pedagogical advances over the years.– lot of description at INFORMS meetings
Focus on a range of techniques– visual learning, – active learning– interactive learning– use of technology– and more
End result: – students like the material better– more interested in follow on subjects– superior mastery of the material over students in
previous years
6
Overview of the Structure of the Class
2 Lectures per week per section (1.5 hours, required)– 1 small morning section, one large afternoon
section– MIT students are used to 1 hour per lecture
Required recitation – weekly quizzes
Nearly weekly homework
2 midterms and a final (and a project)
1 professor, 3 TAs. 150 students
7
Structure of a Lecture
Intro– quotes
– announcements
First half of class
Mental break
Second half of class
game to review the class material
8
Time line for 15.053
1995: Algorithm based
1996: taught 15.053 as an
Excel-based class
1999: taught 15.053 as an
algorithms based class
2001: introduced
PowerPoint into classes;
Mike Metzger was a
student in the class
9
Time line continued
2004: taught both sections of 15.053 in the spring. Mike Metzger was one of my TAs. Hired undergrad graders. TAs helped develop educational materials
2005: introduced required recitations.
2007. dropped required text and replaced with our own notes
CLASS
15.053 class notes
10
Part II: Pedagogy
Imagination is more important than knowledge...• Albert Einstein
You don't understand anything until you learn it more than one way.
• Marvin Minsky
11
Overview of Pedagogy
Pedagogy has evolved from straight lecturing
Techniques adopted– Engaging Model
Formulations
– Algorithm animation using PowerPoint
– Use of games in class
– Active learning (talk to your neighbor)
The Jetson’s diet problem.
Who wants a piece ofpiece of candy?
TV’s hit quiz show comes to 15.053
12
A diet problem and its dual
Jane Jetson is trying to design a diet for George on her home food dispenser so as to satisfy nutritional requirements and minimize cost.
Space Burgers
Saturn Shakes
Venus Flies
Requirements
Carbs 1 4 0 9
Fat 4 2 0 8
Protein 4 1 2 12
Cost (in $100s)
6 6 2
Theme music
13
Spacely Sprockets has decided to make food pellets. Mr. Spacely wants to win over Jane Jetson by making, carbohydrate, fat, and protein pellets. How should he price them to win over Jane and maximize profit?
Upper bound
Carbs Fat Protein on Cost
Space Burgers 1 4 4 6
Saturn Shakes 4 2 1 5.5
Venus Flies 0 0 2 2
Choose Prices
14
Discuss with partners
Usual:– left of the class solves Jane’s primal problem as
best as possible by inspection
– right half of the class solves Spacely’s dual problem by inspection
– we collect answers from both, and observe that the primal opt is >= dual opt.
Do you have any other ways of getting students to understand the intuition behind weak duality?– (2 minutes)
15
Algorithm Animation
Powerpoint has lots of animation tools that are useful for illustrating concepts.
Boxes can appear, and disappear.
Colors and transitions can simulate movement.
And one can use real movement.
And pictures can help.
16
2 Variable Simplex Illustrated
1 2 3 4 5 6
1
2
3
4
5
K
SStart at any feasible corner point.
Move to an adjacent corner point with better objective value.
Continue until no adjacent corner point has a better objective value.
Maximize z = 3 K + 5 S
17
An Alternative Representation
1 2 3 4 5 6
1
2
3
4
5
K
S Introducing:
LPac Man
18
Algorithm Animation: Simplex Pivoting
2 001
3 1 0
2 0 1
0
0
3 1 0
2 0 1
0
0
-2 001
=
= 2
6
x1 x2 x4x3z
= 0
1 0
0 1
00
x4x3
1
0
0
z
-3
-3
-4
-3
-4
3
x1
-3
-4
3
z = 0, x1 = 0, x2 = 0, x3 = 6, x4 = 2.
Non-basic variable x2 becomes basic.
Basic variable x4 becomes non-basic. It was the basic variable in Constraint 2.
Choose column 2.
Choose constraint 2.Pivot on the 2.
2
19
2
x1
-3 3 1 0
-4 0 1
Pivoting to obtain a better solution
=
= 2
6
2
x2 x4x3
3 -2 00
-z
0
0
1 = 0
-2 1 0 .5 1
-1 0 0 1 2
3 0 1 -1.5 3
New Solution: basic variables z, x2 and x3. Nonbasics: x1 and x4.
1
0
0
z
20
What is the maximum number of diamonds that can be packed on a Chinese checkerboard.
Each diamond covers 4 dots.
class exercise: how many can you pack?
21
Here is a best possible
22
23 4
1
5
76
89
10
11
12
For each circle, you can select at most one diamond incident to the circle.
5 slides later
23
An Excellent Integer Program
IP
( )
z = Max
s.t. 1 for each circle
{0,1} for all
dd D
dd S i
d
x
x i
x d D
Let S(i) be the diamonds that are incident with circle i of the Chinese Checkerboard.
zLP = 27.5
And we found a solution by hand with zi = 27
24
Representing the Dual Problem
The dual linear program: choose weights for each circle so that w(d) ≥ 1 for each diamond d. Minimize the weights of the circles.
Assign each circle (or node) a “weight.”
The weight of a diamond d is w(d) = the sum of its node weights.
Class exercise: assigning ¼ to each circle gives a total weight of 30.25. Can you do better?
25
Node Weight
1
1/2
1/3
1/6
0
Each diamond has weight at least 1.
18
1
24
6
27.5jjw
The number of diamonds is at most 27.
26
Who wants a piece of
CANDY?
TV’s hit quiz show comes to 15.053
2727
Who wants another piece of candy
50-50 15.053
A B
C D
What is the advantage of using algorithm animation?
It keeps students engaged
It helps students focuson what is important
It is effective for students who are visual thinkers
A, B, and C.
2828
Who wants another piece of candy
50-50 15.053
A B
C D
What is the advantage of using algorithm animation?
It is effective for students who are visual thinkers
A, B, and C.
2929
Who wants another piece of candy
50-50 15.053
A B
C D
What is the advantage of using algorithm animation?
Next question
A, B, and C.
3030
Who wants another piece of candy
50-50 15.053
A B
C D
What is the advantage of using algorithm animation?
Next question
A, B, and C.
3131
Who wants another piece of candy
50-50 15.053
A B
C D
What is the advantage of using algorithm animation?
Both A and B Neither A nor B
3 6
1 42iji j
x
3 9
1 42iji j
x
Which valid constraint should be added to the integer programming formulation of the TSP to eliminate this solution.
1
2 3
4
5 6
7
8 9
3232
Who wants another piece of candy
50-50 15.053
A B
C D
What is the advantage of using algorithm animation?
Which valid constraint should be added to the integer programming formulation of the TSP to eliminate this solution.
1
2 3
4
5 6
7
8 9
3 9
1 42iji j
x
Both A and B
3333
Who wants another piece of candy
50-50 15.053
A B
C D
What is the advantage of using algorithm animation?
Last Slide
3 9
1 42iji j
x
Which valid constraint should be added to the integer programming formulation of the TSP to eliminate this solution.
1
2 3
4
5 6
7
8 9
3434
Who wants another piece of candy
50-50 15.053
A B
C D
What is the advantage of using algorithm animation?
Last Slide
3 9
1 42iji j
x
Which valid constraint should be added to the integer programming formulation of the TSP to eliminate this solution.
1
2 3
4
5 6
7
8 9
35
Mental Break: Impossibilities
Fooling around with alternating current is a waste of time. Nobody will use it, ever.
Thomas Edison
Rail travel at high speed is not possible because passengers, unable to breathe, would die of asphyxia.
Dr. Dionysus Lardner, 1793-1859
Inventions have long since reached their limit, and I see no hope for future improvements.
Julius Frontenus, 10 A. D.
36
Recitations, Tutorials, Homework, Exams, Etc.The role of the Head TA
37
Structure
Quiz Statistical Break Lesson Summary
38
39
Quiz Problem 1 (4 points total; 2 points each) If exactly one of these points is optimal, which
point(s) could it be? If point B is optimal, must any other of these
point(s) be optimal too? If so, which one(s)?
0 0.5 1 1.5 2 2.5 3 3.5 4 4.50
0.5
1
1.5
2
2.5
3
3.5
4
4.5
A
B
C
D
E
F
G
x1
x 2
40
Statistical Break
Break Between Quiz and Lesson– It takes 5 minutes for them refocus
• Interesting statistics!
Examples:– Favorite Airline
– Exam Scores
– Have You Heard of Welcome Back Kotter
41
Quiz Grades
13
18
65
00
2
4
6
8
10
12
14
16
18
20
10 9 8 7 <7
42
Have you heard of Welcome Back Kotter?
No, 38
Yes, 10
No. 90
Yes. 10
Have you heard of Welcome Back Kotter
43
Recitation-”The Dual” Consider the Following Statements Try to Determine if they Are True or False
– It is possible for an LP to have exactly 2 optimal solutions
– Two different bases (and thus two different tableaus) can have the same corresponding basic feasible solution
– We can have negative z-row coefficients and still be at an optimal solution.
44
Problem 1: More Simplex Tableau Suppose while solving a maximization problem we obtain the following tableau, with the basic variables highlighted in blue:
z x1 x2 x3 x4 x5 rhs 1 a c 0 3 0 10 0 0 d 0 4 1 5 0 1 -5 0 -1 0 b 0 0 e 1 2 0 3
Give conditions on the missing values a, b, c, d, e and required to make the each of the following statements true:
a) The current tableau represents a basic feasible solution in canonical form. b) The current basic solution is an optimal solution, but the set of optimal solutions
is unbounded. c) The current basic solution is a degenerate basic feasible solution. d) The current basic solution is feasible, but the objective function value can be
improved by bringing x2 into the basis and pivoting x3 out.
45
Pivot- Geometry
46
Applications
(Before)
Ebel Mining Company owns two different mines that produce a given kind of ore. After crushing, ore is one of three grades: high, medium or low. Ebel must provide the parent company with at least 12 tons of high grade, 8 tons of medium grade, and 24 tons of low-grade ore per week. It costs $20,000 a day to run the first mine and $16,000 a day to run the second mine. In one day each mine produces the following tonnage:
High Medium Low Mine 1 6 2 4 Mine 2 2 2 12
47
ApplicationsAfter.
Dear 15.053 Class, My name is Jessica Simpson. I have been going through some tough times recently and am having a real problem with one of my cosmetic lines. The info for the line is on the next page. Recently though costs are changing based on market demand in addition to highly fluctuating resource costs. My problem is this we currently have an LP that we solve to find the optimal amount to produce of each product. However, every time a parameter changes, I am always forced to resolve the LP and this takes too long. I was hoping you guys could find a better way. Lately I have just been out of it. For example, Nick and I decided to split our Hummer in half, and now I need to buy a new one. Ohh yea, about the LP it seems to have been misplaced when I was moving out of my Malibu house. Please Help!
- Jessica
48
Interaction
Video clip More about video and educational technology
later
49
Initial Concerns
Students bored in recitation Rebellion to required recitation
– Against MIT Grain
50
What We Found
Main Lesson– By putting out a good product, students didn’t
complain.
– If they are learning, they will be happy
– The Fun Factor helped out
– You can’t please everyone• But you can please 95%
51
Office Hours
Structure– Twice per week for two hours each
– Attendance can be as high as 60 Students
– One TA leads each session• Team teaching is a new approach
– Small vs. large session
– Goal: To help students understand the material on the homework sets
– Management style
52
ExampleAccording to recent reports in US Weekly, after
12 consecutive nights of partying, Jessica has
decided to leave Nick. Currently they are trying to
determine how to optimally divide some of their
possessions. They hire “you” to try and help
them accomplish this task. To start, you ask them
to assign “value points” to all of the items under
consideration such that they total 100. Their
allocation is shown in the table below.
53
The Use of Excel
The main question– How to balance Excel Vs. Theory
Students:– Have disliked Excel
– However, after completing our course often say it is the most useful tool we taught.
54
The Use of Excel
Staff– Hard to grade Excel
– Very important for Management students A new approach
– Take Excel and separate from problem sets and Use as a case study
– To be tried in spring 2007
55
Our Idea
The Diet Problem as an ongoing example and case study– Formulate your problem
– Solve using simplex
– Take dual and interpret as Grocer’s Problem
– Study distribution costs using networks
– Look into decision modeling with stochastic supply
56
Modeling and Abstract modeling
More emphasis on modeling abstract scenarios
Building managerial skills while building on theory
Framing the problem
Min cij xij
s.t. aij xj <= bi for all i
x >= 0
57
Exams-Testing the “Why” not “How”
JB was performing the simplex method; however, at exactly one intermediary step, he made the following error: instead of pivoting in a variable with a negative objective row coefficient, he instead pivoted in a variable a with a positive objective row coefficient. He chooses the exiting variable using the usual minimum ratio test. Assume after this initial error JB continues to solve the problem using the simplex method and does not make any additional errors. Also assume that no basis is degenerate.
58
Testing the “why” not “how”
It is possible that the pivot that JB took will result in a basic solution that is not feasible.
It is possible that the pivot JB took will result in an optimal basis.
It is possible that there are fewer simplex pivots with the error than if the error had not been made.
59
Exam Management
Testing depth of knowledge without penalizing
students who understand the basics
Our Solution: Short Answer Questions
– Say we have an LP with only one variable.
Which of the following could be the set of all
feasible solutions?
• i) A single point, e.g. 2
• ii) A line segment, e.g. [2, 4]
• iii) Two disjoint line segments, e.g, [2, 3] [6, 7].
• iv) A ray or infinite half line, e.g. [2, ∞)
60
Suppose we have a primal-dual pair where
each has a finite optimal solution. Say we
remove the ith constraint for the primal.
What can you say about the dual
corresponding to this new problem?
– i)The dual could be infeasible.
– ii)The dual could still have an optimum solution.
– iii)The dual could be unbounded.
61
Use of Technology
Three recent additions to the subject
– Online PowerPoint reviews
– Homework solution podcasts
– Online videos of recitations and reviews
62
Online PowerPoint Reviews
An alternative to standard textbook readings– Solution: Online Tutorials
Topics:– Abstraction of LP models– Solving systems of equations– Converting an LP to standard form– Degeneracy– Sensitivity analysis using Excel– Integer programming formulations
63
With the Assistance of
Some Friends
64
Degeneracy in Linear Programming
Tim, the turkey
I heard that today’s tutorial is all about Ellen DeGeneres
Ellen DeGeneres
Sorry, Tim. But the topic is just as interesting. It’s about degeneracy in Linear Programming.
Degeneracy? Students at MIT shouldn’t learn about degeneracy. And I heard that 15.053 students have already studied convicts’ sex.
Reverend Jerry Falwell
Actually, they studied convex sets.
65
What is degeneracy? As you know, the simplex algorithm starts at a corner point and moves
to an adjacent corner point by increasing the value of a non-basic variable xs with a negative value in the z-row (objective function).
Typically, the entering variable xs does increase in value, and the objective value z improves. But it is possible that that xs does not increase at all. It will happen when one of the RHS coefficients is 0.
In this case, the objective value and solution does not change, but there is an exiting variable. This situation is called degeneracy.
x1
-3 3 1 0
-4 2 0 1
=
= 2
6
x2 x4x3
3 -2 00
z
0
0
1 = 2
1
00
A basic feasible solution is called degenerate if one of its RHS coefficients (excluding the objective value) is 0.This bfs is degenerate.
66
Nooz, the most trusted name in fox.
Great. I now know what degeneracy is. Now we can move on to other matters.
Ollie, the computationally
wise owl.
No, Nooz**. This tutorial has many more slides. Degeneracy adds complications to the simplex algorithm. And if you understand what occurs under degeneracy, you really understand what is going on with the simplex algorithm.
** As you know, “No, Nooz” is good news.”
Cleaver
Professor Orlin apologizes for this bad pun, but feels that he could not resist the temptation.
67
PodCast’s
MIT Project Launched in Spring 2006– 15.053 selected as launch course for homework
solutions via podcasts
– Over 85% reported using videos on regular basis
– Turned HW Solutions into tutorials
– Review Session 2 Set the current MIT PodCast download record in a single night
• Crashed Server!
68
PodCast’s
Show Example