What is a Hyper-heuristic?quarter.cs.stir.ac.uk/courses/ITNPBD8/lectures/1...Linear Combinatorial...
Transcript of What is a Hyper-heuristic?quarter.cs.stir.ac.uk/courses/ITNPBD8/lectures/1...Linear Combinatorial...
1/16/2017
1
Evolutionary and Heuristic Optimisation (ITNPBD8) Lecture 1: Introduction
Gabriela Ochoa http://www.cs.stir.ac.uk/~goc/
Computing Science and Mathematics,
School of Natural Sciences
University of Stirling, Stirling, Scotland
Content
• Welcome and introductions
• Outline of the module
• Evaluation
• Resources
2
1/16/2017
2
Outline of the module
1. Optimisation problems – Optimisation & search
– Classic mathematical models
– Example problems: Knapsack, Travelling Salesperson, Timetabling
2. Optimisation methods – Heuristics and metaheuristcis
– Single point algorithms
– Population-based algorithms
3. Advanced topics – Fitness landscape analysis
– Multi-objective optimisation
3
Outline of the Module
4
1/16/2017
3
Evaluation
• 10%: Checkpoints in the Lab – Python programming exercises, programming
algorithms from scratch, but part of code will be given – Based on a simple problem: knapsack
• 40%: Assignment – Select a paper applying metaheuristics to a real-world problem. List will be provided. – 15%: Slides in Powerpoint with fixed template – 15% : Summary 1 page in Word – 10%: Oral presentation
• 50%: Exam
5
Resources: Article & Online Book
• Online Book
– Essentials of Metaheuristics, Sean Luke, Department of Computer Science George Mason University
• Survey Article
– Metaheuristics in combinatorial optimization: Overview and conceptual comparison, C Blum, A Roli ACM Computing Surveys (CSUR) 35 (3), 268-308
6
1/16/2017
4
Resources: Books • Burke , E. K; Kendall, G., (Eds.) (2005, 2015) Search
Methodologies: Introductory Tutorials in Optimization and Decision Support Techniques, Springer.
• A.E. Eiben and J.E. Smith (2003,2015), Introduction to Evolutionary Computing, Springer,
• Hoos, H; Stutzle, T. (2004) Stochastic Local Search Foundations and Applications, Elsevier.
• Z. Michalewicz, D.B. Fogel (2000) How to Solve It: Modern Heuristics, Springer.
• Talbi, E-G (2009), Metaheuristics: From Design to Implementation, Wiley.
7
Borg Vogons
Biotop
Art
Life Sciences Social Sciences
Mathematics Physics
Software Engineering
Neural Nets Evolutionary Computing Fuzzy Systems
Computational Intelligence etc
Computer Science etc
Exact Sciences etc
Science Politics Sports etc
Society Stones & Seas etc
Earth etc
Universe
You are here
Positioning of Evolutionary and Heuristic Optimisation
A.E. Eiben and J.E. Smith (2003,2015), Introduction to Evolutionary Computing, Springer
1/16/2017
5
OPTIMISATION PROBLEMS What is Optimisation? What is an Optimisation Problem?
9
Optimisation problems • Wide variety of applications across industry,
commerce, science and government • Optimisation occurs in the minimisation of time,
cost and risk, or the maximisation of profit, quality, and efficiency
• Examples – Finding shortest round trips in graphs (TSP) – Finding models of propositional formulae (SAT) – Science: Determining the 3D-structure of proteins – Operational research, Industry: Planning, scheduling, cutting &
packing, logistics, transportation, communications, timetabling, resource allocation, genome sequencing
– Software engineering: test case minimisation and prioritisation, requirements analysis, code design and repair, etc.
10
1/16/2017
6
Optimisation problems are everywhere!
Logistics, transportation, supply change management
Manufacturing, production lines Timetabling
Cutting & packing Computer networks and Telecommunications
Software - SBSE
11
Optimisation problems General constrained optimisation problem:
Optimisation through search
Iteratively generate and evaluate
candidate solutions.
• Systematic search
• (Stochastic) local search
Search Space: set of candidate solutions. All possible combinations of the decision variables.
Min/Max f( )
x
Subject to:
0)( xgi
0)( xhj
i = 1,…,p
j = 1,…,n
12
1/16/2017
7
Search in Computing Science
1. Search for stored data • Finding information stored in disc or
memory. • Examples: Sequential search, Binary
search
2. Search for web documents • Finding information on the world wide
web • Results are presented as a list of results
3. Search for paths or routes • Finding a set of actions that will bring us
from an initial stat to a goal stat • Relevant to AI • Examples: depth first search, breath first
search, branch and bound, A*, Monte Carlo tree search.
4. Search for solutions • Find a solution in a large space of
candidate solutions • Relevant to AI, Optimisation, OR • Examples: evolutionary algorithms, Tabu
search, simulated annealing, ant colony optimisation, etc.
At least 4 meanings of the word search in CS
13
Search and optimisation in practice
Real-world problem
Model
Solution
Formulation
Algorithm
Problem Model • Problem representation • Constraints • An objective function
Solution to the Model • Feasible candidate solution • Lead to the optimal (or good enough) value of the objective function
Optimisation/search Algorithm • Exact methods • Approximate (heuristic) methods
Many challenging applications in science and industry can be formulated as optimisation problems!
14
1/16/2017
8
Optimisation problems: two categories
Continuous
• Continuous variables
• Looking for a set (vector) of real numbers [45.78, 8.91, 3.36]
• Objective function has a mathematical expression
• Special cases studied in mathematics and OR: Convex, Linear
Combinatorial
• Discrete variables
• Looking for an object from a finite set – Binary digits [1011101010]
– Integer [1, 53, 4, 67, 39]
– Permutation [3,5,1,2,4]
– Graph
• Generally have quite different flavours and methods for solving them • Have become divergent
15
Classic mathematical models
Linear Programs (LP)
• Linear optimisation
• A single objective
• The objective and constraints are linear
• Decision variables, allowed to have any values
• Easy to solve numerically (simplex method)
Importance
• Many applications
• See Wolfram page
Integer Programs (IP)
• LP in which some or all variables are constrained to take on integer values
• Harder to solve. Software packages: Excel, LINGO/LINDO and MPL/CPLEX,
Importance
• problems in which variables required to be integer
• many decisions are essentially discrete (yes/no, go/no-go)
16
1/16/2017
9
Integer program: canonical form maximise c1x1+c2x2+…+cnxn (objective function)
subject to
a11x1+a12x2+…+a1nxn b1 (functional constraints)
a21x1+a22x2+…+a2nxn b2
….
am1x1+am2x2+…+amnxn bm x1, x2 , …, xn Z+ (set constraints)
In vector form: maximise cx (objective function) subject to Ax b (functional constraints) x Zn
+ (set constraints)
17
The knapsack problem • Given a knapsack of capacity W, and a number n of items, each
with a weight and value. The objective is to maximise the total value of the items in the knapsack
Maximise Subject to
18
Examples of Applications • Resource allocation • Cutting and packing problems • Requirements engineering
1/16/2017
10
The knapsack problem • Formulation. Mathematical model
maximise
4x1+2x2+x3+10x4 +2x5
subject to
12x1+2x2+x3+4x4+x5 15
x1,x2,x3,x4,x5 {0, 1}
Xi = 1 If we select item i 0 Otherwise
• Can be formulated as an Integer Programming problem, and solved efficiently using Dynamic Programming
• Binary representation [11010], using heuristic methods
• Search space size = 2n
• n = 100, 2100 ≈ 1030
19
Real-world example of the Knapsack problem
• Consider a cargo company, that has an airplane and need to carry packages.
• Customers state the weight of the cargo item they would like delivered, and the amount they are prepared to pay.
• The airline is constrained by the total amount of weight the plane is allowed to carry.
• The company must choose a subset of the packages (bids) to carry in order to make the maximum possible profit, given the weight limit that they must respect. 20
1/16/2017
11
Travelling salesman problem (TSP)
• A salesman must visit number of cities minimising the total cost of traveling
• Most prominent classical example of an NP-complete combinatorial optimisation problem
• BBC 4 Documentary (min 28:00 – 33:45 TSP): The Secret Rules of Modern Living: Algorithms
• Configurations: permutation (ordering) of cities. – s1= (A B C D), f(s1)= 20+30+12+35= 97
– s2= (A B D C), f(s2)= 20+34+12+42=108
– s3= (A C B D), f(s3)= 42+30+34+35= 141
• Local change (move): swap two cities – s1 = (A B C D)
– s1’ = (A D C B)
Gabriela Ochoa, [email protected] 22
Check: Permutation Generator
1/16/2017
12
ACADEMIC TIMETABLING PROBLEMS
Gabriela Ochoa, [email protected]
Academic timetabling problems
• assigns teachers to a set of specific subjects or topics. Faculty timetabling Faculty timetabling
• assigns subjects to a fixed and specific group of students.
Class-teacher timetabling
Class-teacher timetabling
• ensures that every teacher-subject pair has an available classroom.
Classroom-assignment Classroom-assignment
• assigns one-time events like final exams or special lectures to individual students.
Examination timetabling Examination timetabling
• assigns subjects to individual students Course timetabling Course timetabling
Gabriela Ochoa, [email protected]
1/16/2017
13
The course timetabling problem
• Weekly scheduling of lectures for several university courses • Given number of rooms and time periods • Conflicts between courses are set according to the curricula
published by the University • Components
– Days, Timeslots, and Periods. Periods = the product of the days times the day timeslots.
– Courses and Teachers. Fixed number of lectures on distinct periods (spread). Attended by a number students, taught by a teacher.
– Rooms. With a capacity – Curricula. Group of courses with common students
Gabriela Ochoa, [email protected]
Course timetabling test instances, ITC- 2007
Hard constraints
• Students cannot attend more than one course at a time
• Rooms should be big enough to hold the students
• One course per room, per timeslot
Soft constraints
Avoid as much as possible to:
• Assign students to a course in the last timeslot of the day
• Assign students to 3 or more timeslots in a row
• Assign students to only one event in a particular day
Gabriela Ochoa, [email protected]
• Real-world data from the University of Udine, Italy (24) • For all instances there exists at least one feasible solution (no hard
constraint violations), but it is not known which is the optimal value for the soft constraints.
1/16/2017
14
Course timetabling problem
• E = {e1, e2,…, en} Events (courses
of subjects) Events (courses
of subjects)
• T = {t1, t2 …, ts} Time periods Time periods
• P = {p1, p2 …, ps} Places
(classrooms) Places
(classrooms)
• A = {a1, a2 …, as} Students Students
• quadruple (e,t,p,S) S subset A Assignment Assignment
• complete set of n assignments, that satisfies the constraints
Timetabling solution
Timetabling solution
Gabriela Ochoa, [email protected]
Representation: set of integers representing indexes
Objective function:
Assigns subjects to individual students
Summary of optimisation problems
Real-world problem
Model
Solution
Formulation
Algorithm
Problem Model • Problem representation • Constraints • A fitness function
Solution to the Model • Feasible candidate solution • Lead to the optimal (or good enough) value of the objective function
Optimisation/search Algorithm • Exact methods • Approximate (heuristic) methods
Many challenging applications in science and industry can be formulated as optimisation problems!
28