Welcome to the webinar - Gurobi · Welcome to the webinar Migrating models from other solvers to...
50
Welcome to the webinar Migrating models from other solvers to use the Gurobi solver
Transcript of Welcome to the webinar - Gurobi · Welcome to the webinar Migrating models from other solvers to...
Welcome to the webinarMigrating models from other solvers to use the Gurobi solver
Renan Garcia, Ph.D.
Optimization Support Engineer at Gurobi Optimization
} Ph.D. in Industrial and Systems Engineering, Georgia Tech} Expert in optimization modeling and software development} Over a decade of experience implementing decision support systems
© 2016 Gurobi Optimization
David Nehme, Ph.D.
Principal Consultant at Abremod
} 20 years of software development experience} 15 years implementing CPLEX and Gurobi models} Ph.D. in Operations Research for The University of Texas} Top StackOverflow answerer for both CPLEX and Gurobi tags
© 2016 Gurobi Optimization
Agenda
} Switching is easier than you may think
} Common migration scenarios
} Migrating from OPL
} Migrating from CPLEX Concert API
} Best practices, limitations and considerations
© 2016 Gurobi Optimization
© 2016 Gurobi Optimization
Switching is Easier Than You May Think
Key Migration Issues
} Building the model◦ How do I build my optimization model?◦ Do I build it one constraint at a time, or do I build an entire constraint
matrix?
} Setting solver parameters◦ What solver parameters do I change?◦ What effects are these changes intended to produce?◦ Am I looking for an optimal solution, or just a good feasible solution?
} Computing and extracting the solution◦ How do I extract the solution produced by the solver?
© 2016 Gurobi Optimization
Interfacing with Gurobi
} Command-Line Tool
} Full-featured Interactive Shell
} Matrix-oriented APIs◦ C, MATLAB, R
} Object-oriented APIs◦ C++, Java, .NET, Python
} Modeling systems◦ Commercial: AIMMS, AMPL, Frontline Solvers, GAMS, MPL, …◦ Free: CMPL, JuliaOpt, OSI, PuLP, PYOMO, SolverStudio, YALMIP, …
© 2016 Gurobi Optimization
Do-It-Yourself Resources on www.gurobi.com
} Switching guidance:https://www.gurobi.com/resources/switching-to-gurobi/switching-overview
} Documentation:https://www.gurobi.com/documentation/◦ Quick Start Guides◦ Reference Manual
� APIs� Attributes� Parameters� Tuning� …
◦ Example Tour� 22 functional examples
} Seminars and videos:https://www.gurobi.com/resources/seminars-and-videos/seminars-videos
© 2016 Gurobi Optimization
© 2016 Gurobi Optimization
Common Migration Scenarios
Scenario 1: Migrating with Model Files
} Example: performance testing
} Export model file with virtually no changes needed to existing code ◦ Gurobi supports several file formats (MPS, LP, …)
} Export guidance:https://www.gurobi.com/resources/switching-to-gurobi/exporting-mps-files-from-competing-solvers
} Use Gurobi Command-Line Tool to solve the model ◦ Usage: gurobi_cl [parameters] filename◦ Ex: gurobi_cl TimeLimit=3600 misc07.mps
} "Quick-and-dirty" approach◦ Limited ability to interact with solver (parameters only)◦ For more control, try Interactive Shell
© 2016 Gurobi Optimization
Gurobi Parameters
} Control algorithmic behavior◦ Defaults validated against large
internal test set of models
} Termination criteria◦ Ex. TimeLimit, SolutionLimit
} Tolerances◦ Ex. MIPGap, BarConvTol
} Simplex and barrier◦ Ex. Method, Crossover
} MIP◦ Ex. MIPFocus, ImproveStartTime
} MIP cuts◦ Ex. Cuts, GomoryPasses
} Tuning and distributed algorithms◦ Ex. TuneJobs, WorkerPool
} General◦ Ex. Presolve, OutputFlag
© 2016 Gurobi Optimization
Scenario 2: Using a Modeling System
} Example: model is written in AMPL
} Migrating is extremely easy for solver-independent systems
◦ Obtain license◦ Set solver to Gurobi◦ Convert parameter settings
◦ Ex: in AMPL model file, addoption solver gurobi_ampl;
option gurobi_options 'mipfocus 1';
} Need to migrate your existing model code for single-solver systems◦ OPL, Mosel, …
© 2016 Gurobi Optimization
Gurobi Python Environment
} High-level optimization modeling constructs embedded in Python programming language
} Combines expressiveness of a modeling language with the power and flexibility of a programming language◦ Bring "feel" of a modeling language to the Python interface
} Requires minimal programming skills to get started
} Support all solver and programming needs
} Several seminars on this topic:https://www.gurobi.com/resources/seminars-and-videos/seminars-videos
© 2016 Gurobi Optimization
Interactive Examples
© 2016 Gurobi Optimization
Scenario 3: Porting Matrix-Oriented Code
} Example: C program which calls CPLEX Callable Library, Xpress, …
} Gurobi's C API supports sparse matrix format◦ Standard format used by many solvers
� Simple arrays represent matrix coefficients and their index positions◦ Ex: GRBaddconstrs(), GRBaddvars()◦ Minimal changes required to existing code
} Gurobi also supports advanced features◦ Callbacks◦ Advanced simplex routines (querying tableau rows)◦ …
} Must consider some Gurobi-specific modeling features when porting existing code◦ Gurobi environments◦ Lazy updates◦ Attributes
© 2016 Gurobi Optimization
Gurobi Environments
} Models are built from an environment
} Parameters are set on an environment
} Models get their own copy of the environment ◦ Once a model is created, subsequent parameter changes in parent
environment are not reflected in model environment ◦ Use getEnv() functions to get the environment from model
} Setting parameters in C ◦ Ex: set time limit of 3600 seconds for parent environment
status = GRBsetdblparam(env, "TimeLimit", 3600); ◦ Ex: set presolve level to 2 for model's environment
status = GRBsetintparam(GRBgetenv(model), "Presolve", 2);
© 2016 Gurobi Optimization
Lazy Updates
} Gurobi updates models in batch mode◦ Model creation and updates are efficient
} Must call update() functions to use model elements ◦ Ex: Call after creating a variable before using it in a constraint ◦ May require changes to code for other solvers
} UpdateMode=1 parameter setting allows you to use elements immediately
© 2016 Gurobi Optimization
Attributes
} Unified system to access model elements◦ Attributes work the same across all Gurobi interfaces
} Access via a basic set of get/set functions◦ Attribute name is specified as a parameter◦ Replaces many functions used by other solvers
} Getting/Setting attributes in C◦ Use get/set functions by type (int, double, char, string)◦ Ex: query number of nonzeros in a model
status = GRBgetintattr(model, "NumNZs", &nzs);◦ Ex: query solution vector
double x[NUMVARS];status = GRBgetdblattrarray(model, "X", 0, NUMVARS, x);
◦ Ex: modify constraint RHS to 1status = GRBsetdblattrelement(model, "RHS", cidx, 1.0);
© 2016 Gurobi Optimization
Scenario 4: Porting Object-Oriented Code
} Example: Java program which uses CPLEX Concert Technology
} Gurobi's OO APIs represent models using objects◦ Objects for variables and constraints ◦ Methods are used to create model elements◦ Ex: add simple constraint x + y ≥ 1 in C++
c1 = model.addConstr(x + y >= 1, "c1");
} All Gurobi APIs are just thin layers on top of same native C code
} Must consider same Gurobi-specific modeling features when porting◦ Subsequent parameter changes in parent environment not reflected in
model environment � Java Ex: model.getEnv().set(GRB.IntParam.Presolve, 2);
◦ Must call model's update() method to use elements (unless UpdateMode=1)◦ Use get/set methods on objects to access attributes
� Python Ex: constr.setAttr('RHS', 1.0)
© 2016 Gurobi Optimization
Don't Panic, Your Code Often Looks Similar
} CPLEX Concert C++:IloEnv env; // create empty environmentIloModel model(env); // create empty model
IloNumVar x(env, 0, 10, ILOINT); // add variables
model.add(x);
// ...
model.add(x + 2*y <= 1); // add constraints// ...
} Gurobi C++:GRBEnv env; // create empty environment
env.set(GRB_IntParam_UpdateMode, 1); // enable automatic updates
GRBModel model(env); // create empty model
GRBVar x = model.addVar(0, 10, 0, GRB_INTEGER); // add variables
// ...model.addConstr(x + 2*y <= 1); // add constraints
// ...
© 2016 Gurobi Optimization
Solver-specific Guidance
© 2016 Gurobi Optimization
© 2016 Gurobi Optimization
} Cover two disparate examples◦ OPL ◦ ILOG Concert� Have a C#, and Java Adapters� Working on C++◦ Showing Two approaches� Translating OPL� Using an Adapter for Concert
} General advice on the migrating
Agenda
© 2016 Gurobi Optimization
Switching To Gurobi
} Hurdles◦ Large code base◦ Business Logic embedded in API calls
} Mitigating Factors◦ The time to migrate is not proportional to the size of the
codebase◦ With mps files, you can see how the Gurobi will perform on
your specific models.
© 2016 Gurobi Optimization
Before You Migrate
} Current State◦ Actively adding features?◦ Maintenance only?◦ Do you have regression tests?
} MPS is your friend◦ Evaluate relative performance
} LP is also your friend◦ Name your variables and constraints◦ Testing your code
© 2016 Gurobi Optimization
Migrating From OPL
} OPL includes a domain specific language
} Locked to a Solver (CPLEX)
} Two most likely strategies◦ Move to a solver-agnostic Language� AMPL� AIMMS, GAMS, MPL◦ Migrate to Python
© 2016 Gurobi Optimization
Migrating from OPL to Python
} Why Python
◦ Best of Both worlds� Powerful as a General Purpose Language� As effective as a Domain Specific Language
� Concise� Readable� Learnable
� See Stackoverflow� (http://bit.ly/1QEF0fq)
◦ Huge user base
© 2016 Gurobi Optimization
OPL to Python
} Feature Comparison
OPL PythonTuples, Sets Tuples, SetsRead from Excel openpyxl, xlrdRead from SQL SqlalchemySlicing, Grouping PandasUI Jupyter Notebook.dat format JSONOPLScript Python
© 2016 Gurobi Optimization
} Installing Python◦ Use the Anaconda Distribution� Especially great on Windows� Easy installation for Gurobi Libraries
� http://www.gurobi.com/downloads/get-anaconda
OPL To Python
© 2016 Gurobi Optimization
OPL To Python
} Migrate a complete example
} From a book ◦ Planning and Scheduling in Manufacturing and Services◦ OPL Code is freely available� http://bit.ly/20VpATz◦ Reads from OPL dat file
} Python Code Available on github
© 2016 Gurobi Optimization
} Translate the “.dat” file to JSON
◦ JSON is a standard� Almost all programming languages have readers� OPL reads it� Native format for MongoDB� Many application have JSON exporters
◦ Simple Python script to translate� Most dat files
� without embedded logic� Using PyParsing
� Another library included with Anaconda
Approach
© 2016 Gurobi Optimization
OPL Data
Demand = #[ 2: #[
1: [ 20000 30000 15000 40000 ] 2: [ 0 50000 30000 50000 ]
]#3: #[
1: [ 10000 5000 15000 40000 ] 2: [ 0 10000 0 5000 ] ]#
]#;
"Demand": {"2": {
"Product 1": [ 20000, 30000, 15000, 40000 ],"Product 2": [ 0, 50000, 30000, 50000 ]
},"3": {
"Product 1": [ 10000, 5000, 15000, 40000 ],"Product 2": [ 0, 10000, 0, 5000 ]
}
} Can Translate with Python Script
© 2016 Gurobi Optimization
OPL Model
} Data DeclarationsLang. StatementOPL int RequiredLotSize = ...;
Python RequiredLotSize = d['RequiredLotSize']
OPL float Demand[Stages, Products, Periods] = ...;Python Demand = series_from_json(d['Demand'],
[Stages, Products, Periods])OPL range Products = 1..2;Python Products = ['Product 1', 'Product 2']
© 2016 Gurobi Optimization
OPL Model
} Variable Declarations
dvar float+ x[Factories, Products, Periods];dvar int+ y[Factories, Stages, Products, Periods] in 0..maxint;dvar float+ z[Products, Periods];dvar float+ q2[Products, ZPeriods];dvar float+ v2[Products, Periods];dvar float+ v3[Products, ZPeriods];dvar boolean yb[bnds, Periods];
© 2016 Gurobi Optimization
VariablesOPL dvar float+ x[Factories, Products, Periods];Python x = get_vars('x', Factories, Products, Periods)OPL dvar boolean yb[bnds, Periods];Python yb = get_vars('yb', bnds.index, Periods,
vtype=GRB.BINARY)
© 2016 Gurobi Optimization
Objective function
} OPL} minimize} sum (t in Periods, j in Products, } i in Factories) } ProdCost[i,j]*x[i,j,t]
} Python} model.setObjective(} grb.quicksum([ProdCost[i, j] * x[i, j, t]} for i in Factories} for j in Products} for t in Periods])
© 2016 Gurobi Optimization
Constraints
OPL if (RequiredLotSize > 0)forall (i in Factories, j in Products, t in Periods)
x[i,j,t] == RequiredLotSize*xlots[i,j,t];else
forall (i in Factories, j in Products, t in Periods)xlots[i,j,t] == 0;
Python if RequiredLotSize > 0:[addConstr(x[i, j, t] == RequiredLotSize * xlots[i, j, t])for i in Factories for j in Products for t in Periods]
else:[addConstr(xlots[i, j, t] == 0)for i in Factories for j in Products for t in Periods]
© 2016 Gurobi Optimization
Results From SolverOPL float TotProdCost = sum (t in Periods,
j in Products, i in Factories)
ProdCost[i,j]*x[i,j,t];Python TotProdCost = sum([ProdCost[i, j] * x[i, j, t].x
for t in Periodsfor j in Productsfor i in Factories])
© 2016 Gurobi Optimization
OPL to Python
} Consider moving to Python} If you still like modeling languages, try AMPL
© 2016 Gurobi Optimization
Migration Library for CPLEX Concert API
© 2016 Gurobi Optimization
Migrating From Concert
} Concert◦ C++ API developed in late 90s◦ Java and C# versions followed◦ Good option for using mainstream languages
© 2016 Gurobi Optimization
Migrating from Concert
} Two Strategies◦ Translate code◦ Use an Adapter
© 2016 Gurobi Optimization
} Pattern◦ Object Adapter Pattern
} Physical Examples◦ Electric plugs◦ CO2 Filter on Apollo 13
Adapter
© 2016 Gurobi Optimization
Adapter Approach
} Give applications written against the Concert API, access to the Gurobi solver
} Our Adapter◦ Not a complete implementation of Concert� Most applications use a small subset◦ Enough to make a model run◦ Free to use� Starting point for your application
© 2016 Gurobi Optimization
Adapter Approach
} Working Concert Application
} Unlink CPLEX / Concert libraries
} Add Gurobi Library
} Add adapter library◦ Modify adapter for your code
© 2016 Gurobi Optimization
Example
} Makefile◦ CPPLIB = … -lilo_grb … -lgurobi_c++ -lgurobi65
} Eclipse
© 2016 Gurobi Optimization
Adapter Library
} Available for Java and C#
} Working on C++
} Available on Github◦ https://github.com/abremod/concert2gurobi4cs
© 2016 Gurobi Optimization
Adapter Library
} “Object Adapter”, not a “Class Adapter”} Favor Composition over Inheritance
◦ public class IloCplex extends ilog.concert.Algorithmimplements IloMPModeler {
◦ GRBEnv env;
◦ GRBModel model;◦ private boolean vars_synced = true;◦ private boolean constrs_synced = true;
© 2016 Gurobi Optimization
Adapter Library
} C++ specific
◦ IloXXXX objects are handles� “Pimpl” ◦ Boost shared_ptr<> covers handle functionality◦ class IloRange : public IloExtractable {◦ private:◦ class Impl;
◦ boost::shared_ptr<Impl> _impl;◦ public:◦ IloRange(IloEnv env, double lb, double ub, const char *name=0);
◦ IloColumn operator()(IloNum);
◦ };
© 2016 Gurobi Optimization
Limitations
} Adapter probably won’t work out of the box
} Variables tied to models, not environments◦ No “not extracted exceptions”
} Limited Support for Callbacks
} No support for Goals
So what happens next?
} Request a free evaluation license, if you have not already done so◦ [email protected]◦ [email protected]
} Set up a free consultation with a consultant from [email protected]
} Webinar slides: Will be available in the next day or two
} Abremod tools available from: https://github.com/abremod/ilogrb
} Webinar recording: Will be available next week
