Welcome to the Webinar - Gurobi · • Heuristics can find good feasible solutions quickly but...

Welcome to the Webinar

Choosing a Math Programming Solver

Speaker Introduction

Dr. Kostja Siefen

• Technical Account Manager at Gurobi Optimization

• Ph.D. in Operations Research, University of Paderborn (Germany)

• Many years of experience in the development anddesign of decision support systems usingmathematical optimization methods.

© 2020, Gurobi Optimization, LLC2

Today's Agenda

Introduction

Solver Capabilities & Ecosystem

Switching Solvers

The Selection Process



Introduction

Mathematical Programming

• The term „mathematical programming“ in general describes theuse of a computer to determine the best option from a set ofalternatives.

• Mathematical Programming is better known as Optimization.

• Mathematical Programming is considered a part of OperationsResearch where mathematical methods are used to support management decisions.


Solvers

• A solver is a software that accepts a mathematical model of an optimization problemas input and computes solutions of the model.

• Two main challenges:

• Feasibility: The ability to compute valid solutions to the defined problem

• Optimality: The ability to compute optimal solutions that cannot be improvedany further with respect to defined objective functions.

• The nature of the underlying problem determines the model type. The selected solverneeds to be able to process the given model type.


Optimization Models• Optimization models can be classified by their properties

(see also https://neos-guide.org/optimization-tree)

• Continuous vs. Discrete• Unconstrainted vs. Constrained• Number of objective functions• Deterministic vs. Stochastic

• Very common to distinguish model types based on the type of expressions• See also: https://neos-guide.org/content/optimization-tree-alphabetical

In this presentation, we will focus on Mixed-Integer Optimization (deterministic, linear or quadratically constrained models with one or more objective functions). The large majority of business problems can be expressed through these modeltypes.


https://neos-guide.org/optimization-tree

https://neos-guide.org/content/optimization-tree-alphabetical

Mixed-Integer Optimization• Most business problems can be expressed as a Mixed-Integer Program (MIP)

• Linear constraints & objective function• Integrality constraints on a subset of variables

• MIP models can by definition contain binary and general integer variables. This allows you to modela lot of relationships in real-life business problems:

• Complex cost functions• Yes/No decisions with their respective implications• Arbitrary logical conditions

• Some models in certain industries require quadraticexpressions:

• Multiplication of variables• Mixed-Integer Bilinear


integerallorsome j

T

xuxlbAxtoSubjectxcMinimize

££=

Choosing a Solver

• Since you are here you‘ve probably come a long way:

• The identification of a relevant business problem to be solved• The use of mathematical optimization as a potential tool• Hence the demand for a suitable solver

• A solver is way more than just a set of algorithms

• Q: Gurobi is a commercial solver. Can we be a neutral guide on this topic?• A: We‘ll show you the aspects and what we‘ve got, so you can compare.


Choosing a Solver


Learn Product Capabilities

Identify Decision Criteria

Evaluate Test & Benchmark


Solver Capabilities & Ecosystem

Open Source Solvers

Thanks to an active community there are various Open Source solvers available withoutlicensing costs for commercial use.

• Open Source Solvers are a great way to get started with optimization• Performance can be sufficient for smaller models without numerical issues• Installation & setup is usually very easy

Things to keep in mind:

• Limited development activity and influence from users• Troubleshooting is difficult, no professional product support• Tremendous difference in performance. Always compare to a commercial solver!


Heuristics

Heuristics, also known as „smart strategies“, are sometimes considered an alternative to a solver.The most important properties of heuristics:

• The algorithms are usually quite tailored to the model structure and need to be adjusted whenthis structure changes

• Heuristics can find good feasible solutions quickly but there is no measurement/guaranteeabout the solution quality.

• Heuristics do not need to work on the mathematical formulation of a problem

Gurobi contains a large amount of heuristics that are embedded in the overall framework

Heuristics can be combined with a solver to generate or improve solutions


Modeling Systems & Frameworks

Gurobi can be used from most modeling systems and frameworks (both free and commercial)

Modeling systems and frameworks provide an abstraction layer:• The model is defined in a unified way, independent of the solver interface• After construction, the model is passed to the solver through the API• Switching between solvers becomes very easy

We encourage our users to choose a model system/framework of their preference.• Model solving times are not affected• Model construction times can be significantly higher due to additional overhead• Users are limited to the feature set that is provided by the modeling system/framework


Product Properties


Performance

Support

Features

Technical Capabilities

Documentation & Training Resources

Supported Platforms

Runtime

Robustness

Scalability

Troubleshooting, Performance Tuning & Access to Experts

Features – Technical Capabilities

There are huge differences between solvers when it comes to product features. Technical capabilities can be grouped into different categories:

• Supported Model Types• Interfaces (APIs)• Modeling Support / Productivity Tools• Model Analysis Tools• Solver Interaction• Automated Model Tuning• Client/Server Separation & Distributed Computing


LP

MILP

QCP

MIQP

Convex MIQCP

QP

Gurobi solves the broadest range of problems – regardlessof size…

Features - Supported Model Types


Non

Con

vex

MIQ

CP

… including non-convex

MIQCP, QCP, QP and bi-linear

Features - Supported Model Types

Multi-Objective Models

• Most real-life decision problems have multiple conflicting objectives• Objectives can be either combined (weighted sum) or solved in a hierarchical way• Gurobi supports both approaches through a simple interface:

Just define the objectives and add a weight and a priority.

Multi-Scenario Models

• A thorough „What If“ analysis helps you to learn more about a decision situation• Sensitivity Analysis is important to learn more about the influence of changing parameters• Gurobi can solve multiple scenarios in a single run: In many cases this is faster compared to

solving each scenario in a separate model.


Features - Interfaces


GurobiOptimizer APIs Your Application

Our lightweight APIs make incorporating Gurobi into your application seamless. Our APIs are thin layers around the high-performance core library to maximize model building speed.

Features – Modeling Support / Productivity ToolsAutomatic modeling of logical conditions

• MIP models allow you to model arbitrary logical conditions (boolean algebra). • More advanced features like implied constraints need additional modeling techniques.

• Gurobi makes it easy to model many of these conditions explicitly and takes care of the internal reformulationautomatically.

Automatic linearization of non-linear functions

• Gurobi presolve automatically linearizes expressions, if possible• Piecewise-linear functions• Products of variables

Automatic approximation of non-linear functions

• Gurobi contains an automatic approximation framework• You can define a non-linear function and your acceptable error


Features – Model AnalysisInfeasibility Analysis

• Optimization models can become infeasible due to many reasons (model & data errors) • Knowing that a model has no solution is not very helpful. The interesting question is: Why?• Gurobi contains powerful tools to analyze infeasibilites:

• High-performance computation of a minimal set of contradicting model components (IIS)• Ability to create and solve feasibility relaxations

Solution Pool

Ability to collect and retrieve alternative model solutions (also in a systematic way)

Solution Quality Analysis

Ability to examine a model solution in terms of numerical tolerances


Features – Client/Server & Distributed ComputingGurobi Compute Server is a high-performance optimization service implementation based on state-of-the-art web standards.

Key factors when comparing Compute Server to other service implementations:

• No interface change: The client application uses the exact same API to solve an optimization model comparedto solving on a local machine. No need to rewrite an existing application.

• Flexible deployment: Client/server separation is optional. The decision on where a model should be solved(locally or remote) can be changed at any time.

• Scalability: Gurobi Compute Server can be deployed on a single server machine or within a cluster of multiple powerful machines that can be accessed remotely. Features for job management and high-availability (e.g., load balancing, queuing and failover) are already implemented.

• Performance: Compute Server is a completely integrated into the Gurobi Optimizer. No need for anyintermediate software layer that consumes additional memory or CPU resources.


The technical team behind the solver

The development team

Developing and improving a solver is a very difficult and time-consuming task. It requires a very special set of skills.

The support team

Technical support for an expert tool requires deep knowledge and experience. A dedicated team with quick response times is an insurance to resolve issues quickly.

The content team

An expert tool requires good documentation and extensive material to train new users.


Dr. Ed RothbergCEO

Dr. Zonghao GuCTO

Dr. Tobias AchterbergDirector of R&D

Dr. Robert LuceDeveloper

Olivier NoiretDeveloper

Dr. Michel JaczynskiSr. Architect

Dr. Daniel EspinozaSr. Developer

Michael WinklerDeveloper

Gurobi‘s Development Team


Fernando OrozcoSenior Software Engineer

Dr. Stefan HeinzSenior Developer

The Development Process

The Idea Stack & Customer Requests

• The Gurobi team is quite creative, we have a long list of ideas – but time is limited• Customer feedback helps us to prioritize improvements for future development• Performance remains the key focus

Benchmarking

• We benchmark a lot before something is added to the product• We measure performance on real model instances from our customers• We use computing clusters with hundreds of machines to do that

Intensive Testing

• Correctness and careful implementation is key, we never rush a release• A large set of models and test cases is run against every change to the product


The Gurobi model library(5927 models)

1

10

100

1000

10000

100000

1000000

10000000

100000000

1E+09

1 10 100 1000 10000 100000 1000000 10000000 100000000

Columns

Rows

100,000

1,000,000


Focus

„Because Optimization is all we do“

Gurobi has a single product and our focus is in three areas:

• Technical Superiority

Our vision is clear: We work hard to stay the most powerful solver on the market.

• Ease of Use

If we can make it substantially easier to use our product, we do it.

• Customer Centricity

Our customers’ models and personal feedback drive our activities.


Numerical Challenges

Mathematical computations on a computer suffer from an inherent problem: Limited accuracy

• The infinite set of real numbers is mapped to a finite set of states (64 bits). • In fact, there are infinite numbers without an exact representation.• Emulating exact arithmetics in software is not an options: It is just too slow

Rounding errors everywhere

• Even the most basic mathematical laws do not hold anymore• Violations and infeasibilities are expected and need to be handled properly• Special care is required so that small rounding errors do not sum up to large errors in the results

A numerically stable implementation of optimization algorithms is very challenging

• Requires substantial modifications and extensions of the algorithms• Important to install safeguards everywhere to distinguish actual improvements from random noise.


Product Support

Working with a solver can be greatly improved with access to experts. Our support team can help you in many ways:

• Help with „How To“ questions• Assistance using the solver interfaces in the most efficient way• Help with modeling techniques• Model benchmarking & tuning• Troubleshooting• IT architecture integration

You get unlimited access to our team with all commercial licenses (current M&S required).



Switching Solvers

Common ScenariosIt‘s all about the performance experience:

Model runtimes do not scale with more data• Models tend to become larger because of a business demand• Examples: Longer planning horizons, more products, more personnel, larger networks

The results of the current solver are not robust/reliable• Lack of acceptable solution quality in the available time• Crashes or wrong results

The current solver struggles with increased model complexity• Runtimes explode when additional complexity is added to the model• Examples: Shorter time periods, increased granularity, additional rules, multiple objectives


The value of improved performanceThere can be significant economic value behind better solver performance.

Benefits of reliable, optimal results in shorter time: • Continuous, significant cost savings• Additional time for planners to consider alternative scenarios• Improved automation reliability by avoiding manual intervention in error cases

Most performance improvements are on the algorithmic level• No need to get faster hardware• Nevertheless a good opportunity to check the impact of an upgrade• Better throughput for parallel model solves


RisksSwitching essential parts of a software system always comes with certain risks.

Lack of knowledge• Changing a model generator source code requires expertise• Avoid the „black box effect“ where developers are afraid of touching existing code

Change of system behavior• Existing users may need to get used to different results• Different solvers are likely to have different configuration options

Deployment effort• Rollout of software upgrades to existing users• Additional testing required


Technical Aspects of Switching

Switching to Gurobi is very easy and seamless in most cases

Interface changes• APIs may be different but usually semantically similar (add variable, create expressions, etc.)• Gurobi tries hard to be compatible when reading model files (MPS, LP) from other solvers• Great support from our team

Architecture changes• Great opportunity to consider a client/server separation or service encapsulation• Gurobi runs on Windows, Linux and Mac• Operating systems can be mixed between client and server



The Selection Process

The Decision CriteriaRemember the three main aspects of a solver:• Features• Performance• Support

All these aspects should be properly evaluated before making a choice.

The list of product features is probably larger than your actual need today. When selecting the relevant product features make sure you consider

• The quality of the feature• The need for additional functionality in future models/projects• The scalability of the feature when the use of the solver grows


Product Evaluation

Test before you buyMost important goal: Experience the product and the team behind it

Evaluate the product in your own environmentWe provide you with free evaluation licenses for 30 days, so you have plenty of time

Remove technical obstaclesYou get a dedicated technical contact and access to the support team during your evaluation

Find the right commercial packageYou work with a dedicated account manager to discuss suitable licensing


Performance BenchmarkBenchmark your own models• No need to migrate any code for this• You can export existing models and test performance side-by-side

Consider variability• Some models are sensitive to a higher variance in runtime• Measurements of runtime and solution quality should take this into account• Very easy to evaluate this effect with Gurobi

Building time vs. solving time• Model building times can be significantly reduced through our native APIs• Work with us to use the interfaces in the most efficient way

Determine the tuning potential• Challenge our support team by sending models• Try our recommendations in your environment


Additional Resources

More about switching from existing solvers or modeling languages:Visit www.gurobi.com/switch

More technical resources:Visit www.gurobi.com/resources/seminars-and-videos

Integrating Gurobi into State-of-the-Art Application Architectures:Visit www.gurobi.com/resource/integrating-gurobi-into-application-architectures

Learn more about Performance Tuning:Visit www.gurobi.com/resource/intro-to-tuning-webinar

How to Choose a Math Programming Solver eBook:Visit www.gurobi.com/ebook-choosing-math-solver


http://www.gurobi.com/switch

http://www.gurobi.com/resources/seminars-and-videos

http://www.gurobi.com/resource/integrating-gurobi-into-application-architectures

http://www.gurobi.com/resource/intro-to-tuning-webinar

https://www.gurobi.com/ebook-choosing-math-solver

Your Next Steps

• Try Gurobi 9.0 Now!

• Get a 30-day commercial trial license of Gurobi at www.gurobi.com/free-trial

• Academic and research licenses are free.

• For questions about Gurobi pricing, please contact [email protected] or [email protected]

• A recording of this webinar, including the slides, will be available in roughly one week.


http://www.gurobi.com/free-trial

http://gurobi.com

http://gurobi.de

Thank You – Questions?

Welcome to the Webinar - Gurobi · • Heuristics can find good feasible solutions quickly but...

Documents

Transcript of Welcome to the Webinar - Gurobi · • Heuristics can find good feasible solutions quickly but...