Conic optimization: the past, the present and the...

Conic optimization

Imre Polik

Outline

Optimizationproblems

Examples

Conic optimization

The future of conicoptimization

Literature

Other activities

Conic optimization:the past, the present and the future

Imre Polik

McMaster UniversityAdvanced Optimization Lab

Lehigh UniversityApril 17, 2008

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Literature

Other activities

Outline

1 Optimization problems in general

2 Motivating examples for conic optimizationStability analysis and eigenvalue problemsCombinatorial optimizationOptimization under uncertainty

3 Conic optimizationTheoretical backgroundSoftware tools

4 The future of conic optimizationTheory and algorithmsApplications and software

5 Other activities

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Literature

Other activities

Optimization problems

optimize objective(s)

s.t. constraints are satisfied

Objectives

cost, return

risk

error

SNR

time

heat

lift, drag

. . .

Constraints

laws of nature

manufacturing restrictions

standards

design requirements

resources

tolerances

logical relationships

. . .

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Literature

Other activities

Elements of optimization

(Decision/design) variables x, y, scontinuousdiscretebinary

Problem data (parameters) A, b, c, f, gConstraints/objectives f(x)=

≤0linearquadraticnice nonlinear (convex)ugly nonlinear (nonconvex)

Feasible solution: satisfies all the constraints

Typical problemmin f(x)Ax = b

g(x) ≤ 0h(x) = 0

Conic optimization

Imre Polik

Outline


Examples

Stability analysis

Combinatorialoptimization

Optimization underuncertainty

Other applications

Conic optimization


Literature

Other activities


2 Motivating examples for conic optimizationStability analysis and eigenvalue problemsCombinatorial optimizationOptimization under uncertainty

3 Conic optimization

4 The future of conic optimization

5 Other activities

Conic optimization

Imre Polik

Outline


Examples

Stability analysis



Other applications

Conic optimization


Literature

Other activities

Stability analysis

Theorem (Lyapunov stability)

Trajectories of x = Ax converge to 0 if and only if there is apositive definite P such that ATP + PA is negative definite.

Theorem (Circle criterion)

Linear state-feedback system:

x = Ax+Bw, x(0) = x0

v = Cx

σ(v, w) = (βv − w)T (w − αv) ≥ 0, (sector constraint, α < β)

Equivalent condition for quadratic stability:

P � 0(ATP + PA− 2βαCTC PB + (β + α)CT

BTP + (β + α)C −2

)≺ 0

Conic optimization

Imre Polik

Outline


Examples

Stability analysis



Other applications

Conic optimization


Literature

Other activities

Eigenvalue optimization

Looking for a “good” positive definite matrix

Problem

Given matrices A1, . . . , Am find a nonnegative linearcombination whose smallest eigenvalue is maximal.

SDP formulation

max λm∑

i=1

Aiyi − λI is PSD

yi ≥ 0, i = 1, . . . ,m

Needed for effective stability analysis

Conic optimization

Imre Polik

Outline


Examples

Stability analysis



Other applications

Conic optimization


Literature

Other activities

Relaxation of binary variables

Binary constraints: xi ∈ {0, 1}

LP relaxation: xi ∈ [0, 1], easy, but too weak

Equivalent form:

zi = 2xi − 1

z2i = 1(⇔ zi = ±1)

Matrix form (Z = zzT ):

Z is PSD

diag(Z) = 1

rank(Z) = 1(⇔ Z = zzT )

SDP relaxation is stronger than the LP relaxation

Conic optimization

Imre Polik

Outline


Examples

Stability analysis



Other applications

Conic optimization


Literature

Other activities

Optimization under uncertainty

Sources of uncertainty

measurement errors(random) noiseunpredictability, futurevibrationtechnological limitsmaterial imperfectionscomputer round-off errors

Application areas

medical treatment planningstructural designrobust portfolio selection

Conic optimization

Imre Polik

Outline


Examples

Stability analysis



Other applications

Conic optimization


Literature

Other activities

Solution approaches

Scenario enumeration

include all the possible values of ai

possibly infinitely many...take a subset - column generation

Deterministic approach

put a threshold on the possible errorrewrite the problem - if possible

Probabilistic approach

assume a distribution for the uncertainty (Gaussian)ensure the constraint is satisfied with high probability

Conic optimization

Imre Polik

Outline


Examples

Stability analysis



Other applications

Conic optimization


Literature

Other activities

Example - Robust linear programming

Standard linear programming

min cTx

aTi x ≥ bi, i = 1, . . . ,m

Assume the data is uncertain: ai ∼ N(ai,Σi)We want P (aT

i x ≥ bi) ≥ η, which is equivalent to∥∥∥Σ1/2i x

∥∥∥2≤ Φ(η)

(bi − aT

i x)

Assuming ai ∼ U(ai − βi, ai + βi) leads to an LP

In general, robustification increases complexity

Conic optimization

Imre Polik

Outline


Examples

Stability analysis



Other applications

Conic optimization


Literature

Other activities

Other applications

Quadratic programming

Quadratic linear fractions

r∑i=1

‖Aix+ bi‖2

aTi x+ βi

≤ t

Inequalities with rational powers

x−5/61 x

−1/32 x

−1/23 ≤ t

x1, x2, x3 ≥ 0

Structural/shape design

Sensor network localization

Sum-of-squares optimization

Minimization of univariate polynomials

Conic optimization

Imre Polik

Outline


Examples

Conic optimization

Theoreticalbackground

Software tools


Literature

Other activities


2 Motivating examples for conic optimization

3 Conic optimizationTheoretical backgroundSoftware tools

4 The future of conic optimization

5 Other activities

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Software tools


Literature

Other activities

Symmetric conic optimization in general

The primal-dual problems

min cTx max bT y

Ax = b AT y + s = c

x ∈ K s ∈ K,

The cone K can be

Linear: x ≥ 0Second-order: x0 ≥ ‖x‖2Rotated second-order: x0x1 ≥ ‖x2:n‖, and x0 ≥ 0Semidefinite: x is (can be assembled into) a symmetric,

positive semidefinite matrix, or aproduct of these.

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Software tools


Literature

Other activities

Solving conic optimization

Algorithms: mostly interior point methods

Iterations

SDP: O(√n)

SOCP: O(√

#cones)practice: ≈ 50− 100

Cost of one iteration

SDP: O(mn3 +m2n2 +m3)SOCP: O(m3 + km2n2)much less for sparse data

Problem sizes:

SDP: m ≤ 10000, n ≤ 5000SOCP m ≤ 10000, k ≤ 10000more with sparse data

High accuracy (10−6)

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Software tools


Literature

Other activities

Software

Solvers - mostly open source, last 10 years

SOCP: MOSEK, SeDuMi, LOQO, SDPT3,PENNON

SDP: CSDP, SDPA, SeDuMi, SDPT3, PENSDP

Modeling languages

No commercial support for SDPScarce support for SOCPOnly academic packages (Yalmip, CVX)

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Theory and algorithms

Applications andsoftware

Literature

Other activities


2 Motivating examples for conic optimization

3 Conic optimization

4 The future of conic optimizationTheory and algorithmsApplications and software

5 Other activities

Conic optimization

Imre Polik

Outline


Examples

Conic optimization




Literature

Other activities


More efficient algorithms

IPMs have reached their limitsimplex-like algorithmbetter handling of sparsity for SDPpreprocessing

More general cones

homogeneous conesnonnegative polynomials

Cone intersections

PSD matrix with nonnegative numbersintersection of second-order cones

Integer conic programming

very limited theory/algorithmsefficient cut generation is an open problem

Rounding procedure

getting an exact solution under some conditions

Conic optimization

Imre Polik

Outline


Examples

Conic optimization




Literature

Other activities

Applications and software

Reliability

too many solver failuresinconsistent accuracy

Interfaces

extended and unified input formatscallable libraries

Modeling languages

more commercial involvement

More industrial awareness

interrelated with modeling languages

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Literature

Other activities

1 F. Alizadeh and D. Goldfarb. Second-order coneprogramming. Mathematical Programming, Series B,95:3–51, 2002.

2 A. Ben-Tal and A. Nemirovski. Lectures on Modern ConvexOptimization: Analysis, Algorithms, and EngineeringApplications. MPS-SIAM Series on Optimization. SIAM,Philadelphia, PA, 2001.

3 M. X. Goemans and D. P. Williamson. Improvedapproximation algorithms for maximum cut and satisfiabilityproblems using semidefinite programming. J. Assoc.Comput. Mach., 1994.

4 R. D. C. Monteiro. Primal-dual path-following algorithms forsemidefinite programming. SIAM Journal of Optimization,7:663–678, 1997.

5 M. L. Overton. On minimizing the maximum eigenvalue of asymmetric matrix. SIAM J. Matrix Anal. Appl.,9(2):256–268, 1988.

6 L. Vandenberghe and S. Boyd. Semidefinite programming.SIAM Review, 38:49–95, 1996.

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Literature

Other activities

Other activities

Software development

SeDuMi: software package for conic optimizationoriginally by Jos Sturmin Matlab/C, being reimplemented in Pythonopen source (GPL)

S-lemma, S-procedure

nonconvex quadratic inequalities

Duality theories in optimization

nonconvexnonregularnonexact

Simulation

Canadian Operational Research Society & Visual8Model implemented in Simul82007: 1st prize (team member)2008: results pending (team advisor)

Conic optimization

Imre Polik

Outline


Examples

Conic optimization


Literature

Other activities

Conic optimization:the past, the present and the future

Imre Polik

McMaster UniversityAdvanced Optimization Lab

Lehigh UniversityApril 17, 2008

Conic optimization: the past, the present and the...

Documents

Transcript of Conic optimization: the past, the present and the...