Semidefinite ProgrammingBasics and SOS

Fernando Mário de Oliveira Filho

Campos do Jordão, 20 November 2013

Available at: www.ime.usp.br/~fmario under talks

Conic programming

Complexity: depends on the cone!

V is a real vector space

Given: c, a1, . . . , am 2 V and b1, . . . , bm 2 R

↵x+ �y 2 C 8 x, y 2 C and ↵,� � 0

max hc, xihai, xi = bi for i = 1, . . . , m

x 2 C

h·, ·i is an inner product

Convex cone: set C ✓ V such that

Examples• V = Rn

and C = Rn+: linear programming

• V = Sn and C = cone of PSD matrices: SDP

A, B 2 Sn

hA,Bi = tr(ATB) =

nX

i,j=1

AijBij = trace inner product

() all eigenvalues are nonnegative

A is PSD (also A ⌫ 0): x

TAx � 0 8 x 2 Rn

() A = QTQ for some Q 2 Rn⇥n

() A =Pn

i=1 �iuiuTi , with �i � 0

Semidefinite programmingGiven: C, A1, . . . , Am 2 Sn and b1, . . . , bm 2 R

max hC,XihAi, Xi = bi for i = 1, . . . , mX ⌫ 0

• Generalizes LP: make all matrices diagonal! • Duality theory: comes from conic programming • Solvable in poly-time: under some conditions (later) • Numerical methods: efficient interior-point methods

✓x z

z y

◆⌫ 0 x, y � 0 and xy � z

2 � 0()

Application: polynomial optimizationIs the polynomial p 2 R[x1, . . . , xn] nonnegative?

Sure, if it is a sum-of-squares: p = q21 + · · ·+ q2k

Exercise: A univariate poly is nonnegative iff it is SOS

Pn,d = nonnegative polys, n variables, degree d

⌃n,d = SOS polys, n variables, degree d

Theorem. ⌃n,d ✓ Pn,d with equality i↵

• bivariate polys (n = 2)

• quadratics (d = 2)

• n = 3, d = 4 David Hilbert 1888

SOS is SDPp 2 R[x1, . . . , xn] degree d

v(x) = vector with all monomials degree d/2

Theorem. p is SOS i↵ p(x) = hQ, v(x)v(x)T i for some Q ⌫ 0.

n = 2, d = 6

v(x) =�1, x, x2

, x

3, y, xy, x

2y, y

2, xy

2, y

3�

0

BBBBBBBBBBBBBB@

1 x x

2x

3y xy x

2y y

2xy

2y

3

x x

2x

3x

4xy x

2y x

3y xy

2x

2y

2xy

3

x

2x

3x

4x

5x

2y x

3y x

4y x

2y

2x

3y

2x

2y

3

x

3x

4x

5x

6x

3y x

4y x

5y x

3y

2x

4y

2x

3y

3

y xy x

2y x

3y y

2xy

2x

2y

2y

3xy

3y

4

xy x

2y x

3y x

4y xy

2x

2y

2x

3y

2xy

3x

2y

3xy

4

x

2y x

3y x

4y x

5y x

2y

2x

3y

2x

4y

2x

2y

3x

3y

3x

2y

4

y

2xy

2x

2y

2x

3y

2y

3xy

3x

2y

3y

4xy

4y

5

xy

2x

2y

2x

3y

2x

4y

2xy

3x

2y

3x

3y

3xy

4x

2y

4xy

5

y

3xy

3x

2y

3x

3y

3y

4xy

4x

2y

4y

5xy

5y

6

1

CCCCCCCCCCCCCCA

A small example

v(x) = (1, x, x2, y, y

2, xy)

p(x, y) = x

4 + 2x3y + x

2y

2 � 6x2y � 6xy2 + 10x2 + 10xy + 9y2 � 30y + 25

Qv(x)v(x)T

0

BBBBBBBB@

� 1 x x

2y y

2xy

1 q11 q12 q13 q14 q15 q16

x q21 q22 q23 q24 q25 q26

x

2q31 q32 q33 q34 q35 q36

y q41 q42 q43 q44 q45 q46

y

2q51 q52 q53 q54 q55 q56

xy q61 q62 q63 q64 q65 q66

1

CCCCCCCCA

0

BBBBBBBB@

� 1 x x

2y y

2xy

1 1 x x

2y y

2xy

x x x

2x

3xy xy

2x

2y

x

2x

2x

3x

4x

2y x

2y

2x

3y

y y xy x

2y y

2y

3xy

2

y

2y

2xy

2x

2y

2y

3y

4xy

3

xy xy x

2y x

3y xy

2xy

3x

2y

2

1

CCCCCCCCA

q11 = 25

q23 + q32 = 0

q25 + q52 + q46 + q64 = �6

Q =X

uiuTi =) p =

X(uT

i v(x))2

Sparse SDPA format: Setup

maximize hF0, Y ihFi, Y i = ci for i = 1, . . . , mY ⌫ 0.

1

2

3 Block 4, a diagonal block

Matrices have a block structure

Sparse SDPA format

maximize hF0, Y ihFi, Y i = ci for i = 1, . . . , mY ⌫ 0.

Text file ‘problem.sdpa’:

Matrix entries: hmatrix number i h block number i h i i h j i h entry i

Right-hand side: c1 c2 · · · cmBlock sizes: s1 s2 · · · sN

Number of constraint matrices: m

Number of blocks: N

Remarks: • Indices always start at 1 • Write the size of a block as a negative number to indicate a diagonal block • Only upper-diagonal entries need to be given!

A feasibility problem

p(x) = x

4 + 6x3 + 15x2 � 10x+ 17

F2 =

0

@0 1 01 0 00 0 0

1

AF1 =

0

@1 0 00 0 00 0 0

1

A

F3 =

0

@0 0 10 1 01 0 0

1

A F4 =

0

@0 0 00 0 10 1 0

1

A F5 =

0

@0 0 00 0 00 0 1

1

A

F0 =

0

@0 0 00 0 00 0 0

1

A

maximize hF0, Y ihFi, Y i = ci for i = 1, . . . , 5

Y ⌫ 0.

Is the polynomial

a sum of squares?

c1 = 17, c2 = �10, c3 = 15, c4 = 6, c5 = 1

v(x) = (1, x, x2)

The SDPA fileF2 =

0

@0 1 01 0 00 0 0

1

AF1 =

0

@1 0 00 0 00 0 0

1

A

F3 =

0

@0 0 10 1 01 0 0

1

A F4 =

0

@0 0 00 0 10 1 0

1

A F5 =

0

@0 0 00 0 00 0 1

1

A

F0 =

0

@0 0 00 0 00 0 0

1

A

c1 = 17, c2 = �10, c3 = 15, c4 = 6, c5 = 1

File ‘sos.sdpa’: 5 ▷ Number of constraint matrices!1 ▷ We have only one block!3 ▷ Which is 3x3!17 -10 15 6 1 ▷ The right-hand side 1 1 1 1 1.0 ▷ Matrix entries!2 1 1 2 1.0 !3 1 1 3 1.0 !3 1 2 2 1.0!4 1 2 3 1.0 !5 1 3 3 1.0

CSDP output

p1(x) = �0.719209404617218x2 � 1.21267812518166x+ 4.12310562561766

p2(x) = 0.482351656352516x2 + 4.41136763929902x

p3(x) = 0.500074706343094x2

p

21 + p

22 + p

23 = x

4 + 6x3 + 15x2 � 10x+ 17

Y =

0

@17.0000000000000 �5.00000000000000 �2.96537634217438

�5.00000000000000 20.9307526843488 3.00000000000000�2.96537634217438 3.00000000000000 1.00000000000000

1

A

0.0 0.0 0.0 0.0 0.0 ▷ Dual solution, only numbers!1 1 1 1 2.886751345948129001e-11 ▷ Dual solution, matrix!1 1 2 2 2.886751345948129001e-11 !1 1 3 3 2.886751345948129001e-11 !2 1 1 1 1.700000000000000000e+01 ▷ Primal solution, matrix!2 1 1 2 -5.000000000000000000e+00 !2 1 1 3 -2.965376342174380397e+00 !2 1 2 2 2.093075268434876079e+01 !2 1 2 3 3.000000000000000000e+00 !2 1 3 3 1.000000000000000000e+00

Other uses of SOS

minimum of p is � () p� � is nonnegative

() p is SOS

Formulate as SDP: max{� : p� � is SOS }

p 2 R[x] is nonnegative in [a, b] () 9 q1, q2 SOS s.t.

p = (x� a)(b� x)q1 + q2

Finding the minimum of a polynomial

Nonnegativity in a domain

Duality in conic programmingC ✓ V a cone

Dual cone: C

⇤= {x 2 V : hx, yi � 0 for all y 2 C }

max hc, xihai, xi = bi for i = 1, . . . , m

x 2 C

Primal standard form:

min y1b1 + · · ·+ ymbmy1a1 + · · ·+ ymam � c 2 C⇤

Dual standard form:

strictly feasible: x in interior of C

strictly feasible: y1a1 + · · ·+ ymam � c in interior of C

Duality theorems

x is primal feasible, y is dual feasible =) hc, xi y

Tb

dual is bounded and strictly feasibleIf:primal attains optimal and values coincideThen:

primal is bounded and strictly feasibleIf:dual attains optimal and values coincideThen:

Strong duality:

Weak duality:

Applied to SDP…

Exercise: PSD cone is self-dual

max hC,XihAi, Xi = bi for i = 1, . . . , mX ⌫ 0

Primal standard form:

min y1b1 + · · ·+ ymbmy1A1 + · · ·+ ymAm � C ⌫ 0

Dual standard form:

strictly feasible: X is positive definite

strictly feasible: y1A1 + · · ·+ ymAm � C is positive definite

SDP and duality gone wrong

Optimal not attained:

min �✓↵ 11 �

◆⌫ 0

max �X12 �X21

X11 = 0, X22 = 1

max �X11 �X22

X11 = 0

2X13 +X22 = 1

X ⌫ 0

min y20

@y1 + 1 0 y2

0 y2 + 1 0y2 0 0

1

A ⌫ 0

Positive duality gap:

Optimal: -1 Optimal: 0

Summary and coming soon…

• Generalizes LP • Fits framework of conic programming • Solvable in poly time (under some “mild” assumptions) • Has a duality theory (inherits from conic programming) • One useful application: SOS polynomials

• Approximation algorithms • Maximum-cut problem • Grothendieck inequalities and applications

Semidefinite programming…

Tomorrow: