1

A Convex Polynomial that is not SOS-Convex

Amir Ali AhmadiPablo A. Parrilo

Laboratory for Information and Decision SystemsMassachusetts Institute of Technology

FRG: Semidefinite Optimization and Convex Algebraic Geometry

May 2009 - MIT

2

Deciding Convexity

42

321

22

212

31

41 1632248)( xxxxxxxxxp

Given a multivariate polynomial p(x):=p(x1,…, xn ) of even degree, how to decide if it is convex?

A concrete example:

Most direct application: global optimization

Global minimization of polynomials is NP-hard even when the degree is 4

But in presence of convexity, no local minima exist, and simple gradient methods can find a global min

3

Other Applications

In many problems, we would like to parameterize a family of convex polynomials that perhaps:

serve as a convex envelope to a non-convex function

approximate a more complicated function

fit data samples with “small” error

[Magnani, Lall, Boyd]

To address these questions, we need an understanding of the algebraic structure of the set of convex polynomials

4

Convexity and the Second Derivative

2221

21

2221

21

2221

21

2221

21

19219248969624

969624484812)(

xxxxxxxx

xxxxxxxxxH

But can we efficiently check if H(x) is PSD for all x?

Equivalently, H(x) is PSD if and only if the scalar polynomial yTH(x)y in 2n variables [x;y] is positive semidefinite (psd)

Back to our example:

Fact: a polynomial p(x) is convex if and only if its Hessian H(x) is positive semidefinite (PSD)

5

Complexity of Deciding Convexity

Checking polynomial nonnegativity is NP-hard for degree 4 or larger

However, there is additional structure in the polynomial yTH(x)y:

Quadratic in y (a “biform”)

H(x) is a matrix of second derivatives partial derivatives commute

Pardalos and Vavasis (’92) included the following question proposed by Shor on a list of the seven most important open problems in complexity theory for numerical optimization:

“What is the complexity of deciding convexity of a multivariate polynomial of degree four?”

To the best of our knowledge: still open

6

SOS-Convexity

p(x) sos-convex

p(x) convex

))(())(()()()( yxMyxMyxMxMyyxHy TTTT

As we will see, checking sos-convexity can be cast as the feasibility of a semidefinite program (SDP), which can be solved in polynomial time using interior-point methods.

Defn. ([Helton, Nie]): a polynomial p(x) is sos-convex if its Hessian factors as

for a possibly nonsquare polynomial matrix M(x).

)()()( xMxMxH T

7

SOS-convexity (Ctnd.)

sos-convexity in the literature: Semidefinite representability of semialgebraic sets [Helton, Nie]

Generalization of Jensen’s inequality [Lasserre]

Polynomial fitting, minimum volume convex sets [Magnani, Lall, Boyd]

Question that has been raised:

Q: must every convex polynomial be sos-convex?

NoOur main contribution

(via an explicit counterexample)

8

AgendaNonnegativity and sum of squares

A bit of history

Connection to semidefinite programming

SOS-matrices

Other (equivalent?) notions for sos-convexity

Our counterexample (convex but not sos-convex)

Ideas behind the proof

Several remarks

How did we find it?

Conclusions

9

Nonnegative and Sum of Squares Polynomials

m

ii xqxp

1

2 )()(

Defn. A polynomial p(x) is nonnegative or positive semidefinite (psd) if

Defn. A polynomial p(x) is a sum of squares (sos) if there exist some other polynomials q1(x),…, qm(x) such that

p(x) sos p(x) psd (obvious)

When is the converse true?

nxxp 0)(

10

Hilbert’s 1888 Paper

In 1888, Hilbert proved that a nonnegative polynomial p(x) of degree d in n variables must be sos only in the following cases:

n=1 (univariate polynomials of any degree)

d=2 (quadratic polynomials in any number of variables)

n=2 and d=4 (bivariate quartics)

In all other cases, there are polynomials that are psd but not sos

11

The Celebrated Example of Motzkin

The first concrete counterexample was found about 80 years later!

This polynomial is psd but not sos

13),( 22

21

41

22

42

2121 xxxxxxxxM

12

Sum of Squares and Semidefinite Programming

Unlike nonnegativity, checking whether a polynomial is SOS is a tractable problem

,)( Qzzxp T

Thm: A polynomial p(x) of degree 2d is SOS if and only if there exists a PSD matrix Q such that

where z is the vector of monomials of degree up to d

].,...,,,...,,,1[ 2121dnn xxxxxxz

Feasible set is the intersection of an affine subspace and the PSD cone, and thus is a semidefinite program.

13

SOS matrices

Therefore, can solve an SDP to check if P(x) is an sos-matrix.

Defn. ([Kojima],[Gatermann-Parrilo]):

A symmetric polynomial matrix P(x) is an sos-matrix if

for a possibly nonsquare polynomial matrix M(x).

)()()( xMxMxP T

Lemma: P(x) is an sos-matrix if and only if the scalar polynomial yTP(x)y in [x;y] is sos.

14

PSD matrices may not be SOS

Explicit “biform” examples of Choi, Reznick (and others), yield PSD matrices that are not SOS.

For instance, the biquadratic Choi form can be rewritten as:

However this example (and all others we’ve found), is not a valid Hessian:

15

Equivalent notions for convexity

)2

1(]1,0[,

))1(()()1()(

enoughyx

yxpypxp

yxxyxpxpyp T ,))(()()(

yxyxHyT ,0)(

Basic definition:

First order condition:

Second order condition:

16

Each condition can be SOS-ified

]1,0[))1(()()1()(),( SOSyxpypxpyxg

SOSxyxpxpypyxg T ))(()()(),(

SOSyxHyyxg T )(),(2

Basic definition:

First order condition:

Second order condition:

SOSyxpypxpyxg )()()(),(2

1

2

1

2

1

2

1

2

1 A’

B

C

A

AThm: CBA’

Proof: mimics the “standard” proof closely and uses closedness of the SOS cone

17

A convex polynomial that is not sos-convex

Without further ado...

]1,0[))1(()()1()(),( yxpypxpyxg

))(()()(),( xyxpxpypyxg T

yxHyyxg T )(),(2

B

C

A

Need a polynomial p(x) such that all the following polynomials

are psd but not sos.

18

Our Counterexample

83

63

22

43

42

23

62

82

63

21

43

22

21

23

42

21

43

41

23

22

41

42

41

23

61

22

61

81

30607044

162416335

43254011832)(

xxxxxxx

xxxxxxxxxxx

xxxxxxxxxxxp

Claim:

p(x) is convex: H(x) is PSD

p(x) is not sos-convex: H(x) ≠ MT(x)M(x)

A homogeneous polynomial in three variables, of degree 8.

19

Proof: H(x) is PSD

yxHyxxx T )()( 23

22

21

)()()()( 23

22

21 xMxMxHxxx T

Or equivalently the scalar polynomial

is sos.

Claim:

Proof: Exact sos decomposition, with rational coefficients.

Exploiting symmetries of this polynomial, we solve SDPs of significantly reduced size

44433322211123

22

21 )()( zQzzQzzQzzQzyxHyxxx TTTTT

20

Rational SOS Decomposition

21

Rational SOS Decomposition

22

Proof: H(x)≠MT(x)M(x)

Lemma: if H(x) is an sos-matrix, then all its 2n-1 principal minors are sos polynomials. In particular, all diagonal elements are sos.

Proof: follows from the Cauchy-Binet formula.

Therefore, it suffices to show that

is not sos.

We do this by a duality argument.

63

43

22

23

42

43

21

23

22

21

42

21

23

41

22

41

61

48326420516

300120035401792)1,1(

xxxxxxxxxx

xxxxxxxH

23

Separating Hyperplane

SOS

PSDH(1,1)

µ

SOSww

H

0,

0)1,1(,

24

A few remarks

)1,,(),( 2121 xxpxxp

Our counterexample is robust to small perturbations

Follows from inequalities being strict

A dehomogenized version is still convex but not sos-convex

Minimal in the number of variables

“Almost” minimal in the degree

25

How did we find this polynomial?

sosyxHyxxx

H

r

Tr )()(

0)1,1(,

,fix

23

22

21

parameterize H(x)

add Hessian constraints (partial derivatives must commute)

solve this sos-programSOS

PSDH(1,1)

µ

M

26

Messages to take home…

SOS-relaxation is a tractable technique for certifying positive semidefiniteness of scalar or matrix polynomials

We specialized to convexity and sos-convexity

Three natural notions for sos-convexity are equivalent

Not always exact

But very powerful (at least for low degrees and dimensions)

Proposed a convex relaxation to search over a restricted family of psd polynomials that are not sos

Open: what’s the complexity of deciding convexity?

Our result further supports the hypothesis that it must be a hard problem

27

• Want to know more? Preprint at http://arxiv.org/abs/0903.1287

Questions?