Complexity of Testing Existence of Solutions in Polynomial...
Transcript of Complexity of Testing Existence of Solutions in Polynomial...
![Page 1: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/1.jpg)
Complexity of Testing Existence of Solutions in Polynomial Optimization
+A New Positivstellensatz
Amir Ali AhmadiPrinceton, ORFE
Affiliated member of PACM, COS, MAE, CSML
1ICERM Meeting on Nonlinear Algebra: Core Computational Methods, Septβ18
Jeffrey Zhang Princeton, ORFE
Georgina HallPrincetonINSEAD
Joint work with
![Page 2: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/2.jpg)
Existence of solutions
2
Consider a polynomial optimization problem (POP):
Suppose the optimal value is finite (i.e., POP is feasible and bounded below). We would like to test if there exists an optimal solution, i.e.,
a feasible point π₯β such that π π₯β β€ π π¦ , βπ¦ feasible.
Informally: βCan we replace the `infβ with a `minβ?β
infπ₯ββπ
π(π₯)
s.t. ππ π₯ β₯ 0, π = 1,β¦ ,π.
Remarks:β’ If feasible set is bounded, a solution always exists.β’ If π = 1, a solution always exists.β’ Finiteness of optimal value comes as a βpromiseβ.
![Page 3: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/3.jpg)
3
Motivation
[Nie, Demmel, Sturmfels, βMinimizing polynomials via sum of squares over the gradient idealβ, Math. Prog. 2005]:
βThis assumption [existence of minimizers] is nontrivial, and we do not address the (important and difficult) question of how to verify that a given polynomial π π₯ has this property.β
- An exact algorithm cannot return a solution if there is none!- Existence of solutions essential for algorithms that exploit optimality conditions.
There are algothims that check existence of solutions:β’ Greuet, Safey El Din, βDeciding reachability of the infimum of a multivariate polynomialβ, International
Symposium on Symbolic and Algebraic Computation, 2011β’ Boucero, Mourrain, βBorder basis relaxation for polynomial optimizationβ, Journal of Symbolic
Computation, 2016β’ Greuet, Safey El Din, βProbabilistic algorithm for polynomial optimization over a real algebraic setβ,
SIAM J. on Optimization, 2014β’ Quantifier eliminationβ’ β¦
All have running time at least exponential in dimensionβ¦Can there be a faster algorithm?
![Page 4: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/4.jpg)
4
Existence of a solution guaranteed?
0 1 2
1 Yes Yes
Linear ProgrammingNP-hard to test
(This work)
2 Yes
Linear Algebra
Yes
Frank, Wolfe (1956)
3 Yes Yes
Andronov, Belousov, Shironin (1982)
4 NP-hard to test (This work)
Degree of constraints
Deg
ree
of
ob
ject
ive
minπ₯ββπ
π(π₯)
s.t. ππ π₯ β₯ 0, π = 1,β¦ ,π
![Page 5: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/5.jpg)
Outline
1) NP-hardness of testing existence of solutions
2) Sufficient conditions for existence of solutions
a. Review of SOS and Positivstellensatze
b. An SOS hierarchy for coercivity
3) An optimization-free Positivstellensatz(brief and independent)
5
![Page 6: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/6.jpg)
Outline
1) NP-hardness of testing existence of solutions
2) Sufficient conditions for existence of solutions
a. Review of SOS and Positivstellensatze
b. An SOS hierarchy for coercivity
3) An optimization-free Positivstellensatz(brief and independent)
6
![Page 7: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/7.jpg)
Main hardness results
7
Theorem (AAA, Zhang)
Testing whether a degree-4 polynomial attains its unconstrainedinfimum is strongly NP-hard.
Theorem (AAA, Zhang)
Testing whether a degree-1 polynomial attains its infimum on a set defined by degree-2 inequalities is strongly NP-hard.
Proof: Reduction from 1-in-3 3SAT.
6 (for this presentation)/
![Page 8: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/8.jpg)
1-in-3 3SAT
- Input: A CNF formula with three literals per clause.- Goal: Find a Boolean assignment so that each clause has exactly one true literal.
Not satisfiable.
This problem is NP-hard.
8
π₯1 β¨ Β¬π₯2 β¨ π₯3 β§ (Β¬π₯1 β¨ π₯2 β¨ π₯3) β§ (Β¬π₯1 β¨ Β¬π₯2 β¨ Β¬π₯3)
π₯1 = 1, π₯2 = 1, π₯3 = β1
1 β 1 β 1 β1 1 β 1 β1 β 1 1
π₯1 β¨ Β¬π₯2 β¨ π₯3 β§ (Β¬π₯1 β¨ π₯2 β¨ π₯3) β§ (Β¬π₯1 β¨ Β¬π₯2 β¨ π₯3)
![Page 9: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/9.jpg)
NP-hardness of checking attainment (1/2)
9
Goal: Given any instance of 1-in-3 3SAT, construct a polynomial that attains its infimum if and only if the instance is satisfiable
π = π₯1 β¨ x2 β¨ Β¬π₯3 β§ (Β¬π₯1 β¨ Β¬π₯2 β¨ π₯3)
ππ π₯ β π=1π 1 β π₯π
2 2+ π₯1 + π₯2 β π₯3 + 1
2 + βπ₯1 β π₯2 + π₯3 + 12
Important Property: ππ π₯ has a zero if and only if π is satisfiable
π = π₯1 β¨ x2 β¨ Β¬π₯3 β§ (Β¬π₯1 β¨ Β¬π₯2 β¨ π₯3)
Step 1:
But ππ π always attains its infimum (independent of whether π is
satisfiable) as its highest order component is π=1π π₯π
4.
![Page 10: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/10.jpg)
NP-hardness of checking attainment (2/2)
10
ππ π₯ππ+π¦2 + 1 β π¦π§ 2π β π π( )
Step 2:
ππ ππ, β¦ , ππ, π, π, π =
β’ Infimum is always zero (ππ is a sum of squares; take π = 0, π¦ β 0, π§ =1
π¦).
β’ If π satisfiable, take π = 1, and π₯ the satisfying assignment Infimum attained.β’ If π not satisfiable, ππ does not vanish Infimum not attained.
![Page 11: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/11.jpg)
Outline
1) NP-hardness of testing existence of solutions
2) Sufficient conditions for existence of solutions
a. Review of SOS and Positivstellensatze
b. An SOS hierarchy for coercivity
3) An optimization-free Positivstellensatz(brief and independent)
11
![Page 12: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/12.jpg)
Sufficient conditions for existence of solutions
12
Existence of solutions
Compactness of the feasible set
Coercivity of the objective function
Convexity of the of the objective function and concavity of the constraints
Archimedean Condition
Stable Compactness
SDP hierarchy
SDP hierarchy
Theorem
- Testing whether a polynomial optimization problem satisfies any of these conditions is strongly NP-hard.- Our results are minimal in the degree.
minπ₯ββπ
π(π₯)
s.t. ππ π₯ β₯ 0, π = 1,β¦ ,π
![Page 13: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/13.jpg)
13
Review of sum of squaresand Positivstellensatze
![Page 14: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/14.jpg)
How to prove positivity?
β’ (Tight) lower bounds for polynomial minimization problems
Is π π₯ > 0 on {π1 π₯ β₯ 0,β¦ , ππ π₯ β₯ 0}?
Why prove positivity?
β’ Infeasibility certificates for systems of polynomial inequalities
β’ Dynamics and control (Lyapunov functions)
β’ Stats/ML (shape-constrained regression),β¦
{π1 π₯ β₯ 0, π2 π₯ β₯ 0,β¦ , ππ π₯ β₯ 0}
βπ1 π₯ > 0 on {π2 π₯ β₯ 0,β¦ , ππ π₯ β₯ 0}
empty
β
14
![Page 15: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/15.jpg)
15
Sum of squares and SDP
Ex:
Optimizing over set of sos polynomials is an SDP!
β’ A polynomial π is a sum of squares (sos) if it can be written as
β’ A polynomial π of degree 2π is sos if and only if βπ β½ 0 such that
![Page 16: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/16.jpg)
16
20th century
π π₯ > 0, βπ₯ β βπ
π π₯ > 0,βπ₯ β π = π₯ ππ π₯ β₯ 0}
1927 1993
Putinar
If π π₯ > 0, βπ₯ β π,then π π₯ = π0 π₯ + π ππ π₯ ππ(π₯) ,
where π0, ππ are sos
1974 1991
Schmudgen
If π π₯ > 0, βπ₯ β π,then π π₯ = π0 π₯ + π ππ π₯ ππ π₯ +
ππ πππ π₯ ππ π₯ ππ π₯ + β―, where π0, ππ , β¦ sos
Requires compactness
Artin
If π π₯ β₯ 0, βπ₯ β βπ,then β sos π s.t. π β π sos.
Stengle
PositivstellensΓ€tze
Search for these sos polynomials (when degree is fixed) --->SDP.
Requires the Archimedean
property
![Page 17: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/17.jpg)
Infeasibility proofs for polynomial (in)equalities
17
Stengleβs Positivstellensatz
ππ π₯ β₯ 0, π = 1,β¦ ,π, βπ π₯ = 0, π = 1,β¦ , π‘infeasible
if and only if
there exist polynomials ππ and sos polynomials π such that
β1 = ππβπ + π0 + ππππ + πππππππ + ππππππππππ+β―+ π1β¦πΞ π1β¦ππ.
Search for these sos certificates of infeasibility (when deg. is fixed) ---> SDP.
![Page 18: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/18.jpg)
18
Back to coercivity
![Page 19: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/19.jpg)
Coercivity
19
Definition: A function π is coercive if for every sequence {π₯π} such that ||π₯π|| β β, we have π π₯ β β.β’ A coercive function attains its infimumβ’ Checking whether a polynomial is coercive is NP-hard
We provide a condition which is (i) both necessary and sufficient for a polynomial to be coercive and (ii) amenable to an SDP hierarchy
Past work:β’ Jeyakumar, Lasserre (2014)
β’ SDP hierarchyβ’ Bajbar, Stein (2015)β’ Bajbar, Behrends (2017)
![Page 20: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/20.jpg)
An sos hierarchy for testing coercivity (1/2)
20
Theorem (AAA, Zhang)
A polynomial π of degree π is coercive if and only if for some integer π β₯ 1 the following SDP is feasible
![Page 21: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/21.jpg)
An sos hierarchy for testing coercivity (2/2)
21
Equivalently, p is coercive if and only if there exist an even integer πβ² and a
scalar πβ² for which the set {(π₯, πΎ)|π π₯ β€ πΎ, π₯π2 β₯ πΎπ
β²+ πβ²} is empty.
Theorem (AAA, Zhang)
A polynomial p is coercive if and only if there exist an even integer π > 0and a scalar π β₯ 0 such that for all πΎ β β, the πΎ-sublevel set of p is contained within a ball of radius πΎπ + π.
A function is coercive if and only if all its sublevel sets are compact.
π π₯ = π₯4 + π¦4 β 2π₯3 + π¦2 + 3π₯ + π¦
![Page 22: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/22.jpg)
Coercivity hierarchy revisited
22
Theorem (AAA, Zhang)
A polynomial π of degree π is coercive if and only if for some integer π β₯ 1 the following SDP is feasible
![Page 23: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/23.jpg)
Toy example
23
The polynomial π π₯ = π₯14 + π₯2
2 is coercive as certified by the following algebraic identity:
β1 =2
3π₯12 β
1
2
2
+2
3πΎ β
1
2
2
+2
3πΎ β π₯1
4 β π₯22 +
2
3π₯12 + π₯2
2 β πΎ2 β 2
![Page 24: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/24.jpg)
Outline
1) NP-hardness of testing existence of solutions
2) Sufficient conditions for existence of solutions
a. Review of SOS and Positivstellensatze
b. An SOS hierarchy for coercivity
3) An optimization-free Positivstellensatz(brief and independent)
24
![Page 25: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/25.jpg)
PutinarPutinar
If π π₯ > 0, βπ₯ β π,then π π₯ = π0 π₯ + π ππ π₯ ππ(π₯) ,
where π0, ππ are sos
25
Recall what a Positivstellensatz establishes
π π₯ > 0, βπ₯ β π₯ β βπ ππ π₯ β₯ 0, π = 1, β¦ ,π}
Under the Archimedean
property
Search for these sos polynomials (when degree is fixed) --->SDP.Similar situation for Psatze of Stengle and Schmudgen.
![Page 26: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/26.jpg)
26
An optimization-free Positivstellensatz (1/2)
[AAA, Hall, Math of OR β18] (2018 INFORMS Young Researchers Prize)
π π₯ > 0, βπ₯ β π₯ β βπ ππ π₯ β₯ 0, π = 1, β¦ ,π}
2π =maximum degree of π, ππ
β
β π β β such that
π π£2 β π€2 β1
π π π£π
2 β π€π2 2 π
+1
2π π π£π
4 + π€π4 π
β π π£π2 + ππ€π
2 π2
has nonnegative coefficients,
where π is a form in π +π + 3 variables and of degree 4π, which can be explicitly written from π, ππ and π .
![Page 27: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/27.jpg)
π π₯ > 0 on π₯ ππ π₯ β₯ 0} β
βπ β β s. t. π π£2 β π€2 β1
π π π£π
2 β π€π2 2 π
+1
2π π π£π
4 + π€π4 π
β π π£π2 + ππ€π
2 π2
has β₯ 0 coefficients
Proof sketch:
27
An optimization-free Positivstellensatz (2/2)
π π₯ > 0 on π₯ ππ π₯ β₯ 0} β
βπ β β s. t. π ππ β ππ β1
π π π£π
2 β π€π2 2 π
+1
2π π π£π
4 + π€π4 π
β π π£π2 + ππ€π
2 π2
has β₯ 0 coefficients
π π₯ > 0 on π₯ ππ π₯ β₯ 0} β
βπ β β s. t. π ππ β ππ β1
π π π£π
2 β π€π2 2 π
+π
ππ π ππ
π + πππ π
β ππππ + πππ
π ππ
has β₯ π coefficients
π π₯ > 0 on π₯ ππ π₯ β₯ 0} β
βπ β β s. t. π ππ βππ βπ
π π ππ
π β πππ π π
+π
ππ π ππ
π + πππ π
β ππππ + πππ
π ππ
has β₯ π coefficients
β’ π π₯ > 0 on π₯ ππ π₯ β₯ 0} β π is pd
β’ Result by Polya (1928):
π even and pdβ βπ β β such that π π§ β π π§π2 π
has nonnegative coefficients.
β’ Make π(π§) even by considering π ππ βππ .We lose positive definiteness of
π with this transformation.
β’ Add the positive definite term π
ππ π ππ
π +πππ π
to π(π£2 β π€2) to make it
positive definite. Apply Polyaβs result.
β’ The term βπ
π π ππ
π βπππ π π
ensures that the converse holds as well.
Want to know more? aaa.princeton.edu
![Page 28: Complexity of Testing Existence of Solutions in Polynomial ...aaa/Public/Presentations/AAA_ICERM_18.pdfComplexity of Testing Existence of Solutions in Polynomial Optimization + A New](https://reader035.fdocuments.us/reader035/viewer/2022071214/6042c7e084f82c40cd4c1ca6/html5/thumbnails/28.jpg)
You are cordially invitedβ¦
Princeton Day of OptimizationSeptember 28, 2018
http://orfe.princeton.edu/pdo/
Thank you.