Tight integrality gaps forvertex-cover semidefinite relaxations

in the Lovász-Schrijver Hierarchy

Avner Magen

Joint work with Costis Georgiou, Toni Pitassi and Iannis Tourlakis

University of Toronto

Minimum Vertex Cover

Finding minimum size VC is NP-hard

Exist simple 2-approximations

All known algs are 2 o(1) approximations!

Probabilistically checkable proofs (PCPs) No poly-time 1.36 approximation [Dinur-Safra’02]

Unique Games Conjecture [Khot’02] No poly-time 2 approximation [Khot-Regev’03]

Alternative (concrete) approach [ABL’02, ABLT’06]: Rule out approximations by large subfamilies of algorithms

Linear Programming approach

min iV vi

vi + vj ≥ 1, ij E

vi {0,1} 0 ≤ vi ≤ 1

True Optimum

Optimal Fractional SolutionIntegrality Gap: max

Easy to see IG ≤ 2

for Kn : IG = 2 1/n

SDP: the ultimate remedy?

Vertex Cover on G = (V,E)

Tighter relaxation? Smaller integrality gap?

min iV (1 + v0 · vi)/2

(v0 vi) · (v0 vj) = 0, ij E

|| vi ||2 = 1, vi Rn+1

min iV (1 + x0xi)/2

(x0 xi)(x0 xj) = 0, ij E

|xi| = 1 Hatami-M-Markakis’06:

Integrality gap still 2 o(1), even

with “pentagonal” inequalities

Semidefinite Programming Relaxations

Kleinberg-Goemans’98:

Integrality gap 2 o(1)

Clearly holds in

integral case

vi {1,1}

(v0 vi) · (v0 vj) 0, i,j

(vi vj) · (vi vk) 0, i,j

Charikar’02:

Gap still 2 o(1)

Systematic Approach: Lovász-Schrijver Liftings [LS’91]

Procedures LS0, LS, LS+ for tightening linear relaxations Integral hull in ≤ n rounds Optimize over rth round relaxation in nO(r) time

Very powerful algorithms obtained through small number of rounds: GW’94, KZ’97, ARV’04 algorithms “poly-time” in LS+ All NP in “exponential time”

May view super-constant rounds lower bounds in LS+ models as evidence about inapproximability

Initial Linear Relaxation

Integral Hull

Has PSD constraint Sequence of tighter and tighter SDPs

“Lift” to obtain

SDP Relaxation

n variablesn2 variables

“Project” back

to obtain tighter LP

Previous Lower Bounds for Vertex Cover – without SDP constraints (LS)

[ABLT’06]: Int. gap 2 o(1) after (log n) LS rounds

[Tourlakis’06]: Int. gap 1.5 o(1) after (log2 n) LS rounds

[STT’06b]: Int. gap 2 o(1) after (n) LS rounds

Status of SDP variant LS+

Stronger: one round already Implies clique constraint More generally, gives n-θ(G) lower bound on VC (so

sparse graph are generally not good) Gives rise to SDPs in the “lift” phases.

Integrality gap of 7/6 for LS+ (STT06a)

PCP world: Hastad 0.5-hardness for MAX3XOR and the FGLSS reduction imply 7/6-hardness for VC

AAT05 proved matching LB (for int. gap) in LS+ world for MAX3XOR

STT06b using further ideas from FO06, extend AAT MAX3XOR LB to prove 7/6 int. gap for linear rounds

graph family: FGLSS reduction on random MAX3XOR instances Int. gap 7/6 already after one round

Vertex Cover in LS: results so far

SDP version (LS+)?

Int. gap ≥

2-o(1) ?# rounds

superconsant?

ABLT ’02,STT ’07 NO YES YES

STT ’06 YES NO YES

Charikar ’02 YES YES NO

New result YES YES YES

Main Result

Theorem: Int. gap 2 o(1) for SDPs resulting after(√log n/log log n) LS+ rounds

One LS+ round tighter than [C’02] SDP

SDPs ruled out incomparable to SDPs with (generalized) triangle and pentagonal inequalities (e.g., [HMM’06])

Theorem: Int. gap 2 O(1/√log n/log log n)after O(1) LS+ rounds

Karakostas [K’05] SDP gives 2 (1/√log n) approximation

Use same graph families as [KG’98], [C’02], [HMM’06]SDP solutions rely on sequence of polynomials applying

tensor operations on vectors

xk(xi + xj x0) 0 ij E (x0 xi)(xj – x0) 0 ij E(x0 xi)(x0 xj) 0

vk · (vi + vj v0) 0 ij E(v0 vi) · (v0 vj) = 0 ij E(v0 vi) · (v0 vj) 0

xk(xi + xj x0) 0 ij E (x0 xi)(x0 – xj) = 0 ij E(x0 xi)(x0 xj) 0

Yik + Yjk Y0k 0 ij EY00 Y0i Y0j + Yij = 0 ij EY00 Y0i Y0j + Yij 0

Convert vertex cover LP into an SDP?

Multiply linear inequalities to get valid quadratic constraints.Crucially, add integrality conditions: (x0 xi)xi = 0

E.g.,

Linearize: replace products xixj with linear variables Yij

Lifted SDP in (n + 1)2 variablesProject resulting convex body back onto n + 1 variables Y0i

xk(xi + xj x0) 0 ij E (x0 xi)(xi + xj x0) 0 ij E (x0 xi)(x0 xj) 0

LS+ lift-and-project: the quick guide

min iV xi

xi + xj 1 (i,j) E0 xi 1 i V(x0 = 1)xi + xj x0 0 (i,j) E xi 0 i V x0 xi 0 i V

Yei ,Y(e0ei) K

Y0i = Yii

(x0 xi)xi = 0

(x0 = 1)

Y is PSD

Homogenization:

cone K

= xi

How LS and LS+ tighten VC Relaxation

One round of LS precisely adds “odd-cycle constraints”: For all cycles C in G of odd length,

iC xi ≥ (|C|+1)/2

x1 + x2 + x3 ≥ 2

One round of LS+ adds more: Clique constraints: For all cliques K in G,

iK xi ≥ |K| – 1

min iV xi

xi + xj ≥ 1, ij E

0 ≤ xi ≤ 1

vs. x1 + x2 + x3 ≥ 3/2

Deriving the clique constraints in LS+

0 ≤ x0 – xi) (xi + xj – x0) +((k –x0 – xi)i

2

Edge constraint

i≠j

Let K be a clique of size k in G

SDP condition

xi2 – (k – 1) x0

2

xi ≥ k – 1After projectingi

x K(r) if matrix Y s.t. diagonal is x Y is PSD “columns” K(r 1)

Proving Lower Bounds in LS+ Hierarchies

I.H.

LP relaxation K for G with min VC ~ n:

xi + xj ≥1 ij E

(½, ½,…)

K(1) K(3)

K(2)

Int. gap of K is ≥ 2 – o(1)

(½+, ½+ …)

Use inductive proof: find appropriate Y’s

“Protection”

matrix for xLemma (LS’91):

“Frankl-Rödl” graphs

m-dimensional Hamming cube: n = 2m points

V = {1,1}m

(i, j) E iff (i, j) = (1 )m }

parameter

Theorem: [Frankl-Rödl’87]

Max Ind.Set size |B(v,n/2(1- ))|

m2m(1 2/64)m

Cor: If = (√log m/m) then max IS is o(2m) = o(n)

Graphs used for int.gaps in [KK91, AK94, KG95, C02, HMM06]

(i, j) = (1 )m

o(n)

What’s so wonderful about them?...

Start with a perfect matching

Perturb : edges connect

vertices of Ham. Dist. (1-)n

Vertex Cover = n/2

``Geometric’’ vertex cover = n/2 +O( )

Proof Outline

In induction: need vectors vi to define matrix Yij = vi vj

Show vi exist whenever x {0, 1, ½ + }n and > 6

Ensure S {0, 1, ½ + }n where O()

(/) round lower bound for x = (½ + )1Constant and = (√log m/m)

Int. gap 2 o(1) after (√log n/log log n) rounds

x K(r) if PSD matrix Y s.t.

1. diagonal is x

2. “columns” K(r1) 2’. Show some set S K(r1)

where “columns” conv(S)

(i, j) = (1 )m

VC 1 o(n)

x = (½ + )1

Back to Frankl-Rödl graphs

Natural set {ui} of unit vectors: {1,1}m

(v0 vi) · (v0 vj) = 0, (i, j) E

√m1

Note: ui · uj = 1 2(i, j)/m

Hence (i, j) E ui and uj nearly antipodal

Nearly true for vi = ui

21 for (i, j) E

linear function

F of vi · vj

(i, j) = (1 )m

VC 1 o(n)

ui · uj

1

21

1

F

1

0

1

vi · vj

1

1

Kleinberg-Goemans:

Affine translation onui to obtain vi

F

V = {1,1}m

Use Kleinberg-Goemans vi for LS+?

Fact: One round of LS+ also requires following ineq:

Idea (Charikar): Map ui to wi s.t.

F(wi · wj) 0

F(wi · wj) = 0 if ij E

I.e, when ui · uj = 2 1

How? Use tensoring

(v0 vi) · (v0 vj) 0 i,j

equality whenever ij E

(i, j) = (1 )m

VC 1 o(n)ui · uj

1

21

1

F(vi · vj)1

0

1

vi · vj

1

1

[KG] affine

map on ui

linear

map

F(vi · vj)

F(wi · wj)1

0

1

Desired mapping

on dot-products

Tensoring

u, v Rn

Tensor product: u v Rn2

Value uivj at coordinate (i, j) [n]2

Easy fact: (u v) · (u v) = (u · v)2

Let P(x) = c1xt1 + … + cqxtq

Consider map TP(u) = (c1ut1,…, cqutq)

Example: P(x) = x2 + 4x TP(u) = (u u, 2u) Rn2+2n

TP(u) · TP(v) = (u · v)2 + 4(u · v)2 = P(u · v)

Fact: TP(u) · TP(v) = P(u · v)

22

P determines dot-product

of resulting vectors

Positive coefficients

Back to finding solution for stronger SDP: Use TP

Charikar exhibits appropriate P

(i, j) = (1 )m

VC 1 o(n)

I.e, when ui · uj = 2 1

(v0 vi) · (v0 vj) 0 i,j

equality whenever (i, j) E

F(vi · vj)

ui · uj

1

21

1F

1

0

1

Want wi = TP(ui)s.t. F(wi · wj) min at (i, j) E

ui · uj 11

21

0

KG

C

Charikar sol’n gives one round LS+ lower bound

Charikar vectors define Yij = vi · vj that:

n Diagonal is x = (½ + )1n “Columns” K

x K(r) if PSD matrix Y s.t.

1. diagonal is x

2. “columns” K(r1) I.H.

x = (½ + )1

Can Charikar vectors show “columns” K(1)?

VC = 1 o(n)

Must have seq

of polynomials

Problems: (1) “Columns” not of form (½ + )1 (2) Charikar’s vectors work only for one value

Values distributed like

polynomial of Gaussian

Making non-uniform “columns” uniform

“Columns” we want to continue from not of form (½ + )1

Def [STT]: x K is -saturated if for all ij E so that xi, xj < 1 there is surplus: xi + xj 1 + 2

Lemma [STT]: x is -saturated there exists set of vectorsx(i) {0, 1, ½ + }n in K s.t. x conv({x(i) }).

Can convert “columns” to (essentially) (½ + )1IF “columns” are -saturated

Will be safe to “ignore” 0/1

values distributed like polynomial of Gaussian

Goal: matrix Y for x with “column” saturation ()

Recall P(x) defines TP(u) such that TP(u) · TP(v) = P(u · v)

deg(PC) = O(1/)

Fact: Y has “columns” s.t. some edges never have surplus

Problem: saturation of “close by” edges?

Saturation

Normal. Ham. Dist. from blue edge

Necessary: deg(P) ≥ · m !

For all P

P

Bad saturation zone

The blue edge

~ P(1)P(1-1/m)

≤ P’(1)/m

Is saturation good enough?

= o(m)

Want column saturation O()

Precise technical property needed for P:

| P(ui · uk ) + P(uj · uk) | O()

For all vertices k and all edges ij :

[1, 1]

But ui · uj = 2 1 for all edges ij, so

Need | P(x) + P(y) | O() over R

Red points correspond to 0-1 edges Ignored in saturation calculation

1

1

1

1

12

12

21

21

R

11/m

11/m x

y

Domain of P(x) + P(y)

|ui·uk+uj·uk| 2

|ui·ukuj·uk| 2(1-)

So far: There must be a seq of polys dep. on m. Polynomials must have large degree.

Let x {0, 1, ½ + }n

Take P(x) = (xx 1)m/ + x 1/ + (1- x

Properties: Minimum at ui · uj, ij E P’(1) > m Works as long as > 6 The “Columns” of Y that is produced by

using TP,m(ui) have saturation O()

ui · uj 11 21 0

KG

C

P

arbitrary > 6

Defining the sequence of tensoring polynomials

Putting everything together

Induction: Have x {0, 1, ½ + }n where > 6

Define Y using TP,m(ui)

“Columns” have saturation O()

[STT] Exists S K {0, 1, ½ + }n s.t. “columns” conv(S)

Induction Hypothesis S K(r 1)

Take constant and = (√log m/m)

x K(r) if PSD matrix Y s.t.

1. diagonal is x

2. “columns” K(r1) 2’. Show some set S K(r1)

where “columns” conv(S)

x = (½ + )1

r = (/)

(i, j) = (1 )m

VC 1 o(n)

x K(r)

Requiring that ||vi-vj||2 is l1?

As is, no l1 inequalities are not implied. The results of [HMM] (showing that metric-cut ineqaities

and pentagonal inequalities hold) suggest the examples are still good.

Need to Give Sherali Adams LB introduce dij = ||vi-vj||2

Add more reqs the LS+ proof need to satisfy.

Sherali-Adams [SA’90] Lift-and-Project

Idea: Keep “lifting” but never project!Simulate third, fourth, etc, degree products with linear vars

Only known integrality gap [FK’06]:(log n) SA rounds int. gap ≤ 2 for MAX-CUT

SA+ lower bound would inequalities for lifted variables Triangle, pentagonal, etc., inequalities derivable

E.g., x1x2x3 Y123

LP not SDP version

Relations to Unique Games Conjecture (UGC)

LS+ lower bounds may provide evidence of inapproximability

UGC [Khot’02] implies optimal inapproximability results for Vertex Cover, MAX-CUT, etc

Strong LS+, SA+ lower bounds for VC, MAX-CUT

Thanks