Generic Rounding Schemesfor

SDP Relaxations

Prasad RaghavendraGeorgia Institute of Technology,

Atlanta

``Squish and Solve” Rounding Schemes

[R,Steurer 2009]

Rounding Schemes via Dictatorship Tests

[R,2008]

Rounding SDP Hierarchies via Correlation

[Barak,R,Steurer 2011] [R,Tan 2011]

``Squish and Solve” Rounding Schemes

[R,Steurer 2009]

Max Cut

10

15

3

7

11

Max CUTInput: A weighted graph G

Find:A Cut with maximum number/weight of crossing edges

Fraction of crossing edges

Semidefinite Program:[Goemans-Williamson 94]

Embedd the graph on the N - dimensional unit ball, Maximizing

¼ (Average Squared Length

of the edges)

Eji

jiij vvw),(

2||41

Semidefinite Program[Goemans-Williamson 94]

Variables : v1 , v2 … vn |vi|2 = 1

Maximize

MaxCut

10

15

3

7

11

1

1

1

-1

-1

-1

-1-1

-11 -1

Max Cut ProblemGiven a graph G,Find a cut that maximizes the number of crossing edges

v1

v2

v3

v4

v5

MaxCut Rounding

v1

v2

v3

v4

v5

Cut the sphere by a random hyperplane, and output the induced graph cut.

-A 0.878 approximation for the problem.

[Goemans-Williamson]

SQUISH AND SOLVE ROUNDING

Approximation using Finite Models

¦-CSP Instance =

¦-CSP Instance =finite

variablefolding

(identifyingvariables)

optimal solution for =finite

approximate solution for =

unfolding ofthe assignment

constant time

Challenge: ensure = finite has a good solution

10

15

3

7

11

1

1

-1

-1

-1

-1-1

-1

-1

1

1

11

Approximation using Finite Models

[Frieze-Kannan]For a dense instance =, it is possible to construct finite model =finite

OPT(=finite) ≥ (1-ε) OPT(=)

General Method for CSPs

What we will do :

SDP value (=finite) > (1-ε)SDP value (=)

PTAS for dense instances

Analysis of Rounding Scheme¦-CSP Instance

=¦-CSP Instance

=finite

SDP value ®

SDP value > ® - ²

OPT value¯

rounded value¯

010001001010001001

Hence: rounding-ratio for = < (1+²) integrality-ratio for = finite

unfolding

CONSTRUCTING FINITE MODELS (MAXCUT)

v1

v2

v3

v4

v5

STEP 1 : Dimension Reduction

• Pick d = 1/ Є4 random Gaussian vectors {G1 , G2 , .. Gd} • Project the SDP solution along these directions.Map vector V

V → V’ = (V G∙ 1 , V G∙ 2 , … V G∙ d)v

1

v3v

4 v5

Constant dimensions

STEP 2 : SurgeryScale every vector V’ to unit length

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every vertex to the nearest point in the Є –net

v2v

2

FINITE MODEL Graph on Є –net points

To Show:

SDP value (=finite) > (1-ε)SDP value (=)

Johnson Lindenstrauss Lemma : “Distances are almost preserved under random

projections”

If V’,U’ are random projections of unit vectors U, V on 1/ ε4 directions,

Pr [ |V U – V’ U’| > ∙ ∙ ε] < ε2

STEP 1 : Dimension Reduction•Project the SDP solution along 1/ Є4 random directions.

STEP 2 : SurgeryScale every vector V’ to unit length

STEP 3 : Discretization•Pick an Є –net for the

d dimensional sphere• Move every vertex to the nearest point in the Є –net

For SDP value (=)Contribution of an edge e = (U,V)

|U-V|2 = 2-2 V U ∙

To Show:

SDP value (=finite) > (1-ε)SDP value (=)SDP Vectors for =finite = Corresponding vectors in Є –net

STEP 1With probability > 1- Є2 ,

| |U-V|2 - |U’-V’|2 | < 2Є

STEP 2With probability > 1- 2Є2 , 1+ Є < |V’|2 ,|U’|2 < 1- Є, Normalization changes distance by at most 2Є

STEP 3Changes edge length by at most 2Є

For SDP value (=)Contribution of an edge e = (U,V)

|U-V|2 = 2-2 V U ∙

To Show:

SDP value (=finite) > (1-ε)SDP value (=)SDP Vectors for =finite = Corresponding vectors in Є –net

STEP 1With probability > 1- Є2 ,

| |U-V|2 - |U’-V’|2 | < 2Є

STEP 2With probability > 1- 2Є2 , 1+ Є < |V’|2 ,|U’|2 < 1- Є, Normalization changes distance by at most 2Є

STEP 3Changes edge length by at most 2Є

ANALYSISWith probability 1-3Є2,The contribution of edge e changes by < 6Є

In expectation,For (1-3Є2) edges, the contribution of edge e changes by < 6Є

SDP value (=finite) > SDP value (=) - 6Є – 3Є2

Drawbacks•Running Time(A) On CSP over alphabet size q, arity k

•No explicit approximation ratio

Generic Rounding For CSPs

rounding – ratioA ( ¦ )(approximation ratio) ≥

(1-²) integrality gap of a natural SDP ( ¦ )(SDP is optimal under UGC)

=

[Raghavendra Steurer08]

For any CSP ¦ and any ²>0, there exists an efficient algorithm A,

Unifies a large number of existing rounding schemes, and the resulting algorithm A as good as all known algorithms for CSPs (without dependence on n)

)(2)/1,,(2 npoly

qkpoly

Computing Integrality Gaps

Theorem:

For any CSP ¦ and any ²>0, there exists an algorithm A to compute integrality gap (¦) within an accuracy ²

Running Time(A) On CSP over alphabet size q, arity k

)/1,,(22qkpoly

Run through all instances of size exp(poly(k,q,1/²)

Rounding Schemes via Dictatorship Tests

[R,2008]

Dictatorship TestGiven a function F : {-1,1}R {-1,1}•Toss random coins•Make a few queries to F •Output either ACCEPT or REJECT

F is a dictator functionF(x1 ,… xR) = xi

F is far from every dictator function

(No influential coordinate)

Pr[ACCEPT ] = Completeness

Pr[ACCEPT ] =Soundness

UG Hardness

Rule of Thumb: [Khot-Kindler-Mossel-O’Donnell]A dictatorship test where • Completeness = c and Soundness = αc•the verifier’s tests are predicates from a CSP Λ

It is UG-hard to approximate CSP Λ to a factor better than α

A Dictatorship Test for Maxcut

CompletenessValue of Dictator Cuts

F(x) = xi

SoundnessThe maximum value attained by a cut far from a dictator

A dictatorship test is a graph G on the hypercube.A cut gives a function F on the hypercube

Hypercube = {-1,1}100

Overviewv1

v2

v3

v4

v5

10

15

3

71

1

100 dimensional hypercube

Graph G SDP Solution

CompletenessValue of Dictator Cuts =

SDP Value (G)

SoundnessGiven a cut far from every dictator :It gives a cut on graph G with the same value.

Rounding Scheme:

• Construct the dictatorship test gadget from graph G• Try all possible cuts far from dictator, and obtain a cut back in the graph G.

Guarantee:Algorithm’s Output Value ≥ Soundness of the Dictatorship Test Gadget

UG Hardness

Dictatorship Test

Completeness CSoundness S [KKMO]

UG Hardness“On instances, with

value C, it is NP-hard to output a solution of

value S, assuming UGC”

In our case,

Completeness = SDP Value (G)Soundness < Algorithm’s Output

Cant get better approximation assuming UGC!

The Goalv1

v2

v3

v4

v5

10

15

3

71

1

100 dimensional hypercube

Graph G SDP Solution

CompletenessValue of Dictator Cuts =

SDP Value (G)

SoundnessGiven a cut far from every dictator :It gives a cut on graph G with the same value.

Influences

Definition: Influence of the ith co-ordinate on a function F:{0,1}R [-1,1] under a product distribution μR is defined as:

Infiμ (F) = E [ Variance [F] ]

Random Fixing of All Other Coordinates from μR-1

over changing the ith coordinate as per μ

Definition: A function is τ-quasirandom if for all i, Infi

μ (F) ≤ τ

(For the ith dictator function : Infiμ (F) is as large as variance of F)

v1

v2

v3

v4

v5

Max Cut SDP:

Embed the graph on the N - dimensional unit ball,

Maximizing

¼ (Average Squared Length

of the edges)

v1

v2

v3v

4 v5

Constant dimensional hyperplaneProject to random 1/ Є2 dimensional space.New SDP Value = Old SDP Value + or - Є

100

Dimension Reduction

Making the Instance Harder

v 1

v 2

v 3

v 4

v 5

v1

v2

v3

v4

v5

SDP Value = Average Squared Length of an Edge

Transformations• Rotation does not change the SDP value.• Union of two rotations has the same SDP value

Sphere Graph H :Union of all possible rotations of G.

v 1

v 2

v 3

v 4

v 5v1

v2

v3

v4

v5v 1

v 2v 3

v 4

v 5

SDP Value (Graph G) = SDP Value ( Sphere Graph H)

Making the Instance Harder

v 1

v 2

v 3

v 4

v 5

v1

v2

v3

v4

v5

MaxCut (H) = S

MaxCut (G) ≥ S

Pick a random rotation of G and read the cut induced on it.Thus,

v 1

v 2

v 3

v 4

v 5v1

v2

v3

v4

v5v 1

v 2v 3

v 4

v 5

v1

v2v3

v4

v5

MaxCut (H) ≤ MaxCut(G)SDP Value (G) = SDP Value (H)

Hypercube Graph v1

v2

v3

v4

v5

SDP Solution

100 dimensional hypercube : {-1,1}100

For each edge e, connect every pair of vertices in hypercube separated by the length of e

Generate Edges of Expected Squared Length = d

1) Starting with a random x Є {-1,1}100 ,1) Generate y by flipping each bit of x with probability d/4

Output (x,y)

Dichotomy of Cuts

Dictator CutsF(x) = xi

Cuts Far From Dictators(influence of each coordinate on function F is small)

A cut gives a function F on the hypercube

F : {-1,1}100 -> {-1,1}

Hypercube = {-1,1}100

1

1 1

1

-1 -1

-1

Dictator Cuts

100 dimensional hypercube

v1

v2

v

uv5

For each edge e = (u,v), connect every pair of vertices in hypercube separated by the length of e

Value of Dictator Cuts = SDP Value (G)

Pick an edge e = (u,v), consider all edges in hypercube corresponding to e

Fraction of red edges cut by horizontal dictator .

Fraction of dictators that cut one such edge (X,Y)

Number of bits in which X,Y differ

=|u-v|2/4

=

X

Y

=

Fraction of edges cut by dictator = ¼ Average Squared Distance

Cuts far from Dictatorsv1

v2

v3

v4

v5

100 dimensional hypercube

-1

-1-1

1

1

1

Intuition:

Sphere graph : Uniform on all directions

Hypercube graph : Axis are special directions

If a cut does not respect the axis, then it should not distinguish between Sphere and Hypercube graphs.

v1

v2

v3

v4

v5v 1

v 2

v 3

v 4

v 5

v 1

v 2v 3

v 4

v 5

The Invariance PrincipleCentral Limit Theorem

``Sum of large number of {-1,1} random variableshas similar distribution as

Sum of large number of Gaussian random variables.”

Invariance Principle for Low Degree Polynomials[Rotar] [Mossel-O’Donnell-Oleszkiewich], [Mossel 2008]

“If a low degree polynomial F has no influential coordinate, then F({-1,1}n) and F(Gaussian) have similar distribution.”

Hypercube vs Sphere

H

F:{-1,1}100 -> {-1,1} is a cut far from every dictator.

P : sphere -> Nearly {-1,1} is the multilinear extension of F

By Invariance Principle, MaxCut value of F on hypercube ≈ Maxcut value of P on

Sphere graph H

Rounding SDP Hierarchies via Correlation

[Barak,R,Steurer 2011] [R,Tan 2011]

The Unique Games Barrier

It is Unique Games-Hard to approximate to a factor better than that given by Simple SDP Relaxation for

• Constraint Satisfaction Problems [R08]• Metric Labelling Problems [Manokaran-Naor-R.-Schwartz 08]• Ordering Constraint Satisfaction Problems

[Guruswami-Hastad-Manokaran-R. ]• Kernel Clustering Problems [Khot Naor 09]

• Grothendieck Problem [R.-Steurer 09]

• Monotone-Hard-Constraint CSPs [Kumar-Manokaran-Tulsiani-Vishnoi]

[R-Steurer 09]Unconditionally, Adding all valid constraints on at most 2^O((loglogn)1/4) variables to the simple SDP does not improve the approximation ratio for

Constraint Satisfaction Problems

Metric Labelling Problems Ordering Constraint Satisfaction

Problems

Kernel Clustering Problems

Grothendieck Problem

For the non-believers

Stronger SDP Relaxations

Possibility:``Certain Strong SDP Relaxations yield better

approximations and disprove the Unique Games Conjecture”

(five rounds of Lasserre hierarchy)

Even Otherwise:For what problems do these relaxations help?

How does one use these stronger SDP relaxations?

Difficulty

.

Successes of Stronger SDP Relaxations:• [Arora-Rao-Vazirani] used an SDP with triangle inequalities to improve approximation for Sparsest Cut from log n to sqrt(log n).

• Stronger SDPs for better approximations for graph and hypergraph independent set in [Chlamtac] [Arora-Charikar-Chlamtac] [Chlamtac-Singh]

Very few general techniques to extract the power of stronger SDP relaxations.

Eji

jiij vvw),(

2||41

Semidefinite Program

Variables : v1 , v2 … vn | vi |2 = 1

Maximize

SDP for MaxCutQuadratic Program

Variables : x1 , x2 … xn xi = 1 or -1

Maximize

Eji

jiij xxw),(

2)(41

Relax all the xi to be unit vectors instead of {1,-1}. All products are replaced by inner products of vectors

10

15

3

7

11

1

11

-1

-1

-1

-1-1

-11 -1

Ideally, these vectors are convex combination of integral solutions.

-- the SDP can be thought of as a distribution over cuts

Instead, we force vectors to look like integral solutions locally (on every k vertices)

k-round Lasserre-SDP for MaxCut

1015

3

7111

11

-1-1

-1

-1-1

-11 -1

1 -1 1 -1 …………….. 1 1 1 -1 1 1 - 1

1 -1 -1 -1 …………….. 1 1 1 -1 1 1 1

1 -1 -1 -1 …………….. 1 1 1 -1 1 1 - 1 1 -1 1 -1 …………….. 1 1 1 -1 1 1 - 1

1 1 1 -1 …………….. 1 1 1 -1 1 1 - 1

1 1 1 -1 …………….. 1 1 1 -1 1 1 - 1

-……………………………………………………………………

X1 X2 X3 X4 …………….. X15 …………………. Xn

Local distribution μS

For any subset S of k vertices,

A local distribution μS over {+1,-1} assignments to the set S

Conditioned SDP Vectors {vi|Sα}

For any subset S of k vertices, and an assignment α in {-1,1}k ,

An SDP solution {vi|Sα} corresponding to the SDP solution conditioned on S being assigned α

CorrelationsCorrelation:`` Two random variables are correlated, if the fixing the

value of one changes the distribution of the other’’Measuring Correlation:Mutual information between the two random variables.

]|)([)|( aYXHEYXHa

]Pr[1log]Pr[)(aX

aXXH

Entropy of X Conditional Entropy of X|Y

Mutual Information I(X,Y) = H(X) - H(X|Y)

Global CorrelationGlobal Correlation is the average correlation between random

pairs of vertices in the instance.

GC = E {a,b}[ I(Xa , Xb) ]

Crucial ObservationConditioning the SDP solution on the value of a random vertex Xa reduces average entropy by GC

Proof:average entropy = E{b} H(Xb) average entropy after conditioning Xa = E{a} [E{b} H(Xb | Xa)]

Hence the decrease isE{b} H(Xb) - E{a} [E{b} H(Xb | Xa)] = E{a,b} [H(Xb)- H(Xb | Xa)] = E{a,b} [I(Xb , Xa)]

Progress By Global CorrelationsSuppose an SDP solution has global correlation > ε,Then we sample and condition on the value of a random vertex,Average entropy drops by ε

If global correlation always remains > ε, then after 1/ ε conditionings, the average entropy ≈ 0

The variables are almost frozen, and the conditioned SDP solution is nearly integral.

CorollaryWithin O(1/ ε) conditionings, the global correlation of the SDP solution becomes < ε

Application: Max Bisection

10

15

3

7

11

Max BisectionInput: A weighted graph G

Find:A Cut with maximum number/weight of crossing edges

with exactly ½ of the vertices on each side of the cut.

Halfspace Rounding?

v1

v2

v3

v4

v5

Cut the sphere by a random hyperplane, and output the induced graph cut.

The expected fraction of vertices on each side of the cut is half.

However, the actual number of vertices might always be away from half

-- no concentration

Independence among random variables Concentration (Ex: Chernoff bounds)

Lack of concentration lack of independence

Bounding Variance Let Z1 , Z2 , .. Zn denote the random projections,Suppose the rounding function is F : R [0,1]Fraction of vertices on one side of the cut = E{a} [F(Za)]

Variance of this random variable= EZ [ E{a,b} [F(Za)F(Zb)] - E{a} [F(Za)]E{b} [F(Zb)] ]

= E{a,b} [ Covariance(F(Za), F(Zb)) ] Low global correlation E{a,b} [I(Za ,Zb)] is small

the above variance is small.

CSPs with Global Cardinality Constraint

[R, Tan 2011]Given an instance of Max Bisection/Min Bisection with

value 1-ε, there is an algorithm running in time npoly(1/ε) that finds a solution of value 1-O(ε1/2)

[R, Tan 2011]For every CSP with global cardinality constraint, there is a

corresponding dictatorship test whose Soundness/Completeness = Integrality gap of poly(1/ ε) - round

Lasserre SDP.

Another Application: 2-CSPs on ``expanding instances”

Locally, the constraints of the CSP introduce correlations among the variables.

If the graph is a sufficiently good expander, these local correlations must translate in to global correlations.

10

15

3

7

11

1

1

1

-1

-1

-1

-1-1

-1

1 -1

Low-Rank Graphs

If the adjacency matrix of the graph is “low rank” – approximated by few eigen vectors.

10

15

3

7

11

1

1

1

-1

-1

-1

-1-1

-1

1 -1

Lemma: If the number of eigen values > δ is less than d, then an SDP solution with local correlation > δ has global correlation O(1/d2)

2-CSP on random constraint graphs[Barak-Raghavendra-Steurer]

Given an instance of 2-CSP whose constraint graph is a degree d random graph,

poly(1/ε, k, d) round Lasserre SDP hierarchy has value <

Optimum + od(1)

Another Application

Subexponential Time Algorithm for Unique Games[Arora-Barak-Steurer]

Given an instance of Unique Games with value 1-ε,in time exp(nε), the algorithm finds a solution of value 1-εc

Used a combination of brute force and spectral decomposition, but no SDPs

Subexponential Time Algorithm for Unique Games via SDPs[Barak-Raghavendra-Steurer] [Guruswami-Sinop]

Given an instance of Unique Games such thatnO(ε) – rounds of SDP hierarchy has value with value 1-ε,

there exists an assignment of value 1-εc

Future Work

Can one use local-global correlations to prove [Arora-Rao-Vazirani] or something weaker?

subexponential time algorithms beating the current best for MaxCut, Sparsest Cut?

Thank You

Rounding

1015

3

7111

11

-1-1

-1

-1-1

-11 -1

1 -1 1 -1 …………….. 1 1 1 -1 1 1 - 1

-1 -1 -1 -1 …………….. 1 1 1 -1 1 1 1

1 -1 -1 -1 …………….. 1 1 1 -1 1 1 - 1 1 -1 1 -1 …………….. 1 1 1 -1 1 1 - 1

1 1 1 -1 …………….. 1 1 1 -1 1 1 - 1

1 1 1 -1 …………….. 1 1 1 -1 1 1 - 1

-……………………………………………………………………

X1 X2 X3 X4 …………….. X15 …………………. Xn

Case 1: average entropy < ε,

The SDP solution is nearly integral (it can be rounded to integral solution with value c – O(ε) )

Case 2: average entropy > ε,

if we condition on a random vertex, the average entropy drops by δ

Main Theorem (Informal):

If an instance I of a problem satisfies (c,ε,δ)-global correlation property,

Then, (1/δ)-round SDP solution on the instance I is within O(ε) of the integral value.

Do instances have this global correlation property arise?