Computing dense subgraphs with semidefinite programming · Finding a densest k-subgraph ? Diﬃcult...

Computing dense subgraphs withsemidefinite programming

Jerome MALICK∗ Frederic ROUPIN∗∗

∗CNRS, Lab. Jean Kuntzmann, Grenoble

∗∗LIPN-CNRS, University Paris XIII

Optimization 2011, Lisbon – July 2011

1

A combinatorial optimization problem

Find a subgraph of k vertices with maximum number of edges

Example: graph with n = 8 vertices, best subgraph with k = 4 ?

The densest-subgraph problem:

generalization of max-clique

(difficult) particular case of quadratic knapsack

2

A combinatorial optimization problem

Find a subgraph of k vertices with maximum number of edges

Example: graph with n = 8 vertices, best subgraph with k = 4 ?

The densest-subgraph problem:

generalization of max-clique

(difficult) particular case of quadratic knapsack

3

Finding a densest k-subgraph ?

Difficult problem (NP-hard and more - see e.g. Khot ’05)

Solving to optimality ? Few methods:

– using linear programming (Ekrut ’90)– using reformulation techniques (Pisinger ’06)– using quadratic programming (Billionnet-Elloumi-Plateau ’09)

Compute dense subgraphs for unrestricted graphs with n 6 100

Rem: SDP bounds are very tight - but expensive (Roupin ’04)

Question: use this to be competitive with best methods ?

Objective of our work:1 to study new SDP bounds - that trade tightness for cpu time,

while keeping SDP-like quality !

2 to use them within branch-and-bound for computing densestk-subgraphs

3 to compare with the best: Billionnet-Elloumi-Plateau ’09

4












5












6

Outline

1 Formulation and new semidefinite bounds

2 Relaxed resolution: comparison of the bounds

3 Exact resolution: branch-and-bound procedure

7

Formulation and new semidefinite bounds

Outline




8


Formulation of densest k-subgraph problem

Notation: G = (V,E) unweighted graph (|V | = n)W = (wij) adjancency-matrix (/2)

Initial modelling as {0, 1}-QP with constraintsmax

∑(ij)∈E wij yi yj = y>W y∑

i yi = ky ∈ {0, 1}n

Enforcement of constraints: adding n product constraints⇐⇒ adding (

∑ni=1 yi − k)2 = 0

max y>W y∑i yi = k∑ni=1 yi yj = k yj , j = 1, . . . , n

y ∈ {0, 1}n

9


Formulation of densest k-subgraph problem

Notation: G = (V,E) unweighted graph (|V | = n)W = (wij) adjancency-matrix (/2)

Initial modelling as {0, 1}-QP with constraintsmax

∑(ij)∈E wij yi yj = y>W y∑

i yi = ky ∈ {0, 1}n

Enforcement of constraints: adding n product constraints⇐⇒ adding (

∑ni=1 yi − k)2 = 0

max y>W y∑i yi = k∑ni=1 yi yj = k yj , j = 1, . . . , n

y ∈ {0, 1}n

10


Standard reformulation and lifting

Change of variables (and homogenization):Equivalent formulation as a pure {−1, 1}-QP

max x>Qxx>Qj x = 4k − 2n, j ∈ {0, . . . , n}x ∈ {−1, 1}n+1

SDP lifting (e.g. Lovasz ’79, Goemans-Williamson ’95)

〈X, Y 〉 = trace(XY ) makes x>A x = 〈A, xx>〉X = xx> gives xi ∈ {−1, 1} as Xii = 1

Equivalent formulation as linear SDP with rank-one constraintmax 〈Q,X〉〈Qj , X〉 = 4k − 2n, j ∈ {0, . . . , n}Xii = 1, i ∈ {0, . . . , n}rank X = 1, X < 0

11


Standard reformulation and lifting

Change of variables (and homogenization):Equivalent formulation as a pure {−1, 1}-QP

max x>Qxx>Qj x = 4k − 2n, j ∈ {0, . . . , n}x ∈ {−1, 1}n+1

SDP lifting (e.g. Lovasz ’79, Goemans-Williamson ’95)

〈X, Y 〉 = trace(XY ) makes x>A x = 〈A, xx>〉X = xx> gives xi ∈ {−1, 1} as Xii = 1

Equivalent formulation as linear SDP with rank-one constraintmax 〈Q,X〉〈Qj , X〉 = 4k − 2n, j ∈ {0, . . . , n}Xii = 1, i ∈ {0, . . . , n}rank X = 1, X < 0

12


Idea of the “spherical constraint”

Key remark (Malick ’07):

For all X < 0 satisfying Xii = 1,we have

‖X‖ 6 n + 1

Xii = 1X < 0

n + 1

13


Idea of the “spherical constraint”

Key remark (Malick ’07):

For all X < 0 satisfying Xii = 1,we have

‖X‖ 6 n + 1

‖X‖ = n + 1 ⇐⇒ rank X = 1

“spherical constraint”

n + 1

X < 0Xii = 1

X of rank 1

14


New formulation with the spherical constraint

Replace rank X = 1 by ‖X‖2 = (n + 1)2

New formulation of the k-densest subgraph problem as alinear SDP with one (nonconvex) quadratic constraint

max 〈Q,X〉〈Qj , X〉 = 4k − 2n, j ∈ {0, . . . , n}〈Ei, X〉 = 1, i ∈ {0, . . . , n}X < 0‖X‖2 = (n + 1)2

The difficulty is now concentrated in this spherical constraint...

Drop it: you get the usual SDP relaxation

Don’t want to do it: the SDP bound is tight, but expensive !

Idea: keep the constraint and dualize it !

15


Getting bounds by duality

Dualize the spherical constraint with α ∈ R

θ(α) :=

max 〈Q,X〉 − α(‖X‖2 − (n + 1)2)〈Qj , X〉 = 4k − 2n, j ∈ {0, . . . , n}〈Ei, X〉 = 1, i ∈ {0, . . . , n}X < 0

Weak duality: each θ(α) gives an upper bound

Comparison: θ(α) 6 θ(β) when α 6 β

No gap (!): in theory, bounds as tight as we want !?

θ(α) −→ val(dense subgraph) when α→ −∞

In practice: only θ(α) for α > 0 are tractable...

16




θ(α) :=







17




θ(α) :=







18




θ(α) :=







19




θ(α) :=







20


New family of SDP bounds

In fact, the useful new family of SDP bounds θ(α) with α > 0

Properties:

θ(0) is the standard SDP bound...

...computed by any SDP solver (IP, SB, PenSDP,...)

θ(α) for α > 0 boils down to a SDP least-squares problem...

...computed by nonlinear optimization methods (Malick ’04)

Key practical observation: θ(α) is easier than θ(0) to get !

But: θ(0) 6 θ(α) and θ(α) harder when α→ 0 ! So what ?

Need of a numerical study of the ratio tigthness/cpu cost

21


New family of SDP bounds

In fact, the useful new family of SDP bounds θ(α) with α > 0

Properties:

θ(0) is the standard SDP bound...

...computed by any SDP solver (IP, SB, PenSDP,...)

θ(α) for α > 0 boils down to a SDP least-squares problem...

...computed by nonlinear optimization methods (Malick ’04)

Key practical observation: θ(α) is easier than θ(0) to get !

But: θ(0) 6 θ(α) and θ(α) harder when α→ 0 ! So what ?

Need of a numerical study of the ratio tigthness/cpu cost

22

Relaxed resolution: comparison of the bounds

Outline




23


Technical point 1: how to choose ααα ?

Our strategy = having SDP-like bounds

We fix α = 10−4 – θ(10−4) ≈ θ(0)– cpu time is reasonable

Example on an instance (with n = 300, d = 25%, k = 75)Observe when α→ 0 : convergence to θ(0) + cpu increase

2200

2300

2400

2500

2600

2700

2800

0 0.002 0.004 0.006 0.008 0.01 1

1.2

1.4

1.6

1.8

2

2.2

2.4

θ(α)

CP

U T

ime

(s)

Value of α

SDP Bound

2200

2300

2400

2500

2600

2700

2800

0 0.002 0.004 0.006 0.008 0.01 1

1.2

1.4

1.6

1.8

2

2.2

2.4

θ(α)

CP

U T

ime

(s)

Value of α

SDP Bound

2200

2300

2400

2500

2600

2700

2800

0 0.002 0.004 0.006 0.008 0.01 1

1.2

1.4

1.6

1.8

2

2.2

2.4

θ(α)

CP

U T

ime

(s)

Value of α

SDP Bound

2200

2300

2400

2500

2600

2700

2800

0 0.002 0.004 0.006 0.008 0.01 1

1.2

1.4

1.6

1.8

2

2.2

2.4

θ(α)

CP

U T

ime

(s)

Value of α

SDP Bound

24


Technical point 1: how to choose ααα ?

Our strategy = having SDP-like bounds

We fix α = 10−4 – θ(10−4) ≈ θ(0)– cpu time is reasonable

Example on an instance (with n = 300, d = 25%, k = 75)Observe when α→ 0 : convergence to θ(0) + cpu increase

2200

2300

2400

2500

2600

2700

2800

0 0.002 0.004 0.006 0.008 0.01 1

1.2

1.4

1.6

1.8

2

2.2

2.4

θ(α)

CP

U T

ime

(s)

Value of α

SDP Bound

2200

2300

2400

2500

2600

2700

2800

0 0.002 0.004 0.006 0.008 0.01 1

1.2

1.4

1.6

1.8

2

2.2

2.4

θ(α)

CP

U T

ime

(s)

Value of α

SDP Bound

2200

2300

2400

2500

2600

2700

2800

0 0.002 0.004 0.006 0.008 0.01 1

1.2

1.4

1.6

1.8

2

2.2

2.4

θ(α)

CP

U T

ime

(s)

Value of α

SDP Bound

2200

2300

2400

2500

2600

2700

2800

0 0.002 0.004 0.006 0.008 0.01 1

1.2

1.4

1.6

1.8

2

2.2

2.4

θ(α)

CP

U T

ime

(s)

Value of α

SDP Bound

25


Numerical comparison of standard and new SDP bounds

We compare: θ(α = 10−4) vs. the standard SDP bound θ(0)

Solvers:

– our home-made solver for θ(α) (Malick-Roupin ’10)– SB, bundle method for λmax for θ(0) (Helmberg-Rendl ’00)– CSDP, interior-point method for θ(0) (Borcher ’99)

Test-problems: graphs of Billionnet-Elloumi-Plateau ’09

Collection with 5 instances of graphs for each parameters:

– number of vertices n ∈ {80, 100, 300}– density d ∈ {25%, 50%, 75%}– size of the subgraph k ∈ {n/4, n/2, 3n/4}

For given n : 45 instances (5 for each param. setting)

26


Numerical comparison of standard and new SDP bounds

We compare: θ(α = 10−4) vs. the standard SDP bound θ(0)

Solvers:

– our home-made solver for θ(α) (Malick-Roupin ’10)– SB, bundle method for λmax for θ(0) (Helmberg-Rendl ’00)– CSDP, interior-point method for θ(0) (Borcher ’99)

Test-problems: graphs of Billionnet-Elloumi-Plateau ’09

Collection with 5 instances of graphs for each parameters:

– number of vertices n ∈ {80, 100, 300}– density d ∈ {25%, 50%, 75%}– size of the subgraph k ∈ {n/4, n/2, 3n/4}

For given n : 45 instances (5 for each param. setting)

27


Comparison of the solvers (1)

a result = the mean over 45 instances of same size n

θ(α), α=10−4

n time gap(%)

80 0.15” 0.07%

100 0.19” 0.08%

300 2.81” 0.05%

θ(0) by SB θ(0) by CSDP

time θ(α) time θ(α)1.09” 0.76” 0.34” 0.28”

2.9” 1.63” 0.58” 0.47”

39.64” 25.15” 10.12” 8.05”

Our solver to compute θ(α) is

quick - especially for large problems

tight - mean gap with standard SDP 6 1%reliable - running times are almost constant for given size

Ex: for n = 100, mean standard deviation of cpu times

σθ(α) = 0.02 σSB = 2.32 σCSDP = 0.11

28




θ(α), α=10−4

n time gap(%)

80 0.15” 0.07%

100 0.19” 0.08%

300 2.81” 0.05%


time θ(α) time θ(α)1.09” 0.76” 0.34” 0.28”

2.9” 1.63” 0.58” 0.47”

39.64” 25.15” 10.12” 8.05”





σθ(α) = 0.02 σSB = 2.32 σCSDP = 0.11

29




θ(α), α=10−4

n time gap(%)

80 0.15” 0.07%

100 0.19” 0.08%

300 2.81” 0.05%


time θ(α) time θ(α)1.09” 0.76” 0.34” 0.28”

2.9” 1.63” 0.58” 0.47”

39.64” 25.15” 10.12” 8.05”



tight - mean gap with standard SDP 6 1%

reliable - running times are almost constant for given size


σθ(α) = 0.02 σSB = 2.32 σCSDP = 0.11

30




θ(α), α=10−4

n time gap(%)

80 0.15” 0.07%

100 0.19” 0.08%

300 2.81” 0.05%


time θ(α) time θ(α)1.09” 0.76” 0.34” 0.28”

2.9” 1.63” 0.58” 0.47”

39.64” 25.15” 10.12” 8.05”





σθ(α) = 0.02 σSB = 2.32 σCSDP = 0.11

31



Example: for a graph with n = 80, d = 50%, k = 40

490

500

510

520

530

540

0 0.2 0.4 0.6 0.8 1 1.2 1.4

Boun

d

CPU time (s)

"SDLS"

490

500

510

520

530

540

0 0.2 0.4 0.6 0.8 1 1.2 1.4

Boun

d

CPU time (s)

"SDLS"

490

500

510

520

530

540

0 0.2 0.4 0.6 0.8 1 1.2 1.4

Boun

d

CPU time (s)

"SB"

490

500

510

520

530

540

0 0.2 0.4 0.6 0.8 1 1.2 1.4

Boun

d

CPU time (s)

"SB"

490

500

510

520

530

540

0 0.2 0.4 0.6 0.8 1 1.2 1.4

Boun

d

CPU time (s)

"CSDP"

490

500

510

520

530

540

0 0.2 0.4 0.6 0.8 1 1.2 1.4

Boun

d

CPU time (s)

"CSDP"

32


Technical point 2: back to formulations

Performance of solvers depends on the formulations

2 equivalent {0, 1}-QP for densest subgraph problem...max y>W y

e>y = k(∑

yi − k)2 = 0y ∈ {0, 1}n

max y>W y

e>y = ky>Cj y = k yj , j = 1:ny ∈ {0, 1}n

...lead to 2 equivalentSDP relaxations...

...for which the solversbehave differently !

Choose the best for eachsolver: CSDP vs θ, SB

SDLS θs(10-4) SDLS θp(10-4) SB θs(0) SB θp(0) CSDP θs(0) CSDP θp(0)0

1

2

3

4

5

6

7

8

Graph Size = 100

CPU

Tim

e

33





e>y = k(∑

yi − k)2 = 0y ∈ {0, 1}n

max y>W y

e>y = ky>Cj y = k yj , j = 1:ny ∈ {0, 1}n





1

2

3

4

5

6

7

8

Graph Size = 100

CPU

Tim

e

34





e>y = k(∑

yi − k)2 = 0y ∈ {0, 1}n

max y>W y

e>y = ky>Cj y = k yj , j = 1:ny ∈ {0, 1}n





1

2

3

4

5

6

7

8

Graph Size = 100

CPU

Tim

e

35

Exact resolution: branch-and-bound procedure

Outline




36


Simple branch-and-bound

Characteristics of our branch-and-bound algorithm:

Initialization: greedy algorithm gives

– (good) feasible point– lower bound on optimal solution

Extremely simple branching strategy:

– fixed order of separation– depth-first

Bounding strategy: new SDP bound θ(α) with α = 10−4

– Our solver admits early stops– Warm-restart for sub-problems

Expect the bounding, the rest is rather standard

37


Comparison with the best to get densest subgraphs

Comparison with QCR of Billionnet-Elloumi-Plateau ’09

Quadratic programming approach that mixes nicely– SDP for computing parameters of convex relaxation– CPLEX for branch-and-bound (MIQP to be solved)

Numerical comparison: same instances - same machine

Aggregated results

40 80 100

0

100

200

300

400

500

600

700

800

SDLSQCR

Graph size

Ave

rag

e

CP

U ti

me

38


Comparison with the best to get densest subgraphs

Comparison with QCR of Billionnet-Elloumi-Plateau ’09

Quadratic programming approach that mixes nicely– SDP for computing parameters of convex relaxation– CPLEX for branch-and-bound (MIQP to be solved)

Numerical comparison: same instances - same machine

Aggregated results

40 80 100

0

100

200

300

400

500

600

700

800

SDLSQCR

Graph size

Ave

rag

e

CP

U ti

me

39


Numerical comparison: some more details

80 n

/4 2

5%

80 n

/4 5

0%

80 n

/4 7

5%

80 n

/2 2

5%

80 n

/2 5

0%

80 n

/2 7

5%

80 3

n/4

25%

80 3

n/4

25%

80 3

n/4

75%

100

n/4

25%

100

n/4

50%

100

n/4

75%

100

n/2

25%

100

n/2

50%

100

n/2

75%

100

3n/4

25%

100

3n/4

25%

100

3n/4

75%

0

500

1000

1500

2000

2500

SDLSQCR

Comparable results - with different strategies !

Our bound is more expensive but we prune very well

Ex: nb of nodes in tree for n = 100: 48,675 vs 950,04140


Conclusion on numerical experiments

The bounds θ(α) are interesting– they provide SDP-quality bounds– they are cheaper to get: good ratio tightness/computing-time

The solver for θ(α) combines advantages of SB and CSDP– gives guaranteed upper bounds (like SB)– has a sharp initial decrease (like SB)– is reliable (like CSDP)– we can interrupt it (like SB)

The branch-and-bound using θ(α)– uses SDP-like bounds (all way long) so prunes very well– has performances comparable with the best (that uses CPLEX)

J. Malick and F. RoupinNumerical study of SDP bounds for the k-cluster problemElectronic Notes in Discrete Mathematics: Proceedings of ISCO, 2010

J. Malick and F. RoupinSolving k-cluster problems to optimality with semidefinite programmingTo appear in Mathematical Programming, 2011

41


Conclusion...

Essential points of this work– new formulation of rank-one constraint

– new SDP-like bounds for the densest k-subgraph

– competitive branch-and-bound to compute dense subgraphs

On-going research on these new bounds– generalisation 1: universality using α (Malick Roupin ’11)

– generalization 2: to other combinatorial problemseasy, in theory... but still requires work, in practice...

– densest k-subgraph: simple formulation but challenging forpure-SDP approach (no SDP approach)

Advertisement: next talk of Nathan Krislock on max-cut– different presentation of the family + details on computation

– max-cut admits SDP-based B&B (Wiegele et al ’09)

– managment of inequalities + control on α...

thanks !

42


Conclusion...










thanks !

43


Conclusion... and next talk !










thanks !

44


Conclusion... and next talk !










thanks !

45

Computing dense subgraphs with semidefinite programming · Finding a densest k-subgraph ? Diﬃcult...

Documents

Transcript of Computing dense subgraphs with semidefinite programming · Finding a densest k-subgraph ? Diﬃcult...