A conic cutting surface method for linear-quadratic ... · History of ACCPM Sonnevend (1988), New...

Advanced Optimization Laboratory

McMaster University

A conic cutting surface method for linear-quadratic-

semidefinite programming

Mohammad R. Oskoorouchi

California State University San Marcos

San Marcos, CA

Joint work with

John E. Mitchell

RPI


McMaster University

July 3, 2008

Outline:

Second-order cone: definition and properties

History and recent works

Second-order cone cutting surface method

Optimality conditions

The updating direction: linear cut

The updating direction: multiple SOCC

Complexity results

Linear-quadratic-semidefinite programming

Computational experience


McMaster University

0 0: ( ; ), n

n x x x x x x

Second-order cone:

is a closed convex conen

0:n nx x x

Notations:

iff

iff

n

n

n

n

x y x y

x y x y


McMaster University

Some functions

0 0: ( ; ),

n

nx x x x x x

1 2 0

22

1 2 0

2 2

1 2

1 22

( ) : 2

det( ) :

:

: max ,

F

tr x x

x x x

x

x

2

0

0

2

2 det( ) 2

T

x T

x x xQ

x x x I x x

The analogous operator to in symmetric matrix algebra is the one that maps

any symmetric matrix

into

xQ

S XSX


McMaster University

0 0

1 0 0

1 2

2

1

0 0

and are eigenvalues of

If and then is invertable and

( ; )

( ; ) , ( ; )

0, 0

: (1;

,

0

)

n n

Tx s x s x s s x

x x x x x

x x e

x x x s s s

x

Block Cases

1 2

1x

(x) 2e x ( )

k

k

n n n

x x

T

i

Q Q Q

tr tr x

1 1x ( ; ; ) s ( ;Let and , where ; ) ,ik k i i nx x s s x s

1

22

2

e ( ; ; )

det(x) det( )

x

x max

k

i

iF F

i

e e

x

x

x


McMaster University

History of ACCPM Sonnevend (1988), New algorithms in convex programming based on a notation

of center and on rational extrapolations.

Goffin, Haurie, and Vial (1992), Decomposition and nondifferentiable optimization

with the projective algorithm.

Ye (1992), A potential reduction algorithm allowing column generation.

Ye (1997), Complexity analysis of the analytic center cutting plane method that

uses multiple cuts.

Goffin and Vial (2000), Multiple cuts in the analytic center cutting plane methods.

Luo and Sun (2000), A polynomial cutting surfaces algorithm for the convex

feasibility problem defined by self-concordant inequalities.

Toh, Zhao, and Sun (2002), A multiple-cut analytic center cutting plane method for

semidefinite feasibility problems.

Oskoorouchi and Goffin (2003), The analytic center cutting plane method with

semidefinite cuts.

Oskoorouchi and Goffin (2005), An interior point cutting plane method for the

convex feasibility problem with second-order cone inequalities.


McMaster University

Recent Works on ACCPM

Sivaramakrishnan (2007), A parallel dual decomposition algorithm for polynomial

optimization problems with structured sparsity

Oskoorouchi and Goffin (2007), A matrix generation approach for eigenvalue

optimization

Oskoorouchi and Mitchell (2008), A second-order cone cutting surface method:

complexity and application

Sivaramakrishnan, Martinez, and Terlaky (2006), A conic interior point

decomposition approach for large scale semidefinite programming

Babonneau and Vial (2006), ACCPM with a nonlinear constraint and an active set

strategy to solve nonlinear multicommodity flow problems

Babonneau, Merle and Vial (2006), Solving large scale linear multicommodity

flow problems with an active set strategy and Proximal-ACCPM


McMaster University

Second-order cone cutting surface method

Let

: , and Am T T

y y y cΑ c

1 2

1 2 1 2

be a compact convex set.

Assume:

1) contains a full dimensional ball with radius.

2) There is an oracle that returns cuts.

, ,w , , ; ; ; ,here

W

l

q q

q

m n

n n

n n n

A A A c c c AA c

e are interested in finding a point in this ball


McMaster University

Analytic Center


McMaster University

It is easy to verify that is a strictly concave function on the interior of . Therefore the

maximizer of this function exists and is unique. This maximizer is called the .

From the KK

analytic center

1 1

T optimality conditions is the analytic center of if and only if there exists

0 and 0 such that

0

y

x s

Ax

x s

Ax

where

1( , ) log det log ,

2

: 0,

: 0.

T

T

s s

y

s c A y

s s

s c A

Define the dual barrier function

Analytic Center

Th

e pr

imal bar

rier function:

over

is also strictly concave. Therefore has a unique maximizer over .

1( , ) : log det( ) log

2

, : 0q l

T T

n n

x c x x

x Ax

x c x x

x Ax


McMaster University

The Cartesian product of and gives the primal-dual set of

localization. The corresponding barrier function is defined via

( , , , ) ( , ) ( , ) x s x sx s x s

Optimality Conditions:

The analytic center is uniquely defined:

1

1

0

:

Optimality C

0

: 0

onditions:

T

T

Ax

y

s c A y

x s

Ax

s c - A

x s


McMaster University

Optimality Conditions:

The analytic center is uniquely defined:

1/2

2 2

Approximate analytic cente

0

0

1

r

T

T

F

Ax

y

s c A y

Q xsx

Ax

s c A 0

s e 1


McMaster University

Newton direction:

2

1

Let a strictly feasible point in be given. Since is strictly

concave on , implementing Newton's method to maximize

over yields

The analytic center of the dual then reads

,

,

x

d Q

d x X s

y G g

x xx s

2

2

wh re

.

e

T TG Q AX A

g Q AX c

x

x

A A

A c


McMaster University

So far:


McMaster University

We defined the dual set of localization

and the primal set of localization

and discussed how to compute the analytic center.

: , and A

, : 0q l

m T T

n n

y y y c

x Ax

Α c

x Ax

The algorithm:0 0

, : 0

Let ( , ), a strictly feasible point of (Primal Set of Localization) be given

. Compute , an approximate analytic center of and , an approximate

center of

.

, )

(

q ln n

x A

x

y

x

x

x

S

x A

tep 1

Step

x

2

x

Call the oracle. If is in the stop.

. If the oracle returns a single linear cut , update

. If the oracle returns multiple second-order cone cu

, , 0 : 0

t

-

q ln n

T

T

x Ax

y ball

b d

b

yStep 3

Step 4

x Ax

B

, update

. Find a strictly feasible point of and re

,

turn to S

, :

e t p 1

0

.

q l

p

n np x Ax

y

x u

Ste

u

d

p 5

Ax B


McMaster University

The updating directions: a single LC

1. , where , , and

is a strictly feasible point of .

2. Starting from this point, Newton steps suffices to obtain

an approximate analytic center of

, , d 1-

(1)

xx x x d

O

xx x x

Result 1:

1/2

. .

2 2

max log

d 0

d 1

s t

x

xF

Ad b

Q Sd

x

xs

A

, , 0 : 0q ln n

x Ax bx Ax

Mitchell-Todd direction:


McMaster University

The updating directions: multiple SOCC

1. , where , , and

is a strictly feasible point of .

2. Startin

, , d 1-

( log( 1)g from this point, Newton steps

suffices to obtain an approximate analytic center of

)

xx x x d

O p p

xx u x x

Result 2 :

1/2

. .

2 2

1max log det

2

d 0

d 1

s t

x

xF

Ad

Q Sd

x

xs

u

A Bu

, , : 0s ln np x Axx u Ax Au


McMaster University

Complexity:

3

2

The analytic center cutting surface algorithm finds a point in

the -ball, when the total number of linear and second-order

cone cuts reaches the boun

,

d

mpO

Result 3 :

where is the maximum number of second-order cone cuts

added at the same time and is a condition number of a field

of c s

0

t

u .

p


McMaster University

Linear-quadratic-semidefinite programming problem:


McMaster University

min

. .

,

1,

0, 0, 0

T TC X c x

s t

X Ax b

I X

X x

c x

Ax

x

max

. .

,

,

T

T

T

T

b y z

s t

y zI C

y

A y c

A c

1

Parameters:

, , ,

, , ,

: , ( ) ,

: ,

qs l

q l

s

s

nn n

m n m n m

n m

i i

mnT m T

i i

i

C c

A b

X A X

y y A

c

A

Variables:

, , ,qs lnn n m

X x yx

Primal Dual

Linear-quadratic-semidefinite programming problem:


McMaster University

max

. .

,

,

T

T

T

T

b y z

s t

y zI C

y

A y c

A c

minmax ( )

. .

,

T T

T

T

b y C y

s t

y

A y c

A c

min

The minimum eigenvalue of a symmetric matrix can be cast as an SDP:

( ) (

( ) min ( ) | (

)

) 1, 0T

T

T

T

f y b y C y

f y b y C y

V tr V V

Min-eigenvalue function:


McMaster University

ˆ

min

min

ˆ

Using the Clark generalized gradient of and a cahir rule:

where the orthonormal columns of are the eigenvectors corresponsing to

( - )

( -

( ) | ( ) , ( ) 1,

n p

T

T

m T p

i i

Q

C y

C y

f y b v v Q AQ V tr V V

with multiplicity ˆ) .p

0

0 1

Let

where is large enough to ensure contains the optimal solution.

( ; ) : , , ,m T T T

y z y A y c b y zA c

From optimization to feasibility:


McMaster University

0 0 0 0

0

0

0

ˆLet , be an initial query point. If is differentiable at , then 1 and

reduces to a column vector , and is up

(

dated by

ˆ ˆwhere , w

ˆ; ) : , max( , ( ))

,

i

ˆ

th

T T

m T

i i

y b

y z f y p Q

q

z b y z f y

b

y

q

z

A

d

b ˆ, 1, ... , and .T

q i m d q Cq

0 0

0 0

( ; ) : , max( , ( )

ˆIf is nondifferentiable at , then 1 and is updated by

ˆ ˆˆ ˆˆwhere is a dimensional semidefinite inequality, and ˆand , 1,

,ˆ ˆ )T T

i i

T

T

i i

T

f y p

y zI D p y y B

B

yy z b y z

Q A Q i

f yzI D

ˆ..., , .

T

m D Q CQ

Second-order cone relaxation

302

2

a ba c a b

a b a bc b c

c

On the other hand, every 2 2 principle submatrix of a positive

semidefinite matrix is positive semidefinite

Now consider the semidefinite inequality .

Consider the 2 2 principle submatrix in locations and ,

ˆ ˆT

i j i j

y zI D


McMaster University


1

ˆ ˆ ˆ ˆ 1 00

ˆ ˆ ˆ ˆ 0 1

k kmii ij ii ij

k k kkij jj ij jj

D D B By z

D D B B

ˆ ˆ ˆ ˆ0

ˆ ˆ ˆ ˆ

k k

ii k ii ij k ij

k k

ij k ij jj k jj

D y B z D y B

D y B D y B z

3

ˆ ˆ ˆ ˆ 2

ˆ ˆ ˆ ˆ

ˆ ˆ2 2

k k

ii jj k ii jj

k k

ii jj k ii jj

k

ij k ij

D D y B B z

D D y B B

D y B

ˆ ˆT y zI D


McMaster University

1

k

1

1 1

whe

ˆ

re

contains blocks of SOCC's,

contains linear cuts,

ˆ ˆˆ ˆ: 2 , (

a

) , an

nd

2

ˆ ,

max ,

d ,

kl k

Tk m k k k T k T

l

k

q ii

m

k

k

nk

k k

n p

n

y y z A y z c y

A

y

b

f

zA e

A

c 1

Therefore a semidefinite cut can be relaxed

into second-order cone cuts. Therefore at each

iteration, the set of loca

ˆ dimensional

ˆ ˆ( 1

lization has the fo

)

r

2

m

p

p pp



McMaster University

( ) min{ : 1,..., }T

i if y a y b i m

1( , ) : , and my y y

A simple illustration:


McMaster University

Stopping criterion:


McMaster University

The lower bound is updated at each iteration by construction. Given the definition of , at

the th iteration the following relaxation of the dual is formed:

k

k

max

. .

ˆ ˆ2

ˆ ˆ( )

T

Tk k

k T k

T k

b y z

s t

y z

A y z c

b y z

A e c

1

Stopping criterion:


McMaster University

An upper bound for this problem is obtained by evaluating the objective function of the restricted

primal problem at a feasible point:

ˆ ˆmin ( ) ( )

. .

ˆ ˆ

2 1

0, 0, 0

k T k T k

k k

c x

s t

A x b b

x

x

c x

A x

ex 1

x

On the other hand from optimality conditions of the analytic center we have

ˆ ˆ 0

2 0

k k k k k

k k k

A x b

x

A x

ex 1

Therefore ( / , / , 0) is feasible for restricted primal p

1ˆ ˆ: ( ) (

roblem and

is the updated upper bo d

)

un .

k k T k

k k k

T

k

k

c x

x

c x

x

Convergence behavior:


McMaster University

lc socc p dim gap SOCCSM SDPLR SDPT3 SeDuMi

300, 50, 100 14 46 3 109 9.2e-4 16 29 69 352

300, 100, 600 4 31 4 121 7.8e-4 56 78 225 843

300, 200, 1100 4 44 4 169 8.6e-4 191 146 503 1834

300, 300, 800 11 79 8 518 9.2e-4 739 -- 656 --

300, 300, 1000 5 62 7 335 9.2e-4 561 -- 747 --

500, 50, 200 4 33 4 128 7.6e-4 77 144 247 1245

500, 100, 1000 9 21 3 69 7.3e-4 67 358 1227 2504

500, 200, 500 11 81 5 384 9.1e-4 643 -- 1555 --

500, 200, 2000 9 30 3 94 8.6e-4 398 -- 1791 --

500, 300, 1000 1 44 7 221 4.7 646 -- 2558 --

800, 10, 800 8 1 2 10 4.2e-4 7 159 553 2798

800, 50, 500 1 22 3 58 9.1e-4 77 433 1453 3449

800, 100, 800 2 32 4 114 9.4e-4 313 -- 2811 --

800, 200, 1000 1 28 4 107 3.4e-3 467 -- 5814 --

800, 200, 1500 1 24 5 98 4.5e-3 424 -- 6332 --

, ,s ln m n


McMaster University

lc socc p dim gap SOCCSM SDPLR SDPT3 SeDuMi

1000, 10, 500 2 7 2 16 7.6e-4 14 227 986 3995

1000, 10, 400 11 17 2 45 8.4e-4 110 -- 2097 --

1000, 50, 900 1 19 2 39 8.8e-4 69 -- 2553 --

1000, 100, 500 3 46 3 137 3.9e-3 223 -- -- --

1000, 100, 1000 0 20 3 49 4.3e-3 134 -- -- --

2000, 10, 100 1 12 2 25 7.3e-4 112 -- 4929 --

2000, 10, 500 1 8 2 17 7.1e-4 65 -- 5996 --

2000, 10, 1000 0 7 2 14 4.8e-4 59 -- 6617 --

2000, 20, 100 1 13 3 34 2.6e-3 177 -- 8136 --

2000, 20, 800 1 7 2 15 3.7e-3 70 -- 9050 --

2500, 10, 100 2 6 2 14 4.5e-3 104 -- 9999 --

2500, 20, 50 4 18 3 41 4.3e-3 368 -- 9999 --

2500, 20, 500 6 2 2 10 4.8e-3 87 -- 9999 --

3000, 10, 100 1 9 2 19 4.3e-3 177 -- -- --

3000, 10, 500 0 6 2 12 3.2e-3 106 -- -- --

, ,s ln m n


McMaster University

A conic cutting surface method for linear-quadratic ... · History of ACCPM Sonnevend (1988), New...

Documents

Transcript of A conic cutting surface method for linear-quadratic ... · History of ACCPM Sonnevend (1988), New...