Convex Quadratic Programming for Object Location

Hao Jiang, Mark S. Drew and Ze-Nian Li

School of Computing ScienceSimon Fraser University

2

Introduction Object localization is an important task in computer vision

Template

3

Object localization and labeling Object localization can be formulated as labeling problems.

Consistent labeling Find small cost label assignment. Enforce labeling consistency of neighboring sites.

p

q

fp

fq

c(p,fp)

c(q,fq)Template Target Object

Site Label

Neighboring relation

(q-p) (fq-fp)

4

Previous methods for consistent labeling

Consistent labeling, in general form, is NP-hard. Polynomial time schemes exist for special cases

Dynamic Programming (DP). Max-flow [Ishikawa 2000, Roy 98].

Approximation schemes Greedy (local searching) methods

Relaxation labeling (RL) [Rosenfeld 76]. Iterated Conditional Modes (ICM) [Besag 86]. Need good initialization and easily trapped in local minimum.

Global searching methodsGraduated Non-Convexity (GNC) [Blake & Zisserman 87] Belief Propagation (BP) [Pearl 88, Weiss 2001].Graph Cut (GC) [Boykov & Zabih 2001].

5

Focus of the research

(# of labels in the order of hundred)

(# of labels can be in theorder of thousand or bigger)

Many vision problems such as object matching, large scale motion,

tracking etc will benefit from the solver.

Wellsolved

Hard to solve The focus of

the research

Consistent labeling

Previous methods become slow as the number of labels goes to the order of several thousand.

6

The trick of the proposed scheme In stead of working on the original label space, we

represent the label set with a small number basis labels. We convert the hard problem into a sequence of simpler

problems built using only the basis labels. In this way, the size of the approximation problem is largely

decoupled from the original label set. Each sub-problem is a convex problem and can be globally

optimized. A successive relaxation implementation is used to zero in

the target.

7

The non-linear optimization problem

The labeling problem can be solved by optimizing:

p

q

fp

fq

c(p,fp)

c(q,fq)

(q-p) (fq-fp)

8

Convex relaxation

To convert the labeling cost term into linear functions, for each site, we define a basis label set. Each label can then be represented as a linear combination of the

basis labels. The cost of the label is approximated by the linear combination of

the costs of these basis labels.

t

c(s,t)

J1 J2fs

fs = a * J1 + b * J2

c(s,fs) = a * c(s,J1) + b * c(s,J2)a + b = 1

9

Convex relaxation (Cont’)

The L2 norm smoothness terms do not need additional conversions.

j2 j3

j4j5j1j1 s,j1

+ j2 s,j2+

j3 s,j3

+ j4 s,j4

+ j5 s,j5

c(s,j1)

c(s,j2)

c(s,j3)

c(s,j4)c(s,j5)

c(s,j1)s,j1+ c(s,j2) s,j2

+ c(s,j3) s,j3

+ c(s,j4) s,j4

+ c(s,j5) s,j5

s,j1+ s,j2

+ s,j3

+ s,j4

+ s,j5

=1

10

Convex quadratic program (CQP)

We have the convex quadratic program:

11

In some cases, the CQP is exact

If are binary numbers, the mixed integer program is exactly equivalent to the original problem.

If c(s,t) is convex over t for each s 2 S, the CQP is equivalent to the continuous extension of the original problem.

t t

c(s,t) c(s,t)

Feasible solution Feasible solution

Continuousextension

12

In general it is an approximation

For general problems, the CQP approximates each labeling cost surface with the 3D lower convex hull.

t =(x,y)

c(s,t)

The convexified surfaces are much simpler than the original ones.

x

y

Label cost

13

The CQP can be greatly simplified

The most compact basis labels correspond to the lower convex hull vertices.

This shows a way to simplify the label set in labeling. We can safely discard many labels without worrying about

the problems met in previous methods.

# of original label: 400# of basis label: 20

14

Successive convexification

To improve the approximation we use the successiveconvexification (SC) scheme as follows:

15

Experimental results

Matching Random Dots

16

Experimental results (Cont’)

Matching Random Dots

17

Experimental results (Cont’)

Matching Leaf

18

Experimental results (Cont’)

Matching Face

19

Experimental results (Cont’)

Matching Hand

20

Conclusion Set out an object localization method which can deal with

textureless objects in strong background clutter. Propose a successive convex quadratic method CQP large decouples the number of target image points with the

size of the convex program – It searches the whole target image quickly.

Successive convexification is studied to improve the result recursively.

Successive convexification is a general method can be applied to any convex regularization problem.

21

Thank you!

22

In summery

Label space is approximated with small number of basis labels. Original problem is converted to a sequence of much easier convex

programs and solved by successive relaxation.

The size of the convex program is largely decoupled with the number of candidate labels

Convexification Trust Region Shrinking

Template

Target Image

Matching Costs

LowerConvexHull