Metric Embedding with Metric Embedding with Relaxed GuaranteesRelaxed Guarantees

Ofer NeimanOfer Neiman

Ittai Abraham Yair BartalIttai Abraham Yair Bartal

Embedding metric spacesEmbedding metric spaces

Representation of the metric in a simple and Representation of the metric in a simple and structured space.structured space.

Common target spaces: Common target spaces: llpp, trees (ultra-metrics)., trees (ultra-metrics).

The price of simplicity: The price of simplicity: distortiondistortion, which is the , which is the multiplicative amount by which distances can change.multiplicative amount by which distances can change.

Goal: find low distortion embeddings.Goal: find low distortion embeddings.

A tool for approximation algorithmsA tool for approximation algorithms

Useful for many practical applications.Useful for many practical applications.

Metric embeddingMetric embedding

LetLet X,Y X,Y be metric spaces with metrics be metric spaces with metrics ddxx, d, dyy

respectively.respectively. f : X→Y f : X→Y is an embedding of is an embedding of X X into into YY..

cvud

vfufdc

X

Y

,,

:Xvu

The distortion of The distortion of f f is the minimal is the minimal αα such that for such that for some some cc::

Every metric space on Every metric space on n n points can be embedded into points can be embedded into Euclidean spaceEuclidean space with distortion O(log with distortion O(log nn) and ) and dimension O(logdimension O(log22nn). [Bourgain/LLR]). [Bourgain/LLR]

Basic resultsBasic results

Every metric space on Every metric space on n n points can be embedded points can be embedded into a tree metricinto a tree metric with distortion with distortion

[Bartal/BLMN/RR].[Bartal/BLMN/RR].

n

problemproblem

The lower bounds on the distortion and the dimension The lower bounds on the distortion and the dimension are high, and grow with are high, and grow with nn..

In some cases, weaker guarantees are acceptable..In some cases, weaker guarantees are acceptable..

Some Alternative SchemesSome Alternative Schemes

Probabilistic embeddingProbabilistic embedding: : considering the expected considering the expected distortion. distortion. [Bartal, FRT][Bartal, FRT]

Ramsey theoremsRamsey theorems: : embedding a large subspace of the embedding a large subspace of the original metric. original metric. [BFM, BLMN][BFM, BLMN]

Partial embeddingPartial embedding: : embedding all but a fraction of the embedding all but a fraction of the distances.distances. [KSW, ABCDGKNS][KSW, ABCDGKNS]

MotivationMotivation Estimating latencies (round-trip time) in the internet.Estimating latencies (round-trip time) in the internet.

- the distance matrix is almost a metric.- the distance matrix is almost a metric.

- embedding heuristics yield surprisingly good results...- embedding heuristics yield surprisingly good results...

[Ng+Zhang’02, ST’03, DCKM’04][Ng+Zhang’02, ST’03, DCKM’04]

Practical network embedding requires:Practical network embedding requires:

- Small number of dimensions.- Small number of dimensions.

- No centralized co-ordination.- No centralized co-ordination.

- Linear number of distances measurement.- Linear number of distances measurement.

Finding nearest: copy of a file, service from some server, ect.Finding nearest: copy of a file, service from some server, ect.

(1-(1-εε) partial embedding) partial embedding

X, Y X, Y are metric spaces.are metric spaces.

f : X→Yf : X→Y has ( has (1-1-εε) ) partial partial distortiondistortion at at

most most αα if there exists a set of pairs if there exists a set of pairs GGεε such that: such that:

For all For all pairs (pairs (u,vu,v))єєGGεε..

cvud

vfufdc

X

Y

,,

21

nG

Scaling EmbeddingScaling Embedding

A stronger requirement is a map that will be good for A stronger requirement is a map that will be good for all all εε simultaneouslysimultaneously..

Definition:Definition: an embedding an embedding ff has has scalingscaling distortion distortion DD((εε) if for ) if for anyany εε>0>0, it is an (1-, it is an (1-εε) partial embedding ) partial embedding with distortion with distortion DD((εε).).

Scaling & Average DistortionScaling & Average Distortion

Thm:Thm: every metric space has every metric space has scaling probabilistic embedding scaling probabilistic embedding with distortion with distortion OO(log((log(1/1/εε)) into trees.)) into trees.

Thm:Thm: every metric space has every metric space has scaling embedding with scaling embedding with distortion distortion OO(log((log(1/1/εε)) and dimension )) and dimension OO(log (log nn)) intointo Euclidean Euclidean space.space.

implies constant average distortion!implies constant average distortion!

Applications:Applications: weighted average problems weighted average problems

sparsest cut, quadratic assignment, linear arrangement, ect.sparsest cut, quadratic assignment, linear arrangement, ect.

Any Any λλ-doubling metric space -doubling metric space X X can be embedded into can be embedded into ll22 with (1- with (1-εε) partial distortion ) partial distortion [KSW].[KSW].

Previous workPrevious work

DefinitionDefinition: a metric space : a metric space XX is called is called λλ--doubling if for any doubling if for any r>0r>0, any ball of , any ball of radius radius rr can be covered by can be covered by λλ balls of radius balls of radius r/2.r/2.

logO

Partial embedding into trees with distortion .Partial embedding into trees with distortion . Distortion & Dimension don’t depend on Distortion & Dimension don’t depend on

the size of the size of XX!!

Our ResultsOur Results Partial embedding into Partial embedding into ll22 with distortion and dimension with distortion and dimension

OO(log(1/(log(1/εε)).)).

1O

General theorem converting classical General theorem converting classical llpp embeddings into embeddings into

the partial model.the partial model.

Tight lower bounds. Tight lower bounds.

Appeared in FOCS05 together with CDGKSAppeared in FOCS05 together with CDGKS

Thm:Thm: Any Any subset-closedsubset-closed family of metric spaces family of metric spaces X X , that has, that has for any for any XXєєX X on on n n points, an embedding points, an embedding φφ::XX→ → llpp with with

- distortion - distortion αα((nn).). - dimension - dimension ββ((nn).).

φφ can be converted into (can be converted into (1-1-εε) partial embedding of ) partial embedding of XX with with

- distortion- distortion

- dimension- dimension

Embedding into Embedding into l lpp

1

log1

O

1

log1

log1

O

1O

1O

In practice..In practice..

(1-(1-εε) partial ) partial embedding of any metric space into embedding of any metric space into llpp with distortion with distortion

and dimension [and dimension [Bourgain,Matousek,Bartal]Bourgain,Matousek,Bartal]

(1-(1-εε) partial ) partial embedding of any negative type metric (embedding of any negative type metric (ll11 metrics) metrics) into into ll22 with distortion and dimension with distortion and dimension

[ARV, ALN][ARV, ALN]

(1-(1-εε) partial ) partial embedding of any doubling metric into embedding of any doubling metric into llpp with with distortion and dimension distortion and dimension [KLMN][KLMN]

(1-(1-εε) partial ) partial embedding of any tree metric into embedding of any tree metric into ll22 with distortion with distortion and dimension and dimension [Matousek] [Matousek]

Main ResultsMain Results

p

O1log

1logO

1loglog1logO

1log2O

1log1 pO 1log2O

1loglogO

1logpOe

Definitions Definitions

Let Let rrεε((uu)) be the minimal radius such that be the minimal radius such that ||B(B(u,ru,rεε((uu))| ))|

≥ ≥ εεn.n.

A pair (A pair (u,vu,v), ), w.l.o.g w.l.o.g rrεε((uu) ≥ ) ≥ rrεε((vv):):

has has shortshort distance distance if if dd((u,vu,v) ) << rrεε((uu))

has has medium medium distance if distance if rrεε((uu) ≤ ) ≤ dd((u,vu,v) < 4∙) < 4∙rrεε((uu).).

has has long long distancedistance if 4·if 4·rrεε((uu) ≤ ) ≤ dd((u,vu,v).).

Close DistancesClose Distances

((u,vu,v) is a short pair. ) is a short pair.

Short pairs are Short pairs are ignored - ignored - at most at most εεnn22..

urε(u)

vrε(v)

rε(w)w

Beacon Based EmbeddingBeacon Based Embedding

Randomly choose Randomly choose beacons = beacons = BB..

Each point attached to Each point attached to nearest beacon.nearest beacon.

1

log1

Some More Bad PointsSome More Bad Points

If If dd((u,Bu,B) > ) > rrεε((uu) then) then

is is badbad..

For each For each uuєєXX : :

With probability ½ at With probability ½ at most most 22εεnn22 bad pairs.bad pairs.

urε(u)

vrε(v)

rε(w)w xux ,:

urBud ,Pr

Partial EmbeddingPartial Embedding

Use the embedding Use the embedding φφ::B→lB→lpp.. φφ has distortion guarantee of has distortion guarantee of .. The partial embedding The partial embedding is:is:

1

log1

B

plXhf :

φφ((bb))

1

log1

1

logOk

hh((uu))f f ((uu))

u u attached to attached to beacon beacon bb

0,1

pk

ur 0,1

pk

ur 0,1

pk

ur

bv

bu

Upper BoundUpper BoundWe assume for the pair We assume for the pair (u,v(u,v))::

- Each point has a beacon in its ball.- Each point has a beacon in its ball.

- Both - Both u,vu,v are outside each other’s ball. are outside each other’s ball.

- The mapping - The mapping φφ is a contraction. is a contraction.

urε(u)

rε(v)v

vudbbd vu ,3,

pvu

p

pvu bbdbb ,

ppp

pvudvfuf , 3

pp

pvudvhuh ,

bv

dd((u,vu,v) ) ≥ ≥ 4·max{4·max{rrεε((uu), ), rrεε((vv)})}

Lower Bound - Long DistancesLower Bound - Long Distances

rε(u)u

rε(v)v

bu

d(bd(buu,b,bvv) ≥ ) ≥ dd((u,vu,v)/2 )/2

p

vup

pvu B

bbdbb

,

pp

p

vudvfuf

1log12

,

u

v

Medium Distances??Medium Distances??

There is a problem in this case:There is a problem in this case:

rε(u)

rε(v)

u,v u,v are attached to the same beacon!!are attached to the same beacon!!

The additional coordinates The additional coordinates hh will will guarantee enough contribution..guarantee enough contribution..

Medium DistancesMedium Distances

With probability < With probability < εε the pair the pair ((u,vu,v) will be smaller than half ) will be smaller than half its expectation.its expectation.

Pairs satisfying: Pairs satisfying: rrεε((uu)) ≤≤ d(u,vd(u,v) ≤ 4) ≤ 4rrεε((uu))

[w.l.o.g [w.l.o.g rrεε((uu) ≥ ) ≥ rrεε((vv)) ] ] rrεε((uu),0),0 rrεε((uu),0),0 rrεε((uu),0),0

rrεε((vv),0),0 rrεε((vv),0),0 rrεε((vv),0),0

rrεε((uu)) 00 rrεε((uu))

In expectation In expectation ¼ of the ¼ of the coordinates will be coordinates will be rrεε((uu). ).

With probability With probability ¼ we get ¼ we get rrεε((uu))

hh((uu)-)-hh((vv))

Medium DistancesMedium Distances

With probability With probability ½, ½, 22εεnn22 medium pairs failed, but for medium pairs failed, but for the others:the others:

ppp

pvudurvhuh ,

End of proof!End of proof!

Coarse Partial EmbeddingCoarse Partial Embedding

Another version: ignoring only the short distances Another version: ignoring only the short distances (i.e., from each point to its nearest (i.e., from each point to its nearest εεnn neighbors). neighbors). the dimension increases to the dimension increases to OO(log((log(nn)·)·ββ((11//εε)).)).

Partial Embedding into TreesPartial Embedding into Trees

Thm:Thm: every metric space has (1- every metric space has (1-εε) partial embedding ) partial embedding with distortion into a tree (ultra-metric).with distortion into a tree (ultra-metric). 1O

Ultra-metricsUltra-metrics

Metric on leaves of Metric on leaves of rooted labeled tree.rooted labeled tree.

0 ≤ 0 ≤ ΔΔ((DD) ≤ ) ≤ ΔΔ((BB) ≤ ) ≤ ΔΔ(A).(A). dd((x,yx,y) = ) = ΔΔ((lcalca((x,yx,y)).)).

dd((x,yx,y) = ) = ΔΔ((DD).).

dd((x,wx,w) = ) = ΔΔ((BB).).

dd((w,zw,z) = ) = ΔΔ((AA).).xyz

Δ(A)

Δ(B)Δ(C)

Δ(D)

w

Embedding into Ultra-metricEmbedding into Ultra-metric

Partition Partition X X into 2 sets into 2 sets XX11, X, X22

Create a root labeled Create a root labeled ΔΔ = = diamdiam((XX).).

The children of the root are The children of the root are created recursively on created recursively on XX11, X, X22

Using induction the number Using induction the number of distances we ignore isof distances we ignore is

BB – bad distances for current – bad distances for current level.level.

XXXX11 XX22

Δ

XX11 XX22

222

21 XB

XX

|B|≤ |B|≤ εε|X|X11||X||X22||

Take a point Take a point uu such that | such that |BB((u,u,ΔΔ/2/2)| ≤ )| ≤ n/2.n/2.

Let Let i=1,…,1/i=1,…,1/εε Let Let SSii=A=Ai+1i+1-A-Aii

We need a “slim” shell…We need a “slim” shell… only distances inside the only distances inside the

shell are distorted by shell are distorted by more thanmore than

Where to Cut?Where to Cut?

2, iuBAi

1O

AA11

AAii

uu

AAi+i+11

Case 1: |Case 1: |AA11|< |< εεn.n.

XX11= u, X= u, X22= X\= X\{{uu}}

Where to Cut?Where to Cut?

AA11

AAii

211 11 XXnAB Δ

X\uX\u

uu

Where to Cut?Where to Cut?

Case 2: Case 2:

Choose an Choose an i i such that:such that:

Let Let XX11=A=Ai+½i+½, X, X22 = X\X = X\X11

nA 1

AA11

AAii

uu

AAi+i+11

XX11

XX22

Δ

212

XXS

B i

22

ii An

S

Assume by contradiction for all Assume by contradiction for all

|S|Sii||22 > > εεn|An|Aii||

Then by induction Then by induction |A|Aii| ≥ | ≥ εεn·in·i22. .

which implies which implies |A|Att| ≥ n. | ≥ n.

End of proof!End of proof!

Finding Shell Finding Shell SSii

1

,,1 ti

Lower BoundsLower Bounds

General method to obtain partial lower bounds from General method to obtain partial lower bounds from known classical ones.known classical ones.

ThmThm: given a lower bound : given a lower bound αα for embedding a family for embedding a family

XX into a family into a family YY : : i.e. for any i.e. for any nn there is there is XXєєXX on on n n points and any points and any embedding of embedding of XX requires distortion at least requires distortion at least αα((nn).).

Then there is Then there is X’X’єєXX for which any (1- for which any (1-εε) partial ) partial embedding requires distortion embedding requires distortion

1

The family The family XX must be must be nearly closed under compositionnearly closed under composition!!

• distortion for partial embedding into trees. [Bartal/BLMN/RR].[Bartal/BLMN/RR].

• distortion for partial embedding of doubling or l1 metrics into l2. [NR]

• distortion for probabilistic partial embedding into trees. [Bartal]

Main corollariesMain corollaries

• distortion for partial embedding into lp. [LLR, Mat]

p

1log

1.

1log

1log

Choose Choose XXєєX X such that such that

For each For each xxєєX X create a metric create a metric CCxx

such that such that

- C- CxxєєXX..

- - X’X’ contain many “copies” of contain many “copies” of XX.. Let Let f f be a (1-be a (1-εε) partial ) partial embedding embedding

that ignores the set of edges that ignores the set of edges II. By . By definition .definition .

General ideaGeneral idea

3

1X

nCx 3

ddδδ

XXX’X’

22nI

T: T: vertices intersecting less vertices intersecting less than edges in than edges in II..

For each For each xxєєXX, choose some, choose some

vvxxєєCCxx∩T∩T..

For each pair (For each pair (vvxx,v,vyy) find ) find t t єєCCy y

such that:such that:

Finding a copy of Finding a copy of XX

n nT 1

Itvtv yx ,,,

in T

vx

in T

vy

t

Cx Cy

n2

n2nCC yx 3

Distortion of the CopyDistortion of the Copy

vvxx vvyy

tt

f f has distortion has distortion guarantees for both guarantees for both these distancesthese distances

Its distortion must be at Its distortion must be at leastleast 1

yxyx

yxyx

vtdtvdvvd

vtdtvdvvd

,,,

,,,

dd((t,vt,vyy)) is negligible is negligible