Euclidean Distance Matrices: A Short Walk Through Theory, Algorithms and Applications

Euclidean Distance Matrices:A Short Walk Through Theory, Algorithms and Applications

Ivan Dokmanic, Reza Parhizkar, Juri Ranieri, Miranda Krekovic and Martin Vetterli

“There are three things that matter in property: location, location, location.”

— Lord Harold Samuel, a real estate tycoon in Britain

1

The 2014 Nobel Prize in Physiology or Medicine: for their discoveries of cells thatconstitute a positioning system in the brain.

John O’Keefe,May-Britt Moser and Edvard I. Moser

2

Bat, echoes and navigation

72

70

68

66

74

Freq

uenc

y [k

Hz] 64

72

70

68

66

74

64

Hiryu, Shizuko, et al. "On-board telemetry of emitted sounds from free-flying bats: compensation for velocity and distance stabilizes echo frequency and amplitude." Journal of Comparative Physiology A 194.9 (2008): 841-851.

3

Daniel Kish, a human echo-locator

4

“You took my sonar concept and applied it to every phone in the city. With half the cityfeeding you sonar, you can image all of Gotham. This is *wrong*.”

— Lucius Fox, The Dark Knight

5

Motivation

I We land at the GVA airport and we have lost our map of Switzerland...

I We have only the train schedule!

Lausanne Geneva Zurich Neuchatel Bern

Lausanne 0 33 128 40 66

Geneva 33 0 158 64 101

Zurich 128 158 0 88 56

Neuchatel 40 64 88 0 34

Bern 66 101 56 34 0

Distances in minutes between five Swiss cities

SwitzerlandSwitzerland

Geneva

Lausanne

NeuchatelBern

Zurich

Locations (red) and estimated locations (black) ofthe cities.

6

Sensor network localization problem

Sensor network deployed on a rock glacier

(SensorScope project)

I We measure the distances between the sensor nodes,

I The distances are noisy and some are missing,

I How do we reconstruct the locations of the sensors?

7

Molecular conformation problem

Density map and structure of a

molecule [10.7554/elife.01345]

I We measure the distances between the atoms,

I The distances are given as intervals, and some are missing,

I How do we reconstruct the molecule?

8

Echo SLAM

A

B

C

A) An unknown room, B) a

moving robot, C) a room impulse

response measured by the robot

I A robot moves autonomously in the room,

I At every step it measures the room impulse response,

I How do we simultaneously recover the room shape and therobot location?

9

Overview:

I Motivation

I Euclidean Distance Matrices (EDM) and their properties

I Applications of EDMs

I Algorithms for EDMs

I Conclusions and open problems

10

Euclidean Distance Matrix

I Consider a set of n points in d dimensions: X ∈ Rd×n,

I edm(X) contains the squared distances between the points, edm(X)ij = ‖xi − xj‖2,

I Equivalently, edm(X) is a linear function of the Gramian G = X>X.

I Rank of the Gramian G is d .

11

Rank testing of Euclidean distance matrix (EDM)

I What can we say about ‖xi − xj‖2when x ∈ Rd?

12



‖xi − xj‖2 = 〈x i − x j , x i − x j〉

= 〈x i , x i 〉+ 〈x j , x j〉 − 2 〈x i , x j〉

gijdef= 〈x i , x j〉 , G

def= (gij)

edm(X) = −2G + 1diag(G )T + diag(G )1T

⇒ rank(edm(X)) ≤ rank(G ) + 1 + 1

= d + 2

12



‖xi − xj‖2 = 〈x i − x j , x i − x j〉

= 〈x i , x i 〉︸︷︷︸gii

+ 〈x j , x j〉︸︷︷︸gjj

−2 〈x i , x j〉︸︷︷︸gij

gijdef= 〈x i , x j〉 , G

def= (gij)

edm(X) = −2G + 1diag(G )T + diag(G )1T

⇒ rank(edm(X)) ≤ rank(G ) + 1 + 1

= d + 2

12

EDM properties: rank

I The rank of an EDM depends only on the embedding dimension of the points:

Theorem (Rank of EDMs)

Rank of an EDM corresponding to points in Rd is at most d + 2.

13

EDM properties: essential uniqueness

I EDMs are invariant to rigid transformations (translation, rotations, reflections):

edm(X) = edm(RX + b1>).

14

Counting the degrees of freedom

I Consider n points in Rd , then we have #X = nd degrees of freedom (DoF).

I If we describe the points with a symmetric rank-(d + 2) matrix, we have

#DOF = n(d + 1)− (d+1)(d+2)2 DoF.

I #DOF#X∼ d+1

d : the rank property is not a tight description of an EDM.

n101 102 103 104 105

DoF

101

102

103

104

105d=3

n101 102 103 104 105

DoF

101

102

103

104

105d=1

n101 102 103 104 105

DoF

101

102

103

104

105d=10

15

Forward and inverse problem related to EDMs

Forward problem

Inverse problem

I Forward problem: given the points X, find the EDM edm(X).

I Inverse problem: given the distances ‖xi − xj‖2, find the points X.

• Distances may be noisy,

• Some distances may be missing,

• We may loose the naming of the distances.

• Significantly harder than the forward problem!

16


Forward problem

Inverse problem?

?

?

?







16


Forward problem

Inverse problem







16

Every application has different challenges

?

?

I Problem: reconstruct the network topology.

I Nodes measure distances from each other.

I Random missing entries (e.g. distance).

I Measured entries are noisy.

17

Every application has different challenges

I Problem: reconstruct the molecular shape.

I We measure the distance between atoms.

I Distances are given as intervals.

18

Sometimes we measure distances without names

CorrectWrongI We measure a set of distances, we lost their

labels: combinatorial problem.

I Correct labels generate a low-rank matrix.

I Wrong labels generate a full-rank matrix.

I Hard to solve.

I Harder when the distances are noisy.

19

Overview:

I Motivation





20

Applications: Can you hear the shape of a room?

AI One cannot hear the shape of a drum....

but can one hear the shape of a room?(Using a source and 5+ microphones)

I Each microphone records the roomimpulse response (B), the peaks beingthe reflections on the wall.

I Using the image source model, wetransform walls to points.

I Problem 1: unlabeled distance problem

I Problem 2: some peaks do notcorrespond to any reflection, somereflections may be missing.

21

Applications: Can you hear the shape of a room?

A

B

I One cannot hear the shape of a drum....but can one hear the shape of a room?(Using a source and 5+ microphones)

I Each microphone records the roomimpulse response (B), the peaks beingthe reflections on the wall.

I Using the image source model, wetransform walls to points.

I Problem 1: unlabeled distance problem

I Problem 2: some peaks do notcorrespond to any reflection, somereflections may be missing.

21

Echo sorting algorithm

1 2 3 4 5

I We know the geometry of the microphone array,that is a quadrant of the EDM.

I For each microphone, we have a set of possibledistances w.r.t. the image sources,

I Location of the image sources: combinatorial search

I Rank-based EDM completion is stable to noise andcomputationally feasible.

22


1

2

3

4

5

1 2 3 4 5





22


1

2

3

4

5

??

??

?? ? ??

?1 2 3 4 5





22

It works... at EPFL

1

23

4

5

0.99

2.08

89.9°90.0°

63.2°62.0°

116.8°118.0°

90.190.0°

7.01m7.08m

B

12

3

4 5

2.520.4

100.9° 100.5°

90.3°90.0°

79.1°79.5°

89.7°90.0°

C

° Height

Exp. B 2.70mExp. C 2.71mReal 2.69m

4.55m4.59m

A

23

It works... at the Lausanne cathedral!

D

EFD

DDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDDD

EEEEEEEEEEEEEEEE DDDDDDDDDDDDDDDDDDDDDDDDDD

E

D

12 3

1

2 3

6.48m6.53m

6.71m6.76m

L

24

Applications: Sparse Phase Retrieval

Forward problem

Inverse problem

I (A) We measure a sparse auto-correlation function a(x).

I The ACF is supported on the distances between the elements of the original signal.

I The distances are unlabeled and noisy.

I (B) We recover the support of f (x) by labeling the distances.

I Knowing the support, it is easy to recover the amplitudes.25

Algorithm for Sparse Phase Retrieval

Forward problem

Inverse problem

I Distance measurements are shift invariant, we set the first element at the origin.

I The support of the signal is a subset of the support of the autocorrelation function.

I Greedy algorithm selects the best element from the support of the measured ACF

I Cost function: distance between the support of the ACF of the partial solution and thesupport of the measured ACF

26

Algorithm for Sparse Phase Retrieval

SNR of the ACF

Num

ber o

f del

tas

Phase transition: Noise vs Complexity

−10 −9 −8 −7 −6 −5 −4 −3

4

6

8

10

12

14

16

18

20

22

24 0

0.1

0.2

0.3

0.4

0.5

0.6

0.7

0.8

0.9

1

I Sparse Phase Retrieval using the structure of the EDMs,

I The obtained algorithm is stable to noise.

27

Echo SLAM

A

B

C

I We would like to localize a robot (B) insidean unknown room (A)

I The robot walks autonomously; we knowsome statistics of the steps.

I After each step, the robot emits a sound andmeasures the room impulse response (C).

I Each peak in the RIR likely corresponds toan image source.

I Problem: joint reconstruction of roomgeometry and robot location

28

Echo SLAM with fixed sound source

Estimated ISImage source Measured distance

Robot trajectoryRobot locationRoom

1st step of the robot

I At each step, we measure the RIR anddetermine the distance from the imagesources.

I At the 1st step, initialization of theprobability distribution of the image source

I At every step, the robot movesautonomously and we update the probabilitydistributions.

I The precision of the estimated image sourcesimproves with the steps.

I The estimated image sources are then usedto estimate the room shape.

29




2nd step of the robot






29




3rd step of the robot






29




4th step of the robot






29










29




Estimated room







29

Applications: Multidimensional unfolding of echoes

Microphones Acoustic events

1

32

4 1

2

3

??

??

??

??

??

????

??

??

I Multidimensional unfolding:

• Divide the points in two sets,

• Measure the distances between the points belonging to different subsets,

• Other distances are unknown.

I Example: calibration of microphones with sources at unknown locations.

I Many distances are missing, the measured ones are noisy.30

Applications: Multidimensional unfolding of echoes

MORE DETAILS, REFERENCES

31

Applications: Calibration in ultrasound tomography

A B

I Ultrasound transceivers deployed on a circle (A), their location is unknown.

I Each transceiver can measure the distance w.r.t. the transceivers in front.

I We can fill the EDM, with structured missing entries (B).

I Problem: estimate the locations of the transceiver to improve the imaging performance.

32

Applications: Calibration in ultrasound tomography

−0.1

0

0.1

1200

1500

1800

0 1.01.0−

−0.1

0

0.1

1490

1500

1510

0 1.01.0−

Uncalibrated device Calibrated device

I Solution of the ultrasound tomography when measuring water (1500 m/s).

I Calibration improves significantly the tomographic imaging.

33

Overview:

I Motivation





34

Algebraic algorithm: Multidimensional scaling (MDS)

Classical Multidimensional scaling (MDS)

I Choose x1 = 0, then the first column of D,call it d1, is diag(X>X )

I Thus we can construct the Gram matrix asG = −1

2(D − 1d>1 − d11>)

I The point set is obtained from G = X>X

35

Iterative optimization: s-stress function

Alternating descent on the s-stress function

I Minimize the s-stress,∑(i ,j)∈E

(‖x i − x j‖2 − dij)2

I Alternate between minimizing individualcoordinate component of each point x i

True location Estimated location

Round 1

36




(‖x i − x j‖2 − dij)2



36




(‖x i − x j‖2 − dij)2



Round 1

36




(‖x i − x j‖2 − dij)2



Round 2

36




(‖x i − x j‖2 − dij)2



Round 3

36




(‖x i − x j‖2 − dij)2



Round N

36

Exploiting the EDM cone: Semidefinite relaxations

Semidefinite relaxation

I The Gram matrix is positive semidefinite

minimizeG

∥∥∥W ◦ (D − edm(G ))∥∥∥2

F

subject to (((((((hhhhhhhrank(G ) ≤ d and G ∈ Sn+ ∩ Snc .d12d13

23

The EDM cone for 3 points

37

Algorithms comparison

Algorithm Pros Cons

Classical MDS Very simple No missing distancesHandles noise Unnatural cost function

Alternating Descent Simpleon s-stress Good cost function (s-stress) Convergence issues

Missing distances, noise

SDR Good cost function SlowMissing distances, noise Can’t enforce the embeddingHandles constraints on distances dimension

38

Algorithms: performance w.r.t. missing entries

20 40 60 80 100 120 1400

20

40

60

80

100

Succ

ess

perc

enta

ge

Number of deletions

Alternating Descent

Rank Alternation

Semidefinite Relaxation

Rank Alternation

50 100 1500

0.2

0.4

0.6

0.8

1Semidefinite Relaxation

50 100 150Number of deletions

Rel

ativ

e er

ror

Alternating Descent

Jitter

50 100 150

39

Algorithms: performance w.r.t. number of acoustic events

5 10 15 20 25 30

20

40

60

80

100

Succ

ess

perc

enta

ge

Number of acoustic events


Rank Alternation

Alternating Descent

Alternating Descent

5 10 15 20 25 300

0.2

0.4

0.6Rank Alternation

5 10 15 20 25 30


Number of acoustic events

Rel

ativ

e er

ror

Algorithm 4

Jitter

5 10 15 20 25 30

40

The choice of algorithm is fundamental

41

Overview:

I Motivation





42

Open problems

I Distance matrices on manifolds I Projections of EDMs on subspaces

I Efficient algorithms for distancelabeling

I Analytical local minimum of s-stress

−5

0

5

−5

0

5

0

500

1000

1500

2000

0

200

400

600

800

1000

1200

1400

1600

1800

2000

global min.

local max.saddle

43

Conclusions

Application missing distances noisy distances unlabeled distances

Wireless sensor networks * 4 4 ×Molecular conformation 4 4 ×

Hearing the shape of a room * × 4 4

Indoor localization × 4 4

Calibration * 4 4 ×Sparse phase retrieval * × 4 4

* Applications that we have discussed in this talk.

44

Literature

I I.Dokmanic, R.Parhizkar, J.Ranieri, and M.Vetterli, Euclidean Distance Matrices: A ShortWalk Through Theory, Algorithms and Applications, to appear in IEEE Signal ProcessingMagazine, 2015.

I R.Parhizkar, A.Karbasi, S.Oh, M.Vetterli, Calibration Using Matrix Completion withApplication to Ultrasound Tomography, IEEE Transaction on Signal Processing, 2013.

I I.Dokmanic, R.Parhizkar, A.Walther, Y.M.Lu, and M.Vetterli, Acoustic echoes revealroom shape, Proceedings of the National Academy of Sciences, 2013.

I J.Ranieri, A.Chebira, Y.M.Lu, and M.Vetterli, Phase Retrieval for Sparse Signals:Uniqueness Conditions and Algorithms in preparation.

I R.Parhizkar, Euclidean Distance Matrices: Properties, Algorithms and Applications, PhDThesis EPFL, 2013.

I I.Dokmanic, Listening to Distances and Hearing Shapes: Inverse Problems in RoomAcoustics and Beyond, PhD Thesis EPFL, 2015.

45

Acknowledgments

I Farid Movahedi Naini for helping with the numerical experiments.

I Funding resources:

• Swiss National Science Foundation, Grant 200021 138081.

• European Research Council SPARSAM, Grant 247006.

• Google Focused Research Award.

• Google PhD fellowship.

46

Thank you for your attention!

Googlecar

Batmobile with echolocation

47

Euclidean Distance Matrices: A Short Walk Through Theory, Algorithms and Applications

Science

Transcript of Euclidean Distance Matrices: A Short Walk Through Theory, Algorithms and Applications