AHierarchicalMarkovRandomFieldForRoadNetworkExtractionAndItsApplicationWithOpticalAndSarData.pdf

Introduction Methodology Results Conclusions

A hierarchical Markov random fieldfor road network extraction and its

application with optical and sar data

Talita Perciano1,2 Roberto Hirata Jr.1 Roberto M. C. Jr.1

Florence Tupin2

1Departamento de Computacao

Instituto de Matematica e Estatıstica

Universidade de Sao Paulo

2Departement Traitement du Signal et des Images

Telecom ParisTech

IGARSS 2011

Talita Perciano Road extraction and data fusion IGARSS 2011 26/07/2011 1 / 25


Contents

1 Introduction

2 Methodology

3 Results

4 Conclusions



Motivations and objective

• Advent of new optical (QuickBird, Pleiades) and radar(TerraSAR-X, Cosmo-Skymed) high-resolution satellite sensors

• New perspectives for pattern recognition problems as roadnetwork extraction

• The number of works in the literature exploring high-resolutionimages and multi-sensor image processing is increasing

Objective

Propose a flexible hierarchical Markovian random field based onfeature extraction and road network structure, exploringmulti-sensor data fusion



Problem of road network extraction

• Problem studied since many years as it is an importantstructure for many applications:

• Urban planning• Making and updating maps• Traffic management• Cartography

• Difficult task due to the spatial and spectral features of theroad

• Different automatic and semiautomatic approaches in theliterature

• A two-step approach is explored in this work:

1 Low level: features extraction2 High level: road network reconstruction using contextual

information



Contents

1 Introduction

2 Methodology

3 Results

4 Conclusions



Overview of the method (Tupin et al, 1998 )

1 Extract linear features• Ratio-based detector (D1(x , y)) and a cross-correlation-based

detector (D2(x , y))

D(x , y) =D1(x , y)D2(x , y)

1− D1(x , y)− D2(x , y) + 2D1(x , y)D2(x , y). (1)

��

r2 r3r1

px,y



Overview of the method

2 Road network reconstruction

• Graph modeling: map structures and its relations into a graphwhere each segment is a node and two nodes are connected iftheir corresponding segments share a extremity

• Markovian model: search for the optimal binary labeling byminimizing an energy function defined for the MRF that has adata attachment term (likelihood) and a prior term:

U(l) = Ulikelihood(l , d) + Uprior (l) (2)



Overview of the method

2 Road network reconstruction

1

111

1

10

1

0

1

1

1

Line detection Polygonal approximationand connections

Graph

Final road networkExample of labeling

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��



Features extractionProposed radar and optical fusion (Novelty)

• The ratio and cross-correlation measures are calculatedsimultaneously in the radar and optical images

• The maximum response for each measure is retained• The symmetrical sum is used as before:

D(x , y) =D1(x , y)D2(x , y)

1−D1(x , y)− D2(x , y) + 2D1(x , y)D2(x , y). (3)

��

r2 r3r1

px,y

(a) Radar

��

r2 r3r1

px,y

(b) Optical



Connected component level (Novelty)Proposed method

• Road network reconstruction: the use of connectedcomponents instead of segments

Detect componentsand make connections

1

10

01

1

1

Line detection Graph

Example of labelingFinal road network

��

��

��

��

��

��

��

��

��

��

��

��

��

��



Connected component level (Novelty)Proposed method

• Advantages of using connected components! Simplification of the graph by decreasing considerably itsnumber of nodes! Deal with more complex structures! Take more advantage of the complete structures detected inthe low level

• Process applied in a multi-scale way• A pyramid is created by degrading the resolution (average of

the amplitudes of n × n pixels blocks)• Extraction of the roads in the three scales• Results of each scale are merged together• “Cleaning step” to remove possible redundancies



Road section level (Novelty)Additional high-level step

Built a new graph from the result of the previous road extraction:

• Image is preprocessed to obtain only crossroads and roadsections

• Each road section is a node of the graph and two nodes areconnected is their corresponding sections share a crossroad

• MRF model with the same kind of energy function, but thebest likelihood value is obtained analyzing all three scales ofthe multi-scale pyramid and from both radar and opticalimages

• Simpler and computationally faster step



Road section level (Novelty)Additional high-level step

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

1

1

1

0

11

1 0

1

11

Result from previousMRF

Prolongations

Graph

Example of labeling

Final road network

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��

��



Contents

1 Introduction

2 Methodology

3 Results

4 Conclusions



Results - QuickBird and TerraSAR-X images

(Toulouse)

(a) Optical image (b) Radar image (c) Ground-truth



Results - QuickBird image

(a) Ground-truth (b) Optical result (c) Optical result



Results - TerraSAR-X image

(a) Ground-truth (b) Radar result (c) Radar result



Results - Fusion

(a) Ground-truth (b) Fusion result (c) Fusion result




(Toulouse)

(a) Optical image result (b) Radar image result (c) Fusion result

Figure: Correct detection in red, incorrect detection in black andabsent roads in blue.




(Toulouse)

(a) Optical image (b) Radar image (c) Ground-truth




(Toulouse)

(a) Ground-truth (b) Fusion result (c) Fusion result



Results

Table: Quantitative evaluation of the results.

Data Completeness Correctness Quality

R1 Optical image 41.44% 38.63% 24.99%

Radar image 44.42% 50.82% 31.06%

Fusion 67.39% 56.55% 44.40%

R2 Optical image 45.21% 45.72% 31.06%

Radar image 53.16% 43.77% 31.59%

Fusion 62.69% 57.4% 42.78%



Contents

1 Introduction

2 Methodology

3 Results

4 Conclusions



Discussion and conclusions

• We propose a new framework for road detection composed bythree steps:

• Low-level step (line detection with fusion of optical and radardata)

• First high-level step (connected components)• Second high-level step (road sections and crossroads)

• A hierarchical multi-scale framework that uses informationfrom different sources (radar and optical images)

• The quantitative results show the considerable improvementof detection using the fusion approach



Acknowledgements

Thanks to FAPESP, CAPES (scholarship process number0310-10-7) and CNPq Brazilian agencies for funding.

Contact: [email protected]

Questions?


[email protected]

AHierarchicalMarkovRandomFieldForRoadNetworkExtractionAndItsApplicationWithOpticalAndSarData.pdf

Technology

Transcript of AHierarchicalMarkovRandomFieldForRoadNetworkExtractionAndItsApplicationWithOpticalAndSarData.pdf