© Fraunhofer IOSB

1

Segmentation and classification of man-made maritime objects in TerraSAR-X

images

IEEE International Geoscience and Remote Sensing Symposium Vancouver, Canada

July 27th 2011

Michael Teutsch, email: michael.teutsch@iosb.fraunhofer.de

Günter Saur, email: guenter.saur@iosb.fraunhofer.de

© Fraunhofer IOSB

2

Outline

Motivation

Concept

Segmentation

Classification

Examples

Conclusions and future work

© Fraunhofer IOSB

3

Motivation I

Applications:

Tracking of cargo ship traffic

Surveillance of fishery zones, harbours, shipping lanes

Detection of abnormal ship behaviour, criminal activities

Search for lost containers or hijacked ships

Aims / Challenges:

Detection of man-made objects (not here)

Precise orientation and size estimation

Separation of clutter, non-ships, different ship types

Robustness against various SAR-specific noise effects

Fast processing time

Here: Analyze object appearance, avoid models and prior knowledge

© Fraunhofer IOSB

4

Motivation II: Difficult examples

© Fraunhofer IOSB

5

Concept

© Fraunhofer IOSB

6

Pre-processing

3x3 median filter

Ground Sampling Distance (GSD) normalization to 2.0 meters/pixel

© Fraunhofer IOSB

7

Segmentation I: Structure-emphasizing LBP filter

Timo Ojala et al., „Multiresolution Gray-Scale and Rotation Invariant Texture Classification with Local Binary Patterns”, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 24, no. 7, July 2002.

Rotation invariant uniform LBPs:

Texture primitives:

Local Binary Pattern:

© Fraunhofer IOSB

8

Segmentation II: Structure-emphasizing LBP filter

Rotation invariant variance measure:

Rotation invariant uniform LBPs (texture primitives):

For each pixel position (x,y), fixed P, and varying R:

© Fraunhofer IOSB

9

Segmentation III: Rotation compensation with HOG

A. Korn, „Toward a Symbolic Representation of Intensity Changes in Images“, IEEE Transactions on Pattern Analysis and Machine Intelligence, vol. 10, no. 5, 1988.

© Fraunhofer IOSB

10

Segmentation IV: Rotation compensation with HOG+PCA

PCA FUSION

© Fraunhofer IOSB

11

Segmentation V: Size estimation with row/col. histograms

© Fraunhofer IOSB

12

Segmentation VI: Experimental data set

17 different TerraSAR-X StripMap images

756 manually labeled detections including orientation and length

No ground truth, manual labeling is sensed truth

Labeling inspired by CFAR-detection including potential clutter

Scale normalization to 2.0 meters / pixel

© Fraunhofer IOSB

13

Segmentation VII: Orientation and size estimation results

methodrotation estimation error

median mean

LBP & HOG & PCAwith median filter

5.24° 11.65°

LBP & HOG & PCAwithout median filter

5.99° 12.16°

LBP & HOG 6.71° 12.99°

LBP & PCA 12.09° 24.38°

HOG only 10.68° 23.36°

© Fraunhofer IOSB

14

Segmentation VIII: Examples

© Fraunhofer IOSB

15

Classification I: Classes

clutter (ambiguity)

unstructured shipclutter ship structure 2

ship structure 1non-ship

© Fraunhofer IOSB

16

Classification II: Concept

1. G. Saur, M. Teutsch, „SAR signature analysis for TerraSAR-X based ship monitoring“, Proceedings of SPIE Vol. 7830, 2010.

2. M. Teutsch, W. Krüger, „Classification of small Boats in Infrared Images for maritime Surveillance“, 2nd International Conference on WaterSide Security (WSS), Marina di Carrara, Italy, Nov. 3-5, 2010.

© Fraunhofer IOSB

17

Classification III: Experiments and results

5 classes: clutter, non-ship, unstr. ship, structure 1, structure 2

543 samples with good segmentation and possible manual labeling:

53 clutter, 110 non-ship, 322 unstr. ship, 17 structure 1, 41 stucture 2

362 training samples and 181 test samples

Runtime for segmentation and classification: ~ 2 sec per detection

Classification results:

classifier SVM 1 SVM 2 3-NN cascade

correct rate

96.68 % 93.29 % 91.45 % 80.66 %

© Fraunhofer IOSB

18

Classification IV: Examples

non-ship

unstructured ship

unstructured ship

unstructured shipclutter

ship structure 1

© Fraunhofer IOSB

19

Classification V: Examples for whole processing chain

ship structure 2 ship structure 2unstructured ship

© Fraunhofer IOSB

20

Conclusions Aim: Segmentation and classification of man-made objects in satellite SAR

Challenge: Robustness against various object appearances, noise effects

Segmentation: Pre-processing, structure-emphasizing filter with LBPs, orientation estimation with HOGs and PCA, size estimation with row/column histograms, median orientation estimation error: 5.2°

Classification: Extensive feature calculation, feature evaluation and selection, classification with cascaded SVM and 3-NN, 81% correct classification

Future work Improve size estimation (LBPs instead of row/column histograms?)

More data for classification (esp. structure classes)

Other approaches for 3rd classification-stage (local features?)

Is object structuredness and classifiability based on appearance measurable?

© Fraunhofer IOSB 21

Fraunhofer Institute of Optronics, System Technologies and Image Exploitation IOSB

Karlsruhe Ettlingen Ilmenau

Thanks a lot for your attention!

© Fraunhofer IOSB

22

Segmentation: Orientation estimation error distrib.

© Fraunhofer IOSB

23

Segmentation: Examples – The bad guys