Analisi di immagini iperspettrali satellitari

multitemporali: metodi ed applicazioni

Francesca Bovolo

E-mail: bovolo@fbk.eu

Web page: http://rsde.fbk.eu

Outline

2

Multitemporal image analysis

Multitemporal images pre-processing

Applications

1

Multitemporal information extraction3

2

4

5 Conclusions

Francesca Bovolo

Multitemporal Image Analysis

Multitemporal image analysis: aims at analyzing two or more remote sensing images acquired on thesame geographical area at different times for identifying changes or other kinds of relevant temporalpatterns occurred between the considered acquisition dates.

Several multitemporal problems exist:• Binary change detection.• Multiclass change detection.• Trend analysis in long time series.

3

Binary change detection

Tsunami damages

Before Change After Change

Multiclass change detection

Crop rotation

Before Change After Change

Trend analysis in long

time series

True Color Composite NDVI

Francesca Bovolo

Multitemporal Analysis Block Scheme & Products

4

F. Bovolo, L. Bruzzone, “The Time Variable in Data Fusion: A Change Detection Perspective,” IEEE Geoscience and Remote Sensing

Magazine, Vol. 3, No. 3, pp. 8-26, 2015.

1

N

t1 image

t2 image

X1

X2

tN image

XN

Co-registration

MultipleChange Detection

Radiometriccorrections

1

N

Multitemporal image pre-processingMultitemporalClassification

Map Updating

Trend Analysisin Time Series

BinaryChange Detection

Change Detectionin Time Series

Binary change map

Multiple-change map

Land-cover transition map

Land-cover map @tn+1

Trend map

Seasonal/Annual Change Map

Data Analysis

Francesca Bovolo

Multitemporal Images Co-registration

In multitemporal image processing, poor co-registration is a source of errors and can be modeled as a

noisy component.

Registration noise statistical properties have been modeled by accounting for its multiscale behaviors in

the difference image domain.

5

F. Bovolo, L. Bruzzone, S. Marchesi, “Analysis and Adaptive Estimation of the Registration Noise Distribution in Multitemporal VHR Images,” IEEE Transactions on Geoscience and Remote

Sensing, Vol. 47, No. 8, Part 1, pp. 2658-2671, 2009.

Multitemporal false-

color composite of

misaligned images Registration noise map

Level 0Level 3

0

10-4

T

Differential analysis

Multiscale CVARN polar plot

TRN

RN probability density function

Francesca Bovolo

Multitemporal Images Co-registration

Registration noise information can be used to improve multitemporal analysis by:

Removing noisy pixels in post-processing [1];

Involving registration noise in the co-registration step [2]-[4];

Other more complex paradigms.

6

[1] S. Marchesi, F. Bovolo, L. Bruzzone, “A context-sensitive technique robust to registration noise for change detection in VHR multispectral images,” IEEE Transactions on Image Processing,

Vol. 19, No. 7, pp. 1877-1889, 2010.

[2] Y. Han, F. Bovolo, L. Bruzzone, “An Approach to Fine Co-Registration Between Very High Resolution Multispectral Images Based on Registration Noise Distribution,” IEEE Transactions on

Geoscience and Remote Sensing, Vol. 53, No. 12, pp. 6650 – 6662, 2015.

[3] Y. Han, F. Bovolo, L. Bruzzone, “Edge-Based Registration Noise Identification for VHR Multisensor Images,” IEEE Geoscience and Remote Sensing Letters, Vol. 13, pp. 1231 – 1235, 2016.

[4] Y. Han, F. Bovolo, L. Bruzzone, “Segmentation-based Fine Registration of Very High Resolution Multitemporal Images,” IEEE Trans. on Geoscience and Remote Sensing, in press 2017.

Multitemporal chessboard images

Before co-registration After co-registration

RMSE

(pixels)

STD

(pixels)

Before co-registration 16.87 2.13

SoA 3.70 2.21

Registration noise –

based co-registration0.89 0.37

Francesca Bovolo

Hyperspectral Multitemporal Data Analysis: Challenges

Multitemporal hyperspectral data are highly sensitive to detailed variations of the spectral signatures of

land covers thus when multitemporal images are considered subtle changes/temporal variations can be

detected.

The definition of the concept of change in hyperspectral (HS) images is not clear in the literature and

represents the first challenge.

Only few approaches to multitemporal in hyperspectral images exist in the literature.

State-of-the-art methods are mainly developed for multispectral images and do not explicitly handle the

challenging issues that may arise due to the properties of hyperspectral images like:

• the high dimensionality;

• the presence of noisy channels and redundant information;

• the increase of computational cost;

• the increase of the possible number of changes;

• the high complex change representation and identification;

• the presence of changes at sub-pixel level (i.e., mixed pixels).

7Francesca Bovolo

Hyperspectral Change Concept

8

High spectral resolutionLow spectral resolution

Multitemporal false

color composite

R 823.65nmG 721.90nmB 620.15nm

2004 2007

Hyp

eri

on d

ata

Spectr

al sig

na

ture

variatio

n

Spectr

al sig

na

ture

variatio

n

Francesca Bovolo

Hyperspectral Information Visualization & Management

9

Pseudo binary

change detection

Pseudo binary change detection map

W = {wn, Wc, Wu}

X1

X2

Hierarchical spectral

change analysis

Uncertain pixel

labeling

Change detectionmap

Wu

Wc

XD

wn

Change end-members

Root

Major

changes

Subtle

ch

an

ges

cW

1Cw

2 1 2Cw

3Cw

1 1Cw 1 2Cw

2 2Cw

2 1 1Cw

2Cw

2 1Cw

Change

end-member

Spectr

al sig

natu

re v

ariation

0 1 2×10-4

Tr

F. Bovolo, S. Marchesi, L. Bruzzone, “A Framework for Automatic and Unsupervised Detection of Multiple Changes in Multitemporal Images”, IEEE Transactions on Geoscience and Remote

Sensing, Vol. 50, No. 6, pp. 2196-2212, 2012.

S. Liu, L. Bruzzone, F. Bovolo, P. Du, “Hierarchical unsupervised change detection in multitemporal hyperspectral images,” IEEE Transactions on Geoscience and Remote Sensing, Vol. 53, pp.

244 - 260, 2015. DOI 10.1109/TGRS.2014.2321277.

S. Liu, L. Bruzzone, F. Bovolo, M. Zanetti, P. Du, “Sequential Spectral Change Vector Analysis for Change Detection in Multitemporal Hyperspectral Images,” IEEE Transactions on Geoscience

and Remote Sensing, Vol. 53, pp. 4363 – 4378, 2015.

Francesca Bovolo

Hyperspectral Information Visualization & Management

10

Multitemporal false

color composite

R 823.65nmG 721.90nmB 620.15nm

2004 2007

Proposed method Standard clustering

Francesca Bovolo

Unmixing in Hyperspectral Multitemporal Images

11

+ -

- -

- +

+ -

- -

- +

material class 1

+: pure spectrum - : mixed spectrum

material class 2 material class 3

X1 X2 X1 X2

No-changeChange Multiemporal information extraction can be designed as a

multitemporal spectral unmixing problem at sub-pixel level.

A pure multitemporal endmember (MT-EM) of either

change or no change is a stacked feature vector composed

by a pure spectral signature in both X1 and X2 components.

MT-EM Pool U

Change Analysis

MT-spectral

unmixingUnmixing Model

Abundance

Combination

Un,Uc

AU

Final CD

Map

XS

1 2,

SX X X

Reflecta

nce

X1 X2

No-change

Change

S. Liu, L. Bruzzone, F. Bovolo, P. Du, “Unsupervised Multitemporal Spectral

Unmixing for Detecting Multiple Changes in Hyperspectral Images,” IEEE

Transactions on Geoscience and Remote Sensing, Vol. 54, pp. 2733 - 2748, 2016.Francesca Bovolo

Unmixing in Hyperspectral Multitemporal Images

12

42 endmembers

16 change MT-EMs in Uc

26 no-change MT-EMs in Un

Extracted Endmembers

Change Class 2

Change Class 6

Change Class 4

No-change Class

Abundance Maps

Francesca Bovolo

0

1

Land-Cover Map Updating

13

X1

X2

t1 image

t2 image

Training set T1

CD Driven

Transfer Learning

Initial

Training

set T2 Multitemporal

Active Learning

Land-cover

map at t2

Low-cost definition of training set T2

Cascade

Classification

Final Training set T2

Training set T1

1 2{ , ,..., }

N

1 2{ , ,..., }

Rw w wW

B. Demir, F. Bovolo, L. Bruzzone “Updating Land-Cover

Maps by Classification of Image Time Series: A Novel

Change-Detection-Driven Transfer Learning Approach”,

IEEE Transactions on Geoscience and Remote

Sensing, Vol.51, pp.300-312, 2013. Francesca Bovolo

Applications

Multitemporal analysis of hyperspectral images can be relevant in many applications, among the others

those where small intensity changes and/or changes slow in time may strongly benefit:

Forest analysis (e.g., forest disturbance, forest fires, forest classification, biomass analysis);

Precision agriculture (e.g., crop mapping, crop rotation, crop stress analysis, fertilization);

Water quality (e.g., chlorophyll monitoring, alga bloom);

Vegetation (e.g., desertification, deforestation, vegetation stress);

Etc.

14Francesca Bovolo

Conclusions

PRISMA will guarantee regular multitemporal hyperspectral images.

Several activities have been developed in multitemporal image processing in multispectral images

(satellite and airborne).

Automatic supervised/unsupervised methods are available including:

• Multitemporal images preprocessing.• Binary change detection.• Multiclass change detection.• Trend analysis in long time series.

These methods can been adapted to the characteristics of PRISMA hyperspectral multitemporal images

to generate new products.

15Francesca Bovolo

16Francesca Bovolo