Inverse problems in earth observation & cartography

Inverse Problems in Earth Observation and Cartography

Theme CogB

2

Project and People

Joint CNRS/INRIA/UNSA project team, created in 1998. Project leader: Josiane Zerubia. Members:

3 INRIA: J. Zerubia (DR1), X. Descombes (CR1), I. Jermyn (CR1).

1 CNRS: L. Blanc-Féraud (DR2). 1 postdoc. 10 PhD students. 5 Masters students (ENS Cachan, Sup’Aéro, ENAC, CPE

Lyon). 5 student Interns (ETH Zurich, Szeged University , Sup’Com

Tunis, IIT New Delhi, IIT Roorkee). 1 assistant INRIA (50%), 1 assistant CNRS/UNSA (10%).

3

Research domain

4

Principal goals

Taking into account the physics of the sensor (visible, infrared, radar, laser,…).

Extraction of information relevant for high-level interpretation.

Reconstruction of 3D information (digital elevation map) from 2D data.

Map updating for cartography.

5

Application areas

Commercial Precision farming, GIS,…

Public interest Forestry management,

environmental monitoring, urban planning, cartography,…

Homeland security Intelligence, pre- and post-mission

analysis,…

6

Scientific foundations

Probabilistic approaches MRFs: geometric properties, MRFs on trees,… Stochastic geometry: marked point processes.

Variational approaches Regularization and functional analysis: BV space,

-convergence,… Contours and regions: level sets, higher-order

active contours,… Optimization/Parameter estimation

MCMC, RJMCMC,… Diffusion processes.

7

Main contributions 2001-2005

8

Probabilistic models (I)

Markov Random Fields Markov Random Fields for

hyperspectral data [IEEE TGRS 04]

New models on trees for segmentation [IEEE TGRS 05]

Impact Lyapunov Inst. (01-02); NATO-

Russia (03-04); ECO-NET (05). Contract and transfers: Alcatel

Alenia Space Cannes.

9

Probabilistic models (II)

Marked Point Processes A new general modelling

framework [IEEE SP Mag. 02, IJCV 04, IEEE PAMI 05, IJCV to appear].

New kernels for RJMCMC optimization [LNS Springer 05].

Impact PNTS (04-05); COLORS Arbres

(01); ARC MODE de VIE (05-06). Contracts and transfers: DGA; IGN;

BRGM. Eusipco04 Young Author Best

Paper award.

10

Variational methods (I)

Image decomposition [JMIV05] New algorithm for decomposing an image

into geometry (in BV) + oscillations (in G). Numerically challenging (L1 norm). Applications: restoration, compression,

inpainting…

Impact Math/STIC (02-03); ACI Multim (04-07). Used for IR target detection by DGA. Best PhD thesis prize 05 from EEA Club,

Signal and Image section.

11

Variational methods (II)

Higher-order active contours New method for the inclusion of

geometric information in active contour models. [IJCV to appear]

Reformulation as phase fields to simplify implementation and allow model learning. [ICCV05]

Impact EU NoE MUSCLE (04-08); ACI

QuerySat (04-06). Contract: Alcatel Alenia Space

Cannes. Transfers: U. Szeged, Hungary;

LIAMA, China.

p0p

n(p) n(p0)

Long-range interactions Effect of prior

12

Parameter Estimation

Estimation of model hyperparameters for deconvolution of HR visible and TIR data.

Estimation of sensor parameters for blind deconvolution of HR visible data.

Impact Contracts and transfers: CNES; Astrium;

Sagem. Patent #0110189 (France: 01; Europe,

Israel, USA, Canada, Japan: 02). Usage rights granted to French Space

Agency for future satellites.

13

Application to other domains: astrophysics

Image deconvolution Using complex wavelet packets.

Galaxy filament detection Using a marked point process.

Impact COLORS DECONVOL (02);

COLORS FIGARO (05). Transfers: Côte d’Azur

Observatory. First automatic extraction of

galaxy filaments from a real galaxy catalogue (provided by Harvard).

14

Application to other domains: confocal microscopy

Deconvolution Using Richardson-Lucy algorithm taking

into account the edges of the object (TV regularization) [MRT Journal to appear]

Impact ARC DeMiTri (03-04); P2R Franco Israeli

Programme (05-06). Transfers: Pasteur and Weizmann Institutes.

15

Project-team positioning

16

Positioning: INRIA scientific challenges

Develop multimedia data and multimedia information processing (40%): EU project MOUMIR (00-04). EU NoE MUSCLE (04-08). ACI QuerySat (04-07)

Couple models and data to simulate and control complex systems (15%) ARC MODE de VIE (05-06)

Model living structures and mechanisms (15%) ARC DeMiTri (03-04) P2R Franco-Israeli programme (05-06)

17

Positioning: INRIA (I)

Digiplante: peer. Strong collaboration on forestry growth modelling

(ARC Mode de Vie). Imedia: peer.

Joint partners in EU NoE MUSCLE & ACI QuerySat. Strongly complementary roles: Imedia = database

retrieval; Ariana = image processing. Vista: peer.

Joint partners in EU NoE MUSCLE. Benchmarking of segmentation methods developed in

the two project-teams.

18

Positioning: INRIA (II)

Clime: peer. Works on satellite data, but mainly for atmospheric & meteorological

modelling & on time series. Epidaure/Asclepios: peer.

Joint partners in EU IMAVIS project. Complementary work on biological microscopy imaging but

otherwise different applications. Mistis: peer/competitor.

Collaboration with previous project-team IS2. Methodological overlap (MRFs, variational, & comparison) but

different applications. Odyssée: peer/competitor.

Joint partners in EU IMAVIS project. Shape from shading: different approaches (stochastic vs. PDEs). Variational methods (different applications) & shape information

(different approaches).

19

Positioning: France (I)

Signal & Image Processing department, ENST, Paris: peer/competitor. (Campedel, Maître, Nicolas, Roux, Sigelle, Tupin) Main competitor in France for remote sensing

applications. Joint partners in ACI QuerySat & in PNTS initiative;

joint PhD. PASEO group, MIV team, LSIIT, Strasburg: peer.

(Collet, Jalobeanu) New group: possible collaboration on remote sensing.

20

Positioning: France (II)

CMLA, ENS Cachan: peer. (Aujol, Chalmond, Morel, Younes) Joint partners in ACI Multim & MATH/STIC project. Complementary work on variational and stochastic methods.

CMAP, École Polytechnique, Palaiseau: peer. (Chambolle) Joint partners in ACI Multim & joint work on functional analysis

for image processing.

CEREMADE/Paris XIII: peer/competitor. (Cohen, Dibos) Variational methods for shape description: different approaches

but some overlap of application domains. Complementary work on TV regularization and PDEs.

21

Positioning: international (I)

Stochastic geometry: Ariana is one of the key developers of object

processes for image analysis and understanding. Collaboration with CWI (van Lieshout). Peer. Related to work at Brown (Grenander), Florida State

(Srivastava). Simpler objects but complex inter-object interactions. Peer.

German Space Agency DLR and U. Jaume I have adapted models developed by Ariana. Peer/competitor.

22

Positioning: international (II)

Higher-order active contours: Distinct from other methods for the inclusion of prior

geometric information. Better adapted to remote sensing than the many

template-based methods: U. Bonn (Cremers); U. Florida (Chen, Thiruvenkadam, Huang,

Wilson, Geiser); Yale University (Tagare); U. Mannheim (Kohlberger, Schnorr); UCLA (Soatto); Saarland University (Weickert); Siemens Corporate Research (Paragios); MIT (Leventon, Grimson, Faugeras);… Peers/competitors.

23

Positioning: international (III)

Functional analysis for image processing: Image Processing group at UCLA (Chan,

Esedoglu, Osher, Vese). Also work on functional analytic and PDE approaches. Peer/competitor.

U. Pompeu Fabra Barcelona (Caselles). Related work on image inpainting, and strong links via exchange of students and researchers. Peer.

U. Minnesota (Sapiro). Works more on PDEs than variational methods. Peer/competitor.

CNR Rome (March). Collaboration on -convergence. Peer.

24

Evolution of the objectives during 2001-2005

All the objectives defined in 2001 have been reached except: The work on parameter estimation for blind

deconvolution of optical and infrared satellite images was stopped due to legal problems over industrial property between INRIA-CNRS vs CNES or Sagem).

Despite patent deposit in 2001 and extension in 2002.

Two new topics emerged: Astrophysical image processing. Biological image restoration and deconvolution.

25

Recommendations from previous evaluation

Ariana has continued to: Develop innovative theoretical methods Publish in the best international journals and conferences. Maintain high international visibility.

Ariana has increased its activity in: Validation:

Via data and ground truth obtained from end users (CNES, IAURIF, IGN, IFN, BRGM, Alcatel Alenia Space).

Via end user validation and use (IGN, BRGM). Applications:

Environmental research (IFN, Alcatel Alenia Space, Silogic): trees, fires,…

Image retrieval: Collaborations with INRIA project-teams, French and European

researchers (ACI QuerySat, EU projects MOUMIR and MUSCLE, Math/STIC Tunisia) and end users (CNES, DLR).

26

Goals for the next four years

27

Future: probabilistic methods

Multi-object processes. Different types of objects (e.g. roads, buildings,

trees,…) will be included in a single model in which they mutually interact.

High-risk. Improved models of texture.

Quartic models of the joint statistics of adaptive wavelet packet coefficients.

Medium-risk.

28

Future: variational methods

Image decomposition & restoration. Multispectral extension using inter-band correlation. Medium-risk.

Detection of objects of codimension > 1 in 3D images. New functionals based on Ginzburg-Landau theory. High-risk.

Higher-order active contours & phase fields. Multiscale/wavelet models. Medium- to high-risk.

29

Future: parameter estimation

Parameter estimation for marked point processes using new diffusion processes & comparison to RJMCMC. Medium- to high-risk.

Model learning for higher-order active contours & phase field models. High-risk.

Parameter learning and resolution dependence for adaptive wavelet packet texture models. Medium-risk.

Estimation of image acquisition parameters for the blind deconvolution of microscopy images. High-risk.

30

Future: applications

Automatic urban scene analysis & cartography. Using multi-object marked point processes. Using phase field higher-order active contours. High-risk.

Forest monitoring (tree population counting & classification, disaster damage evaluation). Using marked point processes & Markov random fields. Medium-risk.

Blind deconvolution of microscopy imagery. Using physical models of the image acquisition process and

TV and wavelet regularizations. Medium- to high-risk.

31

Some statistics 2001-2005

Publications: PhDs & Habilitations: 13. Books edited: 2. Journal articles + book

chapters: 44. Conference articles: 80. Research reports: 49.

Contracts: Industrial: 15. Academic: 29.

Software: Transfers: 13. Patents: 2.

Student interns: 23. Visiting scientists: 62. Seminars: ~100. Teaching: ~550h taught.