Vipimage Bernard 2013

1 / 64

3D Echocardiography: recent advances and

future directions

1 University of Lyon, France

O. Bernard1, D. Barbosa2, M. Alessandrini2,

D. Friboulet1, J. D’hooge2

2 K.U. Leuven, Belgium

2 / 64

Outline

Context

Basics on image formation

Ultra realistic simulation

Echocardiographic image processing

Future directions

3 / 64

3D Echocardiographic imaging

In summary

Non invasive modality

Assess mechanical

properties of the heart

such as the strain in

real time

One of the cheapest

modality in 2D and 3D

4 / 64


Clinical useful information

Clinical indices such as the Ejection Fraction (EF)

or the Stroke Volume (SV)

Necessity to perform accurate segmentation

End Diastolic

Volume (EDV)

End Systolic

Volume (ESV)

𝑬𝑭(%) =𝑬𝑫𝑽 − 𝑬𝑺𝑽

𝑬𝑫𝑽∗ 𝟏𝟎𝟎

5 / 64


Clinical useful information

Strain and strain rate measurement

• Opens the door to local cardiac deformation assessment

Necessity to perform motion analysis

6 / 64

BASICS ON ULTRASOUND IMAGE FORMATION

7 / 64

Basics on US image formation

2D Ultrasound probe

Phased array transducer

(less than 192 elements)

Delays on each element

Possibility to focalize the

energy in various part of

the medium

y (elevation)

x (lateral)

z (axial)

Width

Pitch

Kerf Height

8 / 64


Different kind of signals

1

3

2 Rf image

Enveloppe image

B-mode image

9 / 64


Different kind of signals

1

3

2 Rf image

Enveloppe image

B-mode image Image mostly used in 3D

echocardiography

10 / 64


3D Ultrasound probe

2D matrix array transducer

(3000 elements involved)

Technical difficulties in

driving all the elements

Impact spatial resolution

Technical difficulties in

scanning the volume of

interest in real time

Impact temporal resolution

Open head of probe

11 / 64


Image properties

Typical voxel size: 0.80 x 1.00 x 1.00 𝒎𝒎𝟑

Temporal resolution: 20 volumes per second

Long Axis 4 chambers Short Axis Long Axis 2 chambers

12 / 64

ECHOCARDIOGRAPHIC IMAGE PROCESSING

13 / 64


Image properties

Spatial

No clear contours

Noisy nature (speckle)

Temporal

Speckle decorrelation

Linked to frame rate

14 / 64


Current needs

No real consensus on the accuracy of what we can

extract from this modality

Strong need of evaluation platform for quality

assurance of algorithms applied to this modality for:

• Segmentation

• Motion estimation

• Tissue characterisation

15 / 64

ULTRA REALISTIC SYNTHETIC ECHOCARDIOGRAPHIC SEQUENCES

16 / 64

Ultra realistic echocardiographic simulations

State-of-the-art

Ultrasound Simulator

Field II [Jensen et al., 1992]

Cole [Gao et al., 2009]

Creanuis [Varray et al., 2011]

Realistic Simulation

[Gao et al., UMB, 2009]

[Alessandrini et al., ICIP, 2012]

[De Craene et al., IEEE TMI, 2013]

17 / 64


BASICS

18 / 64

Simulator principle

Speckle

pattern

Simulate a realistic point spread

function that characterises the

ultrasound probe

Simulate a medium from a set of

scatterer points with specific

backscattering coefficients

∗

?

How many scatterers ?

Which positions ?

Which amplitudes ?

Which motion ?

sector in degree

depth

in m

m

-40 -30 -20 -10 0 10 20 30 40

20

40

60

80

100

120

19 / 64


MOST ADVANCED SOLUTIONS De Craene et al. – Philips Research France

20 / 64

Most advanced solutions in US simulations

[De Craene et al., TMI, 2013]

Anatomical

model

Obtained from MR segmentation

Electromechanical motion model

Properties

• Realistic motion model

• Need to improve image quality

Contractility Activation [Sermesant et al., TMI, 2013]

Ultrasound simulator

- Inside myocardium: motion derived from the EM model

- Outside myocardium: random scatterers position and motion

- Scatterers amplitudes: simple gaussian distribution

21 / 64


MOST ADVANCED SOLUTIONS Alessandrini et al. – Creatis, France

22 / 64



Improvement of image quality : Image-based approach

Build a simulation based on a real clinical sequence

Learn the motion to be applied to the scatterers inside

the myocardium from the real sequence

23 / 64



Improvement of image quality : Image-based approach

Build a simulation getting inspired by a real clinical

sequence

Learn the scatterers amplitudes from the real sequence

𝐴 = 10

𝐾20𝐼

𝐼𝑀𝐴𝑋− 1

• A: scatterers amplitude

• I : real image intensity

• K: controls dB range of the

resulting image

24 / 64



Real clinical recording

Simulated sequence

Reference motion

Properties

Simulation of surrounding structures

Simulation of image artifacts

No motion model

Only implemented in 2D

25 / 64


FUTURE DIRECTIONS

26 / 64

Future directions in US simulations

How to still improve simulations ?

Merge the model-based simulation with the image-

based one

Anatomical + Electromechanical models

Dedicated registration algorithm

Real clinical recording

27 / 64

Ultra realistic echocardiographic simulations

Future directions

Quantitative comparison of existing 3D

segmentation, motion and strain estimation

techniques

Improving the heart models for the generation of

a set of controlled pathological cases

Creation of a publicly available library of

sequences including clinically relevant cases

28 / 64


FEATURE EXTRACTION

29 / 64

Feature extraction

What kind of features ?

Information support: B-mode image

Most accessible and time efficient support

Edge information

Monogenic signal

Region information

First order statistics computed locally

30 / 64

ECHOCARDIOGRAPHIC FEATURE EXTRACTION

MOST ADVANCED SOLUTIONS Noble et al. – Institute of Biomedical

Engineering, U.K.

31 / 64

Most advanced algorithms in feature extraction

Monogenic signal in few words

Extension of the analytic signal in n-D

Assumption: local image patches have intrinsic

dimensionality one

Efficiently extract local amplitude, local orientation,

local phase and instantaneous frequency in the

direction of maximum energy

𝒄𝒐𝒔(𝝋 𝒙 )

32 / 64


[Rajpoot et al., ISBI, 2009]

Exploit the monogenic signal for edge detection in

3D echocardiography images

Feature asymmetry (FA) operator for phase-

congruency measure in the particular case of step

edges

33 / 64


34 / 64


[Stebbing et al., MEDIA, 2013]

Machine learning based on boundary fragment model

to classify the different detected edges

35 / 64


SEGMENTATION

36 / 64

3D Echo segmentation


Left ventricle

Full myocardium

3D Transesophageal

4 chambers

37 / 64

3D Echo segmentation


Left ventricle

Full myocardium

3D Transesophageal

4 chambers

38 / 64


State-of-the-art in 3D Left Ventricle segmentation

Without prior

Statistical Shape/Appearance Model Leung, ISBI 2010 Zhang, UMB 2013 Butakoff, FIMH 2011

With prior

Supervised tissue classification Lempitsky, FIMH2009 Verhoek, MLMI2011

Machine learning from large databases Yang, IEEE TMI 2011

Graph Cuts Juang, ISBI 2011

Dynamic Programming van Stralen, Academic Radiology 2005

Deformable models

Simplex Meshes Nillesen, Phy. Med. Biol. 2009

Level-sets Rajpoot, MedIA 2011

B-Spline Explicit Active Surfaces Barbosa, UMB 2013

Kalman-based surface tracking Dikici, MICCAI 2012

39 / 64


State-of-the-art : Performance evaluation

Study Year #

Exams Analysis time (s)

EDV MAD (μ±σ mm) R BA (μ±2σ ml)

Van Stralen 2005 14 90+Manual Init 0.93 -10±60 X

Nillesen 2009 5 X X 6.7±4.6 X

Lempitsky 2009 14 2-22 X X X

Leung 2010 99 X 0.95 1.5±40 2.91±1.0

Juang 2011 4 X X X 2.4±3.2

Rajpoot 2011 34 X X -5.0±49 2.2±0.7

Butakoff 2011 10 X X 6.4±14 1.6±1.1

Butakoff 2011 20 X X 3.1±47 1.8±1.9

Verhoek 2011 25 2 X X X

Yang 2011 67 1.5 X 1.3±12 1.3±1.1

Dikici 2012 29 0.08 X X 2.0±X

Barbosa 2013 24 1 0.97 -2.4±23 X

Zhang 2013 50 45-60 0.83 4.2±35 3.2±1.0

40 / 64


State-of-the-art : Performance evaluation

Study Year #

Exams Analysis time (s)

EDV MAD (μ±σ mm) R BA (μ±2σ ml)

Van Stralen 2005 14 90+Manual Init 0.93 -10±60 X

Nillesen 2009 5 X X 6.7±4.6 X

Lempitsky 2009 14 2-22 X X X

Leung 2010 99 X 0.95 1.5±40 2.91±1.0

Juang 2011 4 X X X 2.4±3.2

Rajpoot 2011 34 X X -5.0±49 2.2±0.7

Butakoff 2011 10 X X 6.4±14 1.6±1.1

Butakoff 2011 20 X X 3.1±47 1.8±1.9

Verhoek 2011 25 2 X X X

Yang 2011 67 1.5 X 1.3±12 1.3±1.1

Dikici 2012 29 0.08 X X 2.0±X

Barbosa 2013 24 1 0.97 -2.4±23 X

Zhang 2013 50 45-60 0.83 4.2±35 3.2±1.0

41 / 64

ECHOCARDIOGRAPHIC IMAGE SEGMENTATION

MOST ADVANCED SOLUTIONS Barbosa et al. – Creatis – KULeuven,

France, Belgium

42 / 64

Most advanced algorithms in segmentation

[Barbosa et al., UMB, 2013]

Formalism specifically dedicated for near real time

3D segmentation

Exploit equivalence under specific constraint

between implicit and explicit representation inside

the variational framework of level-set methods

Solve a 3D problem in a 2D space (dimensionality

reduction)

Segmentation performed through a B-Spline

formulation in order to decrease even more

computational time

43 / 64



Interface evolution corresponds to a succession of

simple separable convolutions

with: 𝑬𝒆𝒙𝒑 Energy to minimize

𝒄[𝐤] B-Spline coefficients

𝒈 (𝒙∗) data attachment term

𝜷𝒉 B-spline function

44 / 64


Au et Av : interior and exterior regions

used for the computation of the

local means

𝐱∗ ∈ ℝ2with coordinates {𝑥1, 𝑥2}

𝑰 (𝐱∗) restriction of 𝑰 to interface 𝚪

B(x,y)

x


Evolution equation

With

45 / 64


Results

Validation performed on 24 patients among whom

80% present different cardiac pathologies

All data were manually segmented by 3 experts

Corr. Coeff: 0.97 (EDV), 0.97 (ESV), 0.91 (EF)

Average Cpu time per volume: 25 ms

2.9-GHz 4-Core laptop, with 7.7GB Memory running Fedora

46 / 64


MOTION ESTIMATION

47 / 64

Motion estimation

State-of-the-art in motion estimation

Many algorithms have been proposed

The most well known is based on block matching

Most of them are based on intensity conservation

• Assumption: a moving structure should conserve

its brightness appearance between two consecutive time instants

48 / 64

Motion estimation

State-of-the-art in motion estimation

Without prior

Statistical model Wang, STACOM, 2010 Leung, UMB, 2011

With prior

Mechanical model Papademitris, MEDIA, 2001 Sermesant, MICCAI, 2001

B-Spline transformation Heyde, STACOM, 2013 De Craene, MEDIA, 2012 Piella, STACOM, 2013

Optical flow Alessandrini, TIP, 2013 Tautz, STACOM, 2013 Mansi, IJCV, 2011

Block-matching Isla, JASE, 2011 Crosby, UMB, 2009 Seo, JACC, 2011

49 / 64

Motion estimation

MICCAI’12 Challenge [De Craene et al., TMI, 2013]

Synthetic motion from an Electro-Mechanical model

Simulate normal and pathological cases (13 patients)

Comparison of 5 methods

• 2 B-Spline transformation based methods

• 3 Optical flow based methods

50 / 64

Motion estimation evaluation

Magnitude errors

Globally over a cardiac cycle

For each time instant in the sequence

Motion estimation

51 / 64

Motion estimation

MICCAI’12 Challenge

Average magnitude error over a cardiac cycle

Ischemic sub groups Dyssynchrony sub groups

52 / 64

Motion estimation

MICCAI’12 Challenge

Average magnitude error over a cardiac cycle

Ischemic sub groups Dyssynchrony sub groups

53 / 64

ECHOCARDIOGRAPHIC MOTION ESTIMATION

MOST ADVANCED SOLUTIONS Heyde et al. – KULeuven, Belgium

54 / 64

Most advanced algorithms in motion estimation

[Heyde et al., FIMH 2013]

Free-form deformations in 3D to estimate motion

Model displacement in B-Spline space

Registration performed in a recursive minimization way

𝒅𝑡→𝑡+1 𝒙 = 𝒄[𝑘, 𝑙]𝛽𝜎𝑥 𝑥 − 𝑘 𝛽𝜎𝑦 𝑦 − 𝑙

1,𝑁𝑘 ,[1,𝑁𝑙]

𝐸 = 𝑆 𝑰𝒕, 𝑰𝒕+𝟏, 𝒄 + 𝜆𝑅(𝒄)

with: 𝑬 Energy to minimized

𝑺 Sum of Square diff. measurement

𝑹 Regularization function

55 / 64



Illustration in 2D

Extension in 3D

Derivation of

motion and strain

Image warping Registration

56 / 64



Anatomical shaped control grid

Less control points (efficiency)

Naturally enforce smoothness in the physiologically

relevant directions

Anatomically shaped control grid Regular control grid

57 / 64


Acute myocardial infarct

On going results

Feasibility on 6 clinical data

3 healthy patients + 3 with pathologies

(Acute myocardial infarct)

Average Cpu time per volume:

4 minutes

2.8-GHz 4-Core laptop, with 8.0GB

Memory running Windows

58 / 64

FUTURE DIRECTIONS

59 / 64

Future directions

Key point: image quality

Deeply depend on the capacity to improve or not

3D echocardiographic image quality

Scenario 1: No real improvement

Needs of quantifying what

exactly we could extract from

this modality

Needs of going further in

adapted image processing

Needs of doing real time

processing

60 / 64

Future directions

Key point: image quality

Deeply depend on the capacity to improve or not

3D echocardiographic image quality

Scenario 2: Improvement is possible

Needs of working on the

acquisition process itself

61 / 64

Future directions

3D echocardiographic acquisition improvements

Strong efforts are currently made to improve image quality

Temporal resolution

Compressed sensing in ultrasound

[Wagner et al., IEEE TSP, 2012]

Spatial resolution

Fourier ultrasound imaging

[Garcia et al., IEEE UFFC, 2013]

Modify the image to facilitate motion estimation

Ultrasound-tagging imaging

[Liebgott et al., IEEE UFFC, 2013]

62 / 64

Future directions

Ultrasound-tagging [Liebgott et al., UFFC 2013]

Main idea: reproduce the principle of tagged MR

imaging in ultrasound

63 / 64

Future directions

Classical B-mode image US-tagging

Tagged ultrasound imaging [Liebgott et al., UFFC 2013]

http://www.creatis.insa-lyon.fr/us-tagging/

64 / 64

THANK FOR YOUR ATTENTION

Vipimage Bernard 2013

Documents

Transcript of Vipimage Bernard 2013