Post on 25-Dec-2018
IMAGING DEVELOPMENT IN THE EMBRYONIC
HEART OVER MULTIPLE SPATIAL
DIMENSIONS, MODALITIES, AND TIME-SCALES
Michael Liebling
Electrical & Computer Engineering Department
University of California, Santa Barbara
sybil.ece.ucsb.edu
Large Data Sets in Medical Informatics
Institute for Mathematics and its Applications, University of Minnesota
16 November 2011
<< < > >> B F R I Page 1
Channels
optical properties (absorption,
scattering, refraction index, wavelength)
chemical properties (pH,...)
physical forces,
gene expression level (e.g. via fluorophore)
Space
X, Y, Z
Time
Number of samples
n
Resolution and breadth bottlenecks
• instrument resolution, bandwidth
• available time
• sample integrity
Can we increase imaging resolution in one dimension (space, time,
channels) without a«ecting resolution in the others?
The imaging-space in biology is high-dimensional
<< < > >> B F R I Page 2
Gra
y’s
Anato
my
Day 22 (Human) Adult (Human)Congenital Heart Defects:
occur in 0.8% of newborn infants,
are the leading cause of birth defect related deaths
Heart is Functional (beating!) while it still develops
<< < > >> B F R I Page 3
A
V
Advantages:
• zebrafish are vertebrates
• reproduce externally and rapidly
• relatively transparent embryos
• may be genetically engineered to
express fluorescent markers in spe-
cific tissues [e.g. Tg(gata1:GFP)]
48 hpf (hours post fertilization)
mb: midbrain
ot: otocyst
e: eye
h: heart
yolk: yolk mass
Embryonic Zebrafish Heart is (Almost) Perfect Model
<< < > >> B F R I Page 4
Human umbilical vein endothelial cells exposed to (during 24 h):
Static Conditions (no flow) Laminar Shear Stress
down-regulated, TSS!LSS), thus further indicating that culturedendothelial cells can discriminate between these distinct types offluid mechanical stimulation. These transcriptional profilingdata, analyzed in terms of global patterns of gene regulation,thus confirm and extend our working hypothesis that the endo-thelial cell is a mechanosensitive element in the blood vessel wall,whose phenotype can be modulated at the transcriptional levelby fluid shear stresses (7, 11).
Categorization and Functional Annotation of Differentially ExpressedEndothelial Cell Genes. To begin to comprehend the endothelialphenotypes emerging in this model of biomechanical stimula-tion, we categorized the patterns of expression of known genesby different analytical methods. First, the genes that exhibitedthe greatest degree of change under the experimental conditionsexamined were grouped according to the direction (up or down)of their regulation. In Table 2, the 10 most regulated namedgenes are listed for each condition pair examined (LSS vs. static,TSS vs. static, TSS vs. LSS). Second, an average-linking hierar-chical clustering algorithm (17) was applied to group namedgenes that were similarly regulated by each of the comparisonconditions. As seen in Fig. 2, blocks of genes with similar colorcoding (connoting similar regulation across compared condi-tions) comprised qualitatively different categories. Some cate-gories appear to be enriched for genes associated with particularfunctions. For example, many genes down-regulated by LSS (butless so with TSS) are known to be involved in the process of genetranscription, including Brahma (SWI!SNF matrix associated,subfamily a, member 2) and chromatin assembly factor 1, subunitB (p60). A rigorous analysis of such enrichment awaits asystematic classification of human transcripts into functionalcategories (18). It is notable that Fig. 2 contains examples of allpossible combinations of up- and down-regulation among thethree experimental conditions examined. For example, certaingenes such as connective tissue growth factor were comparablyup-regulated by both LSS and TSS, whereas others, such asCD164 (sialomucin), were comparably down-regulated by bothstimuli, as compared with static control cultures. A subset ofgenes were found to be down-regulated by LSS and up-regulatedby TSS (fibrogenic lymphokine, activating transcription factor5), and vice versa (eukaryotic translation initiation factor 2,serine protease inhibitor Kazal type 2). These latter categoriesare particularly interesting because they may contain pathophysi-ologically relevant genes that would be hypothesized to contrib-ute to biomechanically induced ‘‘atheroprotective’’ or ‘‘athero-prone’’ endothelial phenotypes in the in vivo setting (14). Thisentire data set can be found at a searchable database maintainedby our laboratory at http:!!vessels.bwh.harvard.edu!papers!PNAS2001.
As a third approach, we subjectively grouped highly regulatedgenes, which have known or putative functions in mechanosig-naling, vascular response-to-injury reactions, and atherogenesis(Table 3). We validated the regulation of several of these genesat the mRNA level by using quantitative real-time PCR (Taq-Man), and their translation into protein by using Western blotanalysis (Table 3). This list contains 28 genes of diverse func-tions, including genes involved in lipid metabolism (apolipopro-tein E, long chain fatty acid-CoA ligase 3, megalin); the puri-noreceptor, P2X4, which mediates fluid shear stress-dependentactivation of calcium influx in endothelial cells (19); as well asisoforms of !, " and # subunits of G proteins. The coordinatedup-regulation of trimeric G proteins is interesting because thesemolecules have been implicated in mechanosignaling in responseto fluid shear stress (20, 21). Certain genes involved in vascularresponses to injury also showed significant regulation, includingfibrogenic lymphokine (fibrosin), thioredoxin reductase, andmatrix-gla protein. Thioredoxin reductase is recognized to be animportant modulator of the redox state of the cell (22), whereas
matrix-gla protein is a negative regulator of vascular calcification(23). Other genes whose regulated expression also may berelevant to vascular pathophysiology include connective tissuegrowth factor (CTGF), a pleiotropic growth factor, which can acton endothelial cells, smooth muscle cells, and fibroblasts (24);and CYP1B1, a cytochrome P450 enzyme that catalyzes estradiolhydroxylation and activates exogenous chemicals (25). Finally,we observed significant biomechanical regulation of variousgenes encoding extracellular matrix components, matrix recep-tors, and matrix remodeling enzymes, such as fibronectin, se-creted protein acidic and rich in cysteine (SPARC), laminin "1,
Fig. 3. Changes in cytoskeletal elements of endothelial cells exposed to LSS.(a) Cytoskeleton-related genes showing significant up- or down-regulation inresponse to LSS stimulation. (b) Fluorescence confocal micrographs of theapical (supranuclear) region of HUVEC stained green for F-actin (Top), myosinheavy chain (Middle), and plectin (Bottom). Cells were counterstained withSYTOX (red) to identify nuclei. (Left) HUVEC under static (no flow) conditions;(Right), HUVEC exposed to LSS (10 dyn!cm2 for 24 h).
Garcıa-Cardena et al. PNAS " April 10, 2001 " vol. 98 " no. 8 " 4483
CELL
BIO
LOG
Y
Flow
G. García-Cardeña, et al. PNAS 98(8), pp. 4478–4485, 2001
Cells Remodel when Exposed to Flow (in vitro)
<< < > >> B F R I Page 5
Many Genes Respond to Flow-Induced Shear-Stress (in vitro)
Up-Regulated Genes Down-Regulated GenesTable 1. Genes that increase under shear stress detected by DNA microarray analysis
Group* Gene Ratio (SS!Cont)
Antioxidants Cytochrome P450 1B1 6 h 7.55 ! 1.28AA448157 24 h 9.70 ! 2.71Cytochrome P450 1A1 6 h 3.90 ! 0.48AA418907 24 h 11.15 ! 3.75Heme oxygenase-1 6 h 3.08 ! 0.84T71606 24 h 1.78 ! 0.18NAD(P)H:quinone oxidoreductase (NQO1) 6 h 1.28 ! 0.12AA458634 24 h 2.09 ! 0.32
Proliferation!differentiation Zinc finger protein EZF!GKLF 6 h 4.56 ! 0.69H45668 24 h 2.78 ! 0.54Receptor tyrosine phosphatase 6 h 3.73 ! 0.77AA486403 24 h 1.82 ! 0.05TGF-!-stimulated clone 22 (TSC-22) 6 h 2.18 ! 0.23AA664389 24 h 2.18 ! 0.25Tyrosine kinase receptor precursor 6 h 2.31 ! 0.36TIE-2 H02848 24 h 1.80 ! 0.48Tyrosine kinase HTK 6 h 1.40 ! 0.05T51849 24 h 2.09 ! 0.79E2F transcription factor 5, p130 binding 6 h 2.05 ! 0.71AA455521 24 h 1.56 ! 0.64Human orphaned G protein coupled receptor 6 h 2.39 ! 1.17N53172 24 h 1.13 ! 0.31Glycyl tRNA synthetase 6 h 1.97 ! 0.56AA629909 24 h 1.08 ! 0.19Jagged 1 (Human HJ1) 6 h 2.22 ! 0.37R70685 24 h 1.82 ! 0.51
Vascular tone Argininosuccinate synthetase 6 h 2.41 ! 0.21AA676466 24 h 3.04 ! 0.88"-Galactosidase A precursor 6 h 1.42 ! 0.02AA251784 24 h 2.56 ! 0.23Vasoactive intestinal polypeptide receptor 6 h 1.39 ! 0.20precursor (VIPR1) H73241 24 h 2.13 ! 0.28PGHS-2 6 h 2.20 ! 0.45AA644211 24 h 0.91 ! 0.16
ECM!cytoskeleton Elastin 6 h 2.93 ! 0.42AA459308 24 h 2.26 ! 0.59Connexin 37 6 h 2.45 ! 0.88H44032 24 h 1.48 ! 0.38"-Spectrin 6 h 1.76 ! 0.17T60117 24 h 1.99 ! 0.51Galectin 3 6 h 1.22 ! 0.09AA630328 24 h 2.16 ! 0.56
Immune!inflammation Podocalyxin-like protein 6 h 3.37 ! 0.55N64508 24 h 1.77 ! 0.21CD34 6 h 1.70 ! 0.20AA043438 24 h 2.41 ! 0.22IL-1 receptor, type 1 precursor 6 h 1.51 ! 0.38AA464526 24 h 2.22 ! 0.18Leukocyte elastase inhibitor 6 h 1.06 ! 0.11AA486275 24 h 2.11 ! 0.58
Transcription factor Glucocorticoid-induced leucine zipper protein 6 h 3.35 ! 0.70AA775091 24 h 3.28 ! 0.79
Transport systems Human prostaglandin transporter 6 h 3.33 ! 1.05AA037014 24 h 2.49 ! 0.19Heat shock protein 70 6 h 2.08 ! 0.31AA629567 24 h 0.76 ! 0.09Chromogranin A 6 h 1.05 ! 0.17R36264 24 h 2.07 ! 0.83
Protein modification Paired basic amino acid cleaving system 4 6 h 1.74 ! 0.30AA251457 24 h 2.44 ! 0.21
RNA degradation RNase A family, 1 6 h 0.91 ! 0.18AA485893 24 h 2.01 ! 0.06
Thrombosis S100 calcium binding protein A10 6 h 1.27 ! 0.23AA444051 24 h 1.96 ! 0.32
*Genes were grouped based on function. Many genes can be assigned to more than one group (e.g., transcriptionfactors).
McCormick et al. PNAS " July 31, 2001 " vol. 98 " no. 16 " 8957
ENG
INEE
RIN
G
previous studies of PG H S-2 protein levels under similar shearconditions (18) .
T he most dramatically up-regulated gene expression wasobser ved in cytochromes P450 (C Y P) 1A 1 and 1B 1 (F ig. 1) .C lassically, the C Y P gene families have been associated withcellular detoxification mechanisms. O f the 19 C Y P genes presenton the G F 211, only C Y P 1A 1 and 1B 1 were affected by shearstress in H U V E C . C Y P 1A 1 activity can be induced in humanendothelial cells (E C ) by toxic aromatic hydrocarbons, but notC Y P 1B 1 (19–21) . C Y P 1A 1 has been postulated to participatein endogenous signaling of oxidative processes (22) . F urther-more, there is evidence that the production of an endothelial-derived hyperpolarizing factor from arachidonic acid in endo-thelial cells is catalyzed by C Y P 1A 1; and, loss of this gene incultured E C correlates with dedifferentiation (23) . E xpression ofC Y P 1B 1 in E C is much less studied than C Y P 1A 1. Acomparison of senescent fibroblast, epithelial, and H U V E C celllines revealed C Y P 1B 1 to be up-regulated only in senescentH U V E C (24) . T he strong induction of these C Y P genes by shearstress is consistent with the suggestion that physiological levelsof shear stress are protective for endothelium.
C T G F , which was initially purified and identified fromH U V E C -conditioned medium, is elevated in fibrotic lesions, and
may play a role in the development of fibrotic diseases (25) . I thas also been shown to be highly expressed in vascular cells inatherosclerotic lesions, but not in normal arteries (26) . T heselesions are usually found in regions of low wall shear stress, oftenwith complex, recirculating blood f low patterns (1) . T he strikingdown-regulation of C T G F expression that we obser ve in re-sponse to normal arterial levels of shear stress is consistent withthese findings, and with the hypothesis that physiological arterialshear stress protects against fibrotic and atherosclerotic diseaseprocesses.
M any of the other genes in T ables 1 and 2, whose biologicalfunctions are known, are associated with vascular biologicalpathways that are regulated at least in part by shear stress. B yidentifying additional genes in these systems it will be possible tofurther define the complex mechanisms by which shear stresscontrols them. F or example, shear stress is considered by manyto be the most important stimulus for N O production in E C (27) .Several genes identified as shear stress responsive from our D N Amicroarray analysis may play a role in the regulation of N Oproduction by shear stress, and suggest the following pathwaysfor shear stress regulation of N O production in endothelial cells.G enerating hypotheses in the absence of further supporting datamay be considered excessive, but we believe they provide stimulifor further work.
Table 2. Genes that decrease under shear stress detected by DNA microarray analysis
Group Gene Ratio (SS Cont)
Vascular tone Endothelin-1 6 h 0.21 0.08H11003 24 h 0.20 0.05Caveolin-1 6 h 0.49 0.09AA055835 24 h 0.49 0.13
Extracellular matrix CTGF 6 h 0.13 0.06AA598794 24 h 0.10 0.02Cardiac gap junction (Connexin 43) 6 h 0.58 0.24AA487623 24 h 0.39 0.10Matrilin-2 6 h 0.81 0.11AA071473 24 h 0.52 0.08
Proliferation differentiation Spermidine spermine N1-acetyltransferase 6 h 0.37 0.11R58991 24 h 0.20 0.07Cyr61 6 h 0.35 0.15AA777187 24 h 0.41 0.18S1-5 (fibrillin-like, FBNL) 6 h 0.65 0.06AA875933 24 h 0.21 0.05BMP-4 6 h 0.45 0.03AA463225 24 h 0.58 0.13Aldehyde dehydrogenase 1 6 h 0.50 0.22AA664101 24 h 0.52 0.18Adenylosuccinate synthetase 6 h 0.59 0.29AA431414 24 h 0.74 0.24Gene for H4 histone 6 h 0.80 0.10AA868008 24 h 0.50 0.05
Cytoskeleton Nonmuscle myosin heavy chainB Smemb 6 h 0.48 0.14AA490477 24 h 0.87 0.20
-Tubulin 6 h 0.88 0.14AA865469 24 h 0.50 0.04
GTPases Rho B 6 h 0.63 0.10AA495790 24 h 0.27 0.07Guanylate binding protein 1 (GBP1) 6 h 0.48 0.11AA486850 24 h 0.96 0.24
Transcription factor SL3-3 enhancer factor 2 (SEF2-1A) 6 h 0.40 0.10AA669136 24 h 0.52 0.08
Protein modifier Sialyltransferase 1 6 h 0.54 0.13AA598652 24 h 0.46 0.18
Chemotaxis Monocyte chemotactic protein 1 (MCP-1) 6 h 0.35 0.14AA425102 24 h 0.28 0.07
Unknown Myosin heavy chain homolog (Doc1) 6 h 0.39 0.08W69790 24 h 0.58 0.17
8958 www.pnas.org cgi doi 10.1073 pnas.171259298 McCormick et al.
McCormick et al.
“DNA microarray reveals changes in gene expression of shear stressed human umbilical vein endothelial cells,”
PNAS, 93 (16), 2001
Flow-Sensitive Genes Mediate Cell Remodeling
<< < > >> B F R I Page 6
Reducing the retrograde flow fraction early in the heart development leads
to severe valve dysmorphologies.
J. Vermot, A. Forouhar, ML, et al., PLOS Biology, vol. 7, iss. 11, e1000246, 2009.
Normal valvulogenesis depends on early flow patterns
<< < > >> B F R I Page 7
96hpf
Control Reducing retrograde flow fraction
leads to valve dysmorphologies
100µm
A
V
A
VA
V
J. Vermot, A. Forouhar, ML, et al., PLOS Biology, vol. 7, iss. 11, e1000246, 2009.
Normal valvulogenesis depends on early flow patterns
<< < > >> B F R I Page 8
96hpf
Control Reducing retrograde flow fraction
leads to valve dysmorphologies
100µm
A
V
A
VA
V
AA
V V
J. Vermot, A. Forouhar, ML, et al., PLOS Biology, vol. 7, iss. 11, e1000246, 2009.
Normal valvulogenesis depends on early flow patterns
<< < > >> B F R I Page 9
inhibits atrio-ventricular klf2a expression
Atrium
Ventricle
AtriumVentricle
klf2a, 48hpf klf2a, 48hpf
Control Reduced RFF @36-46hpf
J. Vermot, A. Forouhar, ML, et al., PLOS Biology, vol. 7, iss. 11, e1000246, 2009.
Reducing retrograde flow fraction at 36hpf
<< < > >> B F R I Page 10
Oscillatory flow −→ klf2a expression −→ Valve formation
J. Vermot, A. Forouhar, ML, et al., PLOS Biology, vol. 7, iss. 11, e1000246, November 2009.
⇒ Bad idea to alter (stop) cardiac function to accommodate
time-lapse imaging
How does flow a«ect heart morphogenesis?
<< < > >> B F R I Page 11
Heart
Morphogenesis Heart Function
Cellular Regulation
Goal: develop imaging methods to capture multi images withoutperturbing heart function or morphogenesis
Morphogenesis and function are entangled
<< < > >> B F R I Page 12
Heart
Morphogenesis Heart Function
Cellular Regulation
Time-lapse microscopy
3D microscopyHigh-speed imaging
Fluorescence imaging
Goal: co-localized measurements over multiple dimensions, at
multiple spatial and temporal scales, and via multiple imaging
modalities.
Applicability of traditional imaging is limited
<< < > >> B F R I Page 12
Experimental
System
Microscopy
Image
Processing
Image
Analysis
Modeling
Studying
Heart Development
Integrated approach to imaging allows for feedback
<< < > >> B F R I Page 13
Experimental
System
Microscopy
Image
Processing
Image
Analysis
Modeling
Studying
Heart Development
Integrated approach to imaging allows for feedback
<< < > >> B F R I Page 13
1. Acquire multi-dimensional data
(3D+Time+λ+. . . ) at high frame-rate
2. Merge and segment multi-modal data
3. Joint imaging and untangling
of morphogenesis and function
Steps towards integrated imaging of cardiogenesis
<< < > >> B F R I Page 14
1. Acquire multi-dimensional data
(3D+Time+λ+. . . ) at high frame-rate
2. Merge and segment multi-modal data
3. Joint imaging and untangling
of morphogenesis and function
Steps towards integrated imaging of cardiogenesis
<< < > >> B F R I Page 14
Framerate (frames per second)
2.5 5 10 20 35 68 104
Integration Time (ms)
400 200 100 50 25 12 6
Optimal framerate is a compromise between:
• minimal motion blur
• su»cient photon count
Vermot, Fraser, ML, HFSP Journal 2008, http://hfspj.aip.org/doi/10.2976/1.2907579.
Spatial, Temporal Resolutions and SNR are Linked
<< < > >> B F R I Page 15
Point
Source
~Poisson(n)
Point Spread Function
(Spatial Probability
Density Function ~Airy)
Optical
System
Imm
ob
ile
So
urc
eM
ob
ile S
ourc
e
(velo
city v
)
v
Tv
Integration Time: T
PSF width:
ÉxNew PSF width:
Éx′ = T v + Éxwith T : integration time
v : velocity
Example:
fs = 25 Hz
T = 1/fs = 0.04 s
v = 1 mm/s
T v = 40 µm
Vermot, Fraser, ML, HFSP Journal 2008, http://hfspj.aip.org/doi/10.2976/1.2907579.
Poisson meets Airy
<< < > >> B F R I Page 16
Motion modifies point spread function (PSF)
PSF width: ÉxNew PSF width: Éx′ = T v + Éx with T : integration time, v : velocity
Example: fs = 25 Hz, T = 1/fs = 0.04 s, v = 1 mm/s, T v = 40 µm
Balancing Spatial and Temporal Resolution
<< < > >> B F R I Page 17
Required Spatial
Resolution ∆x (µm)
Velocity of
of Structure
(µm/s)
0.001
0.001
0.01
0.01
0.1
0.1
1
1
10
10
100
100
1000
1000
1f
sp
1fp
01s
f 1p
001
s
f 1p
51m
tuni
se
f 1p
3h
sruo
f 1
d pay
01f
sp001
f sp
001
f 0
sp00,01
f 0
sp
001
f 000,
sp
Cell Migration
Cardio-vascular
cell dynamics
Calcium
Sparks
VesicleTrafficking
Microtubule
Growth
Chromosome
Dynamics
J. Vermot, S.E. Fraser, ML, HFSP Journal (2008), http://hfspj.aip.org/doi/10.2976/1.2907579.
Framerate Requirements Depend on Target Resolution, Structure Velocity
<< < > >> B F R I Page 18
1 2 3 4 5 6 7
1 2 3 4 5 6 7
8 9 10 11
5 6 7 8 9 10 11
12 13 14 15
9 10 11 12 13 14 15
1 2 3
1’ 2’ 3’
4 5 6 7
5 6 7 8 9 10 11
9 10 11 12 13 14 15
Time
Time
Time
Multiple Acquisitions
3) Synchronization
1) Acquisition
2) Cutting
4) Noise Reduction
(e.g. Mean or Median, etc.)
S.K. Bhat, I. Larina, K. Larin, M.E. Dickinson, ML, Optics Letters, 34 (23), 2009.
Multi-Cycle Noise Reduction
<< < > >> B F R I Page 19
Optical Coherence Tomography (50 frames per second) to image beating
heart in E9.5 mouse embryo.
Signal to noise ratio:
57.12 dB 60.71 dB 58.77 dB 61.67 dB
256×512 pixels per frame
S.K. Bhat, I. Larina, K. Larin, M.E. Dickinson, ML, Optics Letters, 34 (23), 3704–06,
2009.
Fast Optical Coherence Tomography at High SNR
<< < > >> B F R I Page 20
2D slice of zebrafish heart (60 fps)
ML, J. Vermot
[38 hpf wildtype zebrafish, BODIPY FL C5-ceramide]
Zebrafish heart @ 2–3 s/volume
Heart beats ∼5 times per scan.
Imaging may be fast in 2D but too slow for 3D
<< < > >> B F R I Page 21
Non-gated acquisition. . .
<< < > >> B F R I Page 22
. . . requires synchronization
<< < > >> B F R I Page 22
4D Imaging via Post-Acquisition Synchronization
<< < > >> B F R I Page 22
Before synchronization After synchronization
A.S. Forouhar, ML
[Tg(cmlc2:GFP) 38 hpf zebrafish, Huai-Jen Tsai, National Taiwan University ]
in “Imaging: a Laboratory Manual”, CSHL Press (NY), Chap. 53, 2011, 823–830.
4D in vivo, Fast Confocal Microscopy is Possible
<< < > >> B F R I Page 23
ML, A.S. Forouhar, M. Gharib, S.E. Fraser, M.E. Dickinson, J. Biomedical Optics, 10 (5), 2005.
Functional Cardiac Imaging at Microscopic Scales
<< < > >> B F R I Page 24
Raw 2D OCT Sequence Denoised and Synchronized
K.V. Larin, I.V. Larina, ML, M. E. Dickinson "Live imaging of early developmental processes in mammalian
embryos with optical coherence tomography " Journal of Innovative Optical Health Sciences, vol. 2, no. 3, pp.
253-259, July 2009.
Optical Coherence Tomography Image Denoising
<< < > >> B F R I Page 25
t
k = 1
k = 2
x
y
s2
Before Synchronization
After Synchronization
ML, H. Ranganathan, Proc. SPIE Wavelets XIII, Vol. 7446, 744602, 2009
Image synchronization problem
<< < > >> B F R I Page 26
0 10 20 30 40 50 60 70 80−1
−0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
Reference
Test
And what if the heartbeat is irregular?
<< < > >> B F R I Page 27
0 10 20 30 40 50 60 70 80−1
−0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
Reference
Test
Rigid synchronization is not adapted
<< < > >> B F R I Page 27
ML, A.S. Forouhar, et al., Dev. Dyn, 2006
[Tg(gata1:GFP) 54 hpf zebrafish]
Method is limited by aperiodicities in data
<< < > >> B F R I Page 28
0 10 20 30 40 50 60 70 80−1
−0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
Reference
Test
Can we compensate for (slight) irregularities?
<< < > >> B F R I Page 29
0 10 20 30 40 50 60 70 80−1
−0.5
0
0.5
1
1.5
2
2.5
3
3.5
4
Reference
Test
Warped
Nonuniform Registration Warps Test Sequence
<< < > >> B F R I Page 29
Acquisition Model
Imeasured(x, zk , t) =ZZZI(x′, z, w−1
k (t))h(x − x′, zk − z) dx′dz,
wk(t): unknown warping functions; h(x, z): microscope point spread function
Hypothesis
|I(x, z, t) − I(x, z, t + T )| ≈ 0, ∀tObjective criterion (minimize to synchronize slice-sequence k ′ to k):
Qk,k′,wk {wk′} =Z L0
ZZR2
|Im(x, zk , wk(t)) − Im(x, zk ′ , wk ′ (t))|dx dt
+λZ T0
ββββ d
dtwk ′ (t) − 1
ββββdt.
Goal:
Recover warping functions wk ′ (t)Additional assumptions:
Warping functions are continuous, non-negative, and monotonically increasing
Initialization: w1(t) = t
Nonuniform temporal registration
<< < > >> B F R I Page 30
’
Goal: Match test sequence to reference sequence
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Matrix to contain dissimilarity measure between pairs of frames
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Similar Frames: low dissimilarity value
Absolute di«erence or
Mean squared di«erence or
Normalized Mutual Information
−SNMI(A,B) = −H(A) + H(B)
H(A,B)
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Dissimilar Frames: higher dissimilarity value
Absolute di«erence or
Mean squared di«erence or
Normalized Mutual Information
−SNMI(A,B) = −H(A) + H(B)
H(A,B)
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Absolute di«erence or
Mean squared di«erence or
Normalized Mutual Information
−SNMI(A,B) = −H(A) + H(B)
H(A,B)
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Dissimilarity matrix as a graph
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Synchronization problem equivalent to finding minimal-cost path
through vertices (extra cost for stretching or shrinking)
Rigid registration:
constrain to paths that are diagonal (135◦)
Non-rigid temporal registration:
Dynamic programming approach,
ML, et al, Proc. ISBI (2006), pp. 1156-1159
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Given a fit in column c, find best fit in column c − 1
Stretching or shrinking sequence yields additional cost
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Mark best fit with arrow and store total cost
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Proceeding recursively from left to right
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
’
Find minimal cost on bottom row and follow graph from right to left
Dynamic Programming Nonuniform Registration
<< < > >> B F R I Page 31
ML, A.S. Forouhar, et al., Dev. Dyn, 2006
[Tg(gata1:GFP) 54 hpf zebrafish]
Comparison of Reconstruction Techniques
<< < > >> B F R I Page 32
. . . without a«ecting frame rate!
(Swept Source) Optical Coherence Tomography
J. Yoo, I.V. Larina, K.V. Larin, M.E. Dickinson, and ML, Biomedical Optics Express, 2(9),
2614–2622, 2011
Mosaicing extends field of view. . .
<< < > >> B F R I Page 33
1. Acquire multi-dimensional data
(3D+Time+λ+. . . ) at high frame-rate
2. Merge and segment multi-modal data
3. Joint imaging and untangling
of morphogenesis and function
Steps towards integrated imaging of cardiogenesis
<< < > >> B F R I Page 34
Bright-field (BF)
+ High frame rates (≥1000 fps)
- Images lack specificity
Fluorescence
+ Excellent specificity
- Low frame rates (≤ 50 fps)
JungHo Ohn
Brightfield microscopy: fast but lacks specificity
<< < > >> B F R I Page 35
J.O
hn
Acquiring Multiple Modalities
<< < > >> B F R I Page 36
a
b
c
d
e
f
g
h
time
timetime
time
Parallel Multi-Channel Imaging Sequential Multi-Channel Imaging (slow)
Sequential Multi-Channel Imaging (fast version, before temporal alignment) Multi-Modal Temporal Alignment
Key idea: Fast sequential acquisition, post-acquisition synchronization
J. Ohn et al. genesis, vol. 49, no. 7, pp. 514–521, 2011.
Multi-channel imaging, revisited for speed
<< < > >> B F R I Page 37
Brightfield Fluorescence (44 fps), Tg(cmlc2:GFP)
Multi-modal images can be synchronized by choosing a mutual
information-based criterion as an image-similarity merit function
ML, H. Ranganathan, Proc. SPIE Wavelets XIII, Vol. 7446, 744602, 2009
Sequentially-acquired (nongated) sequences require synchronization
<< < > >> B F R I Page 38
Brightfield Fluorescence, Tg(cmlc2:GFP)
Multi-modal images can be synchronized by choosing a mutual
information-based criterion as an image-similarity merit function
ML, H. Ranganathan, Proc. SPIE Wavelets XIII, Vol. 7446, 744602, 2009
. . . fast imaging in multiple channels
<< < > >> B F R I Page 38
J. Ohn et al. genesis, vol. 49, no. 7, pp. 514–521, 2011.
Sequentially-acquired (nongated) sequences require synchronization
<< < > >> B F R I Page 39
J. Ohn et al. genesis, vol. 49, no. 7, pp. 514–521, 2011.
Fast imaging in multiple channels
<< < > >> B F R I Page 40
Transgenic zebrafish expressing green fluorescent protein in the heart
ava
v v
av
a
v a
A B C D
ET33-1a ET33-mi3a ET31 GW64C
A myocardium of both the atrium (a) and ventricle (v)
B ventricular myocardium
C atrioventricular canal (av)
D myocardium and endocardium of the ventricle.
K.-L. Poon, ML, et al., “Zebrafish Cardiac Enhancer Trap Lines: New Tools for in vivo Studies of Cardiovascular
Development and Disease,” Dev. Dyn, 239:914–926, 2010. Korzh Lab (a*STAR Singapore)
Problem:
Many structures/cell types can be labeled in di«erent fish, but only a few
simultaneously in single fish!
Transgenic lines reveal di«erent tissue types
<< < > >> B F R I Page 41
High-speed camera
Controller- XY stage- Focus drive- Incubator temperature- Camera trigger- Data storage/process- etc
Incubation chamber
XY stage
Focus drive
Temperature
Objective
sensor
V A
J. Ohn, ML, ISBI’2011, 1549–1552
High-throughput functional imaging
<< < > >> B F R I Page 42
AV
ROI
0.5 1
−0.3
0
0.3
time
(sec)
velocity(mm/s)
D
S
θ
28°C
fish #1
fish #2
#10
29°C
38.1°C
0
1
2
time
(sec)
temperature(°C)
# of samples(n)
velocity
J.
Oh
n,M
L,
ISB
I’2
01
1,
15
49
–1
55
2
Functional signature depends on sample,temperature
<< < > >> B F R I Page 43
−0.3
−0.15
0
0.15
0.3
Velocity(m
m/s)
28°C
0 100 200 Time(ms)
29°C
0 100 200 Time(ms)
30.2°C
0 100 200 Time(ms)
32°C
0 100 200 Time(ms)
33.4°C
0 100 200 Time(ms) 0 100 200 Time(ms)
34.5°C
35.2°C
0 100 200 Time(ms)
36.2°C
0 100 200 Time(ms)
37.3°C
0 100 200 Time(ms)
−0.3
−0.15
0
0.15
0.3
Velocity(m
m/s)
−0.3
−0.15
0
0.15
0.3
Velocity(m
m/s)
J.
Oh
n,M
L,
ISB
I’2
01
1,
15
49
–1
55
2
Increase in heart rate with temperature (28◦C to 38.1
◦C (n = 10)): 47± 5%
Intra-population variability: 18%–30% (velocity amplitude), 4%–8% (heartbeat length).
Functional signature conserved within population
<< < > >> B F R I Page 44
1. Acquire multi-dimensional data
(3D+Time+λ+. . . ) at high frame-rate
2. Merge and segment multi-modal data
3. Joint imaging and untangling
of morphogenesis and function
Steps towards integrated imaging of cardiogenesis
<< < > >> B F R I Page 45
ML, A.S. Forouhar, et al., Dev. Dyn, 2006 | A. Forouhar, ML et al., Science, 2006.
Fast 3D Imaging Reveals Functional Changes
<< < > >> B F R I Page 46
Required Spatial
Resolution ∆x (µm)
Velocity of
of Structure
(µm/s)
0.001
0.001
0.01
0.01
0.1
0.1
1
1
10
10
100
100
1000
1000
1 fp
s
1 fp
10s
1 fp
100
s
1 fp
15
minut
es
1 fp
3 h
ours
1 fp
day
10 fp
s
100
fps
1000
fps
10,0
00 fp
s
100,
000
fps
Cell Migration
Embryonic
Heart
Calcium
Sparks
Cell
Traffick
Microtubule
Growth
Chromosome
Dynamics
J. Vermot, S.E. Fraser, ML, HFSP Journal (2008), http://hfspj.aip.org/doi/10.2976/1.2907579.
Framerate Requirements Depend on Target Resolution, Structure Velocity
<< < > >> B F R I Page 47
Time point 1
90 frames(30 fps)
3 seconds3 seconds
∆z: 5µm
Nz: 6 steps
90 frames(30 fps)
3 seconds3 seconds
∆z: 5µm
Nz: 6 steps
Time point 2 Time point 3
~~
Before synchronization
After synchronization
Time development
2 minutes 2 minutes
Time point 1
A
B
C
D
zy
x
t fast
t slow
t fast t fast
z
t fast
z
t slow
^ t fast^
Time point 2
zy
x
t fast
J.O
hn,H
.-J.T
sai,
ML
,O
rganogenesis
,5(4
),2009.
Capturing morphogenesis & function: combine high-speed with timelapse
<< < > >> B F R I Page 48
A
C
D
B
av
82:52:00
82:54:00
82:56:00
82:58:00J. Ohn, H.-J. Tsai, ML, Organogenesis, 5(4), 2009.
see movie: http://dx.doi.org/10.4161/org.5.4.10568
<< < > >> B F R I Page 49
24 36 48Development Time (Hours)
Heartbeat Time (ms)
0
0
300
600
Atrium
Ventricle
J.O
hn,H
.-J.T
sai,
ML
,O
rganogenesis
,5(4
),2009.
Heart morphogenesis & function—untangled at last
<< < > >> B F R I Page 50
Heart
Morphogenesis Heart Function
Cellular Regulation
Time-lapse microscopy
3D microscopyHigh-speed imaging
Fluorescence imaging
Multi-cycle
image denoising
microscopy Joint
time-lapse/function
imaging
Multi-modal
imaging,merging,
and segmentation
Summary: integrated imaging system allows co-localized measurements in
multi-dimensions and scales while preserving normal heart function and
morphogenesis
Summary
<< < > >> B F R I Page 51
Heart
Morphogenesis Heart Function
Cellular Regulation
Cell fate
Tissue dynamics
Organ morphometry
Cell shape
Cell behavior (division, differentiation, migration, death)
Gene expression patterns
Fluid forces map
Cardiac function
(pumping efficiency,
temporal & spatial
flow patterns)
Multi-cycle
image denoising
Cardiac function
(pumping efficiency, (pumping efficiency,
temporal & spatial temporal & spatial temporal & spatial
Joint
time-lapse/function
imaging
Multi-modal
imaging,merging,
and segmentation
Summary: integrated imaging system allows co-localized measurements in
multi-dimensions and scales while preserving normal heart function and
morphogenesis
Summary
<< < > >> B F R I Page 51
Contact: liebling@ece.ucsb.edu, http://sybil.ece.ucsb.eduUCSB
Sandeep Bhat
JungHo Ohn
Cooper Thomas
Apolonio Arias
Kyle Stewart
Chieh Lee
JeaBuem You
Hiranmayi Ranganathan
Dmitry Fedorov
B.S. Manjunath
Caltech
Arian S. Forouhar
Julien Vermot (@IGBMC)
Le Trinh
Mory Gharib
Scott E. Fraser
Caltech (Quail)
Jen Yang
Rusty Lansford
BCM Houston (Mouse)
Irina Larina
Mary E. Dickinson
U. Houston (OCT)
Kirill Larin
a*star, Singapore
(Heart transgenics)
Kar Lai Poon, Vlad Korzh
Hard- and Software
Imaris (Bitplane AG)
µManager (Nico Stuurman)
Leica Microsystems
Olympus
Carl Zeiss, AIM
Zebrafish
Shuo Lin, UCLA
H.-J. Tsai, Nat. Taiwan U.
Le Trinh, Caltech
ZIRC
Funding
NIH (NHLBI R01 HL078694)
NSF (DMR 0960331)
Hellman Family Fellowship
Santa Barbara Cottage Hospital
Regent’s Junior Faculty Fellowship
Acknowledgments
<< < > >> B F R I Page 52
High-dimensional....................... 2
Human heart.............................. 3
Zebrafish embryos..................... 4
EC and flow ............................... 5
Fraction Retrograde................... 7
klf2a in situ................................. 10
Flow and klf2a model................. 11
Morpho, function, cell resp. ....... 12
Integrated imaging..................... 13
Speed, resolution, SNR............. 15
Poisson Meets Airy.................... 16
Shinkansen blur......................... 17
Velocity vs Resolution Plot ........ 18
OCT Denoising.......................... 20
Gated imaging animation........... 22
cmlc2 raw/regist......................... 23
48hpf heart and hair .................. 24
4D OCT ..................................... 25
Visual Problem explanation ....... 26
What if irregular? ....................... 27
Nonuniform reg. eqs.................. 30
Uniform vs nonuniform .............. 32
Cardiac Mosaicing..................... 33
Brightfield vs fluorescence ........ 35
Microscope Setup...................... 36
Multi-Modal Registration............ 38
ET Transgenic Zebrafish............ 41
Development gata1.................... 46
Velocity vs Resolution (2) .......... 47
Zebrafish cell division ................ 49
Fast and slow cartoon ............... 50
— — — — — — — — — — — 53
Index
<< < > >> B F R I Page 53