Introduction to multispectral imaging · High Content 2016 September 12th-14th 3rd Annual...
Transcript of Introduction to multispectral imaging · High Content 2016 September 12th-14th 3rd Annual...
High Content 2016 September 12th-14th 3rd Annual Conference
Joseph B. Martin Conference Center at Harvard Medical School, Boston, MA
Introduction to multispectral imaging
Bartek Rajwa, PhD
Bindley Bioscience Center Purdue University
Outline •
– – –
• •
– – –
• – –
What is (multi)spectral imaging?
•
–
– –
•
–
– •
Origin of spectral imaging: LANDSAT system
• •
•
•
Remote sensing
Hi-res imaging vs. spectral imaging
COLLECTING THE DATA
Spectral imaging in microscopy - web resources
•
•
•
Collecting a lambda stack
Spectral imaging + microscopy
Spectral imaging – various approaches
Spectral microscopy hardware
•
•
Nikon spectral imaging system
Other multispectral arrangements in microscopy
PROCESSING THE DATA
210 bands unmixed into 7 endmembers
Multispectral small animal imaging
Problem: spectral overlap
TO-PRO-3MitoSOX RedCalcium GreenVybrant DyeCycleViolet stainMonobromobimane(mBBr)
•
•
Let’s revisit
Spectral unmixing overview
Mixing and unmixing
We have two detectors, but some cross-talk happens
Mixing , sorting and sieving…
a = 100
50⎡
⎣⎢
⎤
⎦⎥;M = 0.8 0.3
0.2 0.7⎡
⎣⎢
⎤
⎦⎥
Matrix notation
⎡ ⎤= ⎢ ⎥⎣ ⎦
0.8 0.30.2 0.7
M
What is the definition of “true” signal?
r1 = pa1 + (1− q)a2
r2 = (1− p)a1 + qa2
⎧⎨⎪
⎩⎪
r1r2
⎡
⎣⎢⎢
⎤
⎦⎥⎥=
p 1− q1− p q
⎡
⎣⎢⎢
⎤
⎦⎥⎥
a1
a2
⎡
⎣⎢⎢
⎤
⎦⎥⎥
′ ′= − = +⎧ ⎧⇒⎨ ⎨′ ′= − = +⎩ ⎩
′⎡ ⎤ ⎡ ⎤⎡ ⎤=⎢ ⎥ ⎢ ⎥⎢ ⎥′⎣ ⎦⎣ ⎦ ⎣ ⎦
1 1 2 1 1 2
2 2 1 2 1 2
1 1
2 2
11
s r q s r s q ss r p s r p s s
r sqr sp
The a1 and a2 (abundances) defined for unmixing are different than s1 and s2 (compensated signals) defined for compensation!
a1s1
Just a digression…
−−
⎛
⎝⎜
⎞
⎠⎟⎛
⎝⎜⎜
⎞
⎠⎟⎟=
′′
⎛
⎝⎜
⎞
⎠⎟⎛
⎝⎜⎜
⎞
⎠⎟⎟
−−
⎛
⎝⎜
⎞
⎠⎟⎛
⎝⎜⎜
⎞
⎠⎟⎟=
−
−
⎛
⎝
⎜⎜⎜⎜
⎞
⎠
⎟⎟⎟⎟
⎛
⎝⎜⎜
⎞
⎠⎟⎟
= =
= ÷ = ⋅
Unmixing requires inversion of the mixing matrix
MM−1 = 1= 1 0
0 1⎡
⎣⎢
⎤
⎦⎥
⎡ ⎤ ⎡ ⎤• =⎢ ⎥ ⎢ ⎥
⎣ ⎦ ⎣ ⎦
0.8 0.3 1 00.2 0.7 0 1
?
−⎡ ⎤ ⎡ ⎤ ⎡ ⎤• =⎢ ⎥ ⎢ ⎥ ⎢ ⎥−⎣ ⎦ ⎣ ⎦ ⎣ ⎦
0.8 0.3 1.4 0.6 1 00.2 0.7 0.4 1.6 0 1
Matrix inversion – simple example
a bc d
⎡
⎣⎢
⎤
⎦⎥
−1
= 1ad − bc
d −b−c a
⎡
⎣⎢
⎤
⎦⎥
4 72 6
⎡
⎣⎢
⎤
⎦⎥
−1
= 14 ⋅6 − 7 ⋅2
6 −7−2 4
⎡
⎣⎢
⎤
⎦⎥
= 110
6 −7−2 4
⎡
⎣⎢
⎤
⎦⎥
= 0.6 −0.7−0.2 0.4
⎡
⎣⎢
⎤
⎦⎥
Matrix inversion – cont.
4 72 6
⎡
⎣⎢
⎤
⎦⎥
0.6 −0.7−0.2 0.4
⎡
⎣⎢
⎤
⎦⎥ =
4 ⋅0.6 + 7 ⋅(−0.2) 4 ⋅(−0.7)+ 7 ⋅0.42 ⋅0.6 + 6 ⋅(−0.2) 2 ⋅(−0.7)+ 6 ⋅0.4
⎡
⎣⎢⎢
⎤
⎦⎥⎥
= 2.4 −1.4 −2.8 + 2.8
1.2−1.2 −1.4 + 2.4⎡
⎣⎢
⎤
⎦⎥ =
1 00 1
⎡
⎣⎢
⎤
⎦⎥
Sieved (yet, still mixed) result
•
•
Unmixing (or how to fix a bad sieve)
r = [95,55]; M = 0.8 0.3
0.2 0.7⎡
⎣⎢
⎤
⎦⎥; S = 1 0.4286
0.25 1⎡
⎣⎢
⎤
⎦⎥
aU = M-1r ⇒ aU = 1.4 −0.6−0.4 1.6
⎡
⎣⎢
⎤
⎦⎥ ⋅
9555
⎡
⎣⎢
⎤
⎦⎥ =
10050
⎡
⎣⎢
⎤
⎦⎥
sC = S-1r ⇒ sC = 1.12 −0.28−0.48 1.12
⎡
⎣⎢
⎤
⎦⎥ ⋅
9555
⎡
⎣⎢
⎤
⎦⎥ =
8035
⎡
⎣⎢
⎤
⎦⎥
Example: from n channels to two 2 colors
• •
• •
Generalization: linear mixing of fluorescence signals
r = Ma +n
•
•
OMG!More math!!
Mixing of three signals (overdetermined case)
M =
0.03267974 0.010416670.13071895 0.031250000.32679739 0.072916670.26143791 0.104166670.20915033 0.468750000.03921569 0.31250000
⎡
⎣
⎢⎢⎢⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥⎥⎥⎥
, a = 100 50⎡⎣ ⎤⎦
Overdetermined (multispectral) case
•
•
•
Overdetermined case
aLS = MTM( )−1
MTr
aLS = M−1r
However, if the constraints are imposed…
•
•
( ) ( ){ } { }1min s.t. = 1pT
jJαα α
=∈Δ− − Δ =∑r Mα r Mα
( ) ( )
( ) ( )
11 1
11 1
ˆ ˆ ,whereT T TSCLS LS
T T TL L
P
P
−− −⊥
−− −⊥×
⎡ ⎤= + ⎢ ⎥⎣ ⎦
⎡ ⎤= + ⎢ ⎥⎣ ⎦
α α M M 1 M M 1
I M M 1 M M 1
•
•
( ) 1ˆ ˆ
ˆ( )
TNCLS LS
TNCLS
λ
λ
−⎧ = −⎪⎨⎪ = −⎩
α α M M
M r Mα
( ) ( ){ } { }min s.t. = 0Tjα
α α∈Δ
− − Δ ≥r Mα r Mα
EXPLORATORY ANALYSIS
Principal component analysis
•
•
•
– –
Principal component analysis – textbook explanation
• •
=
=−1
1nT
Y
Y PX
C YY
•
•
−= 1A QLQ
−1
1T
nXX
So, let’s do that (Fisher’s data set)
• •
4.5 5.5 6.5 7.5
4.5
5.5
6.5
7.5
Sepal.Length
2.0
2.5
3.0
3.5
4.0
Sepal.Width
12
34
56
7
Petal.Length
4.5 5.5 6.5 7.5
0.5
1.0
1.5
2.0
2.5
2.0 2.5 3.0 3.5 4.0 1 2 3 4 5 6 7 0.5 1.0 1.5 2.0 2.5
0.5
1.0
1.5
2.0
2.5
Petal.Width
Cx= XXT = QLQT
-0.10 0.00 0.05 0.10 0.15
-0.1
00.
000.
050.
100.
15
var 1
-0.2
-0.1
0.0
0.1
0.2
var 2
-0.10 0.00 0.05 0.10 0.15
-0.2
-0.1
0.0
0.1
0.2
-0.2 -0.1 0.0 0.1 0.2 -0.2 -0.1 0.0 0.1 0.2
-0.2
-0.1
0.0
0.1
0.2
var 3
Another approach to PCA: singular value decomposition
•
•
This could be used to compress images!
SVD and PCA
•
•
• = TX USV
( )( ) ( )( )= =
= = =
= 2
and
,
T T T
TT T T T T
T T
T
X USV XX QLQ
XX USV USV USV VSU V V 1
XX US U
To sum up…
•
•
•
•
-3 -2 -1 0 1 2 3 4
-3-2
-10
12
34
var 1
-1.0
-0.5
0.0
0.5
1.0
var 2
-3 -2 -1 0 1 2 3 4
-0.5
0.0
0.5
-1.0 -0.5 0.0 0.5 1.0 -0.5 0.0 0.5
-0.5
0.0
0.5
var 3
PCA and kernel PCA of lambda stack
-15 -10 -5 0 5 10 15
-15
-10
-50
510
151st Principal Component
2nd
Prin
cipa
l Com
pone
nt
• We can compute PCA on spectral data • Pure endmemebers form a simplex! • Theory of convex sets • Simplest endmember estimation requires
only PCA (or kPCA)
label 1
0
50
100
150
200
0 50 100 150 200
0 50 100 150 200
label 2
0
50
100
150
200
0
50
100
150
200
0 50 100 150 200
label 3
( )1SAM( , ) cosi j i j i js s −= ⋅ ⋅s s s s
PCA performed on a lambda stack
PCA performed on reflected light lambda stack
Example: Raman spectorscopy
•
•
Non-negative matrix factorization (NMF)
2( , )F
F = −W H X WH
Application example: autofluorescence removal
Woolfe, F. et al.., 2011. Autofluorescence Removal by Non-Negative Matrix Factorization. IEEE Transactions on Image Processing 20, 1085–1093. doi:10.1109/TIP.2010.2079810
NMF in histopathology
SCREEN QUALITY ASSESSMENT
Screen quality assessment in multiplexed HCS
•
•
–
–
•
–
…but first – what is Cohen’s d ?
•
δ =
μpos − μneg
σ
d =
Ypos −Yneg
′s, ′s =
npos −1( )spos2 + nneg −1( )sneg
2
npos + nneg − 2
Cohen’s measures distance between normally distributed populations
Z’-factor and Cohen’s d
•
•
′Z = 1−
3 σ neg +σ pos( )μpos − μneg
, ′Z = 1−3 sneg + spos( )
Ypos −Yneg
′Z = 1− 6
d
Cohen’s d and Z’ in multiple dimensions
•
–
–
– –
•
Why 2-D is better than 1-D?
Mahalanobis distance
D2 = dT R−1d = d1,d2 ,…,dm⎡⎣ ⎤⎦
1 r1,2 r1,m
r1,2 1
r1,m 1
⎡
⎣
⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥
−1
d1
d2
dm
⎡
⎣
⎢⎢⎢⎢⎢
⎤
⎦
⎥⎥⎥⎥⎥
.
D = dT R−1d•
• •
D = T 2 nposnneg
npos + nneg
Confidence interval for D
fD =
nposnneg npos + nneg − p −1( )p npos + nneg( ) npos + nneg − 2( ) D2
fD ~ Fp,m+n− p−1 λ( )
So, how can we use this information?
prob Fp,m+n− p−1 λL( )( ) = 1− α2
prob Fp,m+n− p−1 λU( )( ) = 1−α
Conclusions
•
•
•
•
•
Thank you!