Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every...
Transcript of Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every...
![Page 1: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/1.jpg)
{{ }}{{ Choosing and using scientific cameras }}{{ }}KEITH BENNETT, PH.D. | HAMAMATSU PHOTONICS K.K. | FOCUS ON MICROSCOPY| 04.13.2014
![Page 2: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/2.jpg)
Choosing and Using Scientific CamerasChoosing and Using Scientific Cameras
1 Th i bl{1 The image problem{{ Think in photons2{{3 Real cameras are not perfect{4{ Know thyself
5{
{ The Living Image: Case Studies5{ g g
![Page 3: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/3.jpg)
Choosing and Using Scientific CamerasChoosing and Using Scientific Cameras
1{1 The image problem{{ Think in photons2{{3 Real cameras are not perfect{4{ Know thyself
5{
{ The Living Image: Case Studies5{ g g
![Page 4: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/4.jpg)
The Image Problem…g
Courtesy: Prof. Jason SwedlowUniversity of Dundee, ScotlandO Mi E i t
4
Open Microscopy Environment
![Page 5: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/5.jpg)
The Image Problem…g
A pretty picture?A pretty picture?
A measurement?
A resource?
A reference?A reference?
Courtesy: Prof. Jason SwedlowUniversity of Dundee, ScotlandO Mi E i t
5
Open Microscopy Environment
![Page 6: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/6.jpg)
THE IMAGE PROBLEM1{ } THE IMAGE PROBLEM1{ }• Eyes can be fooled
- Not good at quantifying greysbj i- Not objective
- Emphasizes patterns and colors- Viewing environmentLOOK Viewing environment
• Screens are not capable of CAREFULLY
pdisplaying full bit depth
• Image display can (and should be!) manipulated for on screen viewing p g
![Page 7: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/7.jpg)
THREE IDENTICAL IMAGES?1{ }1{ }AA
B
C
![Page 8: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/8.jpg)
THREE IDENTICALLY DISPLAYED IMAGES!1{ }1{ }AA
B
200 photons200 photons
C
200 photons200 photons
500 photons500 photons
1000 photons1000 photons
![Page 9: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/9.jpg)
THREE DIFFERENT INTENSITIES?1{ }1{ }
![Page 10: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/10.jpg)
THREE DIFFERENT DISPLAYS OF THE SAME INTENSITY!1{ }THREE DIFFERENT DISPLAYS OF THE SAME INTENSITY!1{ }
1000 photons1000 photons1000 photons1000 photons
1000 photons1000 photons1000 photons1000 photons
1000 h t1000 h t1000 photons1000 photons
![Page 11: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/11.jpg)
HISTOGRAM AND AREA STATISTICS1{ } HISTOGRAM AND AREA STATISTICS1{ }A B C
Peak (photons) 200 500 1000(p )Mean (photons)
20047.0 117.4 234.7
![Page 12: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/12.jpg)
Choosing and Using Scientific CamerasChoosing and Using Scientific Cameras
1{1 The image problem{{ Think in photons2{{3 Real cameras are not perfect{4{ Know thyself
5{
{ The Living Image: Case Studies5{ g g
![Page 13: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/13.jpg)
THINKING IN PHOTONS2{ } THINKING IN PHOTONS2{ }IIS
THINKING IN• Aren’t ADU’s or grey levels good
enough?PHOTONSREALLY gREALLY
NECESSARY?
SCIENTIFIC CAMERAS SHOULD MEASURE PHOTONS
![Page 14: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/14.jpg)
PHOTONS REALLY MATTER2{ } PHOTONS REALLY MATTER2{ }
Truth
100 photons peakLooks similar butLooks similar, but‐ The histogram is different‐ Information is different
ifi i diff
Perfect camera
‐ Quantification different‐ Lower image contrastPerfect camera Background = Peak/10
![Page 15: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/15.jpg)
REMEMBER SHOT NOISE2{ } REMEMBER SHOT NOISE2{ }
![Page 16: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/16.jpg)
THINKING IN PHOTONS2{ } THINKING IN PHOTONS2{ }h i f i i i i li i d• The information in an image is limited by the number of photons.WHAT’S
• A perfect camera does not produce a perfect image, especially if photons
LIMITINGMY
are limited.
Th i i b f h
MYSCIENCE?
• The minimum number of photons needed depends upon the object imaged resolution and measurementimaged, resolution and measurement requirements (i.e. your experiment).
![Page 17: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/17.jpg)
PHOTONS REALLY MATTER2{ } PHOTONS REALLY MATTER2{ }
ARE YOUCONVINCED?CONVINCED?
![Page 18: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/18.jpg)
Choosing and Using Scientific CamerasChoosing and Using Scientific Cameras
1 h bl{1 The image problem
2{{ Think in photons2{3 Real cameras are not perfect{4{ Know thyself
5{
{ The Living Image: Case Studies5{ g g
![Page 19: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/19.jpg)
{ }3 REAL CAMERAS: THINKING IN PHOTONS{ }3 REAL CAMERAS: THINKING IN PHOTONS
• Makes comparisons among camerasMakes comparisons among cameras meaningful. (ADUs are arbitrary)HOW
DOES THIS • Brings relevance to your data.
• Kno ing the n mber of photons
DOES THISMAKE ME
• Knowing the number of photons and contrast in sample is key to picking the correct camera.
A BETTERMICROSCOPIST? picking the correct camera.
![Page 20: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/20.jpg)
REAL CAMERAS3{ } REAL CAMERAS3{ }I
• Can’t we figure everything out from a camera specs (QE and electronic
ISTHINKING IN camera specs (QE and electronic
specs)?PHOTONSREALLY
[Hint: Maybe, but there’s a better way]REALLY
NECESSARY?
SCIENTIFIC CAMERAS SHOULD MEASURE PHOTONS
![Page 21: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/21.jpg)
3 REAL CAMERAS ARE NOT PERFECT{
{
3 REAL CAMERAS ARE NOT PERFECT{
{
• The GapTHE• Electron multiplying CCDs (EMCCDs)• Simulations comparing perfect to product by spec• All pixels are not created equal
THEWHATAND All pixels are not created equal
• Actual product measurements• Camera noise & visualization
ANDHOW
![Page 22: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/22.jpg)
Why is aWhy is a camera manufacturer
proclaiminghthat
cameras are not perfect?cameras are not perfect?
![Page 23: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/23.jpg)
Because NO camera is perfect&
B d t di hBecause understanding why matters to your sciencematters to your science
![Page 24: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/24.jpg)
WHAT IS THE GAP?3{ } WHAT IS THE GAP?3{ }The difference between the performance of an actual camera and a theoretically perfect camera
{Perfect Camera
The GAP {Actual Camera
![Page 25: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/25.jpg)
UNDERSTANDINGWHY THERE IS A GAP ENABLES:3{ }UNDERSTANDINGWHY THERE IS A GAP ENABLES:3{ }
• Appropriate camera selectionAppropriate camera selection
• Optimized camera usage Better• Optimized experimental design
BetterResults
• More reliable data analysis
![Page 26: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/26.jpg)
THE GAP DEPENDS ON:3{ } THE GAP DEPENDS ON:
{1 S h lCCD
3{ }
{1. Sensor technology EMCCDsCMOS
{2 CQuantum EfficiencyCamera Noise{2. Camera specs Camera Noise
• Read noise• Excess noise• Photo response non uniformity (PRNU)
{• Photo‐response non‐uniformity (PRNU)
Ultra low lighth{3. Input photon level Low Light
IntermediateHighHigh
![Page 27: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/27.jpg)
THE (HYPOTHETICAL) PERFECT CAMERA3{ } THE (HYPOTHETICAL) PERFECT CAMERA
Every photon is converted into one electron100% QE {3{ }
Every photon is converted into one electron100% QE
0 e‐read noise Every electron is digitized exactly as expected every time
{{read noise
0% fixed tt i
y g y p y
Every pixel and amplifier perform identically and predictably
{{
Perfect Camera Signal to Noise Ratio
pattern noise y p p p y p y{
10
100
e R
atio
(SN
R)
In a perfect camera, the SNR of a single pixel is limited only by the physics of photon statistics
1
0.1 1 10 100 1000 10000 100000Si
gnal
to N
oisby the physics of photon statistics… i.e. shot noise.
0Signal (Photons)
![Page 28: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/28.jpg)
REAL CAMERAS ARE NOT PERFECT{ }3 REAL CAMERAS ARE NOT PERFECT{ }3
I EM X2 ORCA Fl h4 0 V2 ORCA R2ImagEM X2 EMCCD: Electron Multiplying CCD
ORCA‐Flash4.0 V2Scientific CMOS Camera
ORCA‐R2Cooled Interline CCD
![Page 29: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/29.jpg)
3 BASIC SPECS: COMPARED{ }3 BASIC SPECS: COMPAREDCCD EMCCD CMOS
{ } { { {Camera Name ORCA‐R2 ImagEM x2 ORCA Flash4 0 V2Camera Name ORCA R2 ImagEM x2 ORCA Flash4.0 V2
QE (550 nm) 70 % 90 % 72 % Read Noise SingleRead Noise Single Frame rms (e‐) 6 < 0.5 (M = 200) 1.5
Full Well Capacity (e‐) 18,000 Gain dependent 30,000, p ,
Dynamic Range 3000:1 Gain dependent 20,000:1
Bit Depth 16 16 16p 16 16 16
Max pixel rate (Mps) 13 18 420
Pixel Size (m) 6.45 x 6.45 16 x 16 6.5 x 6.5Pixel Size (m) 6.45 x 6.45 16 x 16 6.5 x 6.5
Pixel Number 1024 x 1344 512 x 512 2048 x 2048
![Page 30: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/30.jpg)
AMPLIFIERS3{ } AMPLIFIERS3{ }
CCD and sCMOSImportant EMCCD
pdifferences
![Page 31: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/31.jpg)
ELECTRON MULTIPLYING CCDS (EMCCDS)3{ }A t f CCD F t f d
ELECTRON MULTIPLYING CCDS (EMCCDS)3{ }• A type of CCD: Frame transfer and back‐thinned for increased QE
• Frame transfer requires ~ 100s
• Serial devices where each pixel’s chargeSerial devices where each pixel s charge is read out one at a time
• High voltage gain register on sensor for• High voltage gain register on sensor for on‐chip amplification.
O ti t d t th h EM i it• Option to read out through EM circuitry or non‐EM circuit (normal CCD mode)
{{EMCCD architecture}}{{EMCCD architecture}}
![Page 32: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/32.jpg)
CMOS AND CCD AMPLIFIER NOISE3{ } CMOS AND CCD AMPLIFIER NOISE3{ }Output an exact multiple of the input
No noise broadening
Output is a multiple of the input
“Read noise” broadening
Width independent of signal p glevel
CMOS d i 1 5CMOS read noise: 1.5 e‐ rms
![Page 33: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/33.jpg)
EMCCD AMPLIFIER NOISE DEPENDS ON SIGNAL3{ }EMCCD AMPLIFIER NOISE DEPENDS ON SIGNAL3{ }No electron:No electron:‐ Very small noise‐ beautiful blacks
Signal:‐ Broad (excess noise)‐ Long tail: larger apparent
contrast
Signal independent‐ No excess noise‐ Short tail
![Page 34: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/34.jpg)
EMCCDS “DETECT” SINGLE PHOTONS BUT3{ } EMCCDS DETECT SINGLE PHOTONS, BUT3{ }
Peak of 1e‐ output is ~0.4e‐!0.4 e‐ (!!)
Long tail
Signal < (some) noise
S t i di t ib ti ithSymmetric distribution, with noise extending ~+2 (3 e‐) from mean.
Significant overlap
Quantization of ADC not included
![Page 35: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/35.jpg)
EMCCDS CAN’T COUNT3{ } EMCCDS CAN T COUNT3{ }
Outputs from 1e‐ and 2e‐loverlap.
Peak output of 2e‐ input is ~ 1e‐
CMOS not so good either at very low light
2e‐ input, CMOS tail is shorter than EMCCD
![Page 36: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/36.jpg)
EMCCD: SIGNAL DEPENDENT NOISE3{ } EMCCD: SIGNAL DEPENDENT NOISE3{ }Most probable output < mean.
Very long tail
2 = signal
Lots of overlap: 10e‐ & 20e‐Lots of overlap: 10e & 20e
Most probable output = mean
Short tail
2 = 1.5 e‐
CMOS clearly better
![Page 37: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/37.jpg)
EMCCD OUTPUT INCLUDING PHOTON SHOT NOISE3{ } EMCCD OUTPUT INCLUDING PHOTON SHOT NOISE
In simulated probability distribution functions for EMCCD, the output at
3{ }p y p
high gain is not Poisson due to the electron multiplication process!
2 Photon Average Input 10 Photon Average InputGain = 200 Gain = 200
0 0005
0.0006
0.0007
0.0008
[a.u.]
0 00015
0.0002
0.00025
[a.u.] Long tailLong tail
0.0002
0.0003
0.0004
0.0005
roba
bility [
0.00005
0.0001
0.00015
Prob
ability
0
0.0001
‐5 0 5 10
Pr
0
0 5 10 15 20 25 30
P
Photon equivalentPhoton equivalent Photon equivalent
![Page 38: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/38.jpg)
EMCCD VS CMOS AMPLIFIERS3{ } EMCCD VS. CMOS AMPLIFIERS
S h i lifi i
3{ }• Stochastic EM amplification:
– Very low noise without input– Excess noise effectively doubles photoelectron shot noise (Fn2 = 2)A i di ib i– Asymmetric output distribution
• At low light, peak output is much below mean• Long tail• Long tail
• CMOS N i i ith l i t– Noisier with no or very low input
– Noise independent of signal
![Page 39: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/39.jpg)
ELECTRON MULTIPLYING CCDS3{ } ELECTRON MULTIPLYING CCDS3{ }
Are theyAre they really what
you thought?
![Page 40: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/40.jpg)
SIMPLE (PIXEL) SNR EQUATION3{ } SIMPLE (PIXEL) SNR EQUATION3{ }
Terms included: Not included:
QE: Quantum EfficiencyS: Input Signal Photon Number (photon/pixel)Fn: Noise Factor
Dark Noise: Dark current X time; considered negligible
Fn: Noise Factor (= 1 for CCD/sCMOS and √2 for EM‐CCD)
Nr: Readout NoiseM: EM Gain (=1 for CCD / CMOS)
Photo response non uniformity: necessary for image SNR
M: EM Gain (=1 for CCD / CMOS)Ib: Background
![Page 41: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/41.jpg)
RELATIVE SNR: DISPLAYS IMPERFECTIONS PERFECTLY3{ }RELATIVE SNR: DISPLAYS IMPERFECTIONS PERFECTLY3{ }
0.8
0.9
1
) rSNR is the SNR for a
0.5
0.6
0.7
SNR
(rSN
R)
Perfect
QE 70%
rSNR is the SNR for a camera plotted relative to the perfect camera
0.2
0.3
0.4
Rel
ativ
e S
QE 50% rSNR shows differences among cameras over full range of signal level
0
0.1
0.1 1 10 100 1000 10000
Signal (Photons)
g g
Signal (Photons)
![Page 42: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/42.jpg)
READ NOISE REDUCES RSNR ONLY AT LOW LIGHT3{ }READ NOISE REDUCES RSNR ONLY AT LOW LIGHT3{ }
0 9
1.0
70% QE LimitShot NoiseLimited
Read NoiseLimited
50% QE Limit
• Nr(ph): Read noise in photonsis the key low light parameter0.6
0.7
0.8
0.9
R (rSN
R)
Nr(ph) = Nr(e‐)/QE
• QE: always important0.3
0.4
0.5
elative SNR
QE 70%, Nr(ph) = 3
QE 50%, Nr(ph) = 3
0.0
0.1
0.2Re
Same Nr(ph) Nr (e‐) = 2.1 e‐Nr (ph) = 3
0.1 1 10 100 1000 10000
Signal (photons)Nr (e‐) = 1.5 e‐Nr (ph) = 3
![Page 43: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/43.jpg)
EMCCDS: EXCESS NOISE CREATES A GAP3{ }EMCCDS: EXCESS NOISE CREATES A GAP
SNR for CCD / CMOS SNR for EM‐CCD
3{ }SNR for CCD / CMOS SNR for EM CCD
PQEPQESNR
F
PQEPQEMF
PQEMSNRn
2n
PQE
PQE
PQE
FPQEMF
eff
n
n
QE: Quantum Efficiency, 22
QEFQEQE
neff
yP: Input Signal Photon Number, M: EM Gain Fn: Noise Factor (assumes dark current and read noise are negligible)
2Fn
(assumes dark current and read noise are negligible)
![Page 44: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/44.jpg)
EMCCDs3{ } EMCCDs3{ }• Stochastic EM amplification adds excess noise Native QE
• Excess noise effectively lowers the SNR to a detector with½ eQEthe SNR to a detector with ½ the QE
eQE
{{ ff }}{{ Effective QE in EMCCDs }}
![Page 45: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/45.jpg)
MIND THE GAP: PREDICTED PIXEL rSNR PERFORMANCE FOR3{ }COMMON CAMERAS
A camera with the highest The SNR of an EMCCD above 1
{ {
3{ }A camera with the highest SNR at the lowest light level may not be the best at higher light levels
The SNR of an EMCCD above 1 electron/pixel is comparable to a camera with QEeff =QE/2 due to excess noise from EM gain.
1.{ 2. {0.9
1
R)
g g g
0 5
0.6
0.7
0.8
NR
(rSN
R
P f t
1.
2.
0 2
0.3
0.4
0.5
elat
ive
SN Perfect
ORCA-R2 (CCD)
ImagEM X2 (EMCCD)
sCMOS Flash4 0 (100 fps)
0
0.1
0.2
0 1 1 10 100 1000 10000
Re sCMOS Flash4.0 (100 fps)
ImagEM X2 (BT CCD mode)
0.1 1 10 100 1000 10000
Signal (Photon, no background) = 650 nm
![Page 46: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/46.jpg)
ARE ALLPIXELS THE
• Offset non‐uniformity• Photo response non‐PIXELS THE
SAME?uniformity (PRNU)
• Dark signal non‐SAME?uniformity (DSNU)
• Read noise distribution
![Page 47: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/47.jpg)
ACCURATE MEASUREMENT OF THE NUMBER OF PHOTONS3{ } OFFSET NON‐UNIFORMITY3{ }Pixel to pixel variation of readings in the dark
+0.6
‐0.20.2
+1.0
If the zero is incorrect, then absolute measurement is also incorrect.• Most noticeable in dark or low light conditions• Most noticeable in dark or low light conditions.• Usually expressed as DN or e‐, rms.• For scientific cameras, should be less than read noise.
![Page 48: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/48.jpg)
ACCURATEMEASUREMENT OF THE NUMBER OF PHOTONS3{ } PHOTO RESPONSE NON‐UNIFORMTIY3{ }PRNU: pixel to pixel variation of the response to light (DN / photon)‐ QE variation : conversion rate of photon to e‐
(may be spectrum dependent)‐ Electronic gain variation: Conversion factor from e‐ to DN
‐15%
+22%
‐6.5%
Mean: 11.9
If the unit length incorrect, then absolute measurement is also incorrect.g ,•Most noticeable in higher light conditions.•May have spatial pattern, stable over time.U ll d % i•Usually expressed as % maximum.
![Page 49: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/49.jpg)
ACCURATE MEASUREMENT OF THE NUMBER OF PHOTONS3{ } TOTAL FIXED PATTERN NOISE3{ }Total pixel‐to‐pixel variation in the accuracy of the measurement of the number of photons Includesphotons. Includes
• Offset non‐uniformity• Photo‐response non‐uniformity
‐15%
+22%
‐6 5%
Overall specification of the non‐uniformity measurement across the image sensor
‐6.5%
Does not include:• Errors in average QE• Temporal noise (excess noise read noise)Temporal noise (excess noise, read noise)• Dark current and dark current shot
![Page 50: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/50.jpg)
DARK SIGNAL NON UNIFORMITY (DSNU)3{ } DARK SIGNAL NON‐UNIFORMITY (DSNU)
550 nm 850 nm
3{ }Pixel‐to‐pixel variation in dark current
Offset : dark signal x exposure time.Noise : (offset in e‐)
• Proportional to exposure time.• Can be >100 e- / sec for a few pixels, especially for sensors > 0C• For a given image sensor, a multiple of average dark current
Mean: 30157s: 369.5 (1.2%)
Mean: 30508s: 432 (1.4%){How big?
• Doubles for each ~8C increase in sensor temperature• Higher noise for high dark current pixels due to dark shot noise.
.
{Which {Whichtechnologies? { • Mainly sCMOS
• Identify high noise pixels and correct in image• Dark shot noise can NOT be corrected.Correction {{
![Page 51: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/51.jpg)
READ NOISE UNIFORMITY: CCD & EMCCD3{ } READ NOISE UNIFORMITY: CCD & EMCCD
CCDs and EMCCDs: All pixels are readout through the same amplifier and digitization
3{ }CCDs and EMCCDs: All pixels are readout through the same amplifier and digitizationcircuits and therefore read noise is very uniform.Median = spatial rms
1000000
10000000Noise Histogram
6 e (rms)0 4 ( )
10000
100000
s (0.1 e‐
bin)
CCD
EMCCD Gain
6 e‐ (rms) temporal ~8 Mpix / sec
0.4 e‐ (rms) temporal)
100
1000
Num
ber o
f pixel EMCCD. Gain
dependent~18 Mpix / sec
1
10
0 2 4 6 8 10 12 14 16 18 200 2 4 6 8 10 12 14 16 18 20
Temporal read noise (e‐, rms)
![Page 52: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/52.jpg)
READ NOISE UNIFORMITY: CMOS3{ } READ NOISE UNIFORMITY: CMOS
CMOS: Each pixel has an independent amplifier and each column has an independent
3{ }CMOS: Each pixel has an independent amplifier and each column has an independentamplifier. Read noise is pixel dependent“Median” < spatial rms.
1,000,0000.85 e‐Median
10,000
100,000
of pixels
1.51 e‐ spatial rms
100
1,000
Num
ber
(10 s/ row) = 420 Mpix/sec
10
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16Temporal read noise (e‐, rms)
![Page 53: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/53.jpg)
READ NOISE3{ } READ NOISE3{ }• CCDs: Uniform, readout speed
dependent, relatively high.
• EMCCDs: Uniform, gain and readout speed dependent, very
Things to p p , y
low with EM gain > ~50, but relatively high in “normal CCD”
keep in i dmode.
• sCMOS: pixel dependent little
mind• sCMOS: pixel dependent, little
dependence on readout speed for a particular camera.p
![Page 54: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/54.jpg)
MEASURINGMEASURINGTHE
R GAn in‐depth look at noise in CCD, EMCCD and CMOSREAL GAPCCD, EMCCD and CMOS cameras
![Page 55: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/55.jpg)
A CLEARER WAY TO COMBINE CAMERA SPECIFICATIONS3{ }A CLEARER WAY TO COMBINE CAMERA SPECIFICATIONS3{ }• Single Frame rSNR Summarizes whole sensor performancep– QEGain– Gain
– Noise: including spatial rms read noise, excess i d k h t inoise, dark shot noise
– Fixed pattern noises, including offset non‐uniformity and PRNU
– Saturation
![Page 56: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/56.jpg)
ORCA R2 INTERLINE CCD: PREDICTABLE AND ROBUST3{ } ORCA‐R2 INTERLINE CCD: PREDICTABLE AND ROBUST
PRNU is insignificant{1 Bright Image:
3{ }PRNU is insignificant{1. shot noise limited
{ Mean intensity: 17,300 e-: 130.5e-
PRNU: not measurable{2.Single frame read noise histogram has a Gaussian distribution
10000 1.0
Read Noise(Nr/QE)
Shot Noise & QE
100
1000
10000
Coun
t
0.6 0.7 0.8 0.9 58% QE Limit
1
10
78 60 43 25 8 10 27 45 62 80
C
0.10.2 0.3 0.4 0.5
‐78 ‐60 ‐43 ‐25 ‐8 10 27 45 62 80
Dark reading (ph)‐
0.1
10 100 1,000 10,000 signal (photon)
=650 nm
![Page 57: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/57.jpg)
EMCCD: SOME SURPRISING RESULTS3{ } EMCCD: SOME SURPRISING RESULTS
Thickness variations from back- 550 nm 850 nm
{
3{ }Thickness variations from backthinning process causes spectrally-dependent PRNU
• Cannot be removed during
{1.• Cannot be removed during
manufacturing• Must be calibrated by users for their
specific spectrum.
Mean: 30157s: 369.5 (1.2%)
Mean: 30508s: 432 (1.4%)
p p• Individual pixel map required for
correction
0 7
0.8
0.9
1Calculated Single Frame rSNR
0 3
0.4
0.5
0.6
0.7
rSNR
Q % ( ff)
The Gap for EMCCD in CCD mode becomes very wide due to PRNU{2.
0
0.1
0.2
0.3
10 100 1000 10000 100000
eQE: 95% (gain off)Nr/eQE = 8.4 photonsPRNU: 1.4%
wide due to PRNU{10 100 1000 10000 100000
Photons
![Page 58: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/58.jpg)
COMPLEX BEHAVIOR: A CLOSER LOOK AT EMCCD SNR WITH3{ } HIGH AND LOW GAIN
Complex Behavior
3{ }Complex Behavior
tura
tion
0 8
0.9
1- Excess noise (eQE)- PRNU- Saturation
Hi h d i
Sa 90% QE Limit
Excess Noise
0.6
0.7
0.8
e SN
R
- High read noise (34 e- @ M=5, 70 fps)
- Gain hard to measure
Excess Noise
0.3
0.4
0.5Re
lativ
e
系列1Gain = 5
0
0.1
0.2
0 01 0 1 1 10 100 1000 10000
系列1系列2Gain 5
Gain = 400
0.01 0.1 1 10 100 1000 10000
Input photon number (photon)
![Page 59: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/59.jpg)
ORCA FLASH4 0 V2 (SCMOS): A VERY COMFORTABLE SWEET SPOT3{ } ORCA‐FLASH4.0 V2 (SCMOS): A VERY COMFORTABLE SWEET SPOT3{ }The “Sweet Spot”
1.0
Shot Noise & QE
70% QE Limit
0 60.70.80.9 70% QE Limit
0.30.40.50.6
rSNR
0.00.10.2
1 10 100 1,000 10,000Signal (Photons @650 nm)
![Page 60: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/60.jpg)
THE IMAGE SENSOR IS NOT THE CAMERA: PRNU IS SIGNIFICANT IN3{ } “SCIENTIFIC” CMOS IMAGE SENSORS
Raw Data Corrected Data
3{ } {
Bright Image
{
Bright Image
PRNU ~2% PRNU ~0 5%
Si l lifi d d di i i d i l ll l
PRNU: ~2% PRNU: ~0.5%
Signal amplified and digitized in column‐parallel ADC.FPGA provides offset and gain correction to the raw digitized signal.
![Page 61: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/61.jpg)
SCMOS: PIXEL DEPENDENT READ NOISE3{ } SCMOS: PIXEL‐DEPENDENT READ NOISE1,000,000
0.85 e‐Median Single Frame Read
3{ }
Rms read noise matches single frame rSNR.
10,000
100,000
er of p
ixels
1.51 e‐ rmsNoise (measured)
g
100
1,000
Num
be
(10 s/ row) = 100 full fps10
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16Temporal read noise (e‐, rms)
( / ) p
Single Frame Dark Histogram
Does not fit a Gaussian 100
1000
10000
r of p
ixels
distribution, i.e. is not completely modeled by a single “read noise.”1
10
‐19.7
‐17.8
‐15.8
‐13.9
‐11.9
‐10.0
‐8.0
‐6.1
‐4.1
‐2.2
‐0.2 1.7
3.7
5.6
7.6
9.5
11.5
13.4
15.4
17.3
19.3
Num
ber
photon equivalent @ 650 nm
![Page 62: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/62.jpg)
SCMOS: IMPROVING VISUAL IMAGE QUALITY3{ } “NOISY” PIXEL FILTERINGMap high noise pixels and
3{ }T
Correction OFF Correction ON selectively replace value with the average of the surrounding pixels.
AUTO
LUT
A
• Improves contrast & “flicker” with “auto” LUT.
• Small difference with
olled LU
T controlled LUT• Affects only a very small
number of pixels in frame
Contro
30 photons peak, ~10 photons avg.
![Page 63: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/63.jpg)
3 MANAGING READ NOISE{ }3 MANAGING READ NOISECCD EMCCD CMOS
{ }
Read noise expressed in photons is the key specification
{ { {Specs
Read noise expressed in photons is the key specification.Nr (ph) = Nr (e‐)/QE
DistributionUse spatial or singleUse spatial or single
frame rms, not median rms
Optical matchingData collection Analog binning, optical matching
Use slowest clock speed possible
Optical matchingUse pixel noise filter
when possible
Visualization Set lower threshold to a minimum of offset plus 0 to 3 noise standard deviations
Statistical Poisson + uniform Complicated gain Poisson + pixel‐noise model
Poisson + uniform Gaussian
Complicated, gain dependent
Poisson + pixeldependent Gaussian
![Page 64: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/64.jpg)
WHAT ABOUTIMAGES?
• Perfect and real cameras
IMAGES? • Visualization• Histograms• How many photons do
you need?
![Page 65: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/65.jpg)
COMPARING CAMERAS: 1000 PHOTON PEAK3{ } VISUALLY SIMILAR3{ }
T
PerfectsCMOS EMCCD CCD
AUTO
LUT
lled LU
TCo
ntrol
sCMOS:Noise Correction ON
![Page 66: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/66.jpg)
COMPARING CAMERAS: 100 PHOTON PEAK3{ } CAMERA NOISE AND / OR VISUALIZATION MATTER3{ }
T
PerfectsCMOS EMCCD CCD
AUTO
LUT
olled LU
TCo
ntro
sCMOS:Noise Correction ON
![Page 67: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/67.jpg)
HARDER TO SEE IN THE DARK: 30 PHOTON PEAK3{ } CAMERA & VISUALIZATION CRITICAL3{ }
T
PerfectsCMOS EMCCD CCD
AUTO
LUT
rolled LU
TCo
ntr
sCMOS:Noise Correction ON
![Page 68: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/68.jpg)
3{ }HISTOGRAMS: MOST SIMILAR TO THE PERFECT CAMERA
sCMOS EMCCD CCD Perfect
3{ }HISTOGRAMS: MOST SIMILAR TO THE PERFECT CAMERA
sCMOS EMCCD CCD Perfect
hotons
30 ph
photon
sph
oton
s100
100
000 ph
oton
s10
Mean photons: ~35% of peak
![Page 69: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/69.jpg)
HOW MANY PHOTONS DO I NEED WITH A PERFECT CAMERA?3{ }HOW MANY PHOTONS DO I NEED WITH A PERFECT CAMERA?30 100 1000
3{ }UT
AUTO
LUed
LUT
Controll
![Page 70: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/70.jpg)
HOW MANY PHOTONS DO I NEED WITH A REAL CAMERA?3{ }HOW MANY PHOTONS DO I NEED WITH A REAL CAMERA?EMCCDsCMOS CCD Perfect
3{ }0 ph
oton
s
Good enough?
30nsUT
? Bad Good enough?
100 ph
oton
ontrolled L
Good
oton
s
Co Good ? Good
Good1000
pho
Good Good GoodsCMOS:Noise Correction ON Photons : peak intensity in whole, not zoomed, image
![Page 71: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/71.jpg)
HOW TO NARROW THE GAP3{ } HOW TO NARROW THE GAP
1 K h d{
3{ }1 Know what you want to do{ The number of photons required to “see” something depends upon what
you want to see, and how clearly you want to see it, even with a perfect
Turn up the light carefully2{camera.
R l d i lit h h th i h
3 Vi li ti tt
{
{
Real cameras reduce image quality, however when there is enough light, all scientific cameras work well
3 Visualization matters{ Monitor choice, ambient light, LUT settings all make a difference
4 Use the right camera{ Gen II sCMOS cameras have comparable or better image quality than EMCCDs at light levels typically required for visual imaging
![Page 72: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/72.jpg)
CHOOSING AND USING SCIENTIFIC CAMERASCHOOSING AND USING SCIENTIFIC CAMERAS
1 h bl{1 The image problem
2{{ Think in photons2{3 Real cameras are not perfect{4{ Know thyself
5{
{ The Living Image: Case Studies5{ g g
![Page 73: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/73.jpg)
KNOW THYSELF4{ } KNOW THYSELF4{ }
sample contrast
WHAT IS MOSTl ti
frame rate
IMPORTANTFOR YOUR accuracy
resolution
FOR YOUREXPERIMENT?
accuracy
background
![Page 74: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/74.jpg)
CONSIDER THE ENTIRE SYSTEM4{ }CONSIDER THE ENTIRE SYSTEM4{ }
• Lightsheet microscopy (SPIM)• Lightsheet microscopy (SPIM)TWO
EXAMPLES • Single molecule localization microscopy
EXAMPLES
![Page 75: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/75.jpg)
Light Sheet Micoscopy4{ }
Light Sheet Micoscopy4{ } Just like Localization
Microscopy LSM has many faces
LSFM ‐ Light Sheet Fluorescence MicroscopeSPIM ‐ Single Plane Illumination MicroscopeOPM ‐ Oblique PlaneMicroscopysTSLIM – Scanning Thin Sheet Laser Illuminationfaces
Benefits
MicroscopymSPIM – Multidirectional SPIM
Benefits Better sectioning vs. widefield Less photodamage vs. confocal Fast acquisition of large samples
N d l t New developments Multiple Cameras Structured Illumination
75
![Page 76: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/76.jpg)
MuVi SPIM and SIMView4{ }
MuVi‐SPIM and SIMView4{ } Multiple illumination
beams and cameras
Increased isotropy and axial resolutionaxial resolution
Faster scanning with phase Faster scanning with phase or wavelength separation of offset beamsof offset beams
76
![Page 77: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/77.jpg)
4{ } SCMOS IS >20X FASTER THAN EMCCDS4{ }
SPEED!SPEED!
![Page 78: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/78.jpg)
LIGHT SHEET MICROSCOPY4{ } LIGHT SHEET MICROSCOPY4{ }
http://thelivingimage.hamamatsu.com/http://player.vimeo.com/video/74253101
![Page 79: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/79.jpg)
LIGHT SHEET MICROSCOPY4{ } LIGHT SHEET MICROSCOPY4{ }
• Large field of view (high pixel number)• High speed (data rate)• High speed (data rate)• Large dynamic range• Reasonably low noise
CRITICAL CAMERA
CHA AC IS ICS • Reasonably low noise• Rolling shutter synchronized to sample
scanning with variable speed
CHARACTERISTICS
scanning with variable speed
Camera: ORCA Flash4.0 Scientific CMOS
![Page 80: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/80.jpg)
LIGHT SHEET MICROSCOPY4{ } LIGHT SHEET MICROSCOPY4{ }Light sheet microscopy – matching the camera
and optical systemp y
http://www hamamatsu com/sp/sys/en/promotion/mp4/s Lightsheet en htmlhttp://www.hamamatsu.com/sp/sys/en/promotion/mp4/s_Lightsheet_en.html
![Page 81: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/81.jpg)
OPTIMALLY USING THE CAMERA FOR THE TASK4{ }
{4{ }
{{{{
{{{
![Page 82: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/82.jpg)
STANDARD PRACTICE IS NOT THE BEST PRACTICE: USING EMCCD4{ } STANDARD PRACTICE IS NOT THE BEST PRACTICE: USING EMCCD WITH GAIN YIELDS LEAST ACCURATE RESULTS
4{ }
CCD QE: 100%, read noise = 1.8 ph, no background; No fixed pattern noise.
Ad d f J Ch l (Ob L b) N M h10 2013) d i 10 1038/ h 2396Adapted from: J. Chao et al (Ober Lab), Nat. Meth10, 2013) doi:10.1038/nmeth.2396http://www.wardoberlab.com/
![Page 83: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/83.jpg)
UNCORRECTED PRNU CAN LEAD TO LOCALIZATION BIAS4{ } UNCORRECTED PRNU CAN LEAD TO LOCALIZATION BIAS
Localization di t ib ti & bi
Impact of PRNU on localization bias:
4{ }distribution & bias
mEos2 simulation Alexa 647 simulation
0.5% PRNU: 1 – 2 nm @ 100 nm/ pixel
(750 photons)(3000 photons)
Courtesy: Zhen‐li Huang, Huazhong University of Science and Technology, (unpublished)
![Page 84: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/84.jpg)
COMPENSATING READ NOISE VARIATION4{ } COMPENSATING READ NOISE VARIATION4{ }
Courtesy Prof. Joerg Bewersdorf, Yale University
I ti i l ifi d i i t th M i Lik lih d P b bilit M d l li i t d thIncorporating pixel‐specific read noise into the Maximum Likelihood Probability Model eliminates and narrows the asymmetric distribution of localized molecules caused by higher read noise pixels.Courtesy F. Huang, Bewersdorf Lab
![Page 85: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/85.jpg)
MLE RECONSTRUCTION MUST USE A NOISE MODEL4{ } INCLUDING CAMERA NOISEWorst
4{ }
Best
Note: MLE for EMCCDs are also difficult:I t i
Simple and good‐ Inaccurate gain‐ Output PDF not Poisson‐ Even at “high” light, the variance is 2X the mean signal (in photons).
Courtesy: Zhen‐li Huang, Huazhong University of Science and Technology, (unpublished)
![Page 86: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/86.jpg)
SELECTING AND USING CAMERAS: CASE STUDIES4{ } SELECTING AND USING CAMERAS: CASE STUDIES
{4{ }
{{{{
{{{
![Page 87: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/87.jpg)
44{ }44{ }
{Accurate measurement of the distance between two
{ResultsAccurate measurement of the distance between two fluorophores of different colors. distance ~0.77 nm using a dichroic beamsplitter to direct each color of light to separate halves of the CCD camera{halves of the CCD camera.
{Camera Measured PRNU maps for each color. Improved
{Speed: 5 50 s / measurement Imaging: Simultaneous 2 color
{Correctionp p
localization relative accuracy by ~2– 4 nm.
{DetailsSpeed: 5 – 50 s / measurementLight: ~4,000 – 10,000 ph/ mol/frame
~105 ph / mol before bleaching
Imaging: Simultaneous 2 colorCamera: Back‐thinned EM‐CCD, gain off
Nature (2010)| doi:10.1038/nature09163
![Page 88: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/88.jpg)
4{ }4{ }
{Cholera toxin B subunit scale bar: 1 m
Localization Microscopy with Minimal Bleaching. Plasma membrane dynamics for > 60 s (594 frames). 40% better localization precision than “conventional” EMCCD localization
1 m 1 m{Resultsp
Implemented detailed statistical EM noise model into
{{Camera
maximum likelihood reconstruction probability model.{Correction
{Speed: ~60s / reconstructed imageLight: ~100 photons /molecule frame
Mag: 630XCamera: EM‐CCD, Gain ~1000{Details
Courtesy of J. Chao et al (Ober Lab) Adapted from Nat Meth (2013) doi:10.1038/nmeth.2396
![Page 89: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/89.jpg)
Localization Precision “conventional” EMCCD vs. 4{ } sCMOS4{ }
C t f F H B d f L b Y lCourtesy of F. Huang. Bewersdorf Lab, Yale Adapted from F. Huang et al., Nature Methods 10(7): 653‐658 (2013)
![Page 90: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/90.jpg)
MINIMIZING THE GAP: MATCHING THE CAMERA TO4{ }YOUR NEEDS4{ }
Higher AccuracyBetter ResolutionL S l C S i ifi CMOS
ired
Lower Sample Contrast(BT)‐CCD
Scientific CMOSght R
equi
EMCCDLi EMCCD Singlephoton
EMCCD
Speed / Field of ViewFaster (or more pixels)
![Page 91: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/91.jpg)
Choosing and Using Scientific CamerasChoosing and Using Scientific Cameras
1 h bl{1 The image problem
2{{ Think in photons2{3 Real cameras are not perfect{4{ Know thyself
5{
{ The Living Image5{ The Living Image
![Page 92: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/92.jpg)
RESOURCES FOR MICROSCOPISTS5{ } RESOURCES FOR MICROSCOPISTS5{ }
http://thelivingimage hamamatsu comhttp://thelivingimage.hamamatsu.com
![Page 93: Choosing and using scientific cameras€¦ · {3} THE (HYPOTHETICAL) PERFECT CAMERA 100% QE {Every photon is converted into one electron 0 e‐ readread noise {Every electron is digitized](https://reader030.fdocuments.us/reader030/viewer/2022040915/5e8df7508101d9090a06e3e8/html5/thumbnails/93.jpg)
ACKNOWLEDGEMENTSACKNOWLEDGEMENTS
Prof Zhen‐li Huang Huazhong University of Science and TechnologyProf. Zhen‐li Huang, Huazhong University of Science and TechnologyF. Long et al, OPTICS EXPRESS 17741 (2012)
Prof. Joerg Bewersdorf, Yale UniversityF. Huang et al., Nature Methods 10(7): 653‐658 (2013)
Prof. Raimund Ober, Texas Southwestern UniversityJ. Chao et al, Nat Meth (2013) doi:10.1038/nmeth.2396
Prof. Lars Hufnagel, EMBL
Dr. Philip Keller, Janelia Farms
HamamatsuTeruo Takahashi: simulationsHiroyuki Kawai: camera measurementsoyu a a ca e a easu e e tsStephanie Fullerton: presentation guidanceKatsuhide Ito: Lightsheet microscopyEiji Toda: budget
Download 32 f d i 500Download(look on The Living Image)Keith [email protected]
32 fps dynamics. 500 nm scaleCourtesy Vutara / Prof. Bewersdorf