Practical Spectral Photography

Ralf Habel1

Michael Kudenov2

Michael Wimmer1

Institute of Computer Graphics and AlgorithmsVienna University of Technology1

Optical Detection LabUniversity of Arizona2

Ralf Habel 2

Motivation

Spectroscopy is most important analysis tool in all natural sciences

Astrophysics, chemical/material sciences, biomedicine, geophysics,…

Industry applications:Mining, airborne sensing, QA,…

In computer graphics:ColorsMaterial reflectanceSpectral/predictive rendering…

Ralf Habel 3

Spectral Imaging

Records image at narrow wavelength bandsIn visible range not only RGB (3 channels)but many more (6-400 channels)

Result: 3D data cube2 spatial image axis1 wavelength axis

Ralf Habel 4

Spectral Imaging

Usually done with highly specialized devicesMany methods to build devices Scanning slits, rotating mirrors, special sensor, filters, prisms, …Usually scan along one of the data cube axisAll very costly due to opto-mechanical components

“Simplest” spectral imager:Camera + band filtersRequires switching of filters Limited in number of bands

Ralf Habel 5

Motivation

Why not use consumer cameras and equipment for spectral imaging?

High quality, very sensitiveHighly accurate lenses

Practical Constraints:No camera modification No lab/desktop/optical bench setup No expensive components

Ralf Habel 6

CTIS Principle

Computed Tomography Image SpectrometerDiffraction grating parallel-projects 3D data cube in different directions on image plane (sensor):

Ralf Habel 7

CTIS Principle

Computed Tomography Image SpectrometerDiffraction grating parallel-projects 3D data cube in different directions on image plane (sensor):

Sensor records projections of 3D data cube All information needed is recorded in one image“Snapshot” spectrometry

Challenge is to reconstruct 3Ddata cube from projections

Tomographic rec. with ExpectationMaximization More details in paper

Ralf Habel 8

CTIS Principle

Ralf Habel 9

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual image

Ralf Habel 10

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual imageCollimating lens makes light parallel

Ralf Habel 11

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual imageCollimating lens makes light parallelDiffraction grating creates projections

Ralf Habel 12

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual imageCollimating lens makes light parallelDiffraction grating creates projectionsRe-imaging lens focuses on sensor

Ralf Habel 13

CTIS Optical Path

Imaging lens + square/slit aperture creates virtual imageCollimating lens makes light parallelDiffraction grating creates projectionsRe-imaging lens focuses on sensor

Ralf Habel 14

CTIS Optical Path

Built with:Drain pipe & duct tape50mm, 17-40mm and macro lensDiffraction gel ($2 per sheet) in gel holder

Ralf Habel 15

CTIS Camera Objective

Ralf Habel 16

CTIS Camera Objective

Ralf Habel 17

HDR Image Acquisition

No overexposed pixels allowedProjections (diffractions) weaker than center imageAvoids noisy signal where camera response is weak

Ralf Habel 18

Spatial Wavelength Calibration

Mapping from 3D data cube into projectionsLaser pointers (red, green and blue) with known wavelengths shot through a diffusor and pinhole

Monochromatic point light source

Pictures of pinhole give mapping of one voxel in 3D data cube All other projections valuesinterpolated/extrapolated

Ralf Habel 19

CTIS Principle

Ralf Habel 20

Spatial Wavelength Calibration

Ralf Habel 21

Spectral Response Calibration

Spectral response of the diffraction grating + RGB sensor for red, green and blue

Picture of light source with continuous known spectrumWe use calibrated halogen lamp

Ralf Habel 22

Spectral Photography Results

Take HDR picture with CTIS camera objectiveReconstruct 3D data cube for red, green and blue image color channels

Mapping from spatial calibrationCombine RGB spectral response of each pixel to true spectrum with spectral de-mosaicking

Mapping from spectral response calibration

Ralf Habel 23

Spectral Photography Results

Protoype data cube resolutions:

120x120 pixels4.59 nm (54 channels)

Accuracy reduced in high blue and low reds dueto color filtersSlight Expectation Maximization reconstruction artifacts

Nowhere near possible optimum!

Ralf Habel 24

Spectral Photography Results

Ralf Habel 25

Spectral Photography Results

Ralf Habel 26

Future

Better CTIS objectiveDrain pipes and duct tape have their limits…Optimized optical path and componentsMore compact/integrated device

Increase data cube resolution/accuracy:Structured apertureDigital holography – form diffraction/projections in any way

Better solutions to tomographic reconstruction

Is active research in opticsNo vision based approach yet!

Ralf Habel 27

Future

Turning mobile devices into spectrometers - consumer spectroscopy?

8 MP high sensitivity sensorsHDR capabilitiesVery low cost!

“Snapshot” capability: Spectral movies with consumer cameras? Not only good for computer graphics:

Blood sample analysisWater contamination analysisAs part of a TricorderTM

Ralf Habel 28

Practical Spectral Photography

Thank You!