Beyond the visible: A tour to

future of spectroscopy and

imaging

Barmak Heshmat

Dr. Ramesh Raskar

Dr. C. Barsi

1

The big picture• Beyond the visible/IR spectrum (THz spec.)

– New hardware trends

– New computational trends

• Beyond the line of sight (multihop imaging)

– Seeing around the corners

– Seeing through the diffusers

• Beyond the resolvable (subwavelength imaging)

– New hardware trends(course p1)

– New computational trends(course p2)

2

Spectroscopy

• EM waves

• Many types of spectroscopy

3

Wave of spectrometers

• They were all there in the lab but now they are entering consumer market!

– Optical absorption diagnostic

– Raman food analysis

– THz skin, cosmetics, pharm.

Electronics starting to become

portable

Optics starting to

become portable

Example

Just like super computers we still need the accurate lab spectrometers but portable versions can be used in limited applications.

• Raman spectrometer from lab to the key chain!

Tellspec

DeltaNu®

ReporteR™

Smiths Detection

RespondeR™ RCI

Microphazir™

Horiba T64000

?

Hyperspectral and multispectral imaging

6

http://www.markelowitz.com/Hyperspectral.html

7www.markelowitz.com

Measurement samples

8

asri.technion.ac.il

www.popularmechanics.comwww.neo.no

www.bayspec.com www.perception-park.com

Beating the diffraction limit

9

Superlensing Enhanced near field probes

Fluorescence imaging

Super oscillatory lenses

Diffraction limit has limited our resolution in imaging now we are learning ways to go beyond this limit.

Seeking light after scattering

• Going from imaging for human to imaging for computers (measurement in other mathematical spaces

and reconstructing the image)

• Going from single scattering imaging to multi-scattering imaging.

10

2nd Bounce

1st

Bounce3rd

Bounce

Beyond visible/IR spectrum

,

11

New hardware trends• Introductions

• Applications

• PC Switches– New Materials for THz

– Optimizing Excitation of PC Switches

– Nanoplasmonic Structures

• Summary

• Questions?

12

…THz

400

THz

Frequency(Hz)

800

THz

Unique spectroscopy

capabilities

Study of THz dynamics

Faster communication

Imaging and

inspection

13

Why THz

• Noninvasive

• Water in biological systems, protein folding, disease state of tissue

• Vibrational modes for organic molecules

• Picosecond time scale dynamics

14

THz and tissues

• Can measure absorption and refraction index together through pulsed imaging.

15

THz imaging• Security apps, (mm wave <> THz)

• More inspection and analysis apps

16

See a whole gallery here: http://thznetwork.net/index.php/thz-images

Jefferson Lab Ken O, UT, Texas Startiger project

D. Mittleman Rice UQ. Hu, MITBESSY, Germany- (100um res)

THz microscopy

17

R. Kersting, THz-ANSOM 150nmEpithelial tumor cell, A. Tredicuccii, ~15umDiffr

actio

n li

mit

Ordinary

imaging

Near field

imaging

Scanning

probes

D. Zimdars, Picometrix, Inc,

New trends in hardware

18

THz Generation Methods

19

PC Switches

20

Hamamatsu

ZomegaBATOP

Menlo SystemT-Rays

TeraView

21

THz

THz

THz Transmitter

THz

THz

Emitting

THz

Receiving

THz

THz Receiver

22

Tra

nsm

itte

r

Receiv

er

23

H2O,

CO,

Here is what is detected

Temporal profile Frequency composition

Math

24

Our ultimate dream was!

Last 10 yearsin our lab

This yearin our lab

Future, in our hand

The miniaturization process

25

It’s real!

26

Skin

quality

Lung

cancer

agentsBlood

sugar

DrunkReally

Hungry

Cold

Sam

ple

tra

nsm

itta

nce (

Arb

. u

nit

s)

Frequency (Terahertz)27

New Materials for THz

28

Conventional Materials The philosophy of an optical switch defines the desired properties

of the substrate material. highest level of fast photoconductivity modulations:

• high optical density

• high thermal breakdown limit

• high mobility, and Vb and Vsat

• short carrier lifetime (sub-picosecond)

• low dark conductance

• PC switching started by Austin on Si in 1975 (D.H. Auston, Appl. Phys. Lett., 26 (3) 101

(1975))

• C.H. Lee used GaAs in 1977 (C.H. Lee, Appl. Phys. Lett., 30 (2) 84 (1977))

• M.Y. Frankel used LT-GaAs in 1990 (M.Y. Frankel, et al, IEEE Trans on Elec. Devices, 37, 2493, 1990).

29

LT-GaAs

• LT-GaAs has short carrier lifetime (<1ps)

• It has low mobility as well GaAsBi

• Bi is a group V poor metal GaAsBi is shrinking bandgap material

30

GaAsBi Results• 500 GHz bandwidth improvement

• Interesting emissions!

31

Effect of GaAsBi growth condition

• THz emission with variation of different parameters

32

Carbon nanotubes

Increasing the performance with carbon nanotubes

between the gold electrodes of the chip

33

So we made samples.

34

Nanoplasmonic Structures

35

Nanoplasmonics

• Engineering surface electron density wavesin the metallic nanostructures to achieve an enhanced optical response.

• A key property of nanoplasmonics is its capability to efficiently couple light into subwavelength structures.

36

Nanoplasmonics: An Example

Tuning annular nano-apertures

B. Heshmat, D. Li, T. E. Darcie, R. Gordon, " Tuning plasmonic resonances of an annular aperture in

metal plate "Optics Express, Vol. 19, Iss. 7, pp. 5912–5923 (2011). 37

Nanoplasmoincs for THz PC Switches

38

Nanoplasmoincs in THz PC switches

B. Heshmat, H. Pahlevaninezhad,Y. Pang, M. Masnadi, R. Lewis, T. Tiedje, R. Gordon and T. E.

Darcie "Nanoplasmonic Terahertz Photoconductive Switch" Nano letter, accepted. 39

Results of Using Nanoplasmonic Structures

Peak-to-peak response enhancements of 40×, 10×, and 2×, compared to GaAs, LT-GaAs and Commerical device.

40

Past, Present, Future

41

Challenges• THz waves have long wavelength; biological structures, many

important ones, are small…

• Living things need water: THz radiation and water are not “best friends”…

• Unless you work hard, no clear spectroscopic features at THz are visible for many samples.

• Some solutions to above problems are coming out.

42

Summary of new trends in hardware• 100 GHz to 10THz region of EM waves are called THz,

have been unexplored, but we are finally closing the gap.

• Main challenge is detection and generation.

• Major sources and QCLs, schottky diodes, PC switches and nonlinear crystals.

• There is room for enhancement through material, optics and nanoplasmonics.

• Many exciting applications from early cancer detection to inspection of organic materials and faster telecommunication. 43

New computational trends

44

• They also investigated the difference between a random mask and an optimized one.

https://www.youtube.com/watch?v=CWlCa3qbzU0

The optimal block size for the block-based CS is a function of the local image characteristics, and different block sizes can be assigned to different regions.

Summary of computational trends

• Compressive measurements, where you measure the minimum amount of points to reconstruct an image with known priors.

• Layer separation based on pulse features

• Reference-free measurements in THz imaging

• Here is a demo:

55

Beyond the line of sight

56

Time-of-flightIn Situ remote sensing

Require direct path between objects sensor

JPL

Hyperspectral Imaging

Spectroscopic

Monterrey Bay Aquarium Research Institutehttp://www.mbari.org/coastal/

http://earthobservatory.nasa.gov/Features/Lidar/

http://aviris.jpl.nasa.gov/html/aviris.freedata.html

Optical remote sensing

What if there is no direct path?

Receiver

Source

?

Computation + optics

J. Bertolotti, et al. Nature 491 (2012).S. M. Popoff, et al. Nat. Commun. 1 (2010)

• Relies on coherence/correlation

• Small field of view

• Short standoff distance

60Nature Photonics 6, 549–553 (2012)

A. Velten, et al. Nat. Commun. 3 (2012).

Time is a parameter for imaging

x

t Hyperbola

x

Laser

Streak

camera

Diffuser

Object

Time-resolved image formation

Source: Ti:Sapph (λ0 =795nm, but could use other wavelengths)Detector: Streak Camera (δt ≈2ps)Different ray paths register at different times hyperbolic impulse response (x – ct)

Il (x, t) = I0 G(xl, x, ¢x )N(qin )N(qout )R( ¢x )d ct - (rl ( ¢x )+ rc( ¢x ))( )d ¢xò

Time-resolved image formation

Geometry Diffuser Object

Reflectance

Time

constraint

Il R(x),N(qin,out ){ }

1)(0for ,ˆˆ1minarg

12(.)),(

xRIIL

L

l

num

ll

meas

lNxR

Inverse problem

Given a set of streak images

Find the unknown reflectance R(x)

Streak Image

Experimental setup

66

Visible volume

Experimental setup

• Need to know something about diffuser

Unknown reflectance

Unknown reflectance

• Assume object geometry known (can get from previous work)

• Wide field reconstruction

• Works for incoherent light

Moving on to the miniaturization

Time of flight camera

• Continuous wave instead of pulsed

• Cheaper, safer, more compact, but less accurate.

R. Raskar, et al., “Coded Time of Flight Cameras: Sparse Deconvolution to Address Multipath Interference and Recover Time Profiles”, SIGGRAPH

Asia 2013.

3d imaging through turbulence

Solving occlusion problems

www.picassodreams.com/photos/nyc_skyscrapers/

http://www.nasa.gov/vision/earth/lookingatearth/h2005_katrina.html

http://www.fjellandfjord.com/article.php?id=166

http://www.soest.hawaii.edu/GG/HCV/loihi.html

Generalizations for remote imaging

Summary of time of flight imaging• Moving from single scattering to multiscattering

(multihop) imaging

• Different reconstruction techniques that rely on previous optimization techniques can be used.

• Moving from expensive ultrafast hardware to cheaper slow hardware that operates on modulated light

• Now we can recover what is in the visible volume of these cameras

N. Naik, C. Barsi, A. Velten, R. Raskar.“Estimating spatially varying reflectance through scattering layers using time-resolve inversion.” JOSA A.

Two picosecond time resolution

Streak camera details