Page 1 HIP Tour Briefing

Hyperspectral Image Projector (HIP)

•A scene projector for in-lab testing of imaging sensors

using spectrally-realistic scenes

Additional information and

publications available from:

joe.rice@nist.gov

Or go to the NIST website ( www.nist.gov ) and search “HIP”

Page 2 HIP Tour Briefing

Laboratory sources do not match reality very closely

We calibrate with uniform sources…

Example: lamp-illuminated

integrating sphere for reflective bands,

(or blackbody for IR emissive bands)

But reality is spatially non-uniform:

Example: AVIRIS image of

North Island Naval Air Station,

San Diego, CA

Page 3 HIP Tour Briefing

The same situation applies spectrally

0.0000

50.000

100.00

150.00

200.00

250.00

400 450 500 550 600 650 700 750

Ra

dia

nc

e

Wavelength [nm]

Lamps standards and blackbodies offer

only a Planckian-shaped spectrum.

But reality has many different spectra…

Example: ENVI/SMACC was used to find

these 7 endmember spectra from the

San Diego Naval Air Station data cube.

NIST FEL Lamp

0

2000

4000

6000

8000

10000

12000

450 500 550 600 650 700 750

Fig5

EM1EM2EM3EM4

EM5EM6EM7

Inte

nsi

ty (

arb

itra

ry u

nits

)Wavelength (nm)

SMACC Reference: J. Gruninger, A. J. Ratkowski, and M. L. Hoke,

“The sequential maximum angle convex cone (SMACC) endmember model,”

Proc. SPIE 5425, 1-14 (2004).

Page 4 HIP Tour Briefing

0

100

200

300

400

500

600

400 500 600 700 800 900 1000 1100

OLI Matching Absolute

Sp

ectr

al R

adia

nce

(W

/m2sr

um

)

Wavelength (nm)

TOA Solar

Bare Desert Soil

Vegetation

HIP VNIR Limits

Measured from HIP Prototype VNIR Spectral Engine 11/3/10

Modeled Spatial Engine: XGA DMD f/3 with 20% Transmittance

The Hyperspectral Image Projector (HIP) Can Match Typical

Reflected-Solar Radiance Spectra

• The HIP provides enough light to simulate a bright sunny day outside

• Red data plots below show how well the HIP simulates different real-world spectra

Page 5 HIP Tour Briefing

• Broadband Fiber Light Source

– Provides high brightness in a small spot for high spectral resolution

– Fused-silica fiber supercontinuum source for VNIR/SWIR

• Spectral Engine

– Fiber source light is spectrally dispersed across a digital micromirror device (DMD)

– Image reflected from on-pixels of DMD is spatially integrated and determines spectrum

• Spatial Engine

– Uses another DMD to project spectra into customer’s Unit Under Test (UUT) per 2D spatial

abundance images, through a collimator, in sync with spectral engine

• Reference Instrument

– Well-calibrated imaging spectrometer to measure the spectrum of each spatial pixel at the

output, relative to NIST reference standards, to provide truth data

HIP Prototype OverviewSpectral Engine

(Spectra Generator)

Reference InstrumentBroadband Fiber

Light Source

Spatial Engine

(Image Projector) Unit Under Test

Computer

Page 6 HIP Tour Briefing

Supercontinuum Fiber Laser: A “White” Broadband Laser

• Utilizes non-linear effects in a photonic crystal optical fiber to greatly broaden the spectrum of a 1064 nm pump laser.

• Broadband light is generated in a single-mode (5 um core diameter) photonic crystal (holey) optical fiber

– No etendue issues as with lamps or blackbodies.

– Ideally suited for coupling to a spectral engine.

• High power and high spectral resolution:

– 3mW/nm spectral power density from 450 nm to 1700 nm

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

500 1000 1500 2000 2500

SC450 ASD

Powe

r (ar

bitra

ry u

nits

)

Wavelength (nm)

Page 7 HIP Tour Briefing

Digital Micromirror Device (DMD)

• An array of MEMS micromirror elements

• Developed by Texas Instruments (TI)

– 1024 x 768 elements, +/- 12 degree tilt angle

– Aluminum mirrors

– 13.7 micron pitch

– < 24 microseconds mechanical switching time.

• For visible to 2500 nm applications we have used:

TI Discovery 1100 electronics board with an

Accessory Light Processor (ALP) electronics board

(see dlp.com).

• For longer wavelength infrared developments we are using DMDs where the glass window is replaced by a ZnSe window.

• Control algorithms are being written by us using LabVIEW with a USB interface to a standard PC.

• For the prototypes in this proposal, we plan to use the TI Discovery 3000 and ALP3.

MEMS = Micro-Electro-Mechanical System

MAPS = Micromirror Array Projection System

Page 8 HIP Tour Briefing

Principle of the Spectrally Programmable Source

(also called the Spectral Engine)

Dispersing element Spatial Light Modulator(SLM) Recombine the Light

SLM Light Guide

• Enabling Technology: SLM

– Digital micromirror devices (DMDs)

– Liquid crystal on silicon arrays (LCOS arrays)

Page 9 HIP Tour Briefing

How the DMD is used in a spectrally tunable source:

Monochromator Mode

Near IRUV Visible

1024 mirrors

768

mir

rors

Wavelength

Inte

nsi

ty

mirror array

Wavelength and Intensity Selection with a Digital Mirror Device (note mirror # reduced by a factor of 10 for clarity)

Mirrors in "On"Position

Page 10 HIP Tour Briefing

Near IRUV Visible

Digital Mirror Device as a Spectral Light Engine (note mirror # reduced by a factor of 10 for clarity)

1024 mirrors

768

mir

rors

Wavelength

Inte

nsi

ty

mirror array

How the DMD is used to create an arbitrarily programmable

spectrum

Page 11 HIP Tour Briefing

Design Concept for

VNIR/SWIR Spectral Engine

0.0

0.2

0.4

0.6

0.8

1.0

1.2

1.4

1.6

1.8

2.0

500 1000 1500 2000 2500

SC450 ASD

Powe

r (ar

bitra

ry u

nits

)

Wavelength (nm)

Supercontinuum

Fiber Light Source

DMD2

Recombining

System

Dispersing Prisms

DMD1

VNIR SWIR

Input

Output to

Spatial

Engine

SWIR

VNIR

• Splits input into VNIR and SWIR

• One DMD (1024x768) for each range

• Optical modeling in Zemax and FRED

– Prisms selected

– Design meets requirements

❖Spectral range

❖Transmittance

❖Spectral resolution

• Mechanical modeling in Solid Works

VNIR/SWIR spectral

engine fits in 19” rack-

mountable chassis

Source Spectrum

Page 12 HIP Tour Briefing

HIP VNIR-SWIR Spectral Engine

Inside view from back:Outside view from front:

Supercontinuum Light Source

VNIR

Spectral

Engine

SWIR

Spectral

Engine

Page 13 HIP Tour Briefing

HIP VNIR-SWIR Spectral Engine

Supercontinuum

Light Source

VNIR

Spectral

Engine

SWIR

Spectral

Engine

Source

Collimator

Electronics

Section

Output

}}

View from back:

Dichroic

Beamsplitter

Dichroic

Beamcombiner

Page 14 HIP Tour Briefing

VNIR

Spectral

Engine

SWIR

Spectral

Engine

Spatial

Engine

Collimator For Projection

to UUT

Super-

continuum

Source

Power Supply

Page 15 HIP Tour Briefing

0

100

200

300

400

500

600

400 500 600 700 800 900 1000 1100

OLI Matching Absolute

Sp

ectr

al R

adia

nce

(W

/m2sr

um

)

Wavelength (nm)

TOA Solar

Bare Desert Soil

Vegetation

HIP VNIR Limits

Measured from HIP Prototype VNIR Spectral Engine 11/3/10

Modeled Spatial Engine: XGA DMD f/3 with 20% Transmittance

The Hyperspectral Image Projector (HIP) Can Match Typical

Reflected-Solar Radiance Spectra

• The HIP provides enough light to simulate a bright sunny day outside

• Red data plots below show how well the HIP simulates different real-world spectra

Page 16 HIP Tour Briefing

HIP Prototype Specifications

Parameter Specification

Spectral Range450 nm to 2500 nm (VNIR-SWIR)

(extension to 350 nm in progress)

Spectral Resolution5 nm VNIR

8 nm SWIR

VNIR/SWIR/MWIR Sync. Accuracy 1 microsecond

Spatial Format 1024 H × 768 V

Projected FOV** 7.9° H × 5.9° V

Spatial Resolution** 0.135 mrad

Average Spectral Radiance 1000 W/m2srmm

Bit Depth and Frame Rate

12 bits at 250 Hz max;

8 bits per component at180 Hz/N typical*

1 Bit at 11 kHz max

Contrast Ratio 1000:1

Wavelength Accuracy 2 nm

Radiance Accuracy 2%

*N = number of components (i.e. eigenspectra) per frame

**Depends on collimator used. Values shown are for the standard 100 mm collimator. 500 mm collimator also available.

Page 17 HIP Tour Briefing

VNIR Spectral Resolution Test Data

0

0.2

0.4

0.6

0.8

1

628 628.5 629 629.5 630 630.5 631 631.5 632

Single Column 629 nm HIP HR4000 18Oct2010

Respo

nse (

peak-n

orm

aliz

ed

)

Wavelength (nm)

1.5 nmMeets the < 5 nm spec

Spectral Resolution of Test

Instrumentation is 0.2 nm

Page 18 HIP Tour Briefing

SWIR Spectral Resolution Test Data

Meets the < 8 nm spec

Spectral Resolution of Test

Instrumentation is 0.5 nm

0.0

0.2

0.4

0.6

0.8

1.0

1585 1590 1595 1600 1605 1610

IB 1600 nm HIP Slocal 06Jun2011

Re

sp

on

se

(p

ea

k-n

orm

aliz

ed

)

Wavelength (nm)

5 nm

Page 19 HIP Tour Briefing

VNIR-SWIR Spatial Engine

Prototype Mechanical Design

DMD

Interchangeable DMD Driver Electronics PC Boards:

Cooling FansReplaceable

Collimator

Purged, Sealed

Optics SectionLiquid Light Guide

Coupling from

Spectral Engine

Electrical

Connections

to Spectral

Engine

Crossfield/Infiniband (pink)

or

DLi3000/DVI (red)

or

D3000/ALP (green):

Page 20 HIP Tour Briefing

Background: Digital Light Processing (DLP) Projectors

Reference: www.dlp.com

Page 21 HIP Tour Briefing

HIP Image Cube Projection Concept

Spatial Engine

• Projects images with

component spectra into

sensor

• Reference instrument

characterizes output

Reference

Instrument

Sensor

Under Test

Supercontinuum

Light SourceSpectral Engine

• Uses light from supercontinuum source

• Produces programmable spectra that

match component spectra

• Directs these to Spatial Engine

0.0

0.2

0.4

0.6

0.8

1.0

400 450 500 550 600 650 700 750

Pow

er

Wavelength (nm)

Example Spectrum

Input

Hyperspectral

Image CubeComponent

Spectra

Principle-components-type

algorithm reduces image cube

to Component Spectra

and Abundance Images

Abundance

Images

Page 22 HIP Tour Briefing

Input

Image Cube

Example: AVIRIS Image Cube

of San Diego Naval Air Station

First, ENVI/SMACC was used to find these

Endmember Spectra and their Abundances

J. Gruninger, A. J. Ratkowski, and M. L. Hoke, “The sequential maximum angle

convex cone (SMACC) endmember model,” Proc. SPIE 5425, 1-14 (2004).

EM-3

EM-4

EM-2 EM-5

EM-6

Eigenspectra Abundances

EM-1

EM-1

Eigenspectra

EM-2

EM-3

EM-5

EM-4

EM-6

EM-7

Compressive Projection is Used to Achieve Higher Brightness

Then we need only project N = 6 broadband spectra

instead of M = 30+ monochromatic spectra.

Page 23 HIP Tour Briefing

Example Sensor

Test at the HIP

• Used a pushbroom Hyperspectral Imager (HSI) from collaborators at University of

Colorado – This sensor is prototype instrument for NASA.

• Input data was a real scene collected by HSI

• Projected by the HIP and measured by the HSI.

• HSI scanned HIP to simulate ground track motion

HIP Projected, HSI Measured: HSI

Page 24 HIP Tour Briefing

HIP Projection of MODIS Satellite Image Into ORCA Prototype in the Lab

Original Image HIP Image Measured by ORCA Difference Image (at 645 nm)

• ORCA (Ocean Radiometer for Carbon Assessment)

was a NASA prototype ocean color sensor for the

NASA PACE (Pre- Aerosol Cloud Ecosystems) mission.

• Clouds provide stray light that interferes

with the ability to measure ocean color and

bio-chemistry to quantify carbon processes

• HIP difference images potentially allow

measurement of stray light effects with

realistic scenes

Typical Cloud Pixel Typical Ocean Pixel

Page 25 HIP Tour Briefing

Monochromatic Band Projection

•Using the HIP, we projected six MODIS

monochromatic band images into ORCA at the following

band center wavelengths:

“Eigenspectrum #” Wavelength (nm)

1 645

2 667

3 678

4 748

5 859

6 869

•All six images were projected sequentially, but all within

the integration time of ORCA

•The radiance levels at each wavelength had been

adjusted prior to the test, using a calibrated detector at

the output, so that they would have equal radiance

levels for a spatially uniform scene

-0.02

0.00

0.02

0.04

0.06

0.08

0.10

0.12

550 600 650 700 750 800 850 900 950

ASD_111207_002

Ra

dia

nce (

arb

itra

ry u

nits)

Wavelength (nm)

•Measurement of radiance levels of the 6

monochromatic bands from the HIP as

made using an ASD spectrometer:

Page 26 HIP Tour Briefing

ORCA

HIP

Spatial

Engine

Liquid Light Guide:

•Couples HIP Spectral Engine

to HIP Spatial Engine

Translation stage moves HIP between Reference Instruments and ORCA

NIST Reference Instruments:

•Enable measurements of HIP

output radiance (ground truth)

•Includes an ASD spectrometer

and a CCD camera with tunable filters

ORCA at the HIP lab at NIST

Page 27 HIP Tour Briefing

HIP Prototype Specifications

Parameter Specification

Spectral Range450 nm to 2500 nm (VNIR-SWIR)

(extension to 350 nm in progress)

Spectral Resolution5 nm VNIR

8 nm SWIR

VNIR/SWIR/MWIR Sync. Accuracy 1 microsecond

Spatial Format 1024 H × 768 V

Projected FOV** 7.9° H × 5.9° V

Spatial Resolution** 0.135 mrad

Average Spectral Radiance 1000 W/m2srmm

Bit Depth and Frame Rate

12 bits at 250 Hz max;

8 bits per component at180 Hz/N typical*

1 Bit at 11 kHz max

Contrast Ratio 1000:1

Wavelength Accuracy 2 nm

Radiance Accuracy 2%

*N = number of components (i.e. eigenspectra) per frame

**Depends on collimator used. Values shown are for the standard 100 mm collimator. 500 mm collimator also available.