Photon-counting 3D Imaging Ladars · Photon-counting 3D Imaging Ladars M. Jalal Khan 22 May 2014...
Transcript of Photon-counting 3D Imaging Ladars · Photon-counting 3D Imaging Ladars M. Jalal Khan 22 May 2014...
Presentation Name - 1
Author Initials MM/DD/YY
Derived from: Source
Declassify on: Date
Photon-counting 3D Imaging Ladars
M. Jalal Khan
22 May 2014
This slide is UNCLASSIFIED when
detached from classified enclosures
UNCLASSIFIED
UNCLASSIFIED
Presentation Name - 2
Author Initials MM/DD/YY
Geolocated 3D Image
3-D Imaging with Laser Radar
Pulsed
Laser
Receiver
Optics
Focal plane array
of timers
Advantages: - E/O-like resolution: 30 cm from 20 kft
- Radar-like ranging: 20 cm from 20 kft
- Active illumination: Night-time imaging
Disadvantages: - Requires clear line-of-sight (no clouds)
- Active illumination: limited power
results in limited area
Presentation Name - 3
Author Initials MM/DD/YY
3D Imagery Comparison: SRTM vs Ladar UNCLASSIFIED
UNCLASSIFIED
Existing global 3D imagery is low resolution; ladar
enables high resolution imagery
Current Resolution < 1 m Current Resolution ≈ 30 m
Presentation Name - 4
Author Initials MM/DD/YY
Example 3-D Imagery: Kennedy Space Flight Center
Point cloud: 5 cm post-spacing Color: ellipsoid height
Shade: relative intensity
Tie Down
Inset with Space Shuttle. Color coding re-scaled
Approved for Public Release 11-067
Presentation Name - 5
Author Initials MM/DD/YY
Approved for Public Release 11-067
Example 3D Imagery: Grand Canyon
South Rim
Colorado
River Gorge
11 km
5.2 km
ALIRT Collection Parameters: Flight altitude: 21.9 kft MSL (6.68 km)
Ground speed: 270 – 330 kts (140 – 170 m/s)
Time on station: 80 minutes
Collected 8 swaths, each 38 km long (11 km shown)
Swath width & overlap: 1250 m, 50%
Collection date: Sept 23, 2010, ~2am local time
1.43 km
vertical
relief
z = 721 m
z = 2156 m
Product resolution: 1 m
Presentation Name - 6
Author Initials MM/DD/YY
Example 3-D Imagery: Grand Canyon South Rim
Height in Meters
2,250
1,800
1,080
Presentation Name - 7
Author Initials MM/DD/YY
Outline
• Overview of 3-D Ladar
• ALIRT system
– Hardware and processing
– Example collections and data utility
• Ladar system considerations
– Detector technology
– Scanning
– Signal & laser power
– Measurement rates
Presentation Name - 8
Author Initials MM/DD/YY
ALIRT 3-D Imaging System
(Airborne Lidar Imaging Research Testbed)
Vibration Isolated
Optical Bench
Lidar Package
Microchip Laser
Scanned
Line-of-Sight
(±15° × ±23° FOR)
Geiger-Mode
APD Array
32x128, 50 μm
Telescope:
10-cm Aperture
f/10 optics
Operational
since October 2010
Satcom
Window
2 W microchip laser
On-board Processing
NSS / BAE Two-Axis Scan Mirror
Applanix GPS / IMU POS-AV 610
Presentation Name - 9
Author Initials MM/DD/YY
System in Aircraft
Operation and
Processing
Stations
Presentation Name - 10
Author Initials MM/DD/YY
Geiger-Mode Imager: Photon-to-Digital Conversion
• Single-photon sensitivity
• Noiseless readout
• High-bandwidth timing
• Large format
Avalanche
Photodiode
(APD)
Digital
Timing
Circuit
Digitally
Encoded
Photon
Arrival Time
Pixel Circuit
64 x 256 (2009) 32 x 128 (2006)
Lenslet
Array
Avalanche
Photodiode
(APD)
Array
CMOS
Readout
Device Profile
Polyimide
passivation
4 mm
InP
Su
bstr
ate
InGaAsP
absorber
InP
multiplier
InP APD
Incident
Photons
Carrier
Generation
Avalanche
Region
APD Detector
Array
Photon
Presentation Name - 11
Author Initials MM/DD/YY
Level 0
ALIRT Processing Workflow
Level 1 Level 2 Level 3
Raw Data Streams
• Range measurements
• GPS IMU
• Mirror encoders
• GPS base station
• Meta data
Noisy Point Cloud
• All range measurements in one big set
Gridded Point Cloud
• Noise removed by coincidence processing
• Single points created from multiple range measurements
Aggregated Point
Clouds
• Many L2 products intelligently aggregated
Level 4
Transform Coincidence Registration Finishing
Presentation Name - 12
Author Initials MM/DD/YY
Outline
• Overview of 3-D Ladar
• ALIRT system
– Hardware and processing
– Example collections and data utility
• Ladar system considerations
– Detector technology
– Scanning
– Signal & laser power
– Measurement rates
Presentation Name - 13
Author Initials MM/DD/YY
ALIRT Supports Operation Unified Response
• 12 January 2010: Magnitude 7 earthquake
occurs at Haiti
– Epicenter just south of Port-au-Prince
• Significant loss of life and damage to
infrastructure
– Most buildings constructed from
unreinforced concrete
• USSOUTHCOM tasked with coordinating
U.S. response
– Critical need for situational awareness
• ALIRT was asked to provide support
15 January
– Initial flight was on 18 January at 0157 hrs.
Presentation Name - 14
Author Initials MM/DD/YY
Wide Area and High Resolution Combined Haiti Imagery January 2010
• Challenge was to demonstrate high quality mapping capability
• Suburban Port-au-Prince mapped from 10 k-ft, rendered at DTED-6
Presentation Name - 15
Author Initials MM/DD/YY
Trafficability Analysis
18 Jan 2010
0538Z / 0038L Peak ruble height
Street surface
2.5 m
Rubble Volume:
Approx 2.5 m high by
40 m long = 100 m3
Rue Paul V
Administration General
des Contributions
(Tax Office)
Rue Pavee
N
Geometric data allows direct and precise
measure of height and slope, enabling
determination of trafficability by vehicle type
Presentation Name - 16
Author Initials MM/DD/YY
Mapping Mass Migration Day-to-Day change Detection
18 JAN 2010
0531Z / 0031L 24,900 m3 of Tents
19 JAN 2010
0212Z / 2112L 34,700 m3 of Tents
Objects have Departed
Objects have been Added
39%
Increase in
Tents
15 m Departed
Arrived
Presentation Name - 17
Author Initials MM/DD/YY
Assisting Relief Distribution
ID Available Helicopter Landing Zones
HLZ with buffer
zone in green
H-60 Black Hawk
• Daytime operations
• Urban environment
• 50 m buffer area
Nearby obstacles in red
HLZ at National Palace
LIDAR Large Area
Mapping for
Potential Black
Hawk Helicopter
Landing Zones
Area collected 22 Jan 2010
National Palace GEO: 18 32 37 N / 072 20 19 W
Presentation Name - 18
Author Initials MM/DD/YY
Rain Forest in El Yunque National Forest, Puerto Rico
400 m Colorbar:
ellipsoid height
Presentation Name - 19
Author Initials MM/DD/YY
Rain Forest FOPEN - El Yunque National Forest
Collection parameters: Flight altitude: 10 kft AGL
Imaging time: 15 sec in each
of two passes
Laser: 1 W, 8 kHz, 1064 nm
Presentation Name - 20
Author Initials MM/DD/YY
Trail Detection
Human Activity Layer from L2 Data
Colorbar:
height above ground
Trail
MACHETE - 21
JMK 04/11/14
FALCON-I Foliage Penetration System
Dash-8
Orbit
Radius:
3 km
Ladar Resolution
Cell: (0.25 m)3
Altitude:
20 kft / 6 km
UNCLASSIFIED // FOUO
Image human activity layer under
90% obscuration:
− MACHETE ladar resolution: 25 cm
− Collect ellipse with area ~ 1.5 km2
Applications:
− Detect structures / trails / open areas
− Target classification / identification
− Generation of bare-earth maps
− Ingress / egress route planning
Scanned diameter
~ 1.5 km
Presentation Name - 22
Author Initials MM/DD/YY
Outline
• Overview of 3-D Ladar
• ALIRT system
– Hardware and processing
– Example collections and data utility
• Ladar Technology
– Lasers
– Detectors
• Summary
Presentation Name - 23
Author Initials MM/DD/YY
Nd:YAG
1.1 wt.%
Cr4+:
YAG
Input Coating Output Coupler
Output Pump
YA
G
YA
G
Laser Schematic
Gain Medium
Saturable Absorber
Laser Attached
to Pump Fiber
m-Chip
Laser
m-Chip Laser Technology
• Laser properties:
– Small, compact
– Highly efficient
• Minimum power requirements
– Short pulse (<1 ns)
• <10-cm resolution
– High repetition rate
• Fast imaging times
Presentation Name - 24
Author Initials MM/DD/YY
Detector Technology
Detector Format
Single- or Few-Pixel + sophisticated readout
circuitry possible
- lower collection rate
Large Array + high collection rate
- simple pixel readout circuit
De
tec
tor
Op
era
tin
g M
od
e Linear-Mode
+ Simple noise rejection
- Need lots of light; require 1000
detected photons
- Compromised range resolution
Most commercial
systems + COTS: Optech, Leica,
Reigl, etc
+ Simple detection
- Accurate, fast scanning
- Lower collection rate
- Medium range: 0.3 - 3 km
New products + 3D video
+ Simple, Compact
- Short ranges: 0.1 - 1 km
Photon-Counting + Most efficient use of every
photon; require only 10-15
detected photons
+ Tight range resolution
+/- Post-processing
- High data volume
Early
demonstrations
Specialized systems: (e.g. ALIRT, JIGSAW, HALOE,
FALCON-I)
Cameras becoming available
+ Long-range: 3 – 15 km
+ Country-sized collection
rates
- Accurate scanning
103 - 105× more detectors
100
× le
ss
ligh
t
Presentation Name - 25
Author Initials MM/DD/YY
Detector Technology
Detector Format
Single- or Few-Pixel + sophisticated readout
circuitry possible
- lower collection rate
Large Array + high collection rate
- simple pixel readout circuit
De
tec
tor
Op
era
tin
g M
od
e Linear-Mode
+ Simple noise rejection
- Need lots of light; require 1000
detected photons
- Compromised range resolution
Most commercial
systems + COTS: Optech, Leica,
Reigl, etc
+ Simple detection
- Accurate, fast scanning
- Lower collection rate
- Medium range: 0.3 - 3 km
“Flash” Ladar + 3D video
+ Simple, Compact
- Short ranges: 0.1 - 1 km
Photon-Counting + Most efficient use of every
photon; require only 10-15
detected photons
+ Tight range resolution
+/- Post-processing
- High data volume
Early
demonstrations
Specialized systems: (e.g. ALIRT, JIGSAW, HALOE,
FALCON-I)
Cameras becoming available
+ Long-range: 3 – 15 km
+ Country-sized collection
rates
- Accurate scanning
103 - 105× more detectors
100
× le
ss
ligh
t
Presentation Name - 26
Author Initials MM/DD/YY
Detector Technology
Detector Format
Single- or Few-Pixel + sophisticated readout
circuitry possible
- lower collection rate
Large Array + high collection rate
- simple pixel readout circuit
De
tec
tor
Op
era
tin
g M
od
e Linear-Mode
+ Simple noise rejection
- Need lots of light; require 1000
detected photons
- Compromised range resolution
Most commercial
systems + COTS: Optech, Leica, Reigl,
etc
+ Simple detection
- Accurate, fast scanning
- Lower collection rate
- Medium range: 0.3 - 3 km
“Flash” Ladar + 3D video
+ Simple, Compact
- Short ranges: 0.1 - 1 km
Photon-Counting + Most efficient use of every
photon; require only 10-15
detected photons
+ Tight range resolution
+/- Post-processing
- High data volume
Early
demonstrations
Specialized systems: (e.g. ALIRT, JIGSAW, HALOE,
FALCON-I)
Cameras becoming available
+ Long-range: 3 – 15 km
+ Country-sized collection rates
- Accurate scanning
103 - 105× more detectors
100
× le
ss
ligh
t
Presentation Name - 27
Author Initials MM/DD/YY
Area Collection Rate Comparison
5000 ft
20,000 ft 390 km2/hr
2000 km2/hr
Area Collection Rate (ACR): Npixels (photons detected / pixel / pulse) (pulses / sec)
(avg number of detections req. / resolution element)
ACR = area
resolution element
(1000 Detected
photons required)
10 - 15 Detected
photons required
Linear-Mode 3-D Laser Radar
Photon-Counting 3-D Laser Radar
UNCLASSIFIED
UNCLASSIFIED
Presentation Name - 28
Author Initials MM/DD/YY
Conclusions
• Airborne laser radars can rapidly collect human-scale 3-D maps
– Wide-area maps of terrain and urban areas
– Foliage poke-thru
• Lincoln-developed arrays of photon-counting detectors are enabling a new generation of highly capable ladar systems
– Photon-counting = light efficiency = reduced size/weight/power
– Larger arrays = higher measurement rates = reduced operating costs
– High ACR systems must fly higher, thereby requiring photon-counting technology
• Large arrays of Geiger-Mode APDs have been field-proven
Presentation Name - 29
Author Initials MM/DD/YY
Backup
UNCLASSIFIED
UNCLASSIFIED
Presentation Name - 30
Author Initials MM/DD/YY
Geiger-Mode Imager: Single-Photon Detection
• Single-photon sensitivity
• Noiseless readout
• High-bandwidth timing
• Scalable to large arrays
Avalanche
Photodiode
(APD)
Digital
Timing
Circuit
Digitally
Encoded
Photon
Arrival Time
Pixel Circuit
(for each of 4096 pixels)
32 x 128
Photon
APD Device Profile
Polyimide
Passivation InP
Substrate
(250 mm)
InGaAsP
Absorber
InP
Multi-
plier Incident
Photons
Carrier
Generation Avalanche
Multiplication
4 mm
25 mm
Vo
ltag
e
Armed
Time
Multi-Volt Drop
Can Trigger
CMOS Logic
Vbr
Vbr + 3V
Photon
Presentation Name - 31
Author Initials MM/DD/YY
Airborne Lidar Performance Metrics
Performance Metric Desired Capability Limited by…
Operating Altitude Large standoff Power-aperture
Detector sensitivity
Range resolution Distinguish nearby surfaces Pulse width,
Detector timing jitter
Range precision Precise mensuration SNR
Angular resolution Human-scale imagery from stand-
off altitudes
Optical design,
Detector crosstalk
Absolute geolocation Geometric change detection GPS solution
Precision scan system
Area collection rate Collection of large areas in least
possible time (tactical, country-
sized maps
Power-aperture
Detector array size and sensitivity
measurement rate
Size, Weight and Power
(SWaP)
Lowest possible Tradeoff between SWaP and
capability
Onboard Processing Real time or near real time Sensor data rate, SWaP
Presentation Name - 32
Author Initials MM/DD/YY
Two Enabling Technologies Developed at MIT Lincoln Laboratory
1.06-µm, >15-kHz, <0.5-nsec package
Integrated µlens/APD/CMOS-timing Focal Plane Arrays
High-rep-rate, diffraction limited,
passively q-switched, solid-state
microlasers
What’s enabled is high-speed, accurate, 3D imagery collections
32x32x100-µm
2005
32x128x50-µm
2009
64x256x50-µm
2012
Presentation Name - 33
Author Initials MM/DD/YY
3D Ladar Principles: System Components
Ladar Sensor GPS / IMU
Receiver
Optics
Focal plane
array of timers
Pulsed
Laser
Processing
Angle
Encoders
Geolocated 3D Image
Position &
orientation
of telescope
Elapsed time
(range) for
each pixel
Presentation Name - 34
Author Initials MM/DD/YY
Imaging through foliage: How it Works
UNCLASSIFIED // FOUO
• Airborne 3-D ladar images
an obscured region from a
variety of look angles
• Each look produces an
obscured 3-D image of target
• Images from each look angle
are combined
– Holes will “sweep across”
target, filling it in
Presentation Name - 35
Author Initials MM/DD/YY
ALIRT Collection Logistics (Airborne Ladar Imaging Research Testbed)
Processing Center
Final Processing and
Data Delivery
ALIRT Staging Area
Haiti
Miami
Turks and Caicos
Provodenciales
1,100 km 350 km
USSOUTHCOM
Earthquake struck on 12 January 2010, 1643 hrs
Request from SOUTHCOM for participation: 15 January
First imagery available: 18 January
Final flight: 11 February
Presentation Name - 36
Author Initials MM/DD/YY
Damage Assessment National Palace of the Republic of Haiti
High Resolution 3D Target Imaging provides
exquisite 3D data over small areas of interest.
Horizontal Spacing = 15 cm
Damaged
Structures
Perimeter
Fence Normal Roof
Shape
Damage
to Roof
Presentation Name - 37
Author Initials MM/DD/YY
Short-Term Situational Awareness Needs
• Looking for signs of mass migration
– > 750,000 people displaced
– Need to coordinate food and aid distribution
• Ascertaining status of critical infrastructure
– Determining route mobility for relief supply distribution
• Finding helicopter landing zones
• Detecting criminal activity
Presentation Name - 38
Author Initials MM/DD/YY
Maricopa Point, Grand Canyon
Height in Meters
2,250
1,800
1,080
Speed and accuracy: hundreds of km2/hr; geolocated to a basketball-sized error
Presentation Name - 39
Author Initials MM/DD/YY
FALCON-I Foliage Penetration System
Altitude:
20 kft / 6 km
Ground Speed:
80 m/s (170 kts) Image human activity layer under
90% obscuration:
- MACHETE ladar resolution: 25 - 50 cm
- Collect ellipse with area ~ 1.5 km2
Each ground point sampled from
NLOS = 10 angles / orbit
Orbit period: 150 sec
Across-track field of regard: ±46°
Ground spatial resolution: 25 cm
Product post spacing: 25 - 50 cm,
depending on obscuration
0.25 m
0.25 m
Ladar Resolution
Cell
Orbit Radius:
3 km
~2 km
Applications: - Detect structures / trails / open areas
- Target classification / identification
- Generation of bare-earth maps
- Ingress / egress route planning
0.2 m
Presentation Name - 40
Author Initials MM/DD/YY
Geiger-Mode APD Array and Headboard
Headboard Controls Array Function
and Reads Out Data
Flex Print Connects Array With
Headboard
32 x 128 APD Detector Array Straight-On
View of Array