Real-Time Phase-Stamp Range Finder with Improved Accuracy
description
Transcript of Real-Time Phase-Stamp Range Finder with Improved Accuracy
Real-Time Phase-Stamp Range Finder with Improved Accuracy
Akira KimachiOsaka Electro-Communication University
Neyagawa, Osaka 572-8530, Japan
1August 2, 2009 Optical Engineering + Applications, San Diego Convention Center
Outline
• Introduction• Phase-stamp range finder (PSRF)
– Time-domain correlation image sensor (CIS)– Phase-stamp imaging (PSI)– Problem of artifacts
• Accuracy improvement by calibrating the CIS for PSI– Experiment #1: CIS output behavior in PSI– CIS output model for PSI– Compensation for phase stamp errors– Experiment #2: accuracy evaluation
• Conclusions
2
Real-time range imaging
• Demands– Assembly/inspection of industrial products– Environment recognition for robots/vehicles– Observation of mobile objects– Assistance in human workspace
• Active vs. passive range finders– Active methods are preferable in terms of accuracy/reliability
3
# of sensors
Active illumination
Accuracy/reliability
Methods
Active 1 Yes High
Light sectioningStructured lightTime-of-flightTime-stampPhase-stamp
Passive > 1 No LowBinocular stereoMultiple-camera
Real-time active range finders• Robustness vs. depth resolution
– Difficult to establish both
4
Method SensorDepth
resolution
Robustness to disturbance
Ambient illuminationSurface textureTemporal
variationSpatial
variation
Light sectioning
High-speed camera
< 1 mm Low Low Low
VLSI sensor < 1 mm Low High Low
Structured lightHigh-speed camera
< 1 mm Low High High
Time-of-flight (TOF)
VLSI sensor > 1 mm High High High
Axi-Vision Camera
> 1 mm High High High
Time-stamp VLSI sensor < 1 mm Low Low Low
Phase-stampCorrelation image sensor
< 1 mm High High High
flat board @ 400 mm
Objective
• Phase-stamp range finder (PSRF)Kimachi and Ando, Electronic Imaging (2007)
– Time-domain correlation image sensor (CIS)– Frame-rate 3D capture based on “phase stamp” imaging (PSI)
• Problem of artifacts
• Solution– Analyze the behavior of CIS outputs in PSI– Model the CIS outputs for PSI– Calibrate the PSRF by compensating for CIS output errors
5
undulation
~ 3.5 mm rmsrandomnoisepattern
~ 1 mm rms
Correlation image sensor (CIS)
6
Ando and Kimachi, Trans. IEEE ED (2003)
Output images
Average intensity
200x200-pixel CMOS camera
Temporal correlation
: frame integral
frame
Phase-stamp imaging (PSI)
7
Light pulseenergy image
Phase stampimage
Correlation images
uncorrelated
noiseHigher temporal resolutionthan the frame period T
Phase-stamp range finder (PSRF)
8
SOLenergy
ambientillumination
surfacereflectance
(SOL)
Incident lightintensity
Referencesignals
SOLangle
Phasestamp
Correlation images
Range image
: pixel spacing in x
•One SOL scan in one frame•Based on PSI
Kimachi and Ando,Electronic Imaging (2007)
•Range image from single frame •Ambient illumination removed•Surface reflectane canceled
( )ijB tijR
ijz ( , )kQ i j
Experimental PSRF system
9
CIS 200x200-pixel
Frame rate 12.5 fps
Reference signal frequency
50 Hz
Mirror scanning rate
25 Hz
Laser DPSS , 40 mW , 658 nm
Camera lens 25 mm F1.4
PSRF artifacts
• Artifacts in range images– Undulation– Random noise pattern
• Considered to be caused by errors in detected phase stamps
• Temporal correlations may not follow the ideal characteristics with respect to SOL incidence time
10
ijz
ij
( , )kQ i jijt
phase stamp range mapaverage intensity SOL energy
(flat board @ 400 mm)
0
/2
/2
Investigating CIS outputs in PSI
• Capture a sequence of correlation images while shifting pulse occurrence time
11
( , , )kQ i j 0 50 Hz ( 20 ms)f T
Pulse occurrence time 0~20 ms (1 ms step)
Pulse height 0~8, 9 levels
Pulse width 0.2~2 ms (0.2 ms step)
Reference signal amplitude 0.05~0.5 V (0.05 step)0V
ht
0t0t
Results: image average behavior
12
Distortions in from sine functions become severer as increases( )kQ 0V
Image averages of temporal correlations increase monotonicallywith , , and
( )kQ 0Vt h
( , , )kQ i j
Pulse width variedt Pulse height variedh Amplitude varied0V
Undulations in the phase stamp become severer as increases0V
Results: pixel-wise deviation behavior
13
0
/2
/2
Pulse width variedt Pulse height variedh Amplitude varied0V
Pixel-wise deviations of temporal correlations increasemonotonically as , , and 0Vt h
( , , )k i j ( , , )kQ i j
Pixel-wise deviations of computed phase stamps oscillate with 0tij
( , ) (100,100)i j
oscillate with in arbitrary waveforms, randomly with respect to pixel and channel
( , , )k i j 0tk( , )i j
• Ideal characteristic of CIS temporal correlation outputs
• Experiment-based CIS output model for PSI
– Undulation/distortion→ Harmonics , ,
– Pixel-/channel-wise random deviation→ Coefficients , , , ,
– Dependence on light pulse energy → Multiplication by
• Coefficients are estimated from a sequence of PSI images by least-squares fitting
CIS output model for PSI
14
( , )ka i j ( , )kb i j ( , )kc i j ( , )kd i j ( , )ke i j
sin k cos 2 k sin 2 k
,ijA t h
noise
(for fixed )0V
Compensation for phase stamp errorsFor a single-frame set of temporal correlation images
, , and ,
1. Compute the phase stamps
2. Approximate the PSI CIS model by regarding as containing a small error
3. Estimate and pixel-wise by least-squares fitting to the model with and the pre-estimated coefficients , , , ,
4. Obtain the phase stamp estimates15
1( , )Q i j 3( , )Q i j
( , )kQ i j
2 ( , )Q i j
ij1ij
ijij
ij ijA
( , )ka i j ( , )kb i j ( , )kc i j ( , )kd i j ( , )ke i j
Results: CIS calibration for PSI (1)
16
( , ) (100,108)i j
test data = calibration data
•Fitting — over a sequence of •Compensation — on a single frame of
( , , )kQ i j kQ
Results: CIS calibration for PSI (2)
17
108j
test data = calibration data
0 4 mst
0 10 mst
0 17 mst
test data ≠ calibration data
Results: CIS calibration in PSI (3)
18
test data ≠ calibration data
Before compensation
After compensation
Results: artifacts removal in PSRF
19
SOL intensity phase stamp
@ 400 mm
compensateduncompensated
flat board
paper box
can bottle
Compensation Before After
Offsetphase [rad] −2.50 0.00
range [mm] 5.37 0.70
Undulation (rms)
phase [rad] 7.44 1.16
range [mm] 3.45 0.48
Random noise pattern (rms)
phase [rad] 1.81 0.92
range [mm] 1.09 0.55
Conclusions• Artifacts in PSRF outputs have been removed
– CIS outputs were modeled based on PSI experiments– A method for compensating for phase stamp errors in CIS
outputs was proposed– Confirmed in experiments
• Accuracy improved on a PSRF system– Undulation — 3.45 mm → 0.48 mm– Random noise pattern — 1.09 mm → 0.55 mm
20