Post on 24-Dec-2015
Outline Outline
• 3D Reconstruction Based on Range Images
• Color Engineering
• Thermal Image Restoration
3D – Overview 3D – Overview To reconstruct the geometry and texture of a scene in a virtual environment.
--- 3D scanning
Create an arbitrary view by interpolation
Method 1: 2D color-image-based reconstruction
3D – Overview3D – OverviewMethod 2: Range-image-based reconstruction
Range image -----Depth
Intensity image --Reflectance
CCD Image--RGB
3D – Overview3D – Overview Range Image Intensity Image CCD Image
A digital model
Geometric Structure Materials Texture
3D – Overview3D – Overview
Preprocessing
Registration
Mesh
Texture Mapping
Lighting
Shading …..
System architecture
Raw Data (3D coordinates, Intensity, RGB)
Visual Information (Geometry, Texture)
Knowledge from Object Recognition
Modeling
3D – Range Image Registration 3D – Range Image Registration
Image of a left view
Images registered
Image of a right view
Objective
3D – Range Image Registration3D – Range Image Registration
Scheme Range image I1 Range image I2
Description based on geometric features and their interrelationships
……
Construct feature correspondence M and measure the Similarity S between the two images ------ Find the M maximizing S
Feature Extraction: surface, curve, corner point ……
Noise filtering, outlier removing ……
3D – Range Image Registration3D – Range Image RegistrationNoise filtering, outlier removing
Before After
• Polar window filtering,
• Pseudo-median filtering
• Isolated point filtering
3D – Range Image Registration3D – Range Image RegistrationFeature extraction
Surface: adaptive-shape window
Curve, corners: edge evolution
3D – Range Image Registration3D – Range Image RegistrationDescriptions of the geometric features
Interrelationship contained in nested geometric features
Related geometric features are nested
3D – Range Image Registration3D – Range Image RegistrationCorrespondence and similarity measure
Virtual features
Virtual features
3D – Range Image Registration3D – Range Image Registration
Improvement of the registration results
Before After
Global optimization
Color Engineering Color Engineering
-60
-30
0
30
60
90
-90-60
-300
3060
901
1.05
1.1
1.15
1.2
1.25
1.3
1.35
1.4
• Proposed a method for measuring luminance distribution of indoor scenes using a digital camera rather than an expensive luminance meter.
• Proposed a novel radiometric model for CCD sensors and a color self-calibration algorithm based on this model. The objective of this project is to calibrate the color performance of 100 CCDs in a lightfield-rendering system for 3D scene reconstruction.
CCD 1 Fake CCD 2 CCD2
Transformed to be To approximate
Color Engineering Color Engineering • Calibrated a line CCD sensor with poor color performance. The radiometric correlation between r, g, b channels is considered.
11 12 13 14
21 22 23 24
31 32 33 34 1
rr m m m m
gMin g m m m m
bb m m m m
12 13 21 23 31 32, , , , , 0m m m m m m
Thermal Imaging Thermal Imaging •Proposed a radiometric model for infrared cameras and developed relevant model reconstruction methods, which resulted in obtaining a very precise forward function for thermal image restoration. Regularization techniques, such as Tikhonov, Total Variation, and Lasso, were applied to the restoration procedure and their performances were compared.
Thermal camera model
Thermal ImagingThermal ImagingThermal image restoration
Original image Image restored by Tikhonov regularization, Edges are strongly penalized
Image restored by Discontinuity-Adaptive model regularization, Edges are adaptively penalized
Noise is suppressed
Convexity of energy function is well controlled.
Image restored by Total Variation regularization, Edges are preserved
Thermal ImagingThermal Imaging• With the adjustment of the camera setting, the point spread function (PSF) of the camera system can be changed. Therefore we try to develop a semi-blind image restoration algorithm that can recover the original image and the PSF simultaneously.
Iteration 1 Iteration 1
(a)
Original image Original image
Iteration 1 Iteration 1
Iteration 2 Iteration 2
Iteration 3 Iteration 3
Iteration 4 Iteration 4
Iteration 5 Iteration 5
Iteration 6 Iteration 6
Iteration 7 Iteration 7
Iteration 8 Iteration 8
Iteration 9 Final restoration results
Iteration 9 Final restoration results