from Color Management to Omnidirectional...
56
Efficient Regression for Computational Imaging: from Color Management to Omnidirectional Superresolution Maya R. Gupta Eric Garcia Raman Arora
Transcript of from Color Management to Omnidirectional...
Efficient Regression for Computational Imaging:
from Color Management to Omnidirectional Superresolution
Maya R. Gupta
Eric Garcia
Raman Arora
Regression
2
Regression
Regression
Linear Regression: fast, not good enough
Problem: Device Dependent Colors Depend on Device
Color Management For each device, characterize the mapping between the native
color space and a device independent color space.
8/5/2009 7
CIELab (Lab)
ICC Profile
ICC Profile
ICC Profile
ICC Profile
Color Management • For each device, characterize the mapping between the native
color space and a device independent color space.
8/5/2009 8
CIELab (Lab)
ICC Profile
ICC Profile
ICC Profile
ICC Profile
CIELab is a widely used device-independent color space that is
perceptually uniform (i.e. Euclidean distance approximates human
judgement of color dissimilarity)
Color Management • For each device, characterize the mapping between the native
color space and a device independent color space.
8/5/2009 9
CIELab (Lab)
ICC Profile
ICC Profile
ICC Profile
ICC Profile
Mapping from RGB -> CIELab and CIELab -> CMYK can be highly
nonlinear
Gamut mapping: linear transforms not adequate
Skin
tones Skin
tones
Original gamut
Extended gamut
Original Gamut Linear regression Nonlinear regression
Creating Custom Color Enhancements
original transformed by artist to “sunset”
2 hrs. work in Photoshop
Ex: simulating illumination effects
Example Convert an image to how it would look in Cinecolor based on 16 sample color pairs
www.widescreenmuseum.org
Original cinecolor
Color management: speed by LUT
8/5/2009 13
Color management: speed by LUT
8/5/2009 14
Color management: speed by LUT
15
Color management: speed by LUT
Color management: speed by LUT
Color management: speed by LUT
Color management: speed by LUT
Linear Interpolation is linear in the outputs
Linear Interpolation is linear in the outputs
Linear Interpolation is linear in the outputs
Lattice Regression Choose the lattice outputs to minimize the post-linear
interpolation empirical risk on the data:
8/5/2009 23
Lattice Regression Choose the lattice outputs to minimize the post-linear
interpolation empirical risk on the data:
8/5/2009 24
Lattice Regression Choose the lattice outputs to minimize the post-linear
interpolation empirical risk on the data:
Effect of Different Lattice Regression Regularizers
8/5/2009 26
Effect of Different Lattice Regression Regularizers
8/5/2009 27
Lattice Regression Closed Form Solution
9/15/2010 28
Sparse: No more than 7dm non-zero entries (of m2) with cubic interpolation.
Example Color Management Results
Example Color Management Results
9/15/2010 31
Omnidirectional Super-resolution:
Omnidirectional Superres Related Work
State of the Art:
Arican and Frossard 2008-2009 (ICPR 2008 Best Paper Award)
• Interpolation with spherical harmonics
• Alignment with an iterative conjugate gradient approach.
Lattice Regression Approach Finding the correct registration of the low-resolution images is
challenging non-convex optimization problem.
Evaluate a candidate registration:
use lattice regression on image subset -> high-res spherical grid
sum interpolation error for all left-out low res image data
Lattice Regression Approach Finding the correct registration of the low-resolution images is
challenging non-convex optimization problem.
Evaluate a candidate registration:
use lattice regression on image subset -> high-res spherical grid
sum interpolation error for all left-out low res image data
Finding the optimal joint registration is a 3(N-1)-d opt. problem
We use FIPS to find the global optimum.
9/15/2010 36
Visual Homing
START
.
. .
HOME
. .
.
.
Lattice Regression Better For Visual Homing
Some Conclusions
Some Conclusions
Some Conclusions
Some Conclusions
For details, see: • “Optimized Regression for Efficient Function Evaluation,” Eric K. Garcia,
Raman Arora, and Maya R. Gupta, (in review – draft upon request).
• “Lattice Regression”, Eric K. Garcia, Maya R. Gupta, Neural Information Processing Systems (NIPS) 2009.
• “Building Accurate and Smooth ICC Profiles by Lattice Regression,” Eric K. Garcia, Maya R. Gupta, 17th IS&T Color Imaging Conference 2009.
• "Adaptive Local Linear Regression with Application to Printer Color Management," Maya R. Gupta, Eric K. Garcia, and Erika Chin, IEEE Trans. on Image Processing , vol. 17, no. 6, 936-945, 2008.
• "Learning Custom Color Transformations with Adaptive Neighborhoods," Maya R. Gupta, Eric K. Garcia, and Andrey Stroilov, Journal of Electronic Imaging, vol. 17, no. 3, 2008.
• "Gamut Expansion for Video and Image Sets," Hyrum Anderson, Eric K. Garcia, and Maya R. Gupta, Computational Color Imaging Workshop, 2007.
8/5/2009 42
Color is an event
light source
human
cones respond:
human
perceives
color
L = long wave = red
M = medium wave = green
S = short wave = blue
reflection
What does it mean to see black?
light source
human
cones respond
???
human
perceives
color
L = long wave = red
M = medium wave = green
S = short wave = blue
What does it mean to see white?
light source
human
cones respond
???
human
perceives
color
L = long wave = red
M = medium wave = green
S = short wave = blue
What does it mean to see white? images from: www.omatrix.com/uscolors.html
You can see “white” given
light made up of 2-spectra
Color Science Crash Course
• What we see can be represented by three primaries.
8/5/2009 47
Stiles-Burch 10° color matching
functions averaged across 37
observers . Adapted from (Wyszecki
& Stiles, 1982) by handprint.com.
monochromatic
light at some
wavelength
match
mixture of three
primary colors
Color Distances
• CIELab • Based on spectral
measurements of color, integrated over CMF envelopes.
• Euclidean distance between two colors approximates the perceptual difference noticed by a human observer.
• Distance metrics created to correct for perceptual non- uniformities in the space:
8/5/2009 48
image source: www.handprint.com
2-D and 3-D Simulation
8/5/2009 49
d=2
d=3
Color printer
8 bit RGB color patch printed
color patch Human eye
Measure CIEL*a*b*
Color management for printers
Goal: Print a given CIEL*a*b* value. Problem: What RGB value to input?
Inverse Device Characterization
8/5/2009 51
CIELab
Step 1 Sample the device
Step 2 Build an inverse look-up-table
Regression
Look-up-table
Output Measure
8/5/2009 52
Gaussian Process Regression • Models data as being drawn from a Gaussian Process
8/5/2009 53
(L large, σ2 small) (L small, σ2 small) (L large, σ2 large)
• A leading method in geostatistics (2-d regression) also known as Kriging.
• Generally considered a state-of-the-art method by machine learning folks
• Parameters: Covariance Function (length scale L), Noise Power σ2.
Gaussian Process Regression • Models data as being drawn from a Gaussian Process
8/5/2009 54
(L large, σ2 small) (L small, σ2 small) (L large, σ2 large)
• A leading method in geostatistics (2-d regression) also known as Kriging.
• Generally considered a state-of-the-art method by machine learning folks
• Parameters: Covariance Function (length scale L), Noise Power σ2.
• Given Covariance form, parameters can be learned by maximizing marginal likelihood. (i.e. automatically from data).
2-D Simulation
8/5/2009 55
Gaussian Process Regression (Direct) Gaussian Process Regression (to nodes of lattice) Lattice Regression (GPR bias) Lattice Regression (Bilinear bias)
50 Training Samples 1000 Training Samples
3-D Simulation
8/5/2009 56
Gaussian Process Regression (Direct) Gaussian Process Regression (to nodes of lattice) Lattice Regression (GPR bias) Lattice Regression (Bilinear bias)
50 Training Samples 1000 Training Samples
