Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 ©...
-
Upload
fabian-crofton -
Category
Documents
-
view
217 -
download
1
Transcript of Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 ©...
![Page 1: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/1.jpg)
Image Blending and Compositing
15-463: Computational PhotographyAlexei Efros, CMU, Fall 2010
© NASA
![Page 2: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/2.jpg)
Image Compositing
![Page 3: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/3.jpg)
Compositing Procedure1. Extract Sprites (e.g using Intelligent Scissors in Photoshop)
Composite by David Dewey
2. Blend them into the composite (in the right order)
![Page 4: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/4.jpg)
Need blending
![Page 5: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/5.jpg)
Alpha Blending / Feathering
01
01
+
=Iblend = Ileft + (1-)Iright
![Page 6: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/6.jpg)
Affect of Window Size
0
1 left
right0
1
![Page 7: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/7.jpg)
Affect of Window Size
0
1
0
1
![Page 8: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/8.jpg)
Good Window Size
0
1
“Optimal” Window: smooth but not ghosted
![Page 9: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/9.jpg)
What is the Optimal Window?
To avoid seams• window = size of largest prominent feature
To avoid ghosting• window <= 2*size of smallest prominent feature
Natural to cast this in the Fourier domain• largest frequency <= 2*size of smallest frequency• image frequency content should occupy one “octave” (power of two)
FFT
![Page 10: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/10.jpg)
What if the Frequency Spread is Wide
Idea (Burt and Adelson)• Compute Fleft = FFT(Ileft), Fright = FFT(Iright)
• Decompose Fourier image into octaves (bands)– Fleft = Fleft
1 + Fleft2 + …
• Feather corresponding octaves Flefti with Fright
i
– Can compute inverse FFT and feather in spatial domain
• Sum feathered octave images in frequency domain
Better implemented in spatial domain
FFT
![Page 11: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/11.jpg)
Octaves in the Spatial Domain
Bandpass Images
Lowpass Images
![Page 12: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/12.jpg)
Pyramid Blending
0
1
0
1
0
1
Left pyramid Right pyramidblend
![Page 13: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/13.jpg)
Pyramid Blending
![Page 14: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/14.jpg)
laplacianlevel
4
laplacianlevel
2
laplacianlevel
0
left pyramid right pyramid blended pyramid
![Page 15: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/15.jpg)
Laplacian Pyramid: Blending
General Approach:1. Build Laplacian pyramids LA and LB from images A and B
2. Build a Gaussian pyramid GR from selected region R
3. Form a combined pyramid LS from LA and LB using nodes of GR as weights:• LS(i,j) = GR(I,j,)*LA(I,j) + (1-GR(I,j))*LB(I,j)
4. Collapse the LS pyramid to get the final blended image
![Page 16: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/16.jpg)
Blending Regions
![Page 17: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/17.jpg)
Horror Photo
© david dmartin (Boston College)
![Page 18: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/18.jpg)
Results from this class (fall 2005)
© Chris Cameron
![Page 19: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/19.jpg)
Season Blending (St. Petersburg)
![Page 20: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/20.jpg)
Season Blending (St. Petersburg)
![Page 21: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/21.jpg)
Simplification: Two-band Blending
Brown & Lowe, 2003• Only use two bands: high freq. and low freq.• Blends low freq. smoothly• Blend high freq. with no smoothing: use binary alpha
![Page 22: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/22.jpg)
Low frequency ( > 2 pixels)
High frequency ( < 2 pixels)
2-band Blending
![Page 23: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/23.jpg)
Linear Blending
![Page 24: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/24.jpg)
2-band Blending
![Page 25: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/25.jpg)
Don’t blend, CUT!
So far we only tried to blend between two images. What about finding an optimal seam?
Moving objects become ghosts
![Page 26: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/26.jpg)
Davis, 1998
Segment the mosaic• Single source image per segment• Avoid artifacts along boundries
– Dijkstra’s algorithm
![Page 27: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/27.jpg)
min. error boundary
Minimal error boundary
overlapping blocks vertical boundary
__ ==22
overlap error
![Page 28: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/28.jpg)
Seam Carving
http://www.youtube.com/watch?v=6NcIJXTlugc
![Page 29: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/29.jpg)
Graphcuts
What if we want similar “cut-where-things-agree” idea, but for closed regions?• Dynamic programming can’t handle loops
![Page 30: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/30.jpg)
Graph cuts – a more general solution
n-links
s
t a cuthard constraint
hard constraint
Minimum cost cut can be computed in polynomial time
(max-flow/min-cut algorithms)
![Page 31: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/31.jpg)
Kwatra et al, 2003
Actually, for this example, DP will work just as well…
![Page 32: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/32.jpg)
Lazy Snapping
Interactive segmentation using graphcuts
![Page 33: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/33.jpg)
Gradient Domain
In Pyramid Blending, we decomposed our image into 2nd derivatives (Laplacian) and a low-res image
Let us now look at 1st derivatives (gradients):• No need for low-res image
– captures everything (up to a constant)
• Idea: – Differentiate
– Blend / edit / whatever
– Reintegrate
![Page 34: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/34.jpg)
Gradient Domain blending (1D)
Twosignals
Regularblending
Blendingderivatives
bright
dark
![Page 35: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/35.jpg)
Gradient Domain Blending (2D)
Trickier in 2D:• Take partial derivatives dx and dy (the gradient field)• Fidle around with them (smooth, blend, feather, etc)• Reintegrate
– But now integral(dx) might not equal integral(dy)
• Find the most agreeable solution– Equivalent to solving Poisson equation
– Can be done using least-squares
![Page 36: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/36.jpg)
Perez et al., 2003
![Page 37: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/37.jpg)
Perez et al, 2003
Limitations:• Can’t do contrast reversal (gray on black -> gray on white)• Colored backgrounds “bleed through”• Images need to be very well aligned
editing
![Page 38: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/38.jpg)
Gradients vs. Pixels
Can we use this for range compression?
![Page 39: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/39.jpg)
Thinking in Gradient Domain
Our very own Jim McCann::
James McCann Real-Time Gradient-Domain Painting, SIGGRAPH 2009
![Page 40: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/40.jpg)
Gradient Domain as Image Representation
See GradientShop paper as good example:
http://www.gradientshop.com/
![Page 41: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/41.jpg)
Can be used to exert high-level control over images
![Page 42: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/42.jpg)
Can be used to exert high-level control over imagesgradients – low level image-features
![Page 43: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/43.jpg)
Can be used to exert high-level control over imagesgradients – low level image-features
+100
pixelgradien
t
![Page 44: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/44.jpg)
Can be used to exert high-level control over imagesgradients – low level image-featuresgradients – give rise to high level image-
features
+100
pixelgradien
t
![Page 45: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/45.jpg)
Can be used to exert high-level control over imagesgradients – low level image-featuresgradients – give rise to high level image-
features
+100+100+100+100+100
+100
pixelgradien
t
+100+100+100+100+100
+100
pixelgradien
t
![Page 46: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/46.jpg)
Can be used to exert high-level control over imagesgradients – low level image-featuresgradients – give rise to high level image-
features
+100+100+100+100+100
+100
pixelgradien
t
+100+100+100+100+100
+100
pixelgradien
t
image edge
image edge
![Page 47: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/47.jpg)
Can be used to exert high-level control over imagesgradients – low level image-featuresgradients – give rise to high level image-
featuresmanipulate local gradients to
manipulate global image interpretation+100+100+100+100+100
+100
pixelgradien
t
+100+100+100+100+100
+100
pixelgradien
t
![Page 48: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/48.jpg)
Can be used to exert high-level control over imagesgradients – low level image-featuresgradients – give rise to high level image-
featuresmanipulate local gradients to
manipulate global image interpretation+255+255+255+255+255
pixelgradien
t
![Page 49: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/49.jpg)
Can be used to exert high-level control over imagesgradients – low level image-featuresgradients – give rise to high level image-
featuresmanipulate local gradients to
manipulate global image interpretation+255+255+255+255+255
pixelgradien
t
![Page 50: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/50.jpg)
Can be used to exert high-level control over imagesgradients – low level image-featuresgradients – give rise to high level image-
featuresmanipulate local gradients to
manipulate global image interpretation+0
+0
+0
+0
+0
pixelgradien
t
![Page 51: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/51.jpg)
Can be used to exert high-level control over imagesgradients – low level image-featuresgradients – give rise to high level image-
featuresmanipulate local gradients to
manipulate global image interpretation+0
+0
+0
+0
+0
pixelgradien
t
![Page 52: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/52.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
features
![Page 53: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/53.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdges
![Page 54: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/54.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdges
object boundariesdepth discontinuitiesshadows…
![Page 55: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/55.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTexture
![Page 56: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/56.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTexture
visual richnesssurface properties
![Page 57: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/57.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShading
![Page 58: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/58.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShading
lighting
![Page 59: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/59.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShading
lightingshape
sculpting the faceusing shading
(makeup)
![Page 60: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/60.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShading
lightingshape
sculpting the faceusing shading
(makeup)
sculpting the faceusing shading
(makeup)
![Page 61: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/61.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShading
lightingshape
sculpting the faceusing shading
(makeup)
sculpting the faceusing shading
(makeup)
sculpting the faceusing shading
(makeup)
![Page 62: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/62.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShading
lightingshape
sculpting the faceusing shading
(makeup)
![Page 63: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/63.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShading
![Page 64: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/64.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShadingArtifacts
![Page 65: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/65.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShadingArtifacts
noise
sensor noise
![Page 66: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/66.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShadingArtifacts
noiseseams
seams incomposite
images
![Page 67: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/67.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShadingArtifacts
noiseseamscompression
artifacts
blocking incompressed
images
![Page 68: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/68.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShadingArtifacts
noiseseamscompression
artifacts
ringing incompressed
images
![Page 69: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/69.jpg)
Can be used to exert high-level control over imagesgradients – give rise to high level image-
featuresEdgesTextureShadingArtifacts
noiseseamscompression
artifactsflicker flicker from
exposure changes & film degradation
![Page 70: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/70.jpg)
Can be used to exert high-level control over images
![Page 71: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/71.jpg)
Optimization framework
Pravin Bhat et al
![Page 72: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/72.jpg)
Optimization frameworkInput unfiltered image – u
![Page 73: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/73.jpg)
Optimization frameworkInput unfiltered image – uOutput filtered image – f
![Page 74: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/74.jpg)
Optimization frameworkInput unfiltered image – uOutput filtered image – fSpecify desired pixel-differences – (gx, gy)
min (fx – gx)2 + (fy – gy)2
f
Energy function
![Page 75: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/75.jpg)
Optimization frameworkInput unfiltered image – uOutput filtered image – fSpecify desired pixel-differences – (gx, gy)Specify desired pixel-values – d
min (fx – gx)2 + (fy – gy)2 + (f – d)2
f
Energy function
![Page 76: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/76.jpg)
Optimization frameworkInput unfiltered image – uOutput filtered image – fSpecify desired pixel-differences – (gx, gy)Specify desired pixel-values – dSpecify constraints weights – (wx, wy, wd)
min wx(fx – gx)2 + wy(fy – gy)2 + wd(f – d)2
f
Energy function
![Page 77: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/77.jpg)
![Page 78: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/78.jpg)
![Page 79: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/79.jpg)
![Page 80: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/80.jpg)
Least Squares ExampleSay we have a set of data points (X1,X1’), (X2,X2’),
(X3,X3’), etc. (e.g. person’s height vs. weight)
We want a nice compact formula (a line) to predict X’s from Xs: Xa + b = X’
We want to find a and b
How many (X,X’) pairs do we need?
What if the data is noisy?
'22
'11
XbaX
XbaX
'2
'1
2
1
1
1
X
X
b
a
X
XAx=B
.........
1
1
1
'3
'2
'1
3
2
1
X
X
X
b
a
X
X
X
overconstrained
2min BAx
![Page 81: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/81.jpg)
Putting it all together
Compositing images• Have a clever blending function
– Feathering
– Center-weighted
– blend different frequencies differently
– Gradient based blending
• Choose the right pixels from each image– Dynamic programming – optimal seams
– Graph-cuts
Now, let’s put it all together:• Interactive Digital Photomontage, 2004 (video)
![Page 82: Image Blending and Compositing 15-463: Computational Photography Alexei Efros, CMU, Fall 2010 © NASA.](https://reader036.fdocuments.us/reader036/viewer/2022062518/56649c915503460f9494c8a4/html5/thumbnails/82.jpg)