CS448f: Image Processing For Photography and Vision Blending and Pyramids.
Image Fusion by Image Blending Arthur Goshtasby
Transcript of Image Fusion by Image Blending Arthur Goshtasby
Image Fusion by Image Blending
Arthur GoshtasbyWright State UniversityWright State University
Image Registration and Fusion Systems
Image fusionImage fusion
Combining relevant information in two orCombining relevant information in two or more images of a scene into a single highly informative imageinformative image.Fusion methods:
L l l ( i l b d)– Low-level (pixel-based)– Mid-level (region-based)– High-level (description-based)
A. Goshtasby 2
Image compositingImage compositing
Cutting and smoothly stitching together partsCutting and smoothly stitching together parts of different images of a scene into a single highly informative imagehighly informative image.
Image 1 Image 2 Combined image
A. Goshtasby 3
Image fusion as an image blendingImage fusion as an image blending problem
A. Goshtasby 4
1 For each local area in the image1. For each local area in the image domain, find the image providing the most informationproviding the most information.
2. Cut out thef
Image domain
area out of the most informative imageimage.
A. Goshtasby 5
Selected local areas in images
3 Blend the local areas to create the fused3. Blend the local areas to create the fused image.
A. Goshtasby 6
Selecting the best-block imageSelecting the best block image
• Highest contrast• Highest entropy• Highest entropy
255
255
0)log(
iiiG ppE Gray-scale
bgrC EEEE Color
A. Goshtasby 7
The blending functionThe blending function
Assuming (x y ) represents the coordinates ofAssuming (xi,yi) represents the coordinates of the center of the ith block, use
G )(
N
j j
ii
yxGyxGyxW
1),(
),(),(
as the blending function, where
)()( 22 yyxx
is adjusted to the size of the local area
.2
)()(exp),( 2
ii
iyyxxyxG
is adjusted to the size of the local area.A. Goshtasby 8
Image blending algorithmImage blending algorithm
Given images {Ik(x,y): k = 1,…,M}, and assumingGiven images {Ik(x,y): k 1,…,M}, and assuming 1) the image domain is subdivided into N
blocks andblocks, and 2) for block i, the most informative image is
I (x y)Ii.k(x,y),the fused image is computed from:
N
ikii yxIyxWyxO
1. ),(),(),(
A. Goshtasby 9
i 1
Characteristics of the algorithmCharacteristics of the algorithm
• Parameter σ is adjusted to block size andParameter σ is adjusted to block size, and block size is chosen based on image size and/or image resolutionand/or image resolution.
• The blending process does not create information in output that does not exist ininformation in output that does not exist in input.Th h d i il i b d f• The method is versatile, it can be used to fuse various types of images.
A. Goshtasby 10
Application 1:Fusion of multi-exposure imagesOptimizing criterion: Entropy
A. Goshtasby 11
Images courtesy of Paul Debevec, USC
Optimizing criterion: Entropy
Original images courtesy of Shree Nayar Columbia University
A. Goshtasby 12
Original images courtesy of Shree Nayar, Columbia University
Optimizing criterion: Entropy
Original images courtesy of Max Lyons
A. Goshtasby 13
Application 2:Fusion of multi-focus images
Original iimages
Optimizing criterion:
Original images courtesy of Image
Fused image
Contrast
A. Goshtasby 14
Original images courtesy of Image Fusion Systems Research
Optimizing criterion: ContrastOptimizing criterion: Contrast
Originalimages
Fusedimageg
A. Goshtasby 15
Concluding remarksConcluding remarks
• If the type of information to be maximized is known,If the type of information to be maximized is known, this method is guaranteed to produce an image with the highest information content at the desired resolution.
• It can be adapted to image resolution by making block size a function of image resolution.
• By changing the optimization criterion, the method can be used to fuse various types of images.
A. Goshtasby 16
Image fusion referencesImage fusion references1. T. Porter and T. Duff, Compositing digital images, ACM SIGGRAPH Computer
Graphics, vol. 18, no. 3, 1984, pp. 253-259.2. P. Willis, Generalized compositing, ACM SIGGRAPH, 2007, pp. 129-135.3. A. Goshtasby, Fusion of multi-exposure images, Image and Vision Computing,
l 23 2005 611 618vol. 23, 2005, pp. 611-618.4. A. Goshtasby, Fusion of multifocus images to maximize image information,
Defense and Security SYmposium, 17-21 April 2006, Orlando, Florida, 2006.5 Special issue of Information Fusion on Image Fusion A Goshtasby and S5. Special issue of Information Fusion on Image Fusion, A. Goshtasby and S.
Nikolov (Eds.), vol. 8, 2007.6. R. S. Blum and Z. Liu, Multi-sensor Image Fusion and its Applications, CRC
Press, 2005. ,7. F. Sroubek and J. Flusser, Registration and fusion of blurred images, Int’l Conf.
Image Analysis and Recognition, 2004, 124-129.8. C. Genderen and J. L. van Pohl, Multisensor image fusion in remote sensing:
Concepts, methods and applications, Int’l J. Remote Sensing, vol. 19, no. 5, 1998, pp. 823-854.
A. Goshtasby17