Post on 17-Dec-2015
Automatic Image Rescaling Preserving Design Intention
Research Update
Prasad Gabbur
Goal
Simple scaling
Original:
Scaling that preserves the design intention of the objects:
Approach
• Split input image into background (bg) and foreground (fg) layers
• Scale the bg and fg layers separately
Background
Foreground
Background layer scaling
• Scale and shift elements to fit new page
• Ignore aspect ratio
• Classification of background elements
Area Horizontal
(Top, Bottom)
Vertical
(Left, right)
Background layer scaling• Scale and shift of background elements
Scale
Scale + Shift
Scale
+Shift
Sx > Sy Sy > Sx
Area
Horizontal
Vertical
To fit new page
To fit new page width
To fit new page height
Foreground layer scaling
• Scale and shift elements to fit new page
• Preserve aspect ratio
• Classification of foreground elements
Corner
(TL, TR, BL, BR)
Horizontal
(Top, Bottom)
Vertical
(Left, Right)
Foreground layer scaling
Scale
+ Shift
Scale
+ Shift
Scale
+ Shift
Corner
Horizontal
Vertical
Sx > Sy Sy > Sx
Foreground layer scaling
• Scaling preserves aspect ratio (Scale factor = min (Sx, Sy))• Shifting preserves distance ratio (dL/dR = const, dT/dB = const)
dL_old dR_old
dB_old
dT_olddT_new
dB_new
dL_new dR_new
Original Sx > Sy
(dL_new / dR_new)
= (dL_old / dR_old)
Sy > Sx
(dT_new / dB_new)
= (dT_old / dB_old)
HorizontalVertical
Shift of horizontal elements
Shift of vertical elements
Element extraction
• Elements are connected regions in the foreground or background layer
Multilayer image
Background layer(s)
Foreground layer(s)
Extract
alpha
channel
Label
connected
components
Connected component labeling
• Group together spatially connected pixels as a single component
• Each component is assigned a unique integer label
4-connectivity 8-connectivity
Connected component labeling• Region coloring algorithm (4-connected) [Ballard & Brown, 1982]
– Each pixel (Xc) in the image is scanned with the following mask:
XcXl
Xu
new_label = 1
If (Xu Є background & Xl Є foreground), then
label (Xc) = label (Xl)
Else if (Xu Є foreground & Xl Є background), then
label (Xc) = label (Xu)
Else if (Xu Є foreground & Xl Є foreground), then
label (Xc) = min ( label (Xl), label (Xu) )
Else
label (Xc) = new_label
new_label = new_label + 1
Connected component labeling
• Basic region coloring algorithm is slow• Requires multiple passes through the image• A faster version is realized with the help of a
custom data structure
• An array of the above data type can store information about all connected components
• Only one pass through the image is necessary
Connected component labeling
• A single raster scan of the image gives rise to following structure
……………………
Image with two connected components
Data structure at the end of a single image scan
Connected component labeling• Links in the array can be visualized as a tree structure
• Nodes in the tree are equivalent labels of a connected component
1 1 2
3 4
11 12 5 10
6 7 8 9
A fictitious connected component
Connected component labeling
• Trees with different configurations are possible depending on region complexity
• All branches merge at the bottom
One branch Two branches Four branches
Connected component labeling
• Resolving label equivalences
• All the equivalent labels are assigned the least value among them by stepping through the tree
1
1
1
1
1
1
1
1
1
1
1
1
1
2
3
4
5
6
7
8
9
10
11
12
Resolve
Connected component labeling
1 1 2
3 4
11 12 5 10
6 7 8 9
• Resolving label equivalences
1 1 1
1 1
1 1 1 1
1 1 1 1
Resolve
Elements
• Each connected component in the background or foreground layer is an element
• Geometric properties (bounding box center and limits) are computed as part of the labeling process
• Elements are classified based on the geometric properties
Background layer Foreground layer
Scaling issues*
• Scale up– Sparse distribution of pixels in the output
image– Bilinear interpolation to fill in pixel values
• Scale down– Aliasing due to sub-sampling– Low pass filtering before sub-sampling
* Thanks to Jian Fan, HP Labs.
Results• Background layer
Original
Labeled
Scaled (Sx >S y)
Scaled (Sy >S x)
Results• Foreground layer
Original
Labeled
Scaled (Sx >S y)
Scaled (Sy >S x)
Next
• Stitch together the foreground and background layers
• Work on an XML design for input
Thank you!