Unsupervised Automation of Photographic Composition Rules Serene Banerjee and Brian L. Evans
-
Upload
rudolph-powell -
Category
Documents
-
view
217 -
download
0
description
Transcript of Unsupervised Automation of Photographic Composition Rules Serene Banerjee and Brian L. Evans
Unsupervised Automation of Photographic Composition Rules
Serene Banerjee and Brian L. Evanshttp://www.ece.utexas.edu/~bevans/projects/dsc/index.html
Computer Engineering AreaDept. of Electrical and Computer EngineeringThe University of Texas at Austin
1/19/2004 Automation of Composition Rules 2
Motivation Problem: Amateur photographers often take
low-quality pictures with digital still cameras Personal use Professionals who need to document
(e.g.. realtors and architects) Goal: Automate photographic composition rules
and find alternatives to the picture being acquired Analyze scene, including detection of main subject Develop algorithms to automate rules
Main subjectcropped
Too muchbackground
1/19/2004 Automation of Composition Rules 3
Solution Solution #1: Automatically detect main subject
Independent of indoor/outdoor setting or scene Low implementation complexity, fixed-point computation
Solution #2: Automate a few photograph composition rules Rule of thirds for placing the main subject Simulated background blur for motion pictures or depth-of-field
Following rule-of-thirds Blur background for action pictures
1/19/2004 Automation of Composition Rules 4
1: Main subject2: Lenses3: CCD4: Imaging device5: Raw data
Digital Still Cameras Converts optical image to electric signal using charge
coupled device (CCD) Software control
Zoom Focus, e.g. auto-focus filter Shutter aperture and speed White balance: Corrects color distortions
Settings that can be controlled (with added hardware) Camera angle Aspect ratio: Landscape or portrait mode
Produces JPEG compressed images
1/19/2004 Automation of Composition Rules 5
Main Subject Detection Methods Two differently focused photographs [Aizawa, Kodama, Kubota; 1999-2002]
One has foreground in focus, and other has background in focus Significant delay involved in changing the focus
Bayes nets based training [Luo, Etz, Singhal, Gray; 2000-2001] Bayesian network trained on example set and tested later Training time involved: suited for offline applications
Multi-level wavelet coefficients [Wang, Lee, Gray, Wiederhold; 1999-2001] Expensive to compute and analyze wavelet coefficients
Iterative classification from variance maps [Won, Pyan, Gray; 2002] Optimal solution from variance maps and refinement with watershed Suitable for offline applications involving iterative passes over image
1/19/2004 Automation of Composition Rules 6
Proposed Main Subject Detection User starts image acquisition Focus main subject using auto-focus filter Partially blur background and acquire resulting picture
Open shutter aperture (by lowering f-stop) which takes about 1 s Foreground edges stronger than background edges
While acquiring user-intended picture, process blurred background picture to detect main subject Generate edge map (subtract original image from sharpened image) Apply edge detector (Canny edge detector performs well) Close boundary (e.g. gradient vector flow or proposed approximation)
1/19/2004 Automation of Composition Rules 7
Symmetric 3 x 3 sharpening filter For integer and , coefficients are
Integer when dropping 1/(1 + ) term Fractional when -1 – 2 and 1/(1 + ) is power-of-two
Generate edge map Subtract original image from sharpened image Main subject region now has sharper edges
Generate Edge Map
++
++-
f(x,y)g(x,y) fsharp(x,y)Smoothing
filter
0),(
and ),,(*),(),(),,(),(),(
where,),(detect Want to
yxf
yxgkyxfyxfyxfyxfyxg
yxg
sharp
smooth
1
111
)1(1
fsmooth(x,y)
k+
Model for an image sharpening filterSharpening filter
1/19/2004 Automation of Composition Rules 8
Boundary Closure Gradient vector flow method [Xu, Yezzi, Prince; 1998-2001]
Compute gradient Outer boundary of detected sharp edges is initial contour Change shape of initial contour, depending on gradient Shape converges in approximately 5 iterations Disadvantage: computationally and memory intensive
Approximate lower complexity method Select leftmost & rightmost ON pixel and make row between
them ON Can detect convex regions but fails at concavities
1/19/2004 Automation of Composition Rules 9
Automation of Rule-of-Thirds
Goal: Center of mass of the main subject at 1/3 or 2/3 of the picture width (or height) from the left (or top) edge
Solution: For n-D, define function that attains minimum when center of
mass placed as desired and increases otherwise Shift picture so that minimum is attained
Implementation: For 2-D, sum of Euclidean distance from the 4 points Measure which of the 4 points is closest to the current position
of the center of mass Shift picture so that the rule-of-thirds is followed
1/19/2004 Automation of Composition Rules 10
Simulated Background Blurring Goal: Filter the image background and add artistic effects
keeping the main subject intact Solution:
Original image masked with detected main subject mask Region of interest filtering performed on masked image Possible motion blurs
Linear blur: subject or camera motion Radial blur: camera rotation Zoom: change in zoom
Applications Enhance sense of motion where the main subject is moving Digitally decrease the depth-of-field of the photograph
1/19/2004 Automation of Composition Rules 11
Proposed Module
Measure how close rule-of-thirds
followed
Auto-focus filter
Lower f-stop for blur
Filter to generate edge map
Detect sharper edges
Close boundary
Original Image
Automate rule-of-thirds
Simulate background blur
Binary Main Subject Mask
Generated Picture with Rule-of-Thirds
Generated Picture with Blur
1/19/2004 Automation of Composition Rules 12
Implementation Complexity
Number of computations and memory accesses per pixel Main subject detection: convolution with symmetric 3x3 filter, edge
detection, approximate boundary closure Rule-of-thirds: center of mass (1 division, 4 compares) , shift pixels Background blurring: convolution with symmetric 3x3 filter
Digital still cameras use ~160 digital signal processor instruction cycles per pixel
Processing step Multiply-Accumulates /pixel
Comparisons/pixel
Memory accesses/pixel
Main subject detection 18 4 10Rule of thirds 2 1 1 or 3Background blurring 9 4
1/19/2004 Automation of Composition Rules 13
Results (1)Original image with main
subject(s) in focusDetected strong edges with proposed algorithm
Detected main subject mask
Rule-of-Thirds: Main subject repositioned
Simulated background blur
1/19/2004 Automation of Composition Rules 14
Results (2)Original image with main
subject(s) in focusDetected strong edges with proposed algorithm
Detected main subject mask
Rule-of-Thirds: Main subject repositioned
Simulated background blur
1/19/2004 Automation of Composition Rules 15
Results (3)Original image with main
subject(s) in focusDetected strong edges with proposed algorithm
Detected main subject mask
Rule-of-Thirds: Main subject repositioned
Simulated background blur
1/19/2004 Automation of Composition Rules 16
Conclusion Developed automated low-complexity one-pass method for
main subject detection in digital still cameras Processes picture taken with blurred background All calculations in fixed-point arithmetic
Automates selected photographic composition rules Rule-of-thirds: Placement of the main subject on the canvas Simulated background blur: motion and depth-of-field
Applications: digital still cameras, surveillance, constrained image compression, and transmission and display
Copies of MATLAB code, poster, and paper, available athttp://www.ece.utexas.edu/~bevans/projects/dsc/index.html