Image Blur Detection with 2D Haar Wavelet Transform and

Its Effect on Skewed Barcode Scanning

Vladimir Kulyukin Sarat Kiran Andhavarapu

Department of Computer ScienceUtah State University

Outline

● Background● Image Blur Detection with 2D Haar Tile

Clustering● Evaluation of 2D Haar Tile Clustering● Effects of Image Blur Detection on Skewed

Barcode Scanning

Background

Theoretical Foundations

● Mallat & Hwang [1] argue that signals carry information via irregularites

● These researchers show that the local maxima of the wavelet transform detect locations of irregularites

● For example, 2D HWT maxima indicate possible locations of edges in images

Indirect & Direct Blur Detection Methods

● Tong et al. [2] classify image blur detection methods into direct and indirect

● Indirect methods characterize image blur as a linear function IB = B *

IO + N, where I

O is the original image, B is an unknown blur function,

N is a noise function, and is the result image blur and noise are introduced in the image

● Direct methods are based on detection of distinct features directly computed in the images, e.g., corners, edges, color histograms, etc

Edge Classification

Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114.

Edge-Based Blur Detection

● Tong et al. [2] propose a direct method based on 2D Haar Wavelet Transform

● Main assumption of their research is that introduction of blur has different effects on the four main types of edges

● In blurred images, Dirac and A-Step edges are absent whereas G-Step and Roof edges lose their sharpness

● Images are classified as blurred on the basis of presence/absence of Dirac & A-Step edges

Image Blur Detectionwith

2D Haar Tile Clustering

Theory

● Our method (2D Haar Tile Clustering) is based on the hypothesis that it may not be necessary to detect any explicit features such as corners or edges

● Rather, it may be possible to detect regions with pronounced changes without explicitly computing the causes of those changes

● After those regions are detected, they can be combined into larger segments

● Those larger segments can be used to classify images as blurred

Theory

● Sharp images have many easily distinguishable features, e.g., edges, corners, textures

● Blurred images have fewer easily distinguishable features because distinctions among image regions are less pronounced

● Easily distinguishable features consists of regions with pronounced changes

Finding Regions with Pronounced Changes

Image is split into 64 x 64 blocks (aka tiles)

Tile Processing

● Each tile is processed by four iterations of the 2D HWT● The number of iterations is a parameter, and can be

increased/decreased● Each tile is represented by three 2D Haar wavelets:

horizontal change (HC), vertical change (VC), and diagonal change (DC)

● These values are thresholded to retain the tiles with only large changes

Applying 2D HWT to Image

● Image on the right shows two iterations of ordered 2D HWT

● The whiteness of each pixel is proportional to the magnitude of the wavelet in the corresponding array cell

Representation of 64 x 64 Blocks with 2D Wavelets

Each region is represented in terms of 3 wavelets: vertical (first double); horizontal (second double), diagonal (third double); Ignore the numbers after “|”: they are used for debugging

Tile Cluster Buildup with DFS

● After all tiles with pronounced areas are found, as shown in the image on the right

● Now we can run DFS to find all tile clusters

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Cluster Buildup with DFS

Tile Clustering in Sharp Images

Eventually we run out of unmarked tiles with sufficiently high pronounced changes, as shown in the image on the right

Tile Clustering in Blurred Images

Title clusters shown in the right image are found by DFS run on tiles with pronounced changes found in the left image

Tile Cluster Filtering● After the tile clusters are found, two cluster-related rules are used to

classify a whole image as sharp or blurred

● The 1st rule uses the percentage of the total area of the image covered by the found tile clusters

● The 2nd rule uses the number of the tiles in each cluster to discard small clusters

● The 1st rule captures the intuition that sharper images have many tiles with pronounced changes

● The 2nd rule captures the intuition that small clusters should be discarded as irrelevant

2D Haar Tile Clustering: Algorithmic ChainTake Image Find Tiles Find Tile

ClustersFilter Tile

Clusters

BLURRED / SHARP

Evaluation of 2D Haar Tile Clustering&

Effects of Image Blur on Skewed Barcode Scanning

Image Sample

● 500 random RGB images* were selected from a set of 506 smartphone video recordings of common grocery products

● Three human volunteers were recruited to classify each image as blurred or sharp

● An image was classified as sharp if at least two of the three volunteers classified it as sharp – this is the ground truth

*Images are available at https://app.box.com/s/n4s2ve0dajz5gkzqx9vpm1f6fzhw5upz

Three Evaluated Algorithms

● Algorithm 1: Kulyukin, V. & Andhavarapu. S. “Image Blur Detection with 2D Haar Wavelet Transform and Its Effect on Skewed Barcode Scanning.” To appear in Proceedings of the 19th International Conference on Image Processing, Computer Vision, & Pattern Recognition (IPCV 2015). Las Vegas, NV, USA

● Algorithm 2: Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114

● Algorithm 3: Blur effect: perception and estimation with a new no-reference perceptual blur metric.” In Proceedings of SPIE 6492, Human Vision and Electronic Imaging XII, 64920I, San Jose, CA, USA, January 28, 2007. doi:10.1117/12.702790

True & False Positives on Blurred and Sharp Images

Algorithm True Positives on Blurred Images

False Positives on Blurred Images

True Positives on Sharp Images

False Positives on Sharp Images

Algorithm 1 163 4 254 79

Algorithm 2 167 0 183 150

Algorithm 3 81 86 268 65

Ground Truth 167 0 333 0

Relative Difference Table

Algorithm Relative Difference on Blurred Images

Relative Difference on Sharp Images

Algorithm 1 2.39 23.72

Algorithm 2 0.00 45.05

Algorithm 3 51.50 19.52

images arpblurred/shon truth ground theis and algorithmby found

images arpblurred/sh ofnumber theis where,100,max

,fferenceRelativeDi

GA

AGA

GAGA

Observations

● On blurred images, Algorithm 1 & Algorithm 2 do not deviate from ground truth; Algorithm 3 shows a significant deviation (52%)

● On sharp images, Algorithm 1 & Algorithm 3 deviate from ground truth by 20% whereas Algorithm 2 deviates by 45%

Effects of Blur Detection on Barcode Scanning

Sample Sharp Blurred Barcodes in Sharp

Barcodes in Blurred

1 15 15 12 1

2 13 17 11 0

3 16 14 12 0

images. allin barcodes recognized of ratio theand images sharpin only

barcodes recognized of ratio theb/w difference theisGain ed.investigat wasdetection

blur accurate ofeffect theand algorithm scanning barcode skewedour into integrated

wasdetection blur The images. blurred 15 and sharp 15 had sampleeach :evalutorshuman

3by classified images 500 from selected each were images 30 of samples random Three

Sample Blurred Sharp Total Gain

1 1/15 12/15 13/30 0.37

2 0/17 11/13 11/30 0.50

3 0/16 12/14 12/30 0.46

Paper & Code References

[1] Mallat, S. and Hwang, W. L. “Singularity detection and processing with wavelets.” IEEE Transactions on Information Theory, vol. 38, no. 2, March 1992, pp. 617-643.

[2] Tong, H., Li, M., Zhang, H., and Zhang, C. "Blur detection for digital images using wavelet transform," In Proceedings of the IEEE International Conference on Multimedia and Expo, vol.1, pp. 27-30, June 2004.doi: 10.1109/ICME.2004.1394114.

[3] Cretea,F., Dolmierea, T., Ladreta, P., Nicolas, M. “The Blur effect: perception and estimation with a new no-reference perceptual blur metric.” In Proceedings of SPIE 6492, Human Vision and Electronic Imaging XII, 64920I, San Jose, CA, USA, January 28, 2007. doi:10.1117/12.702790.

[4] Python implementation of the blur detection algorithm proposed in reference [2] is available at https://github.com/VKEDCO/PYPL/blob/master/haar_blur

[5] MATLAB implementation of blur detection algorithm proposed in reference [3] is

available at http://www.mathworks.com/matlabcentral/fileexchange/24676-image-blur-metric