Image deblocking using local segmentation By Mirsad Makalic Supervisor: Dr. Peter Tischer.
-
date post
21-Dec-2015 -
Category
Documents
-
view
219 -
download
1
Transcript of Image deblocking using local segmentation By Mirsad Makalic Supervisor: Dr. Peter Tischer.
Image deblocking using local segmentation
By Mirsad Makalic
Supervisor: Dr. Peter Tischer
Presentation Outline
An introduction to JPEG Lossy image compression Discrete Cosine Transform (DCT) and quantization
Local segmentation and prior research Measuring image quality Deblocking Filter Super-Resolution Filter Conclusion
Lossy image compression
JPEG is the most common lossy image compression format Easy to implement Good quality with high compression ratios
Block based transform approach Divide image into 8x8 blocks Perform discrete cosine transform (DCT) on each 8x8
block Quantize the DCT coefficients
Discrete Cosine Transform
Transform 8x8 block of pixels into a set of weighted basis functions (DCT coefficients)
DC coefficient
AC coefficients
DCT - Subbands
An image consisting of just one coefficient from each 8x8 block is called the sub-band image
An image which is divided into 8x8 blocks has 64 possible sub-band images
An 8x8 image split into 2x2 blocks has 4 subbands
Quantization
Exploit visual redundancy Quantization is a many-to-one mapping
Divide each DCT coefficient by a value and round to the nearest integer
The decoder makes a guess from a range of values (pick midpoint by default)
5 6 7 8 9 10 11 12 13 14
= 1Quantizer = 10
Quantization
Coarse quantization introduces artifacts into reconstructed image DCT coefficients are
reconstructed inaccurately
Most visually distracting artifact is blockiness
JPEG compressed image at quality 10 PSNR: 30.41 dB
Deblocking Techniques
Three approaches in the literature:Filter the reconstructed pixel valuesAttempt to reconstruct DCT coefficients more
accuratelyA hybrid approach
Local Segmentation
Most deblocking filters introduce excessive blurringDestroys the structure of the imageEdges lose their sharpness
Local segmentation takes into account the structure of the image
Local Segmentation
Divide a mask of pixels into N segments and filter each segment independently
Average of whole mask = 43.22, average of yellow segment = 20.8
Two questions:How many segments do we use?How do we segment a mask of pixels?
Prior Research
Lukasz Kizewski, BSE (hons) 2004 DC subband approach
Filter using a mask of DC subbands How do we segment a mask of pixels?
Segment the pixel mask using thresholding How many segments do we use?
Do-No-Harm heuristic
Prior Research
Do-No-Harm heuristicTry a 1-segment model (average of the whole
mask) If filtered value is implausible reject and try a 2-
segment model A plausible value is one which falls inside the
quantization range: midpoint +/- ½ QuantumTry a 2-segment model
If still implausible then don’t filter
Prior Research
Results
(a) Unfiltered image (a) Filtered image
Room for improvement
No objective measure used to test the effectiveness of the filterDifficult to make comparisonsDifficult to rate changes in filter
Works only on DC subbandAC subbands contain edge and texture
information A very simple local segmentation method
Measuring image quality
Peak-signal-to-noise-ratioMost commonly used metricDoes not necessarily reflect the subjective
visual quality Generalized Block-Edge Impairment
Metric (GBIM) – H.R. Wu, M. Yuen
Measuring image quality - GBIM
Measures the quality of DCT encoded images Assume that what happens inside a block is the same
as what happens across blocks Take absolute mean difference of pixels inside a block
(vertical/horizontal) Take absolute mean difference of pixels across blocks
(vertical/horizontal) Compare them, if the two differ greatly than it is a sign of
blockiness
100 101 130 131
101 101 132 131
102 100 131 130
102 101 132 129
2x2 block example with vertical blockiness
A new deblocking filter
Filter all coefficientsTreat each 8x8 block as a 64 element vector
where each value in the vector is one of the DCT coefficients
Local segmentation no longer as simple (need to segment masks of vectors)K-Means or K-Nearest Neighbours
DNH needs to work on vectors
A new deblocking filter
Basic structure of filter is same: Create an NxN mask where each item is a 64 element
DCT coefficient vector Segment mask using K-Means or K-Nearest
Neighbours segmentation Check if segmentation produces valid result using
DNH, if not, try different segment If no segmentation produces valid result, leave alone
K-Means
Start with one segment (average of mask) If segmentation is invalid, increase number
of segments by one until a maximum number of segments is reached
Try largest change first
K-Nearest Neighbours
Set the number of nearest neighbours to find as the number of items in the entire mask
Keep decreasing by segment size by one until a valid segment is found
Vector DNH
Center DNH Compare the filtered vector against only the center
vector in the mask Strict Segment DNH
Compare the filtered vector against all the vectors in the segment the center vector is in
Lesser Strict Segment DNH Same as strict segment DNH with some error
tolerance
Results
Best found parameters for the filter: 3x3 vector mask (covers 24x24 pixels) K-Nearest Neighbour segmentation Lesser strict segment DNH with 1% error tolerance
Tested: 5x5 mask, K-Means, Center DNH, Strict DNH etc. Many variations of the filter parameters
Results
(a) JPEG compressed image (b) Filtered image
An improvement of 0.07 dB in PSNR and a reduction of 0.62 in GBIM
Results
(a) JPEG compressed image (b) Filtered image
An improvement of 0.07 dB in PSNR and a reduction of 0.55 in GBIM
Other uses for local segmentation
Super-resolutionCombine multiple slightly different images to
form one higher quality imageCan we extract more information out of a
single image? Neighbouring pixels are similar and share
information Use local segmentation
Super-resolution filter
Very similar to the deblocking filter Instead of using a 64 element vector for each
DCT coefficient, use one element vector containing each pixel value in the mask
How to test if it works?Convert 8 bit image to 4 bits and attempt to
reconstruct back an 8 bit image
Results
Best found parameters for filter:3x3 maskK-Means segmentation with up to three
segmentsCenter DNHMore than one iteration of the filter can further
improve PSNR
Results
An improvement of 1.38 dB in PSNR
(a) An image rounded to 4 bits per pixel
(a) Filtered image (8 bits per pixel)
Results
(a) An image rounded to 4 bits per pixel
(a) Filtered image (8 bits per pixel)
An improvement of 1.32 dB in PSNR
Conclusion
Filtering AC subbands is difficult because most have been quantized to zero or have very large quantization ranges
Most improvements in image quality are from the stricter DNH and better local segmentation techniques
The increase in computational complexity may not be worth the increase in image quality
Conclusion
Super-resolution filter shows a lot of promiseLarge increase in PSNR and image quality
A different DNH heuristic may work better with the super-resolution filter
Future Research
Allow more variation in pixels for the strict DNHAssume local mask is linear and not constant
Try different segmentation techniquesRegion growing
Further investigate iterative filtering
The End
Questions?