Filtering – images restorationusers.polytech.unice.fr/~lingrand/Ens/up/Lesson6-noise.pdf ·...
57
Image processing, 2006, [email protected] 1 Filtering – images restoration University of the Philippines - Diliman 2006 Diane Lingrand [email protected] http://www.polytech.unice.fr/~lingrand
Transcript of Filtering – images restorationusers.polytech.unice.fr/~lingrand/Ens/up/Lesson6-noise.pdf ·...
Image processing, 2006, [email protected] 1
Filtering – images restoration
University of the Philippines - Diliman2006
Diane [email protected]
http://www.polytech.unice.fr/~lingrand
Image processing, 2006, [email protected] 3
Image processing, 2006, [email protected] 4
Image processing, 2006, [email protected] 5
Image processing, 2006, [email protected] 7
Noise sources• Acquisition context
– Under- or over-lighting– Sensors disturbance
• Sensor– Distorsions (geometrical, intensity)
• Sampling– Aliasing effect (out of validity bound of Nyquist-Shannon)– Objects for which the size is the same as the pixel size: salt and
pepper noise• Scene content
– Clouds in satellite images, …• Quantification
– 256 grey levels: not really a limiting factor for human visual system -- or less
• Compression• Transmission
– Data loss, data corruption
Image processing, 2006, [email protected] 8
Noise modeling
• Noise depending or not of the image data • Additive or multiplicative noise• Noise: high frequencies (usually higher than the
image frequencies) => low-pass filtering• Kinds of noise:
– Impulse noise: white Gaussian, exponential...– Salt and pepper noise
Image processing, 2006, [email protected] 9
Salt and pepper noise• nth order : turn n pixels to white and n pixels
to black randomly in an image.– Often expressed as a percentage p (0p1)
• Physical counter-part: dust on a camera film or a scanner, small objects, data loss
• in Java: import java.util.Random– method: int nextInt(int n) gives an integer in [0 n[, with
uniform distribution
– 2 methods :• randomly choose w*h/p pixels to be changed to BLACK
and w*h/p others to be changed to WHITE
• for each pixel: chose randomly an int in [0 1/p[, then if 0, change pixel to WHITE, if 1 change it to BLACK
Image processing, 2006, [email protected] 10
Example of salt and pepper noise
Original image Noisy salt and papper image with p(white) =p(black)=5%
Image processing, 2006, [email protected] 11
Impulse noise
• Probability density function form:
• For �α = 1 : exponential noise
• For �α = 2 : Gaussian noise
Image processing, 2006, [email protected] 12
Gaussian noise
• Additive Gaussian noise with null mean and variance σ 2
• Similar to the acquisition noise• How to implement a Gaussian variable ?
– in Java :
import java.util.Random;The public method double Random.nextGaussian() is a Gaussian random value with mean 0 and standard deviation 1 : to be multiplied by – in C/C++: use rand() from C system library
Image processing, 2006, [email protected] 13
Gaussian noise in C++
• Initializationstruct timeval tv;
struct timezone tz;
gettimeofday(&tv, &tz);
srand(tv.tv_usec);
• Normal random variable distribution between 0 and 1 rand()/(RAND_MAX+1.0f);
• Gaussian variable b = rand()/(RAND_MAX+1.0f);
if(b < epsilon) b = epsilon;
c = rand()/(RAND_MAX+1.0f);
a = -2.0 * Math::log(b);
if(a < 0.0) a = 0.0;
else a *= Math::cos(2.0 * Math::pi * c);
Image processing, 2006, [email protected] 14
Gaussian noise example
Original image Noisy image with a Gaussian noise (std dev 4)
Image processing, 2006, [email protected] 17
Spatial filtering
• Filter applied to the neighborhood of a pixel in an image
• Linear or non-linear filtering– linear : convolution, Fourier Transform– non linear : no convolution, no Fourier Transform
• Processing:– smoothing (blurring), contours strengthening
(sharpening) …
Image processing, 2006, [email protected] 18
Smoothing filters
• Mean filters
• Gaussian filters
• Median filters
• Conservative smoothing
• others
Image processing, 2006, [email protected] 20
Mean filtering
• Also known as averaging or Box filtering
• Principle : a pixel value is replaced by the mean value computed from its neighbors (and itself)
• Convolution with a kernel of size (2n+1 x 2n+1).– ex:
1/91/91/9
1/91/91/9
1/91/91/9 Be careful ifyou make divisionswith integers !
Image processing, 2006, [email protected] 21
Example
• This is a low-pass filtering
• Impairment of contours (blurring)
Image processing, 2006, [email protected] 22
Smoothing a salt and pepper noise :
Kernel size = 3 Kernel size = 5
Image processing, 2006, [email protected] 23
Smoothing a Gaussian noise
Kernel size = 3 Kernel size =5
Image processing, 2006, [email protected] 24
Gaussien filter (Gaussian smoothing)
• 2D Gaussian :– null mean
– variance σ2
• Principle: convolution with a Gaussian• In practice : Gaussian discretization on a
kernel with size (2p+1,2p+1)
Image processing, 2006, [email protected] 25
Gaussian kernels examples
• 3x3[ 1 2 1 ] [ 2 4 2 ] [ 1 2 1 ]
• 7x7[1 1 2 2 2 1 1][1 2 2 4 2 2 1][2 2 4 8 4 2 2][2 4 8 16 8 4 2][2 2 4 8 4 2 2][1 2 2 4 2 2 1][1 1 2 2 2 1 1]
• 5x5[ 1 1 2 1 1 ] [ 1 2 4 2 1] [ 2 4 8 4 2 ][ 1 2 4 2 1] [ 1 1 2 1 1]
Image processing, 2006, [email protected] 26
How to compute a Gaussian kernel ? (1)
• Kernel ? ((2p+1) x (2p+1))
– Integer coefficient (faster computations) followed by a normalization step (division by the sum of coefficient)
– Even more efficient: coefficient as powers of 2
– σ depends on the size of the kernel (p)
Image processing, 2006, [email protected] 27
How to compute a Gaussian kernel ? (2)
• The last coefficient value is 1
• Its real value is obtained by dividing it with the sum of all coefficients ......
...2q
1=2q.e-p2/σ22q.e-p2/(2σ2)
2q.e-p2/(2σ2)
...
...
...
Image processing, 2006, [email protected] 28
Another example of Gaussian kernel
1
273
1 4 4
4
4
4
4
4
4
4
1
1 1
7
7
7 7
16 26
26
2626
16
16 16
41
Image processing, 2006, [email protected] 29
Gaussian filtering after a salt and pepper noise on an
image
Kernel size = 3 Kernel size = 7
Image processing, 2006, [email protected] 30
Gaussian filtering following a Gaussian noise of null mean
(σ = 16)
3 7
Image processing, 2006, [email protected] 31
Conservative filtering
• Principle : only the pixels whose value falls in the interval defined by the neighboring pixels are preserved. If the pixel values falls out of this interval, it is replaced by the closest bound:– Upper bound if the value is higher to that
– Lower bound if the value is lower to that
Image processing, 2006, [email protected] 32
Conservative filtering after a salt and pepper noise
Original image After conservative filtering
Image processing, 2006, [email protected] 33
Conservative filtering after gaussian noise
Original image After conservative filtering
Image processing, 2006, [email protected] 34
Conservative filtering
0
255140
88
110
93
0 6763
0
255255
88
110
93
0 6763
0 ∈ [0 ; 255]
0
255140
88
110
93
62 6763
62
255255
88
110
93
62 6763
0 ∉ [62 ; 255]
Image processing, 2006, [email protected] 35
Median filters
• Principle : a pixel is replaced by his neighbors' values
• Median value = value in the middle of the sorted list of neighbors' values⇒Need to sort values (ex: quick sort ?)
• Enable the suppression of outliers• Good performance when processing images
impaired by salt and pepper noise
Image processing, 2006, [email protected] 36
Median filtering
0
0
255255
67
88
110
93
63
0 6763
088 93 255255 110
00 255 25511063 67 88 93
88
255255
88
110
93
0 6763
sorting
median value
Image processing, 2006, [email protected] 37
Some median filter kernels
• carrés (7x7)[1 1 1 1 1 1 1][1 1 1 1 1 1 1][1 1 1 1 1 1 1][1 1 1 1 1 1 1][1 1 1 1 1 1 1][1 1 1 1 1 1 1][1 1 1 1 1 1 1]
• diamand (7x7)[0 0 0 1 0 0 0][0 0 1 1 1 0 0][0 1 1 1 1 1 0][1 1 1 1 1 1 1][0 1 1 1 1 1 0][0 0 1 1 1 0 0][0 0 0 1 0 0 0]
• en croix (7x7)[0 0 0 1 0 0 0][0 0 0 1 0 0 0][0 0 0 1 0 0 0][1 1 1 1 1 1 1][0 0 0 1 0 0 0][0 0 0 1 0 0 0][0 0 0 1 0 0 0]
• en croix (3x3)[0 1 0]
[1 1 1]
[0 1 0]
• carrés (3x3)[1 1 1][1 1 1][1 1 1]
Image processing, 2006, [email protected] 38
Median filter after pepper and salt noise
kernel 3x3, applied once kernel 3x3, applied twice
Image processing, 2006, [email protected] 39
Median filtering after a gaussian noise
Original image After median filtering
Image processing, 2006, [email protected] 40
Contrast enhancement (Sharpening)
• High-pass filtering (edge crispening)• To counter-balance the effects of
smoothing• Some convolution kernels:
[0 -1 0] [1 -2 1] [-1 -1 -1][-1 10 -1] ou [-2 5 -2] ou [-1 9 -1][0 -1 0] [1 -2 1] [-1 -1 -1]
• Strengthen the noise as well• By lowering the center value, the effect
is amplified (try 5 instead of 10)
Image processing, 2006, [email protected] 41
Example 1:
Salt and pepper noise + 2 times median filtering (3x3)
Image processing, 2006, [email protected] 42
Example 2:
White noise σ = 16 followed by Gaussian filtering (7x7)
Image processing, 2006, [email protected] 43
Preliminary conclusions
• Salt and pepper noise:– Median filtering performs well
• White Gaussian noise :– Gaussian filtering performs rather well
• Draw-backs in both cases :– Contours smoothing
• To go further :– Smoothing only in areas of weak gradients
Image processing, 2006, [email protected] 44
Exercise
• Add noise to images– Gaussian noise– Salt and pepper noise
• Implement restoration filters– Mean filters (2n+1x2n+1), Gaussian filters– Conservative filtering– Median filtering– Contrast enhancement filters
• Test noise and restoration procedures on images
Image processing, 2006, [email protected] 47
Back to Gaussian filtering
• Convolution with a Gaussian :
• with
Image processing, 2006, [email protected] 48
Evolution of an image• We add a temporal parameter t as the
image is seen as an object evolving through time:
u(x) u(x,t)
• Image evolution :– Initialization : u(x,t0) = u0(x)
– State at a given time t : u(x,t) = modified image
– evolution : speed ?
Image processing, 2006, [email protected] 49
Example : heat equation (1)
• Describes heat propagation:
• Linear and parabolic equation
• Interesting property : there exists an closed-form solution which is a Gaussian form :
Image processing, 2006, [email protected] 50
Heat equation (2)
• Exercise : – Check out that the solution is valid
• Consequences :– Correlation between Gaussian smoothing with
variance �σ2=2t and evolution of the image according to the heat PDE (Partial Differential Equation)
– A classical Gaussian filter can be defined as the result of a PDE
Image processing, 2006, [email protected] 51
The Partial Differential Equations
• Old methods, many mathematical studies, …
• How to apply this technique in the image processing area?
• In the general case, there is no known closed-form solution– Numerical implementation of the evolution
Image processing, 2006, [email protected] 52
PDE applied to images restoration
• The drawback of filters seen previously is the lack of contour conservation
• Heat equation :
– We expect : • To smooth part of the images with weak gradent• In strong gradient areas: to smooth the image in a
direction parallel to the contour– Idea : weight by a function depending on the
gradient [Perona-Malik]
s
c(s)
c(|∇u|2)
Image processing, 2006, [email protected] 53
Restoration with contours conservation
• Intuitive idea :– Weak |∇u| : c near to 1 : heat equation
– High |∇u| : c near to 0 : null speed, no smoothing
Image processing, 2006, [email protected] 54
Contour conservation (continued)
• More precisely : divide in two terms (tangential and normal)
• Taking :
• One get :
Image processing, 2006, [email protected] 55
Weighting function
• Constraints :
• Example :c(s) = 1 / 1 + s
decreasing
tends at infinity towards
Image processing, 2006, [email protected] 56
A practical example
Histogramequaliza-
tion
Smoothing with edge conservation
Image processing, 2006, [email protected] 57
Bibliography
• Mathematical problems in image processing: Partial Differential Equations and the Calculus of Variations, par G. Aubert et P. Kornprobst, edition Springer, Applied Mathematical Sciences, vol. 147, 2002 http://www-sop.inria.fr/odyssee/team/Pierre.Kornprobst/book-pde/index.html
