Mahmoud S. Hamid, Neal R. Harvey, and Stephen Marshall

IEEE Transactions on Circuits and Systems for Video Technology, 2003

Genetic Algorithm Optimization of Multidimensional Grayscale Soft Morphological Filters With Applications in Film Archive Restoration

Outline• Introduction• Soft Morphological Filters (SMF)• Genetic Algorithm (GA)

• Introduction• Applying to the File Dirt Problem

• Discussion• Conclusion

Introduction• Film dirt is the common problem in old film archives.• This damage manifests itself as “blotches” of random size,

shape and intensity.• These blotches are nontime correlated.• The cost of conventional restoration are very high.• Some of then can only deal with physical film strip.

• Most of the conventional image sequence restoration algorithms involve median filtering.

• Then, lots of median filter are Introduced.

Soft Morphological Filters (SMF)• Grayscale soft morphological filters.• Two parts of the structuring element : the hard center and

the soft boundary.• Less rigidly in noisy conditions more tolerant to small

variations in the shapes of the objects.

Soft Morphological Filters (cont.)• The structuring system [a,b,r] consists of three parameters:• a is the hard center.• b is called the structuring function.• b\a is the soft boundary.• r is the repetition parameter.• The grayscale soft dilation and the grayscale soft erosion :

Soft Morphological Filters (cont.)• Grayscale soft open-closing and soft close-opening are

combinations of the soft closing and soft opening operations.

Extend to the Spatio-Temporal Domain

• video sequence is a much richer source of visual information than a still image;

• image sequences that contain fast motion always been a problem in the restoration of film archives.

Genetic Algorithm (GA)• Initial Population• Evaluation

• fitness

• Mating Selection• Reproduction• Environmental Selection

GAInitial Population

Evaluation

Mating Selection

Reproduction

Evaluation

EnvironmentalSelection

Final Population

Stop?

Y

N

Next generation

Genetic Algorithm (cont.)

• Structuring function:• a) Hard Center• b) Soft Boundary

• Rank (Repetition parameter)• Sequence of soft morphological operations:

• {soft erode, soft dilate, do-nothing}

Applying the GA Optimization Method to the File Dirt Problem• Fitness should be determined.• Find areas of the uncorrupted image. Artificially corrupt

these ideal image regions.• Fitness value based on some measure of the mean

absolute error (MAE).

Fitness Function

• Fitness for an image in the sequence is a measure of how it is close to the ideal.

• fitness value = 100 means the filter is perfect.

Genetic Operators• Selection: Stochastic universal sampling

• Crossover: Uniform crossover• (probability = 0.75)

• Mutation: Randomly choosing• (probability = 0.03)

• Population Size: 30

Parent Solutions

Discussion• Structuring function size = 773 .• Running for 500 generations.

• Fitness of LUM = 98.56

• Fitness of LUM = 98.56• Fitness of optimized SMF = 99.52

Discussion (cont.)

Discussion (cont.)• To compare with a method which is depend the detection

of the noise using the ROD detector [19] with ML3Dex filter[20].

• It filters the detected noisy pixels and leaves the remaining image pixels untouched.

• Use the same noise detection with optimized SMF.

[19] M. Nadenau and S. Mitra, “Blotch and scratch detection in image sequences based on rank ordered differences,” in Proc. 5th Int. Workshop on Time Varying Image Processing and Moving Object Recognition,Sept. 1996, pp. 27–35.[20] A. Kokaram, Motion Picture Restoration. Berlin, Germany: Springer,1998.

• Fitness of ML3Dex= 99.54• Fitness of SMF with noise detection = 99.88

Discussion (cont.)• SMF could perfectly restore all fast-moving objects.

Conclusion• A technique for the optimization of multidimensional

grayscale soft morphological filters using the GA.• Showed excellent performance in removing dirt from film

and has little effect on the image detail.• The fast-moving objects were restored perfectly.