MRF-BASED TRUE MOTION ESTIMATION USING H.264 DECODING INFORMATION
Yung-Lin Huang, Yi-Nung Liu, and Shao-Yi Chien
Media IC and System Lab
Graduate Institute of Networking and Multimedia
National Taiwan University
Signal Processing Systems (SIPS), 2010 IEEE
Outline
IntroductionMarkov Random Field
Motion Vector Analysis Motion Vector Pre-processing Predictor Selection Simplified Belief Propagation Experimental Results Conclusion
Introduction (1/4) Instead of heuristic approaches, TME can be
formulated as a pixel-labeling problem:
Introduction (2/4) Markov Random Field :
Given an undirected graph G = (V, E) A set of random variables(label) X = (Xv)v ∈ V
Markov properties:
Introduction (3/4) Assigning each pixel a label, can be justified
in terms of maximum a-posterior estimation of a MRF model:
posterior likelihood * prior∝
With negative log probabilities, where the max-product becomes a min-sum.
The max-product algorithm can be used to find an approximate minimum cost labeling of energy functions.
Ed (the data term) & Es (the smoothness term)
Introduction (4/4) The cost energy function of a Markov Random Field
model to estimate the optimal labels { lp } of corresponding pixels:
Ed: the data term that measures the penalty between the labels and the data
Es: the smoothness term that penalizes the coherence between labels
P : the set of all pixelsN : the 4-nearest neighbor pixels
Motion Vector Analysis(1/3) Optical flow datasets are used here because
the ground truth (GT) MV maps are provided:
Check the existence of true MV by similarity:
The existence of TMV:W,H: the width, height of the test sequence
In Fig. 4. Both THx and THy are set to 1, and PSR ranges from 0 to 64.
Motion Vector Analysis(2/3)
The ME strategy(FastFS, FS or EPZS) has little effect on the experimental results.
There are still MVs with true motion trajectory in the H.264-coded MV field.
Motion Vector Analysis(3/3)
Motion Vector Pre-processing
In the proposed algorithm, the block size is fixed in each scale, so the MVs of variable block sizes must be split and merged.
The block merging method takes not only the macroblock types (from H.264) but also neighboring MVs into consideration.
Although the global optimization might modify these bad MVs, the pre-processing costs less efforts.
Predictor Selection(1/2)
In Fig. 4, the probability that true MV exists is high with enough PSR.
We choose PSR=32, when the block size is 16, the range of ±32 pixels ±2 blocks.
The strategy of predictor selection and the MRF model of the proposed algorithm are shown in Fig. 5(a). Nine predictors are selected.
Simplified Belief Propagation (1/3) The multi-scale concept from [7],instead of
pixel-based operation, 4x4 block is taken as the smallest unit.
The belief propagation isoperated from the highest scale (16x16 block) to thelowest scale (4x4 block).
ft: video frame at time t
The basic concept of belief propagation is to perform message passing operation iteratively and approximate global minimum by local messages.
Each pixel requires O(k2) computation for full-search candidates.
The proposed algorithm requires only O(k) computation with predictor selection [7] for each pixel.
Simplified Belief Propagation (2/3)
Simplified Belief Propagation (3/3)[7] Pedro F. Felzenszwalb and Daniel P. Huttenlocher, “Efficient belief propagation for early vision,” Int. J. Comput.Vision, vol. 70, no. 1, pp. 41–54, 2006.
Loopy Belief Propagation approach for MRF:
Messages with the truncated linear model:
Time complexity: O(nk2T)n: the number of pixelsk: the number of possible labelsT: the number of iterations
Time complexity: O(k)n: the number of pixelsk: the number of possible labelsT: the number of iterations
Experimental Results Frame Rate Up Conversion compare with:
Bidirectional overlapped block motion estimation (OBME), MV field smoothing with median filter.
Proposed algorithm has higher PSNR aboutthe camera motion video (mobile calendar) because of the global MV field optimization.
OBME requires full search(FS) with an enlarged search range.
The proposed algorithm has relative lower computational complexity.
Top Related