OPTIMIZED RATE ALLOCATION OF HYPERSPECTRAL IMAGES IN COMPRESSED DOMAIN USING JPEG2000 Part 2
Methods of Video Object Segmentation in Compressed Domain
description
Transcript of Methods of Video Object Segmentation in Compressed Domain
![Page 1: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/1.jpg)
Methods of Video Object Segmentation in Compressed Domain
Cheng Quan Jia
![Page 2: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/2.jpg)
Presentation Outline
• Features for Segmentation in Compressed Domain
• Using Motion Vectors in Segmentation
• Confidence Measure• Conclusion• Q & A
![Page 3: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/3.jpg)
Features for Segmentation in Compressed Domain
An introduction to Compressed Domain
![Page 4: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/4.jpg)
Compressed Domain: Definition
• Compressed Domain refers to video compression techniques that expliots Spatial and Temporal Redundancy using – DCT & Quantization– Motion Compensation
• Examples include MPEG-1/-2/-4, H.261 and H.263
![Page 5: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/5.jpg)
Compressed Domain: Definition
• Opreations in the Compressed Domain involves processing of
– DCT coefficients (from I-macroblocks)– Motion Vectors (from P-/B-macroblocks)
![Page 6: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/6.jpg)
Compressed Domain: Parsing
• Unlike pixel domain, operations in the compressed domain do not require the input bitstream to be decoded
• Instead, they are Parsed
![Page 7: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/7.jpg)
Compressed Domain: Parsing
![Page 8: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/8.jpg)
Features for Segmentation
• After Parsing, we have– DCT coefficients (from I-macroblocks)– Motion Vectors (from P-/B-macroblocks)
• Which coresspond to – Frequencies of texture change– Motion of the macroblock
![Page 9: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/9.jpg)
Using Motion Vectors in Segmentation
![Page 10: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/10.jpg)
Acquiring Dense Motion Field
• Many video object segmentation methods attempt to acquire a dense smooth motion field in order to create object masks
• For this end spatial interpolation and motion accumulation are employed
![Page 11: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/11.jpg)
Motion Accumulation
![Page 12: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/12.jpg)
Motion Accumulation
• Due to the different magnitude and signs of motion vectors, the obtained MVs are normalized, e.g. MVs in B-macroblocks would have their signs reversed
• Filtering is applied to remove non-uniform MV and smooth the motion field
![Page 13: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/13.jpg)
Motion Accumulation
• Chen and Bajic [chen2009] employs MV Integration block-wise and pixel-wise to enhance the Motion Field
![Page 14: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/14.jpg)
Motion Accumulation
Chen and Bajic [chen2009] Babu et al. [babu2004]
![Page 15: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/15.jpg)
Porikli et al.’s Investigation
• The Compression Domain segmentation system published by Porikli et al. [porikli2010] experimented the effect of DCT coefficients and MV on segmentation performance– The DC parameters(for Y, U, V channels) of the I-
frame– Low vertical and horizontal frequency AC values– A spatial energy term– Aggregated motion flow of the corresponding
macroblock
![Page 16: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/16.jpg)
Porikli et al.’s Investigation
• They create a Frequency-temporal data structure for each macroblock with the features and perform volume segmentation
• Their results show that using DCT terms in FT segmentation and using MV in the hierarchical clustering, on average, gives better results.
![Page 17: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/17.jpg)
Porikli et al.’s Investigation
![Page 18: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/18.jpg)
• The Block Matching Process in encoding stage looks for only the best match for a macroblock rather than object motion
Porikli et al.’s Investigation
![Page 19: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/19.jpg)
Confidence Measure of Motion Vectors
![Page 20: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/20.jpg)
• Coimbra and Davies [coimbras2005] try to approximate Lucas–Kanade optical flow in MPEG-2 Compressed Domain
Approximating Optical Flow
![Page 21: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/21.jpg)
• They argue that AC[1] and AC[8] in an I-macroblock can be used as confidence measure
• The confidence update step will have a 8×8 macroblock referencing a 16×16 image block in the I-frame, and the confidence of the motion vector of the macroblock is the weighted average of confidence in the 16×16 window
Confidence Measure
![Page 22: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/22.jpg)
Confidence Measure
Original image MPEG-2 smooth motion field afterconfidence threshold
![Page 23: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/23.jpg)
Conclusion
![Page 24: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/24.jpg)
• Due to block matching process, motion vectors in P-/B- frames do not necessary relate to object motion
• To ensure a motion vector is correlated to object motion, some sort of confidence measure is required
• [coimbras2005] demonstrated that edge strength can be an effective measure
Conclusion
![Page 25: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/25.jpg)
• Problems not discussed here – Camera motion– Changes in illumination– Occlusions
Conclusion
![Page 26: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/26.jpg)
References1. R. V. Babu, K. R. Ramakrishnan, and S. H. Srinivasan.
Video Object Segmentation: A Compressed Domain Approach. IEEE Transactions on Circuits and Systems for Video Technology, 14(4):462–473, April 2004.
2. Y.-M. Chen and I. V. Bajic. Compressed-Domain Moving Region Segmentation with Pixel Precision using Motion Integration. In IEEE Pacific Rim Conference on Computers and Signal Processing, 2009, pages 442 – 447, August 2009.
3. M. T. Coimbra and M. Davies. Approximating Optical Flow Within the MPEG-2 Compressed Domain. IEEE Transactions on Circuits and Systems for Video Technology, 15(1):103–107, January 2005.
4. F. Porikli, F. Bashir, and H. Sun. Compressed Domain Video Object Segmentation. IEEE Transactions on Circuits and Systems for Video Technology, 20(1):2–14, January 2010.
![Page 27: Methods of Video Object Segmentation in Compressed Domain](https://reader036.fdocuments.us/reader036/viewer/2022062423/5681492d550346895db6682a/html5/thumbnails/27.jpg)
Q & A SECTION
Comments and Suggestions