Efficient MPEG Compressed Video Analysis Using Macroblock Type Information
description
Transcript of Efficient MPEG Compressed Video Analysis Using Macroblock Type Information
![Page 1: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/1.jpg)
Efficient MPEG Compressed Video Analysis Using Macroblock Type Information
Soo-Chang Pei, Yu-Zuong Chou
IEEE TRANSACTIONS ON MULTIMEDIA, DECEMBER,1999
![Page 2: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/2.jpg)
Introduction We develop a novel method for video analysis
using the macroblock(MB) type information of MPEG compressed video bitstreams.
Only a simple analysis on MB types of frames is needed to achieve efficient scene change, flashlight, and caption detection.
The advantages of this novel approach are its direct extraction from the MPEG bitstreams after VLC decoding, very low complexity analysis, frame-based detection accuracy and high sensitivity.
![Page 3: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/3.jpg)
Proposed Method SGOP(subgroup of pictures)
SGOP1 are denoted as Pf Bf Br Pr
SGOP2 are denoted as P Bf Br i SGOP3 are denoted as i Bf Br P
![Page 4: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/4.jpg)
Pattern of MB Types of SGOP’s with Abrupt Scene Changes
SCPI (Scene change at a P frame or an I frame) Most MB’s in the two B frames are forward
motion compensated (F mode) A significant number of MB’s in Pr are Intra-
coded (I mode)
x
![Page 5: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/5.jpg)
Pattern of MB Types of SGOP’s with Abrupt Scene Changes(cont.)
SCFB (Scene change at front B frame) Most MB’s in the two B frames are
backward motion compensated (B mode)
There are many I mode MB’s existing in Pr
![Page 6: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/6.jpg)
Pattern of MB Types of SGOP’s with Abrupt Scene Changes(cont.)
SCRB (Scene change at rear B frame) Most MB’s in Bf will be F mode predicted to Pf,
and most MB’s in the Br will be B mode predicted to Pr .
![Page 7: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/7.jpg)
We define the MB in a specific position (x,y) of each frame of a SGOP to form a MB group(MBGxy).
If the pattern of the MB types in a MBGxy follows the above pattern of SCPI,SCFB or SCRB, we can say there is a changing MB in the position (x,y) laying at Pf , Bf , or Br.
![Page 8: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/8.jpg)
Example of gradual scene changes
![Page 9: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/9.jpg)
Pattern of MB Types in SGOP’s with Gradual Scene Changes Dissolve
scene S0 to scene Sn in the transition sequence, length n+1
If S0 =0 ,fade-in. If Sn =0 ,fade-out. The luminance of B frames in SGOP1 (PBBP)
operated by the dissolve effect can be approximately written as
![Page 10: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/10.jpg)
Pattern of MB Types in SGOP’s with Gradual Scene Changes(cont.)
When bidirectional motion compensation is applied to B frames, the compensated frames of three compensation modes(F, B, FB) can be written
It is found that the variance of FB compensated error is smaller than that of F or B compensated error.
B frames in dissolve video sequence are inclined to be interpolative(FB mode) motion compensated.
![Page 11: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/11.jpg)
![Page 12: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/12.jpg)
SFPI(starting frame lies at Pr or I frame) A significant number of MS’s in Pr are I mode because the
change of content. MB’s in B frames are inclined to be F mode The pattern of MB types of SFPI is very similar to the
pattern of SCPI SFFB (starting frame lies at Bf )
Most MB’s in both B frames are FB mode since the frames of this SGOP form a Complete Dissolve SGOP.
SFRB (starting frame lies at Br) the MB’s of Bf are inclined to be F mode and those of Br
are inclined to be FB mode.
x
![Page 13: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/13.jpg)
A significant number of MB’s of Pr in a dissolve video sequence are I mode in both SFFB and SFRB.
The patterns of the MB types of the SGOP including ending frames(EFPI,EFFB,EFRB) are similar and symmetric to that including starting frames.
There are nine combinations.
![Page 14: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/14.jpg)
x
![Page 15: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/15.jpg)
Video sequence involving camera motion
Zooming operation is slow for the purpose of comfortable watching, and can be easily tracked as a special kind of movement and will not greatly affect the MB types of frames.
Slow panning operations would be judged as overall movement of the frame and the MB type information would not show some specific characteristic.
The speeding of fast panning exceeds the tracking ability of motion compensation,so a significant number of MB’s in the P frame are I mode.
The detection process may confuse fast panning operation with dissolve operation because of the similarity.
![Page 16: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/16.jpg)
x
![Page 17: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/17.jpg)
Example of flashlight in news experiment sequence
![Page 18: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/18.jpg)
Pattern of MB Types in SGOP’s with Flashlights
![Page 19: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/19.jpg)
Experimental Result There are 24 abrupt scene changes, five
gradual scene change transitions,f our shots involving camera motion(two fast panning and two zooming sequences) in the test sequence.
Each frame of the sequence is in SIF (352*240) format, which yields 330 (22*15) MB’s in a frame.
The frame structure is IBBPBBPBBPBBPBB (M=3,N=15) .
Total bit rate is 1.5 Mbps
![Page 20: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/20.jpg)
Abrupt scene change detection
![Page 21: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/21.jpg)
Locations of changing MB’s in the test sequence(frame 1 to 100)
![Page 22: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/22.jpg)
Experimental results of abrupt scene change detection ( Tmb = 280)
![Page 23: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/23.jpg)
![Page 24: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/24.jpg)
Gradual scene change detection
![Page 25: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/25.jpg)
Remaining P frames with significant I mode MB’s
![Page 26: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/26.jpg)
Number of FB mode MB’s in B
frames
![Page 27: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/27.jpg)
![Page 28: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/28.jpg)
Flashlight detection
![Page 29: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/29.jpg)
Flashlight detection using news experiment sequence
![Page 30: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/30.jpg)
Caption detection
![Page 31: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/31.jpg)
The locations of changing MB’s in the caption sequence
![Page 32: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/32.jpg)
Abrupt scene change detection
Caption detection with eliminating the abrupt scene changes
![Page 33: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/33.jpg)
Experiments Using Real Video Sequence of Movies and News
![Page 34: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/34.jpg)
Discussion Influence of Bit Rates
![Page 35: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/35.jpg)
![Page 36: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/36.jpg)
Taking the forward motion compensated MB for instance, the attribute of MB’s change from “Not coded” or “Coded, Larger Quantizer Scale” to “Coded, Default” to obtain better residue data reconstruction instead of changing FB to Intra mode.
The change of the bit rates just very slightly affects the occurrences of I,B,F,FB modes is found.
The detection algorithm exploiting the MB types is not affected by bit rates.
![Page 37: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/37.jpg)
Modification of Proposed Method Corresponding to Different GOP Structure
![Page 38: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/38.jpg)
![Page 39: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/39.jpg)
Advantage of Proposed Method
The MB type information can be directly extracted from the bit streams of compressed video after VLC decoding.
Detecting process is quite simple as compared to current DCT methods that require more operations.
Its detection accuracy is on the frame scale, and we can precisely indicate which frame the scene change occurs.
![Page 40: Efficient MPEG Compressed Video Analysis Using Macroblock Type Information](https://reader036.fdocuments.us/reader036/viewer/2022062409/56814683550346895db3a419/html5/thumbnails/40.jpg)
Conclusion We have developed a novel video analysis
method using MB type information,and satisfactory detection precision and speed is obtained.
The method using MB type information benefits from easy data extraction from the bitstream, very simple analysis, frame-based accuracy and high sensitivity to avoid miss detection.