Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE,...
-
date post
21-Dec-2015 -
Category
Documents
-
view
215 -
download
0
Transcript of Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE,...
![Page 1: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/1.jpg)
Scalable Rate Control for MPEG-4 Video
Hung-Ju Lee, Member, IEEE,
Tihao Chiang, Senior Member, IEEE, and
Ya-Qin Zhang, Fellow, IEEE
IEEE TRANSACTIONS ON CIRCUITS AND SYSTEMS FOR VIDEO TECHNOLOGY, VOL. 10, NO. 6, SEPTEMBER 2000
![Page 2: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/2.jpg)
Outline
• Introduction to Rate Control
• Framework of SRC
• New Features of the Proposed SRC
• Experimental Results
• Summary and Conclusions
![Page 3: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/3.jpg)
Network-based Multimedia System
![Page 4: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/4.jpg)
Diminish impact on the video quality
• Due to the delay jitter and available network resources (e.g., bandwidth and buffers)
• Diminish Techniques– Traffic shaping (Transport Layer)– Scalable rate control (SRC, Compression Layer)
![Page 5: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/5.jpg)
Development of an SRC scheme
• To cope with various requirements of different coding environments and applications, a rate-control scheme needs to provide sufficient flexibility and scalability
• Two multimedia application groups– Variable-bit-rate (VBR)– Constant-bit-rate (CBR)
![Page 6: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/6.jpg)
A rate-control scheme must be scalable for
• various bit rates (e.g., 10 kbits/s to 1 Mbits/s),• various spatial resolutions (e.g., QCIF to CCIR-6
01), • various temporal resolutions (e.g., 7.5–30 fps),• various coders (e.g., DCT and wavelet), and• various granularities of video object (VO) (e.g., si
ngle VO to multiple VOs, frame-layer to macroblock (MB)-layer).
![Page 7: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/7.jpg)
Developing a rate-control technique
• two widely used approaches: – 1) an analytical model-based approach and– 2) an operational rate-distortion (R-D) based
approach.
![Page 8: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/8.jpg)
Some Technique
• Lagragian multiplier– Min Σ(D+λR)– Not consider about temporal dependency
• Most of the techniques only focus on a single coding environment, either frame level, object level, or macro level.
![Page 9: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/9.jpg)
Proposed SRC
• Allocating bits among VOs.
• Joint-buffer rate-control scheme
![Page 10: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/10.jpg)
Framework of SRC
![Page 11: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/11.jpg)
Scalable Quadratic Rate Distortion Model
![Page 12: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/12.jpg)
![Page 13: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/13.jpg)
Enhance the R-D model
• The R-D model is not scalable with video contents
• The R-D model does not exclude the bit counts used for coding the overhead including video/frame syntax, motion vectors and shape information.
![Page 14: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/14.jpg)
To solve the target bit rate
![Page 15: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/15.jpg)
Initialization Stage
1) initializing the buffer size based on latency requirement;
2) subtracting the bit counts of the first I-frame from total bit counts;
3) initializing the buffer fullness in the middle level.
![Page 16: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/16.jpg)
Remaining available bits
![Page 17: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/17.jpg)
Framework of SRC
![Page 18: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/18.jpg)
Pre-Encoding Stage
• Target bits estimation– Frame-level bit rate– Object level if desired– MB-level bit-rate estimation if desired
• Further adjustment of the target bits based on the buffer status for each VO
• Quantization parameter calculation
![Page 19: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/19.jpg)
Target bit count for frame level
![Page 20: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/20.jpg)
Consider with buffer fullness
![Page 21: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/21.jpg)
Framework of SRC
![Page 22: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/22.jpg)
Encoding Stage
• Encoding the video frame (object) and recording all actual bit rate;
• Activating the MB-layer rate control if desired.
![Page 23: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/23.jpg)
Framework of SRC
![Page 24: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/24.jpg)
Post-Encoding Stage
• updating the correspondent quadratic R-D model for the entire frame or an individual VO
• Performing the shape-threshold control to balance the bit usage between shape information and texture information
• performing the frame-skipping control to prevent the potential buffer overflow and/or underflow
![Page 25: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/25.jpg)
R-D Model Update
• Selection of Data Points• Calculation of the Model Parameters X1 and X2
• Removal of the Outliers from the Data Set
![Page 26: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/26.jpg)
Shape-Threshold Control
• size conversion process
• shape-threshold setting
![Page 27: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/27.jpg)
Frame Skipping Control
• Objective– To prevent buffer overflow– Fight the problem with continuous frame
skipping
![Page 28: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/28.jpg)
New feature of the proposed SRC• To make the quadratic R-D model more accurate and scalabl
e– Sliding-window data-point selection– Statistical removal of data outliers– Predictive frame-skipping control
• In object-based rate control– Calculating the target bit rate among the VOs
• dynamic target bit-rate distribution among VOs
– Balancing the bit budget between the shape information and the texture information without introducing noticeable distortion
• adaptive shape-threshold control
– Encoding the VOs with proper temporal resolution so that the quality of the composite video frame is sufficient.
![Page 29: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/29.jpg)
Sliding-Window Data-Point Selection
• The sliding-window mechanism is used to adaptively smooth the impact of a scene change for certain number of frames in updating the R-D model.
![Page 30: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/30.jpg)
Statistical Removal of Data Outliers
• Those erroneous data points are defined, in the statistical sense, as the data points whose prediction errors between the actual bit rate and the target bit rate is larger than standard deviations (e.g., as a rule of thumb, is set to one in our experiments)
• To avoid the removal of all data points, the latest data point is always selected in the data set.
![Page 31: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/31.jpg)
Predictive Frame-Skipping Control
![Page 32: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/32.jpg)
Predictive Frame-Skipping Control
![Page 33: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/33.jpg)
Dynamic Target Bit-Rate Distribution Among VOs
• Experimental results show that the significant quality deterioration is experienced in the “gluing” boundary of VOs. Thus, encoding the VOs at the same frame rate is a better alternative to yield better video quality.
![Page 34: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/34.jpg)
Adaptive Shape-Threshold Control
![Page 35: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/35.jpg)
MB-Level Rate Control
![Page 36: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/36.jpg)
Target Bit-Rate Calculation ri for an MBi
![Page 37: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/37.jpg)
Target Bit-Rate Calculation ri for an MBi
![Page 38: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/38.jpg)
![Page 39: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/39.jpg)
Experimental Results
• A. Frame-Level Rate Control
• B. Object-Level Rate Control
• C. MB-Level Rate Control
![Page 40: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/40.jpg)
A. Frame-Level Rate Control
![Page 41: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/41.jpg)
A. Frame-Level Rate Control
![Page 42: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/42.jpg)
B. Object-Level Rate Control
![Page 43: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/43.jpg)
B. VO Level
![Page 44: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/44.jpg)
C. MB-Level Rate Control
![Page 45: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/45.jpg)
C. MB-Level Rate Control
![Page 46: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/46.jpg)
C. MB-Level Rate Control
![Page 47: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/47.jpg)
Summary and Conclusions
New methods or concepts in this paper:1) a more accurate R-D model which is scalable with MAD
2) a dynamically bit-rate allocation among VOs with various coding complexities
3) a sliding-window mechanism for smoothing the impact of scene change
4) an adaptive selection criterion of data points for the R-D model update process
5) an adaptive threshold setting for rate reduction in shape coding
6) an effective frame-skipping control for the prevention of the potential buffer-overflow problem
![Page 48: Scalable Rate Control for MPEG-4 Video Hung-Ju Lee, Member, IEEE, Tihao Chiang, Senior Member, IEEE, and Ya-Qin Zhang, Fellow, IEEE IEEE TRANSACTIONS ON.](https://reader030.fdocuments.us/reader030/viewer/2022032704/56649d6b5503460f94a4b182/html5/thumbnails/48.jpg)
Summary and Conclusions
Advantages of the proposed SRC:1) Low latency and the limited buffer constrains are satisfied for CBR applic
ations
2) The VBR quality is maintained
3) Both the target bit rate and the target frame rate are obtained within a negligible error
4) easy extension to the multiple VOs and MB layer