4G WIRELESS VIDEOCOMMUNICATIONS

Haohong Wang

Marvell Semiconductors, USA

Lisimachos P. Kondi

University of Ioannina, Greece

Ajay Luthra

Motorola, USA

Song Ci

University of Nebraska-Lincoln, USA

A John Wiley and Sons, Ltd., Publication

ayyappan9780470745175.jpg

4G WIRELESS VIDEOCOMMUNICATIONS

Wiley Series on Wireless Communications and Mobile ComputingSeries Editors: Dr Xuemin (Sherman) Shen, University of Waterloo, Canada

Dr Yi Pan, Georgia State University, USA

The Wiley Series on Wireless Communications and Mobile Computing is a series of comprehensive, practicaland timely books on wireless communication and network systems. The series focuses on topics ranging fromwireless communication and coding theory to wireless applications and pervasive computing. The books provideengineers and other technical professionals, researchers, educators, and advanced students in these fields withinvaluable insight into the latest developments and cutting-edge research.

Other titles in the series:

Misic and Misic: Wireless Personal Area Networks: Performance, Interconnection, and Security with IEEE 802.15.4 ,January 2008, 978-0-470-51847-2

Takagi and Walke: Spectrum Requirement Planning in Wireless Communications: Model and Methodology forIMT-Advanced , April 2008, 978-0-470-98647-9

Perez-Fontan and Espineira: Modeling the Wireless Propagation Channel: A simulation approach with MATLAB,August 2008, 978-0-470-72785-0

Ippolito: Satellite Communications Systems Engineering: Atmospheric Effects, Satellite Link Design and SystemPerformance, August 2008, 978-0-470-72527-6

Lin and Sou: Charging for Mobile All-IP Telecommunications , September 2008, 978-0-470-77565-3

Myung and Goodman: Single Carrier FDMA: A New Air Interface for Long Term Evolution , October 2008,978-0-470-72449-1

Hart, Tao and Zhou: Mobile Multi-hop WiMAX: From Protocol to Performance, July 2009, 978-0-470-99399-6

Cai, Shen and Mark: Multimedia Services in Wireless Internet: Modeling and Analysis , August 2009,978-0-470-77065-8

Stojmenovic: Wireless Sensor and Actuator Networks: Algorithms and Protocols for Scalable Coordination andData Communication, September 2009, 978-0-470-17082-3

Qian, Muller and Chen: Security in Wireless Networks and Systems , January 2010, 978-0-470-51212-8

4G WIRELESS VIDEOCOMMUNICATIONS

Haohong Wang

Marvell Semiconductors, USA

Lisimachos P. Kondi

University of Ioannina, Greece

Ajay Luthra

Motorola, USA

Song Ci

University of Nebraska-Lincoln, USA

A John Wiley and Sons, Ltd., Publication

This edition first published 2009 2009, John Wiley & Sons Ltd.,

Registered officeJohn Wiley & Sons, Ltd, The Atrium, Southern Gate, Chichester, West Sussex PO19 8SQ, United Kingdom

For details of our global editorial offices, for customer services and for information about how to apply forpermission to reuse the copyright material in this book please see our website at www.wiley.com.

The right of the author to be identified as the author of this work has been asserted in accordance with theCopyright, Designs and Patents Act 1988.

All rights reserved. No part of this publication may be reproduced, stored in a retrieval system, or transmitted, inany form or by any means, electronic, mechanical, photocopying, recording or otherwise, except as permitted bythe UK Copyright, Designs and Patents Act 1988, without the prior permission of the publisher.

Wiley also publishes its books in a variety of electronic formats. Some content that appears in print may not beavailable in electronic books.

Designations used by companies to distinguish their products are often claimed as trademarks. All brand namesand product names used in this book are trade names, service marks, trademarks or registered trademarks oftheir respective owners. The publisher is not associated with any product or vendor mentioned in this book. Thispublication is designed to provide accurate and authoritative information in regard to the subject matter covered.It is sold on the understanding that the publisher is not engaged in rendering professional services. If professionaladvice or other expert assistance is required, the services of a competent professional should be sought.

Library of Congress Cataloging-in-Publication Data:

4G wireless video communications / Haohong Wang . . . [et al.].p. cm.

Includes bibliographical references and index.ISBN 978-0-470-77307-9 (cloth)

1. Multimedia communications. 2. Wireless communication systems. 3. Videotelephone. I. Wang, Haohong, 1973-

TK5105.15.A23 2009621.384 dc22

2008052216

A catalogue record for this book is available from the British Library.

ISBN 978-0-470-77307-9 (H/B)

Typeset in 10/12pt Times by Laserwords Private Limited, Chennai, India.Printed and bound in Great Britain by Antony Rowe Ltd, Chippenham, Wiltshire.

www.wiley.com

Contents

Foreword xiii

Preface xv

About the Authors xxi

About the Series Editors xxv

1 Introduction 1

1.1 Why 4G? 11.2 4G Status and Key Technologies 3

1.2.1 3GPP LTE 31.2.2 Mobile WiMAX 4

1.3 Video Over Wireless 51.3.1 Video Compression Basics 51.3.2 Video Coding Standards 91.3.3 Error Resilience 101.3.4 Network Integration 121.3.5 Cross-Layer Design for Wireless Video Delivery 14

1.4 Challenges and Opportunities for 4G Wireless Video 15References 17

2 Wireless Communications and Networking 19

2.1 Characteristics and Modeling of Wireless Channels 192.1.1 Degradation in Radio Propagation 192.1.2 Rayleigh Fading Channel 20

2.2 Adaptive Modulation and Coding 232.2.1 Basics of Modulation Schemes 232.2.2 System Model of AMC 252.2.3 Channel Quality Estimation and Prediction 262.2.4 Modulation and Coding Parameter Adaptation 28

vi Contents

2.2.5 Estimation Error and Delay in AMC 302.2.6 Selection of Adaptation Interval 30

2.3 Orthogonal Frequency Division Multiplexing 312.3.1 Background 312.3.2 System Model and Implementation 312.3.3 Pros and Cons 33

2.4 Multiple-Input Multiple-Output Systems 342.4.1 MIMO System Model 342.4.2 MIMO Capacity Gain: Multiplexing 352.4.3 MIMO Diversity Gain: Beamforming 352.4.4 Diversity-Multiplexing Trade-offs 352.4.5 Space-Time Coding 36

2.5 Cross-Layer Design of AMC and HARQ 372.5.1 Background 382.5.2 System Modeling 392.5.3 Cross-Layer Design 412.5.4 Performance Analysis 442.5.5 Performance 45

2.6 Wireless Networking 472.6.1 Layering Network Architectures 482.6.2 Network Service Models 502.6.3 Multiplexing Methods 512.6.4 Connection Management in IP-Based Data Networks 532.6.5 QoS Handoff 54

2.7 Summary 55References 56

3 Video Coding and Communications 59

3.1 Digital Video Compression Why and How Much? 593.2 Basics 60

3.2.1 Video Formats 603.2.1.1 Scanning 603.2.1.2 Color 613.2.1.3 Luminance, Luma, Chrominance, Chroma 64

3.3 Information Theory 643.3.1 Entropy and Mutual Information 653.3.2 Encoding of an Information Source 663.3.3 Variable Length Coding 683.3.4 Quantization 71

3.4 Encoder Architectures 733.4.1 DPCM 733.4.2 Hybrid Transform-DPCM Architecture 773.4.3 A Typical Hybrid Transform DPCM-based Video Codec 793.4.4 Motion Compensation 823.4.5 DCT and Quantization 833.4.6 Procedures Performed at the Decoder 84

Contents vii

3.5 Wavelet-Based Video Compression 863.5.1 Motion-Compensated Temporal Wavelet Transform Using Lifting 90

References 94

4 4G Wireless Communications and Networking 97

4.1 IMT-Advanced and 4G 974.2 LTE 99

4.2.1 Introduction 1014.2.2 Protocol Architecture 102

4.2.2.1 E-UTRAN Overview Architecture 1024.2.2.2 User Plane and Control Plane 1024.2.2.3 LTE Physical Layer 106

4.2.3 LTE Layer 2 1074.2.4 The Evolution of Architecture 1104.2.5 LTE Standardization 110

4.3 WIMAX-IEEE 802.16m 1124.3.1 Network Architecture 1134.3.2 System Reference Model 1144.3.3 Protocol Structure 114

4.3.3.1 MAC Layer 1144.3.3.2 PHY Layer 120

4.3.4 Other Functions Supported by IEEE 802.16m for Further Study 1254.4 3GPP2 UMB 125

4.4.1 Architecture Reference Model 1264.4.2 Layering Architecture and Protocols 127

Acknowledgements 133References 133

5 Advanced Video Coding (AVC)/H.264 Standard 135

5.1 Digital Video Compression Standards 1355.2 AVC/H.264 Coding Algorithm 138

5.2.1 Temporal Prediction 1395.2.1.1 Motion Estimation 1405.2.1.2 P and B MBs 1425.2.1.3 Multiple References 1435.2.1.4 Motion Estimation Accuracy 1435.2.1.5 Weighted Prediction 1445.2.1.6 Frame and Field MV 1445.2.1.7 MV Compression 145

5.2.2 Spatial Prediction 1475.2.3 The Transform 148

5.2.3.1 4 4 Integer DCT and Inverse Integer DCT Transform 1495.2.3.2 8 8 Transform 1505.2.3.3 Hadamard Transform for DC 151

viii Contents

5.2.4 Quantization and Scaling 1515.2.5 Scanning 1515.2.6 Variable Length Lossless Codecs 152

5.2.6.1 Exp-Golomb Code 1535.2.6.2 CAVLC (Context Adaptive VLC) 1545.2.6.3 CABAC 154

5.2.7 Deblocking Filter 1555.2.8 Hierarchy in the Coded Video 156

5.2.8.1 Basic Picture Types (I, P, B, BR) 1575.2.8.2 SP and SI Pictures 157

5.2.9 Buffers 1585.2.10 Encapsulation/Packetization 1595.2.11 Profiles 160

5.2.11.1 Baseline Profile 1605.2.11.2 Extended Profile 1625.2.11.3 Main Profile 1625.2.11.4 High Profile 1625.2.11.5 High10 Profile 1635.2.11.6 High 4:2:2 Profile 1635.2.11.7 High 4:4:4 Predictive Profile 1635.2.11.8 Intra Only Profiles 163

5.2.12 Levels 1635.2.12.1 Maximum Bit Rates, Picture Sizes and Frame Rates 1645.2.12.2 Maximum CPB, DPB and Reference Frames 164

5.2.13 Parameter Sets 1675.2.13.1 Sequence Parameter Sets (SPS) 1675.2.13.2 Picture Parameter Sets (PPS) 167

5.2.14 Supplemental Enhancement Information (SEI) 1675.2.15 Subjective Tests 168

References 168

6 Content Analysis for Communications 171

6.1 Introduction 1716.2 Content Analysis 173

6.2.1 Low-Level Feature Extraction 1746.2.1.1 Edge 1746.2.1.2 Shape 1766.2.1.3 Color 1776.2.1.4 Texture 1776.2.1.5 Motion 1786.2.1.6 Mathematical Morphology 178

6.2.2 Image Segmentation 1796.2.2.1 Threshold and Boundary Based Segmentation 1816.2.2.2 Clustering Based Segmentation 1816.2.2.3 Region Based Approach 1816.2.2.4 Adaptive Perceptual Color-Texture Segmentation 182

Contents ix

6.2.3 Video Object Segmentation 1856.2.3.1 COST211 Analysis Model 1876.2.3.2 Spatial-Temporal Segmentation 1876.2.3.3 Moving Object Tracking 1886.2.3.4 Head-and-Shoulder Object Segmentation 190

6.2.4 Video Structure Understanding 2006.2.4.1 Video Abstraction 2016.2.4.2 Video Summary Extraction 203

6.2.5 Analysis Methods in Compressed Domain 2086.3 Content-Based Video Representation 2096.4 Content-Based Video Coding and Communications 212

6.4.1 Object-Based Video Coding 2126.4.2 Error Resilience for Object-Based Video 215

6.5 Content Description and Management 2176.5.1 MPEG-7 2176.5.2 MPEG-21 219

References 219

7 Video Error Resilience and Error Concealment 223

7.1 Introduction 2237.2 Error Resilience 224

7.2.1 Resynchronization Markers 2247.2.2 Reversible Variable Length Coding (RVLC) 2257.2.3 Error-Resilient Entropy Coding (EREC) 2267.2.4 Independent Segment Decoding 2287.2.5 Insertion of Intra Blocks or Frames 2287.2.6 Scalable Coding 2297.2.7 Multiple Description Coding 230

7.3 Channel Coding 2327.4 Error Concealment 234

7.4.1 Intra Error Concealment Techniques 2347.4.2 Inter Error Concealment Techniques 234

7.5 Error Resilience Features of H.264/AVC 2367.5.1 Picture Segmentation 2367.5.2 Intra Placement 2367.5.3 Reference Picture Selection 2377.5.4 Data Partitioning 2377.5.5 Parameter Sets 2377.5.6 Flexible Macroblock Ordering 2387.5.7 Redundant Slices (RSs) 239

References 239

8 Cross-Layer Optimized Video Delivery over 4G Wireless Networks 241

8.1 Why Cross-Layer Design? 2418.2 Quality-Driven Cross-Layer Framework 242

x Contents

8.3 Application Layer 2448.4 Rate Control at the Transport Layer 244

8.4.1 Background 2448.4.2 System Model 2468.4.3 Network Setting 2468.4.4 Problem Formulation 2488.4.5 Problem Solution 2488.4.6 Performance Evaluation 249

8.5 Routing at the Network Layer 2528.5.1 Background 2528.5.2 System Model 2548.5.3 Routing Metric 2558.5.4 Problem Formulation 2578.5.5 Problem Solution 2588.5.6 Implementation Considerations 2628.5.7 Performance Evaluation 263

8.6 Content-Aware Real-Time Video Streaming 2658.6.1 Background 2658.6.2 Background 2658.6.3 Problem Formulation 2668.6.4 Routing Based on Priority Queuing 2678.6.5 Problem Solution 2698.6.6 Performance Evaluation 270

8.7 Cross-Layer Optimization for Video Summary Transmission 2728.7.1 Background 2728.7.2 Problem Formulation 2748.7.3 System Model 2768.7.4 Link Adaptation for Good Content Coverage 2788.7.5 Problem Solution 2808.7.6 Performance Evaluation 283

8.8 Conclusions 287References 287

9 Content-based Video Communications 291

9.1 Network-Adaptive Video Object Encoding 2919.2 Joint Source Coding and Unequal Error Protection 294

9.2.1 Problem Formulation 2959.2.1.1 System Model 2969.2.1.2 Channel Model 2979.2.1.3 Expected Distortion 2989.2.1.4 Optimization Formulation 298

9.2.2 Solution and Implementation Details 2999.2.2.1 Packetization and Error Concealment 2999.2.2.2 Expected Distortion 2999.2.2.3 Optimal Solution 300

9.2.3 Application on Energy-Efficient Wireless Network 301

Contents xi

9.2.3.1 Channel Model 3019.2.3.2 Experimental Results 302

9.2.4 Application on Differentiated Services Networks 3039.3 Joint Source-Channel Coding with Utilization of Data Hiding 305

9.3.1 Hiding Shape in Texture 3089.3.2 Joint Source-Channel Coding 3099.3.3 Joint Source-Channel Coding and Data Hiding 311

9.3.3.1 System Model 3119.3.3.2 Channel Model 3129.3.3.3 Expected Distortion 3129.3.3.4 Implementation Details 313

9.3.4 Experimental Results 315References 322

10 AVC/H.264 Application Digital TV 325

10.1 Introduction 32510.1.1 Encoder Flexibility 326

10.2 Random Access 32610.2.1 GOP Bazaar 327

10.2.1.1 MPEG-2 Like, 2B, GOP Structure 32710.2.1.2 Reference B and Hierarchical GOP structures 33010.2.1.3 Low Delay Structure 33110.2.1.4 Editable Structure 33110.2.1.5 Others 332

10.2.2 Buffers, Before and After 33210.2.2.1 Coded Picture Buffer 33210.2.2.2 Decoded Picture Buffer (DPB) 334

10.3 Bitstream Splicing 33510.4 Trick Modes 337

10.4.1 Fast Forward 33810.4.2 Reverse 33810.4.3 Pause 338

10.5 Carriage of AVC/H.264 Over MPEG-2 Systems 33810.5.1 Packetization 339

10.5.1.1 Packetized Elementary Stream (PES) 34010.5.1.2 Transport Stream (TS) 34010.5.1.3 Program Stream 343

10.5.2 Audio Video Synchronization 34410.5.3 Transmitter and Receiver Clock Synchronization 34410.5.4 System Target Decoder and Timing Model 344

References 345

11 Interactive Video Communications 347

11.1 Video Conferencing and Telephony 34711.1.1 IP and Broadband Video Telephony 347

xii Contents

11.1.2 Wireless Video Telephony 34811.1.3 3G-324M Protocol 348

11.1.3.1 Multiplexing and Error Handling 34911.1.3.2 Adaptation Layers 35011.1.3.3 The Control Channel 35011.1.3.4 Audio and Video Channels 35011.1.3.5 Call Setup 350

11.2 Region-of-Interest Video Communications 35111.2.1 ROI based Bit Allocation 351

11.2.1.1 Quality Metric for ROI Video 35111.2.1.2 Bit Allocation Scheme for ROI Video 35311.2.1.3 Bit Allocation Models 354

11.2.2 Content Adaptive Background Skipping 35611.2.2.1 Content-based Skip Mode Decision 35711.2.2.2 Budget Adjustment 360

References 366

12 Wireless Video Streaming 369

12.1 Introduction 36912.2 Streaming System Architecture 370

12.2.1 Video Compression 37012.2.2 Application Layer QoS Control 372

12.2.2.1 Rate Control 37212.2.2.2 Rate Shaping 37312.2.2.3 Error Control 374

12.2.3 Protocols 37412.2.3.1 Transport Protocols 375

12.2.4 Video/Audio Synchronization 37612.3 Delay-Constrained Retransmission 377

12.3.1 Receiver-Based Control 37812.3.2 Sender-Based Control 37812.3.3 Hybrid Control 37912.3.4 Rate-Distortion Optimal Retransmission 379

12.4 Considerations for Wireless Video Streaming 38212.4.1 Cross-Layer Optimization and Physical Layer Consideration 383

12.5 P2P Video Streaming 384References 385

Index 389

Foreword4G Wireless Video Communications is a broad title with a wide scope, bridging videosignal processing, video communications, and 4G wireless networks. Currently, 4G wire-less communication systems are still in their planning phase. The new infrastructure isexpected to provide much higher data rates, lower cost per transmitted bit, more flexiblemobile terminals, and seamless connections to different networks. Along with the rapiddevelopment of video coding and communications techniques, more and more advancedinteractive multimedia applications are emerging as 4G network killer apps.

Depending on the readers background and point of view, this topic may be consideredand interpreted with different perspectives and foci. Colleagues in the multimedia signalprocessing area know firsthand the effort and ingenuity it takes to save 1 bit or increasethe quality of the reconstructed video during compression by 0.5 dB. The increased band-width of 4G systems may open the door to new challenges and open issues that willbe encountered for first time or were neglected before. Some topics, such as contentinteractivity, scalability, and understandability may be reconsidered and be given higherpriority, and a more aggressive approach in investigating and creating new applicationsand service concepts may be adopted. On the other hand, for colleagues with communica-tions and networking backgrounds, it is important to keep in mind that in the multimediacontent delivery scenario, every bit or packet is not equal from the perspective of powerconsumption, delay, or packet loss. In other words, media data with certain semantic andsyntactical characteristics may impact the resulting user experience more significantly thanothers. Therefore it is critical to factor this in to the design of new systems, protocols, andalgorithms. Consequently, the cross-layer design and optimization for 4G wireless videois expected to be an active topic. It will gather researchers from various communitiesresulting in necessary inter-disciplinary collaborations.

In this book, the authors have successfully provided a detailed coverage of the funda-mental theory of video compression, communications, and 4G wireless communications.A comprehensive treatment of advanced video analysis and coding techniques, as well asthe new emerging techniques and applications, is also provided. So far, to the best of myknowledge, I have seen very few books, journals or magazines on the market focusingon this topic, which may be due to the very fast developments in this field. This bookhas revealed an integrated picture of 4G wireless video communications to the generalaudience. It provides both a treatment of the underlying theory and a valuable practicalreference for the multimedia and communications practitioners.

Aggelos K. KatsaggelosProfessor of Electrical Engineering and Computer Science

Northwestern University, USA