Jongchurl Park([email protected])
Networked Media LaboratoryDept. of Information and Communications
School of Information & MechatronicsGwangju Institute of Science & Technology (GIST)
System Enhancements and Extension of Uncompressed HD Media Transport System
Lab. SeminarMay 30th, 2009
School OF INFO. & Mecha., GIST
Introduction◦ Uncompressed HD media transport system◦ Networked tiled display & 3DTV
Enhanced uncompressed HD media transport sys-tem◦ Preview video & bi-directional transport system◦ Networked display interface
Extension of uncompressed HD media transport system◦ Multiple streams for free-viewpoint◦ Views in 3D effects
Conclusion & Future works
Contents
School OF INFO. & Mecha., GIST
Introduction
School OF INFO. & Mecha., GIST
No need encoding and decoding proce-dures◦ Highly interactive (i.e., low-delay) media services
Video format◦ 720p: progressive, 60 fps, 1280x720, 20 bits/sample◦ 1080i: interlaced, 30 fps, 1920x1080, 20 bits/sample
Requied bandwidth: 1.2~1.5 Gbps Delay: ~200msec. Interactive application
◦ Video conference◦ Remote lecture (i.e., e-learning)◦ Interactive remote performance (e.g., DancingQ)
Introduction (1/3)-Uncompressed HD media transport system
School OF INFO. & Mecha., GIST
Standard Definition Television (SDTV)◦ Resolution - 480i: 720 x 480, 60 interlaced fields per second
◦ Uncompressed Bandwidth: ~200Mbps Delay: ~100msec.
◦ MPEG2 Bandwidth: 2~6Mbps Delay: 1~2 sec.
High Definition Television (HDTV)◦ Resolution
1080i: 1920 x 1080, 60 interlaced fields per sec. 720p: 1280 x 720, 60 frames per sec. (progressive scan)
◦ Uncompressed Bandwidth: 1.2 ~ 1.5 Gbps Delay: ~100msec.
◦ MPEG2 Bandwidth: 20~80Mbps Delay: 1~2 sec.
Limitation◦ Uncompressed HD video transport system on raw is not enough for display-
ing correct HD videos◦ More realistic presence is required like three dimension
Introduction (2/3)-SD vs. HD
School OF INFO. & Mecha., GIST
Networked tiled display◦ Features
Monitors of Lattice shape , users can use as if tiled display is single monitor Cost-effective ultra-high resolution device (more than 4K) Simultaneously run various applications on local or remote clusters, and share
them by streaming the pixels of each application Users freely move and resize each application's imagery in run-time Color-space
Compatible with RGB16, RGB24 and RGB32
◦ Applications Scalable Adaptive Graphics Environment (SAGE)
The role of display middleware to visualize any kind of pixel-stream to a tiled display color-space
SMeet One Display (SMOD) Realized after SAGE, Integrating collaboration services that include media delivery scheme to offer A/V communication among participants, high-resolution display
service
3DTV◦ “Realistic” ghost-like illusions has always been a goal◦ A new generation of broadcast
Immersive 3D video display in real-time
Introduction (3/3)-Networked Tiled Display & 3DTV
School OF INFO. & Mecha., GIST
Enhanced Uncompressed HD Me-dia Transport System
(Preview Video Support &Bi-directional Transport System)
School OF INFO. & Mecha., GIST
System Overview
The criterion of uncompressed HD media transport system◦ Hardware capture and software display
Specification◦ Ultragrid (USC/ISI) extension◦ XENA HS (AJA Video systems) – Rx & Tx ◦ 1080i (25,50,29.97,59.94,30,60 fps)◦ 6-channel audio (24-bit/48KHz)◦ Video out
XENA HS, SDL, xVideo◦ Audio out
XENA HS, ALSA◦ Platform
Redhat 9, Mandrake 10.1, and Fedora Core 5
6-Channel Audio
Sony HVR-Z1N
5.1 Channel Speaker
Preview
HD Display
1G
1G
High Speed Network
HW Capture
SW Display
Uncompressed HDAV Transport System
School OF INFO. & Mecha., GIST
Preview◦ users can preview and see the current stage of the process before producing into a final form. ◦ lets users to visualize current/final product and correct possible errors easily before finalizing the
product. ◦ Interactive video conference and online forums allow users to preview their place environment
before transport.
Particularly useful on sites with complex markup, where it serves as an op-portunity to identify and correct video quality, light and so on.
Preview Video
A/D Converter
Xena HS Capture
cardGigabit Ethernet
Uncompressed HD
A/D ConverterXena HS Capture
card
Uncompressed HD
A/D Converter
Receiver/Sender ApplicationSender/Receiver Application
Send
Receive
NIC
Speaker
A/D Converter
Uncompressed Audio
Speaker
NIC
Uncompressed Audio
School OF INFO. & Mecha., GIST
1G NIC
Memory
CPU
CPU
HDD (RAID)
Xena card[Capture]
DMA
1G NIC
Memory
HDD (RAID)
TransferModule
[re-organized]
Xena card[Display]
Xena card[Capture]
Xena card[Display]
DMA
CPU
CPUTransferModule
[re-organized]
Node #1 Node #2
Uncompressed HD Media Transport System
HW-based Bi-directional Transport
Alternating system configuration◦ Supporting multiple network interfaces
Verified for bi-directional streams
◦ Multiple HD-SDI interfaces support Separating functions to capture and display Simplifying system handling with bi-directional transport
Hardware performance analysis◦ Requires both HD-SDIs and Network Interface Cards support 1.3Gbps bandwidth
To exploit hardware overload from bulky traffic
◦ Hardware bandwidth requires more than PCI-X 100Mhz extended slot 133Mhz extended slot can support maximum 1 GByte/s
School OF INFO. & Mecha., GIST
0.0
0.1
0.2
0.3
0.4
0.5
0.6
0.7
0.8
0.9
1.0
outgoing video stream (single load)
imcoming video stream (single load)
outgoing video stream (dual loads)
incoming video stream (dual loads)
Frame per second
Occ
urr
ance
rate
Occu
rren
ce
rate
Frame per second
Comparison of frame rate◦ Outgoing/incoming video stream (single load)
61.5% of frame rate maintain at 29.97 fps
◦ Outgoing/incoming video stream (dual load) 99.3% of frame rate maintain at 29.97 fps
Dual load paths improved and stabilized performance than single load network
Experiment Result
School OF INFO. & Mecha., GIST
Enhanced Uncompressed HD Media Trans-port System
- Network Display Interface -
School OF INFO. & Mecha., GIST
System Overview
Interfacing uncompressed HD media transport system and networked tiled display◦ A scalable hundreds of megapixels of contiguous display with cost-
effective prices as possible
School OF INFO. & Mecha., GIST
Networked Tiled Display
Uncompressed HD Video Transport
System
Gigabit Ethernetc
Uncompressed HD video streams
SAIL library &(SAGE)
STP library(SMOD)
Display Modules
Display node 1
Display node 2
Display node 3
Display node 4
Interfacing Module(configuration)
Color Space Conversion Module (YUV422 to RGB24)
SAIL library &STP library
.
.
System Architecture
Video source: Uncompressed HD◦ Captured 1920x1080i format HD
signal by using HD-SDI card◦ Covert analog signal to digital sig-
nal, due to SMPTE-292M by using A/D converter
Interfacing Module◦ Put video data to buffer◦ Color-space conversion
Transport◦ Put frames to buffer and receive
frames by using SAIL or STP library
Display modules◦ Display video on all display nodes
School OF INFO. & Mecha., GIST
Performance enhancement for networked display interface◦ Using binary increase congestion control (BIC) is not good for heavy traf-
fic Long delay over high-speed networks Improving TCP-friendliness and RTT-fairness
◦ Using extended version of BIC (CUBIC) – good for heavy traffic (around 1Gbps bandwidth usage) Increasing TCP transport window size for no acknowledge (NACK) Allocating large enough socket buffers Reconfiguring TCP segment sizes
Maximum MTU packet by using a path MTU discovery function
◦ System tuning (After receiving 1000 packets, it cause freezing HD dis-play) All nodes as well as a controller should be set around MTU 9000 byte Increasing buffer size can solve freezing problem Hard sync stream type makes it more stable
System tuning & Configuration
School OF INFO. & Mecha., GIST
Interfacing without internal network
Cost-effective interfacing system◦ Interfacing system is required to set up both sender and re-
ceiver of uncompressed HD media transport system Directly transmit frames sender to networked tiled display (without re-
ceiver) Using sail library instead of RTP in uncompressed HD media transport sys-
tem
School OF INFO. & Mecha., GIST
Comparison of Bandwidth (BIC and CUBIC)◦ BIC
• Input: 921Mbps (29.97 fps)• Output: 239.4Mbps (7 fps)
◦ CUBIC• Input: 921Mbps (29.97 fps)• Output: 682.1Mbps (23 fps)
CUBIC improved and stabilized performance than BIC• Approximately 16fps difference
Experiment Result
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 210
100
200
300
400
500
600
700
800
BICCUBIC
Time (second)
Bandw
idth
(bps)
Time (sec.)
Ban
dw
idth
(M
bp
s)
School OF INFO. & Mecha., GIST
Environment◦ 8bit uncompressed HD video is
transmitted over 1Gbps (MTU: 9180 Bytes)
Top: Artistic dance performance is deliv-ered in live to international destinations by HW-based playout mode
Middle: Interactive video conferencing scenario with preview HD videos by SW-based playout
Bottom: Interfacing with networked tiled display
Ultra-high-resolution support Appropriate control of frame rate
Experimental Snapshots
Hardware-based Playout for Bi-directional Transport
Receiver View
Sender ViewSender View
Receiver View
Node 1 Node 2
Software-based Playout
Network-based Tiled Display Playout
School OF INFO. & Mecha., GIST
Extension of Uncompressed HD Media Transport System
School OF INFO. & Mecha., GIST
◦ Interactive clustered broadcasting with free-viewpoint display Multiple streams for free-viewpoint Views in 3D effects (Interactive views) Ultra-high-resolution support
Overall Architecture
School OF INFO. & Mecha., GIST
Multiple video streams in single machine◦ Required to handle more TWO uncompressed HD video data (over 2Gbps)
Simplifying system handling multiple ways for transport
◦ Improved configuration of network interface card and HD-SDI interface Up to 4 media interfaces
◦ Verified for bi-directional streams and two ways streams
Free-viewpoint video◦ A multi-view display emits more than two views, a viewer can be positioned anywhere◦ One category of 3D video - the viewer shall be given the flexibility to interactively po-
sition itself
Focus on representations and methodologies to capture and process dynamic scenes
Multiple Streams for Free-viewpoint
School OF INFO. & Mecha., GIST
Generalizing 2D video pixels towards 3D◦ Interfacing imCast to visualization toolkit for 3D
effects Instant virtual freeze-and-rotate effects Put interaction mechanism for mouse/key/time
events
Views in 3D Effects
School OF INFO. & Mecha., GIST
Interactive video service for 3D effects◦ Real-time video display with video rotating and rendering◦ Basic procedure
Source -> mapper -> actor -> renderer -> rendering window◦ Applied mapping and rendering process
Views in 3D Effects
School OF INFO. & Mecha., GIST
Environment◦ 8bit uncompressed HD video is transmitted
Over 1Gbps (MTU: 9180 Bytes) Resolution: 1920 x 960
Uncompressed HD video display in 3D effects◦ Real-time video display with interaction◦ Performance problem
While image actor is working, CPU overhead and user memory increase CPU overhead: 20% to 70~90% User memory: 12.5% to 36.7%
Experiment Result
School OF INFO. & Mecha., GIST
Experimental Snapshots
School OF INFO. & Mecha., GIST
Conclusion & Future Works
School OF INFO. & Mecha., GIST
Conclusion & Future works Conclusion
◦ Enhanced uncompressed HD media transport system Preview video support Networked display interface Bi-directional transport system
◦ Extension of uncompressed HD media transport system (ongoing) Multiple streams for free-view-point Views in 3D effects
Future works◦ Interfacing views in 3D effects to networked tiled display (SAGE)◦ Performance problem (CPU overhead, user memory)
Top Related