Overview of videoconference clients in EVO

VIC [ VIdeoConferencing Tool ]

• VIC is default video client of EVO system

• based on open source project of OpenMash consortium

• the application is written in C++ and OTcl programming languages

• development is focused on following points:

- video codecs (H.261 and H.263)

- performance – IPP (Intel

Performance Primitives) Libraries

- communication interface for external applications

- user’s comfort and GUI

VIC [ H.261 - encoder ]

• new encoder is fully compliant with the H.261 standard

• old encoder in VIC used only INTRA-mode – compression is based on reducing of intra frame spatial redundancy

• added was INTER-mode with inter-frame motion estimation and compensation - difference between two subsequent video frames is encoded - compression is based on reducing of inter frame temporal redundancy

• reduces bitstream (higher compression) - allows to use higher framerate and quality with the same bitstream

frame n frame n+1 difference

VIC [ H.261 - decoder ]

• improvements done on decoder side have influence mainly on compatibility with H.323 hardware devices

• RTP part was changed to avoid artifacts in decoded video from some H.323 clients and to fix bad positions of macroblocks in video stream from MCUs and Polycom VSX

• bug which causes skipping of the last macroblock in RTP packet was fixed (avoids artifacts in decoded video)

MCU Codian - before MCU Codian - nowPolycom VSX7000 - before Polycom VSX7000 - now

VIC [ H.263 codec ]

• compare to H.261, there are two significant differences: resolution of video signal (standardized formats up to 16CIF and custom formats up to 2048x1152) and level of compression and quality

PRESENT STATUS

• currently is implemented only INTRA-frame mode

• development is focused on compatibility with H.323 devices, improvement of codec performance, capturing video with larger resolutions

• Desktop Sharing Mode - VIC is able to capture PC desktop and send it like H.263 video

• one application as video client and desktop sharing client

• advantage of existing infrastructure - signal will pass through network limitations like NAT

• higher compression of the signal using H.263 capabilities

VIC [ H.263 - Desktop Sharing ]

H.261 CIF(352 x 288)

H.261 CIF(352 x 288)H.263 XGA

(1024 x 768)

H.263 VGA(640 x 480)

VIC [ Users Comfort ]

• the same version for Windows, Linux and Mac OS X with the same features

• new GUI – more user friendly, intuitive and easier to customize. All technical settings have been removed from the main settings interface window

• Speaker Mode - VIC is able to identify and highlight video of current speaker - information about who is the current speaker is generated by reflector and carried by specific RTCP packet (SDES)

• new video display modes which automatically arrange video windows on the desktop

- all videos in CIF size

- all videos in QCIF size

- only the video of current speaker in CIF size.

- current speaker in CIF size plus all the other videos in QCIF size

VIC [ Users Comfort ]

• possibility to save all settings from GUI including position on the desktop

• multiple display support for Windows systems - user can choose display, on which the videos from participants will be displayed

• Still Image Mode – allows transmitting of static picture instead of video from camera

• Minimize Mode – hides small video windows on the main VIC window, saves space on the desktop

• and many others improvements like using FireWire cameras on MAC without the IOXperts driver and freezing of video

RAT [ Robust Audio Tool ]

• RAT is default audio client of EVO system

• based on open source project of University College London

• the application is written in C and Tcl/Tk programming languages

• new audio codec was added (G.722) for compatibility with H.323 devices

• priority of development was to unify source code for all supported platforms (Windows, Linux, Macintosh) – required new audio interface for MAC OS X

• ongoing development is focused on splitting audio processing and windows management into two separated threads, what increase performance and stability of the application

Overview of videoconference clients in EVOVIC and RAT onWINDOWS

(17 participants)

12

VIC on LINUX(17 participants)

13

On Mac OS XVIC and RAT on MAC OS X

(9 participants)

Communication Interface[ e-Learning scenario ]

Teacher

Student

Student

Student Student

Communication Interface[ localhost monitoring scenario ]

EVO Client

Localhost monitoring module LISA

RAT

VIC

ALARM: CPU overloaded

Response: decrease framerate, quality, resolution, change video codec, turn off display of big video windows, …

PocketVRVS – the mobile solution

• PocketVRVS is new VRVS videoconferencing client developed for Pocket PC platform

• application is written in C++ and Windows CE API

• it supports H.261 video standard and G.711 (µ-Law) audio standard

• user can choose CIF sized still image to be transmitted (JPEG format)

• the logic of video encoding was adapted to encoding only still images - the process was optimized to reach the compromise between framerate, CPU usage and fast update of video on remote clients

PocketVRVS – the mobile solution

PocketVRVS – the mobile solution

• utilization of INTEL Performance Primitives and Intel compiler for optimization of application performance (Discrete Cosine Transform, color model conversions RGB to YUV)

• received and decoded is only video of current speaker – this functionality is implemented on reflector’s side

• PocketVRVS works inside networks with network address translation (NAT/NAPT)

• new web based interface for Pocket PC clients (designed for smaller screens) includes full functionality available for desktop clients (meeting scheduler, booking, …) and also possibility to connect H.323 device

• installation package of PocketVRVS application with multi- language OS support is intended directly for Pocket PC platform, desktop computer is not needed

H.323 connection interface

PocketVRVS – the mobile solution

PocketVRVS – the mobile solution

PocketVRVSweb interface

PocketVRVS application GUI

DocumentationControl PanelTalk, Listen, Send

and Receive controls

Download PocketVRVS

package

Booking

Connect to any ongoing meeting

Video ScreenRemote Video, Own Video in PIP window

Session InfoVirtual Room,

Participant’s name, …

Program MenuConnection and Session Settings

Conclusions

Work on new video codecs H.263, H.264 in VIC• higher quality and compression of video signals with higher resolution

Implement software echo canceller into RAT• improvement of the quality of transmitted audio signals using intelligent algorithm for echo and noise suppression

Merging VIC and RAT into one application• efficient processing of incoming video and audio streams by one application (common GUI) + video and audio synchronization

Interconnection between VRVS/EVO, RAT and VIC• support of various videoconferencing scenarios (education, tutorials, meetings); reaction on alarms

Support of live video transmission from PocketVRVS send real video from cameras for Pocket PC platform

Thank you for your attention

Additional Slides

• Block scheme of H.261 encoder with highlighted newly developed parts

• Diamond searching algorithm

• Searching area +/- 7

• SAD sum of absolute differences

VIC [ H.261 - encoder ]

Median filterdefined in H.261

standard

• better compatibility with H.323 devices – solutions on encoder side:

• equal RTP packet size (via MBA stuffing) – avoids packet reordering and “freezing” of decoded video

• fixed H.261 payload RTP headers

VIC [ H.263 codec ]

• compare to H.261, there are two significant differences:

Resolution of video signal

• standardized picture formats: sub-QCIF (128x96), QCIF (176x144), CIF (352x288), 4CIF (704x576), 16CIF (1408x1152)

• custom picture formats: resolution up to 2048 x 1152

Level of compression and quality• Arithmetic Coding instead of VLC - significantly fewer bits produced

• Unrestricted Motion Vector Mode - larger vectors and vectors can point outside the picture

• Advanced Prediction Mode - 4 8x8 vectors instead of one 16x16 vector - results in less blocking artifacts

• PB-frames mode - two pictures encoded as one unit

• enhanced error resilience capabilities

• half pixel precision used for motion compensation + many others

VIC [ The Performance ]

• IPP (Intel Performance Primitives) libraries were used for better performance of video codecs in VIC

• IPP allows to replace a part of code, which is heavy on computation – DCT, Motion compensation,…

• automatic processor type detection

• available for Linux and Windows

• Intel compiler optimizes compilation for target processor

• implementation of IPP functions in some H.26x encoder and decoder parts leads to significant decrease in CPU usage

Communication Interface

• interface allowing to receive commands from external applications - like VRVS/EVO client - for interaction with VIC and RAT during runtime

• communication interface uses existing RTCP channel of VIC and RAT applications – doesn’t need additional channel

• remote commands are carried by specific Application Defined (APP) RTCP packets that are understood and subsequently processed only by VIC and RAT applications and ignored by the others

• useful in conference scenarios, when privileged user can control audio and video clients of other participants (e-Learning: teacher vs. students)

• VRVS/EVO client can take appropriate action as reaction on alarm notifications when monitored parameters (system or network) will go beyond a preset threshold