QC and Monitoring in Streaming Media WorkflowsInterra Systems - Overview 2 • Interra Systems was...

WWW.INTERRASYSTEMS.COM

Viktor Till

Application Engineer

[email protected]

QC and Monitoring in Streaming Media Workflows

NETS 2019


Content

• Introduction

• Streaming Media Market

• End-to-End QC & Monitoring

• Interra Systems’ Products for Streaming Media Workflows

• BATON - File Based QC

• ORION MPEG-2 TS Monitoring

• ORION-OTT - Content Monitor

• ORION-OTT - Origin Performance Monitor

• ORION-OTT - Edge Performance Monitor

• ORION Central Manager

• Use case

© Interra Systems1


Introduction


Interra Systems - Overview

2

• Interra Systems was founded in 2000

• Industry leader in end-to-end unified quality control (QC), monitoring and analysis solutions for the digital media industry

• Headquartered in the USA; Engineering and R&D Centers in India

• Global, 24/7 customer support for all geographies

• Over 30 channels partners across the globe


Streaming Media Market


Changes on Media Market

© Interra Systems5

• The global video streaming market is growing

• Millennials watching content on their mobile devices and computers

• In the US, 86% of smartphone users watch video content on their phones

• It is not a simply change from traditional viewing devices, but change in viewing style as well as expectations against service availability and quality

• Applied media formats, production and delivery infrastructures have to reflect to these changes to deliver high quality Over-The-Top (OTT) services

• All of these brings new challenges


© Interra Systems

Streaming Media Challenges

• New technologies, standards and formats

• Variety of device types

• Unmanaged networks became an part of delivery platform

• Accessing content anytime / anywhere / (somehow)

• Dramatic increase in content amount

• Changing media production and delivery approaches

• Transition from a linear workflow infrastructure to OTT

Content QC & monitoring is a must for flawless delivery

6


End-to-End QC & Monitoring


Streaming Media Workflow Example

Video Delivery

Live

Origin ServerPackager /

DRM

Live

VOD

Transcoder

Transcoder

Edge ServersReplica Servers

CDN

8

VOD Repository


© Interra Systems

Why End-to-End QC and Monitoring

• Built-in analytic capabilities of given components of the workflow are important and useful to handle specific issues, but do not address all the aspects of monitoring

• Dedicated QC and monitoring system can cover the whole workflow, keeping focus on overall quality and user experience

• Quality expectations are there at all points of the workflow, but experience is affected by issues raised in previous stages too

• Almost impossible to do all quality checks real time, so proper content preparation is important, thus different type of QC and monitoring features need to complete each others too

• An independent QC and monitoring solution from quality experts gives reliability, also ensuring long-term support and maintenance

9


End-to-End QC & Monitoring

Video Delivery

Live


DRM

Live

VOD

Transcoder

Transcoder


CDN

VOD Source

QC

VOD Out

QC

LIVE Source

Monitoring

LIVE Out

Monitoring

Active OS

Monitoring

Active & Passive

Monitoring

Passive OS

MonitoringPassive Client

Monitoring

10


Interra Systems’s Products

for Streaming Media Workflows


© Interra Systems11

• Enterprise-class, file-based QC

• Over 350 customers world wide

• Advantages:

– Most formats supported

– Most comprehensive quality checks

– Enterprise scalability

• Baton+

– Trend and Analytics for Intelligent, data managed file-based workflows

Interra Systems’ Products

• Enterprise-class, real-time

monitoring of content for

issues in:

– Delivery compliance

– Audio/video quality

– Digital Program / Ad

insertion

– Loudness

– And more…

Check Monitor

• Media Analyzers for standards

compliance of all popular

media/DTV formats

• Includes tools for:

– Buffer analysis, video

comparison, YUV Diff, and

more

• Vega HEVC

• Vega ABR

Debug



End-to-End QC & Monitoring for Linear & OTT Workflows

ORION Central Manager (OCM)

Video Delivery

BATON

ORION ORION ORION-OTT ORION-OTT

BATON ABR

Live Transcoder

Transcoder



DRM

Live

VOD

CDN

ORION-OTT

Comprehensive QC for VOD Baseband quality checks Compression artifacts Check for metadata consistency QC for large volume of content is best addressed in off-line, file-based

environment Ensure each video is as per corporate guidelines for format and quality Doing QC at delivery stage is inefficient, impractical, and risky

Problems not addressed here carry through the delivery chain

Checking content delivery related issuesMonitor Live and VOD streamsError alerts for each service in real time HTTP checks – download time, latency etc.Content Availability, Manifest ValidationDRM support – Without this you are only addressing QoS issuesPerformance, scalability are critical factors here

Thorough ABR and QoE impacting checks at the Delivery stage for best of breed System


Enterprise Class Software-based QC tool

TM

BATON - File Based QC


• Performs the most comprehensive quality checks in the industry, utilizing enhanced audio/video processing, computer vision and machine learning technics

• Offers a hybrid QC platform with a combination of automated & manual QC checks on content

• Provides high performance, enterprise wide scalability

• Integrates with leading MAM / DAM / Transcode / Workflow solutions

• Supports Cloud QC

• Provides sophisticated data analytics, review and content correction

• Direct HTTP content location and DRM support for OTT

BATON® - Next-Generation Hybrid QC Solution for File-based Workflow

a



Most comprehensive quality checks in the industry

• Baseband QC Checks

– Video signal levels, RGB color gamut, color bleeding, blotches, blur, defective pixels, black frames, color bars, chromaphase error, loudness, audio noise, etc.

• Compression Artifacts and In-depth Audio / Video Quality Checks

– Blockiness, Pixelation, Moire pattern, Ringing artifacts, language detection, burnt-in subtitle detection, etc.

• File Integrity & Standards Compliance

– Ensure that the file is not corrupt and has been encoded as per the standard to ensure that the downstream tools are able to play it without issues, including ABR compliances

– Very important in the OTT context (multiple devices, form factors / players from multitude of vendors)

• BATON can perform the following additional checks for ABR content:

– Checks to ensure each segment starts with an independent frame

– Checks to ensure that all variants of the content are properly aligned in terms of number of segments, segment duration, total duration and content structure

– Ensure consistency between metadata and actual content properties

BATON QC Overview



Software-based Monitoring Solution For Linear and OTT Broadcast Workflows

TM

ORION – Content Monitoring Suite


ORION Content Monitoring Suite of Products

The ORION Content Monitoring Suite is a complete package catering to various requirements for monitoring linear and OTT broadcast workflows.

The Suite consists of –

• ORION: For monitoring linear broadcast

• ORION - OTT : For monitoring OTT broadcast

• OCM : For enterprise-wide monitoring




Linear Broadcast OTT Delivery Linear and OTT

ORION – Software-based Real-time Content Monitor

ORION- OTT – Software-based Over The Top Content Monitor

OCM – Software-based Enterprise Level Centralized Monitoring Solution

• Primary users:– Broadcasters

– Telcos

– Video Headends

– Video Equipment Providers

• Ensures superior QoS and QoE for Linear Channels

• Checks for:– Comprehensive IP and TS

Monitoring

– TR101290

– Audio-Video Quality

– Ad-Insertion Monitoring

– Closed Captions & CALM Compliance

• Primary users:– OTT Service Providers

– CDN Providers

– OTT Equipment Providers

• Ensures superior QoS and QoE for Live and VOD OTT services

• Checks for:– ABR Package Integrity

– Audio-Video Quality

– Server and Network Performance

– HTTP Responses

• Primary users:– OTT Users

– Orion Users

• Allows viewing QoS and QoE for Linear Channels as well as Live and VOD OTT services at multiple points in the workflow

ORION – Content Monitoring Suite: Highlights


ORION – Software-based Real-time Content Monitor


Enabling video delivery with superior QoS and QoE

Encoded

Content

IP / ASI / SDI

MPEG-2 Transport] • Comprehensive IP & TS monitoring• Audio-video quality checks• HEVC 4K monitoring with QoE• Ad Insertion monitoring• Closed Captions & CALM compliance• Customizable alerts & reports, Historic logs

Local Content

VOD

Troubleshoot Most Formats Closed Caption CALM Compliance Video Quality Ad Insertion

E n s u r e c o n t ent q u al i t y a n d i n t e g r i t y f r o m a u n i f i ed m o n i t or i ng p l a t f or m


ABR Group Synchronization

• IDR-PTS based synchronization

• Encoder Boundary Point (EBP) based synchronization– EBP is a signaling mechanism for taking continuous streams conditioned for adaptive streaming and creating discrete

chunks of decodable content.

• Using Orion, services can be grouped representing different bitrate variants of an ABR asset, then EBP information can be compared across services within an ABR group.



Formats and Protocols Supported


Formats Protocols Supported

Interface IP (v4), ASI, SDI

Protocols MPEG-2 Transport, DVB, ISDB, and ATSC compliance

Audio Formats MPEG-1/MPEG-2 Audio, AAC/AAC+ Audio, Dolby AC-3, EAC-3 Audio

Video Formats SD/HD/UHD Video, HEVC, H.264/MPEG-4 Part 10 (AVC), MPEG2 Video

Embedded Data Closed Captioning (EIA-608, EIA-708, SCTE-20, DIVICOM, NADBS), DVB Subtitle, Teletext, SCTE-27, SCTE-35, XDS, Text Services

Network Transmission Protocol

UDP/RTP, Multicast (IGMP v1/2/3)


ORION-OTT Introduction


Software-based Over The Top Monitoring Solution

to help you deliver streaming video with superior QoS and QoE

ORION-OTT | Content Monitor

ORION-OTT | Origin Performance Monitor

ORION-OTT | Edge Performance Monitor


• 24x7 Proactive content monitoring required for early fault detection and resolution• Catch issues before your viewers start reporting problems

• For LIVE services, content monitoring at Origin is a must• Real-time environment, no offline QC possible• Cannot handoff content for delivery without checking

– Content availability– ABR compliance– DRM protection– Audio/Video quality

21

Why is Active Monitoring required ?


• Active monitoring does not provide any information regarding real-user sessions

• Passive monitoring at end-devices is required to understand issues during playback at client devices

• Together, Active and Passive monitoring methods complement each other to • Provide an end-to-end network visibility• Determine fault sources in the network

22

Active & Passive Monitoring – complement each other


ORION-OTT | Content Monitor




DRM

Live

VOD

Video Delivery

CDN

Content Monitor

• Deployed at Origin Server/CDN• Created for OTT service providers, CDN providers, OTT equipment providers• Extensive set of Checks

ABR Package Integrity Audio/Video Quality

Active Probe for Content Monitoring


• Package Integrity and ABR compliance

– URL validation, Manifest validation

– ABR Compliance• Start time and duration consistency across segments• Segment metadata mismatch• First frame should be a Key Frame• And more…

• Encryption

– Supports AES-128, CENC (MPD, HLS, HSS), SAMPLE-AES

– DRMs – Widevine, PlayReady, Fairplay, Verimatrix, BuyDRM, Content Key mapping for other DRMs

• Monitoring Content Quality (QoE)– Video Quality (Blockiness, Black frames, Freeze frames)– Audio Quality (Loudness (CALM, ITU, EBU), Level, Silence)

– Conformance checking– Closed Caption/Subtitle Checks

24

ORION-OTT | Content Monitor - Supported Checks


• In-depth monitoring of Live and VOD OTT services

• Intuitive Web-based User interface using intuitive dashboard and drill-down graphs

• Alarms and Triggers for Actions and Debug– Provides error categorization according to severity levels – Major, Minor, Critical

– Provides different alerting mechanisms – E-mail, SNMP; Real-time alarms for quick decision making

– Triggered and manual recordings for profiles and LIVE content

• Rich Reports and Trend Analysis– Provides graphical representation of content delivery quality

– Provides drill-down monitoring views for assets and services under test

– Provides alerts and data analytics for all tests run over last 60 days

– Global Live Alerts page for LIVE Services

• REST APIs for third-party integrations


ORION-OTT | Content Monitor - Features



ORION-OTT | Content Monitor – Feature Examples


Formats and Protocols Supported


Formats Protocols SupportedPackages Apple HLS, MPEG-DASH, HSS, Adobe HDS

Wrappers MP4, MPEG-2 TS

Audio Formats MPEG, AAC, Dolby AC-3 and EAC-3

Video Formats H.264 / AVC, H.265 / HEVC

Embedded Data Formats Closed Captioning (EIA-608, EIA-708, SCTE-20, DIVICOM, NADBS), DVB Subtitle, WebVTT, TTML, SCTE35, ID3 Metadata, External XML Subtitles

Network Transmission Protocol HTTP / HTTPS, RTMP

Checks Content Integrity Checks, Download Failure and Delays, HTTP Errors, Audio/Video Quality Checks

DRM/Encryption All Encryption schemes (CENC, AES-128, SAMPLE-AES) supported by Google Widevine, Microsoft PlayReady, Apple FairPlay, Verimatrix and BuyDRM




ORION-OTT | Origin Performance Monitor


DRM

Live

VOD

Video Delivery

CDN

Origin Performance Monitor

• Deployed at Origin Server

• Monitors performance based on:

Average response times and bandwidth served

Number of requests served, total bytes served

Server availability - Requests served per requests received

HTTP response codes

• Performs manifest conformance checks

Passive Probe for Origin Server Performance Monitoring




ORION-OTT | Edge Performance Monitor


DRM

Live

VOD

Video Delivery

CDN

Edge Performance Monitor

• Monitors QoS & QoE to understand recurrent issues, their causes and the impact of anomalies on your customer base in real time

• Analytics & business intelligence; Collect user data, derive data consumption trends• Monitors real client user experience: Startup time, buffering ratio, number of stalls, play failures• Data collected from the end-devices can be correlated with active monitoring results using OCM

Passive Probe for Client Device Monitoring (powered by NPAW YOUBORA)


Transcoder

Transcoder



DRM

Live

VOD


ORION Central Manager (OCM)

ORION Central Manager

Video Delivery

BATON

ORION ORION

BATON ABR

Live

CDN

ORION-OTTContent Monitor

ORION-OTTOrigin Performance Monitor

ORION-OTTEdge Performance Monitor

• Enterprise-wide visibility

• Single point Control and Monitoring for multiple Linear and OTT probes

• Workflow monitoring for locating error source

• Distributed architecture for flexibility and efficient bandwidth management

Centralized Management


Use Case



Complete QC and Monitoring Solution for Pay TV Operator

• Pay TV operator

• Offers almost 200 broadcast TV channels, 40-50 in HD, delivered via DTH, IPTV

• Then introduced OTT service, which successfully provided subscribers with VOD content and live linear programs

• Later seeing subscriber churn, new competitors are gaining wider audience, like Netflix, etc.

• Already had a Baton system for file-based QC

• Decided to increase service quality by introducing proper end-to-end QC and monitoring solution for the OTT delivery

• Project included: file-based QC, linear monitoring, OTT monitoring


• They receives content from various content providers via Aspera, FTP, Dropbox, HDD etc.

• Media Formats and Processing– File formats: MXF, MOV (Video: H264, MPEG 2, Prores Audio: AAC, AC3, LPCM)– Streaming Formats : MPEG2 TS, HLS, MPD– DRM: Play Ready, Widevine, Fairplay, Verimatrix

• QC and Monitoring checks– Deep A/V QC checks like Video Dropout, Motion jerks etc.– At monitoring they do both QoE and QoS

• Combinational Video and Audio quality Checks• No Monitoring at Blackout Periods

• They broadcasts contents through Satellite and OTT services

• 2 Sites

Video and Content Workflow

© Interra Systems32© INTERRA SYSTEMS CONFIDENTIAL


BroadcastOperations

The Workflow

BATON

Encode / TranscodeOrigin ServerPackager /

DRM

Live

VOD

CDN

Video Delivery

STBEncoder

Video Delivery

IP IRD

Multiplexer

Live

ORION-OTTORION

Case Study33


• Baton installed on physical servers

– Integrated with MAM and limited use of built-in automation (Smart Folders) also exists

– NMS : Integrated via APIs

• Orion OTT

– Installed on AWS with Elemental Servers

– Closed Caption data is available via separate URLs

– Integrated with in-house management servers

– Orion OTT fetches details of Blackout periods via APIs

• Orion

– Installed on physical machine

Integration



• 20+ parallel file based QC

• Monitors around 120 live services– Each live services contain 6-9 variants

– Avg Segment duration 5 sec

– Mix of HD and SD

• Monitors around ~300 VOD assets per day

– Each live services contain 6-9 variants

– Mix of HD and SD

– Segment duration 6 sec

– Duration of each asset (1 minute to 2 hours)

Performance



Quality Matters. Now, More Than Ever.

Reach us at:

Interra Systems, Inc.1601 S. De Anza Boulevard, Suite 212

Cupertino, CA 95014, USA

www.interrasystems.com

QC and Monitoring in Streaming Media WorkflowsInterra Systems - Overview 2 • Interra Systems was...

Documents

Transcript of QC and Monitoring in Streaming Media WorkflowsInterra Systems - Overview 2 • Interra Systems was...