Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.
-
Upload
shana-mclaughlin -
Category
Documents
-
view
220 -
download
0
Transcript of Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.
![Page 1: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/1.jpg)
![Page 2: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/2.jpg)
Measuring Quality of Experience for Successful IPTV Deployments
Dr. Stefan Winkler
![Page 3: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/3.jpg)
Outline
• Challenges– Digital Video Quality Issues– Traditional Measurements (QoS) vs.
Quality of Experience (QoE)
• Possible Solutions– QoE Measurement Approaches– End-to-end QoE Management
• Conclusions
![Page 4: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/4.jpg)
Digital Video Challenges
Demanding traffic profiles High bandwidth streams
High traffic volumes
Live, VOD
Network effects Video impacted heavily with
minor network impairments
Multi-vendor network complicates diagnosis / troubleshooting
High end-user expectations Defined with decades of history
Grow rapidly with HD
Low tolerance for poor quality
New architectures Sensitive video processing
devices create possibility for various impairment sources
Ad-insertion, middleware
Service quality degradations
Difficult diagnosis, troubleshooting
Rising management and
OPEX costs
Higher customer churn
![Page 5: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/5.jpg)
What Drives End-Users
Source: MRG 2007 IPTV Video Quality Survey, available at http://qoe.symmetricom.com
![Page 6: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/6.jpg)
Service Providers View
Source: MRG 2007 IPTV Video Quality Survey, available at http://qoe.symmetricom.com
![Page 7: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/7.jpg)
Service Providers’ View
7Source: MRG 2007 IPTV Video Quality Survey, available at http://qoe.symmetricom.com
![Page 8: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/8.jpg)
Sources of Video Issues
Consider all elements for true end-to-end solution
![Page 9: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/9.jpg)
Compression Artifacts
Original MPEG-2 H.264
![Page 10: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/10.jpg)
PSNR vs. QoE
Same amount of distortion (PSNR) – different perceived quality
Understand & model human vision system
![Page 11: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/11.jpg)
QoS vs. QoE
• Quality of Service– Network-centric– Delay, packet loss, jitter – Transmission quality– Content agnostic
• Quality of Experience– Content impairments – Blockiness, Jerkiness, …– End-user quality– Application driven
QoS QoE
![Page 12: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/12.jpg)
Same network impairmentsPacket Loss: 1%
Delay: 10msJitter: 50us
Bandwidth: 500kbps
Different perceived quality!
QoS vs. QoE
![Page 13: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/13.jpg)
MDI vs. QoE
• Media Delivery Index (MDI)• MDI consists of two metrics:
– Delay Factor (DF)– Media Loss Rate (MLR)
• MDI limitations:– MDI assumes constant bit rate (CBR) traffic– MDI does not consider video payload or content– MDI values are not intuitive– MDI doesn’t correlate with video quality
![Page 14: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/14.jpg)
MDI vs. QoE
0
1
2
3
4
5
V-F
AC
TO
R
0
500
1000
1500
2000
2500
3000
Med
ia L
oss
Jitter1 to 50ms
Packet Drop1 in 500 periodic
Packet Drop1 in 500 poisson
Packet Drop1 in 500 uniform
Packet Drop1 in 500 and Jitter
Duplicate Packets1 in 10
Duplicate Packets1 in 500
Reordering Packets1 in 500
Med
ia L
oss
MO
S
![Page 15: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/15.jpg)
QoS/QoE Cycle
Desired QoE
Perceived QoE
Targeted QoS
DeliveredQoS
End-userService
provider
Alignment gap
Perception gap
Value gap Execution gap
Adapted from ITU-T Rec. G.1000 and COM12–C185–E
![Page 16: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/16.jpg)
Outline
• Challenges– Digital Video Quality Issues– Traditional Measurements (QoS) vs.
Quality of Experience (QoE)
• Possible Solutions– QoE Measurement Approaches– End-to-end QoE Management
• Conclusions
![Page 17: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/17.jpg)
Full-Reference Approach
• Comparison of individual video frames• Offline analysis (capture is required) – lab applications• High detail and accuracy• Alignment procedure
Sender Receiver
Full reference information
Full Ref. QualityMeasurement
![Page 18: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/18.jpg)
No-Reference Approach
• Non-intrusive, in-service measurement• Real-time monitoring applications• No alignment required
Sender Receiver
No-Ref. QualityMeasurement
![Page 19: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/19.jpg)
• Monitoring applications• Correlation of content and network impairments• Encrypted environments
Sender Receiver
Reduced Ref.Measurement
Feature Extraction
Feature Extraction
Reduced-Reference Approach
![Page 20: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/20.jpg)
Content & Network Metrics
20
"Vision is the most highly developed of the human senses, so people are even more sensitive to flaws in video images than, say, the sound of a telephone conversation.”
Ken Wirt, Cisco Vice President Consumer Marketing, Jan 2008
(Correlation Engine)
![Page 21: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/21.jpg)
![Page 22: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/22.jpg)
![Page 23: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/23.jpg)
• Contrast perception– Visibility of different patterns– Frequency dependencies
• Masking effects– Interaction of content and impairments– Texture, edges, luminance– Spatial and temporal masking
• Color perception
Spatial frequency [cpd]Temporal frequency [Hz]
Sen
sitiv
ity
Vision Modeling
Masker contrast
Visibility threshold
Targ
et
contr
ast
Maskingcurve
Thresholdwithoutmasker
![Page 24: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/24.jpg)
End-to-end QoE
Deep Content Analysis(pixel by pixel)
Source content and encoder / transcoder validation
Human Vision System Model
VideoQualityReports
Content Impairments:• Blockiness, blur• Jerkiness• Freeze/black frame• Noise, Color
Network Impairments:• Loss• Delay• Jitter• Bandwidth
Content Stream Analysis:• PES inspection• PCR jitter etc.
Deep Content Analysis (bitstream)
Detect content impairmentsDeep inspection to associate content to timestamps (eg: TS1 = I-Frame) Network (header or stream) Analysis
Detect QoS issuesContent analysis where possible (unencrypted)Inspection of QoS to associate timestamps to impairments (eg: TS1 = Packet Loss)
Q-Advisor
Correlation Engine
TS1 = I-Frame
TS1 = Packet Loss
Packet Loss -> I-Frame
![Page 25: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/25.jpg)
IPTV QoE Management
25
Issue Possible Causes
Blockiness Encoder Transcoder Network Loss
Blur Camera (focus) Encoder Transcoder STB (bad filtering)
Freeze Frame, Jerkiness Encoder (dropped frames)
Network loss Bad synchronization
Black Screen, Blue Screen
No Video Signal (source)
Ads not inserted Major network loss
Color Encoder Camera Transcoder
Video Noise (analog noise)
Camera STB
Noise (digital) Encoder Transcoder
Audio Microphone Encoder (bad mono stereo encoding
Encoder (lip sync) STB
1.0
1. Understand the Service Is there an issue?Does it matter?
1. Understand the Service Is there an issue?Does it matter?
2. Understand the ProblemWhat does the customer see?What is the exact cause?
2. Understand the ProblemWhat does the customer see?What is the exact cause?
3. Understand the SolutionWhat is the impairment source?
3. Understand the SolutionWhat is the impairment source?
1
2
3
4
5
Very Annoying
Annoying
Slightly Annoying
Perceptible
Imperceptible
![Page 26: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/26.jpg)
Conclusions
• QoE is application-driven– Measure both network and content impairments
• QoE is user-oriented– Measure how end-user perceives service issues
• End-to-end quality measurement– Cover different impairment sources– Identify problem causes
![Page 27: Measuring Quality of Experience for Successful IPTV Deployments Dr. Stefan Winkler.](https://reader036.fdocuments.us/reader036/viewer/2022062314/56649d985503460f94a8245f/html5/thumbnails/27.jpg)
Stefan [email protected]
Company:qoe.symmetricom.com
Further Reading:stefan.winkler.net/book.html
Contact Info