The Path from QoS- to QoE-centric Managementof Modern Communication Systems

Jiri Hosek, Ph.D.

Youth School-Seminar, DCCN 2016

RUDN University, Moscow, November 24, 2016

*These slides are intended for educational purposes and include materialpublished by WISLAB group as well as available openly on the Internet.

Lecture’s Content

• Definition of network quality

• Quality of Service (QoS)

• Quality of Experience (QoE)

• Subjective evaluation of modern mobile applications

Evaluation of Service Quality

• Two main approaches how to measure the network service quality

• Objective

• View from the network perspective

• Measuring of traffic parameters and their comparison with pre-defined values

• Different services (applications) have different requirements

• Subjective (Relative)

• View of the end user

• Difficult and time-consuming

• MOS (Mean Opinion Score)

• Substitution of subjective evaluation by mathematical model

MOS Scale

• Used especially for multimedia transmissions

• Numerical expression of the human user’s view on the quality of the network service

Key Network Traffic Parameters

• Dominant measurable parameters of network traffic

• Packet delay

• How long the transmission takes

• Packet delay variation (jitter)

• Difference between the delay of two consequent packets

• Packet loss

• How many packets are lost during the transmission

• Throughput

• Transmission speed (b/s)

• Key Performance Indicators (KPIs)

Uniform Network Traffic Treatment

• As default option in all older network technologies

• Seemingly fair approach

• Each data unit is of the same priority

• No possibility to prefer or defer any type of traffic

• “Best-effort” treatment

• No guarantees, just try

Differentiated Network Traffic Treatment

• Characteristic for modern network technologies

• Traffic classes

• Different requirements on transmission parameters

• Efficient operation of network services

• Identification of data units of different services/flows

• Guarantee of adequate treatment

• 2 basic QoS support mechanisms

• Integrated Service – IntServ

• Differentiated Services – DiffServ

Differentiated Services – DiffServ

• Offers different transmission parameters for different service types

• Every data packet is processed individually

• Fixed network resource sharing

• QoS functions implemented in each router

• Service guarantees

• Service-class based

DiffServ – Traffic Classification and Policing

• Traffic classification into an appropriate category

• Usually based on fields of IP and TCP/UDP header

• Differentiated Service CodePoint (DSCP) field in the IP packet header

• Relative priorities

• Identifies service classes

• Traffic policing

• Ensures, that the incoming traffic satisfies the declared parameters

• Service Level Agreement – SLA

• Network operator / end-user obligations

DiffServ – Packet Scheduling Techniques

• Main packet scheduling mechanisms

• First In First Out – FIFO queuing

• No QoS prioritization

• Priority Queuing – PQ

• Fair Queuing – FQ

• Weighted Fair Queuing – WFQ

FIFO

Priority Queuing

Application of DiffServ Mechanism

• Each network node (router) is performing QoSfunctions individually

• DiffServ routers

• Edge router

• Core router

Subjective Evaluation of Network Services• Customers are demanding constantly increasing quality

of mobile services

• Modern mobile (multimedia) applications are highlydiverse

• Specific requirements and user expectations

• QoS-based evaluation and network control is not enough anymore

• New investments for increasing network quality at all levels.

• Quality of (user) Experience (QoE)

• User eXperience (UX)

• Influenced by many factors

• Technical, socioeconomical, etc.

QoE Key Questions and Goals(from network operator point of view)• How the key network parameters affect the

quality of mobile service perceived by an end user?

• How the type of application affects user‘sevaluation?

• How long are end users willing to wait for specific mobile service and being still satisfied?

• What are the “premium quality” and saturation thresholds for the specific service?

• How to setup and control the mobile network to avoid the over-provisioning?

Experimental Study of QoE in Mobile Networks

• The impact of following variables were investigated:

• End-user device:

• Smartphone Samsung Galaxy Nexus

• Tablet Samsung Galaxy Tab 10.1 inch

• Applications

• Web browsing, file download (DL), file upload (UL)

• YouTube video streaming

• Network and content parameters

• Bit rates (BR) in the range of 32kb/s to 16 Mb/s

• Initial loading delay (connection establishment) in the range of 0 to 11seconds

• YouTube video resolution in the range of 320x180 to 1280x720

• Stalling effects with different duration and repetition

• More than 200 test scenarios evaluated by almost 300 test participants

Experimental Study of QoE in Mobile Networks– Methodology• Laboratory environment

• Completely controllable system

• Test Methods: (rec. ITU-T P.800)

• Absolute Category Rating (ACR):

• Scaling: 5 grade MOS scale:

• 1: Bad

• 5: Excellent

• Acceptability rating:

• Question: Were you satisfied with tested quality?

• Binary answer: YES or NO

Mobile QoE – Automatic Assessment Tool – UserInterface

Questionnaire:Demographic data:Briefing phase:

Web: Sample Results – Quality Rating

• Web browsing, tablet

Web: Sample Results – Acceptability Rating

• Web browsing, smartphone

0

10

20

30

40

50

60

70

80

90

100

0 200 400 600 800 1000 1200

Acc

ep

tab

ility

[%

]

BR [kbps]

0s 1s 3s 5s 7s 11s

Mobile Web QoE – Quality Thresholds

• We specified 3.7 MOS as a quality threshold for “premium quality” service.

• The saturation threshold refers to BR with zero initial loading delay and where quality rating is not more increasing or achieved maximum quality rating or 4.5 MOS.

• This is very practical output in order to avoid quality overprovisioning.

Scenario Notebook Smartphone

Web 256 kbps 256 kbps

File DL 4 Mbps 4 Mbps

File UL 1 Mbps 1 Mbps

Scenario Notebook Smartphone

Web 512 kbps 256 kbps

File DL 4 Mbps 8 Mbps

File UL 4 Mbps 1 Mbps

Youtube: Sample Results (1)

• Quality rating vs. Resolution

• The subjective premium quality threshold (3.7 MOS)

• Resolution [pixels]: 640 x 360

• Saturation threshold (4.5 MOS)

• Resolution [pixels]: 854 x 480

Overprovisioning area

Youtube: Sample Results (2)

• CDF of task’s leaving for infinity loading delay

50 % users wait

more than

104 / 44

seconds

Youtube: Sample Results (3)

• Acceptability rating results vs. single stalling duration – smartphone

1st level

quality

reduction

2nd level quality reduction

Youtube: Sample Results (4)

• Quality rating results vs. multiple stalling – smartphone

1st level

quality

reduction

2nd level quality reduction

Web: Analytical User Experience Prediction

• We need an adequate QoE prediction models based on given results of extensive QoE assessment.

• Analytical tools (regression analysis)

• Calculation of QoE value (MOS) for mobile data services based on input parameters:

• Initial loading delay

• Throughput (bit rate)

ac

cBRc

abMOS

D

2

10

~)(~1

)(

YouTube: Analytical User Experience Prediction

MOS 𝐷, 𝑅 =2,7

1+9,5𝑒0,1𝐷+

10,7

(0,01𝑅)2

𝑒−𝑁0,6𝑆0,2+1

Summary

• Future-generation mobile services are more user- / service-oriented

• QoS-centric evaluation and control of modern wireless networks is not enough

• User-centric / application-centric approach needs to be implemented

• Complex evaluation from network- and user-perspective as well

• QoE modelling is very demanded research topic nowadays

• Not easy task due to high heterogeneity of modern applications and their constant development

• Models need to be updated as users’ expectations are growing

• Standardization in QoE assessment for new mobile services is highly required

Thank you for your attention!

Questions?