APNOMS Special Session(Submission-Hyunmin Lim)Hyunmin_Lim).pdfKT Network Technology Laboratory...

KT Network Technology Laboratory Hyunmin Lim

22Smart Network Solution Leader

Contents

2

3 IPTV Service Quality Indicator

4 ISQM System

Holistic Approach to Define SQI

Conclusion

1 Quality Management

5


1. Quality Management

▣ Quality DefinitionsThe ISO 8402 vocabulary defines quality as “The totality of features and characteristics of a product or services that bear on its ability to satisfy stated and implied needs.”Edwards Deming described it as “Striving for excellence in reliability and functions, by continuous improvement, supported by statistical analysis of causes of failures.”Dr Joseph Juran defined quality as “fitness for use” and an improvement process of “journey from symptom, to cause, to remedy.”James Martin described quality as “being on time, within budget, and meeting user needs.”Phil Crosby defined quality as “Quality is Conformance to requirements” and associated it with the statement “Quality is free.”Bill Perry of Quality Assurance Institute described quality as “High levels of user satisfaction and adherence to requirements.”

▣ Thoughts about QualityQuantitative data are necessary to measure the continuous quality improvement activity.Until recently, corporations have not recognized the importance of quality. However, a new attitude has emerged – quality first among the equals of cost and service. To sum it up, the customer wants value.Quality improvement is not limited to the conformance of the product or service to specifications; it also involves the inherent quality in the design of the system.There is no acceptable quality level because the customer’s needs, values, and expectations are constantly changing and becoming more demanding. Realistically, the customer doesn’t buy a specification; the customer buys the product or service to fulfill a need.Peter Drucker once said, “Customers don’t buy products, they buy results.”Just meeting a customer’s needs is not enough; the organization must exceed the customer’s needs.

▣ Service Quality Management (TMF GB 921)Service quality management encompasses monitoring, analyzing and controlling the performance of the service perceived by customers.



▣ Paradigm ShiftConverged Services require Converged Manageability

Integrated Quality ManagementIn order to maximize customer’s experience

Integrated ManagementIntegrated ManagementSeparate ToolsSeparate Tools

Converged services include applications (“Triple Play”, VoD, music)

Business / service alignment(deploy / monitor / manage / SLM)

Management Convergence

ServiceInfrastructure

AppApp

AppAppAppsApps

Network Convergence

Services Convergence



▣ How do I minimize four QoS viewpoints Gap?

Focus on customer expectations or requirements, often referred to as the voice of the customer.

A considerable amount of effort should be put into research to determine customer expectations.

Top-Down Approach (QoE QoS NP) and System Dynamics Research should be used in order to extract SQI.

Analyze correlation between service quality indicators and improve customer’s satisfaction.

The four QoS Viewpoints or the quality cycleThe four QoS Viewpoints or the quality cycle

From “ITU-T G.1000”

Customer’sQoS

requirements

QoS offered byService provider

Execution gap

Value gap

Alignmentgap

Perceptiongap

Customer Service Provider

Future

Historic

Customer’s perception

of QoS

QoS achieved by service provider



▣ How do I maximize Quality of Experience?

QoE (Quality of Experience) is a term to allow for subjective as well as objective measures of QoS, performance and all aspects of the interaction (experience) with the service or product. (from “TMF SLA Management Handbook”)

QoS (Quality of Service) is the collective effect of service performance which determine the degree of satisfaction of a user of the service. (from “ITU-T E.800”)

NP (Network Performance) is the ability of a network or network portion to provide the functions related to communications between users. (from “ITU-T E.800”)

NPQoE QoS

CoreNetwork

AccessNetwork

AccessNetwork

CPN CPNTE TEUser User

Network Performance

Quality of ServiceQoE QoE

Man-Machine Interface


Network Interface

Network Interface

CoreNetwork

AccessNetwork

AccessNetwork

CPN CPNTE TEUser UserCore

NetworkCore

NetworkAccess

NetworkAccess

NetworkAccess

NetworkAccess

NetworkCPNCPN CPNCPNTE TEUser User

Network Performance

Quality of ServiceQoE QoE



Network Interface

Network Interface


2. Holistic Approach to Define SQI

▣ Holistic Approach

Integrated Service Quality Integrated Service Quality Indicators in the viewpoint of Indicators in the viewpoint of CustomersCustomers

Integrated View

Reflection of CustomerReflection of Customer’’s s Requirements and TrendRequirements and Trend

OneOne--stop Service Quality Viewstop Service Quality Viewof Customers &of Customers & Quality Ticket Quality Ticket Issue for CooperationIssue for Cooperation

RealReal--time Monitoring, time Monitoring, Analyzing, Evaluating & Analyzing, Evaluating & ResponseResponse

Service Quality Indicator Category(Intrinsic, additional, supportive)

77 Cell Matrix Methodology

KQI Development Methodology

SMA Methodology

QFD Methodology

KANO Model Methodology

Various

SQI

Development

Methodologies

Quality Indicator

Characteristics



▣ SQI Category

“Customer’s Perceived Service Quality Index”

Composed of Three Service Quality Indicator Categories (Intrinsic, Additional, Supportive )

Indicators and Weight are changeable depending on target customers, timing etc.

Customer’s Perceived Service Quality

Customer’s holistic experience & recognit ion of service qual i ty

Service Satisfaction of Overall Service Quality including Subscription, Usage, Release etc.

Customer’s Perceived

Service Quality

IntrinsicQuality

(Network Quality)

AdditionalQuality

SupportiveQuality= ++

Holistic ViewOn Customer’sPerception of

Service

Overall NetworkQuality for ServiceDelivery such as

Delay, Jitter& Loss etc.

AdditionalFeatures

in the viewpoint ofContents,

User

Friendliness etc.

SupportiveFeatures

in the viewpoint ofSubscription,

Maintenance etc.

= ++



▣ 77 Cell Matrix Methodology

Network/service 11management by customer

Charging & Billing 10

ConnectionRelease 9

InformationTransfer 8

ConnectionEstablishment 7

Connect-ionquality

Cessation 6

Repair 5

ServiceSupport 4

Servicemanage-ment

Service function

Alteration 3

Provision 2

Sales andPre-contract 1activities

Speed1

Accuracy2

Availability3

Flexibility7

Simplicity6

Security5

Reliability4

Network/service 11management by customer

Charging & Billing 10

ConnectionRelease 9

InformationTransfer 8

ConnectionEstablishment 7

Connect-ionquality

Cessation 6

Repair 5

ServiceSupport 4

Servicemanage-ment

Service function

Alteration 3

Provision 2

Sales andPre-contract 1activities

Speed1

Accuracy2

Availability3

Flexibility7

Simplicity6

Security5

Reliability4

Service quality

Criteria

From “Richtersand Dvorak, 1988”

From “FITCE study, 1993”

From “ETSI ETR003, 1994”

From “ITU-T G.1000, 2001”



▣ KQI Development MethodologyApproach

Step 1: Service ScenariosStep 2: Analyze TimelineStep 3: Identify Service TopologyStep 4: Develop Transaction MatrixStep 5: Develop KQIStep 6: Identify Measurements

From “SLA Management Handbook (TMF), 2005”

Customer Experience Timeline



▣ SMA Methodology

ObjectiveObjective

SMA Objective & Approach SMA process applied in Services

Top-down Approach

User perspective analysis

Utilize Best Practice and Global Standards

Perform Top-Down and Bottom-up approach together appropriately and define optimum KQIs and KPIs

Bottom-upApproach

SMASMA

Analyze available data sources and real world data

SM

A P

rocess –

step b

y step

SMA Constant Insight

1 Analyze Service Scenarios

KQI 3

KQI 2

• Definition• Target• Periodic

KQI 1

Set up the most appropriate KQIs and KPIs

Combined KQIs

Categorize end-user quality factors from Best Practice

Analyze Service Delivery Architecture

Identify Candidate KQIs and KPIs from Best Practice

Analyze available Data Source

Gap analysis based on Best Practice

23

4

67

8Develop KQI formula by analyzing real world data

Define Service Model Architecture

5

SMA Lifecycle

1 2 3 4 7 5 6 8



▣ QFD MethodologyQFD (Quality Function Deployment) is described as “method to transform user demands into design quality, to deploy the functions forming quality, and to deploy methods for achieving the design quality into subsystems and component parts, and ultimately to specific elements of the manufacturing process.”

QFD helps transform customer needs (the voice of the customer) into engineering characteristics for a product or service, prioritizing each product of service characteristic while simultaneously setting development targets for a product or service.The goal of QFD is not only to meet as many customer expectations and needs as possible, but also to exceed customer expectations.

The primary planning tool used in QFD is the house of quality

Measurable Characteristics (EC)5

CustomerRequirements

(CA)

1

RelativeImportanceOf CA

2

CA Importance

Weighting

OtherCAMeasures

(Improvement

Factor, Sales

Point etc.)

3 4

CA -ECInterrelationship

EC -ECInterrelationship

8

7

EC Technical Priorities6

EC Competitive benchmarks9

Targets of EC10



▣ KANO Model Methodology

• A theory of product development which classifies customer preferences into five categories.

• The model represents three major areas of customer satisfaction.- The first area of customer satisfaction, represented

by Performance line, represents explicit requirements.- The second area of customer satisfaction represents

innovations, as shown by Excitement line.- The third and most significant area of customer satisfaction

represents unstated or unspoken requirements, as shownby Basic line.

From “NoriakiKano, 1984”

Excitement FactorsThe factors that increase customer satisfaction if delivered but do not cause dissatisfaction if theyare not delivered. These factors surprise the customer and generate ‘delight’.

Quality Elements Quality Attributes

Performance FactorsThe factors that cause satisfaction if the performance is high, and they cause dissatisfaction if thePerformance is low. Typically these factors are directly connected to customers’ explicit desires.

Basic FactorsThe minimum requirements which will cause dissatisfaction if they are not fulfilled, but do not causeCustomer satisfaction if they are fulfilled (or are exceeded).

Indifferent Factors The customer does not care about this feature.

Reverse Factors The reverse of this product feature was expected by the customer.


3. IPTV Service Quality Indicator

▣ Holistic Approach to Define IPTV SQI

Integrated Integrated

SQISQI

in the in the viewpoint of viewpoint of CustomersCustomers

Integrated View


IPTV Service Quality Feature Survey

Customer Satisfaction Analysis

Quality Management Scenarios

Quality Indicator

Characteristics

SMA

Methodology

QFD

Methodology

KANO

Methodology

Analyze Service Scenarios

Analyze Service Delivery Architecture

Identify Candidate KQIs and KPIs

Analyze Data Sources

Gap Analysis

Identify Candidate QoE, QoS and NPBased on ITU-T, ATIS, DSL Forum etc.

ID Analysis based on Delphi method

HOQ-I & HOQ-II: QoE QoS NP

System Dynamics using Causal LoopDiagram

ISQM SystemISQM System



▣ IPTV SQI Extraction by SMA MethodologyService Usage Scenario Analysis

Service Delivery Architecture Analysis

Access Network Access Service Transfer Data Release ServiceSubscribe Ability to access the

VoD subscription server

Ability to access the VoD subscription server subscription menu page

Ability to send VoD subscription details. Ability to receive VoD subscription confirmation

Exit VoD subscription web page

Browse Ability to access the VoD metadata server

Ability to access the VoD metadata server main menu page

Ability to download the list of available videos

Exit VoD metadata web page

Select Ability to access the VoD content server

Ability to initiate transfer of selected video stream (e.g. RTSP: PLAY)

Unicast streaming of selected video content

Terminate VoD data stream

View Ability to access the VoD content server

Ability to maintain transfer of selected video stream

Unicast Streaming of selected video content with acceptable speech and picture quality



VoD subscription server

Ability to access the VoD subscription server subscription menu page

Ability to send VoD subscription details. Ability to receive VoD subscription confirmation

Exit VoD subscription web page

Browse Ability to access the VoD metadata server

Ability to access the VoD metadata server main menu page

Ability to download the list of available videos

Exit VoD metadata web page

Select Ability to access the VoD content server

Ability to initiate transfer of selected video stream (e.g. RTSP: PLAY)

Unicast streaming of selected video content


View Ability to access the VoD content server

Ability to maintain transfer of selected video stream

Unicast Streaming of selected video content with acceptable speech and picture quality



broadcast TV subscription server

Ability to access the broadcast TV subscription server subscription menu page

Ability to send broadcast TV subscription details. Ability to receive subscription confirmation

Exit broadcast TV subscription web page

Browse Ability to access the broadcast TV metadata server

Ability to access the broadcast TV metadata server main menu page

Ability to download the list of available channel services

Exit broadcast TV metadata web page

Select Ability to access the broadcast TV content server

Ability to select channel and change channels quickly (IGMP: Join and Leave)

IP Multicast Streaming of selected channel content

Terminate broadcast data stream

View Ability to access the broadcast TV content server

Ability to maintain transfer of selected channel service

IP Multicast Streaming of selected channel content with acceptable speech and picture quality

Terminate broadcast TV data stream


broadcast TV subscription server

Ability to access the broadcast TV subscription server subscription menu page

Ability to send broadcast TV subscription details. Ability to receive subscription confirmation

Exit broadcast TV subscription web page

Browse Ability to access the broadcast TV metadata server

Ability to access the broadcast TV metadata server main menu page

Ability to download the list of available channel services

Exit broadcast TV metadata web page

Select Ability to access the broadcast TV content server

Ability to select channel and change channels quickly (IGMP: Join and Leave)

IP Multicast Streaming of selected channel content

Terminate broadcast data stream

View Ability to access the broadcast TV content server

Ability to maintain transfer of selected channel service

IP Multicast Streaming of selected channel content with acceptable speech and picture quality

Terminate broadcast TV data stream

VOD User Actions Mapped to Service Delivery Phase Broadcast TV User Actions Mapped to Service Delivery Phase

SER

Ntopia Switch

UTPSiSi

FES

SiSi

copper

L3 Switch

PE Router

STB

VDSL/IP-ADSL DSLAM

STB

SiSi

Media Switch

VoD Server(H.264)

OLT

L3 switch

STB

FTTH

CP

DP

PP

Broadcast Server

EPG Server

CAS/Server

HDS Authentication Server

Premium Network

Kornet Network

HeadEnd Core IP BackboneCustomer Premises

Transmission Network Access NetworkHeadEnd Core IP BackboneCustomer Premises

Transmission Network Access Network

Points of Interest



▣ IPTV SQI Extraction by SMA MethodologyCandidate KQI and KPI Analysis

cKQI cKQI

KQIKQI

IPTV ServiceIPTV Service

How Goodis my service ?

How Goodis my service ?

Does my service keep running ?

Does my service keep running ?

Does my service work ?

Does my service work ?

How responsive is my service ?

How responsive is my service ?

How well is my problem resolved ?

How well is my problem resolved ?

Service Quality

Service Retainability

Service Accessibility

Service Latency

Service Support

How do I diagnose problem ?

(Operational view)

How do I diagnose problem ?

(Operational view)

Service Diagnosis

53KQI

86Gap KPI

48Available

KPIKPI (134)

17Combined

KQIKQI (70)

KPI KPI KPIKPI

KQIKQI

KPI KPI KPIKPI

CustomerPerspective

KQI Category

cKQI

KQI

KPI

IPTV SQI

KQI in a broad sense



▣ IPTV SQI Extraction by QFD MethodologyIdentify QoE, QoS and NP Candidate

IPTV Video Quality : QoS & QoE (Quarterly Technology & Content Report – February 2007) (Hawley 2007)

MultimediaResearch

Group

Technical Report TR-126 : Triple-play Services Quality of Experience (QoE) Requirements (DSLForum 2006)

DSL Forum

ATIS-0800004 : A Framework for QoS Metrics and Measurements Supporting IPTV Services (ATIS 2006)

ATIS

ITU-T Recommendation I.350 (ITU-T 1993)ITU-T Recommendation Y.1541 (ITU-T 2006)ITU-T FG IPTV-C-0411 (ITU-T FG IPTV 2007)ITU-T FG IPTV-C-0354 (ITU-T FG IPTV 2007)ITU-T FG IPTV-C-0507 (ITU-T FG IPTV 2007)ITU-T FG IPTV-C-0006 (ITU-T FG IPTV 2007)ITU-T FG IPTV-C-0127 (ITU-T FG IPTV 2006)ITU-T FG IPTV-C-0050 (ITU-T FG IPTV 2007)ITU-T FG IPTV-C-0210 (ITU-T FG IPTV 2006)

ITU-T

DocumentReferenceCustomer’s direct

Experience ?Customer’s direct

Experience ?

Whose viewpoint ?

Whose viewpoint ?

QoE QoEIndicator

Yes No

EngineerCustomer

NP

Network Provider

QoS

Service Provider

Which Provider’s

Area ?

Which Provider’s

Area ?



Content Security LevelSecuritySecurity QoE

Out-of-service Frequency

Out-of-Service Duration time

ReliabilityTransmissionQoE

EPG navigation responsiveness

Pause, Play Latency

Channel ChangeLatency

ResponsivenessTransactionQoE

Lip syncLip syncMultimediaQoE

Frame freezing

Frame skipping

Jerkiness

Pause

Object Retention

Object PersistenceAfterimage

Smearing

Block Distortion

Edge Busyness

Blurriness

Shapedistortion

Color ErrorColor Error

ResolutionResolution

Video QoE

Drop outsAudio DistortionAudio QoE

IntrinsicQoE

QoE IndicatorQoE level 2QoE level 1QoELevel 0

Number of Contents

Controller design

Reliability

Completeness (Amusingness)

Usefulness

Rightness

Popular Contents

Quality of Source MaterialContent Quality

Update PeriodContentQuantity

Content QoE

Menu DesignUser

FriendlinessUsability QoE

Addi-tionalQoE

QoE IndicatorQoE level 2QoE level 1QoELevel 0

Customer Churning Rate

A/S Expertness

Fulfillment ProcessingTime

Billing Satisfaction

A/S Friendliness

Subscription ProcessingTime

Expertness

Home Page

Phone Consulting

CustomerCare

Subscription IncreasingRate

Billing

A/S Processing Time

A/S

Termination ProcessingTime

Subscription/Fulfillment/Termination

Support ServiceSuppor-

tiveQoE

QoE IndicatorQoE level 2QoE level 1QoE

Level 0

▣ IPTV SQI Extraction by QFD Methodology



▣ IPTV SQI Extraction by QFD MethodologyID Analysis based on Delphi Method

ID (Importance/Difficulty) Analysis

ⅠII

IIIIV

Difficulty

Importance

1

73.50

3.5

7QoE

ImportanceAverage

DifficultyAverage

① Audio QoE 6.00 4.63

② Video QoE 6.63 5.63

③ Multimedia QoE 6.00 4.50

④ Transaction QoE 6.50 5.13

⑤ Transmission QoE 6.75 4.63

⑥ Security QoE 5.13 3.75

⑦ Usability QoE 6.50 4.50

⑧ Content QoE 6.57 4.57

⑨ Support Service 6.17 3.83

QoEImportance

AverageDifficultyAverage

① Audio QoE 6.00 4.63

② Video QoE 6.63 5.63

③ Multimedia QoE 6.00 4.50

④ Transaction QoE 6.50 5.13

⑤ Transmission QoE 6.75 4.63

⑥ Security QoE 5.13 3.75

⑦ Usability QoE 6.50 4.50

⑧ Content QoE 6.57 4.57

⑨ Support Service 6.17 3.83

4.57

6.25

2

3

4

5

678

9

ID (Importance/Difficulty) Analysis

ⅠII

IIIIV

Difficulty

Importance

1

73.50

3.5

7QoE Indicator

Importance Average

DifficultyAverage

① Resolution 6.25 3.88

② Color Error 5.63 4.00

③ Blurriness 5.38 4.13

④ Edge Busyness 5.75 4.88

⑤ Block Distortion 6.00 4.25

⑥ Smearing 5.75 4.38

⑦ Object Persistence 5.75 4.63

⑧ Object Retention 5.75 4.50

⑨ Jerkiness 6.00 4.63

⑩ Frame Skipping 5.88 4.13

⑪ Frame Freezing 6.13 4.50

QoE IndicatorImportance

AverageDifficultyAverage

① Resolution 6.25 3.88

② Color Error 5.63 4.00

③ Blurriness 5.38 4.13

④ Edge Busyness 5.75 4.88

⑤ Block Distortion 6.00 4.25

⑥ Smearing 5.75 4.38

⑦ Object Persistence 5.75 4.63

⑧ Object Retention 5.75 4.50

⑨ Jerkiness 6.00 4.63

⑩ Frame Skipping 5.88 4.13

⑪ Frame Freezing 6.13 4.50

4.35

5.8423

456 78

91011



▣ IPTV SQI Extraction by QFD MethodologyTwo Phase HOQ

Relative Importance between QoEs

QoE

QoS

Output: Core QoS Element

QoE

QoS

Output: Core QoS Element

QoS

NP

Output: Core NP Element

QoS

NP

Output: Core NP Element

HOQ-I (Phase I): QoE QoS HOQ-II (Phase II): QoS NP

SecuritySecurity QoE

ReliabilityTransmission

QoE

ResponsivenessTransaction QoE

Lip syncMultimedia QoE

Pause

Afterimage

Shape Distortion

Color Error

Resolution

Video QoE

Audio DistortionAudio QoE

SecuritySecurity QoE

ReliabilityTransmission

QoE

ResponsivenessTransaction QoE

Lip syncMultimedia QoE

Pause

Afterimage

Shape Distortion

Color Error

Resolution

Video QoE

Audio DistortionAudio QoE

10.6%

10.6%

7.0%

12.8%

21.9%

6.2%

10.3%

5.9%

7.2%

7.6%

QoE Level 2

0.000

0.050

0.100

0.150

0.200

0.250

0.300

Audio QoE

Video QoE

Multimedia QoE

Transaction QoE

Transmission QoE

Security QoE

QoE Level 1

(0.288)

(0.150)

(0.123)

(0.134)

(0.209)

(0.096)


▣ IPTV SQI Extraction by QFD MethodologyPriority between QoSs

Priority between NPs


6.41%

8.10%

8.19%

8.74%

9.87%

13.52%

16.06%

29.11%

Impact

100.0%

93.59%

85.49%

77.30%

68.55%

58.69%

45.17%

29.11%

Cumulative

Protocol

VoD Server

STB

MPEG Transport Stream

Middleware

Video Frame

Decoding / Decryption

Network Edge & Access

QoS Level 1

6.41%

8.10%

8.19%

8.74%

9.87%

13.52%

16.06%

29.11%

Impact

100.0%

93.59%

85.49%

77.30%

68.55%

58.69%

45.17%

29.11%

Cumulative

Protocol

VoD Server

STB

MPEG Transport Stream

Middleware

Video Frame

Decoding / Decryption

Network Edge & Access

QoS Level 180%60%40% 100%30% 25% 20% 15% 10% 5%

100.0%

100.0%

99.28%

98.55%

96.28%

93.76%

90.90%

88.07%

84.98%

81.88%

78.60%

74.63%

69.86%

64.28%

58.15%

51.56%

44.09%

35.59%

25.65%

13.52%

Cumulative

PSI Data error

STB Command Processing Time

Channel Zapping Time

Bandwidth

VoD Server Delay

RTP Jitter

RTP Delay

IP Packet Delay

IP Packet Jitter

Middleware Processing Time

PCR jitter

PTS error

VoD Server Error

Decoding Delay

Authentication Error

STB Buffer Under/Overflow

Burst Level

Decoding Error

IP Packet Loss

Video Frame Impairment Level

QoS Level 2

0.00%20

0.72%19

0.72%18

2.28%17

2.52%16

2.84%15

2.84%14

3.09%13

3.11%12

3.27%11

3.98%10

4.77%9

5.58%8

6.13%7

6.59%6

7.47%5

8.50%4

9.93%3

12.13%2

13.52%1

ImpactOrder

100.0%

100.0%

99.28%

98.55%

96.28%

93.76%

90.90%

88.07%

84.98%

81.88%

78.60%

74.63%

69.86%

64.28%

58.15%

51.56%

44.09%

35.59%

25.65%

13.52%

Cumulative

PSI Data error



Bandwidth

VoD Server Delay

RTP Jitter

RTP Delay

IP Packet Delay

IP Packet Jitter


PCR jitter

PTS error

VoD Server Error

Decoding Delay



Burst Level

Decoding Error

IP Packet Loss

Video Frame Impairment Level

QoS Level 2

0.00%20

0.72%19

0.72%18

2.28%17

2.52%16

2.84%15

2.84%14

3.09%13

3.11%12

3.27%11

3.98%10

4.77%9

5.58%8

6.13%7

6.59%6

7.47%5

8.50%4

9.93%3

12.13%2

13.52%1

ImpactOrder

90%50%

0.00%

3.48%

13.08%

14.43%

24.20%

44.81%

Impact

100.0%

100.0%

96.52%

83.44%

69.01%

44.81%

Cumulative

Subscriber Line

Protocol

Transmission

Core Network

STB

Access Network

NP Section

0.00%

3.48%

13.08%

14.43%

24.20%

44.81%

Impact

100.0%

100.0%

96.52%

83.44%

69.01%

44.81%

Cumulative

Subscriber Line

Protocol

Transmission

Core Network

STB

Access Network

NP Section

80%60% 100%50% 40% 30% 20% 10%

90%

100.0%0.00%Transmission Error Count(Core)19

100.0%0.00%CPU Load Ratio(Core)18

100.0%0.00%CPU Load Ratio(Access)17

100.0%0.00%Signal To Noise(Access)16

100.0%0.00%CRC Error Count(Access)15

100.0%0.00%Attenuation14

100.0%0.24%Port Up/Down Count(Access)13

99.76%1.83%STB Error Warning Count12

97.94%1.94%Core Phy Error Count(Core)11

96.00%2.18%Core I/F Error Count(Core)10

93.82%2.78%System Reboot Count(Core)9

91.04%3.48%Protocol Error Count8

87.56%3.96%Sub Port Pause Count(Access)7

83.60%7.53%Abnormal Traffic Count(Core)6

76.07%8.10%Memory Usage(STB)5

67.96%13.08%Transmission NE Trouble Count4

54.88%14.27%CPU Usage(STB)3

40.61%19.54%Abnormal Traffic Count(Access)2

21.08%21.08%System Reboot Count(Access)1

Cumulative

ImpactNPOrder

100.0%0.00%Transmission Error Count(Core)19

100.0%0.00%CPU Load Ratio(Core)18

100.0%0.00%CPU Load Ratio(Access)17

100.0%0.00%Signal To Noise(Access)16

100.0%0.00%CRC Error Count(Access)15

100.0%0.00%Attenuation14

100.0%0.24%Port Up/Down Count(Access)13

99.76%1.83%STB Error Warning Count12

97.94%1.94%Core Phy Error Count(Core)11

96.00%2.18%Core I/F Error Count(Core)10

93.82%2.78%System Reboot Count(Core)9

91.04%3.48%Protocol Error Count8

87.56%3.96%Sub Port Pause Count(Access)7

83.60%7.53%Abnormal Traffic Count(Core)6

76.07%8.10%Memory Usage(STB)5

67.96%13.08%Transmission NE Trouble Count4

54.88%14.27%CPU Usage(STB)3

40.61%19.54%Abnormal Traffic Count(Access)2

21.08%21.08%System Reboot Count(Access)1

Cumulative

ImpactNPOrder 50%


▣ IPTV SQI Extraction by QFD MethodologySystem Dynamics using Causal Loop Diagram


Pause

Burst Level

IP Packet Loss


System Reboot(Access)

Abnormal Traffic(Access)

Memory Usage (STB)

CPU Usage (STB)

QoE QoS NP

Shape Distortion

Decoding Error

Video Impairment Level

Resolution


Security

Reliability

VoD Server Error

Lip syncPCR Jitter

PTS Error

Responsiveness


Bandwidth

Decoding Delay

Transmission Error Count

Audio Distortion

Color Error

Afterimage


RTP Delay

RTP Jitter

VoD Server Delay

IP Packet Delay

IP Packet Jitter

PSI Data Error


STB Error Warning Count

Attenuation

CRC Error Count

Signal to Noise

Port Up/Down Count

Protocol Error Count

CPU Load Ratio (Access)

Sub Port Pause Count

System Reboot(Core)

Abnormal Traffic(Core)

CPU Load Ratio (Core)

Transmission Error Count

Core Phy Error Count

Core I/F Error Count

QoE QoS NP

QoE QoS NP



▣ IPTV SQI Extraction by KANO MethodologyCustomer Satisfaction Analysis

Quality Management Scenario

KANO Analysis Result

(18) Customer Care

(17) Billing

(16) A/S

(15) Subscription etc.

(14) Popular Content

(13) Content Quality

(12) Content Quantity

(11) User Friendliness

(10) Security

(9) Reliability

(8) Responsiveness

(7) Lip sync

(6) Pause

(5) Afterimage

(4) Shape Distortion

(3) Color Error

(2) Resolution

(1) Audio Distortion

QoE Level 2

IndifferentUsability QoE

Performance

Content QoE Indifferent

Performance

Support ServicePerformance

Performance

Performance

QoE Level 1 Classification

Audio QoE Indifferent

Video QoE

Performance

Performance

Performance

Indifferent

Performance

Multimedia QoE Performance

Transaction QoE Basic

Transmission QoE Basic

Security QoE Performance

Performance(18) Customer Care

(17) Billing

(16) A/S






(10) Security

(9) Reliability

(8) Responsiveness

(7) Lip sync

(6) Pause

(5) Afterimage


(3) Color Error

(2) Resolution


QoE Level 2

IndifferentUsability QoE

Performance

Content QoE Indifferent

Performance

Support ServicePerformance

Performance

Performance

QoE Level 1 Classification

Audio QoE Indifferent

Video QoE

Performance

Performance

Performance

Indifferent

Performance

Multimedia QoE Performance

Transaction QoE Basic

Transmission QoE Basic

Security QoE Performance

Performance

ReverseReverseQuestionable

ReverseIndifferentBasic

QuestionableExcitementPerformance

ReverseReverseQuestionable

ReverseIndifferentBasic

QuestionableExcitementPerformance

In case of S

uffic

iency

In case of Insufficiency

Satisfaction

Dissatisfaction Satisfaction

23

4

6710

12

1516

1714

18

1 5

1113

89

Quality Management Scenario

(18) Customer Care

(17) Billing

(16) A/S






(10) Security

(9) Reliability

(8) Responsiveness

(7) Lip sync

(6) Pause

(5) Afterimage


(3) Color Error

(2) Resolution


QoE Level 2

CreativeUsability QoE

Short-term

Content QoE Creative

Short-term

Support ServiceShort-term

Short-term

Long-term

QoE Level 1 Scenario

Audio QoE Creative

Video QoE

Long-term

Long-term

Long-term

Creative

Long-term

Multimedia QoE Long-term

Transaction QoE Short-term

Transmission QoE Short-term

Security QoE Long-term

Short-term(18) Customer Care

(17) Billing

(16) A/S






(10) Security

(9) Reliability

(8) Responsiveness

(7) Lip sync

(6) Pause

(5) Afterimage


(3) Color Error

(2) Resolution


QoE Level 2

CreativeUsability QoE

Short-term

Content QoE Creative

Short-term

Support ServiceShort-term

Short-term

Long-term

QoE Level 1 Scenario

Audio QoE Creative

Video QoE

Long-term

Long-term

Long-term

Creative

Long-term

Multimedia QoE Long-term

Transaction QoE Short-term

Transmission QoE Short-term

Security QoE Long-term

Short-term

Creative

ness

Effort

High

Low High

23

4

6710

12

1516

1714

18

1 5

1113

89

Long-termShort-term

Creative

Long-termShort-term

Creative

Medium

Medium

1 2 3 4

5 6 7

10 11 13 14

8 9 12

15 16 17 18

?1

11 13

5


4. ISQM System

▣ ISQM position in an eTOM context

▣ ISQM FeatureThree Layered Architecture

Data Collection, Quality Analysis, Service Quality View

Process Flow

From “TMF eTOM Level 3 Process”

Service Quality Modeling

MonitorServiceQuality Analyze

ServiceQuality

Create SPDRImprove

Service QualityClose SPDR

Track & Manage Service Quality Performance Resolution ReportServiceQuality

Performance

SQI Gathering (KPI)

111

222 333 444 555

666 777

888


4. ISQM System

▣ Required Capabilities

Improve Service Quality

Create Service Performance Degradation Report

Track & Manage Service Quality Performance Resolution6

4

Close Service Performance Degradation Report5

Monitor Service Quality1

Analyze Service Quality

3 Report Service Quality Performance7

•Data Gathering from Quality Data Sources (Probe, NMS

etc)

•Data Filtering Rule Setting & Management

•Data Interlocking Period & Threshold Management

•Real-time Alarm Monitoring

•Performance Related Data Comparison

•SLA/QoS Performance Degradation Report to Problem

Handling Process if needed

•High-level Transfer of the Information about Specific

Service Quality

•Management of Real-time Quality Information through Data Mart

•Judgment on crossing over the threshold of Service Quality

•Rule-based Data Redundancy Removal

•Root Cause Finding through Analyzing Correlation between Service Quality Data서

•Dependency Management between Service Component Elements

•Rule Based KPI & KQI Transformation

•Scheduling & Controlling about Service Performance Data Gathering

•Dynamic KPI/KQI Formula Change

•Pre-defined Action Executed in case of Specific Problem

•Inquiry about the progress of analysis, recovery and improvement of Quality Problem

•Detailed Inquiry about the Quality Event Status of Each Service Component

•Interface with Other System which requires Quality Information & Capabilities of providing Real-time and History Quality Information

•History Management & Inquiry Functionality of Quality Problem

•Offer of Tree-like Graphic Service Model Architecture

•Offer of Customizable GUI Configuration based on User Group

•Offer of Tree-like Service Dependency Architecture

•Transfer Information about Service Performance Degradation to Other Process of SM&O Processes

2

Service Quality Modeling8

•Properties Management of Service Component Type, Data Source etc.

•Management of Dependency Between Service Components

•Editing New KPI/KQIs for New Service Configuration

•Individual Customer Based Service Quality Management

•Detailed Drill Down Inquiry for the Management of Service Model Analysis Result

•Add/Modify/Delete of KPI & KQI

• Transformation from Collected Quality Data to KPI & KQI

• Threshold Management and Monitoring for Each Quality Indicator


4. ISQM System

▣ Functionality & Relationship

ISQMMegaTV, WiBro, SoIP

Service Quality

KQI

KPI

ESB (Enterprise Service Bus)

KPI Data Gathering

Large Data Filtering

Real-time Analysis of Service Quality

E2E Service Quality Information

Long Term Quality Analysis

Customer Care Operational Support Quality Information

Business Planning

Regulator Care

Operational Efficiency

BS Activity Support

SLA Quality InformationCustomer Complaint

Handling

Quality Measurement Infra

TVQMS (QSTB) Packet Probe

NMSCore NMS

Premium-NMS

Multicast-NMS

Head End

HDS, DBS, MOC

VoD Mgt.

STB Web Server

U-CEMS

KSMS

Fault Mgt.

TFMS

Performance/TrafficQuality

IPAS

FaultPerformance

BSS

SLA

NetIS

ICIS

NeOSS

IPASIPAS

provides data for service quality for the CORE Network. It provides data on packet loss, jitter, response times, bandwidth

provides data for service quality for the CORE Network. It provides data on packet loss, jitter, response times, bandwidth

Packet-Probe

TVQMS

Packet-Probe

TVQMS

is an important data source for Service Quality and accessibility. It provides data on audio / video quality , command execution success rates, Support protocols quality

is an important data source for Service Quality and accessibility. It provides data on audio / video quality , command execution success rates, Support protocols quality

U-CEMS DBU-CEMS DB

IP Premium NMS DBIP Premium NMS DB provides data for Service diagnosis . It provides

performance metric for the Premium CORE.provides data for Service diagnosis . It provides performance metric for the Premium CORE.

NeOSS IP CORE DBNeOSS IP CORE DB provides data for Service diagnosis. It provides

performance metric on the Kornetprovides data for Service diagnosis. It provides performance metric on the Kornet

NeOSS IP

Access NMS DB

NeOSS IP

Access NMS DBprovides data for Service diagnosis. It provides performance metric on the access networkprovides data for Service diagnosis. It provides performance metric on the access network

KSMSKSMS provides data for Service diagnosis. It provides

performance metric on the Serversprovides data for Service diagnosis. It provides performance metric on the Servers

provides data for Service retainability and accessibility. It provides performance metric for the STB.provides data for Service retainability and accessibility. It provides performance metric for the STB.


4. ISQM System

▣ Goal

How much customers perceive

their service

qualities?

Paradigm Shift from Network-Centric Quality

Management to Customer & Service-

Centric Management

What kind of indicators

are impacting on

service quality?

Offer of Quality

Information to Find out the Cause

of Problems & Monitor

the Service Quality Status

ISQM [Integrated ServiceQuality Management]

Interworking System(NMS, Probe etc)

ISQMService Quality

KQI

KPI

ESB (Enterprise Service Bus)

Int e

r wo

r ki n

gS

ys

tem

(SM

S, S

LA

, CR

M e

tc)

Monitor Service QualityMonitor Service Quality

Analyze Service QualityAnalyze Service Quality

Create SPDRCreate SPDR

Improve Service QualityImprove Service Quality

Track & Manage Service Track & Manage Service Performance ResolutionPerformance Resolution

Report Service Quality Report Service Quality PerformancePerformance

Close SPDRClose SPDR

Service Quality ModelingService Quality Modeling

Customer Quality Problem Report

(Customer CareCenter)

Inquiry of Customer Service QualtiyInformation & Speedy Action

After Service

(NetworkOperator)

Real-time Service Quality Monitoring & Detection of Quality Degradation Element

Before Service(Marketing) (Investment & Design)

(CxO)

Offer of Quality Information

Decision Making


5. Conclusion


77 Cell Matrix Methodology

KQI Development Methodology

SMA Methodology

QFD Methodology

KANO Model Methodology

Holistic

Approach to

Define

Service

Quality

Indicators

Quality Indicator

Characteristics

Integrated Integrated

SQISQI

in the in the viewpoint of viewpoint of CustomersCustomers

Integrated View ISQM

Service Quality

KQI

KPI

Monitor Service QualityMonitor Service Quality

Analyze Service QualityAnalyze Service Quality

Create SPDRCreate SPDR

Improve Service QualityImprove Service Quality

Track & Manage Service Track & Manage Service Performance ResolutionPerformance Resolution

Report Service Quality Report Service Quality PerformancePerformance

Close SPDRClose SPDR

Service Quality ModelingService Quality Modeling

Paradigm Shift from Network-Centric Quality Management to Customer & Service-Centric Management

APNOMS Special Session(Submission-Hyunmin Lim)Hyunmin_Lim).pdfKT Network Technology Laboratory...

Documents

Transcript of APNOMS Special Session(Submission-Hyunmin Lim)Hyunmin_Lim).pdfKT Network Technology Laboratory...