Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service...
Transcript of Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service...
![Page 1: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/1.jpg)
1 Burton Group Catalyst 2005
Service Level Agreements for VoIP
Alan ClarkCEO, Telchemy
Burton Group Catalyst 2005
![Page 2: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/2.jpg)
2 Burton Group Catalyst 2005
Agenda
• VoIP SLAs– What’s typical– Why typical isn’t ok
• Approaches to SLA measurement• What to measure• The “trust” problem• Final thoughts
![Page 3: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/3.jpg)
3 Burton Group Catalyst 2005
What is an SLA?
• Agreed set of performance parameters that:– Are measurable– Can be easily related to application performance
• In theory - if the SLA is met then the customer is happy
• How well does this work for Voice over IP?
![Page 4: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/4.jpg)
4 Burton Group Catalyst 2005
Service Provider Perspective
• Contractual SLA– Keep SLA as general as possible– Easily measured metrics– Measure at service demarcation point– SLA monitoring
• But……– Want customer to have good experience overall but
minimize level of contractual commitment to this
![Page 5: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/5.jpg)
5 Burton Group Catalyst 2005
Customer Perspective
• Voice is a mission critical application
• SLA should report all issues that affect service quality (i.e. don’t want service provider to claim they meet SLA when service unsatisfactory)
• Need service provider to “make it work”
• SLA helps to focus service provider on delivering agreed quality levels
• Highly reliable voice service more important than refunds
![Page 6: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/6.jpg)
6 Burton Group Catalyst 2005
Actual (typical?) VoIP SLA
Jitter < 20mS
Loss < 0.1%
Latency < 100mS
Availability 99.9%
What does this mean in practice?
![Page 7: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/7.jpg)
7 Burton Group Catalyst 2005
Jitter
0
50
100
150
200
0 0.5 1 1.5 2
Time (Seconds)
De
lay (
mS
)
Average jitter level (PPDV) = 4.5mS
![Page 8: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/8.jpg)
8 Burton Group Catalyst 2005
Packet Loss - also time varying
0
10
20
30
40
50
30 35 40 45 50 55 60 65 70
Time (seconds)
50
0m
S A
vg
e P
ack
et
Loss
Rate
Average packet loss rate = 2.1%
![Page 9: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/9.jpg)
9 Burton Group Catalyst 2005
Leads To Time Varying Call Quality
1
2
3
4
5
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Time
MO
S
0
100
200
300
400
500
0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18
Ban
dw
idth
(kb
it/
s)
Voice
DataHigh jitter/ loss/ discard
Degraded Service Quality (DSQ) Event = MOS less than X for more than N mS
![Page 10: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/10.jpg)
10 Burton Group Catalyst 2005
Delay
2
3
4
5
0 100 200 300 400 500 600
Round trip delay (milliseconds)
MO
S S
core
55dB Echo Return Loss
35dB Echo Return Loss
Conversational problemsEcho problems
![Page 11: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/11.jpg)
11 Burton Group Catalyst 2005
Why does echo make a difference?
IP
EchoCanceller
Gateway
Echo
Round trip delay - typically 50mS+
Additional delay introduced by VoIP makes existing echo problems more obvious
![Page 12: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/12.jpg)
12 Burton Group Catalyst 2005
Customer expectations of service quality
• Listening quality– Clarity, no distracting noise/ pops/ distortion
• Conversational quality– No noticeable delay or echo
• Availability– Always available, does not drop calls
• Signaling quality– Low call setup delay, features work
![Page 13: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/13.jpg)
13 Burton Group Catalyst 2005
A Better VoIP SLA?
99.9% of calls/intervals haveMOS-LQ > 3.9MOS-CQ > 3.8
Degraded Service QualityEvents < 0.1/ hour[DSQ = ….]
Latency < 100mS
Availability 99.9%
Based on either referenceor actual endpoint
Also reflected in MOS-CQ
Availability of media ANDSignaling path
Transient quality problems
![Page 14: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/14.jpg)
14 Burton Group Catalyst 2005
Enterprise Scenario
IP Phones
IP Phones
IP Phones
IP VPN
Gateway
PSTN
![Page 15: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/15.jpg)
15 Burton Group Catalyst 2005
Measuring at ISP Demarcation - active test
Active Test Functions
Test call
![Page 16: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/16.jpg)
16 Burton Group Catalyst 2005
Active test for IP Service SLA
• Uses VoIP calls, to ensure packets are treated identically to “real” VoIP calls
• Use a Reference endpoint - I.e. a fixed configuration, known, virtual IP endpoint
• Test:– Peak times - to understand quality under load
conditions– Off-peak times - to detect problems before they
impact users
![Page 17: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/17.jpg)
17 Burton Group Catalyst 2005
Measuring at user desktop - passive test
RTCP XR
SIP QoSReport
EmbeddedMonitoringFunction
![Page 18: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/18.jpg)
18 Burton Group Catalyst 2005
Passive test for IP SLA
• Most effective for end-to-end measurement
• Embedded quality monitoring function in IP endpoint
• Can measure service quality, signaling reliability……
• Collect data via RTCP XR or SIP QoS reports
![Page 19: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/19.jpg)
19 Burton Group Catalyst 2005
Service Level Metrics - “mine” or “yours”
• Common problem with SLAs– Service provider measures SLA and reports that they
meet SLA– Customer uses different tools and finds that service
provider “does not” meet SLA
• Who is right?
![Page 20: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/20.jpg)
20 Burton Group Catalyst 2005
Overcoming the “somebody else’s problem” problem
• Need– Common measurement methodology– Ability to make the same measurements at the same
time in the same way
• Otherwise– Results will differ and fingers will point
• Solution?– A common (trusted) measurement methodology– A shared (trusted) measurement function that is
accessible to both service provider and customer
![Page 21: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/21.jpg)
21 Burton Group Catalyst 2005
A “Trusted” SLA Monitoring Function
SummaryStats
SummaryStats
ServiceProvider
Enterprise
Non-intrusivemonitoring
Active test agent
Edge router
![Page 22: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/22.jpg)
22 Burton Group Catalyst 2005
SLA Monitoring Function
• Ideally - locate in edge router on customer premise
• Measurement data available to both service provider and customer
• Provides both – Non-intrusive per-call monitoring of live traffic– Active test agent for scheduled testing and
troubleshooting
• Measures SLA in terms of– Estimated call quality level (MOS, R)– DSQ (Degraded Service Quality) events– Loss, jitter, discard, delay………
![Page 23: Service Level Agreements for VoIP - Telchemy 2005... · 2016-05-06 · Active test for IP Service SLA • Uses VoIP calls, to ensure packets are treated identically to “real”](https://reader036.fdocuments.us/reader036/viewer/2022081613/5fba1208b063af09c63dc9ca/html5/thumbnails/23.jpg)
23 Burton Group Catalyst 2005
VoIP SLA Management
• VoIP SLAs– What’s typical– Why typical isn’t ok
• Approaches to SLA measurement• What to measure• The “trust” problem• Final thoughts?
– VoIP does not have such distinct service boundaries as traditional telephony - some form of shared data model is an essential addition to an SLA for collaborative troubleshooting across network boundaries.