Driving Quality Performance Improvements Through Rigorous Testing Of Key Roaming Services Barcelona,...
16
Driving Quality Performance Improvements Through Rigorous Testing Of Key Roaming Services Barcelona, April 17 th 2007
-
date post
15-Jan-2016 -
Category
Documents
-
view
217 -
download
0
Transcript of Driving Quality Performance Improvements Through Rigorous Testing Of Key Roaming Services Barcelona,...
Driving Quality Performance Improvements Through Rigorous Testing Of Key Roaming Services
Barcelona, April 17th 2007
Introduction KEYNOTE SIGOS SIGOS founded in November 1989
April 2006 KEYNOTE aquires SIGOS
KEYNOTE group has currently about 300 employees,Market leader in IP monitoring
KEYNOTE SIGOS has about 80 employees, Market leader in E2E and service testing area
Business Segments
SIGOS Integrated Test Environment – SITE
GlobalRoamer – Roaming Test Service
Test support and consultancy in the telecommunications market
Currently more than 50 operators worldwide are using equipment or services from SIGOS amongst them Vodafone, T-Mobile, Telefonica and Orange
Mobile telecommunication systems are an established technology customers expect them to work
Customers are increasingly traveling and they expect mobile services to work everywhere
Customers are used to their network environment and they expect the same look and feel also abroad VHE
Roaming is still a cash cow – 15% of operator’s revenue from international Roaming, mostly subscribers with high ARPU
The number of mobile services have increased exponentially over the last decade so that traveling people are much more depending on their cell phones than in the past
Importance of QoS
The role of QoS in the area of Roaming
Market saturation has reached in many countries. Operators have to answer new questions like:
How can I attract new customers even in a saturated market?
How can I prevent my customers to churn?
How can I increase ARPU?
How can I check SLAs?
Importance of QoS
The role of QoS in the area of Roaming
Qos in the area of Roaming is as important as QoS in the home network! It is the only key to remain
successful!
How good are the Roaming services really?
Importance of QoS
0,00%
10,00%
20,00%
30,00%
40,00%
50,00%
60,00%
70,00%
80,00%
90,00%
100,00%
Voice SMS MMS USSD CallBack
WAP GPRSDownload
source: GlobalRoamer customer test data
Ser
vice
Ava
ila
bil
ity
There‘s still much money lost due to service unavailability
How good are the Roaming services really?
Importance of QoS
source: GlobalRoamer customer test data
Voice Call Establishment Duration: 8,9 seconds
Mean Opinion Score: 2,97 (fair)
SMS Delivery Time: 12,4 seconds
MMS Delivery Time: 94,5 seconds
Beside availability also quality measurements matter! It’s all about KPIs…
„How long would a customer wait to be connected?“
QoS parameters
Current situation:
KPIs differ between operators according to their internal quality policy based on
service importance portfolio maturation of technology in the country
Roaming related KPIs are rarely specified
Outlook:
Standardized KPIs are recommended by GRQ group
Aim is to measure everywhere the same and to make results comparable!
Unfortunately KPIs are only specified for Voice and SMS so far
To generate all KPIs the operators need different tools
Key Performance Indicators
QoS parameters
The Key Performance Indicators proposed by GRQKPIs Voice - Description KPIs Which tool can generate it
Network Access (customer being able to register on the network)
Location Update success rate Active probing system
Service accessibility (from customer hitting the send bottom until hearing a ring tone)
NER (Network Effectiveness Ratio) Passive monitoring system
PDD (Post Dialling Delay) Both systems
Connection establishment (from customer hitting the send bottom until a successful establishment of the service)
CSSR (Call Setup Success Ratio) Active probing system
REL (ISUPv2 signalling transparency) Passive monitoring system
OCN & RDN (ISUPv2 signalling transparency) Active probing system
Connection retain ability (from a successful establishment of the service until service is terminated)
CCR (Call Completion Rate) Passive monitoring system
ALOC (Average Length of a Call) Passive monitoring system
Connection qualityCLI delivery Active probing system
SpQ (Speech Quality) Active probing system
KPIs SMS KPIs Which tool can generate it
Network Access (customer being able to register on the network)
Location Update success rate Active probing system
Connection establishment (from customer hitting the send bottom until a successful delivery of the SMS)
SMS Delivery Time (SDT) Active probing system
SMS Completion Rate (SCR) Active probing system
different testing methods
Active testing solutions
Real end user simulation
Provide QoS parameters, real-time reports/alarms
Generation of dedicated test calls (billing, new services)
Passive monitoring solutions
No customer experience testing
Provide QoS parameters, real-time reports/alarms
Detailed trace information available covering however only part of the chain
different testing methods
What is the ideal solution?
Usage of all two methods to get the full picture
Data correlation between these systems (intra) and between Roaming partners (inter)
Modest price
What is reality today?
Active and passive testing solutions exist
No data exchange and correlation between the systems, limited or no exchange between Roaming partners
Active testing solution preferred due to best cost-value ratio
Benefits of active probing:
Investment is not that high!
Preventive testing is possible before new services are launched, new network components are introduced and new Roaming agreements are signed
Revenue assurance testing to verify if provided TAP files are matching with the test calls
Immediate reaction is possible and bug fixing likely happens before customers get affected
Customer view can be realistically simulated
Easy implementation and test execution requiring no support from Roaming partner
testing methods to guarantee QoS
testing methods to guarantee QoS
Easy access via the web
Countries of GlobalRoamer Test Locations
GlobalRoamer extension in 2007
GSM/GPRS Test Locations
GSM/GPRS and UMTS/WCDMA Test Locations
GSM/GPRS and UMTS and HSDPA Test Locations
3G
3G
Active probing solution - GlobalRoamer
All services can also be tested within a 3G environment
Test Cases
Initialization
Basic Call
Supplementary Services
SMS
WAP (1.x / 2.0)
GPRS / IP
IVR
MMS
Voice Quality
Push-to-Talk (PoC)
Streaming
Web Browsing
Video Telephony
testing methods to guarantee QoS
Reports based on KPIs
ALARM
Alarms based on user definable thresholds
testing methods to guarantee QoS
One example what active testing can tell you:
lse De Kee (Base Belgium):
"Using GlobalRoamer, we realized that the success rate of our CAMEL calls was only around 50%! Additional to that, we then realized that another 50% of the successful tests weren’t billed! Thanks to GlobalRoamer, we could detect this and immediately take the respective actions."
different testing methods
What shall we do to improve?
GSMA guidelines are a first, very good start to achieve common and comparable GRQ information. Now operators have to use them!
Guideline do not yet include IP and other services
Testing must be intensified and move from an ad-hoc to a continuous process
Vendors must invest to increase worldwide coverage (2G and 3G)
Vendors must implement the GSMA guidelines (or operators must push them to do!)
Interfacing possibility between Active and Passive probing systems;is that something vendors should look at?
Mobile alliances shall use their international presence to intensify passive and active probing
