Alcatel Performance Meeting Chicago

8/3/2019 Alcatel Performance Meeting Chicago

1/39

All Rights Reserved Alcatel-Lucent 2007

Performance Workshop

Alcatel-Lucent

Chicago July 18, 2007


2/39


Agenda

1. Introductions (All)

2. RAMSES (Benjamin Williams)

3. Systems Engineering: Performance Analysis (Paul Boudreaux)

4. Voice Quality (Falguni Sarkar)

5. Q & A (All)


3/39


RAMSES

Alcatel-Lucent

July 16, 2007


4/39


Agenda

1. KPI Team Members/Roles

2. Typical RAMSES KPI Reporting

3. RAMSES Tool Overview

4. Specific examples from RAMSES KPI Reports


5/39


1KPI Team Members / Roles


6/39


KPI Team Members/Roles

KPI Team

- Ben Williams Team Leader, Central Region Prime, XML

Prime

- Imron Hussaini Northeast Region Prime, GSM/GeoProbeco-Prime

- Vrushali Nidgundi South Region Prime, GSM/GeoProbe

co-Prime

- Anand Sane UMA Prime

- Francis B. Smith West Region Prime


7/39


2Typical RAMSES KPI Reporting


8/39


Typical RAMSES KPI Reporting

Weekly reports GSM Weekly Report (reviewed 2nd and 4th Tuesdays of each month via

teleconference)

Covers all (~50) TMO Atriums providing GSM service (i.e. GMSCs, DMSCs: GSM-only +

Phase 2 UMA)

Summarizes TCBH (Time Consistent Busy Hour) for M-F of previous week

MS Word format and includes:

Dashboard Summary, Active Subscribers, Traffic/Events per Subscriber

Traffic Profile, BHCA, CPU Utilization

GOS for Calls, SMS, CAMEL, Paging & Auth., Loc. Update, and HandOver

Trunk Group Report, Call Blocking Report, SS7 Utilization Report (STP andBSC)

Separate Weekly Trend Report for the above GOS Trends


9/39


Typical RAMSES KPI Reporting (cont.)

Weekly Reports (cont.)

UMA Weekly Report (reviewed each Monday via teleconference)

Covers all (12) TMO Atriums providing UMA service (i.e. Phase 1 UMA + Phase 2 UMA DMSCs)

Summarizes TCBH (Time Consistent Busy Hour) for Su-Sa of previous week

MS Word format and includes:

Active Subscribers, Traffic/Events per Subscriber

Traffic Profile, BHCA, CPU Utilization

GOS for Calls, SMS, CAMEL, Paging & Auth., Loc. Update, and HandOver

Trunk Group Report, Call Blocking Report, SS7 Utilization Report (STP and BSC)

UMA Discovery & Registration, UMA Paging, LDAP Queries

KPIs for other UMA Nodes (SGW, GGW, GMLC, SMLC, etc.)

Daily Reports (as needed)

FOAs, Switch launches, UMA service launch, etc.

Typically MS Word format, though sometimes MS Excel

Typically single switch per report

Typically shows all (15-minute) samples for the day in line graph form for each GOS indicator


10/39


3 RAMSES Tool Overview


11/39


RAMSES Tool Overview

RAMSES is an ALU internal tool which is composed of three main parts:

- RAMSES Data Collector Cycles through all supported switches, FTPing all

.tar files in the kpi directory which are not already on the RAMSES server

- RAMSES database A flat-file database containing the post-processedcontents from the .tar files collected from each switch since collection

started for that switch (back to late 2005 for some switches)

- RAMSES Report Generator GUI

- Enables customized (.rtf, .xls, or .html) reports for one or more switches (selected

at run time through the RAMSES GUI)

- Reports are defined in XML and are structured as combinations of individual charts

(histograms, tables, line graphs, etc.) based on KPI definitions- KPIs are defined in XML and are based on the Atrium performance counters

- Ad-hoc charts can be generated from individual KPIs or counters as needed


12/39


4Specific examples from RAMSES KPI Reports


13/39


Specific examples from RAMSES KPI Reports

Example #1:

Histogram chart Active Subscribers (all GSM DMSCs)


14/39


Specific examples from RAMSES KPI Reports (cont.)

Example #2:

Table chart Grade of Service UMA Paging (for all UMA switches)

Ne UNC Paging Attempts UMA Paging SuccessRate

TCBH

ATMSS996_UNC 84.7 evt/h 93.9 % [21-22]

CHMSS966_UNC 78.6 evt/h 94.9 % [17-18]

CSMSS949_UNC 162 evt/h 85.0 % [12-13]

DAMSS987 64.9 evt/h 94.7 % [17-18]

DEMSS931 59.6 evt/h 97.8 % [17-18]

DNMSS935 41.4 evt/h 95.9 % [17-18]

.

.

.


15/39


Specific examples from RAMSES KPI Reports (cont.)

Example #3:

Line chart Loc. Update Attempts vs. Failure Rate for 12-Jul-2007 at newly

launched CLMSS981


16/39


Performance AnalysisAnalysis Of Radio Issues On BSCs Re-Homed To Atrium

MSC

Xavier Ligeret ([email protected])

T-mobile Systems Engineering

North America RSC

July 10, 2007


17/39


Description Of The Issues

Known SRs And Symptoms

Known SRs and DCR/RDS:

- SDCCH drop counter, T3260 Timer

- TCH poor performance (SR 200610002564)

- Increase SDCCH drop after BSC rehome (SR 200704003534)

- Increase paging failure after BSC rehome (SR 200704003538)

- TCH spikes on NEBSC01 and NEBSC97 (SR 200702014483)

- Ciphering timer configurable from the WEM (DCR 173/RDS 6917)

Impacted networks = All networks with BSCs re-homed to Atrium MSC

Symptoms:

- Reported increase of call drops, paging failures

- Statistics suggest poor performance (on BSC counters) on re-homed BSCs comparedwith statistics on BSCs connected to legacy MSCs; though the call drops statisticsdont exceed 3%


18/39


Presentation Of The Issues

Observations Made By TMO (Dinko Dinkov and Frankie Chen)

Issues seem to appear mostly at the borders of areas covered by the re-homedBSCs, with higher probability of RF issues (lower signal strength)

Atrium MSC triggers more requests towards the MS compared with legacy MSCs(more Location Updates and more ciphering), thus:

More signaling on the Radio

The probability where the network tries to contact the MS, while it is in apoor radio coverage, is higher with Atrium, and eventually the risk of calldrop increases

MSC implementation differences found between legacy MSCs and Atrium MSC:

Different handling of DTAP signaling messages during Assignment Procedure

Higher Ciphering frequency (Atrium)

Higher TMSI-reallocation frequency


19/39


Analysis

We first checked that the Atrium MSC was 3GPP compliant, regarding:

The frequencies of the Ciphering and TMSI reallocation procedures, though Atriumfollows the standards in a very strict way

The call flow and the global design to handle scenarios, though bugs may still exist

Re-homing BSCs creates islands of small Location Areas managed by the Atrium MSC,surrounded by legacy MSCs, and increases Signaling messages

The system seen as a whole (from the MSC to the MS) evolves in some extreme situations:

Limit of the radio coverage

With the MS probably often in unstable states (in search for radio signal, )

With message race situations

Where 3GPP can be interpreted in different ways

This is NOT:

A load or dimensioning issue


20/39


Analysis

Summary

+Existing RF issues More signaling onthe Radio Reported moreconnection failures

(according to counters)

A different MSC

implementation

+Existing RF issues Poor statistics on theNetwork Access

More Signaling on the Radio increases the probability that the Network tries to reach the

MS, while it is in a zone of bad radio coverage, and increases the chances to drop the MS

Some implementation differences in the MSC have direct impacts on the BSC counters


21/39


Conclusion

The degradation of the statistics on re-homed BSCs:

Can only be mitigated, but not totally fixed

Shows that there are radio issues, that were partly masked whileusing the legacy MSCs because of the way the BSCs pegged theircounters

ALU will: Help TMO to fine tune the parameters to improve the statistics

Fix the bugs found, if any

Address any change request in the MSC implementation only as anultimate solution

There are clearly radio issues that need further investigations as well


22/39


Backup

Detailed Analysis


23/39


Analysis Of The TCH Issue

Example Of MSC Implementation Impacting BSC Counters (1/2): Ericssons MSC

Implementation (3GPP compliant):

During the Assignment Procedure:

- The MSC suspends the transmission of any CC or MMsignaling messages

- Received messages are queued or discarded (we dontknow exactly yet)

At the end, if the Assignment Procedure is successful:

- The TCH setup is supposedly counted as successful by

the BSC- If the Disconnect is processed by the MSC after theAssignment Procedure, it would supposedly be countedas a normal Disconnect

Main Benefit: Better statistics on the BSC if theAssignment Procedure is successful

Main Drawback(s):

- If the MSC processes the Disconnect: waste ofRadio resources (TCH immediately released aftersetup)

- If the MSC has dropped the Disconnect: it wouldbe a 3GPP violation


24/39


Analysis Of The TCH Issue

Example Of MSC Implementation Impacting BSC Counters (2/2): Atrium MSC

Implementation (3GPP compliant):

- During the Assignment Procedure, the MSC responds toany signaling messages from the MS.

- The MSC lets the BSC suspend the transmission of thosemessages (otherwise the BSC would violate 3GPP)

At the end, the scenario leads to a Call Disconnectionbefore the completion of the Assignment Procedure

Main Drawback: TCH setup is not completed, the BSCsupposedly counts it as a failure

Main Benefit: Radio resource saved, since no TCH setup

Conclusion:

At the end, both Ericsson and Atrium implementationslead ultimately to the Disconnection of the Call, butwith different impacts on the BSC counters


25/39


T-Mobile/ALU Members

Recommendations to fine tune the parameters to adjust to the BSCs and theTMO traffic and support

A dedicated team is working on all these issues since 5/21/07 with a weekly orbi-weekly meeting:

On TMOs side:

TMO coordinator: Pradeep Singh Tier2 RAN Ericsson: Dinko Dinkov

Tier2 Atrium: Michele Kinsey

On ALUs side:

RSC POC: David McNemar and Sean Simmons

RSC System Engineering: Xavier Ligeret

War Room POC: Jim Morris

Other participants:

TMO: R. K. Kumar, Munar Le Roy, Homer Filart,

ALU: Brett Schwarz, Robbie Espiritu,


26/39


Atrium Voice Quality VerificationT-Mobile USA

Norfolk/DC Field Test

Voice and Data Quality/Wireless Performance and RF Core Support

Ping Yu, Wonho Yang, Ahmed Tarraf

July 13, 2007


27/39


Contents

Voice Quality Test ScenariosVoice Quality Test Setup

Voice Quality Test Results

Executive SummaryNext Steps


28/39


Purpose

The Voice and Data Quality and Performance teamperformed Voice Quality field verification for the

GSM/Atrium system in the T-Mobile market from

June 11th to June 14th 2007.

Field verification included:

Voice quality for GSM Mobile-to-Land, Land-to-

Mobile and Mobile-to-Mobile calls

Echo Test for Mobile-to-Land calls


29/39


Voice Quality Test Scenarios

Stationary Tests

Voice Quality Test:

Land-to-Mobile

Mobile-to-Land

Mobile-to-Mobile

Echo Test (ESM1 card)

Mobile-to-Land

Drive Tests

Voice Quality Test:

Land-to-Mobile

Mobile-to-Mobile

Echo Test (ESM1 card)

Mobile-to-Land


30/39


ROMAPlayer

Voice Quality Measurement Setup

Mobile

BSC

Atrium

MGW

BTS

Mobile-Mobile Land-Mobile

Mobile

BSC

MGW

BTS

Signaling link

Speech Path

Analog

Phone

Atrium

PSTN

Switch

Mobile

BSC

MGW

BTS

RF

Tool Kit

Signaling link

Speech Path

Player ROMA

Mobile


31/39


Voice Quality Measurement Tool

Voice Quality was measured using Alcatel-Lucent Real Time

Objective MOS Analyzer (ROMA)

ROMA (Real Time Objective MOS Analyzer)

real-time intrusive tool developed by ALU

based on ITU-T PESQ and on ALU expertise on objective voice quality

evaluation since 1993

is highly correlated with Subjective MOS (~0.93)

runs on a laptop and interfaces with the Mobile via regular sound card and

the 2.5mm audio jack at the mobile

performs automatic detection / segmentation of talk spurts


32/39


MOS = Mean Opinion Score

ITU-T P.800

Rating Speech Quality Listening Effort

5 Excellent Complete Relaxation possible4 Good Attention necessary, little effort

3 Fair Moderate effort required

2 Poor Considerable effort required

1 Bad No meaning understood


33/39


Voice Quality Test Setup

Stationary Tests

Originating mobile, terminating mobile and Land line phone were at

the same location (Switch room location in Norfolk, Virginia)- good

RF location

BSC 26 was used, ESM1 card used

Land-to-Mobile:

Speech injected at Land line (using audio interface box)

Speech recorded at Nokia 6010 mobile (using audio interface box)

Mobile-to-Land

Speech injected at Motorola V330

Speech recorded at land line phone

Mobile-to-Mobile:

Speech injected at Motorola phone

Speech recorded at Nokia phone


34/39


Voice Quality Test Setup (cont.)

Drive Tests

Originating mobile was at van

Drive routes included I464 and 64, experienced both intra BSC and

inter BSC handover (BSCs 23, 26, 28)

ESM1 card used

Land-to-Mobile:

Speech injected at Land line (using audio interface box)

Speech recorded at Nokia 6010 mobile (using audio interface box)

Mobile-to-Land

Live talk test using MotorolaV330 phone and subjective echo at the same

mobile was carefully tested.

ROMA (Realtime Objective MOS Analyzer), developed by VDQPAG, is

used as the EMOS/VQ measurement tool


35/39


Voice Quality Test Results: MOS

Stationary TestsLand-to-Mobile: Call 1 (duration 6+ minutes) - 49 MOS samples

VQ is good

Constant background noise impairment of about 0.2 MOS attributable tomobile audio interface and landline devices.

Background noise impact was a constant during the testing

Statistics MOS

Mean 3.47

Median 3.46Max 3.64

Min 2.82

Standard Dev 0.14

BackgroundNoise Sample


36/39


Executive Summary

ALU Voice and Data Quality and Performance team performed a Voice

Quality evaluation on the Atrium Call Server in NorfolkAtrium Voice Quality was determined good as measured by EMOS

Expected MOS for Good Results based on typical good voice quality score of wirelessnetwork regardless of the technology

Stationary Atrium

Testing

Expected MOS for

Good Results

Measured MOS

(Mean)

Land to Mobile 3.4 3.46

Mobile to Mobile 3.1 3.25

Mobile to Land 3.4 3.54


37/39


Executive Summary

Observed VQ degradations:

Constant background noise impairment of about 0.2 MOS attributable to

mobile audio interface and landline devices.

Infrequent BER audio degradation which is consistent with RF impairments

A sample drive test provided Land to Mobile MOS of 3.21

Next Steps

Run VQ/MOS tests on ESM1 and ESM4 cards in T-M Lab environment.

Run Echo tests on ESM1 and ESM4 cards in T-M Lab environment.


38/39


Performance WorkshopQ&A

Alcatel-Lucent

July 16, 2007


39/39


www.alcatel-lucent.com

Alcatel Performance Meeting Chicago

Documents

Transcript of Alcatel Performance Meeting Chicago