EV-DO RF Optimisation

96
1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE R EL 24.0 RF T OOLS & S ERVICES S YSTEMS E NGINEERING GROUP LUCENT TECHNOLOGIES PROPRIETARY – FOR INTERNAL USE ONLY USE PURSUANT TO COMPANY INSTRUCTIONS

description

CDMA EV-DO RF Optimisation guide

Transcript of EV-DO RF Optimisation

Page 1: EV-DO RF Optimisation

1 X E V- D O R F P E R F O R M A N C E A N A LY S I S

& T R O U B L E S H O O T I N G

G U I D E

REL 24.0

RF TOOLS & SERVICES SYSTEMS ENGINEERING GROUP

LUCENT TECHNOLOGIES PROPRIETARY – FOR INTERNAL USE ONLY USE PURSUANT TO COMPANY INSTRUCTIONS

Page 2: EV-DO RF Optimisation

TA B L E O F C O N T E N T S

0. CHANGE HISTORY.............................................................................................................................. 8

1. INTRODUCTION................................................................................................................................. 11

2. USING THIS GUIDE ........................................................................................................................... 13

3. OVERVIEW OF 1XEV-DO SYSTEM AND OPERATION............................................................. 15 3.1 SYSTEM FUNCTIONAL BLOCKS ..............................................................................................15

3.1.1 Access Network ........................................................................................................... 16 3.1.2 Access Terminal .......................................................................................................... 19

3.2 1XEV-DO SYSTEM OPERATION ...........................................................................................19 3.2.1 Forward link ............................................................................................................... 20 3.2.2 Reverse Link ................................................................................................................ 21

4. SUMMARY OF PERFORMANCE RELATED FEATURES IN PRODUCT RELEASES .......... 22 4.1 PERFORMANCE RELATED FEATURES IN RELEASE 23.0 ....................................................22 4.2 PERFORMANCE RELATED FEATURES IN RELEASE 24.0 ....................................................23 4.3 PERFORMANCE RELATED FEATURES SCHEDULED IN RELEASE 25.0..............................24

5. KPI OPTIMIZATION AND TROUBLESHOOTING ...................................................................... 26 5.1 ESTABLISHED CALL RATE ....................................................................................................27

5.1.1 Call setup process ....................................................................................................... 28 5.1.1.1 Fast Connect Calls ........................................................................................................ 30

5.1.2 RF access failures ....................................................................................................... 31 5.1.2.1 Lack of RF coverage ..................................................................................................... 32 5.1.2.2 Excessive Pilots/Non-dominant Pilots .......................................................................... 33 5.1.2.3 Poorly constructed Neighbor Lists................................................................................ 34 5.1.2.4 Improper PN planning................................................................................................... 35 5.1.2.5 External Interference..................................................................................................... 37 5.1.2.6 Inadequate Search Window Size................................................................................... 38 5.1.2.7 Edge of coverage........................................................................................................... 39 5.1.2.8 RNC border issues ........................................................................................................ 40

5.1.3 Failures due to faulty hardware.................................................................................. 41 5.1.4 Heavy Reverse Link traffic .......................................................................................... 41 5.1.5 TP overload................................................................................................................. 43 5.1.6 Maximum number of connections reached.................................................................. 44 5.1.7 Traffic Channel Activation failure .............................................................................. 46 5.1.8 Authentication error.................................................................................................... 46 5.1.9 High Processor Occupancy rates................................................................................ 47

5.2 DROP CALL RATE ...................................................................................................................49 5.2.1 Lack of RF coverage ................................................................................................... 51 5.2.2 Excessive pilots/Non-dominant pilots ......................................................................... 52 5.2.3 Poorly constructed neighbor lists ............................................................................... 54 5.2.4 Improper PN planning ................................................................................................ 55

Page 3: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.2.5 External interference................................................................................................... 56 5.2.6 Insufficient search window size................................................................................... 58 5.2.7 Failures due to faulty hardware.................................................................................. 59 5.2.8 Heavy reverse link traffic ............................................................................................ 60 5.2.9 Maximum number of legs reached .............................................................................. 62 5.2.10 No resources at the candidate leg ............................................................................... 63 5.2.11 High Processor Occupancy rates................................................................................ 64 5.2.12 Handoff failures .......................................................................................................... 65

5.3 DATA THROUGHPUT...............................................................................................................67 5.3.1 Lack of RF coverage ................................................................................................... 68 5.3.2 Excessive pilots/Non-dominant pilots ......................................................................... 69 5.3.3 Poorly constructed neighbor lists ............................................................................... 71 5.3.4 Improper PN planning ................................................................................................ 73 5.3.5 External interference................................................................................................... 74 5.3.6 Insufficient search window size................................................................................... 76 5.3.7 Failures due to faulty hardware.................................................................................. 77 5.3.8 Heavy reverse link traffic ............................................................................................ 78 5.3.9 High forward link traffic ............................................................................................. 80 5.3.10 Hybrid mode................................................................................................................ 82 5.3.11 Backhaul restrictions .................................................................................................. 83 5.3.12 Handoff rejects ............................................................................................................ 84 5.3.13 Core IP network issues................................................................................................ 86

APPENDIX A METRIC CROSS-REFERENCE.............................................................................. 87

APPENDIX B OVERVIEW OF SPAT3G TOOL............................................................................. 89

LUCENT TECHNOLOGIES PROPRIETARY 3

Page 4: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

L I S T O F F I G U R E S FIGURE 1 FUNCTIONAL BLOCKS IN A LUCENT 1XEV-DO SYSTEM ............................................... 15 FIGURE 2 FUNCTIONAL BLOCKS IN A LUCENT 1XEV-DO OVERLAY ON A LUCENT 3G1X HIGH

SPEED DATA NETWORK......................................................................................................... 16 FIGURE 3 PROTOCOL STACK DIAGRAM FOR LUCENT 1XEV-DO SYSTEM [9] .............................. 19 FIGURE 4 CALL FLOW FOR AN AT INITIATED CONNECTION.......................................................... 29 APPENDIX FIGURE 1 EXECUTIVE SUMMARY................................................................................. 90 APPENDIX FIGURE 2 SM SUMMARY WINDOW AND PEG COUNT DETAILS .................................... 90 APPENDIX FIGURE 3 SM MAP ....................................................................................................... 91 APPENDIX FIGURE 4 SM TREND ................................................................................................... 92 APPENDIX FIGURE 5 ROP SUMMARY – ALARMS ANALYSIS......................................................... 93 APPENDIX FIGURE 7 TRX SUMMARY WINDOW ........................................................................... 94 APPENDIX FIGURE 8 NL ALERT SUMMARY .................................................................................. 95

L I S T O F TA B L E S TABLE A-1 ESTABLISHED CALL RATE METRIC CROSS REFERENCE................................... 87 TABLE A-2 DROP CALL RATE METRIC CROSS REFERENCE ................................................ 87 TABLE A-3 DATA THROUGHPUT METRIC CROSS REFERENCE............................................ 88

LUCENT TECHNOLOGIES PROPRIETARY 4

Page 5: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

List Of Abbreviations 2G Second Generation (IS95A/IS95B) 3G1x Third Generation 1.25MHz (IS2000) 3GPP2 Third Generation Partnership Project 2 A10 Interface that carries user traffic between PCF and PDSN A11 Interface that carries signaling between PCF and PDSN A12 Interface that carries signaling (authentication) between PCF and AAA A13 Interface that carries signaling between source and target PCF AAA Authentication, Authorization and Accounting ACK Acknowledgement AHO Access Handoff AN Access Network AP Application Processor APHO Access Probe Handoff AR Assistance Request AT Access Terminal BTS Base Transceiver Station CAMSHO Channel Assignment into Soft Handoff CBR CDMA Baseband Radio CC Control Channel CDM CDMA Digital Module CLGC Closed Loop Gain Control CRC CDMA Radio Controller CTU Common Timing Unit DRC Data Rate Channel EMS Element Management System EV-DO Evolution Data Optimized EVM EV-DO Modem FCA Failed Connection Attempts FER Frame Error Rate FLM Forward Link Modem FMS Flexent Mobility Server FTP File Transfer Protocol GPS Global Positioning System GRE Generic Routing Encapsulation HCS HDR Cell Site HDLC High Level Data Link Control HDR High Data Rate HOM Handoff Matrix

LUCENT TECHNOLOGIES PROPRIETARY 5

Page 6: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

HRPD High Rate Packet Data IM Inter Modulation IP Internet Protocol KPI Key Performance Indicator LCP Link Control Protocol LIU Line Interface Unit LWS Lucent Worldwide Services MAC Medium Access Control MCC Main CDMA Controller MCR Multi Carrier Radio MIP Mobile Internet Protocol MR Modification Request NAK/NACK Negative Acknowledgement NVM Non Volatile Memory OA&M Operations, Administration & Maintenance OHM Overhead Channel Manager OMP Operation and Management Platform PCF Packet Control Function PDA Personal Digital Assistant PDSN Packet Data Serving Node PPP Point-to-Point Protocol PSK Phase Shift Keying PSMM Pilot Strength Measurement Message QAM Quadrature Amplitude Modulation QoS Quality of Service QPSK Quadrature Phase Shift Keying RAB Reverse Activity Bit RATI Random Access Terminal Identifier RF Radio Frequency RLM Reverse Link Modem RLP Radio Link Protocol RNC Radio Network Controller ROP Read Only Printer RSSI Receive Signal Strength Indicator SCI Slot Cycle Index SM Service Measurements SNR Signal to Noise Ratio SNMP Simple Network Management Protocol SIP Simple Internet Protocol

LUCENT TECHNOLOGIES PROPRIETARY 6

Page 7: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

SPAT3G System Performance Analysis Tool 3G SU Software Update TCA Traffic Channel Assignment TCC Traffic Channel Complete TCP Transmission Control Protocol TDU Time Delay Unit TP Traffic Processor UATI Unicast Access Terminal Identifier UCR UMTS/CDMA Radio UDP User Datagram Protocol URC Universal Radio Controller

LUCENT TECHNOLOGIES PROPRIETARY 7

Page 8: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

0. CHANGE HISTORY

Version Changes DateR21 Initial Release October 2003

R22 Updates reflecting new service measurement counts and metrics, reference web sites, metrics cross-reference table and SPAT3G information for Release 22.

October 2004

R23 Updates reflecting new service measurement counts and metrics for Release 23. Updates include:

• Updated the guide with information about the new reverse link overload control translations and the corresponding peg counts

• Introduced new chapter summarizing performance related features in R23 and R24.

• Added new information to the hybrid mode section about SCI and MSM6500 chipset performance.

• Added a new subsection on high processor occupancy rates • Added a link for EV-DO release letters • Updated appendix B with information about EV-DO specific features

in SPAT3G • Updated acronyms list, references and the metrics cross-reference table

February 2005

R24 Updates reflecting new service measurements counts and metrics for Release 24.

• Added information about Celnet Xplorer in the introduction section • Updated section 4 with performance related features in R24 and 25 • Removed negotiation failures from the Established call rate equation • Modified equation for Fast Connect Failure rate • Added section 5.1.2.7 discussing connection failure rate issues in edge-

of-coverage areas • Added section 5.1.2.8 discussing connection failure rate issues in inter-

RNC border areas • Updated section 5.1.6 with details from Lucent Alert 05-0493 (peg

count decrementing issue) • Updated equation for forward throughput in section 5.3 • Added section 5.3.13 discussing core IP network issues affecting data

throughput • Updated appendix B (SPAT3G overview) with screenshots for ROP

call processing failure analysis • Updated table of contents, references and acronyms list • Updated metrics cross reference list

August 2005

LUCENT TECHNOLOGIES PROPRIETARY 8

Page 9: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

References [1] 1xEV-DO RF Translation Application Note # 0 – Introduction

[2] 1xEV-DO RF Translation Application Note # 1 – Timing, Delay and Access Parameters

[3] 1xEV-DO RF Translation Application Note # 2 – Forward link [4] 1xEV-DO RF Translation Application Note # 3a – Reverse Power control [5] 1xEV-DO RF Translation Application Note # 3b – Reverse Overload control [6] 1xEV-DO RF Translation Application Note # 4 – Handoff [7] 1xEV-DO RF Translation Application Note # 5 – High level Protocol Stack Parameter

Optimization [8] 1xEV-DO RF Optimization Guidelines [9] SRD 1853 – 1x EV-DO Packet Data Call Processing

[10] SRD 1880 – OA&M Requirements for the Network (MSC) portion of a 1xEV-DO System

[11] 1XEV-DO and 3G1X Service Measurement Counts Cross Reference Memo – 12/07/02 ( http://nwswww.wh.lucent.com/~fjiang/1XEVSMcrossRef.doc )

[12] 1xEV-DO Troubleshooting Guide ( http://nwswww.ih.lucent.com/~apxlib/cgi-bin/srchdoc.cgi?entry=d26453+ )

[13] 1xEV-DO Performance Expectations document

[14] White Paper on Base Station and Network Configuration for 1xEV-DO – July 30, 2003 (http://nwswww.wh.lucent.com/~yyang7/hdr/do_wp_2.3.pdf)

[15] CDMA RF Performance Analysis and Troubleshooting Guide

[16] 1xEV-DO RNC, AP Operations, Administration and Maintenance (Doc # 401-614-102)

[17] 1xEV-DO RF Translation Application Note # 6 – Flexible Scheduler

[18] Lucent 1xEV-DO System Architecture, http://nwswww.wh.lucent.com/~gaby/do-work.htm

[19] 1xEV-DO Technology Overview (Physical Layer and Signaling) http://nwswww.wh.lucent.com/~gaby/do-work.htm

[20] Call processing exception reporting for 1x EV-DO: OHM, SFM and PCFA11 CP failure error list document (d32365)

[21] 1xEV-DO Service Measurements Manual (Doc # 401-614-326)

LUCENT TECHNOLOGIES PROPRIETARY 9

Page 10: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Useful Web Pages 1xEV-DO Current Engg http://nwswww.wh.lucent.com/~hdrce/ 1xEV-DO FOA http://nwswww.wh.lucent.com/~1xevfoa/index.html 1xEV-DO Perf Mgmt team http://nwswww.wh.lucent.com/~sobers/DO_Perf_CFT/ 1xEV-DO Release Letters http://nwswww.wh.lucent.com/~hdrce/ APX Library http://nwswww.ih.lucent.com/~apxlib/ AskLucent http://asklucent.web.lucent.com/ Cares Tickets (ARs) http://cares.web.lucent.com/ External Technical Support https://wireless.support.lucent.com/amps/ FID/FDD Lookup http://wngpjm.web.lucent.com/cpdr/ LWS Knowledge Store http://servility.ih.lucent.com/msstpi/ Lucent Alerts http://mss.web.lucent.com/bulflash/bulflash.html Lucent Navigator http://navigator.web.lucent.com/ Lucent Product Documents http://www.cic.lucent.com/cicmulti.html SRD website http://nwswww.wh.lucent.com/~seamps/livelink/ Wireless Service Creation http://rfcoresupport.wh.lucent.com/ & Performance Department Wireless University https://training.lucent.com/SabaWeb

LUCENT TECHNOLOGIES PROPRIETARY 10

Page 11: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

1. INTRODUCTION

This document has been compiled to serve as a guide for understanding concepts and techniques necessary to analyze, troubleshoot and monitor performance in a commercial 1xEV-DO network. The primary intent is to present information on ways to assess current levels of service quality offered by the system as well as ways to augment it by identifying and resolving specific performance issues.

There are two main ways to approach the task of performance troubleshooting and monitoring in any network. The approach often depends on the phase of network operation. The two approaches/phases are described below.

For a new network deployment, single-user based drive test techniques are often employed effectively to optimize performance before turning over the network for commercial operation. It involves characterizing RF performance under controlled conditions with use of several test mobiles. There is little real-user traffic to base system level analysis on.

Once in commercial mode, several parameters influence performance over time such as increase in subscriber usage, shift in traffic pattern, cell additions, carrier growth, hardware problems, environment changes, etc. that make the ongoing performance management critical. In this phase, network performance can be gauged via the use of Service Measurements (SM) and system level tools that derive measurements off the abundant commercial traffic. Controlled single user tests may be needed to supplement system level analysis, mainly in localized areas to fine tune performance.

With the above background in mind, it is important to clarify the approach adopted in this guide. It mainly addresses a 1xEV-DO commercial market with the number of cells and traffic large enough to offer a statistically meaningful analysis based on system level indicators.

The document follows a top-down approach in that it looks at system level performance and problem resolution techniques via Key Performance Indicators (KPI) based on available service measurements for 1xEV-DO1. For each KPI, it supplements the system wide performance management by selective drive test based optimization techniques.

In terms of the scope of the document, another point to note is that this guide primarily addresses performance associated with the sub-system involving Access Terminal (AT), Base Transceiver Station (BTS) and Flexent Mobility Server (FMS) components (explained in Section 3). There is a separate document for end-to-end network troubleshooting using single user call traces and associated diagnostics in more detail [12].

1xEV-DO is a new technology in the initial deployment stages as the guide is being written. Although the initial deployment of the Lucent 1xEV-DO product has adequate SM peg counts to allow computations of many KPIs, additional peg counts will further aid the analysis as they become available in the future. For a one-to-one 1xEV-DO overlay on an existing 2G/3G1x network, some of the SM metrics as well as specialized system level tools (such as Handoff 1 Service Measurement counts available through R24.0

LUCENT TECHNOLOGIES PROPRIETARY 11

Page 12: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Matrix, Undeclared Neighbor List, etc.) from the latter can be utilized for troubleshooting analysis [15].

This guide is written by applying general concepts of performance management from 2G/3G1x systems due to many similarities with 2G/3G1x CDMA systems. As the knowledge base grows from experience in commercial 1xEV-DO networks, additional troubleshooting techniques/principles will be incorporated.

One of the key assumptions made in this guide is that all 1xEV-DO systems will be single carrier. At this point, no multi-carrier deployments are on the immediate horizon, and this simplifies many of the performance considerations typically associated with multi-carrier systems. However, with R25 multiple carriers will be supported for 1xEV-DO.

There are several tools available to retrieve and analyze system level performance data. One such tool is WatchMark Corporation’s Prospect. It primarily allows creating reports (Microsoft Excel output) based on Service Measurement data. Another tool that allows a more integrated analysis of not just SM data, but also other system level information, such as ROP alarms and call processing failures, Translations/Configuration summary and Neighbor List analysis, is the Lucent internal tool, SPAT3G (System Performance Analysis Tool 3G).

Celnet Xplorer is a recently developed tool that was used during performance optimization of Verizon Wireless markets. Celnet Xplorer collects data for each data flow and contains detailed information about RF conditions, mobile type and call status. Celnet Xplorer is currently being developed for a controlled introduction in October 2005 timeframe with a GA version to be available for R26. Analysis of Celnet Xplorer data will be implemented in SPAT3G in a phased manner with basic reports scheduled to be available in SPAT3G v4.7.0 in October 2005.

LUCENT TECHNOLOGIES PROPRIETARY 12

Page 13: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

2. USING THIS GUIDE

The document is intended for Lucent internal use. Typical users will be comprised of LWS personnel specializing in RF performance and troubleshooting as well as Customer Technical Advocates.

To get the most from the guide, familiarity with 2G/3G1x performance management will prove helpful, but not required.

Before listing different ways this guide can be employed, it is important to highlight the fundamental questions the guide addresses related to system performance management. These are:

• KPI/Metrics: How to define/characterize service quality offered to the end user?

• Factors: What are some of the factors that can influence these metrics?

• Pointers: How does one detect/isolate the performance impacting factor/root cause?

• Resolution: How to solve/mitigate the performance impact?

This type of information can be applied effectively in several ways, mainly:

Audit system level performance

It is important to assess network performance on a continual basis. Long-term trending of system level performance indicators helps identify issues related to system bottlenecks, malfunctioning hardware, possible translation entry errors, etc as they develop. The information in the guide will provide useful pointers to aid in the effort.

Note that before embarking on KPI studies, it is important to thoroughly scrub translations per recommended values (Refer to [1]-[7], [17]). Over time, recommendations may change, or translation values may get altered mistakenly (such as fine-tuning some other parameter), or not kept up to date with configuration changes (such as when new resources are provisioned). Cleaning up translation settings often turns out to be a low hanging fruit to pick when it comes to improving network performance.

Targeting root cause for a specific problem request

This may be in response to a trouble ticket opened by the customer, citing degradation in a particular service quality indicator. The goal here is to try to focus on the root cause rather than performing a system level audit of all metrics.

The basic assumption in the document is that most performance issues can be funneled down to a few key manifestations. It is left to the user how to categorize the reported anomaly into one of these few KPIs.

After deducing the impacted KPI, the underlying component or failure cause can be identified using the repository of information offered in this document. It often requires eliminating other

LUCENT TECHNOLOGIES PROPRIETARY 13

Page 14: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

causes in order to narrow down to the real one; therefore, it is important to carefully consider all underlying components. For example, reverse overload may be caused by external interference or 1xEV-DO user load and each has a different resolution strategy.

General Reference

An RF engineer new to the performance management area can learn a great deal on steps to maintain and troubleshoot network performance using the principles outlined in the document. The person may or may not have any experience in the area on prior technologies.

KPIs presented in the document are a good way to start delving into the basics of performance monitoring. The information on failure components can also lead to better understanding of how to apply/make use of secondary performance metrics.

LUCENT TECHNOLOGIES PROPRIETARY 14

Page 15: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

3. OVERVIEW OF 1XEV-DO SYSTEM AND OPERATION

Before delving into the core subject of this document, it is worthwhile to present a basic view of the 1xEV-DO system in terms of architecture and operational principles. These are discussed in the subsequent sections.

3.1 System Functional Blocks

Figure 1 illustrates functional blocks in a standalone Lucent 1xEV-DO system. Figure 2 shows the same for a Lucent 1xEV-DO overlay on a Lucent 3G1x network; color codes differentiate the shared from unique components in each system.

1xEV AN

1xEV Element Management

System

FIGURE 1 FUNCTIONAL BLOCKS IN A LUCENT 1XEV-DO SYSTEM

A 1xEV-DO system can generically be categorized into 2 macro segments. These are:

AN – Access Network is the equipment providing data connectivity between a packet switched data network (typically the Internet) and the access terminals. AT – Access Terminal is the device providing data connectivity to a user. An access terminal may be connected to a computing device such as a laptop personal computer or it may be a self-contained data device such as a personal digital assistant.

Next, we present some of the components in each segment of the network.

LUCENT TECHNOLOGIES PROPRIETARY 15

Page 16: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

1xEV EMS

FIGURE 2 FUNCTIONAL BLOCKS IN A LUCENT 1XEV-DO OVERLAY ON A

LUCENT 3G1X HIGH SPEED DATA NETWORK

3.1.1 Access Network

PDSN – Packet Data Serving Node is one of the key components in a wireless Data network. In the downlink direction (Internet to AT), it converts IP packets received from the Internet into PPP stream and passes on to the 1x EV Controller. In the uplink direction, it terminates the PPP layer, structures PPP packets coming from PCF into IP packets, and directs them away from the DO network to the Internet.

AAA server – AAA stands for Authorization, Authentication and Accounting. The AAA server stores the database that contains service and protocol information for each user registered in the 1xEV-DO network. This information is returned to the PDSN using a secure protocol when the user first establishes a session in the network. The PDSN uses this information for authentication. AAA also administers the billing function by storing accounting information obtained from the PDSN. 1xEV-DO Flexent Mobility System (FMS) – It serves as the heart of Lucent 1xEV-DO call processing and management. Its main hardware components include a bank of servers, known as APs and TPs, described below.

AP – Each FMS frame can contain up to 8 Application Processors (AP). These are actually 4 pairs of APs - both APs in a pair run in full active/backup mode, meaning either one can

LUCENT TECHNOLOGIES PROPRIETARY 16

Page 17: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

take control if the other goes down. AP server is based on the Solaris operating system. Each AP has 2 Traffic Processors (TP) and several cells to manage.

TP – TP runs off VxWorks operating system. A Data session served by a FMS has an associated TP. TP handles the direct traffic processing for the data flow between the cell sites and PDSN.

Each FMS contains 2 Cajun switches for traffic exchange between the APs, TPs and the router (described later).

A Lucent 1xEV-DO FMS can also be classified into software functional blocks. The specific functions are mainly implemented on the AP and the TP:

1xEV-DO Controller – It is responsible for call management and RLP (Radio Link Protocol) control functions in the AP and TP. During setup of a Data session, AP assigns and binds a TP for rest of the session until and unless an idle (dormant) mode inter-PCF transfer occurs. Among several functions, TP is responsible for physical layer channel assignment and managing virtual handoffs between cells. From Data protocol stack perspective, it receives GRE (Generic Routing Encapsulation) packets from PCF, sequences GRE stream into RLP packets to pass on to the serving cell in the forward link. In the other direction, it performs frame-selection to choose the best received physical layer frame from various cells in the Active set of the call, terminates RLP packets and performs GRE wrapping to transport PPP packets to the associated PCF for the session. Another important function it handles is the RLP recovery mechanism. On the reverse link, it is responsible for verifying the integrity of RLP frames received over the physical layer, and requesting re-transmission of RLP frames received in error via NAKs. On the forward link, it buffers and re-transmits RLP packets in response to RLP NAKs received from the AT.

PCF – The term Packet Control Function (PCF) refers to the general PCF functionality, which includes both the A10-PCF and the A11-PCF functionality. The A10-PCF is responsible for the data traffic (A10) connection and resides physically on the TP (Traffic Processor). The A11-PCF handles the signaling (A11) interface and resides on the AP within the FMS hardware architecture. The primary function of the PCF is to buffer and transport PPP packets between the PDSN and FMS using GRE (Generic Routing Encapsulation) protocol. As it is evident from the protocol structure in Figure 3, the GRE encapsulated PPP packets are carried over an Ethernet connection (which could consist of one or more routers) using IP datagrams; this is a local IP network separate from the end-to-end IP packet delivery mechanism between PDSN and AT. Also, note that the interface between PCF and PDSN, is also often called A10/A11 connection, per IS2001. The PCF is also responsible for generating accounting information needed by PDSN for each call.

Router – The router performs an important function of distributing traffic from the controller to the destination cell(s) and aggregating uplink traffic from the cell to the Controller. In addition, it also routes traffic between PCF and the PDSN.

1xEV-DO Element Management System (EMS) – The EMS interfaces with FMS to provide OA&M facilities for 1xEV-DO cells and the FMS. It resides on the standard Lucent OMP server platform. Its OA&M tasks include configuration management on 1xEV-DO cells and FMS –

LUCENT TECHNOLOGIES PROPRIETARY 17

Page 18: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

translation change and verify via the EMS GUI2 (Graphical User Interface), software installation, reboot/restart/status, network provisioning; fault management – alarm monitoring, fault detection and recovery, diagnostics; and, performance management – service measurements gathering and storage. 1xEV-DO Base Transceiver station (BTS) – It is the cell site servicing over-the-air physical connections with the AT to transport RLP frames in either direction. It has an EVM (1xEV-DO Modem) hardware unit (similar to Channel Card Unit or CCU in 2G/3G1x BTS), which consists of Forward Link and Reverse Link Modems also known as FLMs and RLMs to transmit and receive physical/MAC layer frames in the two directions. Another important circuit pack is CDMA Radio Controller (CRC). It has two functional subunits – one is Line Interface Unit (LIU) whose primary function is to terminate the HDLC protocol over T1/E1 backhaul, and the other is Main Cell Controller (MCC) which is responsible for BTS wide OA&M control, such as, alarm management, interface to other circuit packets, NVM updates and diagnostic testing. CBR (CDMA Baseband Radio) is another circuit pack responsible for digital and RF signal inter-conversion. On the forward link, at any given instant, AT is tuned to traffic channel from any one cell. On the reverse link, there can be multiple cells in the Active set that are actively receiving, decoding and passing on traffic channel frame contents to the Controller. In addition, cell transmits certain Control Channels to assist AT with the idle mode operation and call set-up process. It includes broadcast messages (similar to overhead messages such as System and Access Parameters messages in 3G1x) and mobile specific messages (such as page, traffic channel assignment, etc.). Another key element is its assignment of MAC Indices. MAC Indices are similar to Walsh Codes. There are a total of 64 MAC Indices, one reserved for each of the overhead channels and the rest assigned to active calls. MAC Indices are important to distinguish signaling messages as well as reverse power control bits sent on the forward link for individual calls. Finally, BTS is also responsible for managing the inner and outer closed loops of reverse power control with the AT.

2 EMS GUI is the interface in the EMS that allows entering / modifying translation parameter values for various components in the BTS and the FMS.

LUCENT TECHNOLOGIES PROPRIETARY 18

Page 19: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

PCF PDSN 1xEV BTS

1xEV AT

CallControl

Airlink T1/E1

IP IP

802.3T1 T1

HDLC HDLC

GRE GRE PPP PPP

IP IP TCP UDP

TCPUDP

FixedEndSystem

RLP RLP

Internet

1xEV Controller

100 Base T

IPIP 802.3802.3

Router

IP IP

A10Interface

AAA

FrameSelector

RLP

UDPIP

UDP IP

PHY PHY 802.3MAC MAC

TCP

Ethernet

802.3

TCP

FIGURE 3 PROTOCOL STACK DIAGRAM FOR LUCENT 1XEV-DO SYSTEM [9]

3.1.2 Access Terminal

Also known as the mobile, it can operate in one of 2 different modes, based on the layers of the protocol stack it manages.

• Connected to laptop: In this split function mode, AT terminates physical, MAC and RLP layers in the Protocol stack. All higher layers (PPP, TCP/UDP, IP, Application, etc) are terminated in the laptop itself.

• Self-connected unit (such as PDA): In contrast to the above mode, AT terminates all the layers in itself. All the functions mentioned on the AN side have peers in the AT (e.g., physical layer channel establishment, reverse power control, RLP recovery, sequencing/de-sequencing between RLP packets and PPP stream, protocol conversion between PPP and IP, as well as between TCP/UDP and IP, etc).

The analysis and troubleshooting discussions in the remainder of the document mainly involve AT, BTS and 1xEV Controller components of the network.

3.2 1xEV-DO System Operation

Unlike 3G1x technology (defined by IS2000 standards), which can support both Voice and Data calls in a standard 1.25Mhz carrier, 1xEV-DO is a Packet Data only technology. It is defined by

LUCENT TECHNOLOGIES PROPRIETARY 19

Page 20: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

IS856 standards. The 1xEV-DO air interface, mobile and network components have been carefully designed to offer much higher data rates/throughputs to the end-user on the forward link compared to 3G1x systems. The reverse link is largely similar to that of 3G1x.

3.2.1 Forward link

The 1xEV-DO forward link is the key differentiator that sets the technology apart from 3G1x. It offers peak rates of 2.4 Mbps compared to 153.6 kbps in 3G1x. The various rates include 38.4, 76.8, 153.6, 307.2, 614.4, 921.6, 1228.6, 1843.2 and 2457.2 kbps. The forward link is time multiplexed; there is no concept of Walsh code sharing among logical channels/users. There is also no concept of separate fundamental and supplemental traffic channels per user. Each user is allotted entire sector-carrier power at any instant without any power control for the transmission at a given rate.

The forward link transmission is in terms of slots; each slot is 1.66ms. The Pilot channel structure also differs considerably compared to 3G1x systems. All cells, for a small portion within every slot, send the Pilot channel synchronously. The mobile computes received Ec/Io (or Signal to Noise Ratio) for each Pilot during this synchronous interval. This SNR measurement is one of the key parameters used by the mobile to select and request a specific rate. Unlike 3G1x systems, the rate decision process is in the mobile and not in the cell. This allows for quicker rate adaptation to changing RF conditions, quicker transition to a stronger sector to receive forward link data, and minimal signaling overheads, compared to 3G1x systems. The mobile selects a specific rate to meet a target packet error rate (hard-coded to 1% in current mobiles). It is important to note that the transmitted cell site power to the user stays constant, so the mobile has to adjust rates to maintain the QoS (in 3G1x systems, cell has to adjust rate and transmit power to maintain QoS).

1xEV-DO uses a more sophisticated modulation technique to support high-speed data rates – 16-QAM or 8-PSK modulation (lower rates use conventional QPSK). The other distinguishing 1xEV-DO attribute is the use of Incremental Redundancy on forward link. Basically, for multiple slot transmissions (except the higher couple rates, all other rates span over multiple slots), the cell sends data first and then redundancy bits in an incremental manner only if the mobile is having trouble decoding the contents. This results in high utilization of the physical layer.

On the forward link, only one sector is transmitting all the data to a given user. As mentioned above, it is the mobile that requests cell to transmit on a specific rate. While in an active call, the mobile constantly sends these requests via DRC (Data Rate Channel) bits on the reverse link. It addresses these DRCs to a specific cell from which it desires transmission. Such a cell is called DRC pointed cell. When another pilot in its Active set becomes stronger, the mobile starts pointing its DRC to this new cell. This process of switching active sector transmission is called Virtual Soft handoff. It is similar to the anchor transfer of forward link supplemental channel performed by the cell site in Lucent 3G1x systems; in case of 1xEV-DO, the process is managed by the mobile and inherently faster.

The initial Lucent 1xEV-DO system uses a “proportionally fair” scheduler, which attempts to maximize the aggregate sector throughput while managing delay (data starvation) experienced by the disadvantaged users. Cell evaluates DRCs requests from all mobiles and chooses the one with the highest rate. Due to short-term RF variations, different users experience SNR peaks at different times; the one experiencing strong signal gets the data - this is the “proportional” part where user served and hence, the assigned rate is based on signal strength. Considering long-

LUCENT TECHNOLOGIES PROPRIETARY 20

Page 21: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

term averages, everybody gets roughly equal transmission time (number of slots) – this is the “fair” part.

3.2.2 Reverse Link

As mentioned earlier, there are many similarities in the reverse link between 1xEV-DO and 3G1x systems. The supported rates are 9.6, 19.2, 38.4, 76.8 and 153.6 kbps. There is also a Pilot channel transmitted by each mobile on the reverse channel to aid coherent demodulation at the cell site. The soft/softer handoff process and the Active set maintenance procedures are similar (Tadd, Tdrop, Tcomp, TTdrop concepts of IS2000). The mobile can enter into a maximum of 6-way soft handoff, depending on translation settings. The mobile transmission is power controlled by the cell site to maintain a 1% reverse frame error rate. Unlike on the forward link where the target error rate is hard coded at the mobile, the one on the reverse link can be controlled by a translation called “Reverse Target Frame Error Rate” on the Sector EMS GUI page.

There are some differences, however. The physical layer frames are 26.66ms in duration (20ms frames used for 3G1x). The reverse link carries DRC bits to report forward link requested rates. There is no concept of separate fundamental and supplemental channels. There is only one logical traffic channel per mobile – the mobile dynamically changes rate based on available data backlog and a set of rate transition probabilities. The mobile starts with low transmission rate and ramps it up based on these probabilities. It includes Reverse Rate Indication information on each frame to indicate the corresponding transmission rate.

As mentioned above, each mobile decides the transmission rate on its own. To control the reverse link integrity and prevent RF overload, cell has two options. One is to transmit a Rate Limit message on the forward link control channel limiting the maximum rate used by the mobiles. The other mechanism is to set the Reverse Activity Bit (RAB) on. RAB is sent on the Medium Access Control channel – the MAC channel contains RAB, reverse power control commands and DRC lock bit for Active calls on the sector. When the RAB bit is on, some of the users can probabilistically halve their rate on the next frame. The percentage of such users is specified in translations. If the bit is off, the users can double their transmission rate unless they are already at maximum rate (153.6kbps).

LUCENT TECHNOLOGIES PROPRIETARY 21

Page 22: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

4. SUMMARY OF PERFORMANCE RELATED FEATURES IN PRODUCT RELEASES

This section summarizes the key performance related features that are part of the current release (R24.0) and the previous release (R23.0). Features that are scheduled for the next release (R25.0) are also discussed to provide the user with information about future performance impacting features in the product. For a comprehensive list of features available in each release, the user is encouraged to refer to the release letters.

4.1 Performance related features in release 23.0

The following performance related features are available in Cell/RNC release 23.0:

• FID-8048.1 1xEV-DO Translation parameters for 1xEV-DO Reverse Link Overload Control: The reverse link overload control parameters as implemented in release 2.0 (FID-8048.0) are tunable parameters, which are preset in the system and cannot be viewed and provisioned by the user. This feature enhances the previous functionality by allowing the user to provision the overload control parameters manually. The user will be able to provision the parameter via the EMS GUI on a per system (service node) basis or per sector basis, with the per sector data overriding the per service node data.

• FID-10752.0 Test Carrier Support in 1xEV-DO: This feature enables 1xEV-DO carrier access to be restricted to a special class of mobiles. With this feature, a new translation is provided at the EMS for the user to indicate to the system that a particular carrier is put in test mode.

• FID-10607.0 1xEV-DO RNC Performance Improvements: This feature delivers performance improvements in a number of areas that allow for capacity increases to be accommodated. A large part of this feature and the most significant change is to the traffic allocation algorithm that now uses TP-Pooling to attempt to balance the loads evenly on each TP. The capacity improvements include increase in support from 80 cells to 100 per RNC, from 10,000 AT sessions per AP to 20,000, from 5,000 AT sessions per TP to 10,000, and from 65,000 AT sessions per frame (PCFA11) to 120,000.

• FID-10795.0 Service measurement architecture changes for inter-SN and inter-RNC active handoffs: This feature provides accurate service measurement reporting for active soft handoffs between cells that have the same cell IDs but are on different service nodes.

• Support for Subnet ID: This release provides a feature to support Subnet ID on the cell that is globally unique according to 1xEV-DO standards (IS-835). The issue is that the 1xEV-DO network uses Subnet ID instead of System ID (SID) as the network identification. Subnet ID is the 104 Most Significant Bits of the Sector ID or UATI and is broadcast over the air as part of network overhead information in Sector Parameters Message.

LUCENT TECHNOLOGIES PROPRIETARY 22

Page 23: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

4.2 Performance related features in release 24.0

The following performance related features are available in Cell/RNC release 24.0.

• FID-12205.0 Target sector handoff service measurements: This feature provides a set of 1xEV-DO service measurements that is pegged against the target sector for soft/softer handoffs. The current set of soft/softer handoff service measurements for 1xEV-DO is based upon the sector that received the Route Update message from the AT - which could be the strongest member of the active/serving set of sectors for that AT. This existing set is used to identify hand-out problems for a "cluster" of sectors but not for an individual sector. The actual conditions contributing to handoff failures may often be at the target/candidate sector for the handoff – hence the need for a set of target sector handoff service measurements to help identify individual sectors with hand-in difficulties.

• FID-12373.0 1xEV-DO Prior session: In 1xEV-DO, a critical problem is that many AT's (mobiles) are following the logic to reconnect to a prior session as part of an Inter-RNC Idle Hand-off as specified by the standards. However, the Lucent system does not perform this functionality, and instead brings up a new session and strands the older session, causing interruptions in service and blocking system resources. This critical feature will implement the Prior Session method to bring the system into compliance with the standards. New service measurements will be needed to count Prior Session metrics separately. In addition, if this feature is mapped back into releases where service measurements cannot be added, only the non-SM part will be mapped. This feature has resulted in improved performance in Inter-RNC idle handoff scenarios.

• FID-10898.0 1xEV-DO NeighborSector form improvements: Currently, the NeighborSector configuration form requires the service provider to manually enter the IP addresses of the neighbor sector associated primary AP and associated backup AP, and other redundant information already entered at the Service Node level. The NeighborSector form needs to be revised so that it no longer requests the redundant data. This feature is expected to be delivered in R24 SU3.

• FID-11003.0 1xEV-DO OA&M shared indicators: This feature implements the capability to show shared cell equipment in base stations that support both EV-DO and CDMA carriers. Status display pages will be updated to show the existence of EV-DO carriers in mixed cells. This will alert technicians that both CDMA and EV-DO carriers may be effected by changes in the status of shared equipment.

• FID-10801.0 EMS “Read-only” display option for 1xEV-DO: This feature provides for a "Read-Only" display option for users accessing the 1xEV-DO system via the EMS GUI.

• FID-9123.0 Bulk provisioning capability for 1xEV-DO: This feature provides a mechanism to allow service providers to manage their data provisioning in a way that supports bulk updates. Previously, only Lucent engineers had access to this capability.

• FID-8219.0 Support for multiple 1xEV-DO carriers phase 1: This feature provides support for multiple 1xEV-DO carriers in the same cell. It includes Modcell 1.0/2.0/3.0/4.0 and all the required changes in the network (1xEV-DO RNC) and OA&M (EMS). This feature supports load balancing and only one carrier per URC.

LUCENT TECHNOLOGIES PROPRIETARY 23

Page 24: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• FID-8219.01 1xEV-DO infrastructure changes for multiple carriers: This is a feature that provides infrastructure changes to support Multiple DO Carriers in a subsequent release. The feature is applicable to the OMP, DO-RNC, and both 1xEV-DO only cells and 1xEV-DO/3G-1x mixed mode cells. These changes include modifications to existing parameters and introduction of new parameters to the following EMS Configuration Management Screens: Service Node, BTS, CBR, EVM, Sector and CDM. Additionally, the format of the output file for DO service measurements is changed for both RNC and HCS sector counts to reflect their actual sector-carrier granularity. Within each sector, the counts will be reported for each carrier..

• FID-8880.4 1xEV-DO support for MCR: This feature provides the 1xEV-DO support for the Multi-Carrier Radio (MCR) board on a Modcell 4.0 shared radio cell. The MCR supports a maximum of 8 Cellular or up to 11 PCS carriers over a contiguous 15 MHz bandwidth on one board. Furthermore, on-board measurement devices allow for CLGC functionality without the TDU. This feature only allows the use of the MCR in a shared radio mode. Therefore, the MCR needs to be equipped in both CDMA & EVDO translations. While multiple CDMA carriers can be configured on the radio, only a single EVDO carrier is allowed.

• FID-9121.0 Software update capability for 1xEV-DO: This feature introduces a Software Update (SU) capability for the 1xEV-DO system. The 1xEV-DO product currently has downtime associated with all software deliveries. This capability introduces the ability to update the 1xEV-DO RNC Software during maintenance upgrades without system downtime.

• FID –10735.0/10735.1 OMP-FX/EMS backward compatibility: This Feature allows the EMS user to provision “non-global objects” such as Cell, Sector, CDM etc from a nth release OMP to a n-1 RNC using Bulk Provisioning. Provisioning includes operations such as Add, Remove, Update and Replace. Phase 2 of this Feature allows the EMS user to provision “global objects” such as PDSN, Service Node etc from a nth release OMP to a n-1 RNC using Bulk Provisioning.

4.3 Performance related features scheduled in release 25.0

The following performance related features are scheduled to be available in Cell/RNC release 25.0. It must be noted that the feature content might be changed by the R25.0 release time frame.

• FID-12370.0 Service Measurements Enhancement Phase 1: This feature shall provide enhanced service measurements collection for 1xEV-DO calls with greater resolution, improved accuracy for performance metrics and greater granularity for failures due to other reasons. Some of the proposed peg counts include Fast Connect Assignment counts, Paging counts, Peak and average active counts, RAN authentication failures counts and AN/AT initiated connection failure pre/post TCA counts.

• FID-12146.0 Handoff Matrix: This feature is to create the EV-DO handoff matrix functionality. Handoff Matrix data will allow the service providers to make adjustments to the EV-DO neighbor lists to improve handoff performance. The tool shall provide data similar to the existing CDMA HOM tool, which shows the number and types of handoffs from all the faces on an ECP to every other face handed off to. In addition, it will provide the

LUCENT TECHNOLOGIES PROPRIETARY 24

Page 25: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

date/timestamp information showing when the data was collected and data on undeclared neighbors.

• FID-10858.1 Call processing exception report enhancements: This feature adds assert reporting and additional throttling controls for the asserts and 1xEV-DO call processing exception reports. In addition, this feature gives the operator the option to filter call processing exception reports into a separate ROP log file.

• FID-10519.0 1xEV-DO Cell processor overload control: The purpose of the feature is to ensure overload control for the cell processors. This feature will help the system performance by providing a protection algorithm against overload thus taking care of issues before they result in performance degradation.

• FID-10607.2 FMS/RNC Carrier count and session capacity increase: This feature will support the increase of maximum number of 3-sector carriers and cells to 130 from 100. In addition, this feature will also support the maximum number of sessions per TP to 13,000 and the maximum number of active connections per TP to 520. This will result in the AP supporting maximum number of 26,000 sessions.

• FID-8219.0/8219.1 Support for 1xEV-DO multiple carriers: This feature provides support for multiple 1x EV-DO carriers in the same cell. It includes Modcell 1.0/2.0/3.0/4.0 and all the required changes in the network (1xEV-DO RNC) and OA&M (EMS). This feature supports only one carrier per URC. It also supports load balancing.

• FID-9123.1 1x EV-DO Bulk provisioning enhancements: This feature provides a series of enhancements for the generally available version of the bulk provisioning capability introduced by FID9123.0.

• FID-11003.1 1x EV-DO Cell OA&M convergence for mixed networks: The purpose of this feature is to develop a fully integrated fault and configuration management interface for cell equipment in mixed networks that contain EV-DO only cells, CDMA only cells, and mixed EV-DO/CDMA cells. This interface will allow customers to more easily manage their mixed data/voice network. When the convergence feature is available, EV-DO cell OA&M functionality will migrate from the DO-RNC platform to the CDMA RCS platform. This results in a single display, control, and configuration point for cell equipment supporting both EV-DO and CDMA. EV-DO Call processing provisioning and status will continue to be supported by the RNC and EV-DO EMS.

LUCENT TECHNOLOGIES PROPRIETARY 25

Page 26: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5. KPI OPTIMIZATION AND TROUBLESHOOTING

At a higher level, the performance of a 1xEV-DO system can be quantified via a few primary metrics, named here, the KPIs. When used collectively, they are indicative of the quality of service experienced by a typical end user. In a more general sense, they may also be applied to characterize performance in any wireless Data network.

These KPIs are:

• Established Call Rate

• Drop Call Rate

• Data Throughput

As may be evident from the names, the KPIs together help gauge the quality of a network in terms of its ability to offer successful call set-ups, minimal unintended call interruptions and fast download speeds to an end user – all essential ingredients for a rich Data experience.

In the following sections, we treat the KPIs individually. For each KPI, we first provide a high level definition and then, explore the underlying elements/parameters that may negatively influence it along with the associated troubleshooting/rectification techniques.

One key point to note is that the ability to isolate a factor influencing any of the KPIs hinges on the degree to which the problem manifests. The latter itself depends on many factors such as the extent of the problem area, amount of call activity in the problem area, usage patterns, persistence of the problem, etc. In general, larger than normal degradation in a metric stands out much easier and hence, facilitates its own detection and resolution.

LUCENT TECHNOLOGIES PROPRIETARY 26

Page 27: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.1 Established Call Rate

In essence, this KPI represents the success rate associated with setting up a Data call. Mathematically, it can be expressed as follows:

RequestsConnectionFailuresConnectionquestsConnectionRateCalldEstablishe −

=Re

The metric can be computed on a per sector basis.

The various terms in the above equation are computed from SM peg counts collected in the AP and the TP on a per-sector basis as follows:

Connection Failures – sum of (counts below are pegged in the TP):

• AT-init Connection Attempt failures-Reverse link not acquired • AN-init Connection Attempt failures-Reverse link not acquired • AT-init connection attempts failures – no response to traffic channel assignment • AN-init connection attempts failures – no response to traffic channel assignment • AN-initiated connection attempt failures – no resources available • AT-initiated connection attempt failures – no resources available • AT-initiated Connection Attempt Failures – other failures • AN-initiated Connection Attempt Failures – other failures

Connection Requests – sum of (counts below are pegged in the AP):

• AT-initiated Connection Request • AN-initiated Connection Request

A Connection Request is an event where the AT makes a request to establish a call with the AN, or vice versa. If the cell is able to assign a Traffic channel, the connection may still fail before establishing a stable traffic link for several reasons, which are aggregated as Connection Failures.

The cell may not be able to assign Traffic channel for several reasons. One reason could be it experiences trouble activating traffic channel element typically due to hardware error in the EVM.

The other reason could be that the cell is already processing maximum number of connections and cannot free up resources to serve additional users. Any type of call processing overload (such as both the TPs of an AP going in overload due to excessive traffic load) can also prevent Traffic channel allocation. These two causes essentially represent call blocking due to resource shortage. Note that per the current implementation, the call is never denied due to any power overload conditions unlike in 2G/3G1x networks.

Once assigned a traffic channel, the connection may still fail for several reasons. A call setup failure in this mode is primarily linked to some type of RF impairment (poor RF coverage, pilot pollution, excessive RF loading, etc.)

LUCENT TECHNOLOGIES PROPRIETARY 27

Page 28: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

The Established Call Rate may alternatively be viewed as the deficit between Connection Requests and the calls that did manage to come up stable on the Traffic Channel. A complementary metric commonly used is called Access Failure rate. It can be represented as (1 – Establish Call Rate) and signifies failed call or access attempts.

Service providers define their own metrics to track performance. Verizon wireless has created Failed Connection Attempts (FCA) as an equivalent metric for Established call rate. Lucent engineers might encounter this metric when attending to Verizon wireless specific performance issues. At this point, the FCA metric equation is the same as Lucent’s Established call rate equation.

To improve the Established Call Rate, it is important to understand the constituent factors influencing the deficit. We catalogue and explain these in the subsequent sections. But, first we briefly describe the call setup process.

5.1.1 Call setup process

Figure 4 illustrates a typical flow of messages between AT and the AN for an AT initiated call. It bears many similarities with a typical origination call attempt in a 2G/3G1x network.

When the AT receives the command to establish a call (such as, from the laptop connected to AT when user tries to establish a fresh PPP session or reactivate a dormant3 session), AT first checks to see it is current with the overhead messages, and if not, updates them.

On the next available access channel slot, the AT sends a Connection Request message via an access probe. Along with the Connection Request message, the AT always sends a Route Update message listing the received pilot strengths. If a response is not received within a finite time, the AT will retransmit the probe. It will keep sending the probes (based on the “Number of Access Probes” configuration parameter value in the Sector and Service Node forms), each at incrementally higher power, until it receives AcAck (Acknowledgement) from the cell.

Once the cell is ready with the necessary resources and once the cell receives the DRC Cover Indication from the controller, it sends a Traffic Channel Assignment message (TCA) to the AT. If there were multiple pilots in the Route Update message, cell will transmit TCA on all the sectors.

On receipt of the TCA, the AT initiates reverse link traffic channel transmission. There is no traffic channel information sent at this time, only DRC and reverse pilot bits. If there were multiple pilots in the TCA, the mobile will include all these Pilots in its Active set. The AT will

3 The first time AT establishes traffic channel with the AN, it is called a new session. The session may last for several hours or days depending on the AN configuration. Within a session, the AT can switch between active and dormant modes numerous times. Data is exchanged between AT and AN in active mode over traffic channels. After a short period (typically of the order of a few seconds depending on the AT/AN setting) of inactivity, the AT goes into dormancy, that is, tears the traffic link and goes into idle mode. When a fresh set of data appears, either side (AT or AN) can initiate a connection out of dormancy (generically termed reactivation) depending on who has data to send. Note that a new session is always initiated by the AT. The reactivation can either be from the AT or AN.

LUCENT TECHNOLOGIES PROPRIETARY 28

Page 29: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

gradually increase its transmit power until it receives acknowledgement from the cell site or it times out.

Command received by the AT to initiate a call. AT updates overhead messages and waits for the next Access Slot to transmit

Configuration Negotiation Procedures

Ack

Traffic Channel CompleteTraffic Channel Complete

RTC Ack Send RTC Ack

Mobile Acquired IndicationSend DRC + Pilot and ramp up Reverse Traffic Channel

Traffic Channel Assignment Send Traffic Chn Assignment

DRC Cover Indication

AcAck Allocate Traffic Channel Response

Allocate Traffic Channel Request

Route Update & Connection Request Message

Controller BTSAT

FIGURE 4 CALL FLOW FOR AN AT INITIATED CONNECTION

When cell site acquires the reverse link transmission, it sends an acknowledgement called RTCAck to the AT. Once the mobile receives RTCAck message, it sends a Traffic Channel Complete (TCC) message to the cell site. Cell site acknowledges it back with an Ack message.

If there are any negotiation procedures to be completed, a set of messages is exchanged next to arrive at the proper configuration. For example, there are certain parameter values that the

LUCENT TECHNOLOGIES PROPRIETARY 29

Page 30: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

mobile assumes by default per standards. This saves the control channel bandwidth (in 2G/3G1x networks, such information is transmitted via overhead messages – such as via System Parameters message and Access Parameters message). If the cell site is configured with non-default parameter settings, it must negotiate (exchange) these with the mobile. After the negotiation is finished, the call setup is complete at this time and the actual Data contents can now be carried over the air. However, any negotiation failure does not contribute to the Established Call Failure rate metric.

Many of the access enhancements first introduced by IS95B standards are also available in a 1xEV-DO system. For example, after sending the first probe and while waiting for the AcAck, the mobile can switch to a stronger Pilot. This is similar to Access Probe Handoff (APHO) specified in IS95B/IS2000 standards. After receiving the AcAck and while waiting for the TCA, the mobile can again switch to the previous or different stronger Pilot. This is similar to Access Handoff (AHO) specified in IS95B/IS2000 standards. Finally, the TCA can contain multiple Pilots that the mobile can place in its Active set as it enters Traffic channel. This is similar to Channel Assignment into Soft Handoff (CAMSHO) specified in IS95B/IS2000 standards. Notes that these features are implemented as core functionalities in Lucent 1xEV-DO system and cannot be disabled via translations.

The process for AN initiated call (akin to a mobile termination attempt in 2G/3G1x network) is similar to the above except that the cell site first pages the mobile to invoke Connection Request message from the AT. In a wireless Data network, all AN initiated calls are typically reactivations from dormancy; the network does not initiate a fresh session with the mobile. A mobile on the other hand is capable of initiating a new session or reactivation from dormancy.

5.1.1.1 Fast Connect Calls

Fast Connect is a special type of AN initiated call. When the AT has been dormant for 5 seconds or less (controlled by the tunable parameter called “Suspend Timer”) and the AN has a fresh set of data to transmit to this AT, the AN uses this procedure to bring a dormant session back onto the Traffic channel. Basically, it just sends the TCA to the AT, which consists of Pilots in the AT’s Active set at the time prior call release. The idea is that 5 second is a relatively short interval and the AT must have been in the vicinity, so it is safe to locate it on the prior set of Pilots. It is appropriately called Fast Connect since there is not much time spent on the Control/Access channel message exchange and the attempt is reactivate the Traffic channel quickly.

The AN sends the TCA once it has ascertained necessary resources on the AN. This results in the peg for Fast Connect Requests. If the cell times out waiting for TCC from the mobile, it pegs it as Fast Connect Failure – No response to traffic channel assignment. If the cell does not acquire the reverse link after the TCA was sent, then the Fast Connect Failure – Reverse link not acquired count is pegged. Both the above counts are pegged on the AP on a per AP basis. Per sector granularity is not available. These counts are not pegged towards any of the counts for AN Initiated calls.

For such calls, Fast Connect Failure rate metric (for AT/AN Initiated calls) can be derived. It can be defined on a per AP basis as follows:

LUCENT TECHNOLOGIES PROPRIETARY 30

Page 31: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

RequestsConnectFastacqnotRLtFailuresFastConnec

TCAtoespNoFailuresConnectFast

rateFailureConnectFast ___R_ +

=

One may want to compute a composite metric for Established Call Rate consisting of counts for AT Initiated, AN Initiated and Fast Connect calls together at the per sector-carrier level. However, since the pegs for Fast Connect do not have the granularity down to the sector-carrier basis, the Established Calls rates will have to be computed separately for AT/AN Initiated calls (per sector-carrier) and Fast Connect calls (per AP).

Many of the factors commonly impact AT/AN Initiated calls as well as Fast Connect Calls. These are discussed in the subsequent sections.

5.1.2 RF access failures

As the name suggests, this type of failures impacts access performance due to RF performance limiting factors. By definition, the RF access failures influencing the deficit between Connection Request and successful call setup occur after a Traffic channel is assigned4 by the AN, but before Traffic Channel Complete message is received from the AT. The failures represent the inability to complete call setup due to inadequate signal strength received by the AT and/or AN. Basically, the RF impairments break down the message flow and/or the signal acquisition resulting in an incomplete call setup.

Call setup is a multi-stage process, and hence there are various places where RF breakdown may occur. There are separate SM counts to trap failures (for both AN initiated and AT initiated connections) at each stage. These are:

• After sending a Traffic Channel Assignment Message to the AT, AN times out waiting to acquire DRC channel from the AT. The SM count to trap this is called Connection Attempt Failure – Reverse link not acquired.

• After AN successfully acquires AT on the reverse link, it sends an Acknowledgement message, called RTCAck, to the AT. AT is expected to Ack it back with the TCC message. If AN does not receive the TCC after 3 attempts of transmitting the RTCAck, it will result in a failed call attempt. The SM count to capture this event is called Connection Attempt Failure - No response to Traffic Channel Assignment. Note that the peg count is a misnomer; it conceptually represents “no receipt of TCC”, not “no response to TCA”.

All the above stages apply regardless of an AT or AN initiated Connection Request. Accordingly, there are separate SM counts for connection requests (that is, AT or AN initiated

4 From AT perspective, failure may occur much sooner but AN will not know until it times out waiting for response to the Traffic Channel Assignment.

LUCENT TECHNOLOGIES PROPRIETARY 31

Page 32: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Connection Request), connection failures and separate metrics (AN initiated RF failure rate and AT initiated RF failure rate) to track them.

Next, we elaborate on potential RF factors influencing the failures along with some mitigation strategies. Their impact may be felt anywhere along the call setup process although with varying degrees. It is not uncommon to see similar symptoms for problems induced by more than one of the RF factors. Thus, a comprehensive understanding of all RF factors is called for to be able to eliminate unrelated RF factors and single out the root cause.

5.1.2.1 Lack of RF coverage

Concept/Reason for failure

AT in weak coverage area (such as AT receive power less than –105 dBm and best Pilot SNR < -10dB, mobile transmit power > 20 dBm), by definition, suffers from excess pathloss resulting in its inability to close one or both the links. Typically, the reverse link runs out of coverage first given that the 1xEV-DO forward link has roughly 10 dB advantage (Pilot channel transmitted at full power). Areas suffering from heavy shadowing such as in-building locations or stretch of route heavily blocked by building, tree, etc can likely experience this type of failure.

Failure Symptoms and Identification strategies

Such problems are best detected via drive tests. The drive test area can be narrowed to within the coverage of a few cells by identifying cells via SM, which tend to exhibit unusual degradation in many QoS indicators, including:

• Low Established call rate • High Drop Call rate (defined in section 5.2) • High Page Failure rate • High RLP re-transmission rates (ratio of RLP Octets Re-transmitted to RLP Octets

Transmitted on the forward link, ratio of Missing RLP Octets Requested to RLP Octets Received on the reverse link – these counts are collected in the TP for the DRC pointed sector).

Suggested Mitigation Techniques

Having identified select areas, drive test based RF optimization techniques can be employed as detailed in [8]. The process typically entails analyzing drive test data to arrive at a suitable conclusion, such as, increasing cell power or adjusting antenna azimuth, or in the worst case adding repeater5 (especially for in-building environments) or cell to fill in low signal strength areas.

Improving the coverage may also result in secondary effects such as overall performance improvement as interference is mitigated due to reduced traffic channel power requirement.

5 Current 1xEV-DO deployments are based on regular cells; generic repeaters may first require thorough characterization to ensure they maintain decent transmit SNR in the forward direction.

LUCENT TECHNOLOGIES PROPRIETARY 32

Page 33: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.1.2.2 Excessive Pilots/Non-dominant Pilots

Concept/Reason for failure

Presence of too many pilots in an area often impacts forward link performance due to interference it creates to the pilot(s) being actively tracked by the AT. Typically, there is no dominant pilot in such a problem area, that is, there are several pilots with weak comparable strengths, often accompanied by constant pilot thrashing (dominant sector changing quickly). Another scenario is where a distant sector overshoots to create/contribute to a localized non-dominant pilot region.

Call setups are susceptible to failure in multiple pilot regions since AT has no single strong pilot to tune to. While AT can enter into Traffic Channel with multiple pilots in the Active set, there may still be some degradation since AT can tune to only a single pilot at any given time on the forward link. Any delay in completing the call set-up (for example, AT may miss the first RTC_ack message before it tunes to a stronger pilot, the message will be resent a few frames later) could impair the established call rate.

Failure Symptoms and Identification strategies

There are several ways to attain a pointer to the potential existence of this problem.

On a relative basis, cells exhibiting higher than normal rate of connection requests with 3 or more Pilots provide one such indication. There are also separate SM counts to flag “Connection Requests with 2 Pilots” and “Connection Requests with 3 Pilots” events. All these counts are reported in the AP for the sector that received the Connection Request. [Sum of AT Initiated Connection Requests and AN Initiated Connection Requests] minus [Sum of these multiple pilot counts] reflect the number of Connection Requests with single pilot. The following four metrics in SPAT3G can verify the existence of the pilot pollution problem:

• Connection Request with One Pilot (%) • Connection Request with Two Pilots (%) • Connection Request with Three Pilots (%) • Connection Request with Four or more Pilots (%)

Guidelines similar to the one used to detect pilot pollution in 2G/3G1x networks, based on SM counts for the number of Tadd pilots reported on the PSMM, can be used. Higher than normal rate of Soft/Softer HO fail due to maximum number of legs reached (ratio of SO_SOR_HOFF_FAIL_MAX_NO_LEGS_REACHED and SO_SOR_HOF_ATTMPT) when the Maximum Legs in Handoff configuration parameter is set to a value higher than 3 can also indicate pilot pollution problems.

Another pointer, though a weak one6, is excessive activity for Soft and/or Softer handoff attempts. It is likely that other KPIs such as Dropped call and Data throughput may also exhibit impairment in these areas.

6 Soft/softer handoff activity is more symbolic of mobility, but along with several other indicators tends to suggest a potential multi-pilot region.

LUCENT TECHNOLOGIES PROPRIETARY 33

Page 34: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

For one-to-one overlay on a Lucent 2G or 3G1x network, data and tools from the underlying 2G/3G1x infrastructure may also be employed for identification of excess-pilot areas. Note, however, that this type of comparative analysis will be valid only if the antennas are shared across technologies, or in physical proximity with similar attributes (azimuth, downtilt, height, beam width)

• 2G/3G1x cells showing high secondary CE usage relative to Primary (or Total) CE usage, or Total Walsh Code relative to Primary Erlangs, are potential indicators to excess pilot regions.

• Handoff Matrix and Undeclared Neighbor List analysis tools in SPAT3G can be used on the underlying 2G/3G1x network to flag cells overshooting from a distant region. Every effort must be made to contain their coverage/reduce their interference in order to not only benefit call set-up rate, but also to improve overall performance (drop call rate, throughput, etc).

Suggested Mitigation Techniques

Similar to the Lack of RF coverage reason, drive tests can be employed to analyze and resolve such problem areas. Typical RF optimization techniques include adjusting cell site power and/or antenna downtilt of one or more neighboring cells to create a dominant pilot or fewer visible pilots. Antenna downtilt usually are more effective in controlling coverage of overshooting sectors. Refer to [8] for more details.

5.1.2.3 Poorly constructed Neighbor Lists

Concept/Reason for failure

Not having a fine tuned neighbor list is a common source of access failures. In essence, it prevents AT from being on the best pilot during the access process. A poor neighbor list may typically be missing a nearby pilot to which AT will otherwise handoff to, or may have integrity issues, such as, lack of reciprocity or suffer from PN ambiguity, discussed below.

Failure Symptoms and Identification strategies

The first check for cells with high access failures should be to review the neighbor list entries and ensure they follow basic rules, such as other sectors of the same cell as well inward facing sectors of the surrounding first tier cells are included as neighbors and the correct AP IP addresses are populated for the neighbor sectors.

If the Soft Handoff Attempt to Assign rate metric is high, it typically indicates that some of the neighbor sectors have incorrect AP IP addresses listed in the NeighborSector form.

The Neighbor List Alert (NL Alert) feature in the SPAT3G tool can be employed effectively to fine tune neighbor lists. The integrity can be verified in terms of:

• Reciprocity - if cell X is on neighbor list of cell Y, then so should be Y on the neighbor list of X.

LUCENT TECHNOLOGIES PROPRIETARY 34

Page 35: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• No PN ambiguity - cell X should not have two cells with same PN on its neighbor list7, or if cell X and Y are in handoff, where X sponsors neighbor cell A and cell Y sponsors cell B, A and B should not be configured with the same PN8. Otherwise, it is possible that AT ends up establishing a traffic link on the weaker of the two cells with the same PN, thereby, impacting access performance due to noise component brought in by the stronger non-traffic bearing PN at the AT demodulator. The 2-way ambiguity problem is likely to have larger impact on the dropped calls

• Cross-face omission – It is a good idea to include other sectors in the cell as high priority neighbors.

• AP IP alert – The IP addresses of the neighbor sector’s AP’s have to be defined correctly in order for handoffs to occur.

For a one to one overlay on a 2G/3G1x network, use of HO matrix and Undeclared Neighbor List analysis tools in SPAT3G can be used on the underlay network to detect missing neighbors. This can help improve neighbor lists and hence overall performance on both the networks.

Suggested Mitigation Techniques

Fix any reciprocity issues and resolve PN ambiguity by changing PN offset of one of the two cells with common PN. Ensure that the neighbor sectors have the correct AP IP addresses populated.

If it is a standalone deployment, it is best to shape neighbor lists with drive tests. Neighbor list omissions can be detected for example, where a call drops and shows up on a strong Pilot that was not in the prior Active set or the Neighbor List message sent to the mobile. Opening the Remaining set can also help identify missing neighbors, if they are strong enough for the AT to detect and report them, either via Route Update or Searcher info message. Refer to [8] for more details on drive test based neighbor list refinement techniques. As traffic grows and more system level 1xEV-DO tools/SM capabilities are available (such as HO Matrix and Undeclared Neighbor List), further fine-tuning of the neighbor lists can be performed.

5.1.2.4 Improper PN planning

Concept/Reason for Failure

Improper PN planning could result in PN assignments such that AT cannot resolve signal from 2 different cells. There are two common scenarios in which this problem manifests.

• PN offsets are time-shifted versions of the same PN sequence. Each PN offset is 64 chips apart. Large propagation delays, as in the case of signal from a distant overshooting cell, can alter the identity of a PN at the AT, which in the worst case could make it

7 Commonly known as one-way PN ambiguity.

8 Commonly known as two-way PN ambiguity.

LUCENT TECHNOLOGIES PROPRIETARY 35

Page 36: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

indistinguishable from a nearby cell. When AT sends Route Update message, it will include this common PN it sees. The call will be set-up only on the close-in cell, not on the far-away cell since it is actually transmitting on a different PN. As a result, only the close-in cell will key up traffic. When AT attempts to demodulate traffic channel, it will not only include valid traffic energy from the close-in cell, but also weigh in noise component from the distant cell. Consequently, the traffic signal-to-noise ratio will nose-dive, impacting access performance.

• Another scenario is where two cells are assigned the same PN, but are not physically separated enough such that at the AT the two cells becomes indistinguishable. This triggers the same performance impacting phenomenon as described above in the case of two far-way cells with different PNs.

In both cases, the problem appears because there is not enough separation in the PN offset of the two cells. Prudent PN planning can help minimize the problem. The first scenario is mitigated by using adequately large pilot_increment (available as Pilot PN Sequence Offset Index Increment in the Service Node translation form) so that AT can still resolve signals from cells received with large relative difference. Special attention should be heeded for situations where signal is expected to propagate much farther than typical range, for example, due to low loss experienced over water bodies, signal from elevated cellsites, etc. The second scenario requires proper PN re-use margin.

Failure Symptoms and Identification strategies

The NL Alert feature in SPAT3G is designed to uncover PN ambiguity issues and can potentially help identify the problem with inadequate PN reuse margin. See the previous section (5.1.2.3) for more details. However, this approach cannot detect all problem configurations. In such cases, drive tests are often effective, though not simplistic, in isolating potential problem(s).

A common symptom to look for is where FER on the forward link appears poor while the mobile receive power and Pilot Ec/Io remain at decent levels. This follows from the discussion earlier that traffic signal to noise ratio weakens since the AT ends up demodulating noise from the cell not in the Active set. Usually, this is a very telling indicator of two or more cells being received with same PN (either of the two problem scenarios) assuming there are no other issues (such as cell software/hardware defects). To confirm the problem, the cell suspected to be facing PN interference should be turned off to verify if the Ec/Io associated with this PN still remains strong. If it does, the next step will be to identify cells in the vicinity configured with same PN or explore the possibility of a distant overshooting cell. Latter may require meticulous investigation of terrain/morphology/cell elevation to identify the potential for a benign propagation environment in the area.

Suggested Mitigation Techniques

Once the failure scenario is identified, it is relatively easy to resolve the problem. For the first scenario (overshooting cell), standard RF optimization techniques, such as antenna downtilt and/or azimuth adjustments can be attempted to contain its coverage. The other option is PN re-assignment, but care should be exercised to ensure it does not become a problem elsewhere. Using a large PN increment will also mitigate the problem, though will require quite a bit of PN planning re-work.

LUCENT TECHNOLOGIES PROPRIETARY 36

Page 37: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

The second problem scenario can be resolved by maintaining sufficient number of cells between PN re-use. A judicious design at the outset will go a long way in minimizing a potentially large amount of effort spent resolving it later.

5.1.2.5 External Interference

Concept/Reason for Failure

Presence of interference raises the noise floor, which raises the amount of power needed to overcome it, further elevating the noise levels. Since the cell always transmits at fixed full power, AT may not be able to decode the signal in presence of strong interference levels on the forward link. On the reverse link, the AT can run out of power needed to close the link in the wake of strong external interference. In either case, the Established Call Rate will be sacrificed. The level of impact, and hence, the ability to detect/resolve, may depend on the amount of interference. An intermittent/weak transmitter may often be very difficult to identify.

There are several flavors of external interference that could impact performance of a network. It can exist on the forward or the reverse link.

• An unlicensed transmission within the CDMA band is the most common cause of external interference. Usually spectral clearance tests are performed in a new deployment to identify and rectify such sources. If the tests were not executed, such as, when additional spectrum is allocated to an existing operator, continued operation of interference sources could cause significant performance issues as the commercial service is activated on the carrier.

• Interference can also be caused by spill over of a legitimate transmission from the adjacent channel due to narrow spectral spacing, and/or, inadequate filter roll-offs implemented by the transmitter or the receiver.

• Another flavor of interference is where inter-modulation (IM) products, generated in the receiver front-end amplifier due to a strong signal on a neighboring channel (although within the amplifier band), cause significant performance degradation.

Failure Symptoms and Identification strategies

Detecting presence of interference is not straightforward and locating the exact source may be even more difficult. It often requires shutting down the desired transmission (that is, turn off service) to pinpoint the root cause. Thus, it is best to isolate external interference upfront before pressing system into commercial system.

The most telling indicator on the reverse link is much higher levels of RSSI rise, Eb/No setpoint and RFER, all happening as a result of inflated noise floor. There are SM counts available to inspect these per sector performance metrics – RSSI (Short and Long Term Average RSSI Rise pegged in the MCC), reverse Eb/No setpoint (Total Setpoint for Reverse Outer Loop Power Control – separate count for each reverse frame rate ranging from 0 to 153.6kbps, pegged in the TP), and RFER (Reverse Link Frame Error and Total Reverse Link Frame counts, pegged in the TP). Corresponding performance metrics for these peg counts available in SPAT3G include: RSSI (One Second Average RSSI Rise, One Second Peak RSSI Rise, Ten

LUCENT TECHNOLOGIES PROPRIETARY 37

Page 38: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Second Average RSSI Rise, and Ten Second Peak RSSI Rise), and RFER (Reverse Frame Error rate, Reverse Retransmission Failure rate and EVM DRC Erasure rate).

From prior field experience with such type of interference in 2G/3G1x networks, several sectors facing the interference source will exhibit a simultaneous increase in the RSSI, Eb/No setpoint and RFER. This is one of the surest indicators of external interference. To pinpoint the problem, more expensive/elaborate means such as use of spectrum analyzer at/around the cell site may be called for.

On the forward link, drive testing may help provide initial pointers to the presence of external interference. Basically, the higher interference will result in very high mobile receive power, but poor Ec/Io (Io goes up due to interference) and poor fwd FER. This, however, may also be caused to due to missing a neighbor or an inadequate search window size; hence, these have to be eliminated first along with any potential hardware/software issues. Turning off some cells or service in the affected area may be required to allow for thorough spectral clearance tests.

On either link, one way to isolate the problem related to IM interference is by using an in-line attenuation to the mobile or the cell. If the measured signal power at either end (mobile receive power on the forward link, or cell RSSI on the reverse link which is supposedly much higher than the noise floor) increases at a much faster rate than the additional attenuation value, it suggests the existence of IM products. Basically, the IM harmonics created due to amplifier non-linearity de-emphasize at a larger rate than the additional attenuation value while the desired inband signal attenuates at the nominal rate; therefore, the composite signal still attenuates at a larger rate.

Suggested Mitigation Techniques

As above, identifying the precise location/source of the interference is the most difficult first step. Once the source is identified, the remedy is simply to turn it off.

5.1.2.6 Inadequate Search Window Size

Concept/Reason for Failure

There are two main search windows whose sizes are of interest – Active and Neighbor sets. When searching the PN space to track/detect pilots in its various sets, the AT utilizes a finite window width in order to keep the search time small. The size is optimized to suit local terrain and cell topology in order to maximize the likelihood of AT finding a pilot and its multipath components in the smallest possible time.

Different considerations go into determining Active and Neighbor set search window size. Accordingly, there are two failure modes.

• When Active set search window size is not configured large enough to allow AT to receive/track a strong multipath component. Typically, the default search window size is good enough to cover most of the significant components. In some rare cases, especially in large delay spread environments, this problem may surface. AT cannot combine/demodulate the multipath since it falls outside the search window, rendering it as an interferer.

LUCENT TECHNOLOGIES PROPRIETARY 38

Page 39: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• When AT fails to detect a neighbor due to an inadequate Neighbor search window size. More than the presence of multipath, it is the relative path difference at the AT between the reference and the neighbor cells that drives this window width. Since PN offset transmitted by each cell is a time-shifted version of a single PN sequence, the difference in propagation delays to the AT requires the Neighbor search window to be wide enough to span the largest delay differential. Otherwise, a particular neighbor may fall outside the search window, causing it to interfere with the Active set.

The call may fail in either case due to RF impairment if the mobile is not able to detect a strong pilot or a multipath component, and instead attempts to setup a call on a weaker one.

When changing Active set search window size, it is usually a good idea to make the one used at the cell site for reverse link demodulation, called Reverse Traffic Window Size, as well as for acquiring soft handoff leg, called Data Window size, comparable. Both the translation parameters are on the EMS GUI for BTSs.

Failure Symptoms and Identification strategies

SM counts do not provide a clear clue as to the existence of a search window problem.

For an overlay network, a basic sanity check is recommended to ensure Active and Neighbor search window sizes are comparable to the 2G/3G1x underlay.

Drive test is often the most productive method for identifying search window issues. LDAT3G can be used to alert and recommend proper neighbor search window sizes. Based on the mobile logs, it employs the phase-offset information for various pilots reported on the Route Update Message for an active call to identify potential for larger neighbor search windows. However, it cannot identify situations where the existing neighbor search window is small enough to let AT even detect the neighbor Pilot. Towards that end, another tool that could be of particular significance, especially in tough to resolve problem areas, is a pilot scanner, which employs independent GPS time reference to provide an accurate view of multipath components from each cell.

Suggested Mitigation Techniques

Once the need for opening the search window for an Active or Neighbor set is identified, it can be increased via translations. If a proper size is not known, it can be expanded in smallest possible increments until performance is improved and/or mobile logs confirm proper detection at the problem area.

5.1.2.7 Edge of coverage

Concept/Reasons for failure

At the edge of a 1xEV-DO system, where 1xEV-DO coverage ends and 3G1x coverage continues, ATs attempt session setups even in poor RF conditions. The ATs attempt to setup sessions several times if they fail. This results in the number of connection attempts peg count to increase resulting in a poor value for the Established Call rate metric.

LUCENT TECHNOLOGIES PROPRIETARY 39

Page 40: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Failure Symptoms and Identification strategies

For sectors at the edge of 1xEV-DO coverage, it is likely that the Established Call rate metric is degraded. The primary cause for failure will likely be the connection attempt failures due to no traffic channel complete received from the AT. It is also likely that a majority of the failures in these regions are caused by connection requests associated with RATI session setup requests. RATI session setup requests is computed as the difference between the Session setup requests and the Inter-subnet idle transfer attempts counts.

This problem can also be identified by collecting data using the Celnet Xplorer tool for the cells in the coverage border. Analysis reports will provide a view of how many of the connection attempts are coming from ATs in poor coverage area and are far away from the border cells.

Suggested Mitigation Techniques

A R24 private load is being proposed to trial a potential solution for this problem. In this private load, connection attempts from ATs in poor RF (poor Ec/Io values) and too distant from the cell site can be denied. If this mitigates the problem, this proposal will become part of a future software load.

5.1.2.8 RNC border issues

Concept/Reasons for failure

ATs are required to re-register with the system as it crosses an RNC border. RNC borders are defined by the change in color codes. In such border areas, it is likely that the ATs ping-pong between the RNCs resulting in paging failures, misdirected data and connection failures.

Failure Symptoms and Identification strategies

For sectors at the RNC border, it is likely that the Established Call rate metric and the AT/AN initiated connection attempt failure rate is degraded. However, with the current service measurements, it is not easily possible identify this problem. Drive data analysis in the RNC border area would provide a clear picture of the problem.

Suggested Mitigation Techniques

RNC borders need to be carefully engineered in order to minimize such performance problems. The following factors need to be considered while engineering RNC borders:

• Define RNC border in low data usage areas

• Avoid RNC border that runs parallel to major highways/roads

• Avoid RNC borders near major office buildings, water bodies and island cells (a few cells in a RNC surrounded by cells on a different RNC)

• If a new RNC is needed in heavy usage area (capacity need), instead of adding new cells on to a new RNC, re-distribute cells evenly across RNCs

LUCENT TECHNOLOGIES PROPRIETARY 40

Page 41: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.1.3 Failures due to faulty hardware

Concept/Reason for failure

Presence of faulty or mis-configured hardware can influence the performance on the affected cell. Hardware failures can occur in various forms and intensity. For example, a CRC or EVM board entering into faulty operation registers sudden increase in access failures. At times, the affected hardware device may only show one errant behavior, such as, increase in call setup failures, but other indicators such as dropped call rate remain sane.

A different flavor of hardware problem could be a faulty RF antenna cable with high return loss likely resulting in smaller than desired cell footprint. In many cases, the hardware simply goes into a faulty state, the hardware itself is fine – software restore or stable clear rectifies the problem.

It is also possible to witness intermittent hardware outage instead of an outright malfunction, such as loss of T1/E1 during certain hours.

Failure Symptoms and Identification strategies

A first check should involve careful review of alarms generated and flagged by fault management module of the network. Certain issues associated with receive path (such as RSSI imbalance across main and diversity paths), malfunctioning hardware devices, backhaul outages, etc. can easily be identified through the alarms.

The ROP analysis report in SPAT3G provides information on these alarms at the AP, TP and BTS levels. A pareto analysis of call setup failures by specific hardware type based on the ROP data can offer a tell-tale sign of issues with a specific hardware/cell based on the over-abundant concentration of the failures. To make this type of analysis meaningful, however, care should be taken to normalize the failures with the amount of carried traffic.

Suggested Mitigation Techniques

Once the suspected malfunctioning hardware is identified, the first step is to attempt to rectify through a remote software restore or download. If the performance continues to suffer, it may require cell visit to reseat the hardware and in some cases, to ensure tight cable connection. If all on-site diagnostics fail, the last step will be to replace the hardware.

5.1.4 Heavy Reverse Link traffic

Concepts/Reasons for failure

As the reverse link load increases, either due to higher number of active users or increased data upload activity, the overall interference floor seen by each cell increases. In order to overcome the raised interference floor, AT has to transmit at much higher power levels. An AT already at the cell edge can no longer close the reverse link as its signal arrives with a weak signal-to-noise ratio at the cell. This would sacrifice access performance. In many cases, the access probe itself may not be heard at the cell; as a result, no SM counts will be pegged, but end-user experience will degrade.

LUCENT TECHNOLOGIES PROPRIETARY 41

Page 42: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Failure Symptoms and Identification strategies

The indication of high reverse overload condition can be inferred by increase in several SM metrics, relative to normal levels observed on other sectors

• High values for peg counts introduced in R23 to measure Reverse Link Overload Control including Short Term Average Fast Control Low Load count (SM_SHTM_AVGLLC_FASTCTRL), Short Term Average Fast Control Medium Load count (SM_SHTM_AVGMLC_FASTCTRL) and Short Term Average Fast Control High Load count (SM_SHTM_AVGHLC_FASTCTRL). These peg counts are available on a per sector basis.

• High Reverse Frame Error rate (ratio of Reverse Frame Error and Total Reverse Frame counts, pegged in the TP)

• High average setpoint for various rates (ratio of Total Setpoint for Reverse Outer Loop Control and Total Number of Frames for Reverse Outer Loop Control at each rate, pegged in the TP)

• Increase in Number of Connections Force Released and Abnormal Release rate, pegged in the MCC

• Higher values for One Second and Ten Second Average/Peak RSSI Rise, pegged in the MCC

• Large number of Total Connection Requests per hour, pegged in the AP

Some of these indicators may also be influenced by reasons other than heavy reverse traffic, such as:

• External interference resulting in elevated RSSI levels, high RFER and high average set points.

• A malfunctioning device in the RF receive path, such as UCR or low noise amplifier, causing consistent or intermittent RSSI spikes, thereby, reducing the coverage.

• A component breakdown in the RF receive path, or improper antenna cable connection, incapacitating one of the two diversity branches. This would tend to increase the Eb/No requirement as well as the power control variation. Not only will this require increase in power transmitted by the mobile, but the variation will also push the cell further into instability.

Hence, careful investigation of all possible causes may be required before a particular one can be confirmed. Though not always necessary, one distinction may be that there are large number of users consistently on the cell, as can be judged from Total Connection Requests per hour (sum of AT and AN Initiated Connection Requests). It is less likely that just a few users consistently load up the cell, usually a pointer, to external interference or faulty hardware.

Suggested Mitigation Techniques

LUCENT TECHNOLOGIES PROPRIETARY 42

Page 43: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

It is always a sound practice to track/trend various performance indicators, especially, Connection Requests and short/long term average/peak RSSI rise to be able to take a timely corrective action instead of waiting till the last moment when performance is impacted.

Once the cell is identified as a heavy traffic-bearing cell impacting access performance due to reverse link loading or overload, there are a few options available to offload the traffic:

• Reduce the overload control thresholds to lower the reverse transmission rates. With R23, the HDR Reverse Link Overload Control (HROC) tunable parameters have become translation parameters. The Lucent Reverse Overload Control mechanism utilizes these translatable thresholds to manage the integrity of reverse link performance. Refer to [5] for an overview of operation and associated translations. Lowering these thresholds will reduce the probability that AT will transmit at a higher rate, under load. Lower transmission rates require less AT transmit power on an average. In essence, reverse overload will kick-in at lighter loading levels thereby mitigating coverage reduction at the expense of throughput performance on the reverse link.

This is not a preferred solution in the long-term. It may, however, be acceptable as a stopgap measure instead of denying service to users until longer-term measures, stated next, are pursued. However, the customer must be made aware of the trade-offs before attempting any threshold adjustments.

• One solution is to increase Pilot overlap such that other links get into the active set of the call, mitigating the traffic channel power requirement lowering the overall interference. From the perspective of mitigating AT transmit power requirement, higher handoff diversity should never be undesirable. However, careful consideration is warranted to characterize potential impact on the forward link performance as well as increase in cell resource utilization (e.g., channel element hardware).

• Add a 1xEv-DO carrier. As scoped earlier, this document focuses on single carrier 1xEV-DO deployments, so no further discussion is included here. It is mentioned here for future considerations. In general, assuming the service provider has the required spectrum, this is the most common approach used to offload traffic used in 2G/3G1x networks.

• Add a cell to serve the heavy traffic area. If adding a carrier is not a possibility due to lack of spectrum, adding a cell (geographically separated from the high traffic cell(s)) is often an effective, although a long-lead, solution to improve coverage and hence, permit higher levels of offered traffic per unit area. Care must be taken to ensure proper RF optimization is performed to minimize excessive pilot with cells in the vicinity – otherwise it may impact forward link performance.

5.1.5 TP overload

Concept/Reasons for failure

There is a safeguard mechanism to block further calls on the Traffic Processor (TP) as its processor occupancy exceeds a certain threshold. This is to preserve Data performance for

LUCENT TECHNOLOGIES PROPRIETARY 43

Page 44: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

existing users as the traffic served by the TP increases beyond the designed limits. Doing so will minimize packet delays or drops as TP is unable to keep up with the load. Calls blocked due to TP overload will result in the deficit between Connection Requests and Established calls.

Note that there are two TPs per Application Processor (AP) in the FMS, so blocking will only occur if both of them are in overload.

The TP occupancy threshold to trigger an overload is controlled by a tunable parameter called “TP Utilization Threshold”. It is currently set to 85%.

Failure Symptoms and Identification strategies

There are a host of SM counts and metrics on TP utilization. There is a direct count for Number of New Sessions Denied due to TP overload (corresponding metric in SPAT3G is Sessions Denied due to TP Overload). The count is actually pegged at the AP when both its TP are in overload. Other metrics at the TP level related to overload are Number of Times TP in Overload as well as Total Overload Duration.

However, it is a sound practice to monitor TP Processor Usage, pegged per TP as well as the packet drop rate (Ratio of counts on number of dropped packets and packet arrival rate, when TP is in overload, pegged in each TP) on a regular basis to allow preemptive action.

Suggested Mitigation Techniques

Normally, if Lucent recommendations for system design and provisioning are followed, it will allow proper projections to minimize TP overload. Refer to [14] for Lucent recommended Network Engineering Guidelines for 1xEV-DO systems. Should such a situation arise, however, one solution may be to re-configure and allocate fewer BTSs per AP/TP assembly. This way each TP will serve smaller fraction of the traffic in a geographical area, providing relief to its processor occupancy.

Over time, software/hardware upgrades may also become available to reduce the occupancy for the same amount of serviced traffic, alleviating the need to re-configure cell to AP/TP mapping.

5.1.6 Maximum number of connections reached

Concept/Reasons for failure

Currently, the scheduler in the ASIC at a BTS can support a maximum of 48 users per sector. When the number of active users reaches 48 (including soft/softer connections), successive call attempts are denied on the sector, lowering the established call rate.

There is also another ASIC limitation that a 3-sector cell can support a maximum of 93 channels. Although these are pooled across 3 sectors, it is possible to trigger this per-cell bottleneck before any individual sector hits the 48-user limit.

Note that standards allow 64 MAC Indices per sector. After allowing for overhead channels, only 59 are available for assignment to active users. If and when the existing ASIC limitations are removed, the available MAC Indices will become the per-sector bottleneck.

LUCENT TECHNOLOGIES PROPRIETARY 44

Page 45: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

The maximum allowed users on a sector are controlled by a translation called “Maximum Number of Users”. Refer to [1] and [5]. When the cell reaches this threshold and receives a new call attempt, it first attempts to make some room by attempting to push an active user into dormancy if its dormancy timer is about to expire9. However, if there is no such user, the Connection Request is rejected by sending Connection Deny message to the AT. This lowers the Established Call rate.

In 1xEV-DO, a critical problem is that many AT's (mobiles) follow the logic of reconnecting to a prior session as part of an Inter-RNC Idle Hand-off as specified by the standards. However, the Lucent system does not perform this functionality, and instead brings up a new session and strands the older session, causing interruptions in service and blocking system resources. This could cause Established Call rate to degrade even in good RF coverage areas.

This has been fixed in RNC/Cell release 24.0, with the implementation of the Prior Session method to bring the system into compliance with the standards. New service measurements have been added to count Prior Session metrics separately. In addition, if this feature is mapped back into releases where service measurements cannot be added, only the non-SM part will be mapped.

Failure Symptoms and Identification strategies

The bottlenecks described above are easy to spot in SM. They are measured at the sector level as Number of Times Maximum Connections Reached.

In R24.0 RNC load, there is a software error that causes the counter for the number of active users per sector to not be decremented and this results in blocked calls/connections if the counter reaches the provisioned maximum. Refer to Lucent alert # 05-0493 for additional information and mitigation techniques.

Suggested Mitigation Techniques

One solution is to reduce the coverage of the cell in an attempt to offload traffic to the neighboring cells/sectors. This, however, assumes the neighboring cells have the room to handle more traffic, which may not be the case all the time. An alternative is to add multiple carriers. This is a more graceful solution than adjusting coverage, which may not always yield desired traffic offloading. At this point, only single carrier deployments are considered. Where there are spectrum availability issues, adding a cell may be the last resort solution. This, of course, requires longer lead times in order to accommodate zoning, cell installation, RF calibration and optimization stages before turning the cell over to commercial service.

9 There is a mechanism to release calls that have been idle for certain time and nearing their dormancy timer expiration. The idea is that these calls are likely to go into dormancy soon, so releasing them sooner may not cause much end-user impact while at the same time reducing interference to others.

LUCENT TECHNOLOGIES PROPRIETARY 45

Page 46: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.1.7 Traffic Channel Activation failure

Concept/Reasons for failure

This represents a traffic channel allocation failure at the time of call setup. In essence, it can be construed as Channel element initialization failure. AN will send a Connection Deny message to the AT upon encountering the failure. The problem is likely to be due to some type of hardware failure in the EVM board at the cell site. Note that this case is different than when cell has reached the Maximum Number of Users limitation.

Failure Symptoms and Identification strategies

This type of failure is accumulated into a SM count called Traffic channel Allocation Failure for setup – Resources Not Available in the Candidate Sector. The count is pegged in the AP for the candidate sector not able to allocate traffic channel. Note that this count is not separated for AT and AN Initiated call attempts. It is also important to note that the Connection Request will only be denied if all sectors listed in the Route Update message sent along with the Connection Request message cannot allocate any resources.

It is also possible that ROP logs this failure, which could help with easier identification.

Suggestion Mitigation Strategies

When there is a lot of traffic channel activation failures concentrated on a particular CE or FLM/RLM, one corrective action will be to stable clear the cell. If that doesn’t help, running remote diagnostics or reloading the card is suggested. If that does not help, site visit is called for. First, reseat the card manually and bring it to back to service. If the problem persists, as a last resort, replace the affected channel element hardware.

5.1.8 Authentication error

Concept/Reasons for failure

There are two stages of authentication. The first one is for new session that involves authentication with the PDSN and AAA as part of PPP setup. Any failures are logged in the AAA. This information exchange occurs after the traffic channel has been established; so these failures do not impact Established Call Rate.

The second stage of authentication is of more interest, since it directly influences the Established call rate. Once a session is successfully established, each subsequent transition from dormant to Active state undergoes this authentication stage using the protocol parameters set at the beginning of the session. In essence, this is authentication of the access channel and it is performed before any traffic channel is assigned. The purpose is to validate that the requesting AT is the true owner of the session previously established.

Note that this stage does not involve much message exchange. Security information from the MAC header of the Connection Request message is decoded and validated at the AN. As a result, any failure is more likely to be due to a real authentication failure rather than RF reasons. In this

LUCENT TECHNOLOGIES PROPRIETARY 46

Page 47: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

case, AN sends a Connection Deny message instead of Traffic Channel Assignment message. It contributes to the deficit between Connection Requests and Traffic channel assignments.

Failure Symptoms and Identification strategies

There are several counts and metrics to indicate authentication failure during various sub-phases of the access layer authentication. These are pegged on the TP. These include:

• RAN Authentication Failure rate

• Number of Security Protocol Negotiation Failures

• Number of Session Key Length Negotiation Failures

• Number of key Response message timeouts

• Number of Session Security Digest Mismatches

• Number of Session Security ATKeyComplete message time-outs

• Number of packet discards in SHA-1 Authentication Protocol

• Number of Authentication Failures for postponed Security packets on old 1XEV_call Control

• Number of authentication failures for postponed security packets on new 1XEV-Call Control

• Number of Encryption Attribute negotiation failures

Given the lack of per-sector granularity, the impact of authentication failures on the Established Call Rate on a sector basis is difficult to establish. In any event, it is important to determine if particular TP(s) is experiencing unusual amount of authentication failures in an attempt to curb them.

Suggested Mitigation Techniques

Once an authentication breach is suspected, first step should be to determine the type of AT. Since this information is not available in system level tools such as ROP that custom developer traces may be needed. In any event, the goal is to detect any incompatibility between the AT and AN. This can be confirmed if the failures are pre-dominantly occurring on the same make/model of ATs.

After the failures are limited to a few ATs, a logical step will be to turn in their identification to the operator for further action.

5.1.9 High Processor Occupancy rates

Concept/Reasons for failure

LUCENT TECHNOLOGIES PROPRIETARY 47

Page 48: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

When equipment processor occupancy levels exceed certain thresholds, it is likely that new connections are blocked and existing connections are dropped. There are several components in the 1xEV-DO system for which occupancy levels can be monitored including the Application Processor (AP), Line Interface Unit (LIU), Forward Link Modem (FLM) and Reverse Link Modem (RLM).

In order to improve the performance of the RLM, the DRC per rate peg counts have been disabled in R23.0. A new implementation to peg these counts without impacting the RLM is being devised and is likely to be fixed as an MR in release 24.0. Cell sites loaded with older releases would still be pegging the DRC per rate counts.

Failure Symptoms and Identification strategies

With R23, there are several service measurement counts that have been introduced to monitor the occupancy levels. High values for these counts over several days could potentially result in blocked and dropped calls. These counts include:

• AP Peak Processor Occupancy (AP_PEAK_PROC_OCCUPANCY)

• AP Average Processor Occupancy (AP_AVG_PROC_OCCUPANCY)

• LIU Peak Processor Occupancy (LIU_PEAK_PROC_OCCUPANCY)

• LIU Average Processor Occupancy (LIU_AVG_PROC_OCCUPANCY)

• FLM Peak Processor Occupancy (FLM_PEAK_PROC_OCCUPANCY)

• FLM Average Processor Occupancy (FLM_AVG_PROC_OCCUPANCY)

• RLM Peak Processor Occupancy (RLM_PEAK_PROC_OCCUPANCY)

• RLM Average Processor Occupancy (RLM_AVG_PROC_OCCUPANCY)

Suggested Mitigation Techniques

If the AP occupancy levels are high and with high traffic volume, consider re-homing the cell site to another AP with lower occupancy levels. If LIU, FLM and/or RLM occupancy levels are high, consider off-loading traffic from the cell in question to surrounding cell sites.

LUCENT TECHNOLOGIES PROPRIETARY 48

Page 49: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.2 Drop Call Rate

This metric is commonly viewed as a measure of unintended interruptions during a Data session, primarily due to loss of RF link or handoff failures. Mathematically,

sConnectiondEstablisheReleasesConnectionUnintendedRateCallDrop =

As defined above, it is the ratio of unintended or abnormal call releases to the total number of Established Connections. Established Connections are those that were assigned Traffic channel and came up stable on the traffic channel, that is, they completed call setup process successfully. The metric can be computed on a per-sector basis.

There are two SM counts, the sum of which provides the total number of dropped calls or the unintended connection releases. Both these counts are collected in the TP and pegged on the DRC pointed sector at the time of call drop. The counts are:

• Connection Released – RF Link Lost – this happens when AN declares loss of RF link when all the legs lose lock on the reverse link DRC (see below)

• Connection Released – Other Reasons – as the name suggests, there could be several reasons for this, such as, call states not in sync between the AP and TP, cell cannot build/send messages, timeouts, internal software error, etc.

• Soft/Softer Handoff Failures – No AT Response – this represents connection releases due to failure to receive the Traffic Channel Confirmation message from the AT to the Traffic Channel Assign message sent by the AN to add or drop a handoff leg.

The term Established Connections is simply the numerator in the metric for Established Call Rate discussed in section 5.1.

A call may be dropped in one of the following common ways:

• Cell can no longer decode DRCs. When cell receives 4 or fewer good DRCs in any 5-second sliding interval, it declares loss of RF link and drops the call. If there are multiple legs supporting the call, all legs must lose the lock on the DRCs before the call is released. This can occur under following conditions:

o Although AT is transmitting, cell cannot “hear” the mobile (due to pathloss, external interference, mobile has tuned away to receive a voice call on 3G1x or has spent too much time on a particular 3G1x search in hybrid mode10, etc.).

10 In hybrid mode, AT periodically searches 3G1x while in an active 1xEVDO connection. If it receives an incoming voice termination, it will take the voice call and not tune back to 1xEVDO until after the voice call is released. At times, it may spend a long time on 3G1x, such as, in weak 3G1x coverage areas or when performing idle handoffs, autonomous registrations, etc. This could manifest as a Lost call on the 1xEVDO system. A future enhancement will attempt to remove such tune-away induced releases from Lost call peg by estimating AT’s designated 3G1x page slot and the time it “disappears” from 1xEVDO relative to this slot.

LUCENT TECHNOLOGIES PROPRIETARY 49

Page 50: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

o AT can no longer decode the forward link and cannot recover before the DRC supervision timer expires (240ms), and stops transmitting.

o AT times out, waiting for response to an Ack required message (such as Route Update Message or Traffic Channel Complete message) even after transmitting it 3 times, and stops transmitting.

• Handoff failure. Cell sends a Traffic Channel Assignment message to indicate change in the Active set (drop or add a Pilot) to the AT. AT responds with Traffic Channel Complete (TCC) message. However, if the cell times out waiting for the TCC, it de-allocates the traffic channel to the mobile. This is another flavor of dropped call11 and currently gets pegged as Soft/Softer Handoff Failures – No AT Response12. A separate metric Dropped Call rate due to HO failures has been defined in SPAT3G to measure the effect of handoff failures.

In all of the above mechanisms, cell sets the “FT_valid” field in the Quick_config message sent on the forward link Control channel to 0, thereby, signaling the phone to get off the traffic channel.

Given that the AT always listens to only one Pilot any given time on the forward link, it results in a make before break type transition when the AT changes the serving sector. This is called Virtual Soft handoff and is unique to 1xEV-DO. One may think the lack of full soft handoff diversity may increase the likelihood of handoff failures and drop calls. However, 1xEV-DO AT has the ability to switch to a dominant sector much faster, compensating for the lack of soft handoff continuity.

Note that, to a Data user, a dropped call itself may not inhibit experience so much as it does to a voice user. This is because either the network or the AT may be able to re-establish the connection soon after a Drop call without requiring end-user intervention. This is a feature available on many wireless Data systems including 1xEV-DO.

A re-activation upon drop is possible, especially, if cell or RF conditions are conducive for a successful call setup. For example, a neighbor list omission can result in a dropped call, but the AT can quickly acquire Sync on this strong Pilot leading to successful call setup/resumption of data transfer. However, if the dropped call was due to lack of RF coverage or cell problem, re-activation will likely fail.

One can argue that the inherent ability to re-establish connection makes the drop call rate less significant for Data systems. Nevertheless, this is an important metric tracked by Data service operators for several reasons:

11 A future enhancement at the cell may include ignoring the TCC timeout. Instead the cell will keep repeating TCA until it receives TCC from the mobile, or the DRC supervision timer at the mobile expires. The latter will lead to cell not being able to decode the DRCs and forcing it to release the call. This follows from the line of thinking that a call should not be torn down due to failed handoff; if the RF link impairment persists, the call should be dropped subsequently when the RF link is lost.

12 In a future release, a separate SM peg count will be available to represent connection releases due to handoff failures. At that time, it can replace the soft/softer handoff failure count in the dropped call equation.

LUCENT TECHNOLOGIES PROPRIETARY 50

Page 51: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• Carry over of mindset from voice networks that drop calls offer tangible view of performance interruption on an otherwise stable call.

• Drop calls are often reflective of prevailing poor RF conditions, which will impact Data performance in other ways (e.g., low throughput).

• Some drop calls may not necessarily result in successful re-activation, especially in areas of poor RF coverage or cell software/hardware malfunction.

• Perceived delay due to drop/successive call setup may be unacceptable, especially, for certain time-critical applications, such as voice over IP, or audio/video streaming.

Next we discuss each of the potential contributors to Drop call rate in detail. Many of these are similar in nature to those responsible for impact on the Established call rate.

5.2.1 Lack of RF coverage

Concept/Reason for failure

AT in weak coverage area (such as AT receive power less than –105 dBm and best Pilot SNR < -10dB, mobile transmit power > 20 dBm), by definition, suffers from excess pathloss resulting in its inability to close one or both the links. Typically, the reverse link runs out of coverage first given that the 1xEV-DO forward link has roughly 10 dB advantage (Pilot channel transmitted at full power). Areas suffering from heavy shadowing such as in-building locations or areas blocked by buildings, trees, etc. can likely experience this type of failure.

A drop call due to insufficient RF coverage is generally preceded by high frame error activity on one or both the links, and increased mobile transmit power.

Failure Symptoms and Identification strategies

Cells suffering from lack of RF coverage tend to exhibit simultaneous degradation in several performance metrics. Besides Drop Call rate, these include Established Call rate, Page Failure rate, Reverse Frame Error rate, RLP re-transmission rates (ratio of RLP Octets Re-transmitted to RLP Octets Transmitted on the forward link; ratio of Missing RLP Octets Requested to RLP Octets Received on the reverse link, all these counts are pegged in the TP), and data throughput (discussed in section 5.3) performance. Hence, it requires correlating several performance indicators to identify the root cause.

Even when the aforementioned metrics show a correlated degradation, it may not be straightforward to establish lack of RF coverage as the root cause for dropped calls. The problem may be best detected and resolved via drive tests. The drive test area can be whittled down to the coverage of a few cells by identifying suspect cells via SM.

For one-to-one overlay on a 2G/3G1x network, comparison with the underlying network performance as well as review of the existing coverage information can help isolate the root cause quicker. For example, weak coverage areas often exhibit high average transmit power per user on the forward link. Of course, other possible factors discussed in the following sections (Pilot

LUCENT TECHNOLOGIES PROPRIETARY 51

Page 52: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

pollution, Neighbor list/search window issue, Island mode problem) will have to be eliminated first.

Suggested Mitigation Techniques

Having identified specific areas, drive test based RF optimization techniques can be employed as detailed in [8]. The process typically entails analyzing drive test data to arrive at a suitable conclusion, such as, increasing cell power or adjusting antenna azimuth, or in the worst case adding repeater13 (especially for in-building environments) or cell to fill in low signal strength areas.

Improving the coverage typically has a positive effect on the overall performance. This also follows from the claim above that lack of RF coverage impacts several other performance indicators. Improvement in coverage provides higher link margin to combat RF performance inhibitors such as shadowing, fading, etc.

5.2.2 Excessive pilots/Non-dominant pilots

Concept/Reason for failure

Presence of too many pilots in an area often impacts forward link performance due to interference it creates to the pilot(s) being actively tracked by the AT. Typically, there is no dominant pilot in such a problem area, that is, there are several pilots with comparable strengths, all weak, often accompanied by constant pilot thrashing (dominant sector changing quickly). Another scenario is where a distant sector overshoots to create/contribute to a localized non-dominant pilot region.

Calls are susceptible to getting dropped in multiple pilot regions since AT has no single strong pilot to tune to. Any dropped calls will primarily be due to inability to complete handoff sequences due to break in the signaling.

Failure Symptoms and Identification strategies

There are several ways to attain a pointer to the potential existence of this problem.

On a relative basis, cells exhibiting higher than normal rate of connection requests with 3 or more Pilots provide one such indication. There are also separate SM counts to flag “Connection Requests with 2 Pilots” and “Connection Requests with 3 Pilots” events. All these counts are reported in the AP for the sector that received the Connection Request. [Sum of AT Initiated Connection Requests and AN Initiated Connection Requests] minus [Sum of these multiple pilot counts] reflect the number of Connection Requests with single pilot. The following four metrics in SPAT3G can verify the existence of the pilot pollution problem:

• Connection Request with One Pilot (%) • Connection Request with Two Pilots (%)

13 Repeaters have not been applied to 1xEV-DO deployments so far and should be verified for their transmit signal to noise level integrity.

LUCENT TECHNOLOGIES PROPRIETARY 52

Page 53: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• Connection Request with Three Pilots (%) • Connection Request with Four or more Pilots (%)

Guidelines similar to the one used to detect pilot pollution in 2G/3G1x networks, based on SM counts for the number of Tadd pilots reported on the PSMM, can be used. Higher than normal rate of Soft/Softer HO fail due to maximum number of legs reached (ratio of SO_SOR_HOFF_FAIL_MAX_NO_LEGS_REACHED and SO_SOR_HOF_ATTMPT) when the Maximum Legs in Handoff configuration parameter is set to a value higher than 3 can also indicate pilot pollution problems.

Another pointer, though a weak one14, is excessive activity for Soft and/or Softer handoff attempts. It is likely that other KPIs such as Established call rate and Data throughput may also exhibit impairment in these areas.

For one-to-one overlay on a Lucent 2G or 3G1x network, additional data and tools from the underlying 2G/3G1x infrastructure may also be employed for identification of excess-pilot areas. Note, however, that this type of comparative analysis will be valid only if the antennas are shared across technologies, or in physical proximity with similar attributes (azimuth, downtilt, height, beam width).

• 2G/3G1x cells showing high secondary CE usage relative to Primary (or Total) CE usage, or Total Walsh Code relative to Primary Erlangs, are potential indicators to excess Pilot regions.

• Handoff Matrix and Undeclared Neighbor List analysis tools in SPAT3G can be used on the underlying 2G/3G1x network to flag cells overshooting from a distant region. Every effort must be made to contain their coverage/reduce their interference in order to not only benefit call set-up rate, but also to improve overall performance (drop call rate, throughput, etc).

Suggested Mitigation Techniques

Similar to the Lack of RF coverage reason, drive tests can be employed to analyze and resolve such problem areas. Best way to resolve the problem is to attend to the root cause – that is, to mitigate pilot pollution. Typical RF optimization techniques for this include adjusting cell site power and/or antenna downtilt of one or more neighboring cells to create a dominant pilot or fewer visible pilots. Antenna downtilt usually are more effective in controlling coverage of overshooting sectors. Refer to [8] for more details.

Many customers prefer to increase the “Maximum Legs in Handoff” to 6 to reduce dropped call rate in such problem areas. Larger Active set not only increases the likelihood of having a pilot at

14 Soft/softer handoff activity is more symbolic of mobility, but along with several other indicators tends to suggest a potential multi-pilot region.

LUCENT TECHNOLOGIES PROPRIETARY 53

Page 54: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

AT’s disposal when it momentarily becomes dominant15, it also tends to mitigate reverse link power. This, however, comes at the expense of increased resource utilization on either link. On the forward link, the cost is mostly limited to increase in MAC Index usage, on the reverse link the impact is in terms of increase in CE hardware and backhaul occupancy. Increasing the “Maximum Legs in Handoff” translation (Service Node/General section of EMS GUI) may also serve as a stopgap measure while more resource intensive changes such as antenna adjustments are being planned.

5.2.3 Poorly constructed neighbor lists

Concept/Reason for failure

Not having a fine tuned neighbor list is a common source of dropped call failure. A poor neighbor list may typically be missing a nearby pilot to which AT will otherwise handoff to, or may have integrity issues, such as, lack of reciprocity or suffer from PN ambiguity, discussed below. In essence, any of these neighbor list issues renders a nearby pilot a strong source of interference to the Active set pilots, causing a call to drop.

Failure Symptoms and Identification strategies

The first check for cells with high dropped call rate should be to review the neighbor list entries and ensure they follow basic rules, such as other sectors of the same cell as well inward facing sectors of the surrounding first tier cells are included as neighbors and the correct AP IP addresses are populated for neighbor sectors.

If the Soft Handoff Attempt to Assign rate metric is high, it typically indicates that some of the neighbor sectors have incorrect AP IP addresses listed in the NeighborSector form.

The Neighbor List Alert (NL Alert) feature in the SPAT3G tool can be employed effectively to fine tune neighbor lists. The integrity can be verified in terms of:

• Reciprocity - if cell X is on neighbor list of cell Y, then so should be Y on the neighbor list of X.

• No PN ambiguity - cell X should not have two cells with same PN on its neighbor list16, or if cell X and Y are in handoff, where X sponsors neighbor cell A and cell Y sponsors cell B, A and B should not be configured with the same PN17. Otherwise, it is possible that AT ends up establishing a traffic link on the weaker of the two cells with the same PN, thereby, impacting

15 In 1xEV-DO, since AT tunes to only one Pilot at a time on the forward link, there is no increase in combining diversity due to a larger Active set. However, a larger Active set may still be beneficial in some cases. For example, it is better to keep a “bouncing” Pilot in the Active set longer, potentially saving the call when it momentarily becomes strong rather than let it swap in and out of the Active set. The latter could result in a dropped call if there is break in handoff messaging due to marginal RF conditions.

16 Commonly known as one-way PN ambiguity.

17 Commonly known as two-way PN ambiguity.

LUCENT TECHNOLOGIES PROPRIETARY 54

Page 55: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

access performance due to noise component brought by the stronger non-traffic bearing PN at the AT demodulator.

• Cross-face omission – It is a good idea to include other sectors in the cell as high priority neighbors.

• AP IP alert – The IP addresses of the neighbor sector’s AP’s have to be defined correctly in order for handoffs to occur.

For a one to one overlay on a 2G/3G1x network, use of HO matrix and Undeclared Neighbor List analysis tools in SPAT3G can be used on the underlay network to detect missing neighbors. This can help improve neighbor lists and hence overall performance on both the networks.

Neighbor list issues often result in call “drags” before eventually causing call drops. Depending on the severity of the impact, this may manifest in terms of high RSSI levels on the approaching cell as the AT power ramps up in the absence of connection to this cell.

Suggested Mitigation Techniques

Fix any reciprocity issues and resolve PN ambiguity by changing PN offset of one of the two cells with common PN. Ensure that the neighbor sectors have the correct AP IP addresses populated.

If it is a recent deployment without a 2G/3G1x overlay, it is best to shape neighbor lists with drive tests. Neighbor list omissions can be detected for example, where a call drops and shows up on a strong Pilot that was not in the prior Active set or the Neighbor List message sent to the mobile. Opening the Remaining set can also help identify missing neighbors, if they are strong enough for the AT to detect and report them, either via Route Update or Searcher info message. Refer to [8] for more details on drive test based neighbor list refinement techniques. As traffic grows and more system level DO tools/SM capabilities are available (such as HO Matrix and Undeclared Neighbor List), further fine-tuning of the neighbor lists can be performed.

5.2.4 Improper PN planning

Concept/Reason for Failure

Improper PN planning could result in PN assignments such that AT cannot resolve signal from 2 different cells. There are two common scenarios in which this problem manifests.

• PN offsets are time-shifted versions of the same PN sequence. Each PN offset is 64 chips apart. Large propagation delays, as in the case of signal from a distant overshooting cell, can alter (shift) the identity of a PN at the AT, which in the worst case could make it indistinguishable from a nearby cell. When AT sends Route Update message, it will include this common PN it sees. The call will be setup only on the close-in cell, not on the far-away cell since it is actually transmitting on a different PN. As a result, only the close-in cell will key up traffic. When AT attempts to demodulate traffic channel, it will not only include valid traffic energy from the close-in cell, but also weigh in noise component from the distant cell. Consequently, the traffic signal-to-noise ratio will take a hit, impacting drop call performance.

LUCENT TECHNOLOGIES PROPRIETARY 55

Page 56: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• Another scenario is where two cells are assigned the same PN, but are not physically separated far enough such that they become indistinguishable at the AT. This produces the same performance-limiting phenomenon as described above.

In both the cases, the problem appears because there is not enough separation between the PN offset of the two cells. Prudent PN planning can help minimize the problem. Using adequately large Pilot_increment (available as Pilot PN Sequence Offset Index Increment in the Service Node translation form) helps mitigate the first scenario by allowing the AT to resolve signals from two distant cells. Special attention should be heeded for situations where signal is expected to propagate much farther than typical range, for example, due to low loss experienced over water bodies, signal from elevated cellsites, etc. The second scenario requires proper PN re-use margin.

Failure Symptoms and Identification strategies

The NL Alert feature in SPAT3G is designed to uncover PN ambiguity issues and can potentially help identify the problem with inadequate PN reuse margin. See the previous section for more details (5.2.3). Note, however, that this approach cannot detect all problem configurations. In such cases, drive tests are often effective, though not simplistic, in isolating potential problem(s).

A common symptom to look for is where FER on the forward link appears poor while the AT Receive power and Pilot Ec/Io remain at decent levels. This follows from the discussion earlier that traffic signal to noise ratio weakens since the AT ends up demodulating noise from the cell not in the Active set. Usually, this is a very telling indicator of two or more cells being received with same PN (either of the two problem scenarios) assuming there are no other issues (such as cell software/hardware defects).

To confirm the problem, the cell suspected to suffer from PN interference should be turned off to verify if the Ec/Io associated with this PN still remains strong. If it does, the next step will be to identify cells in the vicinity configured with same PN or explore the possibility of a distant overshooting cell. Latter may require meticulous investigation of terrain/morphology/cell elevation in an attempt to identify the potential for a benign propagation environment in the area.

Suggested Mitigation Techniques

Once the failure scenario is identified, it is relatively easy to resolve the problem. For the first scenario (overshooting cell), standard RF optimization techniques, such as antenna downtilt and/or azimuth adjustments can be attempted to contain its coverage. The other option is PN re-assignment, but care should be exercised to ensure it does not become a problem elsewhere. Using a large PN increment will also mitigate the problem, though will require quite a bit of PN planning re-work.

The second problem scenario can be addressed by maintaining sufficient number of cells between PN re-use. A judicious design at the outset will go a long way in minimizing the amount of effort spent fixing it later.

5.2.5 External interference

Concept/Reason for Failure

LUCENT TECHNOLOGIES PROPRIETARY 56

Page 57: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Presence of interference raises the noise floor, which raises the amount of power needed to overcome it, further elevating the noise levels. As the cell or the AT runs out of power to close the link, call reliability will suffer, dropped call being one of the common consequences. The level of impact, and hence, the ability to detect/resolve, may depend on the amount of interference. An intermittent/weak transmitter may often be very difficult to identify.

There are several flavors of External Interference that could impact performance of a network. It can exist on the forward or the reverse link.

• An unlicensed transmission within the CDMA band is the most common cause of interference. Usually spectral clearance tests are performed in a new deployment to identify and rectify such sources. If the tests were not executed, such as, when additional spectrum is allocated to an existing operator, continued operation of interference sources could cause significant performance issues as the commercial service is activated on the carrier.

• Often, interference is also caused by spill over of a legitimate transmission from the adjacent channel due to narrow spectral spacing, and/or, inadequate filter roll-offs implemented by the transmitter or the receiver.

• Another flavor of interference is where inter-modulation (IM) products generated at the receiver amplifier due to a strong signal on neighboring channel (although within the amplifier band) causes significant performance degradation.

Failure Symptoms and Identification strategies

Detecting presence of interference is not that straightforward and locating the exact source may be even more difficult. It often requires shutting down the desired transmission (that is, turn off service) to pinpoint the root cause. Thus, it is best to isolate external interference upfront before pressing system into commercial system.

The most telling indicator on the reverse link is much higher levels of RSSI rise, Eb/No setpoint and RFER, all happening as a result of inflated noise floor. There are SM counts available to inspect these per sector performance metrics – RSSI (Short and Long Term Average RSSI Rise pegged in the MCC), reverse Eb/No setpoint (Total Setpoint for Reverse Outer Loop Power Control – separate count for each reverse frame rate ranging from 0 to 153.6 kbps, pegged in the TP), and RFER (Reverse Link Frame Error and Total Reverse Link Frame counts, pegged in the TP). Corresponding performance metrics for these peg counts available in SPAT3G include: RSSI (One Second Average RSSI Rise, One Second Peak RSSI Rise, Ten Second Average RSSI Rise, and Ten Second Peak RSSI Rise), and RFER (Reverse Frame Error rate, Reverse Retransmission Failure rate and EVM DRC Erasure rate).

On the forward link, drive testing may help provide initial pointers to the presence of external interference. Basically, the interference will result in very high mobile receive power, but poor Ec/Io (Io goes up due to interference) and poor fwd FER. This, however, may also be caused due to a missing neighbor or an inadequate search window size; hence, these have to be eliminated first along with any potential hardware/software issues. Turning off some cells or service in the affected area may be required to allow for thorough spectral clearance tests.

On either link, one way to isolate a problem related to IM interference is by using an in-line attenuation to the mobile or the cell. If the measured signal power at either end (mobile receive

LUCENT TECHNOLOGIES PROPRIETARY 57

Page 58: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

power on the forward link, or cell RSSI on the reverse link which is supposedly much higher than the noise floor) increases at a much faster rate than the additional attenuation value, it suggests the existence of IM products. Basically, the IM harmonics created due to amplifier non-linearity de-emphasize at a larger rate than the additional attenuation value while the desired inband signal attenuates at the nominal rate; therefore, the composite signal still attenuates at a larger rate.

Suggested Mitigation Techniques

As above, identifying the precise location/source of the interference is the most difficult first step. Once the source is identified, the remedy is simply to turn it off.

5.2.6 Insufficient search window size

Concept/Reason for Failure

There are two main search windows of interest – active and neighbor sets. When searching the PN space to track/detect pilots in its various sets, the AT utilizes a finite window width to keep the search time small. The size is optimized to suit local terrain and cell topology in order to maximize the likelihood of AT finding a Pilot and its multipath components in the smallest possible time.

Different considerations go into determining Active and Neighbor set search window sizes. Accordingly, there are two failure modes.

• When the Active set search window size is not configured large enough to allow AT to receive/track a strong multipath component. Typically, the default search window size is good enough to cover most of the significant components. In some rare cases, especially in large delay spread environments, this problem may surface. AT cannot combine/demodulate the multipath since it falls outside the search window, rendering it as an interferer.

• When the AT fails to detect a neighbor due to an inadequate Neighbor search window size. More than the presence of multipath, it is often the relative path difference at the AT between the reference and the neighbor cells that drives this window width. Since PN offset transmitted by each cell is a time-shifted version of a single PN sequence, the difference in propagation delays to the AT requires the neighbor search window to be wide enough to span the largest delay differential. Otherwise, a particular neighbor may fall outside the search window, causing it to interfere with the Active set.

Not adding a strong Neighbor to the Active set may cause the call to drag, potentially increasing the reverse link interference to other users, and also causing dropped calls on the vicinity cells.

When changing Active set search window size, it is usually a good idea to make the one used at the cell site for reverse link demodulation, called Reverse Traffic Window Size, as well as for acquiring soft handoff leg, called Data Window size, comparable. Both parameters on the BTSs section of the EMS GUI.

LUCENT TECHNOLOGIES PROPRIETARY 58

Page 59: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Failure Symptoms and Identification strategies

SM counts do not provide a clear clue as to the existence of a search window issue.

For an overlay network, a basic sanity check is recommended to ensure Active and Neighbor search window sizes are comparable to the 2G/3G1x underlay.

Drive test is often the most productive method to identify search window issues. LDAT can be used to alert and recommend proper neighbor search window sizes. Based on the mobile logs, it employs the phase-offset information for various pilots reported on the Route Update Message for an active call to identify potential for larger neighbor search windows. However, it cannot identify situations where the existing neighbor search window is small enough to let AT even detect the neighbor pilot. Towards that end, another tool that could be of particular significance, especially in tough to resolve problem areas, is a Pilot scanner, which employs independent GPS time reference to provide an accurate view of multi-path components from each cell.

Suggested Mitigation Techniques

Once the need for opening the search window for an Active or Neighbor set is identified, it can be increased via translations. If a proper size is not known, it can be expanded in smallest possible increments until performance is improved and/or mobile logs confirm proper detection at the problem area.

5.2.7 Failures due to faulty hardware

Concept/Reason for failure

Presence of faulty or mis-configured hardware can influence the performance on the affected cell. Hardware failures can occur in various forms and intensity. For example, a CRC or EVM board entering into faulty operation registers sudden increase in dropped calls. At times, the affected hardware device may only show one errant behavior, such as, increase in drop calls, but other indicators such as access failure rate remain sane. A different flavor of a hardware problem could be a faulty RF antenna cable with high return loss likely resulting in smaller than desired cell footprint. In many cases, the hardware simply goes into a faulty state, the hardware itself is fine – software restore or stable clear rectifies the problem.

It is also possible to witness intermittent hardware outage instead of an outright malfunction, such as loss of T1/E1 during certain hours.

Often issues with the hardware on a cell prevent the AT from entering into soft handoff as it approaches this cell. This leads to the call being dragged until the AT can no longer communicate with the existing Active set cells, eventually registering a dropped call on one of the neighboring cells supporting the call. One particular case is where the timing unit in the cell drifts away abnormally due to a hardware defect. This puts the affected cell out of sync with the neighboring cells. As a result, soft handoffs are blocked in both directions (traffic entering or leaving the cell) since a neighboring PN takes a different identity due to time shift. This problem is commonly known as “island cell” phenomenon and impacts drop call performance more than access performance.

LUCENT TECHNOLOGIES PROPRIETARY 59

Page 60: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Failure Symptoms and Identification strategies

A first check should involve careful review of alarms generated and flagged by fault management module of the network. Certain issues associated with receive path (such as RSSI imbalance across main and diversity paths), malfunctioning hardware devices, backhaul outages, timing units, etc. can easily be identified through the alarms. It is also likely that hardware related failures will result in increased count for Connection Released – Other Reasons (pegged in the TP) and available as a metric in SPAT3G as Connection Release rate Other Reasons.

The ROP analysis report in SPAT3G provides information on these alarms at the AP, TP and BTS levels. A pareto analysis of call set-up failures by specific hardware type based on ROP data can offer a tell-tale sign of issues with a specific hardware/cell based on the over-abundant concentration of the failures. To make this type of analysis meaningful, however, care should be taken to normalize the failures with the amount of carried traffic.

It is easy to detect the “Island cell” problem via SM. Typically, only the dropped call rate shoots up while the access failures remain the same. Further, the soft handoff activity dwindles in both directions. Handoff activity can be inferred from SM count – Soft and Softer handoff attempts (pegged in the AP). Note, however, that softer handoffs (and soft handoffs within the same cell) may still complete successfully, since the sectors share the same timing unit. However, soft handoff with the neighboring cells will not be requested since cell will view phase reported on the Route Update Message potentially as a Remaining Set Pilot. Similarly, soft handoff activity on the neighboring cell will also reduce.

An “Island cell” problem can also be confirmed via drive tests by looking for strong unexpected pilot PNs near the problem cell as well as absence of strong pilot PNs associated with the cell. A Pilot scanner with its independent GPS source can facilitate this investigation.

Suggested Mitigation Techniques

Once the suspected malfunctioning hardware is identified, the first step is to attempt to rectify through remote software restore or download. If the performance continues to suffer, it may require cell visit to reseat the hardware and in some cases, to ensure tight cable connection. If all on-site diagnostics fail, the last step will be to replace the hardware.

5.2.8 Heavy reverse link traffic

Concepts/Reasons for failure

As the reverse link load increases, either due to higher number of active users or increased data upload activity, the overall interference floor seen by each cell increases. In order to overcome the raised interference floor, AT has to transmit at much higher power levels. An AT already at the cell edge can no longer close the reverse link as its signal arrives with a weak signal-to-noise ratio at the cell. This would sacrifice call reliability leading to call drops in some instances.

Failure Symptoms and Identification strategies

The indication of high reverse activity can be inferred by increase in several SM counts, relative to normal levels observed on other cells:

LUCENT TECHNOLOGIES PROPRIETARY 60

Page 61: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• High values for peg counts introduced in R23 to measure Reverse Link Overload Control including Short Term Average Fast Control Low Load count (SM_SHTM_AVGLLC_FASTCTRL), Short Term Average Fast Control Medium Load count (SM_SHTM_AVGMLC_FASTCTRL) and Short Term Average Fast Control High Load count (SM_SHTM_AVGHLC_FASTCTRL). These peg counts are available on a per sector basis.

• High Reverse Frame Error rate (ratio of Reverse Frame Error and Total Reverse Frame counts, pegged in the TP)

• High average setpoint for various rates (ratio of Total Setpoint for Reverse Outer Loop Control and Total Number of Frames for Reverse Outer Loop Control at each rate, pegged in the TP)

• Increase in Number of Connections Force Released and Abnormal Release rate, pegged in the MCC

• Higher values for One Second and Ten Second Average/Peak RSSI Rise, pegged in the MCC

• Large number of Total Connection Requests per hour, pegged in the AP

Some of these indicators may also be influenced by reasons other than heavy reverse traffic, such as:

• External interference resulting in elevated RSSI levels, high RFER and high average set points.

• A malfunctioning device in the RF receive path, such as UCR or low noise amplifier, causing consistent or intermittent RSSI spikes, thereby, reducing the coverage.

• A component breakdown in the RF receive path, or improper antenna cable connection, incapacitating one of the two diversity branches. This would tend to raise Eb/No requirement as well as cause more power control fluctuations. Not only will this heighten AT transmit power requirements, but also push the cell further into instability.

Hence, careful investigation of all possible causes may be required before a particular one can be confirmed. Though not always necessary, one distinction may be that there are large number of users consistently on the cell, as can be judged from Total Connection Requests per hour (sum of AT and AN Initiated Connection Requests). It is less likely that just a few users consistently load up the cell, usually a pointer, to external interference or faulty hardware.

Suggested Mitigation Techniques

It is always a sound practice to track/trend various performance indicators, especially, Connection Requests and RSSI rise to allow preemptive action.

Once the cell is identified as a heavy traffic-bearing cell impacting dropped call performance due to reverse link loading or overload, there are a few options available to offload the traffic:

LUCENT TECHNOLOGIES PROPRIETARY 61

Page 62: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• Reduce the overload control thresholds to lower the reverse transmission rates. With R23, the HDR Reverse Link Overload Control (HROC) tunable parameters have become translation parameters. The Lucent Reverse Overload Control mechanism utilizes these translatable thresholds to manage the integrity of reverse link performance. Refer to [5] for an overview of operation and associated translations. Lowering these thresholds will reduce the probability that AT will transmit at a higher rate, under load. Lower transmission rates require less AT transmit power on an average. In essence, reverse overload will kick-in at lighter loading levels thereby mitigating coverage reduction at the expense of throughput performance on the reverse link.

This is not a preferred solution in the long-term. It may, however, be acceptable as a stopgap measure instead of denying service to users until longer-term measures, stated next, are pursued. However, the customer must be made aware of the trade-offs before attempting any threshold adjustments.

• One solution is to increase pilot overlap such that other links get into the active set of the call, mitigating the traffic channel power requirement lowering the overall interference. From the perspective of mitigating AT transmit power requirement, higher handoff diversity should never be undesirable. However, careful consideration is warranted to characterize potential impact on the forward link performance as well as increase in cell resource utilization (e.g., channel element hardware).

• Add a 1xEV-DO carrier. As scoped earlier, this document focuses on single carrier 1xEV-DO deployments, so no further discussion is included here. It is mentioned here for future considerations.

• Add a cell to serve the heavy traffic area. If adding a carrier is not a possibility due to lack of spectrum, adding a cell (geographically separated from the high traffic cell(s)) is often an effective, although a long-lead, solution to improve coverage and hence, permit higher levels of offered traffic per unit area. Care must be taken to ensure proper RF optimization is performed to minimize excessive Pilot with cells in the vicinity – otherwise it may impact forward link performance.

5.2.9 Maximum number of legs reached

Concept/Reasons for failure

Handoff additions may only happen to the extent there is room in the AT’s Active set. The maximum Active set size is governed by the translation called “Maximum Legs in Handoff”. It is in the Service Node/General section of the EMS GUI. When the Active set gets full, the only way to add a pilot is to wait for one of the Active set members to drop18. To drop a pilot, its Ec/Io strength must stay below the “Pilot Detection Threshold” for at least “Drop Timer Value” seconds. These two translation parameters are in the EMS GUI section for Sectors.

18 Lucent 1xEV-DO system supports a mechanism to simultaneously add and drop Pilots. However, when the Active set is full and a candidate Pilot is available, this shuffle does not take place until the drop condition for one of the Active set Pilots has been met.

LUCENT TECHNOLOGIES PROPRIETARY 62

Page 63: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

One can see that in certain situations, waiting to drop off a weak pilot can delay addition of a more “worthwhile” pilot. The delay could result in a dropped call if the mobile has already dragged into the coverage of a candidate pilot before establishing a connection with it.

Such events are more likely to happen in pilot polluted environments. Usually, Lucent recommendation of setting the “Maximum Legs in Handoff” to 4 is adequate to minimize such dropped calls. However, it is conceivable that the problem may still happen in poorly optimized or tough to optimize RF environments.

Failure Symptoms and Identification strategies

It is easy to spot this phenomenon from SM. There is a count in the AP called Soft and Softer handoff attempts not processed – Maximum number of Legs Reached to peg it.

Suggested Mitigation Techniques

One method to mitigate the dropped calls occurring due to full active set is to allow more pilots in the Active set. That is, the “Maximum Legs in Handoff” translation can be increased to 6.

Note that there could be potential downsides to expanding the Active set (e.g., use up more MAC Indices on the forward link and increase hardware utilization on the reverse link). It may still be a good interim step, however, especially if the problem areas are localized and not severe in nature.

A better/longer-term solution would be to resort to traditional drive test based RF optimization techniques and attempt to improve dominant coverage in the region (see section 5.2.2). This will limit the likelihood of encountering a full Active set when a strong candidate pilot needs to be added. Another option after exhausting efforts to improve dominant coverage in the area is to experiment with higher values of “Pilot Detection Threshold” and “Pilot Drop Threshold”. These translation parameters are in the Sectors section of EMS GUI and the generally recommended values are –9 and –11 dB respectively. Higher values will tend to keep Active set smaller. In future, the availability of dynamic add/drop thresholds (similar to IS95B handoffs enhancements) will also help reduce Active set size.

5.2.10 No resources at the candidate leg

Concept/Reasons for failure

When the candidate leg to be added to the Active set has run out of resources, the handoff add attempt will not go through. As a result, the effective SNR on one or both the links will degrade due to growing pathloss as AT moves away from the existing Active set cells and approaches the cell it was denied handoff with.

The poor SNR levels due to call drag at some point will diminish the AT’s ability to decode the forward link. The mobile can also make reverse link power control errors and after a while unable to reach the cell as the transmit power reaches its ceiling. The high transmit power will also raise the interference levels to the nearby cells. All this could lead to increase in drop calls on the cells in the vicinity.

LUCENT TECHNOLOGIES PROPRIETARY 63

Page 64: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Failure Symptoms and Identification strategies

There is a separate SM count pegged at the AP level to facilitate investigation of handoff rejects due to resource shortages. It is called Soft and softer handoff failures – lack of resources in the candidate sector. Increase in this count on a particular sector will likely be accompanied by higher drop call rate on the neighboring sectors (prior to the drop, the call will be in active connection with these cells) as well as on this resource constrained sector (due to higher reverse link interference caused in the vicinity by the call drags).

With R24, a new set of target sector soft/softer handoff service measurements is available. In addition to the count listed in the previous paragraph, the Soft/Softer handoff failures due to no resources peg count, pegged at the target sector, provide a view of handoff blocking in the target sectors. Analyzing such counts for neighboring sectors, for example in a geographical map, provides a high level view of where handoffs are being blocked.

Any lack of resources will typically be due to cell running out of channel elements. Note that with one FLM/RLM (forward link/reverse link modem card), there are 93 CEs available on a 3-sector cell. In rare cases, where a sector takes on a large share of traffic, it could run out of MAC Indices (max 59 MAC Indices available per sector) before all CEs on the cell are used or the other limitation of 48 Max Number of Users19 per sector limit is reached.

Suggested Mitigation Techniques

One solution is to reduce the coverage of the cell in an attempt to offload traffic to the neighboring cells/sectors. This, however, assumes the neighboring cells have the room to handle more traffic, which may not be the case all the time.

An alternative is to add multiple carriers. This is a more graceful solution than adjusting coverage, which may not always yield desired traffic offloading. Release 25 will support multiple carriers for 1xEV-DO.

Where there are spectrum availability issues, adding a cell may be the last resort solution. This, of course, requires longer lead times in order to accommodate zoning, cell installation, RF calibration and optimization stages before turning the cell over to commercial service.

5.2.11 High Processor Occupancy rates

Concept/Reasons for failure

When equipment processor occupancy levels exceed certain thresholds, it is likely that new connections are blocked and existing connections are dropped. There are several components in the 1xEV-DO system for which occupancy levels can be monitored including the Application 19 The Maximum Number of Users constraint may block new call attempts only. Handoffs are not blocked. So, if a sector receives a lot of soft handoff traffic, the total number of users supported can exceed the limit, potentially bumping against the MAC Index limit.

LUCENT TECHNOLOGIES PROPRIETARY 64

Page 65: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Processor (AP), Line Interface Unit (LIU), Forward Link Modem (FLM) and Reverse Link Modem (RLM).

Failure Symptoms and Identification strategies

With R23, there are several service measurement counts that have been introduced to monitor the occupancy levels. High values for these counts over several days could potentially result in blocked and dropped calls. These counts include:

• AP Peak Processor Occupancy (AP_PEAK_PROC_OCCUPANCY)

• AP Average Processor Occupancy (AP_AVG_PROC_OCCUPANCY)

• LIU Peak Processor Occupancy (LIU_PEAK_PROC_OCCUPANCY)

• LIU Average Processor Occupancy (LIU_AVG_PROC_OCCUPANCY)

• FLM Peak Processor Occupancy (FLM_PEAK_PROC_OCCUPANCY)

• FLM Average Processor Occupancy (FLM_AVG_PROC_OCCUPANCY)

• RLM Peak Processor Occupancy (RLM_PEAK_PROC_OCCUPANCY)

• RLM Average Processor Occupancy (RLM_AVG_PROC_OCCUPANCY)

Suggested Mitigation Techniques

If the AP occupancy levels are high and with high traffic volume, consider re-homing some cell sites to another AP with lower occupancy levels. If LIU, FLM and/or RLM occupancy levels are high, consider off-loading traffic from the cell in question to surrounding cell sites.

5.2.12 Handoff failures

Concept/Reasons for failure

When the AN directs the AT to perform a handoff, it expects the AT to respond back indicating a successful handoff. If there is no response from the AT, the handoff is considered unsuccessful and the connection is dropped, resulting in increased drop call rate.

Failure Symptoms and Identification strategies

The Soft/Softer handoff failures due to no AT response peg count measures the number of failed soft/softer handoffs at the source sector that resulted in drop calls. With R24, there are new counts that measure the soft/softer handoff failure at the target sector. The Soft/softer handoff failure due to no AT response at target sector peg count can be used in conjunction with the

LUCENT TECHNOLOGIES PROPRIETARY 65

Page 66: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Soft/Softer handoff attempt at target sector to compute the handoff failure rate at the target sector that contributes to drop calls. These metrics are available as Soft/softer handoff failure rate and Soft/softer handoff failure rate at target sector in SPAT3G.

In addition, analysis of call processing failures and associated error codes provides additional details for the reason for handoff failures. SPAT3G call processing failure analysis report provides per sector level summary of all failures along with unit information and error codes. Refer to [20] for a look-up table that translates the error codes to detailed failure cause.

Neighbor list analysis often reveals PN ambiguities that could result in handoff failures. NL Alert analysis in SPAT3G provides a comprehensive report of such neighbor list inconsistencies. In addition, if there are too many strong pilots vying for handoffs, handing off to one target sector with interference from other strong neighbor pilots could also contribute to failed handoffs. Pilot pollution problems are dealt with in more detail in section 5.2.2.

Suggested Mitigation Techniques

If there are neighbor list ambiguities that are contributing to increased handoff failures, appropriate steps need to be taken to resolve such ambiguities (section 5.2.3 discusses this in detail). Typical RF optimization techniques to mitigate pilot pollution include adjusting cell site power and/or antenna down-tilt of one or more neighboring cells to create a dominant pilot or fewer visible pilots (section 5.2.2 discusses this in detail).

LUCENT TECHNOLOGIES PROPRIETARY 66

Page 67: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.3 Data Throughput

Data throughput is defined as the amount of data transferred per unit time. It can be measured at various layers. Unlike 3G1x, in 1xEV-DO systems, the peak forward and reverse data rates are different, with the peak forward data rate being 2.4Mbps and the peak reverse data rate being 153Kbps.

Consistent with the current Service Measurement capabilities, we define it at the RLP layer, as follows:

For R21 and earlier:

100036008

××

=dtransmitteoctetsRLPOriginalofNumberThroughputDataForward

framesverseofNumberframeperRateframesverseofNumberThroughputDataverse

ReReRe ×

=

For R22 and later:

( )( )001667.0*2160000*)8^10/(%

1000/1024SlotsBusy

ratesdifferentatPacketsofNumberThroughputDataForward ×=

×

=

02667.0*0

0

ReKbpsthangreaterrateofreceivedframesofNumber

frameperRateKbpsthangreaterrateofreceivedframesofNumber

ThroughputDataverse

The unit is in kilobits per sec (kbps). These are direct SM counts pegged in the TP on the DRC pointed sector. It is important to use only the original RLP data, not include the re-transmitted data, in order to obtain a more accurate view of the end-user experience. The constants “0.001667” and “0.02667” in the throughput equations represent the frame size in the forward and reverse link respectively. The forward data throughput metric is available as “Avg Fwd link MAC data rate” and the reverse data throughput metrics is available as “Average Rev link burst rate” in SPAT3G.

Also note that on the forward link, it is possible that some of the data never makes it to the end-user even after a retry; there is no way to obtain that view from SM – normally it represents a small percentage (of the order of 0.02%) and the above is a good approximation. Data throughput defined above can be obtained on a per sector basis.

On the forward link, the throughput is impacted either because the mobile is requesting lower DRC rates (poor RF conditions, interference, bad cell site hardware corrupting the transmit Signal to Noise Ratio, etc.), or the mobile is not getting assigned enough time slots as the scheduler is

LUCENT TECHNOLOGIES PROPRIETARY 67

Page 68: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

serving other user(s) on the cell. Note that there is no power control on the forward link. Cell always transmits at full power. It is up to the mobile to adjust the requested rates based on the short-term estimates of SNR (similar to Ec/Io) to meet 1% packet error rate. The mobiles measure SNR for small portions in each time slot (that is, when all sectors are transmitting Pilot channel). The received SNR is a function of RF conditions at the mobile (fading, coverage, interference from other Pilots) and the transmitted SNR (in case of cell site hardware failure). It is interesting to note that since 1xEV-DO has a time multiplexed forward link, the received SNR at the AT is not a function of other active users on the cell. This is in contrast to 2G/3G1x systems, where RF loading from other users impacts pilot Ec/Io at the mobile.

On the reverse link, the throughput is impacted when the reverse frame error rate is not being met (due to poor RF conditions or interference), or the mobile is transmitting at lower rates (either because it does not have adequate power to transmit at higher rates such as when the pathloss is high, or cell is instructing to do so as part of reverse overload control due to high traffic demand).

In subsequent sections, we discuss the constituent causes for low throughputs in more details. One area that is not treated in any of the discussions in the document is low throughput due to configuration problems in certain network elements (such as PDSN, router between PCF/PDSN, etc.). Such problems are likely to be systemic (a large number of calls in the system will be impacted) and should have been resolved either prior to commercial service operation or soon thereafter. The user is encouraged to refer to [12] and [13]. Reference document [12] has information on network configuration troubleshooting (such as using test call traces). Reference document [13] contains information on throughput expectations in a variety of environments.

The throughput is impacted negatively for MSM5500-based ATs due to the limitations in its mobile IP (MIP) implementation [13]. In MIP, the MSM5500-based AT experiences additional latency resulting in a maximum TCP throughput of 1.46Mbps. It has been observed that the throughput performance for MSM6500-based ATs is significantly better than the MSM5500-based ATs. ATs with MSM6500 chipsets are also expected to have a more efficient ramp-up algorithm following a tune-away, which improves reverse link throughput performance. In addition, these ATs also tend to complete a 1xEV-DO to 3G1X handoff faster than ATs with the older chipsets.

In hybrid mode, there will be a perceivable impact on throughput based on Slot Cycle Index settings [13]. The lower the slot cycle index setting the higher the impact on both the forward and reverse link throughput rates. It has been estimated that with a SCI setting of 0 and in SIP mode, there could be throughput degradation of up to 20% in the forward link and 30% in the reverse link compared to an EV-DO only mobile.

5.3.1 Lack of RF coverage

Concept/Reason for failure

AT in weak coverage area (such as AT receive power less than –105 dBm and best pilot SNR < -10dB, mobile transmit power > 20 dBm), by definition, suffers from excess pathloss affecting SNR at the receiver on either link.

LUCENT TECHNOLOGIES PROPRIETARY 68

Page 69: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Lower received SNR can not only reduce the transmitted data rates, but can also trigger re-transmission due to potential for high packet error rates. In either case, achieved throughput will suffer in the weak coverage area.

Typically, the reverse link runs out of coverage first given that the 1xEV-DO forward link has roughly 10 dB advantage (Pilot channel transmitted at full power). Areas suffering from heavy shadowing such as in-building locations or areas blocked by buildings, trees, etc can likely suffer from low throughputs on the either link.

Failure Symptoms and Identification strategies

Typically, cells suffering from low Data throughput due to ATs in weak coverage area will also tend to exhibit lower packet transmission rates. There are separate SM counts to measure this on either link. Additional SM derived metrics that should also tend to show correlated degradation include:

• Relatively larger number of DRC Rate Requests at lower forward link rates (counts pegged in the Modem)

• High RLP re-transmission rate (ratio of RLP Octets Re-transmitted to RLP Octets Transmitted on the forward link; ratio of Missing RLP Octets Requested to RLP Octets Received on the reverse link – all these counts pegged in the TP)

• High Physical layer NACK rate (ratio of Total Number of Physical Layer ACKs Received to Total Number of Physical Layer NACKs Received on the forward link only – both counts pegged in the modem)

• High Reverse Frame error rate (ratio of Reverse Link Frame Error Count to Total Reverse Link Frame Count – both counts pegged in the TP)

Ascertaining degradation using the above metrics does not lead to confirming weak coverage area as the root cause for low throughput. Such problems are best investigated further via drive tests. The drive test area can be narrowed to within the coverage of a few cells by identifying cells via SM, which tend to exhibit higher degradation in throughput compared to average.

Suggested Mitigation Techniques

Having identified select areas, drive test based RF optimization techniques can be employed as detailed in [8]. The process typically entails analyzing drive test data to arrive at a suitable conclusion, such as, increasing cell power or adjusting antenna azimuth, or in the worst case adding repeater (especially in in-building environments) or cell to fill in low signal strength areas.

Improving the coverage may also result in other benefits such as improved access and dropped call rate in the region.

5.3.2 Excessive pilots/Non-dominant pilots

Concept/Reason for failure

LUCENT TECHNOLOGIES PROPRIETARY 69

Page 70: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Presence of too many pilots in an area often impacts forward link performance due to interference it creates to the pilot(s) being actively tracked by the AT. Typically, there is no dominant pilot in such a problem area, that is, there are several pilots with weak comparable strengths, often accompanied by constant Pilot thrashing (dominant sector changing quickly). Another scenario is where a distant sector overshoots to create/contribute to a localized non-dominant pilot region.

Since AT decodes traffic on only one pilot, reduced SNR due to interference from other pilots results in lower DRC rate requests. As a result throughput will decrease on the forward link. Low throughput may also occur due to gaps in transmission during virtual soft handoffs, as the pilots thrash back and forth.

Failure Symptoms and Identification strategies

There are several ways to attain a pointer to the potential existence of this problem.

On a relative basis, cells exhibiting higher than normal rate of connection requests with 3 or more Pilots provide one such indication. There are also separate SM counts to flag “Connection Requests with 2 Pilots” and “Connection Requests with 3 Pilots” events. All these counts are reported in the AP for the sector that received the Connection Request. [Sum of AT Initiated Connection Requests and AN Initiated Connection Requests] minus [Sum of these multiple pilot counts] reflect the number of Connection Requests with single pilot. The following four metrics in SPAT3G can verify the existence of the pilot pollution problem:

• Connection Request with One Pilot (%) • Connection Request with Two Pilots (%) • Connection Request with Three Pilots (%) • Connection Request with Four or more Pilots (%)

Guidelines similar to the one used to detect pilot pollution in 2G/3G1x networks, based on SM counts for the number of Tadd pilots reported on the PSMM, can be used. Higher than normal rate of Soft/Softer HO fail due to maximum number of legs reached (ratio of SO_SOR_HOFF_FAIL_MAX_NO_LEGS_REACHED and SO_SOR_HOF_ATTMPT) when the Maximum Legs in Handoff configuration parameter is set to a value higher than 3 can also indicate pilot pollution problems.

Another pointer, though a weak one20, is excessive activity for Soft and/or Softer handoff attempts. It is likely that other KPIs such as Dropped call and established call rates may also exhibit impairment in these areas. Other derived SM metrics (re-transmission rate, NACK rate) discussed in the prior section may also show degradation on the forward link.

For one-to-one overlay over Lucent 2G or 3G1x network, additional data and tools from the 2G/3G1x infrastructure may also be employed for identification of excess Pilot areas.

• 2G/3G1x cells showing high secondary CE usage relative to Primary (or Total) CE usage, or Total Walsh Code relative to Primary Erlangs, are potential indicators of excess Pilot regions.

20 Soft/softer handoff activity is more symbolic of mobility, but along with several other indicators tends to suggest a potential multi-pilot region.

LUCENT TECHNOLOGIES PROPRIETARY 70

Page 71: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• Handoff Matrix and Undeclared Neighbor List analysis tools in SPAT3G can be used on the underlying 2G/3G1x network to flag cells overshooting from a distant region. Every effort must be made to contain their coverage/reduce their interference in order to not only benefit call set-up rate, but also to improve overall performance (drop call rate, throughput, etc).

Suggested Mitigation Techniques

Similar to the Lack of RF coverage reason, drive tests can be employed to analyze and resolve such problem areas. Typical RF optimization techniques include adjusting cell site power and/or antenna downtilt of one or more neighboring cells to create a dominant Pilot or fewer visible Pilots. Antenna downtilt usually are more effective in controlling coverage of overshooting sectors. Refer to [8] for more details.

When improving pilot dominance, care should be taken not to pull back coverage to the extent that the reverse link performance begins to suffer. Higher handoff diversity is usually beneficial to the reverse link as it mitigates AT transmit power. Reverse link performance should be verified under load to ensure the impact on mobile transmit power requirements is minimal.

5.3.3 Poorly constructed neighbor lists

Concept/Reason for failure

Not having a fine tuned neighbor list is a common source of throughput performance degradation. In essence, it prevents AT from pointing its DRC to the best Pilot. As it continues to tune to a Pilot with weaker SNR, it requests lower DRC rates, thereby, impacting forward link throughput. The reverse link throughput may also suffer due to “call drag” resulting in higher effective path loss. The “call drag” in general will cause higher interference levels in the area due to higher mobile transmit power, impacting reverse link throughput for other mobiles.

A poor neighbor list may typically be missing a nearby pilot to which AT will otherwise handoff to, or may have integrity issues, such as, lack of reciprocity or suffer from PN ambiguity, discussed below.

Rather than the outright neighbor list omissions whose impact is clearly evident in terms of dropped call and access failures, the marginal pilots require more due diligence to detect since the throughput impact will be a more “gradual or wider spread” phenomenon rather than a single event. Nevertheless, it is important to detect and add such pilots to improve the achieved throughputs, especially on the forward link. The reverse link improvement may be subtler, since it may accrue only as a side benefit in terms of reduced interference (AT transmit power is mitigated due to reduced call drags).

Failure Symptoms and Identification strategies

The first check for cells with low throughput should be to review the neighbor list entries and ensure they follow basic rules, such as other sectors of the same cell as well inward facing sectors of the surrounding first tier cells are included as neighbors and the correct AP IP addresses are populated for the neighbor sectors.

LUCENT TECHNOLOGIES PROPRIETARY 71

Page 72: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

If the Soft Handoff Attempt to Assign rate metric is high, it typically indicates that some of the neighbor sectors have incorrect AP IP addresses listed in the NeighborSector form.

The Neighbor List Alert (NL Alert) feature in the SPAT3G tool can be employed effectively to fine tune neighbor lists. The integrity can be verified in terms of:

• Reciprocity - if cell X is on neighbor list of cell Y, then so should be Y on the neighbor list of X.

• No PN ambiguity - cell X should not have two cells with the same PN on its neighbor list21, or if cell X and Y are in handoff, where X sponsors neighbor cell A and cell Y sponsors cell B, A and B should not be configured with the same PN22. Otherwise, it is possible that AT ends up establishing a traffic link on the weaker of the two cells with the same PN, thereby, impacting access performance due to noise component brought by the stronger non-traffic bearing PN at the AT demodulator. In some cases, AT may have trouble decoding control channel contents if

• Cross-face omission – It is a good idea to include other sectors in the cell as high priority neighbors.

• AP IP alert – The IP addresses of the neighbor sector’s AP’s have to be defined correctly in order for handoffs to occur.

For a one to one overlay on a 2G/3G1x network, use of HO matrix and Undeclared Neighbor List analysis tools in SPAT3G can be used on the underlay network to detect missing neighbors. This assumes similarity in footprint (power settings and antenna attributes). This can help improve neighbor lists and hence overall performance on both the networks.

Suggested Mitigation Techniques

Fix any reciprocity issues and resolve PN ambiguity by changing PN offset of one of the two cells with common PN. Ensure that the neighbor sectors have the correct AP IP addresses populated.

If it is a recent deployment without a 2G/3G1x overlay, it is best to shape neighbor lists with drive tests. Neighbor list omissions can be detected for example, where a call drops and shows up on a strong pilot that was not in the prior Active set or the Neighbor List message sent to the mobile. Opening the Remaining set can also help identify missing neighbors, if they are strong enough for the AT to detect and report them, either via Route Update or Searcher info message. Refer to [8] for more details on drive test based neighbor list refinement techniques. As traffic grows and more system level 1xEV-DO tools/SM capabilities are available (such as HO Matrix and Undeclared Neighbor List), further fine-tuning of the neighbor lists can be performed.

21 Commonly known as one-way PN ambiguity.

22 Commonly known as two-way PN ambiguity.

LUCENT TECHNOLOGIES PROPRIETARY 72

Page 73: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.3.4 Improper PN planning

Concept/Reason for Failure

Improper PN planning could result in PN assignments such that AT cannot resolve signal from 2 different cells. There are two common scenarios in which this problem manifests.

• PN offsets are time-shifted versions of the same PN sequence. Each PN offset is 64 chips apart. Large propagation delays, as in the case of signal from a distant overshooting cell, can alter the identity of a PN at the AT, which in the worst case could make it indistinguishable from a nearby cell. When AT sends Route Update message, it will include this common PN it sees. The Active set will include only the close-in cell, not on the far-away cell since it is actually transmitting on a different PN. As a result, only the close-in cell will key up traffic. When AT attempts to demodulate traffic channel, it will not only include valid traffic energy from the close-in cell, but also weigh in noise component from the distant cell. Consequently, the traffic signal-to-noise ratio will nose-dive, impacting data throughput performance.

• Another scenario is where two cells are assigned the same PN, but are not physically separated enough such that at the AT the two cells becomes indistinguishable. This triggers the same performance impacting phenomenon as described above in the case of two far-way cells with different PNs.

In both cases, the problem appears because there is not enough separation in the PN offset of the two cells. Depending on the severity of interference generated on the forward link, there could be more series consequences such as access and dropped call failures. If that is not the case, the small amount of interference can create pocket areas of low Data throughput on the forward link, potentially escaping visibility from the SM.

Prudent PN planning can help minimize the problem. As mentioned above, if the interference effect is not severe, the small but pervasive throughput degradation may evade detection via SM, or worse, become difficult to detect even via drive tests. This makes proper PN design even more critical.

The first scenario is mitigated by using adequately large Pilot_increment (available as Pilot PN Sequence Offset Index Increment in the Service Node translation form) so that AT can still resolve signals from cells received with large relative difference. Special attention should be heeded for situations where signal is expected to propagate much farther than typical range, for example, due to low loss experienced over water bodies, signal from elevated cell sites, etc. The second scenario requires proper PN re-use margin.

Failure Symptoms and Identification strategies

The NL Alert feature in SPAT3G is designed to uncover PN ambiguity issues and can potentially help identify the problem with inadequate PN reuse margin. See the previous section for more details. However, this approach cannot detect all problem configurations. In such cases, drive tests are often effective, though not simplistic, in isolating potential problem(s).

LUCENT TECHNOLOGIES PROPRIETARY 73

Page 74: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

A common symptom to look for is where FER on the forward link appears poor while the mobile receive power and Pilot Ec/Io remain at decent levels. This follows from the discussion earlier that traffic signal to noise ratio weakens since the AT ends up demodulating noise from the cell not in the Active set. Usually, this is a very telling indicator of two or more cells being received with same PN (either of the two problem scenarios) assuming there are no other issues (such as cell software/hardware defects). To confirm the problem, the cell suspected to be facing PN interference should be turned off to verify if the Ec/Io associated with this PN still remains strong. If it does, the next step will be to identify cells in the vicinity configured with same PN or explore the possibility of a distant overshooting cell. Latter may require meticulous investigation of terrain/morphology/cell elevation to identify the potential for a benign propagation environment in the area.

Suggested Mitigation Techniques

Once the failure scenario is identified, it is relatively easy to resolve the problem. For the first scenario (overshooting cell), standard RF optimization techniques, such as antenna downtilt and/or azimuth adjustments can be attempted to contain its coverage. The other option is PN re-assignment, but care should be exercised to ensure it does not become a problem elsewhere. Using a large PN increment will also mitigate the problem, though will require quite a bit of PN planning re-work.

The second problem scenario can be resolved by maintaining sufficient number of cells between PN re-use. A judicious design at the outset will go a long way in minimizing a potentially large amount of effort spent resolving it later.

5.3.5 External interference

Concept/Reason for Failure

Presence of interference raises the noise floor, which raises the amount of power needed to overcome it, further elevating the noise levels. As the cell or the AT runs out of power needed to close the link, DRC rate request on the forward link and transmitted rate (on either link) decreases impacting throughput performance. The level of impact, and hence, the ability to detect/resolve, may depend on the amount of interference. An intermittent/weak transmitter may often be very difficult to identify.

There are several flavors of External Interference that could impact performance of a network. It can exist on the forward or the reverse link.

• An unlicensed transmission within the CDMA band is the most common cause of interference. Usually spectral clearance tests are performed in a new deployment to identify and rectify such sources. If the tests were not executed, such as, when additional spectrum is allocated to an existing operator, continued operation of interference sources could cause significant performance issues as the commercial service is activated on the carrier.

• Interference is also caused by spill over of a legitimate transmission from the adjacent channel due to narrow spectral spacing, and/or, inadequate filter roll-offs implemented by the transmitter or the receiver.

LUCENT TECHNOLOGIES PROPRIETARY 74

Page 75: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• Another flavor of interference is where inter-modulation (IM) products generated at the receiver amplifier due to a strong signal on neighboring channel (although within the amplifier band) causes significant performance degradation.

Failure Symptoms and Identification strategies

Detecting presence of interference is not straightforward and locating the exact source may be even more difficult. It often requires shutting down the desired transmission (that is, turn off service) to pinpoint the root cause. Thus, it is best to isolate external interference upfront before pressing system into commercial system.

The most telling indicator on the reverse link is much higher levels of RSSI rise, Eb/No setpoint and RFER, all happening as a result of inflated noise floor. There are SM counts available to inspect these per sector performance metrics – RSSI (Short and Long Term Average RSSI Rise pegged in the MCC), reverse Eb/No setpoint (Total Setpoint for Reverse Outer Loop Power Control – separate count for each reverse frame rate ranging from 0 to 153.6kbps, pegged in the TP), and RFER (Reverse Link Frame Error and Total Reverse Link Frame counts, pegged in the TP). Corresponding performance metrics for these peg counts available in SPAT3G include: RSSI (One Second Average RSSI Rise, One Second Peak RSSI Rise, Ten Second Average RSSI Rise, and Ten Second Peak RSSI Rise), and RFER (Reverse Frame Error rate, Reverse Retransmission Failure rate and EVM DRC Erasure rate).

From prior field experience with such type of interference in 2G/3G1x networks, several sectors facing the interference source will exhibit a simultaneous increase in the RSSI, Eb/No setpoint and RFER. This is one of the surest indicators of external interference. To pinpoint the problem, more expensive/elaborate means such as use of spectrum analyzer at/around the cell site may be called for.

On the forward link, drive testing may help provide initial pointers to the presence of external interference. Basically, the higher interference will result in very high mobile receive power, but poor Ec/Io (Io goes up due to interference) and poor fwd FER. This, however, may also be caused to due to missing a neighbor or an inadequate search window size; hence, these have to be eliminated first along with any potential hardware/software issues. Turning off some cells or service in the affected area may be required to allow for thorough spectral clearance tests.

On either link, one way to isolate problem related to IM interference is by using an in-line attenuation to the mobile or the cell. If the measured signal power at either end (mobile receive power on the forward link, or cell RSSI on the reverse link which is supposedly much higher than the noise floor) increases at a much faster rate than the additional attenuation value, it suggests the existence of IM products. Basically, the IM harmonics created due to amplifier non-linearity de-emphasize at a larger rate than the additional attenuation value while the desired inband signal attenuates at the nominal rate; therefore, the composite signal still attenuates at a larger rate.

Suggested Mitigation Techniques

As above, identifying the precise location/source of the interference is the most difficult first step. Once the source is identified, the remedy is simply to turn it off.

LUCENT TECHNOLOGIES PROPRIETARY 75

Page 76: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.3.6 Insufficient search window size

Concept/Reason for Failure

There are two main search windows whose sizes are of interest – Active and Neighbor sets. When searching the PN space to track/detect Pilots in its various sets, the AT utilizes a finite window width in order to keep the search time small. The size is optimized to suit local terrain and cell topology in order to maximize the likelihood of AT finding a pilot and its multipath components in the smallest possible time.

Different considerations go into determining Active and Neighbor set search window size. Accordingly, there are two failure modes.

• When the Active set search window size is not configured large enough to allow AT to receive/track a strong multipath component. Typically, the default search window size is good enough to cover most of the significant components. In some rare cases, especially in large delay spread environments, this problem may surface. AT cannot combine/demodulate the multipath since it falls outside the search window, rendering it as an interferer.

• When the AT fails to detect a neighbor due to an inadequate Neighbor search window size. More than the presence of multipath, it is often the relative path difference at the AT between the reference and the neighbor cells that governs this window width. Since PN offset transmitted by each cell is a time-shifted version of a single PN sequence, the difference in propagation delays to the AT requires the Neighbor search window to be wide enough to span the largest path differential. Otherwise, a particular neighbor may fall outside the search window, causing it to interfere with the Active set.

Both cases of interference impact Data performance on the forward link as the AT requests lower DRC rates amidst reduced SNR.

On the reverse link, not adding a strong Neighbor to the Active set may cause the call to drag, potentially lowering the achieved throughput as well as increasing interference to other users.

When changing Active set search window size, it is usually a good idea to make the one used at the cell site for reverse link demodulation, called Reverse Traffic Window Size, as well as for acquiring soft handoff leg, called Data Window size, comparable. Both translation parameters on BTSs section of EMS GUI.

Failure Symptoms and Identification strategies

SM counts do not provide a clear clue as to the existence of a search window problem.

For an overlay network, a basic sanity check is recommended to ensure Active and Neighbor search window sizes are comparable to the 2G/3G1x underlay.

Drive test is often the most productive method to identify search window issues. LDAT can be used to alert and recommend proper neighbor search window sizes. Based on the mobile logs, it

LUCENT TECHNOLOGIES PROPRIETARY 76

Page 77: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

employs the phase-offset information for various Pilots reported on the Route Update Message for an active call to identify potential for larger neighbor search windows. However, it cannot identify situations where the existing neighbor search window is small enough to let AT even detect the neighbor Pilot. Towards that end, another tool that could be of particular significance, especially in tough to resolve problem areas, is a Pilot scanner, which employs independent GPS time reference to provide an accurate view of multipath components from each cell.

Suggested Mitigation Techniques

Once the need for opening the search window for an Active or Neighbor set is identified, it can be increased via translations. If a proper size is not known, it can be expanded in smallest possible increments until performance is improved and/or mobile logs confirm proper detection at the problem area.

5.3.7 Failures due to faulty hardware

Concept/Reason for failure

Presence of faulty or mis-configured hardware can influence the performance on the affected cell. Hardware failures can occur in various forms and intensity. For example, a CRC or a FLM/RLM or a CBR board entering into faulty operation registers sudden degradation in throughput while other indicators such as access and dropped calls may remain sane. In some cases, the affected board impacts the transmit Signal to Noise Ratio to an extent that the mobile measuring decent absolute receive power is not able to demodulate Pilot, thereby, requesting low forward link rates. A different flavor of a hardware problem could be a faulty RF antenna cable with high return loss likely resulting in smaller than desired cell footprint. In many cases, the hardware simply goes into a faulty state, the hardware itself is fine – software restore or stable clear fixes the problem.

It is also possible to witness intermittent hardware outage instead of an outright malfunction, such as loss of T1/E1 during certain hours.

Failure Symptoms and Identification strategies

The first check should involve careful review of alarms generated and flagged by fault management module of the network. Certain issues associated with receive path (such as RSSI imbalance across main and diversity paths), malfunctioning hardware devices, backhaul outages, etc. can easily be identified through the alarms.

The ROP analysis report in SPAT3G provides information on these alarms at the AP, TP and BTS levels. A pareto analysis of call set-up failures by specific hardware type based on ROP data can offer a tell-tale sign of issue with a specific hardware/cell based on the over-abundant concentration of the degraded throughput activity.

Suggested Mitigation Techniques

Once the suspected malfunctioning hardware is identified, first step is to attempt to rectify through software restore or download/restore remotely. If the performance continues to suffer, it may require cell visit to reseat the hardware and in some cases, to ensure tight cable connection. If all on-site diagnostics fail, last step will be to replace the hardware.

LUCENT TECHNOLOGIES PROPRIETARY 77

Page 78: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.3.8 Heavy reverse link traffic

Concepts/Reasons for failure

As the reverse link load increases, either due to higher number of active users or increased data upload activity, it elevates the overall interference floor seen by each cell. In order to overcome the raised floor, AT has to transmit at much higher power levels. An AT already at the edge can no longer close the reverse link as its signal arrives with a weak signal-to-noise ratio at the cell. This would sacrifice data throughput achieved on the reverse link, especially for the users at the edge of the coverage.

A closely associated, but an extreme form of revere link loading is when it pushes the cell into reverse overload. [5] provides an overview of the Reverse Overload Control algorithm. Some of the highlights are as follows.

Reverse overload control

Lucent 1xEV-DO network employs a proprietary algorithm to manage integrity of the reverse link. Key objectives are:

• To protect coverage and end-user Data performance on the reverse link

• To manage reverse link loading within tolerable limits to avoid instability

• To preserve integrity of DRC and ACK channels critical for forward link Data performance

Current implementation offers a choice between two control mechanisms:

• Rate limit message - reduces the reverse Data transmission rate used by all users in a sector based purely on the number of active users.

• Reverse Activity Bit (or RAB) - controls rate of all users based on RAB bit coupled with a standards-defined probability decision matrix for rate transition. This is a dynamic mechanism based on estimated reverse link loading and RSSI.

Either option constrains the transmitted rate to protect the reverse link, reducing data throughput when overload conditions are met.

Failure Symptoms and Identification strategies

The indication of high reverse loading condition can be inferred by increase in several SM counts, relative to normal levels observed on other cells:

• High values for peg counts introduced in R23 to measure Reverse Link Overload Control including Short Term Average Fast Control Low Load count (SM_SHTM_AVGLLC_FASTCTRL), Short Term Average Fast Control Medium

LUCENT TECHNOLOGIES PROPRIETARY 78

Page 79: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Load count (SM_SHTM_AVGMLC_FASTCTRL) and Short Term Average Fast Control High Load count (SM_SHTM_AVGHLC_FASTCTRL). These peg counts are available on a per sector basis.

• High Reverse Frame Error rate (ratio of Reverse Frame Error and Total Reverse Frame counts, pegged in the TP)

• High average setpoint for various rates (ratio of Total Setpoint for Reverse Outer Loop Control and Total Number of Frames for Reverse Outer Loop Control at each rate, pegged in the TP)

• Increase in Number of Connections Force Released and Abnormal Release rate, pegged in the MCC

• Higher values for One Second and Ten Second Average/Peak RSSI Rise, pegged in the MCC

• Large number of AT/AN Initiated Connection Requests per hour, pegged in the AP

Regardless of whether throughput is degraded by heavy reverse activity or overload, some of these indicators may also be influenced by reasons other than 1xEV-DO traffic, such as:

• External interference resulting in elevated RSSI levels, high RFER and high average set points.

• A malfunctioning device in the RF receive path, such as CBR/UCR or low noise amplifier, causing consistent or intermittent RSSI spikes, thereby, reducing the coverage.

• A component breakdown in the RF receive path, or improper antenna cable connection, incapacitating one of the two diversity branches. This would tend to increase the Eb/No requirement as well as the power control variation. Not only will this require increase in power transmitted by the mobile, but the variation will also push the cell further into instability.

Hence, careful investigation of all possible causes may be required before a particular one can be confirmed. Though not always necessary, one distinction may be that there are large number of users consistently on the cell, as can be judged from AT/AN Initiated Connection Requests. It is less likely that just a few users consistently load up the cell, usually a pointer, to external interference or faulty hardware.

Suggested Mitigation Techniques

It is always a sound practice to track/trend various performance indicators, especially, Connection Requests and RSSI rise to allow timely corrective action instead of waiting till performance becomes unacceptable.

Once the cell is identified as a heavy traffic-bearing cell impacting throughput performance due to reverse link loading or overload, there are a few options available to offload the traffic:

LUCENT TECHNOLOGIES PROPRIETARY 79

Page 80: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

• Increase the overload control thresholds to allow higher reverse transmission rates. With R23, the HDR Reverse Link Overload Control (HROC) tunable parameters have become translation parameters. The Lucent Reverse Overload Control mechanism utilizes these tunable thresholds to manage the integrity of reverse link performance. Refer to [5] for an overview of operation and associated translations. The ones of interest are “Slow and Fast Thresholds - 1, 2 and 3”. Raising these thresholds will increase the probability that AT will transmit at a higher rate, under load. However, this may come at the expense of elevated reverse interference levels, potentially impacting access, drop call and reverse FER performance. Improvement in throughput due to higher transmission rates will to some extent be capped by higher reverse FER.

This is not a preferred solution in the long-term. It may, however, be acceptable as a stopgap measure until longer-term measures, stated next, are pursued. However, customer must be made aware of the trade-offs before attempting any threshold adjustments.

• One solution is to increase pilot overlap such that other links get into the Active set of the call. This will tend to mitigate the traffic channel power requirement and hence, the overall interference. This would enable ATs to sustain higher transmission rates at acceptable frame rate, improving the reverse link throughput. However, careful consideration is warranted to characterize potential impact on the forward link performance (more pilot overlap could hurt pilot dominance) as well as increase in cell resource utilization (e.g., channel element hardware).

• Add a 1xEv-DO carrier. As scoped earlier, this document focuses on single carrier 1xEV-DO deployments, so no further discussion is included here. It is mentioned here for future considerations.

• Add a cell to serve the heavy traffic area. If adding a carrier is not a possibility due to lack of spectrum, adding a cell (geographically separated from the high traffic cell(s)) is often an effective, although a long-lead, solution to improve coverage and hence, permit higher levels of offered traffic per unit area. Care must be taken to ensure proper RF optimization is performed to minimize excessive Pilot with cells in the vicinity – otherwise it may impact forward link performance.

5.3.9 High forward link traffic

Concept/Reasons for failure

There is an inherent scheduler gain on the 1xEV-DO forward link that tends to increase overall (or aggregate) throughput supported by a sector as the number of active users increases, to a certain point. The “proportionally fair” scheduler rides along the SNR peaks experienced by individual users at different times. In essence, each user is served when it experiences decent SNR (thereby, requesting high DRC rate), and will be allowed to starve when other users are experiencing much better SNR levels.

However, as with any wireless system, the shared air-interface bandwidth is limited; as a result, the individual throughput will begin to suffer eventually in a 1xEV-DO system as the usage increases. It can happen with fewer users if their applications are data intensive (such as, file

LUCENT TECHNOLOGIES PROPRIETARY 80

Page 81: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

transfer, email attachments, video streaming, etc). On the other hand, a relatively larger number of users can be supported per 1xEV-DO cell if the usage is primarily based on low demand data applications (such as web browsing). In any event, action is warranted to offload traffic and improve individual user throughput beyond certain traffic levels.

Failure Symptoms and Identification strategies

Growing disparity between the requested and allocated rates is a good measure of data starvation at the users. There are two SM counts whose comparison may offer an approximation for this metric - DRC Rate Requests and Total Packets Transmitted on the Forward link – each provided for various rates and pegged in the Modem. However, there are couple issues with this type of analysis, which will require experimenting with commercial data in order to confirm validity and calibrate expectations. One issue is that the DRC Rate Request does not reflect the amount of the data backlog at the AN, necessary to obtain truer view of the starvation. Second, each packet may span over several DRC requests, so there may not be one-to-one correspondence between the two quantities.

Another metric to trend is the total data transmitted (RLP Octets Transmitted on the Forward link, pegged in the TP) per established call (numerator of the metric for Established Call Rate in section 5.1). Reduction in the metric will in general point to increased end-user starvation.

Suggested Mitigation Techniques

As with combating any traffic related bottlenecks, it is always a good practice to track/trend various performance indicators, such as ones mentioned above, to be able to anticipate and take a timely corrective action.

Once the cell is identified as one suffering from degraded throughput performance due to user contention on the forward link, there are a few options available to offload the traffic:

• Investigate potential for RF optimization to fine-tune performance. It will be a good first step to audit the cell for any issues related to coverage, pilot dominance, neighbor list omissions, search windows size, external interference, etc. There may not be any systemic or blatant issues at this stage of commercial operation, but the presence or development of several marginal issues can come together to degrade performance over time. For example, recently added cells in the area not well optimized, could cause overshoot/non-dominance in the coverage of the cell under investigation.

• Add a 1xEv-DO carrier. As scoped earlier, this document focuses on single carrier 1xEV-DO deployments, so no further discussion is included here. It is mentioned here for future considerations.

• Add a cell to serve the heavy traffic area. If adding a carrier is not a possibility due to lack of spectrum, adding a cell (geographically separated from the high traffic cell(s)) is often an effective, although a long-lead, solution to improve coverage and hence, permit higher levels of offered traffic per unit area. Care must be taken to ensure proper RF optimization is performed to minimize excessive Pilot with cells in the vicinity – otherwise it may impact forward link performance.

LUCENT TECHNOLOGIES PROPRIETARY 81

Page 82: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.3.10 Hybrid mode

Concept/Reasons for failure

Hybrid mode operation affords AT the flexibility to receive voice calls on a 2G/3G1x system while on a 1xEV-DO Data session. The downside is that the AT has to periodically tune to 2G/3G1x system to look for pages. The time spent away from the 1xEV-DO system will interrupt data transfer, lowering data throughput. How often it searches the underlying system (that is, stays out of 1xEV-DO system) depends on the Slot Cycle Index (SCI)23 setting. Larger values mean mobile can stay longer on the 1xEV-DO system and hence, smaller impact to throughput. It has been estimated that with a SCI setting of 0 and in SIP mode, there could be throughput degradation of up to 20% in the forward link and 30% in the reverse link compared to an EV-DO only mobile.

It is possible that the AT makes conservative rate requests (transmit on the reverse link) immediately upon return from the underlying system. This will have further impact on throughput compared to non-hybrid mode operation. This type of logic will likely be AT implementation specific.

Failure Symptoms and Identification strategies

There is no SM count to reflect hybrid mode operation. Check the AT configuration to see if this operation is allowed. Work with the service provider to obtain information on throughput performance of various terminal types in hybrid mode. The goal will be to verify that a particular AT model/make performs within the average range of other well-characterized terminals. This step will be important during initial 1xEV-DO deployments where behavior of different vendor terminals in hybrid mode is not well understood.

It would also be useful to analyze the percentage of AT models/makes that use the older MSM5500 chipset versus the ones that use the newer MSM6500 chipsets. ATs with MSM6500 chipsets are expected to have a more efficient ramp-up algorithm following a tune-away, which improves reverse link throughput performance. In addition, these ATs also tend to complete a 1xEV-DO to 3G1X handoff faster than ATs with the older chipsets.

Suggested Mitigation Techniques

As mentioned earlier, if hybrid mode is enabled, work with the service provider to identify any issues with hybrid mode performance of most prevalent terminals in the market. Another thing to discuss with the customer is raising SCI. The current recommendation is to set the SCI value as 2 for 1xEV-DO PCMCIA card based ATs. As mentioned earlier, this will allow AT to tune away from the 1xEV-DO carrier less frequently causing fewer gaps in data transfer. It must be noted that the SCI can also be programmed at the mobile. The mobile will use the smaller of the SCI value sent by the system and its programmed value. For example, if desired the service providers have the option of setting the system SCI value to 2, program EV-DO hybrid mode AT to 2 and program 3G1X mobiles to 1. 23 The effective SCI for the AT is the smaller of the one programmed in the AT and the maximum SCI, a cell site parameter.

LUCENT TECHNOLOGIES PROPRIETARY 82

Page 83: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

5.3.11 Backhaul restrictions

Concept/Reasons for failure

To realize full throughput capabilities of the 1xEV-DO air-interface, it is important to minimize system bottlenecks elsewhere. A system can only deliver as good a Data experience as its weakest link can offer. One such link, which requires important considerations, is the T1/E1 backhaul.

Lucent 1xEV-DO system utilizes “unchannelized” T1/E1s for backhaul transport between the cell and the FMS. (Note: Lucent 2G/3G1x infrastructure, in contrast, uses packet pipes, which can be viewed as logical slices of T1/E1. Each packet pipe consists of multiple DS0s).

To mitigate T1/E1 related throughput bottlenecks, Lucent currently recommends a minimum of 2 T1/E1s per 3-sector single-carrier cell24. This is based on certain assumptions from Systems Engineering on RLP layer throughput to be supported per sector as well as T1/E1 capacity. [14] shows a range of supported throughputs based on the number of T1/E1s. One such scenario is as follows.

Assuming uniform traffic across sectors and the usable T1/E1 capacity of ~1100 and ~1400 kbps, 2 T1s/E1s per cell will be adequate to support 680 kbps/760 kbps RLP layer throughput per sector. This is roughly 80%/92% of the 850kbps throughput that a typical 1xEV-DO sector may need to offer when serving all dual-antenna ATs, with minimal backhaul bottlenecks. In reality, sector loading is not uniform and hence, sectors can individually deliver peak throughput of 850kbps using 2 T1/E1s per cell.

In some situations, service providers may not be able to provision backhaul per Lucent recommended guidelines due to cost issues or during the interim period as system is evolved over time. One scenario likely to be encountered especially in some of the initial 1xEV-DO deployments is use of 1 T1/E1 on each link. This will limit aggregate sector throughputs achieved on the cell on the forward link.

Failure Symptoms and Identification strategies

Evaluate the T1/E1 service measurement counts – DL_UPLINK_AVG_THRUPUT, DL_UPLINK_PEAK_THRUPUT, DL_DOWNLINK_AVG_THRUPUT and DL_DOWNLINK_PEAK_THRUPUT

Anytime maximum achieved Data throughput on a sector is suspected to be much lower than the average seen on a cell with adequately provisioned backhaul, it is a good idea to check number of T1/E1s configured on the cell. If they are not configured per Lucent recommendations, the service provider should be informed.

24 Two T1s are necessary to achieve a decent throughput performance on the forward link. Only one is needed on the reverse link. The difference stems from the asymmetry in maximum air-interface transmit rates as defined in IS856. However, since T1s are bi-directional, Lucent recommends minimum of two.

LUCENT TECHNOLOGIES PROPRIETARY 83

Page 84: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

If the provisioning is adequate, look for issues related to T1/E1 outage, or defective T1/E1 terminating hardware at the cell. Hardware outage/defects are more likely to cause intermittent or sudden impact as opposed to under-provisioning, which will result in consistently reduced levels of throughput since the rollout of 1xEV-DO carrier.

Suggested Mitigation Techniques

After confirming that the degraded throughput performance is on account of under-provisioned T1/E1s, alert it to the operator and suggest appropriate backhaul adds.

5.3.12 Handoff rejects

Concept/Reasons for failure

This category of handoff failures is defined as those that do not result in Traffic Channel Assignment message sent to include a candidate Pilot.

Handoff attempts may be denied if a candidate cell is unable to assign resource due to shortage, or if the current Active set size is already at the maximum allowed translation value. In some situations, errors such as traffic channel activation failure on the candidate cell can also block handoff requests. Another reason would be the candidate cell losing GPS synchronization, also known as “Island cell” problem.

In any case, the SNR on one or both the links will degrade due to growing pathloss as the AT approaches the cell it was denied connection with.

The poor SNR levels due to call drag will lower the transmitted rates, thereby, impacting Data throughput. On the reverse link, higher AT transmit power levels will also create more interference, further degrading Data throughput performance for all users.

In many cases, the failed handoffs will also result in dropped calls.

Failure Symptoms and Identification strategies

There are separate SM counts pegged in the AP to facilitate investigation of handoff rejects due to resource shortages. These are:

• Soft and softer handoff failures – lack of resources in the candidate sector

• Soft and softer handoff failures – max number of legs reached

Miscellaneous handoff rejects (such as, due to traffic channel activation failure) will be pegged under Soft and softer handoff failures – Other Reasons in the AP.

As the cell begins to hit any of the above limits, other performance indicators, mainly dropped call rate, will also suffer.

Any lack of resources will typically be due to the cell running out of channel elements. Note that with one FLM/RLM, there are 93 CEs available on a 3-sector cell. In rare cases, where a sector

LUCENT TECHNOLOGIES PROPRIETARY 84

Page 85: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

takes on a large share of traffic, it could run out of MAC Indices (max 59 MAC Indices available per sector) before all CEs on the cell are used or the other limitation of 48 Max Number of Users25 per sector limit is reached.

The handoff rejects due to Max Number of Legs reached is a straightforward one to identify directly based on the SM count mentioned above.

Miscellaneous handoff reject causes may be more difficult to identify. If they occur at an unacceptable rate, development support may be needed for more detailed tracing in order to get to the root cause.

Suggested Mitigation Techniques

The workarounds depend on the root cause.

• Lack Of Resources In The Candidate Sector:

One solution is to reduce the coverage of the cell in an attempt to offload traffic to the neighboring cells/sectors. This, however, assumes the neighboring cells have the room to handle more traffic, which may not be the case all the time.

An alternative is to add multiple carriers. This is a more graceful solution than adjusting coverage, which may not always yield desired traffic offloading. At this point, only single carrier deployments are considered.

Where there are spectrum availability issues, adding a cell may be the last resort solution. This, of course, requires longer lead times in order to accommodate zoning, cell installation, RF calibration and optimization stages before turning the cell over to commercial service.

• Max Number Of Legs Reached:

One method to mitigate handoff failures due to full active set is to allow more pilots in the Active set. That is, the “Maximum Legs in Handoff” translation can be increased to 6.

Note, however, that there could be potential downsides to expanding the Active set (e.g., use up more MAC Indices on the forward link and increase hardware utilization on the reverse link). It may still be a good interim step, however, especially if the problem areas are localized and not severe in nature.

A better/longer-term solution would be to resort to traditional drive test based RF optimization techniques and attempt to improve dominant coverage in the region (see section 5.2.2). This will limit the likelihood of encountering a full active set when a strong candidate pilot needs to be added. Another option after exhausting efforts to improve dominant coverage in the area is to experiment with higher values of “Pilot Detection Threshold” and “Pilot Drop Threshold”. These translation parameters are in the Sectors section of EMS GUI and the generally 25 The Maximum Number of Users constraint may block new call attempts only. Handoffs are not blocked. So, if a sector receives a lot of soft handoff traffic, the total number of users supported can exceed the limit, potentially bumping against the MAC Index limit.

LUCENT TECHNOLOGIES PROPRIETARY 85

Page 86: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

recommended values are –9 and –11 dB respectively. Higher values will tend to keep Active set smaller. In future, the availability of dynamic add/drop thresholds (similar to IS95B handoffs enhancements) will also help reduce Active set size.

• Miscellaneous Reasons:

As mentioned earlier, this will require more detailed tracing to identify the root cause and fix accordingly. Simple steps such as restoring the cell, reseating/replacing the FLM/RLM, identifying issues with reference drifts might first be attempted to mitigate/eliminate handoff failures due to other reasons.

5.3.13 Core IP network issues

Concept/Reasons

Performance issues in the core IP network will directly affect the data throughput. These issues in the IP network could include latency, packet drop rate and degraded throughput. Data to and from the user flows through the PCF, PDSN and several routers. Sub-optimal performance of any of these network elements or the interface between these elements could result in degraded data throughput rates.

Failure Symptoms and Identification strategies

The general failure symptom is lower forward and reverse data throughput rates observed from service measurements. In addition, there are some secondary metrics that indicate the performance of the link between the PCF and the PDSN.

Higher than normal rate of PCF-PDSN failure rate (ratio of Errors in A10 interface and RLP octets sent to PCF) indicates that there are performance issues in the A10 interface between the PCF and the PDSN.

Call processing failure and alarm analysis in SPAT3G also provides some information on failures on the PCF and the link between the PCF and the PDSN. Error codes and reject codes on the call processing failures provides additional information on the failure [20].

Suggested Mitigation Techniques

One way to clearly identify and mitigate the problem in the core IP network is to use the Data Quality of Service (DQoS) tool. The DQoS tool collects and analyzes data from different nodes in the network. Correlated analysis of data from different monitoring points provides information about which network node and/or link is performing non-optimally. However, deploying this tool requires physically hooking up monitoring points at different points.

LUCENT TECHNOLOGIES PROPRIETARY 86

Page 87: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

APPENDIX A METRIC CROSS-REFERENCE

Table A-1 Established Call Rate Metric Cross Reference

KPI Metric Name Sections Established Call rate Abnormal Release rate 5.1.4 AN initiated RF Failure rate 5.1.2 AT initiated RF Failure rate 5.1.2 Connection Request with Four or More Pilots (%) 5.1.2.2 Connection Request with One Pilot (%) 5.1.2.2 Connection Request with Three Pilots (%) 5.1.2.2 Connection Request with Two Pilots (%) 5.1.2.2 Established Call rate 5.1 EVM DRC Erasure rate 5.1.2.5 Fast Connect Failure rate 5.1.1.1 Number of times Maximum Connections Reached 5.1.6 Number of Times TP is in Overload 5.1.5 One Second Average RSSI Rise 5.1.2.5, 5.1.4 One Second Peak RSSI Rise 5.1.2.5, 5.1.4 Page Failure rate 5.1.2.1 RAN Authentication Failure rate 5.1.8 RATI Session Setup Requests 5.1.2.7 Reverse Frame Error rate 5.1.2.5, 5.1.4 Reverse Retransmission Failure rate 5.1.2.5 Sessions Denied due to TP Overload 5.1.5 Soft Handoff Attempt to Assign rate 5.1.2.3 Ten Second Average RSSI Rise 5.1.2.5, 5.1.4 Ten Second Peak RSSI Rise 5.1.2.5, 5.1.4 Total Connection Requests 5.1.4 TP Overload Duration 5.1.5 TP Processor Usage 5.1.5

Table A-2 Drop Call Rate Metric Cross Reference

KPI Metric Name Sections Drop Call rate Abnormal Release rate 5.2.8 Connection Release rate Other Reasons 5.2.7 Connection Request with Four or More Pilots (%) 5.2.2 Connection Request with One Pilot (%) 5.2.2 Connection Request with Three Pilots (%) 5.2.2 Connection Request with Two Pilots (%) 5.2.2 Drop Call rate 5.2

LUCENT TECHNOLOGIES PROPRIETARY 87

Page 88: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Dropped Call rate due to HO Failures 5.2 Established Call rate 5.2.1 EVM DRC Erasure Rate 5.2.5 One Second Average RSSI Rise 5.2.5, 5.2.8 One Second Peak RSSI Rise 5.2.5, 5.2.8 Page Failure rate 5.2.1 Reverse Frame Error rate 5.2.1, 5.2.5, 5.2.8 Reverse Retransmission Failure rate 5.2.5 Soft Handoff Attempt to Assign rate 5.2.3 Soft Handoff Failure rate 5.2.12 Soft Handoff Failure rate at target sector 5.2.12 Ten Second Average RSSI Rise 5.2.5, 5.2.8 Ten Second Peak RSSI Rise 5.2.5, 5.2.8 Total Connection Requests 5.2.8

Table A-3 Data Throughput Metric Cross Reference

KPI Metric Name Sections Data Throughput Abnormal Release rate 5.3.8 Average Forward Data Burst rate [Kbps] 5.3 Average Reverse Outer Loop Frames rate [Kbps] 5.3 Connection Request with Four or More Pilots (%) 5.3.2 Connection Request with One Pilot (%) 5.3.2 Connection Request with Three Pilots (%) 5.3.2 Connection Request with Two Pilots (%) 5.3.2 EVM DRC Erasure rate 5.3.5 Forward Transmit Data rate 5.3 One Second Average RSSI Rise 5.3.5, 5.3.8 One Second Peak RSSI Rise 5.3.5, 5.3.8 PCF PDSN Failure rate 5.3.13 Reverse Frame Error rate 5.3.1, 5.3.5, 5.3.8 Reverse Retransmission Failure rate 5.3.5 Soft Handoff Attempt to Assign rate 5.3.3 Soft Handoff Failure due to other reasons 5.3.12 Ten Second Average RSSI Rise 5.3.5, 5.3.8 Ten Second Peak RSSI Rise 5.3.5, 5.3.8

LUCENT TECHNOLOGIES PROPRIETARY 88

Page 89: EV-DO RF Optimisation

APPENDIX B OVERVIEW OF SPAT3G TOOL

System Performance Analysis Tool 3G (SPAT3G) is a PC-based tool that is designed for rapid troubleshooting, system performance analysis and RF optimization of CDMA, EVDO and UMTS networks. SPAT3G for EVDO has the ability to analyze Service Measurements (SM), ROP, Neighbor list and Translations/configuration data simultaneously. SPAT3G supports EVDO releases 21 and beyond.

SPAT3G structure

There are two components in SPAT3G – the OMP scripts and the PC GUI tool. The OMP scripts have to be installed on the OMP to collect SM, ROP, and Translations/configuration data on a daily basis. A cron job can be set up on the OMP to run these scripts a few minutes past midnight every day to collect the relevant data. At the completion of the cron job, a single zipped file is generated that is an archive of the SM, ROP, and Translations/configuration data that was collected for that day. The naming convention used for the zipped file is SN-Number_yyyymmdd.sph (e.g. 1_20040901.sph). With the SPAT3G v.4.6.0 release, the OMP scripts have been changed to allow users to better control data collection. Users may now specify a different date, release, range of hours, System ID, ECP or SN number using command line options.

The sph file needs to be downloaded to the PC from the OMP everyday for further analysis. The PC portion of the SPAT3G tool processes the sph file(s) along with the cells information (e.g. cells.mdb from LCAT) and Street maps and creates a SPAT3G database. This database is used in displaying the SM performance metrics, ROP analysis, Translations summary, and NL Alert analysis. The cell information and street maps can be combined into a “GeoScheme” in SPAT3G which can be re-used to create multiple SPAT databases for the same market (e.g. one database for each month of the year). The next few sections explain the different types of EVDO-specific analyses that are available with SPAT3G once the data has been processed.

Service Measurements analysis

The sph files contain raw service measurement peg counts that are used within the PC portion of SPAT3G to compute performance metrics based on hard-coded equations (e.g. Connection Failure rate, Forward Data rate etc.). These performance metrics are available on a per SN, per RNC/FMS, per AP, per TP, per cell and per sector depending on the peg counts used for calculations.

Appendix Figure 1 shows the executive summary of SPAT3G. The executive summary shows key performance metrics on a per system level for the whole day (24-hour), for the busy hour for each day for which SM data is available, or for the individual hours of the day. Each metric is color-coded red when it exceeds a certain threshold. This enables the user to rapidly identify which metric is under-performing in a system. All the performance metrics are classified under multiple categories available as individual tabs in SPAT3G. For example, Dropped Call rate and Connection Failure rate metrics are available under the Failures tab in SPAT3G.

LUCENT TECHNOLOGIES PROPRIETARY – FOR INTERNAL USE ONLY USE PURSUANT TO COMPANY INSTRUCTIONS

Page 90: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

APPENDIX FIGURE 1 EXECUTIVE SUMMARY

APPENDIX FIGURE 2 SM SUMMARY WINDOW AND PEG COUNT DETAILS

LUCENT TECHNOLOGIES PROPRIETARY 90

Page 91: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Appendix Figure 2 shows the per cell level view of a certain metric. Double-clicking on any grid opens up a definition window for the peg count or the metric. The formula used for the calculation of the metric is available as part of the display window along with the threshold that is used to color-code the metric. SPAT3G metrics are also available in map view format. Appendix Figure 3 shows one such example of a performance metric on a map. Each cell/sector is color-coded according to the value of the metric that is being displayed.

SM metrics and peg counts can also trended if data from multiple days is available. Appendix Figure 4 shows an hourly trend of a metric for about 12 day’s worth of data.

APPENDIX FIGURE 3 SM MAP

LUCENT TECHNOLOGIES PROPRIETARY 91

Page 92: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

APPENDIX FIGURE 4 SM TREND

ROP Analysis

SPAT3G analysis of EVDO ROP data is done on a per AP, per TP and per cell level. In addition to the EVDO alarm analysis, the EVDO ROP analysis feature in SPAT3G has been updated to include call processing failure analysis. Appendix Figure 5 shows the system level view of EVDO AP, TP and cell alarms. Appendix Figure 6 shows the cell level view of call processing failures.

LUCENT TECHNOLOGIES PROPRIETARY 92

Page 93: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

APPENDIX FIGURE 5 ROP SUMMARY – ALARMS ANALYSIS

APPENDIX FIGURE 6 ROP SUMMARY – ALARMS ANALYSIS

LUCENT TECHNOLOGIES PROPRIETARY 93

Page 94: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

Translations/configuration Summary

The existing translations settings for a system can be viewed using SPAT3G. As part of the OMP scripts, SPAT3G picks up the XML file that is generated on the OMP using the cmexportrun utility. The XML file contains the translations/configuration settings from all the database forms and for all cell/sectors in the system.

SPAT3G color-codes any translation setting that is different from Lucent recommended values. Comparison of this setting can also be made against specific user-set values also and this is color-coded differently.

A parameter definition window pops up when the user double-clicks on a particular translation grid. The translations summary can also viewed for all the cells/sectors at the same time or for one cell/sector at a time. The difference between translations settings for two different cells or two different days can also be done in SPAT3G. Appendix Figure 7 shows a sample translations/configuration summary window with a parameter definition window.

APPENDIX FIGURE 7 TRX SUMMARY WINDOW

LUCENT TECHNOLOGIES PROPRIETARY 94

Page 95: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

NL Alert Analysis

NL Alert feature enables the optimization of EVDO neighbor lists. The different types of warnings that are generated include Reciprocity alert, one-way and two-way PN ambiguity alerts, AP IP alert and reciprocal search window alert.

In addition, the NL Alert tabular report, a map view is available for the current neighbor list and for all the individual alerts. This enables the user to visualize where the neighbor list problem is located.

SPAT3G also displays distance between source and target sectors for all NL Alerts if cell location information is available as a GeoScheme. This data may be used to prioritize NL Alert fixes. Appendix Figure 8 shows a sample NL Alert summary window with a map indicating the locations of the cells involved in the alert.

APPENDIX FIGURE 8 NL ALERT SUMMARY

Other features in SPAT3G

LUCENT TECHNOLOGIES PROPRIETARY 95

Page 96: EV-DO RF Optimisation

1X EV-DO RF PERFORMANCE ANALYSIS & TROUBLESHOOTING GUIDE

LUCENT TECHNOLOGIES PROPRIETARY 96

There are other features in SPAT3G that have not been described in detail in the previous sections that are useful for performance analysis. One such key feature is the ability to create a subset. A subset could be created as a collection of a few RNCs/FMSs, cells or sectors (e.g. cells within a cluster). The subset’s performance can be compared against the performance of the system as a whole. Another useful feature is the ability to create a superset, which allows the users to aggregate service measurements data for a group of days.

The user defined metrics feature allows the users to create their own metrics. With this feature, users are allowed to define a new service measurement metric or modify an existing service measurement metric.

The user defined category feature allows the users to create a new category with a set of service measurement peg counts and/or metrics. Once created, the user defined category will be available as a new tab in the SM table window.

Even in the absence of SM and/or ROP files, SPAT3G can also be used to view translations summary and perform NL Alert analysis.

Any table that is displayed in SPAT3G can also be exported as a comma-delimited file or as an MS Excel spreadsheet. This file can then be used for further analysis if the need exists. SPAT3G also has the mechanism to report bugs or feature requests. This can be done from the “Help” menu option and an email from the user’s computer is sent automatically to the SPAT team.

User preferences can be set from the “Preferences” menu option. Some examples of preferences are setting working directory, and defining custom ranges and labels for maps.

A detailed SPAT3G for EVDO training course is available (LTW609) from the Wireless University website. A detailed SPAT3G user guide is also available from within the tool.