KPI Optimization Test Plan for LTE
-
Upload
huzaif-zahoor -
Category
Documents
-
view
330 -
download
8
Transcript of KPI Optimization Test Plan for LTE
-
8/10/2019 KPI Optimization Test Plan for LTE
1/42
Alcatel-LucentProprietaryAlcatel-Lucent - Proprietary
This document contains proprietary information ofAlcatel-Lucent and is not to be disclosed or used
except in accordance with applicable agreements.Copyright 2006 Alcatel-Lucent Technologies
Unpublished and Not for PublicationAll rights reserved
Subject: Alcatel-Lucent KPI Optimization Test Plan
for LTE
Date: April 22, 2010
Version 1.0
ABSTRACT
This document provides a detailed test plan for KPI Optimization of the LTE fieldtechnology deployment as detailed in the Verizon High Level Test Plan. The tests will beexecuted in Boston, MA and surrounding areas. The primary objective of the
Optimization is to validate basic KPI functionality, evaluate performance of LTE Air-
Interface functionalities. The scope of test cases included in this KPI Test Plan spansseveral areas including access, latency, coverage and Capacity
http://all.alcatel-lucent.com/wps/portal/!ut/p/kcxml/04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLd4y3dAXJgFku-pHIIgbxjnABX4_83FT9IH1v_QD9gtzQiHJHR0UACtrxRg!!/delta/base64xml/L3dJdyEvd0ZNQUFzQUMvNElVRS82X0FfOUU!http://all.alcatel-lucent.com/wps/portal/!ut/p/kcxml/04_Sj9SPykssy0xPLMnMz0vM0Y_QjzKLd4y3dAXJgFku-pHIIgbxjnABX4_83FT9IH1v_QD9gtzQiHJHR0UACtrxRg!!/delta/base64xml/L3dJdyEvd0ZNQUFzQUMvNElVRS82X0FfOUU! -
8/10/2019 KPI Optimization Test Plan for LTE
2/42
Alcatel-LucentProprietary 2See notice on first page
Revision History
Version Description Date
1.0 Initial version of test plan that will be executed for
KPI Optimization
04/22/10
Table 1: Revision History
References
[1]LTE KPI Optimization High Level Test Plan, Version 1, April 22 2010, VerizonWireless
-
8/10/2019 KPI Optimization Test Plan for LTE
3/42
Alcatel-LucentProprietary 3See notice on first page
Glossary of Terms
DLLS Down-Link Load SimulatorEAT Enhanced Analysis Tool
ePC Enhanced Packet Core
GbE Gigabit EthernetLLDM LGE Logging and Diagnostic ModuleLMT Local Maintenance Tool
MME Mobility Management Entity
MPLS Multi Protocol Label SwitchingMS Management Server
NPO Network Performance Optimization
OAM Operation, Administration and Management
OLSM Open Loop Spatial MultiplexingRAMSES Role-based Access Management Security SystemRRH Remote Radio Head
SAE System Architecture EvolutionTLS Transparent LAN Service
VLAN Virtual LAN
-
8/10/2019 KPI Optimization Test Plan for LTE
4/42
Alcatel-LucentProprietary 4See notice on first page
Table of Contents
ABSTRACT.................................................................................................................................................... 1
REVISION HISTORY ................................................................................................................................. 2
REFERENCES ............................................................................................................................................. 2
1 INTRODUCTION ............................................................................................................................... 72 KPI OPTIMIZATION TARGET .......................................ERROR! BOOKMARK NOT DEFINED.
3 OPTIMIZATIOIN SYSTEM .............................................................................................................. 8
3.1 DEPLOYMENTSYSTEM ARCHITECTURE .................................................................................... 83.2 AIR INTERFACE OVERVIEW ..........................................................................................................113.3 ACCESS TERMINALS ......................................................................................................................113.4 LTEENODEBFUNCTIONS .............................................................................................................113.5 LTEMMEFUNCTIONS .................................................................................................................113.6 LTESAEFUNCTIONS ...................................................................................................................123.7 DATA LAPTOP CONFIGURATION ...................................................................................................123.8 APPLICATION SERVERS .................................................................................................................123.9 RFOPTIMIZATIONTOOLS ........................................................................................................12
3.9.1 eDAT Tool ...........................................................................................................................133.9.2 Enhanced Analysis Tool (EAT) ............ ........... .......... ........... .......... ........... .......... ........... .....133.9.3 Agilent tool .......... ........... .......... ........... .......... ........... .......... .......... ........... .......... ........... .......143.9.4 WPS .....................................................................................................................................143.9.5 WINDS ................................................................................................................................143.9.6 SYNCRO TEST ....................................................................................................................14
4 PROCESS OVERVIEW .....................................................ERROR! BOOKMARK NOT DEFINED.
4.1 DEPLOYMENTSITE LOCATIONS ...............................................................................................174.2 SITE READINESS ..................................................................ERROR!BOOKMARK NOT DEFINED.
4.2.1 Spectrum Clearance ............................................................................................................134.2.2 Antenna Audit ......... ........... ........... .......... ........... .......... .......... ........... .......... .......... ........... ....134.2.3 Sector Verification...............................................................................................................14
4.2.4 Baseline Existing System .......... ........... .......... .......... ........... .......... ........... .......... ........... .......144.3 RFOPTIMIZATIONPLANNING ..............................................................................................194.3.1 Perform Parameter Audit ........... .......... ........... .......... ........... .......... ........... .......... ........... .....134.3.2 Validate Initial Neighbor lists .............................................................................................134.3.3 Tool Readiness ....................................................................................................................144.3.4 Define Clusters ........... ........... .......... ........... .......... ........... .......... .......... ........... .......... ...........144.3.5 Drive Route Planning .......... .......... ........... .......... ........... .......... ........... .......... ........... .......... ..14
4.4 RFOPTIMIZATIONEXECUTION ............................................................................................174.4.1 Cluster Optimization ...........................................................................................................134.4.2 System Verification ..............................................................................................................13
5 TEST CASES ......................................................................................................................................20
5.1 PHYSICAL LAYER THROUGHPUT TESTS PEAK ...............................................................................205.1.1 Single User Downlink Physical Layer Throughput Test Peak .................................................205.1.2 Single User Uplink Physical Layer Throughput Test Peak .....................................................28
5.2 RLCTHROUGHPUT TESTS PEAK .................................................................................................305.2.1 Downlink RLC Throughput Peak Test .......... .......... ........... .......... ........... .......... .......... ........... ..305.2.2 Uplink RLC Throughput Peak Test .........................................................................................31
5.3 PHYSICALLAYERTHROUGHPUT TESTS MEDIAN ..................................................................305.3.1 Downlink Physical Layer Throughput Median Test .................. ........... .......... ........... .......... ....305.3.2 Uplink Physical Layer Throughput Median Test ....................................................................31
-
8/10/2019 KPI Optimization Test Plan for LTE
5/42
Alcatel-LucentProprietary 5See notice on first page
5.4 RRCSETUPFAILURERATE ....................................................................................................335.5 ATTACHFAILURERATE.........................................................................................................345.6 ATTACHDELAY .......................................................................................................................355.7 SERVICEREQUESTFAILURERATE .....................................................................................365.8 SERVICEREQUESTDELAY ....................................................................................................375.9 DEDICATEDBEARERACTIVATIONFAILURERATE ........................................................395.10 DEDICATEDBEARERACTIVATIONDELAY .......................................................................405.11 DEDICATEDBEARERDROPRATE ........................................................................................415.12 CONTEXTDROP .................................................................ERROR!BOOKMARK NOT DEFINED.5.13 RRCDROP ..................................................................................................................................335.14 ACCESSRACHLATENCY .......................................................................................................345.15 DLPHYSICALTHROUGHPUT5TH.%-ILE ..............................................................................355.16 ULPHYSICALTHROUGHPUT5TH.%-ILE ..............................................................................365.17 RLCARQ/HARQRETRANSMISSIONRATE ..........................................................................375.18 PACKETLATENCY(ROUND-TRIP DELAY) ..........................................................................395.19 S1/X2HANDOVERFAILURERATE .......................................................................................405.20 S1/X2HANDOVERINTERRUPTIONTIME,INTRA-ENB .....................................................415.21 S1/X2HANDOVERINTERRUPTIONTIME,INTER-ENBERROR!BOOKMARK NOT DEFINED.5.22 INTRA/INTERMMETAUFAILURERATE ............................................................................335.23 PAGINGPERFORMANCE ........................................................................................................345.24 ATTACHDELAY .......................................................................................................................355.25 IRATHANDOVERFAILURERATE ........................................................................................365.26 RF-SINR ......................................................................................................................................375.27 RF-RSR[ ......................................................................................................................................39
-
8/10/2019 KPI Optimization Test Plan for LTE
6/42
Alcatel-LucentProprietary 6See notice on first page
List of TablesTable 1: Revision History ................................................................................................................ 2Table 2: DEPPLOYMENT Cell Locations .................................... Error! Bookmark not defined.Table 3: SNR of Different Cell Locations ...................................... Error! Bookmark not defined.
List of FiguresFigure 1: LTE DEPLOYMENT Network Architecture .................................................................. 9Figure 2: LTE DEPLOYMENT Network Transport Configuration ............ Error! Bookmark not
defined.Figure 3: LTE DEPLOYMENT Network Transport Configuration ............ Error! Bookmark not
defined.Figure 4: EAT configuration in LTE DEPLOYMENT ................................................................. 14
-
8/10/2019 KPI Optimization Test Plan for LTE
7/42
Alcatel-LucentProprietary 7See notice on first page
Introduction
KPI OPTIMIZATION TARGET
Category Sub-Category Scope Target
ValuePerformance-
Accessibility
RRC Setup Failure Rate C
0.70%
Attach Failure Rate L 2.50%
Attach Delay L 2 seconds
Service Request Failure Rate C
2.50%
Service Request Delay L 0.5 second
Dedicated Bearer Activation
Failure Rate
N
1.50%
Dedicated Bearer Activation Delay N 0.5 seconds
Performance-
Retainability
Dedicated Bearer Drop Rate N
1.20%
Context Drop N 1.20%
RRC Drop C 1.20%
Performance-
Integrity
Access RACH Latency L
0.5 secondsDL/UL Physical Layer Throughput,
peak
L
60/20 Mbps
DL/UL RLC Throughput, peak L
55/18 MbpsDL/UL Physical Layer Throughput,
median
C
7/3 Mbps
DL/UL Physical Layer Throughput,
5th
%-ile
R
1/0.5 Mbps
RLC ARQ/HARQ Retransmission
Rate
N
1%
-
8/10/2019 KPI Optimization Test Plan for LTE
8/42
Alcatel-LucentProprietary 8See notice on first page
Packet Latency (round-trip delay) L 30 msec
Performance-
Mobility
S1/X2 Handover Failure Rate C
1.20%
S1/X2 Handover Interruption
Time, intra-eNB
L
100 msec
S1/X2 Handover Interruption
Time, inter-eNB
L
100 msec
Intra/Inter-MME TAU Failure
Rate
R
2%
Paging Performance R 95%
IRAT handover failure rate R
RF-SINR Percent Included Area > 13 dB
SINR
R
10%
Percent Included Area > -5 dB
SINR
R
90%RF-RSRP Percent Included Area < 143 dB RL
OPL (referenced to full-power
signal)
R
90%
DEPLOYMENT SYSTEM
1.1 Trial System Architecture
This section provides a high level description of the LTE system architecture and a
description of all involved entities and interfaces between entities. As shown inFigure 1,the LTE architecture network is composed of:
Multiple eNodeBs
An ePC encompassing two functions: MME and SAE Gateway
Metro Ethernet Backhaul
Applications servers
As shown in the figure, the cluster of eNodeBs will be connected via the 7705 and 7750
routers to the other network elements such as the ePC and application servers. Each
eNodeB will consist of a D2U and three TRDUs. Each D2U will consist of a uCCMs
(controller with interface to backhaul) and three eCEMs (modems; one for each sector).Each eCEM will be connected via fiber optic cable to the TRDU.
-
8/10/2019 KPI Optimization Test Plan for LTE
9/42
Alcatel-LucentProprietary 9See notice on first page
Figure 1: LTE Deployment Network Architecture
eNB
LTE RAN
MetroEthernetbackhaul
MetroEthernetbackhaul
eNB
OPTICALFIBER
OPTICALFIBER
7750
IPIP
MME/SAE GW
VoIP
SMS
VoIP
SMSTCP/U DP
HTTP
TCP/U DP
HTTP
VIDEOVIDEO
VIDEOVIDEO
LTE Core
Monitoring
WMSEAT server
EATAgilent
MU serverPDM
Analyser
CISCO PIX
RAMSES
Gate VPN
Mediationdevice
RAMSES
Gate VPN
RAMSES
Gate VPN
Mediationdevice
SUN Sparc/5620 SAM
RAMSES
NTPserver
CLIENTS
hub hub
hub
Coppershielded7705
7705
-
8/10/2019 KPI Optimization Test Plan for LTE
10/42
Alcatel-LucentProprietary 10See notice on first page
The information below details the hardware description and the associated OAM
equipment list for the Trial network:
eNodeB
o D2U V2 (1 uCCM + 3 eCEM)
o TRDU (remote-radio-heads comprising of amplifiers and filters), 40W Tx
power Backhaul
o 7750 SR (service router)
o 7705 SAR (service aggregation router)
Transport services
o Cisco TLS EVC
ePC
o PDN/MME/SAEGW - IPD ATCA
Security
o Cisco PIX
LAN switchingo Module from 7750
Remote supporto RAMSES
In order to manage the eNodeBs, a complete OAM system has to be designed to host the
following functions:
Configuration management
Fault management Performance management
Traces
Additional OAM systems include:
LMT to configure and set up the D2U platform to commission IP addresses,
DLCP server, and default gateway. The LMT connects locally to console port
LMT to configure the ePC complex (MME, S-GW and PDN GW) (connects
locally to console port)
Management Server (MS) to configure the eNB and display eNB status and faultinformation
Network Performance Optimization (NPO) to collect performance counts and
measurements. The MS and the NPO are together referred to as the LTE
Management System Server
MS and NPO clients to interface to MS and NPO servers Netscreen firewall to protect the LTE network elements from intrusion Netscreen Gate firewall to filter access to the RAMSES Mediation system
RAMSES Remote Access and RAMSES Mediation PC to provide access control
and authentication on remote access to the LTE network elements
5620 S/W product managing the monitoring aspects on 7750
-
8/10/2019 KPI Optimization Test Plan for LTE
11/42
Alcatel-LucentProprietary 11See notice on first page
1.2 Air Interface Overview
Main inputs to setup LTE air interface during the Optimization are:
10 MHz spectrum bandwidth in the Upper Band C (700770 MHz) Number of frequency carriers: 1
3 sectors per eNB site
3 TRDUs per eNB site SFBC/MIMO in DL and SIMO in UL
Cross pole and Vertical pole antennas
1.3 Access terminals
LGE G Series UEs will be used during the Optimization
1.4 LTE eNodeB functions
The eNodeB hosts the following functions:
Functions for Radio Resource Management: Radio Bearer Control, RadioAdmission Control, Connection Mobility Control, Dynamic allocation ofresources to UEs in both Uplink and Downlink (scheduling)
Routing of user plane data towards SAE gateway
Scheduling and transmission of paging messages (originated from the MME)
Scheduling and transmission of broadcast information (originated from the MMEor OAM).
Measurement and measurement reporting configuration for mobility and schedulingfunctions
1.5 ALU 7750 Service Router FunctionsThe SR7750 is an edge router that will host the following functions:
Link aggregation
DSCP mapping
VLAN and LAN switching
IP router to reach different Application servers
1.6 ALU 7705 Service Aggregation Router Functions
The ALU 7705 is a Service Aggregation Router (SAR) that offers:
A service-oriented capability to the RAN IP/MPLS RAN transport solution
1.7 LTE MME Functions
The MME hosts the following functions:
Idle mode mobilityo S1 connection establishment
-
8/10/2019 KPI Optimization Test Plan for LTE
12/42
Alcatel-LucentProprietary 12See notice on first page
o Idle to active mode transition
o Active to idle mode transition
Session managemento QoS control
S1 handling during HO
1.8 LTE SAE Functions
The SAE Gateway hosts the following functions:
Multiple bearer support (one default and one dedicated)
S1 GTP-U bearer endpoint Idle mode handling: bearer suspension with paging request
S1 path switch during handover
1.9 Data Laptop Configuration
The access terminal will interface with a data laptop to support Pings, FTP, UDP, and
HTTP data transfers. The laptop should be configured per the recommended parametersto optimize performance and provide an appropriate comparison to existing data. Theserecommendations include:
Windows 2000 Professional or Windows XP edition
IP header compression (VJ compression) turned OFF PPP software compression OFF except for HTTP data transfer
128Kbytes TCP window size
MTU of 1500 bytes
1.10 Application ServersApplication servers will be used during the Trial to provide the data content for the
various tests. These servers will be provided by Alcatel-Lucent and they need to be easily
accessible and not blocked or restricted by low bandwidth pipes.
Windows 2000 Server Edition will be used as a data server, which should reside as close
to the PDN gateway as possible. This will eliminate performance uncertainty due to any
external network delay. Data applications available through this server will be:
UDP
DOS FTP - TCP/IP Ping
1.11 Test tools
In order to validate functionality and quantify performance, a variety of test tools will be
used.
Traffic Generation Tools:
DOS-FTP, WINDS, Ping To generate TCP/IP and UDP based traffic data for
measuring data capacity and network latency
-
8/10/2019 KPI Optimization Test Plan for LTE
13/42
Alcatel-LucentProprietary 13See notice on first page
Ping scripts for access tests
Logging tools
Agilent is a Diagnostic Monitor for logging and analyzing over-the-air Network
system performance and parameters
Enhanced Analysis Tool (EAT): to collect internal traces generated by the eNB
Analysis tools
Agilent Protocol analyzer
KPI collector and generator Packet Data Monitoring (PDM) tool: a distributed data performance, analysis and
troubleshooting service
1.11.1 PDM Tool
PDM is a distributed data performance analysis and troubleshooting service for packet
data, providing end-to-end and per-link data quality analysis. It can be used for the
following functions:
Vendor independent Packet data network monitoring capabilities
Consistent and automated testing and analysis of packet networks
Characterization of end-user perceived performance in terms of throughput,latency, dropped packets, etc.
Ability to monitor / test packet network to support time sensitive applications,
such as VoIP
Identification of links/components requiring maintenance and optimization, and
monitoring of link and end-to-end performance
Enables precise data correlation across the entire network Reduces resource requirements with an automated and remote controlled sniffer
system
1.11.2 Enhanced Analysis Tool (EAT)
The EAT stores internal traces of up to 15 eNodeBs. EAT runs on a Linux PC which is
connected to one or several eNodeBs via Ethernet (Figure 2). It connects to each eNodeB
and configures the trace service by providing a destination IP address (i.e. its own IP
address) and which traces to activate. EAT does not know the exact moment the traces
start, so it has to listen via a socket server. Traces are received via UDP.
-
8/10/2019 KPI Optimization Test Plan for LTE
14/42
Alcatel-LucentProprietary 14See notice on first page
Figure 2: EAT configuration in LTE Trial
1.11.3 Agilent tool
The Agilent DNA is a protocol analyzer and can be used for user-plane analysis. It offersa scaleable, distributed probing architecture and the raw data is collected at the S1 level.Owing to the true client-server architecture, each user client is able to test independently.To enable detailed protocol analysis, Hardware Intelligent Packet Processing (HI-PI2) atline rate will be required. At the user-plane, the analysis will also require a separation ofthe signalling and payload packets to be able to correlate and analyze events at bothlevels.
1.11.4 Wireshark
Wireshark is a network protocol analyzer. Wireshark offers the capabilities to capture
network data elements and provide some metrics on the data snooping.
1.11.5 SyncroTest
SyncroTest is an automated test tool to control test mobiles (called Probes) remotely using acentral Master Controller console. (SeeFigure 3: SyncroTest Architecturebelow) SyncroTest
probes control the functionality of LLDM/Agilent, WINDS and FTP to generate traffic andmonitor the connections from the probes point-of-view. The log data is then sent back to themaster control for analysis. Probes have also been developed to support remote control of EAT
so that testing and data collection can be synchronized.
A single SyncroTest master controller can control at least 8 Agilent/LLDM/WINDS probes andan EAT probe simultaneously eliminating the need for an RF engineer in each drive test vehicle.
LTE RAN
OAM
LTE CORE7750
BackboneBackbone
CLIENTS
hub
Eng rules for EAT: Traces are transported insideUDP payloads trace transportUDP/IP/Eth (standard Ethernetframes)
Trace content officially released(5.3 Mbps) by System designteam.
Traces forwarded inside VLANOAM and routed thanks to LANswitch to EAT server (dest IP @used)
Internal eNB traces activated bydefault Limited export tooutside (up to EAT server) TBC
Traces coming from standardEthernet port of the eNB. Debugport not used.
Eng
rules for EAT: Traces are transported insideUDP payloads trace transportUDP/IP/Eth (standard Ethernetframes)
Trace content officially released(5.3 Mbps) by System designteam.
Traces forwarded inside VLANOAM and routed thanks to LANswitch to EAT server (dest IP @used)
Internal eNB traces activated bydefault Limited export tooutside (up to EAT server) TBC
Traces coming from standardEthernet port of the eNB. Debugport not used.
EAT server
EAT
To get EAT traces at eNB site:
a local access point to OAM VLAN required
either from an existing connection point
or from an external hub/switch to come with on eNB site
-
8/10/2019 KPI Optimization Test Plan for LTE
15/42
Alcatel-LucentProprietary 15See notice on first page
SyncroTest uses self healing TCP connections with each probe to direct the probe and it receivesperiodic heartbeats from the probe to update the probes status.
Figure 3: SyncroTest Architecture
1.11.6 Data Analysis Tool (eDAT)
eDATisapost-processingtoolthatallowsausertoanalyzeRFperformanceKPIs.eDATtakesinputfrombothUElogsandeNodeBlogs.eDATprovidesastandardizedapproachtoanalyze
metrics intheRadioAccessNetwork. eDATusestheUELogsgeneratedbyAgilent/LLDMas
inputfromtheUEperspective.TheeNodeBlogsaregeneratedbyEATandprovideanadditional
inputtoeDATtoanalyzeKPIsfromtheeNodeBperspective.Thefollowingdiagramillustrates
theeDATconfigurationfortheLTEDeployment.
-
8/10/2019 KPI Optimization Test Plan for LTE
16/42
Alcatel-LucentProprietary 16See notice on first page
Figure 4: eDAT LTE Trial Configuration
Once the data has been collected, eDAT post-processes the UE and eNodeB log files to
generate the analyses of the KPIs. eDAT is a standalone tool that does not need to be
connected to the fixed infrastructure. The output includes maps, graphs, plots, reports
and message decoding. All these can be used to evaluate the RF performance of the LTETrial network. eDAT makes use of event timestamps and locations of the UEs to
geographically plot out the data.
KPIsKPIsKPIs
eNodeB
LDAT
EventsKPIs Messages
eNodeB logsEAT
LTE
UE
LGE-LLDM
UE logs
KPIsKPIsKPIs
eNodeB
LDAT
EventsKPIs Messages
eNodeB logsEAT
LTE
UE
LGE-LLDM
UE logs
KPIsKPIsKPIs
eNodeB
LDAT
EventsKPIs Messages
eNodeB logsEAT
LTE
UE
LGE-LLDM
UE logs
-
8/10/2019 KPI Optimization Test Plan for LTE
17/42
-
8/10/2019 KPI Optimization Test Plan for LTE
18/42
Alcatel-LucentProprietary 18See notice on first page
Drive routes and Key Performance Indicators will be the same as the ones used later for
System Verification. It is important to keep the drive routes and KPIs identical for
performance validation and comparison purposes. Drive routes and KPIs must be agreedupon with the customer.
1.14 RF Optimization PlanningThe Optimization planning phase ensures system and tool readiness for RF Optimizationbefore beginning the actual drive testing.
1.14.1 Perform RF Parameter Audit
RF parameters must be inspected for consistency with the LTE parameter catalogue.The RF parameter settings used in the network can be obtained from the NDP project
database. These settings are then audited using the LTE parameter catalogue WPS
1.14.2 Validate Initial Neighbor Lists
An important step within the RF Optimization preparation phase is associated with theneighbor list verification. The complete neighbor lists in the LTE network are required to
compare the neighbor relations with network design plots. Neighbor relations need to be
verified for recent updates, validity and appropriateness. The recommended strategy is tohave a minimum number of neighbor relations in the neighbor lists. The neighbor lists
used in the network can also be obtained from the WPS project database.
1.14.3 Tool Readiness
Appropriate drive test tools and post-processing tools, need to be prepared foroptimization.
1.14.4 Define Clusters
Approximately 15-19 eNodeBs should be combined into one cluster. The actual numberused is based on the network expansion as well on topographical environment. The
clusters are selected to provide a center eNodeB with tow rings of surrounding eNodeBs .
1.14.5 Drive Route Planning
Drive routes need to be defined for Sector Verification, Cluster Optimization and SystemVerification. Coverage prediction plots, morphology and clusters can define all drive test
routes.
The drive route should maintain a distance equal to of the cell site radius for sector
verification.
The routes for Cluster Optimization shall consist of major roads, highways and hotspots.Total time to drive all routes in atypical cluster should be approximately 6 to 8 hours.
Additional border route is chosen by the way it crosses the cluster borders witout goinginto the cluster areas.
The System Verification drive route are used to collect the metrics for the Exit Criteria.
The routes are a combination of individual clusters.
-
8/10/2019 KPI Optimization Test Plan for LTE
19/42
Alcatel-LucentProprietary 19See notice on first page
1.15 RF Optimization Execution
The RF Optimization Execution consists of drive tests, problem area identification,
verification drives, and final drives to ensure completion of Exit Criteria. The core
activity is to provide system tuning, as well as data collection and reporting. LTEnetwork optimization would be performed under loaded network conditions.
1.15.1 Cluster Optimization
The Cluster Optimization consists of three phrases:
Unloaded Cluster Optimization
Loaded Cluster Optimization
Cluster Performance VerificationDuring the first Cluster Optimization phase, a measurement drive is performed under
unloaded network conditions using the optimization route. Once the data from the first
phase are collected, problem spots are identified and optimized. The unloaded drive testidentifies coverage holes, handover regions and multiple pilot coverage areas. It also
spots eventual overshooting sites (as interferences is minimal) from areas belonging to
neighbor clusters. The first pass might lead to correction of neighbor lists andadjustments of the fundamental RF parameters such as transmit powers and/or antenna
azimuths and antenna tilts. The drive test information highlights fundamental flaws in the
RF design under best-case conditions
The second Cluster Optimization phase is performed under loaded conditions. The drive
routes for the loaded Cluster Optimization will be exactly the same routes as those used
for the unloaded measurements drives. Loading the cell will cause an increase of negativeSNR valuses, identify potential coverage holes, result in higher BLER, result in lower
mobility throughput, and more dropped calls. The objective is to fix the problems
observed by the field teams. This involves the fine-tuning of RF parameters such as thetransmit power or handover parameters. Antenna re-adjustments (e.g. down-tilts,
azimuths, patterns/types or heights) are also occasional performed.
The Cluster performance is measured against the cluster Exit Criteria. The exit drivespurpose is to verify and to confirm specific Exit Criteria demanded by the customer.
The final statistics from the cluster exit drive are presented to the customer for approval.
These statistics contain plots as well as data in tabular form.
1.15.2 System Verification
System Verification is the final phase of the RF Drive Test Based Optimization activity
and it focuses specifically on collecting overall performance statistics. It is performedunder loaded conditions with all cells activated. System Verification involves fusion ofthe previously optimized clusters and once again is required to demonstrate that Exit
Criteria are met system-wide.
The final statistics from the System Verification are presented to the customer forapproval.
-
8/10/2019 KPI Optimization Test Plan for LTE
20/42
Alcatel-LucentProprietary 20See notice on first page
1.16 Test Cases
The default test mode in the DL will be the open loop spatial multiplexing (OLSM) and
for the UL it will be SIMO.
DLLS (down-link load simulator) will be used to generate interference on the DL of the
neighboring cells for loading purposes.
1.17 Single User Throughput Test Peak
1.17.1 Single User Downlink Airlink Throughput Peak Test
Test Objectives:Test Validate the performance by conducting single-user stationary and
limited mobility tests on pre-selected locations in an embedded sector and in a limited
drive route within the same sector, respectively. The tests shall be performed using both
UDP applications for performance comparison, and under 50% cell loading conditions.
Test Description:Tests will be executed in Close-Loop Spatial Multiplexing (CLSM).
For the stationary tests, the test UE will be located at selected locations in an embedded
sector corresponding to the appropriate SNR ranges for Near Cell (NC). See for SNRranges. For the mobility tests, the test UE will be driven according to the predefined drive
route.
DLLS will be used to load the DL of the cells neighboring the target cell. Loading will begenerated by occupying portions of the Resource Blocks (RB). For example, to generate
a cell loading of 50%, 50% of the total DL RBs will be occupied.
Agilent Tool, Backhual Bandwidth: 100 Mbps, SINR: 17 to 21 dB, Loaded Conditions:
50%
.
Test Routes:
Near Cell, SNR17dB21 dB
Terminal Speeds:Stationary, Limited Mobility (25-30)km/hr
Test Set Up:
A drive test van will be used with rooftop mounted antennas 1. Ensure that 500MB file is available at the servers for downloads2. Ensure that Tx/Rx Antenna Correction is below 20%3. Ensure that UE is reporting Rank of 24. Ensure that antenna on the Test van are cross polarized5. Ensure that TCP Window size of client laptop is set to 512kbytes
-
8/10/2019 KPI Optimization Test Plan for LTE
21/42
Alcatel-LucentProprietary 21See notice on first page
Procedure:
Action Response
1 Park test Van at predetermined location on NCroute.
2 Open Agilent and connect UE to the Agilenttool.
3 Power the UE and ensure that the right port isassigned to the UE. Check that GPS is workingon Agilent and Winds
4 Open LGE LTE CM and click on Connect UE would start the attach process
5 Ping the Application Server to make sure UEhas acquired an IP address
IP address verified
6 Open Winds UDP and configure the rightadapter. Populate the fields with right values
7 Start logging Agilent and click Request onWinds
8 Log Data for 3 mins. UE log files collected
9 Stop the Winds UDP sessions. Stop logging onAgilent
10 Repeat Steps 7 -9 for two more runs
11 Save the UE and Winds files
Key Metrics:
1. Physical Layer Downlink Airlink Throughput Peak2. Application Layer Throughput3. Initial Block Error Rates
4. Residual Block Error Rates5. Scheduled Transport Format distribution
Expected Result:Peak Tput should be 60Mbps
Expected Test Duration: 0.5 day
1.17.2 Single User Uplink Airlink Throughput Peak Test
Test Objectives:Test Validate the performance by conducting single-user stationary and
limited mobility tests on pre-selected locations in an embedded sector and in a limited
drive route within the same sector, respectively. The tests shall be performed using bothUDP applications for performance comparison, and under no loading.
Test Description:Tests will be executed in Close-Loop Spatial Multiplexing (CLSM).
For the stationary tests, the test UE will be located at selected locations in an embedded
-
8/10/2019 KPI Optimization Test Plan for LTE
22/42
Alcatel-LucentProprietary 22See notice on first page
sector corresponding to the appropriate SNR ranges for Near Cell (NC). See for SNR
ranges. For the mobility tests, the test UE will be driven according to the predefined driveroute.
Agilent Tool, Backhual Bandwidth: 100 Mbps.
Test Routes:Near Cell, SNR17dB21 dB
Terminal Speeds:
Stationary, Limited Mobility (25-30)km/hr
Test Set Up:
A drive test van will be used with rooftop mounted antennas 1.Ensure that 500MB file is available at the client laptop for uploads
2.Ensure that SIR target is set to 18dB at the eNodeB
Procedure:
Action Response
1 Park test Van at predetermined location on NC
route.
2 Open Agilent and connect UE to the Agilent
tool.
3 Power the UE and ensure that the right port isassigned to the UE. Check that GPS is working
on Agilent and Winds4 Open LGE LTE CM and click on Connect UE would start the attach process
5 Ping the Application Server to make sure UEhas acquired an IP address
IP address verified
6 Open Winds UDP and configure the rightadapter. Populate the fields with right values
7 Start logging Agilent and click Send onWinds
8 Log Data for 3 mins. UE log files collected
9 Stop the Winds UDP sessions. Stop logging on
Agilent10 Repeat Steps 7 -9 for two more runs
11 Save the UE and Winds files
-
8/10/2019 KPI Optimization Test Plan for LTE
23/42
Alcatel-LucentProprietary 23See notice on first page
Key Metrics:
1. Physical Layer Uplink Airlink Throughput Peak2. Application Layer Throughput3. SIR target4. Initial Block Error Rates
5. Residual Block Error Rates6. Scheduled Transport Format distribution
Output:Physical Layer Uplink Airlink Throughput Peak should be recorded.
Expected Result:Peak Tput should be 20Mbps
Expected Test Duration: 0.5 day
1.17.3 Single User Downlink RLC Throughput Peak Test
Test Objectives:Test Validate the performance by conducting single-user stationary and
limited mobility tests on pre-selected locations in an embedded sector and in a limited
drive route within the same sector, respectively. The tests shall be performed using both
UDP applications for performance comparison, and under 50% cell loading conditions.
Test Description:Tests will be executed in Close-Loop Spatial Multiplexing (CLSM).
For the stationary tests, the test UE will be located at selected locations in an embeddedsector corresponding to the appropriate SNR ranges for Near Cell (NC). See for SNRranges. For the mobility tests, the test UE will be driven according to the predefined drive
route.
DLLS will be used to load the DL of the cells neighboring the target cell. Loading will be
generated by occupying portions of the Resource Blocks (RB). For example, to generate
a cell loading of 50%, 50% of the total DL RBs will be occupied.
Agilent Tool, Backhaull Bandwidth: 100 Mbps, SINR: 17 to 21 dB, Loaded Conditions:
50%
.
Test Routes:Near Cell, SNR17dB21 dB
Terminal Speeds:Stationary, Limited Mobility (25-30) km/hr
-
8/10/2019 KPI Optimization Test Plan for LTE
24/42
-
8/10/2019 KPI Optimization Test Plan for LTE
25/42
Alcatel-LucentProprietary 25See notice on first page
1.17.4 Single User Uplink RLC Throughput Peak Test
Test Objectives:Test Validate the performance by conducting single-user stationary andlimited mobility tests on pre-selected locations in an embedded sector and in a limited
drive route within the same sector, respectively. The tests shall be performed using both
UDP applications for performance comparison, and under no loading.
Test Description:Tests will be executed in Close-Loop Spatial Multiplexing (CLSM).
For the stationary tests, the test UE will be located at selected locations in an embedded
sector corresponding to the appropriate SNR ranges for Near Cell (NC). See for SNR
ranges. For the mobility tests, the test UE will be driven according to the predefined driveroute.
Agilent Tool, Backhaul Bandwidth: 100 Mbps.
Test Routes:Near Cell, SNR17dB21 dB
Terminal Speeds:Stationary, limited Mobility (25-30) km/hr
Test Set Up:
A drive test van will be used with rooftop mounted antennas 1.Ensure that 500MB file is available at the client laptop for uploads
2.Ensure that SIR target is set to 18dB at the eNodeB
Procedure:
Action Response
1 Park test Van at predetermined location on NC
route.
2 Open Agilent and connect UE to the Agilenttool.
3 Power the UE and ensure that the right port isassigned to the UE. Check that GPS is working
on Agilent and Winds
4 Open LGE LTE CM and click on Connect UE would start the attach process
5 Ping the Application Server to make sure UEhas acquired an IP address
IP address verified
6 Open Winds UDP and configure the rightadapter. Populate the fields with right values
-
8/10/2019 KPI Optimization Test Plan for LTE
26/42
Alcatel-LucentProprietary 26See notice on first page
7 Start logging Agilent and click Send onWinds
8 Log Data for 3 mins. UE log files collected
9 Stop the Winds UDP sessions. Stop logging onAgilent
10 Repeat Steps 7 -9 for two more runs
11 Save the UE and Winds files
Key Metrics:1. Physical Layer Uplink RLC Throughput Peak
2. Application Layer Throughput3. SIR target4. Initial Block Error Rates5. Residual Block Error Rates6. Scheduled Transport Format distribution
Output:
Peak Uplink RLC layer throughput should be recorded
Expected Result:Peak Tput should be 18Mbps
Expected Test Duration: 0.5 day
-
8/10/2019 KPI Optimization Test Plan for LTE
27/42
Alcatel-LucentProprietary 27See notice on first page
-
8/10/2019 KPI Optimization Test Plan for LTE
28/42
-
8/10/2019 KPI Optimization Test Plan for LTE
29/42
Alcatel-LucentProprietary 29See notice on first page
6 Open Winds UDP and configure the right
adapter. Populate the fields with right values
7 Start logging Agilent and click Send on
Winds
8 Log Data for 3 mins. UE log files collected
9 Stop the Winds UDP sessions. Stop logging on
Agilent10 Repeat Steps 7 -9 for two more runs
11 Save the UE and Winds files
Key Metrics:1. Physical Layer Uplink RLC Throughput Peak
7. Application Layer Throughput8. SIR target9. Initial Block Error Rates10.Residual Block Error Rates11.Scheduled Transport Format distribution
Output:
Peak Uplink RLC layer throughput should be recorded
Expected Result:Peak Tput should be 18Mbps
Expected Test Duration: 0.5 day
Key Metrics:7. Physical Layer Downlink Throughput Mean
8. Application Layer Throughput9. Initial Block Error Rates10.Residual Block Error Rates11.Scheduled Transport Format distribution
-
8/10/2019 KPI Optimization Test Plan for LTE
30/42
Alcatel-LucentProprietary 30See notice on first page
1.18 Best Effort Sector Throughput Tests
1.18.1 Downlink Best Effort Sector Throughput
Test Objectives:Evaluate the sector throughput for multiple UEs at stationary locations
and for limited mobility drive tests within the same sector. Tests will be conducted under
unloaded and loaded conditions, and for UDP and FTP applications.
Test Description:Three different scenarios will be tested: Open-Loop Spatial Multiplexing (OLSM), SFBC
and SIMO
For the stationary tests, 8 test UEs will be placed at selected locations corresponding to the
appropriate SNR ranges for Near Cell (NC), Mid Cell (MC), and Cell Edge (CE) locations.
The 8 UEs will be placed in a (2, 4, 2) configuration: 2 UEs at NC, 4 UEs at MC and 2UEs at CE locations (shown as 242 in Tables below). The 8 UEs will be placed in four
different vans (V1-V4 in Tables below) with 2 UEs in each van. See Error! Reference
source not found.for SNR ranges. Four different sets of (2, 4, 2) configurations will betested (Loc1-4 in Tables below).
For the mobility tests the test UEs will be driven according to a predefined limited mobility(single sector) drive route.
DLLS will be used to load the DL of the cells neighboring the target cell. Loading will be
generated by occupying portions of the Resource Blocks (RB). For example, to generate acell loading of X% (CLX in Tables below), X% of the total DL RBs will be occupied.
Three different loading conditions will be used: 0%, 50% and 100%.
Three different scenarios of active Sectors will be tested: only target sector active (1S inTables below), all three sectors of target cell active (3S), and all cells in the cluster active
All tests will be conducted using the default scheduler setting.
Test Setup:1. One or more drive test vans will be used with rooftop mounted antennas
Key Metrics:
1. Physical Layer Throughput2. Application Layer Throughput (UDP/FTP)
3. Initial Block Error Rates4. Residual Block Error Rates5. Scheduled Transport Format distribution
-
8/10/2019 KPI Optimization Test Plan for LTE
31/42
Alcatel-LucentProprietary 31See notice on first page
1.18.2 Uplink Best Effort Sector Throughput
Test Objectives:Evaluate the sector throughput for multiple UEs at stationary locations
in an embedded sector and for limited mobility drive tests within the same sector. Tests
will be conducted under unloaded and loaded conditions, and for UDP and FTP
applications.
Test Description:For the stationary tests the test UEs will be located at selected
locations corresponding to the appropriate SNR ranges for Near Cell (NC), Mid Cell
(MC), and Cell Edge (CE). See Error! Reference source not found.for SNR ranges. Forthe mobility tests the test UEs will be driven according to a predefined limited mobility
(single sector) drive route
Loading is generated by placing loading UEs in neighboring cells at pre-selectedlocations. Loading of 100% will result in an IoT of TBD dB in the target cell, while a
loading of 50% will result in an IoT of TBD dB in the target cell.
All tests will be conducted with the default scheduler setting.
Test Setup:1. One or more drive test vans will be used with rooftop mounted antennas2. Each stationary location consists of a unique set of (NC,MC,CE) locations
Key Metrics:1. Physical Layer Throughput2. Application Layer Throughput (UDP/FTP)
3. Initial Block Error Rates4. Residual Block Error Rates5. Scheduled Transport Format distribution
-
8/10/2019 KPI Optimization Test Plan for LTE
32/42
Alcatel-LucentProprietary 32See notice on first page
1.18.3 Uplink MU-MIMO Sector Throughput
Test Objectives:Evaluate the sector throughput with MU-MIMO at stationary locations
and for limited mobility drive tests. Tests will be conducted under unloaded and loaded
conditions, and for UDP and FTP applications.
Test Description:For the stationary tests the test UEs will be located at selected
locations corresponding to the appropriate SNR ranges for Near Cell (NC), Mid Cell
(MC), and Cell Edge (CE). See Error! Reference source not found.for SNR ranges. Forthe mobility tests the test UEs will be driven according to a predefined limited mobility
(single sector) drive route.
Loading is generated by placing loading UEs in neighboring cells at pre-selected
locations. Loading of 100% will result in an IoT of TBD dB in the target cell, while a
loading of 50% will result in an IoT of TBD dB in the target cell.All tests will be conducted using the default scheduler setting.
MU-MIMO implementation allows for up to 4 UEs paired (2 pairs).
Test Setup:1. One or more drive test vans will be used with rooftop mounted antennas2. Each stationary location consists of a unique set of (NC,MC,CE) locations
Key Metrics:1. Physical Layer Throughput2. Application Layer Throughput (UDP/FTP)
3. Initial Block Error Rates4. Residual Block Error Rates5. Scheduled Transport Format distribution
-
8/10/2019 KPI Optimization Test Plan for LTE
33/42
Alcatel-LucentProprietary 33See notice on first page
1.19 Downlink Scheduler
Test Objectives:Evaluate the Scheduler performance for multiple UEs at stationary locationsand for limited mobility drive tests within the same sector. Tests will be conducted under loadedconditions with UDP and FTP applications. Three Scheduler settings will be tested: proportional-
fair (PF), conservative (CO), and aggressive (AG).
Test Description:
Settings common to all tests:- Open-Loop Spatial multiplexing (OLSM) mode- 100% loading on DL of neighbor cells
For the stationary tests, 8 test UEs will be placed at selected locations corresponding to the
appropriate SNR ranges for Near Cell (NC), Mid Cell (MC), and Cell Edge (CE) locations. The 8UEs will be placed in four different vans with 2 UEs in each van. The number of active UEs will
be increased incrementally to illustrate the scheduling gain. For the stationary cases, eachcombination of a subset of the 8 UEs will be depicted as a triplet (x, y, z) in the tables below torepresent the number of active UEs at each of the three locations. For the cases with a mix of
stationary and mobility UEs, each combination of a subset of the 8 UEs will be depicted as aquartet (x, y, z, m) where m will denote the number of mobility UEs. See Error! Reference
ource not found.for SNR ranges.
For the mobility tests the test UEs will be driven according to a pre defined limited mobility
(single sector) drive route.
DLLS will be used to load the DL of the cells neighboring the target cell. Loading of 100% willbe used for the tests.
Test Setup:1. One or more drive test vans will be used with rooftop mounted antennas
Key Metrics:
1. Physical Layer Throughput2. Application Layer Throughput3. Initial Block Error Rates4. Residual Block Error Rates5. Scheduled Transport Format distribution
-
8/10/2019 KPI Optimization Test Plan for LTE
34/42
Alcatel-LucentProprietary 34See notice on first page
1.20 Uplink Scheduler
Test Objectives:Evaluate the uplink scheduler performance for multiple UEs at stationarylocations and for limited mobility drive tests within the same sector. Tests will be conductedunder loaded conditions with UDP and FTP applications. Three Scheduler settings will be tested:
proportional-fair (PF), conservative (CO), and aggressive (AG).
Test Description:
For the stationary tests, 8 test UEs will be placed at selected locations corresponding to the
appropriate SNR ranges for Near Cell (NC), Mid Cell (MC), and Cell Edge (CE) locations. The 8UEs will be placed in four different vans with 2 UEs in each van. The number of active UEs will
be increased incrementally to illustrate the scheduling gain. For the stationary cases, eachcombination of a subset of the 8 UEs will be depicted as a triplet (x, y, z) in the tables below torepresent the number of active UEs at each of the three locations. For the cases with a mix of
stationary and mobility UEs, each combination of a subset of the 8 UEs will be depicted as aquartet (x, y, z, m) where m will denote the number of mobility UEs. See Error! Reference
ource not found.for SNR ranges.
For the mobility tests the test UEs will be driven according to a pre-defined limited mobility
(single sector) drive route.
All tests will be executed with 100% UL loading. The uplink loading will be generated by placingloading UEs in the neighboring cells to generate an IoT corresponding to 100% loading in thetarget sector.
Test Setup:1. One or more drive test vans will be used with rooftop mounted antennas
Key Metrics:
1. Physical Layer Throughput2. Application Layer Throughput3. Initial Block Error Rates4. Residual Block Error Rates5. Scheduled Transport Format distribution
-
8/10/2019 KPI Optimization Test Plan for LTE
35/42
Alcatel-LucentProprietary 35See notice on first page
1.21 Latency C-plane
Test Objectives:To assess the control plane latency associated with call setup events
Test Description:
This test will determine the call setup time. The delay will be measured from the firstRACH attempt to the time the UE completes traffic channel setup. This test will be
executed with UE in BE and GBR modes.
Tests will also be executed to measure mobile terminated connection setup time. Thesetests will also be executed with UE in BE and GBR modes.
Procedure:
BE C-Plane Latency
1. Set the log mask for the DM tool to include the debug messages.
2. Initiate a call from the test UE with the UE in BE mode
3. Initiate a call from the network side to the UE with UE in BE mode
4. Repeat 100 times each
QoS C-Plane Latency5. Initiate a call from the test UE with UE in GBR QoS mode
6. Initiate a call from the network side to the UE with UE in GBR QoS mode
7. Repeat 100 times each
Key Metrics (per UE):1. Call Setup Time
5.5.1 C-Plane LatencyTest Case Priority Test Case
Description
Call Setup QoS of the Test
UE
Number
of UEs
5.5.1.1 H UE_Init_NC_BE UE Initiated BE 1
5.5.1.2 H UE-Init_NC_QoS UE Initiated GBR 15.5.1.3 H UE_Term_NC_BE UE Terminated BE 1
5.5.1.4 H UE_Term_NC_QoS UE Terminated GBR 1
-
8/10/2019 KPI Optimization Test Plan for LTE
36/42
Alcatel-LucentProprietary 36See notice on first page
1.22 Latency U-plane
Test Objectives:To assess the end user experienced latency. To measure the round tripdelay from the time a packet is generated at the IP level to the time a response is
received.
Test Description:This test will be conducted with a total of 8 UEs placed at different sector locations (NC,
MC, EC). Ping tests with 32 bytes/1462 bytes will be executed on the test UE (BE/GBR
QoS modes) while bi-directional IP traffic will be run on the other UEs to generate DLand UL loading. The number of loading UEs will be varied in the tests.
Procedure:
U-Plane Latency
1. Execute 32 byte Ping tests on the 8 UEs one at a time for 30 seconds each.
2. Execute 32 byte Ping tests on the test UE1 with bi-direction IP traffic running on theother loading UEs (3 UEs, 5 UEs and 7 UEs)
3. Repeat steps 1 and 2 the test with a ping payload size of 1462 Bytes.
4. Repeat steps 1-3 with UE in GBR mode
Key Metrics (per UE):1. Resource Utilization2. Transport Format Distribution3. Latency4. Physical Layer Throughput5. Application Layer Throughput {UDP}6. Initial Block Error Rates7. Residual Block Error Rates
-
8/10/2019 KPI Optimization Test Plan for LTE
37/42
Alcatel-LucentProprietary 37See notice on first page
1.23 Quality of Service
Test Objectives:To assess the QoS performance of an LTE UE with VoIP and HTTPapplications in various multi-UE loading scenarios
Test Description:
VoIP QoS Test:This test will be executed with the test UE in VoIP mode. Tests will be executed under
different loading conditions. The loading UEs will be executing BE traffic. The number ofloading UEs will be varied from 3 to 7 and they will be placed at a Near Cell location.
HTTP QoS Test:
This test will be executed with the test UE in GBR mode running HTTP application. Testswill be executed under different loading conditions. The loading UEs will be executing BE
traffic. The number of loading UEs will be varied from 3 to 7 and they will be placed at a
Near Cell location.
Procedure:
GBR VOIP QoS Test
1. Configure test UEs MAC Downlink Scheduler with the following settings:a. VoIP Flag = Trueb. Initial MCS: 5 (QPSK, Code Rate 0.438)c. HARQ Max Number of Transmissions: 1
2. Configure test UE with the following UL/DL TFT information:a. Remote IP address/subnet maskb. Port Range for RTP/RTCP: 1000010010c. Protocol: UDP
3. Configure test UE with the following QoS Information:a. QoS Class Identifier (QCI): 1b. UL/DL MBR: (not used)c. UL/DL GBR: (not used)
4. Initiate a call from UE1 to IxChariot to simulate voice traffic. Incrementally add UEswith best effort IP transfers until you have 7 active UEs.
5. Repeat steps 14 for the Mid Cell and the Cell Edge geometries
GBR HTTP QoS Test6. Repeat steps 1-3 and activate IxChariot. Initiate an HTTP session at the Near Celllocation. Incrementally add UEs with best effort IP transfers until you have 7 active
UEs. Repeat the tests for both the Mid Cell and the Cell Edge locations
Key Metrics (per UE):1. Resource Utilization2. Transport Format Distribution
-
8/10/2019 KPI Optimization Test Plan for LTE
38/42
Alcatel-LucentProprietary 38See notice on first page
3. Latency4. Mean Opinion Score5. Physical Layer Throughput6. Application Layer Throughput {UDP}7. Initial Block Error Rates
8. Residual Block Error Rates
-
8/10/2019 KPI Optimization Test Plan for LTE
39/42
Alcatel-LucentProprietary 39See notice on first page
1.24 Coverage Testing
Test Objectives:Validate the coverage for single-UE tests on the pre-selected drive route.
Tests will be conducted under interfered and non-interfered conditions for UDP application.
Test Description: Test UE will be driven according to the pre-selected drive route from
Near Cell (NC) to Cell Edge (CE) until call drops. Uplink (UL) interference is generated byplacing loading UEs in neighboring cells at pre-selected locations. DLLS will be used to
generate DL interference in the neighboring cells.
Interference of 100% will result in an Interference over Thermal (IoT) of TBD dB in the
target device (i.e., cell for UL and UE for DL), while interference of 50% will result in anIoT of TBD dB in the target device. Both UL and DL physical-layer data rate and Signal to
Interference plus Noise Ratio (SINR) will be measured and signaling will be recorded in the
tests.
Procedure:
UL Tests
1. Set SINR target in neighboring cells to control the power of loading UEs
2. Place loading UEs in neighboring cells at pre-selected locations to generate desired IoT in the
target cell3. Make a UDP Best Effort (BE) call to the target cell on the test UE and measure both UL and DL
physical-layer data rates, SINR, and record the signaling messages
4. Place the test UE in a van and drive on a pre-selected route from NC to CE of the target cell untilthe call drops
5. Repeat steps 1 to 4 for each interference condition
DL Tests
1. Set DLLS in neighboring cells to generate desired DL interference levels
2. Make a UDP BE call on the test UE from the target cell and measure both UL and DL physical-
layer data rates, SINR, and record the signaling messages
3. Place the test UE in a van and drive on a pre-selected route from NC to CE of the target cell until
the call drops4. Repeat steps 1 to 3 for each interference condition
Key Metrics:1. Physical Layer Throughput2. SINR
3. PDCCH error rate
-
8/10/2019 KPI Optimization Test Plan for LTE
40/42
Alcatel-LucentProprietary 40See notice on first page
1.25 Handover
Test Objectives:Evaluate handover performance in following scenarios:- Intra-Site (different sectors within one eNodeB)
- Inter-Site (different eNodeBs)
- Loaded and Unloaded Destination eNodeBs
Test Description:
Test UE will be driven along two routes:
Handover Route comprising of intra and inter eNodeB handovers between 3-4sectors. On this route, additional UEs will be stationed in each sector along the
drive route. These UEs will load both downlink and uplink of their respective
sectors with BE traffic.
Cluster Route comprising of intra and inter eNodeB handovers in the entire 10
eNodeB Cluster. Only DL loading will be generated via DLLS on this route.
Non-guaranteed and guaranteed Quality of Service (QoS) tests will be conducted via Best
Effort (BE) and Guaranteed Bit Rate (GBR) QoS classes, respectively.
Application performance will be measured quantitatively and subjectively. Whilequantitative measurements are throughput (physical-layer data rate) and latency,
subjective performance will be based on users perception of the application. For
example, quality of a Voice over IP (VoIP) call can be either clear,choppy but
audible,or not audible.
Both UL and DL SINR (Signal to Interference plus Noise Ratio) will be measured andSignaling messages will be recorded in the tests.
Key Metrics:
1. Physical Layer Throughput2. SINR3. Latency
-
8/10/2019 KPI Optimization Test Plan for LTE
41/42
Alcatel-LucentProprietary 41See notice on first page
1.26 V-Pol vs. Cross-Pol
Test Objectives:Compare the performance of Vertically and Cross polarized antennaconfigurations
Test Description:
Settings common to all tests:- Open-Loop Spatial multiplexing (OLSM) mode- 100% loading on DL of neighbor cells- Stationary
A single UE will be used for all the tests. Tests will be executed in NC, MC and EC locationswith UDP and FTP applications on both DL and UL.
For the V-pol (vertically polarized) tests, both the eNodeB and UE antennas will be set to the V-
pol configurations. Similarly, for the X-pol (cross polarized) tests, both the eNodeB and UEantennas will be set to the X-pol configurations.
DLLS will be used to load the DL of the cells neighboring the target cell. Loading of 100% willbe used for all the tests.
Test Setup:1. One test van will be used with rooftop mounted antennas
Procedure:
1. Set the eNB and UE to V-pol configuration2. Run DL/UL UDP/FTP tests in each of NC, MC and EC locations3. Repeat for X-pol configuration setting
Key Metrics:
1. Channel Correlation Statistics at UE for DL tests2. Physical Layer Throughput3. Application Layer Throughput4. Initial Block Error Rates5. Residual Block Error Rates6. Scheduled Transport Format distribution
-
8/10/2019 KPI Optimization Test Plan for LTE
42/42
Appendix A: Performance Metrics
The following metrics will be collected during the trial execution phase. The list shallinclude (but not necessarily limited to):
Air Interfaceo UE Tx powero RSSI
o SINRo BLERo Retransmission statistics (HARQ and RLC)
o Transport Format
o Number of resource blocks (DL/UL)o Channel rank statisticso MIMO mode (Tx diversity or Spatial Multiplexing)
o Serving sector
o Location (GPS)o UE Velocity
Throughputo Individual user throughput and aggregated sector throughputo UDP individual user throughput and aggregated sector throughput
o TCP individual user throughput and aggregated sector throughput
o User statistics (peak rates, average rates, standard deviations)
Latency
o U-plane latency
o Connection set up timeso Handover interruption time within the same site and across different sites