Nokia Siemens LTE E2E Field Network Performance

8/13/2019 Nokia Siemens LTE E2E Field Network Performance

http://slidepdf.com/reader/full/nokia-siemens-lte-e2e-field-network-performance 1/79

Nokia Siemens Networks LTE

Radio Access, Rel. RL15,

Operating Documentation Pre-release, Issue 01

LTE E2E Field Network Performance -

Definitions of Key Performance Indicators

DN0972412

Issue 01 DRAFT APPROVED

Approval Date 2010-06-22



2 DN0972412


LTE E2E Field Network Performance - Definitions of Key Performance Indicators

Id:0900d805807a32c5

The information in this document is subject to change without notice and describes only the

product defined in the introduction of this documentation. This documentation is intended for the

use of Nokia Siemens Networks customers only for the purposes of the agreement under whichthe document is submitted, and no part of it may be used, reproduced, modified or transmitted

in any form or means without the prior written permission of Nokia Siemens Networks. The

documentation has been prepared to be used by professional and properly trained personnel,

and the customer assumes full responsibility when using it. Nokia Siemens Networks welcomes

customer comments as part of the process of continuous development and improvement of the

documentation.

The information or statements given in this documentation concerning the suitability, capacity,

or performance of the mentioned hardware or software products are given "as is" and all liability

arising in connection with such hardware or software products shall be defined conclusively and

finally in a separate agreement between Nokia Siemens Networks and the customer. However,

Nokia Siemens Networks has made all reasonable efforts to ensure that the instructions

contained in the document are adequate and free of material errors and omissions. Nokia

Siemens Networks will, if deemed necessary by Nokia Siemens Networks, explain issues which

may not be covered by the document.

Nokia Siemens Networks will correct errors in this documentation as soon as possible. IN NO

EVENT WILL Nokia Siemens Networks BE LIABLE FOR ERRORS IN THIS DOCUMENTA-

TION OR FOR ANY DAMAGES, INCLUDING BUT NOT LIMITED TO SPECIAL, DIRECT, INDI-

RECT, INCIDENTAL OR CONSEQUENTIAL OR ANY LOSSES, SUCH AS BUT NOT LIMITED

TO LOSS OF PROFIT, REVENUE, BUSINESS INTERRUPTION, BUSINESS OPPORTUNITY

OR DATA,THAT MAY ARISE FROM THE USE OF THIS DOCUMENT OR THE INFORMATION

IN IT.

This documentation and the product it describes are considered protected by copyrights and

other intellectual property rights according to the applicable laws.

The wave logo is a trademark of Nokia Siemens Networks Oy. Nokia is a registered trademark

of Nokia Corporation. Siemens is a registered trademark of Siemens AG.

Other product names mentioned in this document may be trademarks of their respectiveowners, and they are mentioned for identification purposes only.

Copyright © Nokia Siemens Networks 2010. All rights reserved

f Important Notice on Product SafetyElevated voltages are inevitably present at specific points in this electrical equipment.

Some of the parts may also have elevated operating temperatures.

Non-observance of these conditions and the safety instructions can result in personal

injury or in property damage.

Therefore, only trained and qualified personnel may install and maintain the system.

The system complies with the standard EN 60950 / IEC 60950. All equipment connected

has to comply with the applicable safety standards.

The same text in German:

Wichtiger Hinweis zur Produktsicherheit

In elektrischen Anlagen stehen zwangsläufig bestimmte Teile der Geräte unter Span-

nung. Einige Teile können auch eine hohe Betriebstemperatur aufweisen.

Eine Nichtbeachtung dieser Situation und der Warnungshinweise kann zu Körperverlet-

zungen und Sachschäden führen.

Deshalb wird vorausgesetzt, dass nur geschultes und qualifiziertes Personal die

Anlagen installiert und wartet.

Das System entspricht den Anforderungen der EN 60950 / IEC 60950. Angeschlossene

Geräte müssen die zutreffenden Sicherheitsbestimmungen erfüllen.



DN0972412


3

LTE E2E Field Network Performance - Definitions ofKey Performance Indicators

Id:0900d805807a32c5

Table of ContentsThis document has 79 pages.

Summary of Changes . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 7

1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 9

2 Measurement Methods and Reference Conditions . . . . . . . . . . . . . . . . 11

2.1 Introduction . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 11

2.2 Reference Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.1 Distinguished Cell Positions. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 12

2.2.2 Channel Models. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 13

2.2.3 Template for the Description of Reference Conditions . . . . . . . . . . . . . 14

2.3 Measurement Scenarios . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3.1 Throughput Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

2.3.1.1 Cell Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 17

2.3.1.2 Peak User Data Rate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 19

2.3.2 Latency Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.2.1 Round Trip Time Measurement . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 20

2.3.2.2 Signaling and Media Delay Measurements . . . . . . . . . . . . . . . . . . . . . . 21

2.3.2.3 Reference Conditions for Latency Measurements. . . . . . . . . . . . . . . . . 22

2.3.3 Service Accessibility Measurements . . . . . . . . . . . . . . . . . . . . . . . . . . . 23

2.3.4 Reliability Measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.3.4.1 Transport Error Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 25

2.3.4.2 Service Drop Rates . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 27

2.3.5 Mobility (Handover) Measurements. . . . . . . . . . . . . . . . . . . . . . . . . . . . 282.4 Measurement Setup . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.4.1 Field Network Cluster. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 30

2.4.2 Trial Lab. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.4.3 Measurement Data Collection . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.4.3.1 Subscriber and Equipment Traces. . . . . . . . . . . . . . . . . . . . . . . . . . . . . 31

2.4.3.2 PM Counters . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 32

2.5 Measurement Reporting . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 33

3 Definitions of Key Performance Indicators (KPIs) . . . . . . . . . . . . . . . . . 34

3.1 Overview . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 34

3.1.1 Application Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 35

3.1.2 LTE Network Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.1.2.1 LTE E2E Network Service . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 36

3.1.2.2 Radio Bearer Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 37

3.2 KPI Definition Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38

3.3 Application Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.3.1 Data Download / Upload (FTP) Service KPIs . . . . . . . . . . . . . . . . . . . . 39

3.3.1.1 (FTP) Service Accessibility Ratio. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 39

3.3.1.2 (FTP) Completed Session Ratio . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 40

3.3.1.3 (FTP) Session Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 41

3.3.1.4 (FTP) Service Access Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 42

3.3.1.5 (FTP) User Data Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 453.4 LTE Network Services . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46



4 DN0972412



Id:0900d805807a32c5

3.4.1 LTE E2E Network KPIs. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.4.1.1 Attach Time. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 46

3.4.1.2 Detach Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 49

3.4.1.3 Attach Success Rate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 513.4.1.4 Service Request Time (EPS), UE Initiated . . . . . . . . . . . . . . . . . . . . . . . 53

3.4.1.5 Service Request (EPS) Time, Network Initiated . . . . . . . . . . . . . . . . . . . 57

3.4.1.6 Service Request (EPS) Success Rate . . . . . . . . . . . . . . . . . . . . . . . . . . 58

3.4.1.7 Service (EPS Bearer) Drop Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 60

3.4.1.8 Handover Procedure Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 62

3.4.1.9 Handover Success Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 65

3.4.1.10 Paging Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 67

3.4.1.11 Paging Failure Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 69

3.4.1.12 (LTE) Round Trip Time . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 70

3.4.1.13 (LTE) User Data Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 71

3.4.1.14 (LTE) Packet Loss Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 723.4.1.15 (LTE) Service Interrupt Time (HO) . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 73

3.4.2 Radio Bearer KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.4.2.1 (RB) Packet Loss Rate . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 74

3.4.2.2 (RB) User Data Rate. . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 75

3.4.2.3 Cell Throughput . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 76

4 References . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 78



DN0972412


5


Id:0900d805807a32c5

List of FiguresFigure 1 LTE Bearer Service Architecture . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 10

Figure 2 Distribution of Users for Cell Throughput Measurements . . . . . . . . . . . 17

Figure 3 Reference Cluster for Field Performance Verification Tests . . . . . . . . . 30

Figure 4 KPIs for Packet Switched Data Services . . . . . . . . . . . . . . . . . . . . . . . . 43

Figure 5 Attach Procedure w/ Initial EPS Bearer Establishment (3GPP 23.401). 48

Figure 6 UE-Initiated Detach Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 50

Figure 7 EPS Bearer Setup Procedure, Part 1 . . . . . . . . . . . . . . . . . . . . . . . . . . 55

Figure 8 EPS Bearer Setup Procedure, Part 2 . . . . . . . . . . . . . . . . . . . . . . . . . . 56

Figure 9 Handover Procedure: Intra MME/S-GW, Inter eNB via X2 . . . . . . . . . . 63

Figure 10 Handover Procedure: Intra MME/S-GW, Inter eNB via S1 . . . . . . . . . . 64

Figure 11 Paging Procedure . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 68



6 DN0972412



Id:0900d805807a32c5

List of TablesTable 1 Channel Models and Doppler Frequencies . . . . . . . . . . . . . . . . . . . . . . 13

Table 2 Description of Reference Conditions . . . . . . . . . . . . . . . . . . . . . . . . . . . 15

Table 3 KPI Categories . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 16

Table 4 Reference Conditions for Cell Throughput Measurements . . . . . . . . . . 18

Table 5 Reference Conditions for User Data Rate Measurements . . . . . . . . . . . 20

Table 6 Reference Conditions for Latency Measurements . . . . . . . . . . . . . . . . . 23

Table 7 Reference Conditions for Success Rate Measurements . . . . . . . . . . . . 24

Table 8 Reference Conditions for Packet Loss Rate Measurements . . . . . . . . . 26

Table 9 Reference Conditions for Service Drop Rate Measurements . . . . . . . . 28

Table 10 Reference Conditions for Handover Measurements . . . . . . . . . . . . . . . 29

Table 11 Overview of Application Service KPIs . . . . . . . . . . . . . . . . . . . . . . . . . . 35

Table 12 Overview of LTE E2E Network Service KPIs . . . . . . . . . . . . . . . . . . . . . 36

Table 13 Overview of LTE Radio Bearer Service KPIs . . . . . . . . . . . . . . . . . . . . 37Table 14 KPI Definition Template . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . . 38



DN0972412


7


Summary of Changes

Id:0900d805807a32af

Summary of Changes

Issue History

Issue

number

Date of issue Reason for update

01 DRAFT 2010-06-14 First issue for RL15 pre-release



8 DN0972412



Id:0900d805807a32af

Summary of Changes



DN0972412


9


Introduction

Id:0900d805807a32b1

1 IntroductionThe current document provides the following information:

• Definitions of End-to-End Key Performance Indicators (E2E KPIs) for Long-TermEvolution (LTE)

• Descriptions of measurement methods for field and lab trials

KPIs are basic network quality indicators for offer processing and field acceptance tests.

The KPIs specified in this document are E2E related KPIs; they help network operators

and Nokia Siemens Networks to determine the quality of deployed LTE networks.

In order to prove that the performance of an operating network matches the targeted

quality, field acceptance tests are carried out. As multiple factors may influence the per-

formance of an operational network, detailed conditions, under which network KPIs are

valid, must be specified and agreed upon. These include system performance, network

planning and dimensioning, measurement campaign planning and parameterization.

These conditions will be reffered to as "reference conditions" in the sequel.

LTE system architecture was designed to meet the following major goals. These are

also the constant focus of lab and field performance tests (for more details, see Refer-

ences 7. [3GPP25.913]):

• Improved system capacity & coverage

• High user data rates (peak upload and download rates)

• Reduced latency (one-way delay and RTT)

• Simplified architecture, IP based transport infrastructure

• Superior user experience

• Seamless connection to legacy networks (GSM, UMTS, 3GPP2/CDMA)

These improvements can be assessed with KPI measurements as defined in this docu-

ment.

The KPIs are defined for different service levels (see References 20. [3GPP 36.300],

and Figure 1 LTE Bearer Service Architecture):

a) Application Services

b) LTE E2E Network Service

c) Radio Bearer (RB) Service

d) IP based LTE Bearer Services: X2, S1, S5/S8 and External

The KPIs of group B (LTE E2E Network Service) characterize the quality of the LTE

network as IP transport service for applications. They are in the center of field perfor-

mance investigations, because they characterize the overall LTE network performance.

The Application Service KPIs of group A (Application Services) show the user-perceived

quality of application services as impacted by the LTE network. The KPIs of this group

are service specific. The following services are of common interest of mobile network

operators:

• Web Browsing

• Data Upload / Download

• Audio Video Streaming

• Conversational Audio Video (VoIP and Video Telephony)

This paper concentrates on the Data Upload / Download (File Transfer Protocol

(FTP)/Transmission Control Protocol (TCP)) application service, because it is the

http://general_glossary.pdf/















10 DN0972412



Id:0900d805807a32b1

Introduction

support of this service that stands in focus of the first commercial NSN LTE product

release.

In addition, KPIs for the radio link (RB Service, group C - Radio Bearer Service) are also

specified, because the radio link is the most critical component of the E2E transportpath. The other bearers, the E-UTRAN Radio Access Bearer (E-RAB) and Evolved

Packet System (EPS) bearers are composed of the RB Service and of the IP based

bearer services (of group D - IP based LTE Bearer Services: X2, S1, S5/S8 and Exter-

nal).

Monitoring the performance of all bearer services is necessary to fully understand the

E2E transport behavior of the LTE network and to optimize the end-user perceived

quality of application services running on top of it.

Figure 1 LTE Bearer Service Architecture

While the KPI definitions (see Definitions of Key Performance Indicators (KPIs)) are

independent of LTE product releases, Measurement Scenarios focuses on the first

product release in order to restrict the number of possible scenario combinations (e.g.

by not considering scenarios with Guaranteed Bit Rate (GBR) bearers).









DN0972412


11


Measurement Methods and Reference Conditions

Id:0900d805807a32b3

2 Measurement Methods and Reference Condi-

tions

2.1 Introduction

The measurement method to be applied for field performance verification depends on

the specifics of the measured Key Performance Indicator (KPI). End-user perceived

KPIs are measured differently from network performance KPIs, and the conditions for

delay measurements are different from those for accessibility KPIs. For this reason, this

Section discusses measurement methods according to KPI categories.

Since the end-user experience of application services is mostly dependent on through-

put and latency, these aspects together with Handover (HO) scenarios constitute the

core part of field performance verification measurements.

While the scope of KPI definitions follows the performance goals of the LTE system (fordetails see References 7. [3GPP25.913]), the selected methods and conditions of field

performance tests have to ensure that typical conditions in operational networks are

covered. Additionally, the measurement conditions also include relevant impairment

factors of the radio link, because the radio link has the most influence on E2E perfor-

mance. Especially, those characteristics of the Orthogonal Frequency-Division Multi-

plexing (OFDM) encoding technique, which have side effects for radio network

performance are considered in the selection of conditions, e.g.:

• Sensitivity to Doppler shift

• Sensitivity to frequency synchronization problems

• Sensitivity to multipath effects and cell area (urban, sub-urban, rural)

• Dependency on used bandwidth and frequency range

• Dependency on used multiple antenna techniques - Multiple Inputs Multiple Outputs

(MIMO) (transmit diversity, spatial multiplexing)

These aspects have a major bearing on the design of measurement scenarios. For

example, investigating the impact of Doppler shift on network performance, e.g. on user

and cell data rates, on delay and packet loss KPIs requires that high speed movement

becomes part of the measurement scenario. Another aspect would be the multipath

reception, which decreases the impact of the Doppler shift. Thus, the measurement

setup should also include space diversity if worst-case conditions are being discussed

measurements with these conditions have to show, how the investigated KPI figure

depends of the velocity and terminal category of the receiving user.

Some of the measurement scenarios might be much too costly to implement in live

network environments, or it may be impossible to ensure the desired radio conditions

during the whole duration of the test. In such cases, lab trials with simulated radio links

are acceptable as replacement of live network tests.

The combination of all different options for reference conditions and measurement sce-

narios would result in a very large number of test cases, which is simply not feasible. It

will be the task of test specifications to select options and scenarios out of the outlined

possibilities for a given LTE performance verification campaign considering the avail-

able resources and time constraints for test execution.











12 DN0972412



Id:0900d805807a33b4

2.2 Reference Conditions

Performance figures can only be interpreted correctly if the exact conditions under which

they have been acquired are known. This chapter describes the "reference conditions"

under which LTE field performance tests should be executed. They are formulated onan abstract level.

2.2.1 Distinguished Cell Positions

Operators are keen to know the distribution of throughput and latency KPI figures (e.g.

peak user data rate, cell UL/DL throughput, network delay, etc.) across the cells of a "ref-

erence cluster". However, it is impossible or would be associated with prohibitive high

costs to measure these KPIs in a live network environment in so many cell positions that

a Cumulative Distribution Function (CDF) can be calculated for 5%, 50%, or 95% of the

cell area. For this reason, it is proposed to replace the distribution function by KPI figures

measured on three characteristic positions of the cell (in the sequel, the term "cell" will

be used to denote both omni-directional and directional cells):

a) Very Good radio conditions (line of sight close to cell center within 50m, low path

loss)

b) Medium radio conditions (BTS distance of ~1/2 cell radius, channel with medium

noise conditions, medium path losses)

c) Cell Edge conditions (cell overlapping area, receive power from neighbor cells at

about the same level).

The radio conditions are formulated in terms of Signal to Interference and Noise Ratio

(SINR) and receive power levels. The measured KPI figures in Good, Medium and Cell

Edge positions replace their distribution across the LTE cell and will be considered as

their Peak, Average and Minimum values, respectively. The three variants are given bymean and 95% values calculated from all measurement samples, where the number of

samples is to be derived from the required confidence level.

If no suitable cell positions are available in the reference cluster, the tests can be

executed in the laboratory, where the radio conditions are to be simulated. If automated

test execution is possible, even the calculation of CDF function values at 5%, 50% and

95% distribution is an option.

This shows that often the full set of results can only be produced by combined field and

lab trials. In certain cases, even simulations might become necessary to accomplish

(incomplete) measurement results. For this reason, it is best practise to measure all E2E

field network KPIs in vendor lab before going into field tests with the operator.

The description of test conditions (see Table 2 Description of Reference Conditions, row

"Setup") specifies which test methods are recommended.







DN0972412


13


Id:0900d805807a33b4

2.2.2 Channel Models

Several factors of the radio link have influence on the E2E performance of application,

or LTE network services. Their importance is dependent on the category of the KPI to

be measured. The following parameters are checked for each scenario:

• UE category

• Antenna systems on UE and eNB

• Position and distribution of terminals in the cell, terminal movement

• Load conditions in the measured cell (UL/DL, data volume)

• Receive power level, interference and noise conditions, diversity and fading

• Cell area, or clutter type (urban, sub-urban, rural)

In order to reduce the number of possible combinations, mobile terminal speed and

radio propagation conditions for different environments and distances were combined in

"channel models". The relevant channel models for LTE are described in References 30.

Annex B of [3GPP36.521-1]. The following three channel models are defined: • EPA - Extended Pedestrian A

• EVA - Extended Vehicular A

• ETU - Extended Typical Urban

In addition, 3GPP proposes some combinations of channel models and Doppler fre-

quencies, which are complemented in Table 1 Channel Models and Doppler Frequen-

cies with scenarios for

• stationary UEs (EPA0, EVA0 and ETU0), and

• high speed train (ETU250)

The high speed train scenario with app. 250 km/h uses a carrier frequency of 900 MHz

(former GSM band) since it is assumed that initial LTE deployments will use this fre-quency in rural areas. If the assumption is not true for a given customer trial, the Doppler

frequency will change proportionally to the applied carrier frequency (500Hz @ 1.8 GHz,

583 Hz @ 2.1 GHz, 722 Hz @ 2.6 GHz). The current document uses only these models

as reference models.

Model Source Maximum Doppler Frequency

EPA0 0 Hz (stationary UE)

EVA0 0 Hz (stationary UE)

ETU0 0 Hz (stationary UE)

EPA5 3GPP36.521-1, Annex B; see 30. 5Hz

EVA5 3GPP36.521-1, Annex B; see 30. 5Hz

EVA70 3GPP36.521-1, Annex B; see 30. 70Hz

ETU70 3GPP36.521-1, Annex B; see 30. 70Hz

ETU250 250 Hz

(with 900 MHz carrier frequency)

ETU300 3GPP36.521-1, Annex B; see 30. 300Hz

Table 1 Channel Models and Doppler Frequencies



14 DN0972412



Id:0900d805807a33b4

Channel model conditions can be set in lab trials with simulated radio links precisely, but

they cannot be ensured in field network tests. In field trials, radio conditions with best

suit to the channel model are used to calibrate trial lab results.

2.2.3 Template for the Description of Reference Conditions

Table 2 Description of Reference Conditions includes the channel model together with

other parameters to summarize the essential conditions for measurement scenarios.

The column "Reference Condition" gives textual descriptions and possible settings as

abstract values only.

The meaning of rows is as follows:

• "Antenna" identifies the number, type and modulation of antennas used on the UE

and on the eNB side. The reference UE configuration comprises two receive

antennas in DL and one antenna in UL. (However, it may not be freely selectable in

reference clusters of operational networks.) • "The "Application" parameter identifies the test application as a real application, or

an equivalent replacement (test script), which allows the collection of information

necessary to calculate / extrapolate the KPI figure.

• The "Setup" parameter identifies the environment, where the KPI can best be mea-

sured. It is not always the reference cluster of the live network. Some conditions, e.g.

SINR are easier to set in the lab.

– The "Reference Cluster" denotes a selected area of the operator´s live network

performance verification tests. In the cluster typical load conditions apply, as

generated by real users -stationary or mobile - and the radio conditions are

determined by cell location (rural, or city area, high, or low buildings, etc.). The

conditions are given and cannot be modified during the trial.

– The "Trial Lab" is either a vendor or operator lab used for executing performance

tests, where exact measurement conditions can be ensured. The lab allows the

background traffic mix, load levels, and radio conditions to be precisely set. The

simulation of mobile users of high velocity is also possible.

– "Simulation" results are applied where field and even lab trials are expensive or

cannot be executed in the given time or budget.

• "Concurrent Load" describes the load which is to be generated in addition (i.e. back-

ground traffic) to the traffic produced by active test terminals in the measured cell.

Load conditions in neighbor cells are the operational load in field network clusters,

and are set to medium load in the lab.

• "Drive test "Routes" are defined for mobile terminals in live LTE environments. In

trial labs, terminal movement and hand-over are simulated (by varying the channel

conditions).

• "Backhaul Capacity" defines the necessary throughputs of transport links if their

values deviate from default and are important for test execution, e.g. to prevent a

transport link becoming bottleneck in user data rate, or cell throughput measure-

ments. It also defines other parameters, like Packet Delay (PD), Packet Delay Vari-

ation (PDV), Packet Loss Rate (PLR) and their expected values. These parameters

are observed in field trials, and are set to their maximal allowed values in the lab.







DN0972412


15


Id:0900d805807a33b4

The number of test executions (sample size) is not defined as a parameter in the tem-

plate, but it is to be defined for each measurement scenario so that the calculated KPI

figure has the required statistical confidence.

Parameter Reference Condition

Terminal (UE)

Type Mobile or smart phone, laptop with LTE card, or w/

modem

Category UE categories 1-5 according to [3GPP36.306]; see 21.

Antenna Type: SISO, MIMO1*2, MIMO2*2, MIMO4*2, etc.

Modulation: QPSK, 16QAM, 64QAM

Position Good, medium and cell edge (for each terminal)

Distribution Equal, or non-equal distribution of "1 to n" UEs

Network Environment

Channel Model EPA, EVA, ETU with different Doppler frequencies

Backhaul Capacity Capacities of X2, S1-U, S1-MME transport l inks.

QoS Configuration Link weight and priority assigned to the flow(s)

Concurrent Load Unloaded, or concurrent UL/DL data volumes

Application Application service or traffic generation for LTE E2E,

EPS, E-UTRAN, EPC bearers.

Traces Trace points & data to be collected

Setup Trial Lab, Reference Cluster, or Simulation

Route In reference cluster, drive test route description.

Table 2 Description of Reference Conditions



16 DN0972412



Id:0900d805807a33b6

2.3 Measurement Scenarios

Measurement Scenarios describes the measurement methods and recommended sce-

narios according to KPI categories summarized in Table 3 KPI Categories. The first

column indicates the respective QoS terminology defined by the ITU [ITU-T.E800], andreferenced by 3GPP in References 12. [3GPP32.410] and 17. [3GPP32.450] for LTE

related KPIs. The third and fourth columns indicate the selected terminology and expla-

nation for its usage in the current paper.

Since the KPI category is part of all KPI definitions (see KPI Definition Template ), it is

easy to lookup the corresponding measurement method to each KPI.

2.3.1 Throughput Measurements

Throughput is the primary metric for the characterization of the LTE radio technology

(Orthogonal Frequency Division Multiple Access (OFDMA) in DL, Single-Carrier Fre-

quency Division Multiple Access (SC-FDMA) in UL). It shows the data rate that can be

provided to users among different radio conditions. The throughput of an LTE cell is

expressed as a function of the number of concurrently active users and the amount of

aggregate traffic that the cell can still support. The throughput the LTE system can offer

depends on many factors:

• Channel environment (e.g. stationary or mobile, speed) and fading conditions.

• Reception conditions impaired by traffic load levels, and by interference between the

cells, in short by the user's SINR.

• Network layout, type of antenna.

• Position of users in the cell (implies e.g. path loss and fading).

ITU QoS

Category

3GPPKPI

Category

Terminology

Used

Meaningfor LTE Field Network

Performance

Accessibility Accessibility Accessibility Service accessibility, e.g. Attach, or

EPS Bearer Setup Request success

rates.

Reliabil ity Retainability Reliability Once established, the reliability of theservice, e.g. VoIP call, or EPS bearer

drop rates.

Integrity Throughput,

Latency

Throughput,

Latency

Promise to deliver the service with a

certain quality, level of conformance.

E.g. cell throughput, or user data rates,

packet delay.

Availability Availability not used Relationship between service up and

down times, e.g. cell, or LTE network

availability

not defined Mobility Mobility Different mobility related metrics, e.g.

hand-over success or drop rates, laten-

cies.

Table 3 KPI Categories







DN0972412


17


Id:0900d805807a33b6

Thus, it is obvious that the location of the mobile terminal in the cell will strongly influ-

ences the data rate that can be experienced by the user. For this reason, the capacity

of an LTE cell is characterized by two distinguished metrics, one viewing capacity from

the operator's point of view (How many users can be supported by the cell?), the other

from the end-user perspective:

• Cell Throughput, and

• Peak User Data Rate

2.3.1.1 Cell Throughput

The metric "Cell Throughput" shows the sustainable aggregate cell capacity available to

a number of "n" users. In order to approximate a typical operating environment, the

users are distributed uniformly in the cell and use a typical mix of applications concur-

rently. The Figure 2 Distribution of Users for Cell Throughput Measurements shows how

users are arranged for cell throughput measurements. Each user position is character-

ized by different combinations of path loss, receive signal, and neighbor cell interference

strengths.

Figure 2 Distribution of Users for Cell Throughput Measurements

The measurement is done with stationary terminals to avoid the impact of movement

and Hand-Over (HO). The latter implies that for those mobiles that are located at the celledge, HO must be suppressed on the UE or by the network. The proposed reference

conditions for cell throughput measurements are summarized in Table 4 Reference

Conditions for Cell Throughput Measurements.



18 DN0972412



Id:0900d805807a33b6

The default backhaul capacity of S1-U transport links are to be controlled and modified

if necessary to prevent this link becoming a bottleneck during the measurement. The

modified value should become default for subsequent measurements.

Cell throughput measurements should be executed with an increasing number of UEs

starting with 1. While the first test case corresponds to the peak user data rate measure-

ment, the subsequent ones illustrate the dependency of this KPI on the number of active


Terminal

UE Type Laptop with LTE card

UE Category Different categories, KPI specific

Antenna Types: SISO, MIMO 1*2, 2*2, …;

Position Good, medium, cell edge (see Distinguished Cell Position)

Distribution "1-n" terminals, equal distribution in the cell

Network Environment

Channel Model Stationary outdoor - EPA0, EVA0, ETU0 with different SINRs

Backhaul Capacity Capacities of X2, S1-U, S1-MME transport links.

QoS Configuration Non-real-time QoS, equal weights and priorities

Concurrent Load Unloaded (no concurrent load)

Application Data upload / download (FTP); separate UL/DL and concurrent

transmission.

Traces UE, Uu, eNB, S1-U, … FTP Server

Setup Reference cluster in live network

Route N.a.

Table 4 Reference Conditions for Cell Throughput Measurements



DN0972412


19


Id:0900d805807a33b6

users. The results (cell throughput and peak user data rate) are to be illustrated together

in a diagram like the example below:

2.3.1.2 Peak User Data Rate

The "Peak User Data Rate" metric describes the data capacity that is available to one

user in a cell. Data rate is the most important single factor which influences the end-user

experience of non-real-time application services, like web browsing, email, File Transfer

Protocol (FTP) up/downloads, and interactive gaming, due to its impact on (signaling

and data) transfer delays. The goal of field performance measurements is to investigate

the sensitivity of this metric to LTE impairment factors. The realistically achievable data

rates under concurrent load (i.e. with concurrently active users in the cell) in stationary

case are shown with cell throughput measurements as described in Cell Throughput.

Since the available peak user data rate is influenced by any or all of the following con-

ditions,

• Channel environment (mobility, stationary)

• Radio conditions (signal power, path loss, SINR)

• Aggregate cell load (i.e. intra-cell and inter-cell traffic)

• Restriction of user data rates (e.g. by terminal category)

• Link sharing weights (Quality of Service (QoS) configuration)

• Backhaul capacity

the measurement scenarios include variations of these parameters. User data rates are

also measured in cell edge conditions, even including hand-over. The reference condi-







20 DN0972412



Id:0900d805807a33b6

tions for user data rate tests are summarized in Table 5 Reference Conditions for User

Data Rate Measurements.

Ideally, the results of field or lab trial measurements of this KPI would be a full distribu-

tion of Peak User Data Rate values (a Cumulative Distribution Function (CDF)) across

the cell. For practical reasons, however, it is sufficient to produce the values in selected

cell positions as described in Distinguished Cell Position.

Radio positions (good, average and poor SINR) that are best suited to the channel

model of stationary users is selected. Mobility tests with up to 50 km/h can be executed

in field reference clusters, but higher speeds, e.g. high speed train with ~350 km/h are

to be executed in simulated a lab environment.

2.3.2 Latency Measurements

2.3.2.1 Round Trip Time Measurement

The benchmark measurement for finding the E2E latency of LTE access networks for

data applications is the Round Trip Time (RTT) measurement. RTT is measured with

the Ping application of the UE´s operating system or with a comparable measurement

tool. It records the time difference between sending an Internet Control Message

Protocol (ICMP) Echo Request to an IP host, and the reception of the corresponding

ICMP Echo Reply message.

The measurement can be executed between a mobile terminal and an IP host, which is

configured to respond to ICMP Echo Requests. This way, round trip delays can be

measured between UE - eNB (LTE Uu), UE - S/P-GW (E-UTRAN Radio Access

Parameter Reference ConditionTerminal

UE Type Smart phone and laptop with LTE card

UE Category Corresponding UE categories

Antenna Types: SISO, MIMO 1*2, 2*2, etc.

Position Good, medium, and cell edge (see Distinguished Cell Position)

Distribution One single terminal

Network Environment

Channel Model EPA, EVA stationary and mobile w/ 3, 50 km/h


QoS Configuration Non-real-time QoS, equal weights and priorities

Concurrent Load Unloaded

Application FTP upload, download; separate

Traces UE, eNB, FTP Server; Uu, S1-U, S1-MME, X1, ...

Setup Reference cluster in live network

Route Drive test route passing good and medium cell positions, and crossing cell borders.

Table 5 Reference Conditions for User Data Rate Measurements









DN0972412


21


Id:0900d805807a33b6

Bearer(E-RAB), EPS) and UE - AS (LTE E2E), respectively. The measurement can

(and should) be executed in both directions.

The Ping application sends successive ICMP Echo Requests

• either upon reception of the ICMP Echo Reply to the previous request,

• or after a predefined period of time (in the range of ~100ms to 1s)

This is done to ensure that only one Ping request/response is under way at any given

time. This way, the delay time will have no bias caused by queuing, or scheduling delays

along the transport path. The total number of measurements per cycle should be chosen

so that results have the required statistical confidence level. The recommended number

of measurements per IP host is ≥ 100.

If the measurement is executed over the radio link (LTE Uu), the first IP packet might

suffer additional delay caused by the dynamic allocation of radio resources to non-real-

time (non-GBR) bearers. For this reason, the round trip delay of the first packet should

not be considered in statistic calculations. On the other hand, the first ping measurement

can be used to determine UE state transition delays if the UE starts from non-registered

or idle states (from non-registered to registered, or from idle to connected states). The

periodicity of Pings needs to be selected so that the once allocated radio bearer is kept

over the whole duration of the measurement.

The test is to be done for stationary and mobile end users. The measurement with sta-

tionary UE should be performed under average radio link conditions with normal SINR.

For moving UEs, two different positions are to be selected, one in average radio link con-

ditions and normal SINR, and another in the cell overlapping area. It is important to

make sure that handovers between cells are avoided. The reference conditions should

be monitored during the test using UE performance measurements; also, relevant trace

data can be collected from the network.

In lab trials, the measurements need to be executed with and w/o concurrent load.

From the measurement results (samples) mean figures, standard deviation and 95%

delays need to be calculated.

2.3.2.2 Signaling and Media Delay Measurements

If the bandwidths of UL and DL bearers in RTT measurements are identical, the bench-

mark value of UL and DL latencies can be given as 1/2 RTT.

However, latency KPIs of signaling procedures on the network or application level (e.g.

Radio Access Bearer (RAB) service setup, or VoIP call setup flows) are often complex

and include the exchange of more than one messages of different sizes. In addition, thelink capacities used in UL and DL are often different, e.g. in case of the application

services data download, gaming or video streaming.

In these cases, the RTT or 1/2 RTT figures are not sufficient to characterize the latencies

incurred by the mobile access network. Here, the time synchronization of network

elements is the central problem, since the start and stop triggers of the measurement lie

on different Network Elements (NE) and interfaces.

In case of measuring latencies over the radio link, the time needed to attach the mobile

to the network and to setup the necessary radio resources is a significant component.

In these cases, the measurement should be made with different initial states of the

mobile, e.g. power off, idle, connected and active states. LTE State Models lists the dif-









22 DN0972412



Id:0900d805807a33b6

ferent states of Mobile Terminations (MT) in 3GPP LTE specifications, see References

3. [3GPP23.401].

In field tests, latencies should be measured with the typical initial state of the mobile. The

initial state will depend on the measurement scenario, e.g. Idle for network attach, orConnected for EPS bearer setup.

Depending on the location of trigger points, end-user visible delays, LTE network

delays, different bearer (RB, E-RAB and EPS) delays, transport and NE processing

delays can be described similarly. The following aspects need to be specified for latency

measurements:

• Start and stop triggers of the media or signaling flow. The trigger point is defined by

the message type, NE and interface; it identifies where time stamps are to be

recorded.

• In case of measuring one-way delays, the method of time synchronization.

• Trace points and analysis tools. In addition to trace points in the network, traces may

be needed on mobile terminals when measuring end-user visible latencies, or E2E

LTE network delays.

2.3.2.3 Reference Conditions for Latency Measurements

Field measurements of latencies are requested under reference radio conditions (equal

to cell medium), which are specified by the acceptable ranges of UL/DL SINR and of

receive power levels. In addition, the following variations of conditions are planned (see

also Table 6 Reference Conditions for Latency Measurements):

• Unloaded versus loaded cell conditions

• Stationary UE versus mobile on the move

• Drive route for measurements with velocity from medium to cell edge position includ-

ing hand-over





DN0972412


23


Id:0900d805807a33b6

Those samples that were taken in measurements during which the specified conditions

(reference, or explicitly modified ones) were not met, have to be excluded from the sta-

tistic calculations.

2.3.3 Service Accessibility Measurements

Accessibility KPIs characterize the accessibility of a service, or service element to its

users. They are expressed as a ratio of successful service requests to the total number

of attempts. (This group of KPIs corresponds to the category of "service accessibility

KPIs" in References 12. [3GPP32.410].) The KPI value is calculated as:

Success Rate = (# successful service requests / # total attempts) * 100 [%]

For this test to be meaningful, a significant number of samples need to be collected. E.g.

to get the success rate with 0.2% granularity, at least 500 samples are needed.

Service request attempts that are rejected due to insufficient access rights, bad user

authentication, or errors in request parameters are not considered and should be taken

out of the samples used for calculating the metric.

The measurements should be executed with the following variations of reference condi-

tions:


Terminal

UE Type Mobile phone, smart phone, laptop with LTE card

UE Category UE category 1-5 according to [3GPP36.306]; see References 21.

Antenna One antenna: e.g. SISO

Position Medium, overlapping area

Distribution One terminal

Network Environment

Channel Model EVA0, EVA70 (stationary, mobile w/ ~50 km/h)

Backhaul Capacity Capacities of X2, S1-U transport links.

QoS Configuration Real-time and non-real-time QoS, the latter with equal weightsand priorities

Concurrent Load unloaded and loaded with concurrent UL/DL data of medium

volume (below cell target capacity)

Application Data upload download and real-time over EPS bearer.

Traces Trace points, send & receive packet counts

Setup Field Reference Cluster, Trial Lab

Route From radio position with reference condition (medium) to overlap-

ping area including hand-over.

Table 6 Reference Conditions for Latency Measurements



24 DN0972412



Id:0900d805807a33b6

• Application is selected according to the mobile network service procedure that is

tested. Test scripts are used for the automated execution of service requests, for the

evaluation of responses and for the calculation of the KPI figure.

• Mobile terminals in stationary and mobile use. • The mobile use includes drive routes among good and medium receive conditions,

as well as a route that crosses cell overlapping areas. In the latter case, hand-over

is executed. The frequency of handovers follows the assumptions of the NSN refer-

ence traffic model for LTE networks.

• Unloaded and loaded cell conditions. Concurrent load is to be generated in lab trials

with low and medium levels (as a certain percentage of the cell target load). In field

network tests, where a certain background load is given, load levels are to be traced

to allow for scaling (calibrating) of lab trial results.

The reference conditions are summarized in Table 7 Reference Conditions for Success

Rate Measurements.

Due to the high number of test executions, the measurements can be executed in lab

environment with simulated air links. At least the stationary tests should be executed in

field network cluster. Automated test execution is needed in both environments.

In addition to the KPI figures calculated by the test script, test results should include

online performance measurement statistics, which are gained by monitoring the perfor-

mance counters of LTE network elements (i.e. on the eNB, MME, S-GW, P-GW). Please


Terminal

UE Type Laptop with LTE card, or modem.

UE Category UE category according to [3GPP36.306]; see References 21.


Position Good, medium and cell edge area.

Distribution n.a. (one terminal only)

Network Environment

Channel Model EVA0, EVA70: stationary, mobile w/ ~50 km/h


QoS Configuration n.a.

Concurrent Load Unloaded or concurrent UL/DL data volumes of low and medium

levels.

Application Procedure dependent test script for automated execution of dif-

ferent applications.

Traces Data collected by test script. Monitoring of onl ine statistic coun-

ters.

Setup Field Network Cluster and Trial Lab.

Route Real drive test routes (in Cluster), or simulated with varying

channel models (in Lab).

Table 7 Reference Conditions for Success Rate Measurements



26 DN0972412



Id:0900d805807a33b6

One KPI value (measured sample) is calculated for each scenario as:

Packet Loss Rate = (# not received packets / # sent packets) * 100 [%]

The PLR is given as mean value over all samples of the same measurement scenario.

For this test to be meaningful, a significant number of samples needs to be collected.

For this reason, the test should run for at least one hour in the lab.

It is not necessary to make a detailed analysis of the reasons behind packet losses, as

it is difficult to store extremely large amounts of trace data. However, if professional test

equipment is available for this analysis, the measurement setup may include it.

Online performance statistics on PLR can be collected during the measurement if the

corresponding Performance Management (PM) counters are available on LTE network

elements (i.e. on the P-GW). Please note, however, that the PLR figures collected may

not be identical to the PLR measured on the test UE and on the AS.


Terminal


UE Category UE category tbd. according to [3GPP36.306]; see 21.


Position Good and medium in originating cell, drive across hand-over

area.


Network Environment


Backhaul Capacity n.a.QoS Configuration n.a.

Concurrent Load Loaded & unloaded in source & destination cells

Application Data download (UDP) and video streaming.

Traces UE and eNB, mobility events

Setup Reference Cluster, Trial Lab.

Route Drive test route passing best and medium cell positions, and

crossing cell borders.

Table 8 Reference Conditions for Packet Loss Rate Measurements





DN0972412


27


Id:0900d805807a33b6

2.3.4.2 Service Drop Rates

Attach, Service Request, Register, etc. are examples of service requests on different

service levels; service requests establish a relationship between the UE and the network

or application. The relationship is normally terminated with a "Release Request" by theUE. In case of network errors (e.g. handover failures), or insufficient radio resources the

relationship can be terminated unexpectedly. The ratio of such "abnormal" terminations

to the number of all established relationships is given by "drop rate" KPIs.

For certain services, like bearer creation or VoIP call, the service drop rate is also related

to the duration of the session. This is motivated by the fact that the possibility of network

errors, e.g. hand-over failures is higher if the service duration is longer. In such cases,

the service usage time should also be measured, and the drop ratio expressed in [1/s]

units instead of [%]. The current version of this document does not consider session time

for service drop rates. It will be added in the next update of this document.

In order to measure the ratio, the given service request has to be executed in a suffi-

ciently large number of times to assume statistical significance. E.g. each test scenario

should be executed ≥ 100 times. After the relationship has been established, common

operations of the service are to be executed for a typical period of service usage time.

The LTE traffic model can deliver input for this.

Due to the complexity of the test setup and the long time needed to execute the tests,

the measurements are better executed in a lab environment with simulated air links and

automated service execution.

The measurements are to be executed under the following varying conditions:

• Mobile terminal in stationary and mobile use

• The mobile use includes drive routes among good and medium reception conditions,

as well as routes that cross cell overlapping areas. In the latter case, a handover isexecuted. If executed in the trial lab, these conditions are set in simulated radio links,

i.e. the "drive route" is simulated.

• Unloaded and loaded cell conditions. Concurrent load is to be generated up to full

(or target) load level of the cell.



28 DN0972412



Id:0900d805807a33b6

2.3.5 Mobility (Handover) Measurements

Mobility KPIs from a group because all of them are related to hand-over procedures.

They could, however, be discussed in the Latency and Accessibility groups, as well,

because HO-related KPIs are either service interrupt time, or success ratio type KPIs.

A successful hand-over is understood to be any successfully performed make-before-

break procedure. For its measurement, a DL User Datagram Protocol (UDP) data

stream is generated as the only load during the whole measurement. The test is per-

formed in an overlapping area between two cells. The overlapping region of a cell is

defined by the level of radio conditions (DL SINR and receive power levels) which are

identical for neighboring cells. It can be expressed as the difference between receive

sensitivity levels of neighboring cells. I.e. at the border line of the overlapping region the

relative receive sensitivity level is 0 dB.

Drive routes need to be specified so that they begin in good radio conditions of the orig-

inating cell and proceed directly through the hand-off area and end in high signal

strength and good SINR in the other cell. Drive routes should proceed directly through

the hand-off area in order to prevent multiple handoffs caused by stop lights, stop signs,

traffic congestion as well as to prevent extending of the time period in the hand-over

region. Routes must be carefully selected to avoid areas where handoff is driven through

lack of coverage, which leads to breaks in connection. This is the most common cause

of so-called "false failures".


Terminal


UE Category UE category tbd. according to [3GPP36.306]; see References

21.


Position Good, medium and cell edge area.


Network Environment


Backhaul Capacity Capacities of X2, S1-U, S1-MME transport links.QoS Configuration Link weight and priority assigned to the flow(s)

Concurrent Load Unloaded or concurrent UL/DL data volumes up to full cell load.

Application n.a.

Traces Trace points & data to be collected

Setup Trial Lab with simulated air link conditions.

Route The drive test routes are simulated using different channel

models and varying radio conditions (receive level, SINR).

Table 9 Reference Conditions for Service Drop Rate Measurements





DN0972412


29


Id:0900d805807a33b6

In order to measure handover KPIs, the logging of mobility events is necessary in mobile

terminals. A successful handover is marked by an entry in the log, which carries the

identifier of the new cell (destination).

The HO KPIs are measured under the following variations of conditions (Table10 Reference Conditions for Handover Measurements):

• Unloaded versus loaded cell conditions for Intra RAT, ideal conditions in target cell

for Inter Radio Access Technology (RAT) (GSM, UMTS) hand-over scenarios.

• Mobile terminals with vehicular speed.

• Drive routes from good reception condition (in the originating cell) over hand-over

area to a good position on the destination cell.

The drive test has to be repeated at least 20 times to get the required level of statistical

confidence. All reasons for handover failures (no resources in target cell, service not

supported, etc.) should be excluded.


Terminal

UE Type Smart phone, laptop with LTE card

UE Category UE category tbd. according to [3GPP36.306]; see References

21.


Position Good/medium in originating cell directly across hand-over area to

good/medium position in new cell.

Distribution n.a.

Network Environment


Backhaul Capacity n.a.

QoS Configuration n.a.

Concurrent Load Unloaded and loaded destination cell in UL/DL.

Application UDP data stream in DL with hand-offs acc. to reference traffic

model.

Traces MT, Source and Target eNBs, mobility events

Setup Reference Cluster, Trial Lab

Route In reference cluster, drive test route description.

Table 10 Reference Conditions for Handover Measurements



30 DN0972412



Id:0900d805807a33b8

2.4 Measurement Setup

2.4.1 Field Network Cluster

For field performance verification tests of KPIs to be executed in live network environ-

ments, the following LTE Reference Cluster is recommended (see also Figure

3 Reference Cluster for Field Performance Verification Tests):

• Sufficient number and positioning of UEs as specified by the measurement scenario.

All UEs are inside of the cells of the reference cluster.

• eUTRAN including at least three eNBs, one MME, and an integrated S-GW/P-GW.

• Each eNB supporting three LTE cells, each with predefined minimum radio condi-

tions at cell borders.

• Provisioning of network equipment and setting of configuration parameters accord-

ing to the guidelines of the LTE network operator and agreed with the vendor.

The recommendation considers constraints of the NSN LTE product release.

The Reference Cluster environment must not change during the test campaign (no

changes to network configuration and parameter values).

Traces are to be activated on the UE, and on the network elements MME and S/P-GW

as required by the measurement scenario and reference conditions of the given KPI.

Trace data that can be collected on the interfaces LTE-Uu, S1, X1 and S11 are

described in References 14. [3GPP32.423]. Traces should be taken such that the inves-

tigated KPIs (e.g. latencies) are not impacted. If this cannot be avoided, the measure-

ments should be repeated without traces.

Figure 3 Reference Cluster for Field Performance Verification Tests

In addition to Figure 3 Reference Cluster for Field Performance Verification Tests, the

final diagram of a test specification should contain indications of:

• At least three eNBs with three cells each

• Trace points on all NE interfaces

• Monitoring station or test equipment with adequate protocol stack (e.g. K12xx) todecode IP and radio network layer protocol messages



DN0972412


31


Id:0900d805807a33b8

• Test workstations (WS) to generate background load

• WS for post-processing of traces

• Operations, Administrations and Maintenance Performance Management (OAM

PM) for monitoring counters, displaying of statistics

2.4.2 Trial Lab

A description of the Trial Lab should be given with the performance verification test

specifications.

2.4.3 Measurement Data Collection

The following ways of performance data collection are possible in KPI measurements:

• Collection of traces from terminals, network elements and network interfaces

• Collection and evaluation of counters from network elements by OAM PMThe preferred way of collecting performance data in a live network is via PM counters.

It does not add additional load to the network and is a permanent source of information

that can be monitored all the times. However, PM counters do not exist for all categories

and for all individual KPIs. Typically, no counters are defined, or can be implemented for

latency KPIs. For this reason, terminal, server and network traces are also collected.

Since tracing puts additional processing load on these elements, traces are activated

only on operator demand and for the time of test execution. On the other hand, traces

include very detailed information on call level, which allows performance analyses of

individual mobiles and operations. With traces, it is possible to go further in monitoring

and optimization operations. For more details on LTE MMS and E-UTRAN traces, see

References 13. [3GPP32.421] and 14. [3GPP32.423].

2.4.3.1 Subscriber and Equipment Traces

Subscriber and UE Traces should be activated on the UE, and on the network elements

MME and S/P-GW of the Reference Cluster, as required by the scenario description

(see Reference Conditions) of the given KPI measurement.

Collected measurement data is transferred to an external server for post-processing and

for evaluation. This data is necessary to check if the test run was executed w/o failure

(e.g. dropped call analysis) and if the measured figure can be considered for statistic cal-

culations and statements. Moreover, it helps to verify if the reference conditions were

met (e.g. RF coverage and capacity) during the whole measurement. Traces deliver

additional information for tuning and optimizing the Reference Cluster itself.

The capability to log data on any interface at call level for a specific user (e.g. Interna-

tional Mobile Subscriber Identity (IMSI)) or mobile type (e.g. International Mobile Equip-

ment Identity (IMEI)), or service initiated by a user makes it possible to get information

which cannot be deduced from Performance Measurements such as the end-user per-

ception of Quality of Service (QoS) during a call (e.g. requested QoS vs. provided QoS),

and correlation between protocol messages and RF measurements. Moreover, as

opposed to OAM Performance Measurements, which provide values aggregated on an

observation period (i.e. interval statistics), Subscriber and UE traces give instantaneous

values for a specific event (e.g. call, location update, etc.).















32 DN0972412



Id:0900d805807a33b8

2.4.3.2 PM Counters

Performance measurements are mandatory in all field test scenarios. This provides per-

formance related information similar to operational environments.

The PM counters of special interest are given with the KPI description (see Overview).

This level of detail is provided in accordance with the availability of PM counters in the

latest LTE product release and in 3GPP specifications.



DN0972412


33


Id:0900d805807a33ba

2.5 Measurement Reporting

Some measurement scenario descriptions already contain hints about the form of the

expected measurement reports (e.g. in Peak User Data Rate). A complete description

will be provided with the field verification test plan.



34 DN0972412



Id:0900d805807a32b6

Definitions of Key Performance Indicators (KPIs)

3 Definitions of Key Performance Indicators

(KPIs)

3.1 Overview

Key Performance Indicators (KPIs) are defined for application and for LTE network

services (see Figure 1 LTE Bearer Service Architecture):

a) Application Services

b) LTE E2E Network Service

c) Radio Bearer Services: E-RAB, RB

d) Transport Bearer Services: X2, S1, S5/S8 and External

e) LTE Network Elements

The Application KPIs are service specific. The following services are considered: • Web Browsing

• Data Upload / Download

• Audio Video Streaming

• Conversational Voice (VoIP)

The current paper focuses on the application service “Data Download/ Upload (FTP)

Service KPIs", and on “LTE E2E Network Service" KPIs in line with the feature scope of

the LTE release.

The performance indicators of the categories D and E are useful for the decomposition

of E2E performance figures. Their output can help to locate potential performance bot-

tlenecks in the field, and to optimize the overall performance of the LTE network. E.g.

the KPI values of category D can be the basis for Service Level Objectives (SLOs) for

IP backhaul networks if these are operated by third party ISPs. Because SLOs are very

important, maximum acceptable values are specified for them as assumptions for the

achievement of performance targets for application and LTE network service KPIs.

KPIs are distinguished according to the measurement setup, which should be used for

their verification. They are denoted "Lab" KPIs, if field network trials are not recom-

mended with them due to high costs or technical limitations. These KPIs are measured

in vendor or operator laboratories only. The other category of KPIs is called "Field" KPIs,

because they should be measured in Reference Network Clusters of operational net-

works. However, also "Field" KPIs have to be verified in the laboratory before measure-

ments are executed in the field. Lab results serve as benchmark figures and will be used

to calibrate (scale) field network measurement results.

After the overview of all KPI categories, detailed descriptions are given in Application

Services and LTE Network Services.



DN0972412


35



Id:0900d805807a32b6

3.1.1 Application Services

The full set of application service KPIs is summarized in Table 11 Overview of Applica-

tion Service KPIs. The following IP based Packet Switched (PS) data services are char-

acterized with a common set of KPIs:

• Data Download / Upload (FTP)

• Web browsing (HTTP)

• Email (POP/SMTP)

The Data Download / Upload service is selected for detailed investigations and verifica-

tion in the LTE network, because of the Transmission Control Protocol (TCP) windowing

mechanism.

# KPI Name KPI

Category

Trial

Network

Chapter

PS Data Services (FTP, HTTP, etc.)

1 Service Accessibility Ratio [%] Accessibility Field (FTP) Service Accessibility Ratio

2 Completed Session Ratio [%] Reliability Field (FTP) Completed Session Ratio

3 Service Access Time [s] Latency Field (FTP) Service Access Time

4 Session Time [s] Latency Field (FTP) Session Time

5 User Data Rate [Mbps] Throughput Field (FTP) User Data Rate

Table 11 Overview of Application Service KPIs







36 DN0972412



Id:0900d805807a32b6


3.1.2 LTE Network Services

3.1.2.1 LTE E2E Network Service

The full set of LTE E2E network service KPIs is summarized in Table 12 Overview of

LTE E2E Network Service KPIs. Some of the KPIs are defined as "Lab", which means

that they are verified in in-house (or vendor) labs, but not in the field.

# KPI Name KPI

Category

Trial

Network

Chapter

LTE E2E Network Service, Control

Plane

6 Attach Time [ms] Latency Field Attach Time

7 Detach Time [ms] Latency Lab Detach Time

8 Attach Success Rate [%] Accessibility Field Attach Success Rate

9 Service Request (EPS) Time [ms] Latency Field Service Request Time (EPS), UE Ini-

tiated,

Service Request (EPS) Time,

Network Initiated

10 Service Request (EPS) Success Rate

[%]

Accessibility Field Service Request (EPS) Success

Rate

11 Service (EPS) Drop Rate [%] Reliability Field Service (EPS Bearer) Drop Rate

12 Handover Procedure Time [ms] Mobility Lab Handover Procedure Time

13 Handover Success Rate [%] Mobility Field Handover Success Rate

14 Paging Time [ms] Latency Lab Paging Time

15 Paging Failure Rate [%] Accessibility Lab Paging Failure Rate

LTE E2E Network Service, User Plane

16 (LTE) Round Trip Time (RTT) [ms] Latency Field (LTE) Round Trip Time

17 (LTE) User Data Rate [Mbps] Throughput Field (LTE) User Data Rate

18 (LTE) Packet Loss Rate (PLR) [%] Reliability Field (LTE) Packet Loss Rate

19 (LTE) Service Interrupt Time (HO) [ms] Mobility Lab (LTE) Service Interrupt Time (HO)

Table 12 Overview of LTE E2E Network Service KPIs



DN0972412


37



Id:0900d805807a32b6

3.1.2.2 Radio Bearer Services

# KPI Name KPI

Category

Trial

Network

Chapter

Radio Bearer Service, User Plane

20 (RB) Packet Loss Rate UL / DL [%] Reliability Lab (RB) Packet Loss Rate

21 (RB) User Data Rate [Mbps] Throughput Field (RB) User Data Rate

22 Cell Throughput [Mbps] Throughput Field Cell Throughput

Table 13 Overview of LTE Radio Bearer Service KPIs



38 DN0972412



Id:0900d805807a3c4c

3.2 KPI Definition Template

For each KPI, relevant information is provided according to the structure shown in Table

14 KPI Definition Template. The fields in the KPI definition are marked as mandatory, or

optional according to References 12. [3GPP32.410].

The "KPI Object" field identifies the service, or network element to be characterized:

• Services: AS or one of the 3GPP bearer services: E2E network (LTE), EPS, E-RAB,

RB, X2, S1, S5/S8 and External. The bearer services are uniquely defined by their

end-points, i.e. by the network interfaces of the LTE architecture.

• Network Elements: eNB, MME, S/P-GW.

The following "KPI Categories" are used: Accessibility, Reliability (Retainability),Latency, Throughput, Mobility and Quality.

The PM measurements and KPIs of special interest are given with indication of their

relationship to the field network KPI, e.g. if and how to calculate the field network KPI

value out of them.

KPI Name M Mandatory long name of the KPI.

KPI Type, Category, Object, Unit M Type: Ratio, Mean, CDF, etc. [3GPP32.814]; see References 19.

Category: Accessibility, Reliability, Latency, Mobility, etc.

Object: Application Service, E2E, E-RAB, RB, X1, etc.

Unit: %, ms, Mbps, Erlang, etc.

Definition M Description of the KPI. End-user or network view. Basic information

about begin and end trigger points.

Methodology of measuring O Test environment: field or lab test, or simulation.Measurement method:

according to KPI Category, plus specifics for the KPI (if any).

Assumptions and pre-conditions O Important information for measurement execution, i.e. scenario

description including terminal category, channel model, load condi-

tions, etc.

Formula (Logical) M Provided for KPIs, e.g. Success Rate & Drop Rate which are calculated

from some other metrics (measured)

Message flow, trigger points O Provided for latency KPIs, for monitoring of procedure correctness, and

for success rate KPIs to indicate trigger points where PM counters are

incremented.

Related KPIs O Reference to related KPIs

Related PM counters O Reference to related domain specific PM counters with indication of

how to aggregate them to the KPI.

Table 14 KPI Definition Template



DN0972412


39


Id:0900d805807a3c21

3.3 Application Services

3.3.1 Data Download / Upload (FTP) Service KPIs

Download of reference files with different sizes from the Internet, e.g. SW images, pic-

tures, or video, as well as upload of the same files by FTP. For FTP protocol details, see

References 33. [RFC979].

3.3.1.1 (FTP) Service Accessibility Ratio

(FTP) Service Accessibility Ratio

KPI Category, Object, Type, Unit M Category: Accessibility, Object: AS (UE, FTP server),

Type: Ratio, Unit: [%]

Definition M The service accessibility ratio denotes the probability that the user canestablish the necessary bearer (EPS) and access the FTP service suc-

cessfully (see References 39. [ETSI102.250-2]).

Service access covers starting the FTP client on the UE, setting up

mobile access and creating a TCP connection to the FTP server.

Measurement method O Accessibility measurement, see Service Accessibility Measurements.

Field and lab trial.

Assumptions, pre-conditions O Conditions for accessibility measurements see Service Accessibility

Measurements

Formula (logical) M

Related KPIs O (FTP) Completed Session Ratio

Related PM counters O No PM counters exist for this KPI

Message flow, trigger points O Trigger points:

• Start: ftp get / put command issued on the UE

• Stop: first data byte sent / received by the UE, or ftp indicates

network error (e.g. timeout expired)

FtpCmdSR

number_of(successful_ftp_commands)

number_of(total_ftp_commands)--------------------------------------------------------------------------------------------------------- 100%⋅=



40 DN0972412



Id:0900d805807a3c21

3.3.1.2 (FTP) Completed Session Ratio

(FTP) Completed Session Ratio

KPI Category, Object, Type, Unit M Category: Availability, Object: AS (UE, FTP server),


Definition M The completed session ratio is the proportion of completed FTP

sessions and sessions that were started successfully (see References

39. [ETSI102.250-2]). The session is not completed if a predefined

timer expires, or if a network failure has occurred that could not be

repaired by automatic restarts.

Measurement method O Availability measurement, see Service Accessibility Measurements.


Assumptions, pre-conditions O Conditions for availability measurements see Service Accessibility

Measurements.The size of the file used for data upload / download, or the session

duration time is to be specified for a concrete requirement on the target

value of this KPI.

Formula (logical) M

Related KPIs O (FTP) Service Accessibility Ratio

Related PM counters O No PM counters exist for this KPI.

Message flow, trigger points O Trigger points:

• Start: ftp get / put command issued on the UE.

• Stop: after error free execution of the command, ftp returns with a

prompt and indicates the number of bytes transmitted.

FtpSessionSRnumber_of(completed_sessions)

number_of(successfully_started_sessions)------------------------------------------------------------------------------------------------------------------- 100%⋅=



DN0972412


41


Id:0900d805807a3c21

3.3.1.3 (FTP) Session Time

(FTP) Completed Session Ratio

KPI Category, Object, Type, Unit M Category: Latency, Object: AS (UE, FTP server),

Type: Mean and max (95%) values, Unit: [s]

Definition M It is the overall duration of the download or upload of reference files

from / to the FTP server (ref. to [ETSI102.250-2]). The elapsed time is

measured between start and end trigger points.

Measurement method O Latency measurement, see Latency Measurements

Assumptions, pre-conditions O Conditions for latency measurements see Reference Conditions for

Latency Measurements

Formula (logical) M

Related KPIs O (FTP) Service Access Time, Service Request (EPS) Time


Message flow, trigger points O Trigger points UI:

• Start: ftp client started on the UI.

• Stop: after error free execution of the "ftp get / put" command,

receiving prompt with the number of bytes transmitted.

Trigger points MT / R if (Download):

• Start: First [SYN] sent to setup a TCP connection. • Stop: Reception of the last data packet containing content.

Trigger points MT / R if (Upload):

• Start: First [SYN] sent to setup a TCP connection.

• Stop: Receive [FIN, ACK] to last data packet sent with file content.

Ft pS essi onTim e [s] tsessionend tsessionstart–=



42 DN0972412



Id:0900d805807a3c21

3.3.1.4 (FTP) Service Access Time

Figure 4 KPIs for Packet Switched Data Services shows the phases of a data download

/ upload (FTP) service session using GPRS access network as example (see Refer-

ences 39. [ETSI102.250-2]). In case of an LTE network, GPRS would be replaced byLTE and an EPS bearer would correspond to the "PDP-Context". The full data download

/ upload session is subdivided into the phases Service Access and Data Transfer.

In this document, the Service Access Time is defined as independent KPI. It corre-

sponds to the phase "IP Service Access" on the Figure 4 KPIs for Packet Switched Data

Services. (The EPS bearer setup time is defined as LTE E2E network KPI in Service

Request Time (EPS), UE Initiated.) The trigger points for the Service Access Time KPI

are defined on two levels:

• end-user interface (UI), i.e. on command line level, and

• TE - MT interface, i.e. on UE R interface level.

On the command line level, the end of the mobile network access phase is recognizedby the ftp> prompt. By this time, the FTP client is initialized, the terminal is attached to

the LTE network and the EPS bearer has been created. The Session is started by

issuing the "ftp get / put" command and terminates with the display of the results of the

data transfer (e.g. the # of bytes transmitted.). After the "get / put" command has been

be issued, a TCP connection to the FTP server is created and the data transfer phase

begins.

Trigger events of the Service Access Time KPI cannot be seen on the terminal UI. In

order to find them, it is necessary to analyze the message flows on the R interface level.

The Session (and the IP Service Access phase, see Figure 4 KPIs for Packet Switched

Data Services) begins with setting up the TCP connection to the FTP server and termi-

nates with the transmission (send, or receive) of the last IP packet with file content

payload. The phases Service Access and IP Service Access end, and the Data Transfer

phase begins with the transmission (send, or receive) of the first IP packet with file

content payload.





DN0972412


43


Id:0900d805807a3c21

Figure 4 KPIs for Packet Switched Data Services



44 DN0972412



Id:0900d805807a3c21

(FTP) Service Access Time

KPI Category, Object, Type, Unit M Category: Latency, Object: AS (UE, FTP server),

Type: Mean and max (95%) values, Unit: [s]

Definition M It is the time period needed to access the FTP service successfully,

from starting the ftp client to the point of time when the first data packet

is sent or received (see References 39. [ETSI102.250-2]). It is

assumed that the UE is already attached to the mobile network and an

EPS bearer exists.

Measurement method O Latency measurement, see Latency Measurements.


Latency Measurements.

Formula (logical) M

Related KPIs O (FTP) Session Time, Service Request (EPS) Time


Message flow, trigger points O Trigger points DL/UL:

• Start: "ftp get / put" command issued on the UI

• Stop: first data packet containing file content received / sent on the

R interface.

FtpServ iceAccessT ime [s] tco nte nt sent or received tftp command started–=



DN0972412


45


Id:0900d805807a3c21

3.3.1.5 (FTP) User Data Rate

It is to be noted that the measurement of the User Data Rate (FTP) KPI (named after

[ETSI102.250-2]; see References 39.) is executed as a single user measurement

without concurrent load in the cell, since the goal is to find the maximum available FTP

data rate for one user.

(FTP) User Data Rate UL / DL

KPI Category, Object, Type,

Unit

M Category: Throughput, Object: AS (UE, FTP server),

Type: Min (CDF 5%), Mean (CDF 50%), Max (CDF 95%) values, Unit: [Mbps]

Definition M After the connection to the FTP server has been successfully established, the

parameter describes the average data transfer rate measured over the data

transfer phase (see Figure 4 KPIs for Packet Switched Data Services). This

means that successful Service Access (availability of an EPS bearer and

connection to the FTP service) is a prerequisite to the data transfer. The data

transfer phase should also conclude successfully.

The data rate is measured on application level in UL/DL direction.

Measurement method O Throughput measurement, see Peak User Data Rate. Single user measure-

ment both in field and lab trials.

In order to eliminate the impact of service startup times, the measurement

begins with the transmission of first IP packets with file content payload.

Since the data rates can vary strongly from session to session, a sufficiently

large number of measurement executions is needed (>10) to get statistical

confidence.

The size of the large reference file should be selected such that the data

transfer phase does not take less then thirty seconds. Optionally, a small ref-

erence file can be measured in addition, but the number of executions should

be the same. The measured User Data Rates should be displayed in

diagrams as a function of time to show the impact of FTP slow start.

Assumptions, pre-conditions O Conditions for capacity measurements see Peak User Data Rate.

The FTP server will be connected directly to the SGi interface. Additional

delays or bottlenecks between the S/P-GW and the server should be

excluded.

The initial status of UE before starting the ftp application:

EMM-REGISTERED, ECM-CONNECTED.

The FTP application will be executed in binary mode, and the file to be trans-

ferred will also be binary.

Formula (logical) M

Related KPIs O (FTP) Service Access Time, (FTP) Session Time


Message flow, trigger points O The average throughput is measured from the opening of the data connec-

tion to the end of the successful transfer of the content (file, e-mail or web

page). The trigger points for the Data Transfer phase can be seen in Figure

4 KPIs for Packet Switched Data Services. For more details see References

39. [ETSI102.250-2], chapters 4.6.1.7 and 4.6.2.7.

FtpMeanDataRateUL/DLtransferred_data_volume_UL/DL[bytes] 8⋅

transfer_time [s]--------------------------------------------------------------------------------------------------------------------=



46 DN0972412



Id:0900d805807a3c48

3.4 LTE Network Services

This chapter includes detailed KPI definitions for LTE Network Services.

3.4.1 LTE E2E Network KPIs

3.4.1.1 Attach Time

Attach Time, Control Plane

KPI Category, Object, Type, Unit M Category: Latency, Object: End-user, LTE (UE R interface),

Type: Mean and max (95%) value, Unit: [ms]

Definition M With Attach, the mobile terminal registers at the LTE network. At the

end of the procedure the UE is authenticated, and a default (nGBR)

bearer is established.The Attach Time is the interval between the connection request and the

acknowledgement of the positive response by the UE.

Measurement method O Latency measurement, see Latency Measurements. Field and lab trial.

Stationary users only.

Assumptions, pre-conditions O Conditions for latency measurements, see Reference Conditions for

Latency Measurements. UE status before measurement: EMM-

DEREGISTEREDMAP inter-working according to [3GPP29.002], see

References 10., is assumed between MME and HSS (no Diameter).

Formula (logical) M tbd.: mean value and 95% from all measured samples

Message flow, trigger points O Trigger points on UE R interface (see Figure 5 Attach Procedure w/

Initial EPS Bearer Establishment (3GPP 23.401), References 3.

[3GPP23.401]):

• Begin: 1. RRC CONNECTION REQUEST sent by UE (see also

References 26. [3GPP36.331]) carrying NAS: Attach Request in its

body.

• End: 21. RRC DIRECT TRANSFER message with NAS: Attach

Complete in its body sent by UE (see References 4.

[3GPP24.008]).

Related KPIs O a) Attach Success Rate.

b) No 3GPP defined KPI for Attach Time found (exists).

c) Attach Setup Time KPI in [ETSI102.250-2], see References39.

chapter 4.6.3.2.

Related PM counters O Attach Time related measurements are not defined in 3GPP Release 8

documents.

However, the same trigger events for counting Attach requests and

successful attachment completions can be used as Begin and End

triggers for the measurement of elapsed time on the eNB.

At tach Time [ms] t At tac h Complete t At tach Request–=



DN0972412


47


Id:0900d805807a3c48

For initial Attach, the following specifics of the message flow in Figure 5 Attach Proce-

dure w/ Initial EPS Bearer Establishment (3GPP 23.401), have to be considered:

• A signaling bearer (SRB1) is to be established, before an Attach Request can be

sent, i.e. the UE is expected execute a contention based Random Access (RA) pro-cedure.

• Step 3. is not executed, since the IMSI is queried from the UE directly using Step 4.

The optional Identity Request / Response messages in step 5.b do not have to be

exchanged if the MEI is already returned by the UE using the Security Mode

Complete message as part of step 5.a.

• Steps 7-11 are not executed, no EPS bearers are to be deleted.



48 DN0972412



Id:0900d805807a3c48

Figure 5 Attach Procedure w/ Initial EPS Bearer Establishment (3GPP 23.401)



DN0972412


49


Id:0900d805807a3c48

3.4.1.2 Detach Time

Detach Time, Control Plane

KPI Cate-

gory, Object,

Type, Unit

M Category: Latency, Object: LTE (UE R interface)

Type: Mean and max (95%-ile) values, Unit: [ms]

Definition M With an explicit Detach request the UE informs the LTE network that it does not want to

access the EPS any longer. At the end of the procedure all EPS bearers of the UE are

released.

The Detach Time is the interval between the Detach Request and the reception of a Detach

Accept message by the UE. No Detach Accept is sent by the network if the cause for Detach

is switching the UE off.

Measurement

method

O Latency measurement, see Latency Measurements Lab trial.

Stationary users only.

Assumptions,

pre-conditions

O Conditions for latency measurements see Reference Conditions for Latency Measurements.

UE status before measurement: EMM-REGISTERED .

Formula

(logical)

M

Message flow,

trigger points

O Trigger points on UE R interface (see Figure 6 UE-Initiated Detach Procedure,

[3GPP23.401]):

• Begin: 1. RRC UL Information Transfer sent by UE carrying NAS: Detach Request in its

body.

• End: 11. RRC DL Information Transfer message with NAS: Detach Accept in its body

received by the UE.

Related KPIs O a) Attach Time

b) No 3GPP defined KPI for Detach Time found (exists).

Related PM

counters

O Detach Time related measurements are not defined in 3GPP Release 8 documents.

Detach Time [ms] tDetachRequest tDetachAccept–=



50 DN0972412



Id:0900d805807a3c48

Figure 6 UE-Initiated Detach Procedure



DN0972412


51


Id:0900d805807a3c48

3.4.1.3 Attach Success Rate

Attach Success Rate, Control Planee

KPI Cate-

gory, Object,

Type, Unit

M Category: Accessibility,

Object: LTE (UE R, eNB Uu interface),


Definition M The Attach Success Rate is defined as the ratio between the number of successful registra-

tions and the number of all requests. This is the probability that a user can attach to the LTE

network at any moment of time.

The calculated success ratio figure excludes attach requests, which are rejected by authen-

tication failures. On the other hand, network attach requests which are terminated by timer

expiry (due to the unavailability of some LTE resource) are considered as unsuccessful reg-

istrations.

If the success rate is calculated on the eNB by counting incoming RRC requests, RRC CON-

NECTION REQUEST retries are to be excluded, since they would increase the overallnumber of establishment attempts, and thus reduce the success ratio.

Measurement

method

O Accessibility measurement, see . Field and lab trial. Stationary users only.

For defining the success ratio, the same series of "Attach Time" measurements can be used.

The KPI cannot be measured as end-user perceived ratio on certain mobiles, which do not

indicate network attachment status to the end-user.

Assumptions,

pre-conditions

O Conditions for accessibility measurements see Service Accessibility Measurements.

See also conditions in Attach Time.

Formula

(logical)

M

Message flow,

trigger points

O Trigger events on UE R / eNB Uu interface (see Figure 5 Attach Procedure w/ Initial EPS

Bearer Establishment (3GPP 23.401)):

• Success: 21. RRC DIRECT TRANSFER message with L3 NAS: Attach Complete in its

body (see References 4. [3GPP24.008]) sent by the UE / received by eNB

• Attempts: 1. RRC CONNECTION REQUEST messages (with cause Attach Request)

sent by UE / received by eNB.

Related KPIs O a) Attach Time, Service Request (ESP) Success Rate.

b) E-RAB Accessibility" KPI in [3GPP32.450], see References17. chapter 6.1.1. Since an

SRB and S1 connection, as well as initial EPS bearer(s) have to be created during the

Attach procedure (see Figure 5 Attach Procedure w/ Initial EPS Bearer Establishment(3GPP 23.401)), its success rate is dependent of the success rates of those component

bearers. The success rate of initial EPS bearers is given by the following expression (for

“additional” EPS bearers, refer to the Service Request (EPS) Success Rate):

c) See "Attach Failure Ratio" KPI in References39. [ETSI102.250-2], Chapter 4.6.3.1

defined as the inverse probability of successful network attachment.

LTENwAttSRnumber_of_successful_attachments

number_of_all_attempts-------------------------------------------------------------------------------------------------- 100%⋅=



52 DN0972412



Id:0900d805807a3c48

Related PM

counters

O a) Related 3GPP measurements are defined in [3GPP32.425],see References15., for eNB:

Signaling Radio Bearer (SRB) creation:

• Attempts: RRC.ConnEstabAtt.Cause = Attach

• Success: RRC.ConnEstabSucc.Cause = Attach • Failures: RRC.ConnEstabFail.Cause = Attach

UE associated logical S1 connection (S1-AP):

• Attempts: S1SIG.ConnEstabAtt

• Success: S1SIG.ConnEstabSucc

Initial EPS setups during Attach:

• Attempts: SAEB.EstabInitAttNbr.QCI

• Success: SAEB.EstabInitSuccNbr.QCI

where QCI identifies the SAE Bearer level quality of service class.

• Failures: SAEB.EstabInitFailNbr.Cause

where Cause identifies the cause resulting in the setup failure.

b) The corresponding eNB counters are:

• Attempts: SIG_CON_EST_ATT due to Attach

• Successful attachments: SIG_CON_EST_SUCC

• Failures: SIG_CON_EST_FAIL

For the calculation of success, or failure ratios, it is necessary to have identical Causes

for _ATT, _SUCC and _FAIL types of counter groups.

Since the EPS_SETUP_ATT/_SUCC counters do not differentiate between initial and

subsequent EPS bearer setups, they cannot be set in direct relation to the Attach SR KPI.





DN0972412


53


Id:0900d805807a3c48

3.4.1.4 Service Request Time (EPS), UE Initiated

Service Request Time (EPS Bearer Setup), UE Initiated, Control Plane

KPI Category, Object, Type, Unit M Category: Latency, Object: EPS (UE R, eNB Uu interface),

Type: Mean and max (95%) values, Unit: [ms]

Definition M It is the time taken by the LTE network to setup an EPS bearer on

request by the UE. The EPS bearer can be new (dedicated), or an

existing one (e.g. the default EPS bearer). The latter is needed to re-

assign Uu radio and S1 bearer resources to the existing EPS bearer of

a previously Idle UE.

The EPS bearer has to be created or activated before IP packets can

be exchanged, i.e. the Service Request creates the IP link dynamically

over the mobile access (LTE) network.

The UE Initiated EPS Bearer Setup Time is the interval between the

submission of the message RRC CONNECTION REQUEST (see Ref-erences 26. [3GPP36.331]) carrying a NAS: Service Request message

in its body, and the reception of the RRC CONNECTION RECONFIG-

URATION response by the UE. The Bearer Setup procedure is based

on the Service Request procedure, see References 3. [3GPP23.401]

and 20. [3GPP36.300].

Measurement method O Latency measurement, see Latency Measurements. Field and lab trial.

Stationary users only. Dedicated, or default EPS bearer setup.

The response time can be measured on the UE R interface, or on the

eNB Uu interface. The latter metric shows the delay component of the

E-UTRA network only.

End-user delays cannot be measured, since terminals do not indicatethe corresponding state change toward the end-user.

The setup times of the RB and S1 bearers are part of the EPS setup

time and can be determined using the same sequence of measure-

ments.



UE status before measurement: EMM-REGISTERED , but otherwise

different initial states (idle, dormant or active).

Formula (logical) M Mean value and 95% from all measured samples.Service Request

Message flow, trigger points O The message flow diagram is shown in Figure 7 EPS Bearer Setup Pro-

cedure, Part 1 and Figure 8 EPS Bearer Setup Procedure, Part 2. The

user initiated EPS bearer setup time is measured between the trigger

points BUE and EUE:

• Begin: RRC CONNECTION REQUEST sent by the UE carrying a

NAS: Service Request message in its body.

• End: corresponding RRC CONNECTION RECONFIGURATION

COMPLETE message or first PDU in UL sent by the UE (see Ref-

erences 4. [3GPP24.008]).

Service Request Time [ms] tRRC_Reconfig tRRC_Request–=



54 DN0972412



Id:0900d805807a3c48

Related KPIs O a) Attach Time, network initiated Service Request (EPS Bearer Setup)

Time.

b) No 3GPP defined KPI for EPS bearer setup found (exists).

Related PM counters O The eNB delay budget of the overall Service Request Time can be

measured using two counters of [3GPP32.425], see References 15.

• Mean EPS Setup Time: SAEB.EstabTimeMean.QCI

• Max EPS Setup Time: SAEB.EstabTimeMax.QCI

The latency is measured between the trigger points T1, T2 as can be

seen in Figure 8 EPS Bearer Setup Procedure, Part 2.



DN0972412


55


Id:0900d805807a3c48

Figure 7 EPS Bearer Setup Procedure, Part 1

D

LTE-Uu S1- MME S11 S6a

UE eNodeB MME S-GW / P-GW HSS

S1-AP: UL NASTRANSPORT

(L3: AUTHENTICATION AND CIPHERING

REQUEST)

Begin measurementRRC CONNECTION

REQUEST

MAP: Send Authentication Info

Setup SRB1

RRC CONNECTION

SETUP

S1-AP: INITIAL UEMESSAGE

(L3 NAS: SERVICEREQUEST)

MAP: Send Authentication Info Ack

RRC: DL INFORMATIONTRANSFER


REQUEST)

If authentication

vectors are not

available in MME,

authenticate UE

RRC: UL INFORMATIONTRANSFER


RESPONSE)

S1-AP: DL NASTRANSPORT


RESPONSE)

If authentication

timer has expired,

setup security

association for user

plane RB

UE EMM-REGISTEREDand ECM-IDLE

RRC CONNECTION

SETUP COMPLETE

(L3 NAS: SERVICEREQUEST)

L2 SCTP: Create BearerRequest

L2 SCTP: Create BearerResponse

Activate EPS

bearer setup in

ePC

DL data

L2 SCTP: DL DATANOTIFICATION

BN

BUE

Paging Procedure



56 DN0972412



Id:0900d805807a3c48

Figure 8 EPS Bearer Setup Procedure, Part 2

The service request times for UE and for network initiated EPS bearer setups are

measured between the trigger points BUE and EUE, and BN and EN, respectively.These trigger points are colored green in the diagram.

The trigger points T1 and T2 denote events, which increment the PM counters

EPS_BEARER_ATT and EPS_BEARER_SUCC on the eNB. They are colored yellow.

These counters can be used on the PM server to calculate EPS bearer setup success

rates as seen by the eNB. Please note that this value will be different from that of the

Service Request Success Rate KPI defined in this document (see Service Request

(EPS) Success Rate). The eNB counters calculate the RB (and not the EPS) setup rate.



S1-AP: INITIAL

CONTEXT

SETUP REQUEST

PM Counter:

EPS_SETUP_ATT

RRC SECURITY MODE

COMMAND

RRC CONNECTION

RECONFIGURATION

COMPLETE

S1-AP: INITIAL

CONTEXT

SETUP COMPLETEL2 SCTP: Update Bearer

Request

L2 SCTP: Update Bearer

Response

First UL PDU

T1

T2

RRC CONNECTION

RECONFIGURATION

RRC SECURITY MODE

COMMAND COMPLETE

Setup user

plane RB

Means “Delay DL

Packet Notification

Request” to stop/

enable DL data

transfer PM Counter:

EPS_SETUP_SUCC

End measurement

First DL PDU

EUE

EN

S1-AP: INITIAL

CONTEXT

SETUP FAILURE



DN0972412


57


Id:0900d805807a3c48

3.4.1.5 Service Request (EPS) Time, Network Initiated

The network initiated Service Request Time is not described from the UE point of view,

because the delay component seen on the UE is not the complete time the downlink

data packet is delayed before it can be forwarded from the P-GW to the UE. The network

initiated EPS bearer setup time is dependent on the mobility state of the mobile terminal.If the terminal is idle, paging becomes necessary.

Service Request (EPS Bearer Setup) Time, Network Initiated, Control Plane

KPI Category, Object, Type, Unit M Category: Latency, Object: EPS (S-GW S11 interface),

Type: Mean and max (95%) values, Unit: [ms]

Definition M It is the time taken by the LTE network to set up an EPS bearer on

request by the P-GW. The EPS bearer has to be created before IP

packets can be sent (DL) to the UE if the UE has no proper EPS bearer

for the given IP packet flow. The network initiated Service Request

Time includes a Paging Time (ref. to Paging Time) if the UE is idle. The

EPS Bearer Setup procedure is based on the Paging and Service

Request procedures according to References 26. [3GPP36.331] and 3.

[3GPP23.401].

Measurement method O Latency measurement, see Latency Measurements. Lab trial and sta-

tionary users only. Default EPS bearer setup. The setup times of E-

RAB and RB bearers, as well as the Paging time are part of the EPS

setup time and can be calculated from the same sequence of measure-

ments by taking appropriate traces.


Latency Measurements. UE status before measurement: EMM-REG-

ISTERED, but otherwise different initial states (idle, dormant or active).

Formula (logical) M Mean value and 95% from all measured samples.Service Request

Message flow, trigger points O The message flow of the Service Request procedure is shown in Figure

7 EPS Bearer Setup Procedure, Part 1 and Figure 8 EPS Bearer Setup

Procedure, Part 2. The network initiated service request time is

measured between the trigger points BN and EN on the S-GW S11

interface:

• Begin: L2 SCTP: DL DATA NOTIFICATION sent to MME

• End: L2 SCTP: Update Bearer Response sent to MME

If the service request is triggered by the arrival of a DL packet, the inter-

rupt time of the packet stream is measured on the S-GW as the latency

between:

• Begin: arrival of the DL packet at the S-GW toward the idle UE,

• End: forwarding of the same packet to the UE connected.

Related KPIs O Attach Time, Paging Time, E-RAB and RB Setup Times.

Related PM counters O The eNB delay budget of the overall Service Request (ESP) Time is

identical to that of the UE initiated procedures. See related PM counters

and measurements there.

Service Request Time [s] tRRC_Reconfig tRRC_Request–=



58 DN0972412



Id:0900d805807a3c48

3.4.1.6 Service Request (EPS) Success Rate

Service Request (EPS) Success Rate, Control Plane

KPI Category,Object,

Type, Unit

M Category: Accessibility, Object: EPS (UE R, eNB Uu),


Definition M This KPI is defined as the ratio between successfully established EPS bearers

compared to the overall number of EPS bearer establishment attempts. It corre-

sponds to the probability that a user or the LTE network can establish an EPS bearer

at any moment in time.

Requests that are terminated by timer expiry (due to the unaccessibility of some LTE

resource) are considered as unsuccessful attempts. Authentication errors (requests

rejected by the MME) are included in the total number of failures.

Only the first RRC CONNECTION REQUEST is to be considered, since counting

retries of the same message would increase the overall number of bearer establish-

ment attempts, and thus reduce the success ratio.

Measurement

method

O Accessibility measurement, see Service Accessibility Measurements. Field and lab

trial. Stationary users only. For calculating the success ratio, the same series of

"Service Request (EPS) Time" measurements can be used.

Assumptions,

pre-conditions

O Conditions for accessibility measurements see Service Accessibility Measurements.

See additional conditions in Service Request (EPS) Time, Network Initiated.

Formula (logical) M

Message flow,

trigger points

O The message flow diagram is shown in Figure 7 EPS Bearer Setup Procedure, Part

1 and Figure 8 EPS Bearer Setup Procedure, Part 2. The trigger points for UE initi-ated EPS bearer setup success ratio on the R interface are:

• Success: RRC CONNECTION RECONFIGURATION COMPLETE message

sent by the UE, see EUE in Figure 8 EPS Bearer Setup Procedure, Part 2.

• Total: an attempt is made to send an RRC CONNECTION REQUEST with appro-

priate establishment cause by the UE, see BUE in Figure 7 EPS Bearer Setup

Procedure, Part 1.

Trigger points on the eNB Uu interface:

• Success: RRC CONNECTION RECONFIGURATION COMPLETE message

received by the eNB.

• Total: an attempt is made to receive an RRC CONNECTION SETUP

COMPLETE MESSAGE by the eNB with L3 NAS: SERVICE REQUEST in its

body.

The measurement on the eNB side is an approximation of the success ratio as per-

ceived by the end-user, since it does not include SRB setup failures. The UE side

delivers more precise results.The exact calculation of network initiated EPS bearer

setup success ratios uses the triggers BN and EN, which are also shown in Figure

7 EPS Bearer Setup Procedure, Part 1 and Figure 8 EPS Bearer Setup Procedure,

Part 2.

EPSSRnumber_of(RRC_CONN_RECONFIGURATION_COMPLETE)

number_of(RRC_CONNECTION_REQUEST)----------------------------------------------------------------------------------------------------------------------------------------------------------------------- 100%⋅=



DN0972412


59


Id:0900d805807a3c48

Using online PM measurements on the eNB to count the total number of EPS bearer

setup requests (EPS_SETUP_ATT ) and the successfully terminated ones

(EPS_SETUP_SUCC ), an EPS setup success rate can be calculated (see trigger points

T1 and T2). This ratio is, however, not identical to the user perceived success rate, since

the latter includes additional failure causes, like failures in setting up the SRB signaling

bearer and the ePC part of the EPS bearer, as well as UE authentication errors.

Related KPIs O Service Request (EPS) Time, user and network initiated.The "E-RAB Accessibility"

KPI of [3GPP32.450], see References 17. chapter 6.1.1 corresponds well with this

KPI, considering explicitly created "additional" EPS bearers:

For initial EPS bearers, see KPI Attach Success Rate.

Related PM counters O a) 3GPP defines the following measurements for additional EPS bearers:

h. Additional EPS setup attempts: SAEB.NbrAttEstabAdd.QCI

i. Successful setups: SAEB.EstabAddSuccNbr.QCI.

j. Failures: SAEB.EstabAddFailNbr.Cause

For initial EPS bearer setup measurements, see Attach procedure in Attach

Success Rate.

b) The corresponding eNB counters are:

• EPS_SETUP_ATT, EPS_SETUP_SUCC, EPS_SETUP_FAIL each counts

both initial and additional EPS setups (i.e. no differentiation is made). See

also trigger points T1 & T2 in Figure 8 EPS Bearer Setup Procedure, Part 2.



60 DN0972412



Id:0900d805807a3c48

3.4.1.7 Service (EPS Bearer) Drop Rate

Service (EPS Bearer) Drop Rate, Control Plane

KPI Category, Object, Type, Unit M Category: Availability, Object: EPS (UE R, eNB Uu),


Definition M It is the ratio between abnormally released bearers and the overall

number of established EPS bearers. An abnormal release is defined as

any EPS bearer termination that was not triggered by the mobile user

(from UE side). Thus, it reflects the probability that an established

bearer is aborted due to insufficient network resources.

Dropping the bearer becomes visible to the end-user if an application

service is actively using it. If the application automatically re-estab-

lishes the bearer, it remains unnoticed by the user.



Assumptions, pre-conditions O Conditions for availability measurements see Service Accessibility

Measurements.The ratio is calculated on the same series of Service

Request (EPS) Time measurements (see Service Request Time

(EPS), UE Initiated and Service Request (EPS) Time, Network Initi-

ated).

Formula (logical) M

Message flow, trigger points O Trigger points on UE R / eNB Uu interfaces:

• Success: submission of the message RRC CONNECTION

RECONFIGURATION COMPLETE by the UE, see References26.

[3GPP36.331], or reception by the eNB.

• Drop: successful establishments minus terminations by the user,

i.e. the UE submitting a L3DETACH REQUEST (see References 4.

[3GPP24.008]) carried within the RRC message UL NAS TRANS-

PORT (see References 26. [3GPP36.331]), or the eNB receiving

this message.

Measurements on the eNB side produce equivalent results to measure-

ments on the UE side. The drop ratio calculated this way is valid for

both UE initiated EPS bearers.

For the calculation of a network initiated EPS bearer drop rate, ePC ini-

tiated L3 DETACH REQUEST -s needs to be considered.

EPSBearerD Rnumber_of(dropped_calls)

number_of(successful_calls)------------------------------------------------------------------------------ 100%⋅=



DN0972412


61


Id:0900d805807a3c48

The EPS bearer release rate determined by online PM counters on the eNB is equiva-

lent to the value of the Service (EPS) Drop Rate KPI.

Related KPIs O a) Service Request (EPS) Success Rate.

b) The "E-RAB Retainability" KPI of [3GPP32.450], see References

17. in chapter 6.2.1, is an abnormal EPS release rate, which is

related to the above Drop Rate definition, but also considers thesession duration (its unit is [1/s]):

The necessary measurements are defined by 3GPP (see below).

Related PM counters O a) The following online PM counters of the eNB can be used to calcu-

late the Service Drop Rate:

EPS_SETUP_SUCC, EPS_REL_EPC_INI, EPS_REL_ENB_INI.b) 3GPP defines the following measurements for an abnormal EPS

bearer release: SAEB.RelAct.QCI, SAEB.SessionTimeUE and

SAEB.SessionTimeQCI.QCI

R2

SAEB.RelAct.[QCI]

QC I

∑

SAEB.SessionTimeUE--------------------------------------------------------------=



62 DN0972412



Id:0900d805807a3c48

3.4.1.8 Handover Procedure Time

Handover Procedure Time, Control Plane

KPI Category, Object,

Type, Unit

M Category: Mobility (Latency), Object: LTE (UE R interface),

Type: Mean and max (95%) value, Unit: [ms]

Definition M It denotes the total time needed for the hand-over procedure as seen by the

UE. It begins by receiving a Handover Command from the SeNB and ends by

sending the Handover Confirm response to the TeNB by the UE. Its relevance

is the discontinuity of the IP packet f low in the user plane that is implied by it

(also called service interruption). The value of the HO Procedure Time KPI

depends on the hand-over scenario. The following HO scenarios are distin-

guished (though not directly seen by the UE):

• intra LTE intra- and inter-frequency mobility

• inter RAT mobility (LTE ↔ 2G/3G) • intra vs. inter eNB, the latter via X2, or S1 interface

• intra vs. inter MME/S-GW

Remark: The value of the HO Procedure Time itself does not depend on the

direction of an ongoing data transfer (UL, or DL). It is the LTE Service Interrupt

Time (HO) which will be different.

Measurement method O Mobility (HO) measurement, see Mobility (Handover) Measurements. Lab

and field trials, mobile user scenarios only, crossing of cell borders included.

The handover time should be measured on the UE R interface. However,

measurements on the source eNBs also provide good approximations of the

handover time as perceived by the UE. They even have some advantage,

since the SeNB sees the full handover procedure, the handover preparationphase included.

End-user delays can be measured with test equipment only, since terminals

do not indicate cell changes toward the end-user.

Assumptions, pre-conditions O Conditions for handover measurements see Mobility (Handover) Measure-

ments. UE status before measurement: registered and connected.

Formula (logical) M Mean value and 95% from all measured samples.

Message flow, trigger points O The HO procedure time is seen by the UE as the elapsed time between (see

message flow in Figure 9 Handover Procedure: Intra MME/S-GW, Inter eNBvia X2):

• BUE: UE receives Handover Command in the body of an RRC Connection

Reconfiguration request from source eNB (SeNB).

• EUE: UE sends Handover Confirm to the target eNB (TeNB) encoded in

an RRC Connection Reconfiguration Complete message.

Related KPIs O a) (LTE) Service Interrupt Time, (RB) Service Interrupt Time, HO Success

Rate.

b) No 3GPP defined KPI for HO procedure time found.

Related PM counters O HO Procedure Time related measurements are not defined in 3GPP Release

8 documents.

Handover Procedure Time [ms] tHO_Confirm tHO_Command–=



DN0972412


63


Id:0900d805807a3c48

Figure 9 Handover Procedure: Intra MME/S-GW, Inter eNB via X2



64 DN0972412



Id:0900d805807a3c48

Figure 10 Handover Procedure: Intra MME/S-GW, Inter eNB via S1



DN0972412


65


Id:0900d805807a3c48

3.4.1.9 Handover Success Rate

Handover Procedure Time, Control Plane


Type, Unit

M Category: Mobility (Accessibility), Object: eNB (X2 interfaces),


Definition M The Handover Success Rate is the ratio between successfully executed (commit-

ted) HO procedures and the number of all Handover attempts.

Measurement method O Mobility (HO) measurement, see Mobility (Handover) Measurements. Field and

lab trial.

For calculating the success ratio, the same series of "Service Interrupt Time" mea-

surements can be used. For restrictions in measurement scenarios, refer to the

description of the KPI "(LTE) Service Interrupt Time".

The measurement is to be executed for different HO scenarios (see description

with the KPI Handover Procedure Time).

If practical limitations make it difficult carry out a sufficient number of performance

tests with terminals, eNB online statistics can be used as approximations (as

described below). In case of Inter-RAT scenarios, the eNB statistics cover the LTE

leg of the hand-off only. HO failures in the 2G/3G leg have to be collected with cor-

responding statistics of the BTS/NodeB.

Assumptions,

pre-conditions

O Conditions for mobility measurements see Mobility (Handover) Measurements.

See additional conditions at the KPI “(LTE) Service Interrupt Time".

Formula (logical) M

Message flow,trigger points

O The success rate perceived by end-users is approximated with the success rateas seen on the source eNB. Trigger points on the eNB X2 interface:

• Total #: Handover Request sent by SeNB to TeNB (X2).

• Success #: UE Context Release message received by SeNB from TeNB.

In case of Intra eNB HO procedures, no Handover Request is sent, both trigger

points are to be counted by the (Source) eNB internally.

Related KPIs O a) Handover Procedure Time

b) “E-UTRAN Mobility Success Rate" in [3GPP32.450], see References17.

Chapter 6.5.1, is a success rate, which applies to all Intra- and Inter-RAT

mobility scenarios:

The KPI is calculated by multiplying the success rates for HO preparation and

execution phases. The relationship of variables in this expression and the HO

measurements defined in [3GPP32.425],see References 15., is still to be clar-

ified.

HOSRnumber_of(Handover_Confirm)

number_of(Handover_Request)-------------------------------------------------------------------------------------- 100%⋅=



66 DN0972412



Id:0900d805807a3c48

The calculation of HO Success Rate does not differentiate between different causes of

a negative hand-over decision. All rejected HO attempts are considered as unsuccessful

handovers.

Related PM counters O a) 3GPP defines measurements / counters on eNB to support HO success ratio

calculations (see References 15. [3GPP32.425] chapter 4.3). Examples for

selected scenarios:

Intra-RAT, intra-eNB handovers: • # HO attempts: HO.IntraENBOutAtt.Cause

• # HO success: HO.IntraENBOutSucc.Cause

where Cause identifies the cause for handover

Intra-RAT, inter-eNB handovers:

• # HO attempts: HO.InterENBOutPrepAtt

• # HO attempts: HO.InterENBOutAtt.Cause

• # HO success: HO.InterENBOutSucc.Cause

Handover measurements on neighbor cell basis:

• # HO attempts: HO.OutAttTarget.Cause

• # HO success: HO.OutSuccTarget.Cause

Inter-RAT handovers (LTE -> 2G/3G):

• # HO attempts: HO.IartOutAtt.Cause

• # HO success: HO.IartOutSucc.Cause

b) LTE product related counters on eNB which support the calculation of

handover success rates:

Intra-RAT, intra-eNB handovers:

• # HO attempts: INTRA_HO_PREP

• # HO success: INTRA_HO_SUCC

Intra-RAT, iner-eNB handovers:

• # HO attempts: INTER_HO_PREP

• # HO success: INTER_HO_SUCC

Inter-RAT handovers (LTE -> 2G/3G):

• # HO attempts: INTER_RAT_HO_PREP

• # HO success: INTER-RAT_HO_SUCC



DN0972412


67


Id:0900d805807a3c48

3.4.1.10 Paging Time

Paging Time, Control Plane

KPI Category, Object, Type, Unit M Category: Latency, Object: LTE (MME, S-GW; S11, S1-U),

Type: Mean and max (95%) value, Unit: [s]

Definition M Paging is initiated :

• By the arrival of a DL packet at the S-GW for an UE whose location

is not known, i.e. its location information does not exist in the S-GW,

or

• By network initiated EPS bearer setup requests (ISR flag is set)

It denotes the total procedure time from starting the paging request in

DL and terminating it with the subsequent service request (EPS bearer

setup) of the UE after it has been located. Thus, the paging time is per

definition the difference between network and UE initiated ServiceRequest Times.

Trigger events can be defined on the MME and S-GW as initiators of

the paging request, either on the S11 (MME - S-GW) or on the S1-U

(UE - S-GW) interfaces, see "Trigger points" below.

Measurement method O Latency measurement, see Latency Measurements. Lab trial with

mobile user scenario only, crossing of cell borders included.

The paging time can be measured on the S11 (MME - S-GW), or S1-U

(UE - S-GW) interfaces. End-user delays cannot be measured, since

paging is always initiated by the network.



UE status before measurement: registered, but idle (not connected).

Formula (logical) M The paging time is calculated as the difference between network initi-

ated service request and UE initiated service request times.

Message flow, trigger points O See message flow in Figure 11 Paging Procedure.The paging time can

be approximated on the MME S11 with:

• Start: L2 SCTP: DL Data Notification received for an UE

• Stop: L3 NAS: SERVICE REQUEST received from the same UE

On S-GW S1-U, latency is measured between:

• Start: DL packet arrives for an idle UE

• Stop: DL packet is forwarded to the connected UE

The latter latency is a good metric of DL "service interrupt time" for an

IP packet stream caused by paging. The paging time has to be calcu-

lated as the measured latency minus the UE initiated Service Request

Time (EPS).

Related KPIs O a) Service Request Time (EPS), UE and network initiated.

b) No 3GPP Release 8 defined KPIs for Paging Time found.

P agi ng Time [s] tSRT network initiated tSRT UE initiated–=



68 DN0972412



Id:0900d805807a3c48

Figure 11 Paging Procedure

Related PM counters O Paging Time related measurements are neither defined in 3GPP

Release 8, nor in LTE reference documents.



UE EMM-REGISTERED

and ECM-IDLE

DL Packet

L2 SCTP: DL Data

Notification

RRC: DL INFORMATION

TRANSFER

(L3 NAS: PAGING)

D

e

l

ay

L2 SCTP: DL Data

Notification Ack

Paging Procedure

[3GPP TS 36.413]

S1-AP: DL NAS

TRANSPORT

(L3 NAS: PAGING)

To all cells of all TAs

UE in EMM-REGISTERED

and ECM-CONNECTED

UE Triggered Service

Request Procedure

[3GPP TS 23.401]

Forward DL Packet to UE

L2 SCTP: Stop Paging

BN

EN

T1

T2L2 SCTP: DL Data

Notification Reject

Retry (optionally)

T

i

m

e

r

Paging Failure

Paging Attempt



DN0972412


69


Id:0900d805807a3c48

3.4.1.11 Paging Failure Rate

Paging Failure Rate, Control Plane

KPI Category, Object, Type, Unit M Category: Accessibility, Object: MME (S1-AP, S11),


Definition M The Paging Failure Rate is the ratio between unsuccessful paging

requests and the number of all paging attempts initiated by the MME.

Retries of the same paging request by the MME are not counted as new

attempts. Similarly, multicasting the same request to more than one

eNBs (in the UE´s tracking areas) is considered as one attempt.

Measurement method O Accessibility measurement, see Service Accessibility Measurements.

Field and lab trials.

For calculating the failure ratio, the same series of "Service Request

(EPS) Success Rate" measurements can be used, however, restricted

to the network initiated bearer setup scenario. For restrictions in mea-

surement scenarios, see also the KPI “Service Request Success Rate".

The existence of online performance counters for paging on the MME

can help to avoid the execution of performance tests repeatedly.

Suitable counters are indicated in the message flow in Figure

11 Paging Procedure (trigger points T1 and T2).

Assumptions, pre-conditions O Conditions for accessibility measurements, see Service Accessibility

Measurements.

See additional conditions at the KPI "Service Request Success Rate".

Formula (logical) M

Message flow, trigger points O Trigger points on MME (see T1: Attempt and T2: Failure on the Figure

11 Paging Procedure):

• Total #: different L3 NAS Paging Requests sent to an eNB (S1-AP).

• Failure #: DL Data Notification Reject messages sent to S-GW

(S11)

Related KPIs O a) Paging Time, Service Request Success Rate - network initiated.

b) No Paging related KPI is defined in 3GPP Release 8 documents.

Related PM counters O Directly applicable trigger events are T1 and T2 as shown in Figure

11 Paging Procedure.

a) 3GPP defined counter (see References15. [3GPP32.425] Chapter

4.7) on eNB is:

• Discarded Paging Records: PAG.DiscardedNbr

b) LTE product related counters defined for eNB in:

• Requests: RRC_PAGING

• Responses: SIG_CON_EST_ATT.Cause = PagingResponse

Using these counters, the Paging Failure Rate can be calculated as:

Paging FR =( 1 - SIG_CON_EST_ATT.Cause = PagingResponse /

RRC_PAGING *100%)

PagingFR number_of(Paging_Failures)number_of(Paging_Attempts)-------------------------------------------------------------------------------- 100%⋅=



70 DN0972412



Id:0900d805807a3c48

3.4.1.12 (LTE) Round Trip Time

Round Trip Time (RTT), User Plane

KPI Category, Object, Type, Unit M Category: Latency, Object: LTE (UE R, P-GW Gi interfaces),

Type: min (CDF 5%), mean (CDF 50%), max (CDF 95%) values,

Unit: [ms]

Definition M RTT in UL is the interval between sending a datagram by the UE &

receiving the corresponding reply from an IP peer entity connected to

the Gi interface of the P-GW.

RTT in DL is the interval between sending a datagram to the UE &

receiving the corresponding reply by the IP host (peer entity).

Measurement method O Latency measurement, Latency Measurements. Field and lab trials.

Stationary users in different cell positions, which are uniformly distrib-

uted across the cell.

RTT is measured with the Ping application between the UE and an IP

host (peer entity). As average RTT figure, the output of the Ping appli-

cation is used (e.g. "MS-DOS >ping -n count -l size host_name"). Size

of ICMP packets: 32, …, MTU, 2x MTU.

Ping should be executed at least 100 times to reduce the impact of the

first ICMP message, which triggers the setup of radio bearer establish-

ments in UL and DL directions.



UE status before measurement: registered and connected.

The IP peer entity should be located as close as possible to the SGi

interface. Additional delays between the P-GW and the IP host should

be avoided or minimized.

Formula (logical) M Mean value and 95% from all measured samples.

Message flow, trigger points O Trigger points both from UE, and from IP peer entity view:

• Start: ICMP ECHO REQUEST

• Stop: ICMP ECHO REPLY

Related KPIs O a) Network Delay UL / DL KPI for LTE, RB and IP based transport

bearers X2, S1, S5/S8)

b) No 3GPP defined KPI for LTE RTT found.

c) See "{Service} Round Trip Time" KPI definition for Ping service in

[ETSI102.250-2], see References 39. Chapter 4.6.3.6.

Related PM counters O Measurements of round trip times are neither defined in 3GPP nor in

LTE reference documents.

Round Trip Time [ms] tICMP Echo Reply tICMP Echo Request–=



DN0972412


71


Id:0900d805807a3c48

3.4.1.13 (LTE) User Data Rate

The User Data Rate KPI has been defined on E2E network level to verify if the capacity

of large non-GBR EPS bearers (QoS profile with large maximum bit rates) can in fact beexploited by IP based services. The same holds for the E-RAB and RB bearers being

part of the EPS. Especially, the maximum achievable data rate of the RB bearer is inter-

esting, since it is the critical resource of the LTE network. For this reason, the Peak Data

Rate is defined as a separate KPI on the RB level (see (RB) Packet Loss Rate), too.

Because of this relationship (containment) between EPS, E-RAB and RB, user data rate

KPIs can be verified with the same series of measurements. The KPIs are interesting as

values calculated over the full duration of long data transfers (~minutes). Of equal

interest are peak values observed for a short time period only (~seconds).

Please note that data rate KPIs have also been defined for application services, e.g.

User Data Rate KPI of the Data Download / Upload service, but these do not character-

ize the capability of the LTE network alone, but also depend on the application service

and its protocols.

User Data Rate UL / DL, User Plane


Type, Unit

M Category: Throughput, Object: LTE (UE R if, IP peer entity)

Type: Min (CDF 5%), Mean (CDF 50%) Max (CDF 95%) values, Unit: [Mbps]

Definition M The metric describes the data speed available to one user of the LTE network on

UDP/IP level. It is given as the maximum (95%-ile) value that can be observed over

a short period of time (e.g. of 1s) and as a mean value that characterizes longer data

transfer periods (minutes). Its value distribution over the radio cell is given as a

function of the SINR. The maximum value is often referred to in the literature as

instantaneous "Peak Throughput" that is achieved in optimal radio conditions. The

user data rate can be given for a single user active in the cell (single user data rate),

or to one of several concurrently active users.

Measurement method O Throughput measurement, see Throughput Measurements. Stationary and mobile

user. Cell positions: best, medium, cell edge, including different HO scenarios. The

data rate is measured in UL/DL direction with UDP/IP traffic over a time period of

several minutes. Signaling delays, e.g. initial radio bearer setup has no impact on the

result, because transient times at the beginning and end of the data transfer are not

considered.

Assumptions,

pre-conditions

O Conditions for throughput measurements see Peak User Data Rate.

The IP host used as traffic generator (e.g. with Iperf) should be connected directly to

the SGi interface. Additional delays and bandwidth restrictions between the S/P-GW

and the server need to be excluded.

Formula (logical) M

Message flow,

trigger points

O Trigger points on the UE R interface:

• UL: UE sending UDP/IP packets to the IP peer entity.

• DL: IP peer entity sending UDP/IP packets to the UE.

Related KPIs O (RB) User Data Rate, (FTP) User Data Rate.

Related PM counters O Measurements of user data rate on LTE level (R <-> Gi) are neither defined in 3GPP,

nor LTE reference documents.

UserDataRate trans fer red_data_volume [bytes] 8⋅transfe r_time [s] 1000⋅

---------------------------------------------------------------------------------------------------=



72 DN0972412



Id:0900d805807a3c48

3.4.1.14 (LTE) Packet Loss Rate

(LTE) Packet Loss Rate UL / DL, User Plane

KPI Category, Object, Type, Unit M Category: Availability, Object: LTE (UE R interface, IP peer entity)

Type: Ratio, Min (CDF 5%), Mean (CDF 50%) Max (CDF 95%) values

Unit: [%]

Definition M This is the ratio between the numbers of lost or corrupted IP packets,

and of all IP packets sent. Corrupted IP packets are those that contain

bit errors in their headers or in their payload.Packets with "residual", i.e.

undetected errors are not counted as lost.


The EPS bearer is to be configured such that corrupted IP packets are

not delivered to the application.

Measured in loaded and unloaded network, under different radio condi-

tions. Stationary and mobile users.

The packet loss rate should be measured in test scenarios, where han-

dovers occur. The number of handovers should be oriented toward the

NSN reference traffic model, or, in operator trials, toward the operator´s

traffic model.

Assumptions, pre-conditions O Conditions for availability measurements, see Service Accessibility

Measurements.

Measured in loaded and unloaded network, under different radio condi-

tions. Stationary and mobile user including HO scenarios.

The ratio can be calculated on a series of User Data Rate (see Peak

User Data Rate) measurements.

Formula (logical) M

Message flow, trigger points O UL / DL:

• Total: packages sent at UE R if / IP peer entity

• Lost: Total minus packets received at IP peer entity / UE R if.

Related KPIs O a) (RB) Initial Radio Block Error Rate (BLER)

b) No PLR KPI (LTE level) is defined in 3GPP and ETSI documents.

Related PM counters O Measurements of packet loss rate on LTE level (R <-> Gi) are neither

defined in 3GPP nor in LTE reference documents.

PLRnumber_of(lost_corrupted_packets)

number_of(all_packet_sent)------------------------------------------------------------------------------------------------ 100%⋅=



DN0972412


73


Id:0900d805807a3c48

3.4.1.15 (LTE) Service Interrupt Time (HO)

(LTE) Service Interrupt Time (HO), User Plane

KPI Category, Object, Type, Unit M Category: Latency, Object: LTE (UE R if)

Type: Mean and max (95%-ile) value, Unit: [ms]

Definition M The Service Interrupt Time is the interval between the last

sent/received IP packet of a continuous UL/DL data stream in the old

cell and the first sent/received user IP packet in the new cell measured

on the UE (also called "user plane break").

The value of the KPI depends of the handover scenario. See HO sce-

narios at the KPI Handover Procedure Time.

Measurement method O Mobility (HO) measurement, see Mobility (Handover) Measurements.

Only mobile user scenarios including cell border crossing. The test

application is an FTP upload/download service, or an isochronous

UP/IP flow (iperf) maintained during handover.

Assumptions, pre-conditions O Conditions for availability measurements, see Mobility (Handover)

Measurements.

In inter eNB "HO via X2" scenariosthe X2 interface shall be enabled

between the source and target eNBs.

Formula (logical) M

Mean value and 95% from all measured samples.

Message flow, trigger points O The trigger points for service interrupt time (from UE point of view):

• BUE: UE sends/receives last packet to/from source eNB

• EUE: UE sends/receives first packet to/from target eNB.

See Figure 9 Handover Procedure: Intra MME/S-GW, Inter eNB via X2

Related KPIs O a) HO Procedure Time, Voice Interrupt Time (HO)

b) No related KPI definitions found in 3GPP Rel. 8 documents.

Related PM counters O Measurements of service interrupt times caused by HO are neither

defined in 3GPP nor in LTE reference documents.

Service Interrupt Time [ms] t fir st packet to/from TeNB tla st packet to/from SeNB–=



74 DN0972412



Id:0900d805807a3c48

3.4.2 Radio Bearer KPIs

3.4.2.1 (RB) Packet Loss Rate

(RB) Packet Loss Rate UL / DL, User Plane

KPI Category, Object, Type, Unit M Category: Availability, Object: RB (UE R interface, eNB)

Type: Ratio, Min (CDF 5%), Mean (CDF 50%) Max (CDF 95%) values

Unit: [%]

Definition M This is the ratio between the numbers of lost or corrupted IP packets,

and of all IP packets sent. Corrupted IP packets are those that contain

bit errors in their headers or in their payload.

Packets with "residual", i.e. undetected errors are not counted as lost.

Measurement method O The KPI is not measured on its own. Traces are to be taken on the UE

and eNB during (LTE) Packet Loss Rate measurements, see (RB)Packet Loss Rate. Lab trials with results for internal use.

Assumptions, pre-conditions O See comment above.

Formula (logical) M

Message flow, trigger points O UL / DL:

• Total #: packages sent on UE R / eNB

• Lost #: Total minus packets received on eNB / UE R

Measured at the upper SAPs of the L2 PDCP.

Related KPIs O a) Initial Radio Block Error Rate (BLER)

b) No PLR KPI (RB level) is defined in 3GPP and ETSI documents.

Related PM counters O a) The following measurements are defined in LTE for the number of

discarded (dropped) PDUs on the eNB in DL:

• Discarded PDCP SDUs: PDCP_SDU_DISCARD

• Discarded RLC blocks: SDU_DISCARD_DL_DTCH

b) The following measurements are defined by 3GPP in

[3GPP32.425], see References 15. chapters 4.4.3-4, for packet

(PDCP SDU) loss and drop rates:

• Drop rate on eNB in DL: DRB.PdcpSduDropRateDl.QCI

• Loss rate on Uu in DL: DRB.PdcpSduAirLossRateDl.QCI

• Loss rate on Uu in UL: DRB.PdcpSduLossRateUl.QCI

A packet drop rate on eNB in UL is not defined.The loss rate values

may refer to the L2 protocols PDCP, RLC or MAC (see

[3GPP32.314] chapter 4.1.5.1-3)

RBPLRnumber_of(lost_corrupted_packets)

number_of(all_packet_sent)------------------------------------------------------------------------------------------------ 100%⋅=



DN0972412


75


Id:0900d805807a3c48

3.4.2.2 (RB) User Data Rate

(RB) User Data Rate UL / DL, User Plane

KPI Category, Object, Type, Unit M Category: Throughput, Object: RB, eNB,

Type: Min (CDF 5%), Mean (CDF 50%), Max (CDF 95%) values,

Unit: [Mbps]

Definition M The metric describes the UDP/IP data rate achievable by one user. It

can be given as single user data rate if only one user is active in the

cell, or as multi-user data rate for a given number of concurrently active

users.

The user data rate distribution over the cell is given as function (CDF)

of the radio conditions characterized by the SINR.

The time variation of the KPI value in a given cell position is given with

its maximum (95%) (also referred to as instantaneous "Peak User Data

Rate"), and with its average (mean) value.The Peak User Data Rate corresponds to the Cell Throughput KPI on

UDP/IP level, when the number of active users in the cell is n=1 and

best radio conditions are observed.

The User Data Rate is also called "User Throughput" in the literature.

Measurement method O Throughput measurement, see Throughput Measurements. Stationary

and mobile user. Cell positions: best, medium, cell edge, including dif-

ferent HO scenarios. Field and lab trials.

The eNB assigns all resources (radio blocks) to this user, except for the

radio blocks used for signaling and controlling. Measured on UDP/IP

level for all UE categories (1-5). With the help of trace analysis, PDCP,

RLC, MAC and PHY layer throughputs can also be defined.With and w/o concurrency in UL / DL traffic.

The mean user data rate is calculated over a few minutes, peak data

rate values are averaged over short periods of time (e.g. 1s).

Assumptions, pre-conditions O Conditions for capacity measurements, see Peak User Data Rate.

All transport bearers between the UE and IP peer entity should have

higher capacity than the radio link in order to avoid bandwidth bottle-

necks.

Formula (logical) M

Message flow, trigger points O Trigger events on the eNB:

• UL: eNB receiving PDUs

• DL: eNB sending PDUs

Measured at the upper SAPs of the protocols PDCP, RLC, MAC or

PHY.

Related KPIs O a) (LTE) User Data Rate, Cell Throughput.

b) See 3GPP defined KPIs in Cell Throughput.

Related PM counters O See 3GPP defined measurements with the KPI in Cell Throughput.

RBUserDataRatetransferred_data_volume [bytes] 8⋅

transfer_time [s] 10

6–

⋅

-------------------------------------------------------------------------------------------------=



76 DN0972412



Id:0900d805807a3c48

3.4.2.3 Cell Throughput

Cell Throughput UL / DL, User Plane

KPI Category, Object, Type,

Unit

M Category: Throughput, Object: UE, RB Uu,

Type: Min (CDF 5%), Mean (CDF 50%), Max (CDF 95%) values, Unit: [Mbps]

Definition M The metric shows the sustainable aggregate throughput of the cell (in UL/DL)

available to "n" stationary users distributed uniformly in the cell and running a

typical mix of applications. The "cell throughput" is the sum of all bits trans-

ported in all radio blocks carrying PDUs (i.e. bits in UL-SCH / DL-SCH trans-

port blocks) during one second. The cell capacity is also given as peak value

(called peak cell capacity, or throughput), which is defined as the aggregate

throughput of "n" users all located in best radio conditions. The cell throughput

value is defined here on PHY level, but could be given for other protocol levels

(UDP/IP, PDCP, RLC, MAC), too. When the (peak, average) cell throughput

is expressed on UDP/IP level, it corresponds to the (peak, mean) user datarate value at comparable radio conditions.

Measurement method O Throughput measurement, Throughput Measurements. Stationary users uni-

formly distributed in the cell. UE categories according to application mix. Peak

value measured with one user in LoS cell position using UDP/IP load to

approach the full buffer condition.

Field trial without target value (only to learn the available capacity).

Measured with UDP/IP traffic on PHY layer (UL-SCH / DL-SCH transport

block bits) over several minutes. With the help of trace analysis PDCP, RLC

and MAC layer values can also be given.

Assumptions, pre-conditions O Conditions for throughput measurements see Attach Success Rate. All IP

transport bearers (S1, S5/S8, SGi) between the UE and IP peer entity should

have higher bandwidth than the cell capacity to avoid bandwidth bottlenecks.

Formula (logical) M

Message flow, trigger points O Trigger points on the eNB Uu interface:

• UL: eNB receiving PDUs / RLC blocks / transport data blocks

• DL: eNB sending PDUs / RLC blocks / transport data blocks

Related KPIs O a) (RB) User Data Rate

b) 3GPP defines the KPI "E-UTRAN IP Throughput" in [3GPP32.450], seeReferences 17. chapter 6.3.1.

CellThroughputtransferred_data_volume [bytes] 8⋅

transfer_time [s] 106–

⋅

-------------------------------------------------------------------------------------------------=



DN0972412


77


Id:0900d805807a3c48

Related PM counters O a) The following measurements are available on eNB in LTE reference doc-

ument, which are related to the Cell Throughput KPI:

• CELL_THROUGHPUT_PDCP (separate counters for min, mean and

peak values, both for UL and DL on Uu interface) • PDCP_SDU (same for one cell)

b) The following measurements are defined by 3GPP in [3GPP32.425]

Chapters 4.4.1 for cell data (PDCP SDU) rates:

• Average data rate in DL: DRB.PdcpSduBitrateDl.QCI

• Maximal data rate in DL: DRB.PdcpSduBitrateDlMax

• Average data rate in UL: DRB.PdcpSduBitrateUl.QCI

• Maximal data rate in UL: DRB.PdcpSduBitrateUlMax

The data rate values are measured on the eNB at the upper SAP of L2 PDCP

and refer to SDU bits. SDUs that are forwarded over X2/S1 to another eNB

are subtracted from the overall bit count.



78 DN0972412



Id:0900d805807a32b8

References

4 References

1. [3GPP21.905] 3GPP TR 21.905; v 8.x.x; Vocabulary for 3GPP Specification

2. [3GPP23.203] 3GPP TS 23.203; V8.7.0 (2009-09); Policy and charging controlarchitecture

3. [3GPP23.401] 3GPP TR 23.401; v.8.x.x; General Packet Radio Service (GPRS)

enhancements for Long Term Evolution (LTE) access

4. [3GPP24.008] 3GPP TR 24.008; V8.5.0 (2009-03); Mobile Radio Interface Layer 3

Specification; Core network protocols; Stage 3

5. [3GPP24.228] 3GPP TR 24.228; V5.15.0 (2006-10); Signaling flows for the IP mul-

timedia call control based on Session Initiation Protocol (SIP) and Session Descrip-

tion Protocol (SDP);Stage 3 (Release 5)

6. [3GPP24.229] 3GPP TR 24.229; V8.8.0 (2009-06);IP multimedia call control

protocol based on Session Initiation Protocol (SIP) and Session Description

Protocol (SDP);Stage 3 (Release 8)7. [3GPP25.913] 3GPP TR 25.913; V8.0.0 (2008-12); Requirements for Evolved

UTRA (E-UTRA) and Evolved UTRAN (E-UTRAN)

8. [3GPP25.943] 3GPP TR 25.943; V8.0.0 (2008-12); Deployment Aspects (Release

8)

9. [3GPP26.114] 3GPP TS 26.114 V8.2.1 (2009-03); IP Multimedia Subsystem (IMS);

Multimedia Telephony; Media handling and interaction (Release 8)

10. [3GPP29.002] 3GPP TS 29.002, V8.x.x; Mobile Application Part (MAP)

11. [3GPP29.060] 3GPP TS 29.060; v 8.x.x; General Packet Radio Service (GPRS);

GPRS Tunneling Protocol (GTP) across the Gn and Gp interface

12. [3GPP32.410] 3GPP TS 32.410; V8.0.0 (2009-03); Telecommunication manage-

ment; Key Performance Indicators for UMTS and GSM (Release 8)13. [3GPP32.421] 3GPP TS 32.421, V8.4.0 (2008-12); Telecommunication manage-

ment; Subscriber and equipment trace; Trace concepts and requirements

14. [3GPP32.423] 3GPP TS 32.423, V8.0.0 (2009-03); Telecommunication manage-

ment; Subscriber and equipment trace, Trace data definition & management


ment; PM Performance Measurements, E-UTRAN (Release 8)


ment; Performance Management (PM); Performance measurements, Evolved

Packet Core network (EPC)


ment; Key Performance Indicators for E-UTRAN: Definitions (Release 8)18. [3GPP32.451] 3GPP TS 32.451, V8.0.0 (2009-03); Telecommunication manage-

ment; Key Performance Indicators for E-UTRAN: Requirements (Release 8)


ment; UTRAN and GERAN Key Performance Indicators (KPI), (Release 7)

20. [3GPP36.300] TS 36.300; v 8.x.x; Evolved Universal Terrestrial Radio Access (E-

UTRA) and Evolved Universal Terrestrial Radio Access Network (E-UTRAN);

Overall description; Stage 2

21. [3GPP36.306] TS 36.306; V8.3.0 (2009-03); Evolved Universal Terrestrial Radio

Access (E-UTRA); User Equipment (UE) Radio Access Capabilities


Access (E-UTRA); Physical Channels and Modulation




References


Access (E-UTRA); Multiplexing and channel coding


Access (E-UTRA); Physical layer - Measurements25. [3GPP36.314] TS 36.314; V8.1.0 (2009-03); Evolved Universal Terrestrial Radio

Access (E-UTRA); Layer 2 - Measurements


Access (E-UTRA); Radio Resource Control (RRC) protocol specification

27. [3GPP36.401] TS 36.401; v 8.x.x; Evolved Universal Terrestrial Radio Access

Network (E-UTRAN); Architecture description

28. [3GPP36.413] TS 36.413; V8.6.1 (2009-06); Evolved Universal Terrestrial Access

Network (E-UTRAN) ; S1 Application Protocol (S1 AP)

29. [3GPP36.423] TS 36.423; v 8.x.x; Evolved Universal Terrestrial Radio Access

Network (E-UTRAN); X2 Application Protocol (X2AP)

30. [3GPP36.521-1] 3GPP TS 36.521-1 V8.2.1 (2009-06); Evolved Universal TerrestrialRadio Access (E-UTRA) User Equipment (UE) conformance specification, Radio

transmission and reception, Part 1: Conformance Testing


Access (E-UTRA); Radio Frequency (RF) system scenarios

32. [RFC792] IETF RFC 792: Internet Control Message Protocol

33. [RFC979] IETF RCF 979: File Transfer Protocol

34. [RFC2679] IETF RCF 2679: A One-way Delay Metric for IPPM, September 1999

35. [RFC2680] IETF RCF 2680: A One-way Packet Loss Metric for IPPM, September

1999

36. [RFC2681] IETF RCF 2681, A Round-trip Delay Metric for IPPM, September 1999

37. [RFC3261] IETF RCF 3261, SIP - Session Initiation Protocol, June 2002

38. [RFC4960] IETF RFC 4960, Stream Control Transmission Protocol

39. [ETSI102.250-2] ETSI TS 102 250-2 V1.3.1 (2005-07); Speech Processing, Trans-

mission and Quality Aspects (STQ); Part 2: Definition of Quality of Service parame-

ters and their computation

40. [ITUE500] ITU-T Recommendation E.500, Traffic intensity measurement principles,

11/98

41. [ITUE800] ITU-T Recommendation E.800, Definitions of terms related to quality of

service, 09/2008

42. [ITUG107] ITU-T Recommendation G.107 (03/2003), The E-model, a computational

model for use in transmission planning

Nokia Siemens LTE E2E Field Network Performance

Documents

Transcript of Nokia Siemens LTE E2E Field Network Performance