Lte kpi dt guide & measure method.

42
LTE KPI DT GUIDE & MEASURE METHOD SECRET Product name Confidentiality level LTE RNP/RNO Internal Open Product version Total 34 pages LTE KPI DT GUIDE & MEASURE METHOD Prepared by Hu Wei Date 2011-11-29 Reviewed by Date Reviewed by Date Approved by Date Huawei Confidential No spreading without permission Page 1 of 42

Transcript of Lte kpi dt guide & measure method.

Page 1: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

Product name Confidentiality levelLTE RNP/RNO Internal OpenProduct version

Total 34 pages

LTE KPI DT GUIDE & MEASURE METHOD

Prepared by Hu Wei Date 2011-11-29

Reviewed by DateReviewed by DateApproved by Date

Huawei Technologies Co., Ltd.All rights reserved.

Huawei Confidential No spreading without permission Page 1 of 35

Page 2: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

Revision record

Date Revision version change Description

Author

2011-11-29 V 1.0 First version finished

Hu Wei 00135858

Huawei Confidential No spreading without permission Page 2 of 35

Page 3: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

Contents

1 SSV Test........................................................................42 KPI Test.........................................................................63 Measure Method..........................................................33

Huawei Confidential No spreading without permission Page 3 of 35

Page 4: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

1 SSV Test

1) First should check the azimuth and tilt, any problem the work should be

stopped and directly inform Huawei the problems.

2) The DT teams should select points as below (updated: 1,4,7 are canceled,

only 6 points needed) to do the network entry/ Download/ upload/ ping test.

The points location can be changed as the real signal condition.

3) If the teams find any problem (include but not only: wrong BSID or PCI; low

throughput; high ping delay) in the test they should report it to coordinator

right now. Take care on the feeder across! At last finish and submit the SSV

report (refer to the attachment).

4) If there is no cross feeder, the terminal should access to correct BSID and

Huawei Confidential No spreading without permission Page 4 of 35

Page 5: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

correct PCI. If there is cross feeder, please report to Huawei SSV

supervisor. And the remaining test for this site should be stopped.

5) Throughput test should be done with FTP. Currently, there is one Huawei

FTP is available for throughput test. FTP IP: ftp://10.218.240.115/ user

name: huawei, password: huawei; ftp://10.212.195.139/ user name: wimax,

password: wimax. Please use FTP software (suggest to use filezilla,

because it can download several files together.) to download the files (at

least 10 files together) at least for 1 minute. Using first stopwatch of DU

meter to record the average DL/UL throughput during test.

  

6) All the throughput test mentioned in this document must be

performed in this way.

7) Ping test: click “start” button on your desktop and then: Start-> Run-> input

”cmd”-> enter-> input ”ping 10.218.240.115 –n 100” or ”ping

10.212.195.139 –n 100” > enter-> wait until output 10 ping delay result on

the screen-> record the average ping delay

8) Take a snapshot which includes all the above test result (ping and

Huawei Confidential No spreading without permission Page 5 of 35

Page 6: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

download, upload), signal information, PCI, terminal Tx power, etc.

9) Record each point’s coordinates.

Huawei Confidential No spreading without permission Page 6 of 35

Page 7: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

2 KPI Test

If the SSV test is clear, the teams can go ahead to the KPI test. KPI test include

Drier test and Points test.

1) Tools prepared:

i. LTE Dongle (E398s)

ii. Genex Probe(V2R3)

iii. Genex Probe HardwareDog

iv. GPS

2) Softwares prepared:

v. Install software of LTE Dongle and Agent

vi. Inster LTE Dongle in your laptop, you will see the picture on “My

Computer”, then Double-click this picture, auto install the software.

vii. Next phase install “Hisi UE Agent” .

viii. Please confirm the LTE Dongle、GPS and Probe Dog in your laptop.

ix. Open Hisi UE Agent

Huawei Confidential No spreading without permission Page 7 of 35

Page 8: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

x. Open Genex Probe(V2R3)

xi. New LTE template:

xii. Then clean up template like this:

Huawei Confidential No spreading without permission Page 8 of 35

Page 9: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xiii. New Engineering Parameter

xiv. Import File

Huawei Confidential No spreading without permission Page 9 of 35

Page 10: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xv. Select File, the Engineering Paremeter will be provided by huawei.

xvi. Finish Import File

Huawei Confidential No spreading without permission Page 10 of 35

Page 11: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xvii. Engineering Apply

Huawei Confidential No spreading without permission Page 11 of 35

Page 12: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xviii. Device Configure

xix. Add MS

Huawei Confidential No spreading without permission Page 12 of 35

Page 13: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xx. Finish MS Add

Huawei Confidential No spreading without permission Page 13 of 35

Page 14: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxi. Add GPS

xxii. Finish GPS Add

Huawei Confidential No spreading without permission Page 14 of 35

Page 15: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxiii. New Test Plan

Huawei Confidential No spreading without permission Page 15 of 35

Page 16: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxiv. Loading-in Test Plan, the test plan will be provided by huawei. But you

need to change FTP Upload file to your local file(20M).

Huawei Confidential No spreading without permission Page 16 of 35

Page 17: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

Huawei Confidential No spreading without permission Page 17 of 35

Page 18: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxv. Open Map

xxvi. Open Geoset File

Huawei Confidential No spreading without permission Page 18 of 35

Page 19: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxvii. Select Map File(TAR file), the map file will be provided by huawei

Huawei Confidential No spreading without permission Page 19 of 35

Page 20: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxviii. Set Cell Radius

Huawei Confidential No spreading without permission Page 20 of 35

Page 21: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

Huawei Confidential No spreading without permission Page 21 of 35

Page 22: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxix. Set Lable

Huawei Confidential No spreading without permission Page 22 of 35

Page 23: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

Huawei Confidential No spreading without permission Page 23 of 35

Page 24: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxx. Finish all setting, then “connect” to LTE Dongle and GPS.

Huawei Confidential No spreading without permission Page 24 of 35

Page 25: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

xxxi. Please begin the log recording before DT, you need to open test plan, then

begin to log recording and save.

Huawei Confidential No spreading without permission Page 25 of 35

Page 26: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

3) Begin DT

i. The DT will follow before confirmed wimax DT boundary. Using Huawei

E398s USB terminal and Huawei Genex Probe DT software to perform the

DT. During the DT the engineers should parallel perform network entry,

minimum guarantee DownLoad/UpLoad test. By this after the DT can get

the test result of coverage probability/ successful setup rate / session

establish delay/ dropped session rate/ minimum rate guarantee of

downstream&upstream/ handoff/ wireless link burst error rate.

ii. All the streets should be covered. The speed should less than 30km/h. In

the DT engineers should always pay attention on the PCI/RSRP/SINR. If

any problem happened they should click the “pause” button to pause the

test and after solving continue the test. Any problems like KPI fail to

achieve must be informed to Huawei on time. After DT the log should be

transferred to Huawei.

Huawei Confidential No spreading without permission Page 26 of 35

Page 27: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

iii. In the DT the team should always take care the PCI and performance, the

bad performance DT log or wrong test log will need repeating work. It is ok

that feedback the problems to Huawei when you meet some problems.

4) Begin PT

i. Cell edge coverage probability

To the cell edge coverage probability, select 20 edge points (SINR is higher

than 0dB but lower than 5dB & RSRP is lower than -100dB) to perform ftp

download, each point last 1 min. Record the detail coordinates and test results.

The RSRP and SINR must be check from Probe, you can use test plan to

download and take a Snapshots like the “Average downstream throughput –

sector level” pictures, check the RSRP and SINR like this picture:

Acceptable range: successful rate> 80%

Target range: successful rate> 85%

ii. Average downstream throughput – terminal level

To the average downstream/upstream throughput (terminal level), select 3

points of one sector (RSRP=-105~-95 / -95~-80/ >-Huawei Confidential No spreading without permission Page 27 of 35

Page 28: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

80、SINR=0~10/15~25/25~35) to perform the ftp download/ upload for 1 min.

Record the average throughput of the 3 points. This test should be implemented

for 20% of sectors per cluster.

Please pay more attention to 3 points:

RSRP=-105~-95,SINR=0~10, keep the DL throughput between 5M to 15M

RSRP=-95~-80, SINR=15~25, keep the DL throughput between 25M to 35M

RSRP>-80, SINR=25~35, keep the DL throughput between 45M to 55M

The RSRP must be check from Probe, then you can test plan to finish this

work.This test plan PT will be provided by huawei. And you need to take a

Snapshots like the “Average downstream throughput – sector level” pictures.

DL:

Acceptable range: the result should higher than 4 Mbps ;

Target range: the result should higher than 4 Mbps;

UL:

Acceptable range: the result should higher than 1 Mbps ;

Target range: the result should higher than 1 Mbps ;

iii. Average downstream throughput – sector level

To the average downstream/upstream throughput (sector level), use 2 terminal

to perform the ftp download/ upload for 1 min at the same time. One point

must be meet the condition of Rank2 SINR1 >20 dB. The other point must be

meet the condition of MCS (in 16QAM) between 10 to 16(as below table). This

test should be implemented for 20% of sectors per cluster.

Huawei Confidential No spreading without permission Page 28 of 35

Page 29: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

Please pay more attention to the DL throughput of one point must over 35M.

The DL throughput of the other point must over 15M.

In addition, you need to add MCS to your PT logs, like this snapshot to open

MCS, and check DL MCS:

Huawei Confidential No spreading without permission Page 29 of 35

Page 30: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

The Rank2 SINR1 and MCS must be check from Probe, you can use FileZilla or

test plan, this test plan PT will be provided by huawei. And you need to take a

Snapshots like this:

Huawei Confidential No spreading without permission Page 30 of 35

Page 31: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

DL Snapshot

UL Snapshot

Huawei Confidential No spreading without permission Page 31 of 35

Page 32: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

iv. Maximum round trip latency

To the Maximum round trip latency, select 1 edge point to perform the ping test.

This point must be meet the condition of -100<RSRP<-90.The ping packet size

is 32KB and the ping repeat time is 100. Record the average ping delay. This

test should be implemented for 20% of sectors per cluster.

Acceptable range: the average value is less than 60 ms;

Target range: the average value is less than 50 ms;

v. Maximum Jitteryf

Calculate the maximum jitter based on above ping result. 5 biggest jitter values

can be canceled. This test should be implemented for 20% of sectors per

cluster.

Acceptable range: less than 30 ms;

Target range: less than 20 ms;

Huawei Confidential No spreading without permission Page 32 of 35

Page 33: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

The LTE Stationary KPI template in here, please finish this PT test report use

this template.

5) Each Work Output:

i. SSV report;

ii. DT logs.

Snapshots for Cell edge coverage probability, sector level throughput, terminal

level throughput; the result summary table (refer to attachment)

Round trip latency test detail by excel file;

Huawei Confidential No spreading without permission Page 33 of 35

Page 34: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

3 Measure Method

KPI Test includes DT and PT (point test).1) DT:

i. The DT will follow before confirmed wimax DT boundary. Using Huawei E398s USB terminal and Huawei Genex Probe&Assistant DT software to perform the DT. During the DT the engineers should parallel perform network entry, minimum guarantee DownLoad/UpLoad test. By this after the DT can get the test result of coverage probability/ successful setup rate / session establish delay/ dropped session rate/ minimum rate guarantee of downstream&upstream/ handoff/ HSR/ wireless link burst error rate.

ii. To the coverage probability, need to show out the statistic of points percentage of RSRP>-110dBm&SINR>-3dB, show the RSRP & SINR & PCI map picture.

iii. Successful setup rate= ERAB Setup Success Counter / ( ERAB Setup Success Counter + ERAB Setup Failure Counter);

iv. Session establishment delay= T(Attach CMP)-T(Attach REQ); v. Dropped session rate= ERAB Abnormal Rel Counter / ERAB Setup

Suc Counter.vi. To the minimum rate guarantee of downstream&upstream, the

engineers will perform ftp download and upload by one fixed size file (DL 80MB and UL 20MB) with non-GBR account, the download&upload will be ceaselessly repeated until finish the DT. Then calculate the average throughput of these tests. The test result will be filtered as contract requirements and only the valid service result will be calculated. This after filter result will be the KPI result of this test item.

vii. Handover successful rate(HSR)= HO Success Counter / ( HO Success Counter + HO Failure Counter);

viii. In any 1 min of the DT cannot happen more than 6 times HO, if this not happen, the handoff test is passed.

ix. To the wireless link burst error rate, calculate the average RBLER in the DT, the average RBLER of the DT should be less than 1% to achieve the contract KPI.

x. All the above test result will be filtered as the RF test condition defined in contract and the final result listed in the report should base on the test result after this filter. The service drop or test fail happened at bad RF points defined by contract will be excluded.

Huawei Confidential No spreading without permission Page 34 of 35

Page 35: Lte kpi dt guide & measure method.

LTE KPI DT GUIDE & MEASURE METHOD

SECRET

Besides, Huawei should provide all the original logs when submit the KPI reports.

2) PT:xi. To the cell edge coverage probability, select 20 edge points (SINR

is higher than 0dB but lower than 5dB & RSRP is lower than -95dBm) to perform ftp download, each point last 1 min. Do a statistic for the successful service points percentage.

xii. To the average downstream/upstream throughput (terminal level), select 3 points (RSRP=-105~-95 / -95~-80/ >-80) to perform the ftp download/ upload for 1 min. Record the average throughput of the 3 points. This test should be implemented for 20% of sectors per cluster

xiii. To the average downstream/upstream throughput (sector level), use 2 or 3 terminals (can be negotiated by Mobily & Huawei to detail test requiremtns) to perform the ftp download/ upload for 1 min at the same time. In the 2 terminals condition the modulation should be around 64QAM and 16QAM. This test should be implemented for 20% of sectors per cluster.

xiv. To the Maximum round trip latency, select 1 edge point (RSRP=-100~-90) to perform the ping test. The ping packet size is 32KB and the ping repeat times is 100. Record the average ping delay. This test should be implemented for 20% of sectors per cluster.

xv. Calculate the maximum jitter based on above ping result. 5 biggest jitter values should be canceled. This test should be implemented for 20% of sectors per cluster.

xvi. As contract requirements, if the site has other commercial users, the throughput KPI is not must achieved, DT team just normally record the test result & the slot occupy rate from NOC M2000. For that the test is a long term work and the ftp problem is always met, as to avoid any delay by the ftp server, to all the ftp related test items Huawei can use either Mobily ftp server (ftp.bayanat.com.sa) or Huawei server.

xvii. Huawei should provide all the original logs when submit the KPI reports.

xviii. KPI test report template

Huawei Confidential No spreading without permission Page 35 of 35