AZQ POC Comparative Report Ver1 2 Apr'14 Pune

2013 Nokia Solutions and Networks. All rights reserved.

CCMT-AZQ POC Report (Pune) April, 2014

2 2013 Nokia Solutions and Networks. All rights reserved. 16-May-14

RSO-India_2014-017_CCMT-AZQ POC Report/ Ver 1.0/ RSO - Radio / Praveen Wason For internal use

Agenda

Objective and Scope of Work

Responsibility Matrix

POC Details

2G Drive Test Results City Cluster

1

2

3

4

3G Drive Test Results City Cluster 5

2G & 3G Drive Test Results Highway 6



Agenda

3G MultiRAB and Data Drive Test Results City Cluster

7

Feature Comparison Drive Test Tools 8

KPI Comparison Drive Test Tools

9



Objective and Scope of Work

Field trial of AZQ handset based drive test tool Feasibility check for AZQ tool in carrying out various ND/MS outdoor drive test scenarios Comparative result analysis of AZQ tool with conventional Laptop based DT kits Feature and capability comparison of AZQ tool v/s conventional Laptop based DT kits Conclusion of AZQ tool performance on the basis of drive test results, with support from circle team Replacing existing conventional laptop based DT kits with AZQ tool to achieve cost efficiency in terms of reduced manpower and drive test vehicles

Dry run of AZQ tool at Gurgaon office before actual field trial and team deployment Simultaneous drive test with both AZQ and TEMS/Nemo kits on the same drive route/cluster Post processing of drive test data and final report preparation Validation and Analysis of AZQ test results Comparison of AZQ tool features and performance with TEMS/Nemo conventional laptop based drive test kit

Objective

Scope of Work



Responsibility Matrix

R (Responsible) S (Support) J (Jointly) I (Information)

Sr. No. Activity Description RSO PMO COM

1 Preparation of drive test scenarios for POC R I I

2 Finalization of POC report format R I I

3 Provisioning of AZQ handsets as per drive test scenarios R

4 Provisioning of dedicated FTP server for data testing R

5 Arrangement of drive test team with TEMS/Nemo Kit R S

6 Tool setup and readiness J J

7 Product related support from AZQ tool vendor during the POC / Trial R

8 Data Post Processing & Report Preparation R S S

9 Validation of POC test results J J S



POC Details

Pune City Map & Drive Test Route



POC Details

Drive Test Details

A Drive activity was carried out in Pune City in the month of April 2014. The details are as follows:

Total route covered in KMs : Approx 350 KMs.

Measurement Date : 14 March 2013 to 18 April 2014

Measurement time during the day : 11 AM to 7:30 PM

Drive Route : Intra City & Highway

Drive Test Tool: TEMS s/w ver: 13.1.6 & AZQ Handsets

UE Detail: TEMS (SE W995 & HSPA+ Sierra Air Card 310U), AZQ (Sony Xperia E & Sony Xperia SP handset)

Post Processing Tool: Gladiator & AZQ

Laptop: I3 with 4 GB RAM



POC Details

Drive Test Methodology Voice Call

Mobile Originated Short Calls (Dedicated Mode)

Call Duration : 120 sec

Wait time before attempting next call: 10 sec for 2G Calls & 30 sec for 3G Calls

Call Establish time : 15 sec (max)

All calls should be made to Mobile or Switch test number

Mobile Originated Long Calls (Dedicated Mode)

Call Duration : Continuous (590 sec)

Wait time before attempting next call: 10 sec for 2G Calls & 30 sec for 3G Calls

Call Establish time : 15 sec (max)

All calls should be made to Mobile or Switch test number

Idle Mode

Idle mode drive is to be done to collect coverage data for 2G & 3G n/w with handset



POC Details

Drive Test Methodology Data Call

Packet Data Test Sequence Attach and PDP Activation

Ping/RTT Test - 32 bytes on IP address: 182.19.95.68

Wait: 5 sec

FTP Download File Size 500 KB for 2G and 2 MB for 3G

Wait: 5 sec

FTP Upload File Size 100 KB for 2G and 15 MB for 3G

Wait: 5 sec

HTTP Browsing (1 Web Site) - www.cricinfo.com

Wait time between each browsing: 5 sec

Detach and PDP Deactivation



POC Details

Server Details

PING: 182.19.95.68 (Public Server)

FTP DL: 93.183.30.178 (AZQ Server @ NSN Noida)

FTP UL: 93.183.30.178 (AZQ Server @ NSN Noida)

HTTP Browsing: www.cricinfo.com



POC Details

Drive Test Scenarios

Vodafone Airtel Vodafone Airtel Vodafone Airtel Vodafone Airtel

(Idle Mode) (Idle Mode) (Short call) (Short call) (Idle Mode) (Idle Mode) (Short call) (Short call)

Vodafone Airtel Vodafone Airtel Vodafone Airtel Vodafone Airtel - -

(Long Call ) (Long Call ) (Data call) (Data call) (Long Call ) (Long Call ) (Data call) (Data call)

Vodafone Idea Vodafone Idea Vodafone Idea Vodafone Idea

(Idle Mode) (Idle Mode) (Short call) (Short call) (Idle Mode) (Idle Mode) (Short call) (Short call)


(Long Call ) (Long Call ) (Data call) (Data call) (Long Call ) (Long Call ) (Data call) (Data call)


(Idle Mode ) (Idle Mode ) (Long Call) (Long Call) (Idle Mode ) (Idle Mode ) (Long Call) (Long Call)

Vodafone Airtel Vodafone Airtel Vodafone Airtel Vodafone Airtel

(Idle Mode ) (Idle Mode ) (Long Call) (Long Call) (Idle Mode ) (Idle Mode ) (Long Call) (Long Call)


(Multi RAB1) (Multi RAB2) (Data call) (Data call) Multi RAB1 Multi RAB2 (Data call) (Data call)

1

2

5 UMTS

3

4

Drive

Test

Scenario

6 GSM

UMTS

MS 1 (Handset) MS 2 (Handset)

CCMT - AZQ Drive Test Tool

Technology

MS 1 (Handset) MS 2 (Handset) MS 3 (Handset) MS 4 (Handset)

MS 5

(HSPA+ Data

Card)

MS 6

(HSPA+ Data

Card)

MS 3 (Handset) MS 4 (Handset)

7

4

Drive

Test

Activity

UMTS

UMTS

TEMS Drive Test Tool

3

2 GSM

1 GSM




2G Drive Test

Voice (Idle, Short, Long Call)

&

Data (Ping, FTP D/L, FTP U/L & HTTP)



2G Drive Test Results City Cluster Drive Test Activity - 1

TEMS MS 1: Vodafone 2G Idle Mode (SE W995)

AZQ MS 1: Vodafone 2G Idle Mode (Sony Xperia SP)

TEMS MS 2: Airtel 2G Idle Mode (SE W995)

AZQ MS 2: Airtel 2G Idle Mode (Sony Xperia SP)

TEMS MS 3: Vodafone 2G Short Call (SE W995) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 10 Sec

AZQ MS 3: Vodafone 2G Short Call (Sony Xperia E) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 10 Sec

TEMS MS 4: Airtel 2G Short Call (SE W995) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 10 Sec

AZQ MS 4: Airtel 2G Short Call (Sony Xperia E) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 10 Sec




TEMS MS 1: Vodafone 2G Long Call (SE W995) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 10 Sec

AZQ MS 1: Vodafone 2G Long Call (Sony Xperia SP) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 10 Sec

TEMS MS 2: Airtel 2G Long Call (SE W995) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 10 Sec

AZQ MS 2: Airtel 2G Long Call (Sony Xperia SP) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 10 Sec

TEMS MS 3: Vodafone 2G Data Call (SE W995) PING (93.183.30.178) FTP DL (500 Kb), FTP UL (100 Kb), HTTP Browsing

AZQ MS 3: Vodafone 2G Data Call (Sony Xperia E) PING (93.183.30.178) FTP DL (500 Kb), FTP UL (100 Kb), HTTP Browsing

TEMS MS 4: Airtel 2G Data Call (SE W995) PING (93.183.30.178) FTP DL (500 Kb), FTP UL (100 Kb), HTTP Browsing

AZQ MS 4: Airtel 2G Data (Sony Xperia E) PING (93.183.30.178 FTP DL (500 Kb), FTP UL (100 Kb), HTTP Browsing



Rx Level (Full) Idle Mode

Vodafone -TEMS

Airtel -TEMS

Vodafone -AZQ

Airtel -AZQ



Rx Level Sample Comparison

Vodafone

Airtel



Rx-Quality Plots (Long Calls)

Vodafone -TEMS Vodafone -AZQ

Airtel -TEMS Airtel -AZQ



Rx-Quality Sample Comparison

Vodafone

Airtel



C/I Plots (Long Calls)





C/I Sample Comparison

Vodafone

Airtel



FER Plots (Long Calls)





FER Sample Comparison

Vodafone

Airtel



AMR FR/HR Distribution (Long Calls)



AMR: FR

AMR: Likely HR

AMR: FR

AMR: Likely HR



AMR FR/HR Sample Comparison

Vodafone

Airtel



AMR Codec Distribution Long Calls AMR:4.75 Kbps AMR:5.15 Kbps AMR:5.90 Kbps AMR:6.70 Kbps AMR:7.40 Kbps AMR:7.95 Kbps AMR:10.20 Kbps AMR:12.20 Kbps


Airtel -TEMS Airtel -AZQ AMR:4.75 Kbps AMR:5.15 Kbps AMR:5.90 Kbps AMR:6.70 Kbps AMR:7.40 Kbps AMR:7.95 Kbps AMR:10.20 Kbps

AMR:12.20 Kbps



AMR Codec distribution- Sample Comparison

Vodafone

Airtel



Blocked Calls on Rx-Level (Sub) Plot

0 Block Call 0 Block Call

1 Block Call



0 Block Call



Drop Calls on Rx-Quality Plot

1 Drop Call 0 Drop Call






HO Failure on Rx Level (Sub) Plot

0 HO Fail

0 HO Fail

1 HO Fail

2 HO Fail





2G Voice Statistics

Sr. No. Description Vodafone - TEMS Vodafone -AZQ Airtel - TEMS Airtel - AZQ

1 Total Call Attempts 72 71 73 71

2 Block Call Rate (%) 0% 0% 1.3% 0%

3 Drop Call Rate (%) 1.3% 0% 1.3% 0%

4 Call Completion Success Rate (%) 98.7% 100% 98.7% 100%

5 Call Setup Success Rate (%) 100% 100% 98.7% 100%

6 Average Call Setup Time (sec) 2.13 2.23 3.04 2.25

7 HOSR 100% 99.51% 100% 99.23%



RLC Throughput DL





RLC DL Throughput - Sample Comparison

Vodafone

Airtel



RLC Throughput UL





RLC UL Throughput - Sample Comparison

Vodafone

Airtel



Application Throughput (Kbps) For FTP

TEMS AZQ



PING RTT (ms)

TEMS AZQ VF Avg. Ping 308.2 ms Airtel Avg. Ping 470 ms



EDGE/GPRS Sample Distribution Plot



GPRS

EDGE

GPRS

EDGE



Packet Channel Coding DL GPRS/EDGE

Range Color

CS-1

CS-2

CS-3

CS-4

MCS-1

MCS-2

MCS-3

MCS-4

MCS-5

MCS-6

MCS-7

MCS-8

MCS-9





Channel Coding DL - Sample Comparison

Vodafone

Airtel



Number of Timeslots DL GPRS/EDGE





Timeslots DL - Sample Comparison

Vodafone

Airtel

0 Timeslot Allocation % is on higher

side. This is due to the reason that

web page could not be opened

during HTTP data session and

timeout value for the same was

d e f i n ed a s 12 000 se co nd s .



Number of Timeslots UL GPRS/EDGE





Timeslots UL - Sample Comparison

Vodafone

Airtel

0 Timeslot Allocation % is on higher

side. This is due to the reason that

web page could not be opened

during HTTP data session and

timeout value for the same was

d e f i n ed a s 12 000 se co nd s .



2G Data Statistics

Sn Description Vodafone -

TEMS Vodafone - AZQ Airtel - TEMS Airtel - AZQ

1 FTP D/L Average Download TPUT (Kbps) 68 55.15 43 60.62

2 FTP U/L Average Download TPUT (Kbps) 35 37.06 29 46.01

3 HTTP Setup Success Rate (%) 100% 88.89% 100% 92.11%

4 HTTP Average Throughput (Kbps) 91 28.62 55 39.89

5 GPRS Attach Success Rate (%) 95.65% 100% 89.13% 100%

6 PING Average RTT (ms) 324 308.2 508 470




3G Drive Test

Voice (Idle, Short, Long Call)

&







TEMS MS 2: Idea 3G Idle Mode (SE W995)

AZQ MS 2: Idea 3G Idle Mode (Sony Xperia SP)

TEMS MS 3: Vodafone 3G Short Call (SE W995) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 30 Sec

AZQ MS 3: Vodafone 3G Short Call (Sony Xperia E) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 30 Sec

TEMS MS 4: Idea 3G Short Call (SE W995) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 30 Sec

AZQ MS 4: Idea 3G Short Call (Sony Xperia E) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 30 Sec




TEMS MS 1: Vodafone 3G Long Call (SE W995) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 30 Sec

AZQ MS 1: Vodafone 3G Long Call (Sony Xperia SP) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 30 Sec

TEMS MS 2: Idea 3G Long Call (SE W995) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 30 Sec

AZQ MS 2: Idea 3G Long Call (Sony Xperia SP) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 30 Sec

TEMS MS 3: Vodafone 3G Data Call (SE W995) PING 93.183.30.178) FTP DL (500 Kb), FTP UL (100 Kb), HTTP Browsing

AZQ MS 3: Vodafone 3G Data Call (Sony Xperia E) PING (93.183.30.178) FTP DL (500 Kb), FTP UL (100 Kb), HTTP Browsing

TEMS MS 4: Idea 3G Data Call (SE W995) PING (93.183.30.178) FTP DL (500 Kb), FTP UL (100 Kb), HTTP Browsing

AZQ MS 4: Idea 3G Data (Sony Xperia E) PING (93.183.30.178) FTP DL (500 Kb), FTP UL (100 Kb), HTTP Browsing



RSCP Idle Mode


Idea -TEMS Idea -AZQ



RSCP Plot Sample Comparison

Vodafone

Idea



Ec/No Sample Comparison

Vodafone

Idea



Block Call on RSCP Plot Short Call







AMR Codec Distribution Long Call

Vodafone -AZQ

Idea -AZQ Range % Samples Color

AMR:4.75 Kbps 0 0

AMR:5.15 Kbps 0 0

AMR:5.90 Kbps 0 0

AMR:6.70 Kbps 0 0

AMR:7.40 Kbps 0 0

AMR:7.95 Kbps 0 0

AMR:10.20 Kbps 0 0

AMR:12.20 Kbps 99.96 21958

AMR-WB:6.60 Kbps 0 0









Rangea % Samples Color

AMR:4.75 Kbps 8.16 1891

AMR:5.15 Kbps 0 0

AMR:5.90 Kbps 4.38 1016

AMR:6.70 Kbps 0 0

AMR:7.40 Kbps 76.87 17820

AMR:7.95 Kbps 0 0

AMR:10.20 Kbps 0 0

AMR:12.20 Kbps 0.13 29

AMR-WB:6.60 Kbps 0.85 196


AMR-WB:12.65 Kbps 9.58 2220







AMR codec data could not be

populated in Gladiator tool

AMR codec data could not be

populated in Gladiator tool



3G Voice Statistics

Sn Description Vodafone - TEMS Vodafone -AZQ Idea - TEMS Idea- AZQ


2 Block Call Rate (%) 3.12 0 3.12 0

3 Drop Call Rate (%) 0 0 0 12.8

4 Call Completion Success Rate (%) 100 100 100 88.2

5 Call Setup Success Rate (%) 96.88 100 96.88 100


7 HOSR 100 100 100 100



RLC Throughput DL





RLC Throughput Distribution DL

Vodafone

Idea



RLC Throughput UL





RLC Throughput Distribution UL

Vodafone

Idea



Application Throughput (Kbps) For FTP

TEMS AZQ



PING RTT (ms)

TEMS AZQ VF Avg. Ping 300 ms Idea Avg. Ping 370 ms



Packet Technology Plot





HSDPA Modulation Technique





IRAT HO failure (3G to 2G & 2G to 3G)

0 HO Fail 0 HO Fail

0 HO Fail 0 HO Fail





BLER Plot - Data Call





BLER Sample Distribution

Vodafone

Idea



3G Data Statistics


TEMS Vodafone - AZQ Idea - TEMS Idea - AZQ

1 FTP D/L Average Download TPUT (Kbps) 837 900 473 487

2 FTP U/L Average Download TPUT (Kbps) 496 265 171 177

3 HTTP Setup Success Rate (%) 100 100 100 100

4 HTTP Average Throughput (Kbps) 156 143 221 151

5 GPRS Attach Success Rate (%) 100 100 100 100

6 PING Average RTT (ms) 324 300 508 370



2G & 3G Drive Test Results Highway

2G / 3G Drive Test

Voice (Idle & Long Call)

&




2G & 3G Drive Test Results Highway Drive Test Route



2G & 3G Drive Test Results Highway Drive Test Activity 5



TEMS MS 2: Idea 3G Long Call (SE W995) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 30 Sec

AZQ MS 2: Idea 3G Long Call (Sony Xperia SP) Called Number: Test Number, Call Duration: 590 Sec, Wait Time: 30 Sec



RSCP Idle Mode





RSCP Plot Sample Comparison

Vodafone

Idea



Ec/No Plot Long Call





Ec/No Plot Sample Comparison

Vodafone

Idea



Block Call on RSCP Plot Long Call

1 Block Call 2 Block Calls



Idea -TEMS Idea - AZQ



Drop Call on Ec/No Plot Long Call


0 Drop Calls 3 Drop Calls

0 Drop Call





IRAT HO failure (3G to 2G) Long Call

0 HO Fail 0 HO Fail

0 HO Fail 0 HO Fail





AMR Codec Distribution Long Call

Vodafone -AZQ

Idea -AZQ

Range Percent(%) N Color

(0) AMR:4.75 Kbps 3.44 236

(1) AMR:5.15 Kbps 0 0

(2) AMR:5.90 Kbps 2.23 153

(3) AMR:6.70 Kbps 0 0

(4) AMR:7.40 Kbps 20.22 1388

(5) AMR:7.95 Kbps 0 0

(6) AMR:10.20 Kbps 0 0

(7) AMR:12.20 Kbps 12.34 847

(8) AMR-WB:6.60 Kbps 0.26 18

(9) AMR-WB:8.85 Kbps 0 0

(10) AMR-WB:12.65 Kbps 61.44 4217

(11) AMR-WB:14.25 Kbps 0 0

(12) AMR-WB:15.85 Kbps 0 0

(13) AMR-WB:18.25 Kbps 0 0

(14) AMR-WB:19.85 Kbps 0 0

(15) AMR-WB:23.05 Kbps 0 0

(16) AMR-WB:23.85 Kbps 0 0


(0) AMR:4.75 Kbps 0.93 66

(1) AMR:5.15 Kbps 0.41 29

(2) AMR:5.90 Kbps 0.72 51

(3) AMR:6.70 Kbps 3.27 231

(4) AMR:7.40 Kbps 0.81 57

(5) AMR:7.95 Kbps 0 0

(6) AMR:10.20 Kbps 1.55 110

(7) AMR:12.20 Kbps 92.24 6526

(8) AMR-WB:6.60 Kbps 0 0

(9) AMR-WB:8.85 Kbps 0 0

(10) AMR-WB:12.65 Kbps 0 0

(11) AMR-WB:14.25 Kbps 0 0

(12) AMR-WB:15.85 Kbps 0 0

(13) AMR-WB:18.25 Kbps 0 0

(14) AMR-WB:19.85 Kbps 0 0

(15) AMR-WB:23.05 Kbps 0 0

(16) AMR-WB:23.85 Kbps 0 0



2G & 3G Drive Test Results Highway Drive Test Activity 6



TEMS MS 2: Airtel 2G Long Call (SE W995) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 10 Sec

AZQ MS 2: Airtel 2G Long Call (Sony Xperia SP) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 10 Sec



Rx Level (Full) Idle Mode


Airtel -TEMS Airtel - AZQ



Rx Level Sample Comparison

Vodafone

Airtel



Rx-Quality Plots (Long Calls)





Rx-Quality Sample Comparison

Vodafone

Airtel



C/I Sample Comparison

Vodafone

Airtel



FER Plots (Long Calls)





FER Sample Comparison

Vodafone

Airtel



AMR FR/HR Distribution (Long Calls)



Range Colour % Samples

FR 46.6 3332

Likely HR 53.4 3819

Range Colour (%) Samples

FR 21.7 1493

Likely HR 78.3 5396



AMR FR/HR Sample Comparison

Vodafone

Airtel



AMR Codec Distribution Long Calls



Range % Samples Color AMR:4.75 Kbps 8.33 574 AMR:5.15 Kbps 0 0 AMR:5.90 Kbps 3.74 258 AMR:6.70 Kbps 0 0 AMR:7.40 Kbps 66.2 4564 AMR:7.95 Kbps 0 0

AMR:10.20 Kbps 0 0 AMR:12.20 Kbps 21.66 1493

Range % Samples Color AMR:4.75 Kbps 3.87 277 AMR:5.15 Kbps 0 0 AMR:5.90 Kbps 9.74 697 AMR:6.70 Kbps 0 0 AMR:7.40 Kbps 39.76 2845 AMR:7.95 Kbps 0 0

AMR:10.20 Kbps 0 0 AMR:12.20 Kbps 46.57 3332



AMR Codec distribution- Sample Comparison

Vodafone

Airtel



HO Failure on Rx Level (Sub) Plot

1 HO Fail

0 HO Fail

1 HO Fail

1 HO Fail





3G MultiRAB and Data Drive Test Results City Cluster

(3G Locked Mode)

3G MultiRAB Drive Test

Voice (Short Call)

Data (FTP D/L

3G Data Drive Test (Dual Mode)





3G MultiRAB and Data Drive Test Results City Cluster Drive Test Activity - 7

TEMS MS 1: Vodafone 3G Short Call (SE W995) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 30 Sec PING (93.183.30.178) FTP DL (2 Mb)

AZQ MS 1: Vodafone 3G Short Call (Sony Xperia E) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 30 Sec PING (93.183.30.178) FTP DL (2 Mb)

TEMS MS 2: Idea 3G 3G Short Call (SE W995) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 30 Sec PING (93.183.30.178) FTP DL (2 Mb)

AZQ MS 2: Idea 3G Long Call 3G Short Call (Sony Xperia E) Called Number: Test Number, Call Duration: 120 Sec, Wait Time: 30 Sec PING (93.183.30.178) FTP DL (2 Mb)

TEMS MS 3: Vodafone 3G Data Call (HSPA+ Data Card) PING (93.183.30.178) FTP DL (15 Mb), FTP UL (2 Mb), HTTP Browsing

AZQ MS 3: Vodafone 3G Data Call (Sony Xperia SP) PING (93.183.30.178) FTP DL (15 Mb), FTP UL (2 Mb), HTTP Browsing

TEMS MS 4: Idea 3G Data Call (HSPA+ Data Card) PING (93.183.30.178) FTP DL (15 Mb), FTP UL (2 Mb), HTTP Browsing

AZQ MS 4: Idea 3G Data (Sony Xperia SP) PING (93.183.30.178) FTP DL (15 Mb), FTP UL (2 Mb), HTTP Browsing



RLC Throughput DL (HSPA+ Data Card v/s AZQ Xperia SP Handset)





RLC Throughput Distribution DL (HSPA+ Data Card v/s AZQ Xperia SP Handset)

Vodafone

Idea



RLC Throughput UL (HSPA+ Data Card v/s AZQ Xperia SP Handset)





RLC Throughput Distribution UL (HSPA+ Data Card v/s AZQ Xperia SP Handset)

Vodafone

Idea



Application Throughput (Kbps) For FTP (HSPA+ Data Card v/s AZQ Xperia SP

Handset)

TEMS AZQ



PING RTT (ms) (HSPA+ Data Card v/s AZQ Xperia SP Handset)

TEMS AZQ VF Avg. Ping 335.3 ms Idea Avg. Ping 307.7 ms



Packet technology Plot (HSPA+ Data Card v/s AZQ Xperia SP Handset)





HSDPA Modulation Technique (HSPA+ Data Card v/s AZQ Xperia SP Handset)



Range % Samples Color

QPSK 34.99 3261

16 QAM 0.26 24

Mixed 64.75 6035


QPSK 32.25 2276

16 QAM 0.3 21

Mixed 67.46 4761



IRAT HO failure (3G to 2G & 2G to 3G) (Data call)

0 HO Fail 0 HO Fail

0 HO Fail 0 HO Fail





3G Data Statistics (HSPA+ Data Card v/s AZQ Xperia SP Handset)


TEMS Vodafone - AZQ Idea - TEMS Idea - AZQ

1 FTP D/L Average Download TPUT (Kbps) 1135 1231 436 582

2 FTP U/L Average Download TPUT (Kbps) 562 573 317 438

3 HTTP Setup Success Rate (%) 100 98.21 100 100

4 HTTP Average Throughput (Kbps) 260 168 229 272

5 GPRS Attach Success Rate (%) 100 100 100 100

6 PING Average RTT (ms) 397 572 308 404



RSCP Dedicated Mode (MultiRAB Mode)





Ec/No Plot Short Call (MultiRAB Mode)





Ec/No Plot Sample Comparison (MultiRAB Mode)

Vodafone

Idea



Block Call on RSCP Plot Short Call (MultiRAB Mode)

4 Block Calls 2 Block Calls






Drop Call on Ec/No Plot Short Call (MultiRAB Mode)


3 Drop Calls 2 Drop Calls





Serving System Plot Short Call (MultiRAB Mode)






(0) AMR:4.75 Kbps 0 0

(1) AMR:5.15 Kbps 0 0

(2) AMR:5.90 Kbps 0 0

(3) AMR:6.70 Kbps 0 0

(4) AMR:7.40 Kbps 0 0

(5) AMR:7.95 Kbps 0 0

(6) AMR:10.20 Kbps 0 0

(7) AMR:12.20 Kbps 99.68 4057

(8) AMR-WB:6.60 Kbps 0 0

(9) AMR-WB:8.85 Kbps 0 0

(10) AMR-WB:12.65 Kbps 0 0

(11) AMR-WB:14.25 Kbps 0 0

(12) AMR-WB:15.85 Kbps 0 0

(13) AMR-WB:18.25 Kbps 0 0

(14) AMR-WB:19.85 Kbps 0 0

(15) AMR-WB:23.05 Kbps 0 0

(16) AMR-WB:23.85 Kbps 0 0


(0) AMR:4.75 Kbps 0 0

(1) AMR:5.15 Kbps 0 0

(2) AMR:5.90 Kbps 0 0

(3) AMR:6.70 Kbps 0 0

(4) AMR:7.40 Kbps 0 0

(5) AMR:7.95 Kbps 0 0

(6) AMR:10.20 Kbps 0 0

(7) AMR:12.20 Kbps 0 0

(8) AMR-WB:6.60 Kbps 0.47 30

(9) AMR-WB:8.85 Kbps 0 0

(10) AMR-WB:12.65 Kbps 99.18 6275

(11) AMR-WB:14.25 Kbps 0 0

(12) AMR-WB:15.85 Kbps 0 0

(13) AMR-WB:18.25 Kbps 0 0

(14) AMR-WB:19.85 Kbps 0 0

(15) AMR-WB:23.05 Kbps 0 0

(16) AMR-WB:23.85 Kbps 0 0

AMR Codec Distribution Short Call (MultiRAB Mode)

Vodafone -AZQ

Idea -AZQ



3G Voice Statistics (MultiRAB Mode)

Sn Description Vodafone - TEMS Vodafone -AZQ Idea - TEMS Idea- AZQ


2 Block Call Rate (%) 11.1 6.4 0 0

3 Drop Call Rate (%) 0 0 15 10.5

4 Call Completion Success Rate (%) 100 100 85.0 89.5

5 Call Setup Success Rate (%) 88.9 93.6 100 100


7 HOSR 100 100 100 100



RLC Throughput DL (MultiRAB Mode)





RLC Throughput Distribution DL (MultiRAB Mode)

Vodafone

Idea



RLC Throughput UL (MultiRAB Mode)





RLC Throughput Distribution UL (MultiRAB Mode)

Vodafone

Idea



Application Throughput (Kbps) For FTP (MultiRAB Mode)

TEMS AZQ



Packet technology Plot (MultiRAB Mode)





HSDPA Modulation Technique (MultiRAB Mode)



Range % Samples Color QPSK 38.27 2043

16 QAM 0.41 22 Mixed 61.32 3274


QPSK 20.25 845

16 QAM 0.77 32

Mixed 78.98 3295



Feature Comparison Drive Test Tools





Feature DescriptionTEMS - Laptop

Based DT Tool

NEMO - Laptop

Based DT Tool

TEMS Pocket -

Handset Based DT

Tool

AZQ - Handset

Based DT Tool

Operating System Multiple OS Multiple OS Google Android OS Google Android OS

Technology Supported (GSM, EDGE, GPRS, WCDMA, HSPA,LTE,Wi-Fi)

Handset Based Scanning

HTTP file and Web download

FTL DL/UL

External Scanner Connectivity

Ping

SMS

Voice Quality MOS (Mobile to Mobile,Mobile to IVR)

IP packet capture

Idle Mode Data Collection

Wi-Fi scanning

Multi-RAB (concurrent voice and data)

Time scheduled and event triggered automatic tests

Automated logfile upload on the central server

Automated logfile post processing and report generation

Data views for idle, dedicated, RACH, data

bearer, script information, etc.

User-customizable data views

Data session event viewer for script

progress tracking

SQI MOS

Positioning with internal or external GPS




Feature DescriptionTEMS - Laptop

Based DT Tool

NEMO - Laptop

Based DT Tool

TEMS Pocket -

Handset Based DT

Tool

AZQ - Handset

Based DT Tool

L3 message header and full text viewer

Wide range of L3, call and radio events

with audio alerts

Line charts / distribution charts

Event markers in maps and logs

Indoor testing

Outdoor map with global satellite

Plot cell site locations and antenna

directions in outdoor maps

Import of cell site data for added cell

information

RAT Lock

Screen capture

Replay collected data directly in the handset

Automatic report processing

Multi-Rab Testing

Display of L3 message and other Window on the Phone

Email Send Receive Test

Post Processing of Log file on 3rd part Post Processing tool

MMS Testing

Cell Locking

WAP Testing

Tool Stability Stable Stable Stable Stable

Ease of Operations Good Good V.Good V.Good




Test

ScenarioKPI Description Technology TEMS Nemo AZQ Remarks

Average Call Setup Time(s) 2G & 3G

Rx. Level Distribution 2G & 3G

CSSR % 2G & 3G

CCSR % 2G & 3G

Rx. Quality Distribution 2G

TCH FR HR distribution 2G & 3G

HOSR % 2G

IRAT HOSR % 3G

Downlink Edge Codec Distribution 2G Cant lock to EDGE/GPRS

Uplink Edge Codec Distribution 2G Cant lock to EDGE/GPRS

Downlink GPRS Codec Distribution 2G Cant lock to EDGE/GPRS

Uplink GPRS Codec Distribution 2G Cant lock to EDGE/GPRS

SIR 3G

FER 2G

SMS E2E Delivery Success % 2G & 3G

SMS E2E Delivery Time (sec) 2G & 3G

MMS E2E Delivery Success % 2G & 3G

MMS E2E Delivery Time (sec) 2G & 3G

Voice

VAS




Test

ScenarioKPI Description Technology TEMS Nemo AZQ Remarks

BLER D/L 3G

GPRS Attach Success % 2G & 3G

GPRS Attach Delay (sec) 2G & 3G

PDP Context Activation Success % 2G & 3G

PDP Context Activation Delay (sec) 2G & 3G

FTP Uplink Success % 2G & 3G

FTP Uplink RLC Throughput (Kbps) 2G & 3G

FTP Downlink Success % 2G & 3G

FTP Downlink RLC Throughput (Kbps) 2G & 3G

HTTP Browsing Success % 2G & 3G

HTTP Browsing Download Time (sec) 2G & 3G

IRAT HO % 3G

RTT 2G & 3G

CQI Distribution 3G

Modulation Technique Distribution (QPSK, 16 QAM

& 64 QAM)3G

DL and UL timeslot allocation 2G

Latency (msec) 2G & 3G

MCS Coding Scheme 2G & 3G

PDP Activation Success Rate % 2G & 3G

PDP Activation Succes Time (sec) 2G & 3G

HSDPA CSSR % 3G Cant lock to R99/HSDPA/HSUPA

HSUPA CSSR % 3G Cant lock to R99/HSDPA/HSUPA

R99 CSSR % 3G Cant lock to R99/HSDPA/HSUPA

Data



Thank You

AZQ POC Comparative Report Ver1 2 Apr'14 Pune

Documents

Transcript of AZQ POC Comparative Report Ver1 2 Apr'14 Pune