8/15/2019 BM Report Template

1/38

Benchmarking ReportTemplate

8/15/2019 BM Report Template

2/38

• Executive Summary

 – Summary of City

• Benchmarking Details

 – Data Collection Project Summary

 – Drive Route• Test Overview

 – Voice Tests

 – Data

 –

Strategic Locations• Voice & Data Quality Weightage

• Performance Ranking

Contents

8/15/2019 BM Report Template

3/38

Executive Summary

>

8/15/2019 BM Report Template

4/38

Data Collection project Summery

>

8/15/2019 BM Report Template

5/38

Drive Route

>

>

8/15/2019 BM Report Template

6/38

6

Test Overview-VoiceContinuous call for 3 minute duration with 10 seconds of

idle time. Speech samples (DL and UL) are sent andreceived which provide the MOS score for voice quality.

8/15/2019 BM Report Template

7/38

7

Test Overview-Data

>

No. HTTP Wbsites1 www.google.com

2 www.facebook.com

Test Details Specifications 3 www.numbuzz.com

Attach & PDP Activation Before every Test 4 www.airtel.in

Ping 3 times - 32 bytes on www.google.co.in 5 www.cricinfo.com

FTP Download 10 Mb for EvDo and 15 Mb for UMTS from server 6 www.vuclip.com

FTP Upload 1 Mb for EvDo and 1 Mb for UMTS to server 7 www.youtube.com

HTTP 10 different websites 8 www.admob.com

Email send 100 Kb Email to SMTP server 9 www.zedge.net

Email Receive 100 Kb Email from POP3 server 10 www.nokia.com

Number of Operators Tested>

Data Test Alogorithm

8/15/2019 BM Report Template

8/38

8

Test Overview-Strategic Locations

>

>

No. WAP Sites1 wap.google.com

2 m.facebook.com

3 m.orkut.com

Test Details Specifications 4 wapin.com

WAP Test 10 different WAP sites 5 m.cricinfo.com

FTP Download 10 Mb for EvDo and 15 Mb for UMTS from server 6 m.vuclip.comEmail Send 100 Kb Email to SMTP server 7 m.youtube.com

Email Receive 100 Kb Email from POP3 server 8 m.admob.com

SMS SMS send to same number (160 charectors) 9 www.m.zedge.net

MMS MMS send to same number 15 Kb 10 m.sonyericsson.com

Number of Operators Tested

>

Data Test Alogorithm

8/15/2019 BM Report Template

9/38

9

Voice & Data KPI Weightage

Coverage Calculation = (A%*10+ B%*8 + C%*6 +D%*1) / 10

Grade RSCP in dBm

 A : 0 to -74

B : -74 to -82

C : -82 to -90

D : -90 to -98

8/15/2019 BM Report Template

10/38

10

Performance Ranking

Performance CST Rank Performance CSSR Rank Performance CCSR Rank Performance MOS Rank Performance Coverage Rank

Bad 15.0 0 Bad 90% 0 Bad 90% 0 Bad 1.0 0 Bad 90% 0

Poor 12.0 2 Poor 92% 2 Poor 92% 2 Poor 2.0 2 Poor 92% 2

Average 10.0 4 Average 94% 4 Average 94% 4 Average 2.8 4 Average 94% 4

Fair 5.0 6 Fair 96% 6 Fair 96% 6 Fair 3.5 6 Fair 96% 6

Good 3.0 8 Good 98% 8 Good 98% 8 Good 4.0 8 Good 98% 8

Excellent 2.0 10 Excellent 100% 10 Excellent 100% 10 Excellent 5.0 10 Excellent 100% 10

>2

4

6

8

10

8/15/2019 BM Report Template

11/38

• Voice Quality Rank

• Data Quality Rank

• Voice Benchmarking

 – Call setup time.

 – Call Setup Success rate.

 – Call Completion rate.

 – Call Completion Success rate comparison with CCR.

 – Voice quality UL & DL MOS.

 – Coverage

 –

Plot of UL MOS. – Plot of DL MOS.

 – Voice Codec Usage

 – Handover Performance

 – Dropped call analysis

Contents-Voice Benchmarking

8/15/2019 BM Report Template

12/38

12

Voice Quality Ranking

>

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10CST CSSR CCSR MOS COV TOTAL

8/15/2019 BM Report Template

13/38

13

Data Quality Ranking

>

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10eMail HTTP SMS MMS

8/15/2019 BM Report Template

14/38

14

Data Quality Ranking

>

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10

2

4

6

8

10Attach

PDP

Context FTP-DL FTP-UL Ping TOTAL

8/15/2019 BM Report Template

15/38

15

Call Setup Time (CST)

8/15/2019 BM Report Template

16/38

16

Call Setup Success Rate (CSSR)

8/15/2019 BM Report Template

17/38

17

Call Setup Failure Plot

8/15/2019 BM Report Template

18/38

18

Call Completion Rate

8/15/2019 BM Report Template

19/38

19

Call Completion Success RateVs. Call Completion Rate

8/15/2019 BM Report Template

20/38

20

Dropped Call Plot

8/15/2019 BM Report Template

21/38

21

Voice Quality - MOS

8/15/2019 BM Report Template

22/38

22

MOS Plot

8/15/2019 BM Report Template

23/38

23

Receive Level

8/15/2019 BM Report Template

24/38

24

Coverage Plot

8/15/2019 BM Report Template

25/38

25

Codec Distribution

8/15/2019 BM Report Template

26/38

26

Handover Performance

Handover Details Value

WCDMA soft handover(Num) 3222

WCDMA soft handover failure(Num) 0

WCDMA soft handover success rate (%) 100%

GSM handover(Num) 203

GSM handover failure(Num) 1

GSM soft handover success rate (%) 99.51%

WCDMA -> GSM handover(Num) 22

WCDMA-> GSM handover failure (Num) 0

WCDMA -> GSM handover success rate (%) 100%

WCDMA handover rate (%) 93.50%

GSM handover rate (%) 5.86%

WCDMA -> GSM handover rate (%) 0.64%

8/15/2019 BM Report Template

27/38

27

Dropped Call Analysis

8/15/2019 BM Report Template

28/38

• FTP Performance

 – FTP DL & DL

• HTTP Performance.

• Email Performance

• GPRS Attach & PDP Context

• SMS

• MMS

• Technology distribution

• Modulation distribution

• RLC Throughput

• CQI Distribution

• MCS Distribution

Contents-Data benchmarking

8/15/2019 BM Report Template

29/38

29

FTP Performance

8/15/2019 BM Report Template

30/38

30

HTTP Performance

>

>

8/15/2019 BM Report Template

31/38

31

Email Performance

8/15/2019 BM Report Template

32/38

32

Attach & PDP Context Performance

8/15/2019 BM Report Template

33/38

33

Technology Distribution

8/15/2019 BM Report Template

34/38

34

Modulation Distribution

>

8/15/2019 BM Report Template

35/38

35

RLC DL Throughput

8/15/2019 BM Report Template

36/38

36

CQI Distribution

8/15/2019 BM Report Template

37/38

37

MCS Distribution

8/15/2019 BM Report Template

38/38

38