Methodical Procedure to Measure the Data Transmission ... · PDF fileMethodical Procedure to...
Transcript of Methodical Procedure to Measure the Data Transmission ... · PDF fileMethodical Procedure to...
Methodical Procedure to Measure the Data
Transmission Speed in Mobile Networks
in Accordance with the LTE Standard
Draft: release 2013 – 03 – 28
___________________________________________________________________
Methodical Procedure to Measure Data Transmission Speed in Mobile Networks in accordance with the LTE Standard
Section 1
Purpose of the document
This document specifies in more detail the procedure of measuring the data transmission speed specified in item 5 of the Office’s document: Annex 3 to the Invitation to Tender for the Award of the Rights to Use Radio Frequencies for Providing a Public Communications Network in the 800 MHz, 1800 MHz and 2600 MHz Bands [1].
Section 2
Definition of concepts
For the purposes of this methodology, the following terms have the following meanings:
a) Measurement square Kx,y is a 100x100m standardised square with an exactly defined GPS location and orientation in accordance with the Czech territory coverage document prepared by the Office. The x, y indices show the relative location to the x, y axes, which are not interconnected with the maps of the Czech Republic. Each square has a unique identifier, denoted as ID, which consists of the row and column number of the initial graticule. Each square’s attributes include its association with community and district, the number of population in the square and the information whether it belongs to motorways, expressways or rail corridors;
b) Rank k measurement sample is a continuous time interval of 1 second during which the transmitted test data volume w(k) is measured in bytes, where k is a positive integer, indicating the sample’s rank in time;
c) Rank k sample of data transmission speed ),(,
kKvyxd
is the speed of data transmission pertaining
to measurement square Kx,y, where k is a positive integer indicating the sample’s rank in time. The value of data transmission speed in bits per second is obtained from the volume of data w(k) in bytes transmitted in the kth second-interval as follows: )(8),(
,kwkKv
yxd ;
d) Average data transmission speed )(,yxd
Kv is the data transmission speed taken from all the N
data transmission speed samples ),(,
kKvyxd
measured in the given measurement square Kx,y as
follows:
N
k
yxdyxdkKv
NKv
1
,,),(
1)( ;
e) The required minimum value of data transmission speed mind
v is the speed value of 2 Mbit/s
(2,000,000 bit/s) for the downlink direction within 7 years from the finality of the acquisition of the allocation. Afterwards the minimum value of data transmission speed is increased to 5 Mbit/s (5,000,000 bit/s) for the downlink direction. This speed applies to one piece of mobile equipment and one SIM;
f) Relative coverage success rate R(Kx,y) is a dimensionless number between 0 and 1 (or percentage between 0 and 100 %) indicating the rate of successful coverage of a given stationary point or
measurement square Kx,y for the defined limit mindv . It is calculated as follows:
)(
)()(
,
,
,
yx
yxOK
yxKN
KNKR , where NOK is the number of measurement samples in the given measurement
square Kx,y, or which the condition of min,),(
dyxdvkKv is satisfied, and N is the total number of
measurement samples of data transmission speed in the given measurement square Kx,y.
This parameter is determined so as to ensure that R(Kx,y) ≥ 0.5;
g) Measurement time Tm is a continuous time interval in seconds, during which the test data are transmitted and the data transmission speed is measured;
h) Interval between measurements Tp is a continuous time interval in seconds, during which no test data are transmitted and the data transmission speed is not measured;
i) Measurement period T = Tm+Tp is a time interval in seconds consisting of two immediately succeeding intervals: the measurement time and interval between measurements;
j) Measurement series is the sequence of a finite number M of successive measurement periods T, where M is a positive integer;
k) Number of repetitions is a series of measurements whose number is L, where L is a non-negative integer.
Section 3
Initial conditions of data transmission speed measurement
(1) A measuring terminal or an equivalent piece of equipment (modem and computer) is used for the data transmission speed measurement. The software in the terminal should be able to record the time course of data transmission speed at one-second intervals and then to make statistical calculations, using the measured data, under the following conditions:
a) The measurement is performed using a measuring device via the network of the operator being tested against a server with guaranteed Internet connectivity of at least 1 Gbit/s and with a computing performance which at least ensures that the measurement is not adversely affected;
b) The measurement is performed on business days between 7 a.m. and 7 p.m., unless the measurement itself requires the measurement to be performed at any other time;
c) SIM cards with activated public (personal) mobile internet access service with the highest available data limit, or with no limit, are used for the measurement;
d) The measurement is performed on the IP layer, using the TCP protocol downlink from the server to the measuring terminal;
e) UMTS measurement will be performed as alternative to the LTE measurement for a transient period of 5 years in accordance with [1] so that the measuring terminal (modem) is primarily set at the LTE mode: if LTE is unavailable, it automatically switches to UMTS;
f) The current location of the measuring terminal will be monitored by a GPS receiver with a ±5 m position identification error and with a 5% maximum probability of position identification error;
g) The location of the measuring antenna must ensure that the adverse impacts of the measurement car on the measurements being taken is minimised.
(2) All measurements must be performed with three TCP connections simultaneously open under the following conditions:
a) The method of measurement and of interpreting the results is specified in detail in the IETF RFC 6349 document [2], unless otherwise provided;
b) TCP flows are generated continuously during the entire time of measurement;
c) With respect to the assumed transmission speed and the value of delay in data flow transmission, the size of the TCP reception window will be set at a value of at least 64 kB for each open session;
d) The size of the packets must be set so as to prevent packet fragmentation in the entire network being tested.
Section 4
Measuring data transmission speed
(1) Data transmission speed is measured at a single point with no movement of the measuring terminal (stationary measurement) or in a drive test. The calculations of coverage and data transmission speed are described in detail in Section 5 below.
(2) Stationary measurement of data transmission speed in mobile networks will be performed [1] as follows:
a) During random measurement to check compliance with the conditions specified in the tender, measurements will be taken within four consecutive hours at least four times in each hour with an interval between measurements of at least 10 min. (see Section 5, Subsection (1));
b) In the case of an investigation upon a complaint of interference with network operation or complaint of failure to maintain the data speed guaranteed on the basis of agreement between the client and the service operator, the measurement will be taken for a period of 1 hour (see Section 5, Subsection (2)).
(3) In stationary measurement (measurement at a single point with no movement of the measuring terminal), the requirement for the desired speed is met if and when:
a) The transmission speed reaches the required value (2 Mbit/s or 5 Mbit/s) in at least 50% of the measurement samples;
b) The average speed for all measurements reaches at least 75% of the required value (1.5 Mbit/s or 3.75 Mbit/s);
c) Should the above conditions of speed fail to be met, one repeated measurement will be taken for verification.
(4) For the purposes of checking the coverage of a populated area, continuous measurement of the coverage of such an area is performed by drive test measurement at a drive speed of 40 km/hour (if this is impossible due to the conditions prevailing at the time of measurement, a lower speed is used) along the major roads pertaining to the measurement site. The requirement for the desired data transmission speed is met if and when:
a) The transmission speed in the given 100x100m square reaches the required value (2 Mbit/s or 5 Mbit/s) in at least 50% of the measurement samples;
b) The average speed for the measurements reaches at least 75% of the required value (1.5 Mbit/s or 3.75 Mbit/s);
c) One repeated measurement will be taken for verification in the squares where the above speed conditions are not met;
d) Major roads should be understood to mean highways and local roads through towns/villages and the village greens and town squares.
(5) For the purposes of checking the coverage of motorways and speedways [1], continuous measurement is performed by drive test measurement at a drive speed of 60 km/hour (if this is impossible due to the conditions prevailing at the time of measurement, a lower speed is used) along the entire length of the road. Data communication must be available in at least 90% of the measurement squares that are crossed by the road. The requirement for the desired data transmission speed is met if and when:
a) The measurement is performed four times in succession, i.e. by driving twice in one direction and twice in the other direction along the measured segment of the motorway or speedway. The measurement samples from all four measurements are taken as a whole, i.e., as a file of values obtained in the given 100x100m squares;
b) The transmission speed in the given 100x100m square reaches the required value (2 Mbit/s or 5 Mbit/s) in at least 50% of the measurement samples;
c) If the number of measurement samples in the given 100x100m square is lower than 5, the given square is not used for coverage calculation.
(6) Coverage of transit rail corridors will be measured, using the Office’s equipment, near the railway lines (parallel roads, railway stations, flyovers or crossings) in the cases of complaints. In the case of measurement at one point, the stationary measurement procedure is used in accordance with item (3) above. In the case of measurement along parallel roads, continuous measurement by the drive test procedure is used in accordance with item (4) above.
Section 5
Calculation of coverage and data transmission speed
(1) Stationary measurement of coverage according to Section 4 Subsection 1(a) will be performed in a series of measurement periods T, the number of the periods being M = 16, with a measurement time Tm = 300 s and with an interval between measurements Tp = 600 s.
(2) Stationary measurement of coverage according to Section 4 Subsection 1(b) will be performed in a series of measurement periods T, the number of the periods being M = 4, with a measurement time Tm = 300 s and with an interval between measurements Tp = 600 s.
(3) For the stationary measurement, a measurement square Kx,y is defined, corresponding to the GPS position identified during the measurement.
(4) For the stationary measurement, the number of measurement samples N is calculated as follows: N = M.Tm.(1+L), where L is the number of repetitions.
(5) For the drive test measurements, no measurement time Tm and no interval between measurements Tp are determined. The measurement is taken on a continuous basis and its length depends on the area being tested, i.e. on the number and composition of the measurement squares Kx,y, in which the coverage measurements are being performed.
(6) For the drive test measurements, a measurement square Kx,y, corresponding to the GPS position identified during the measurement, is defined for each measurement sample.
(7) For the drive test measurements, the number of measurement samples )(,yx
KN of data transmission
speed in the given measurement square Kx,y is defined on the basis of Item (6) by summing up all the
samples belonging to the given square.
(8) The requirement for the desired data transmission speed in the given measurement square Kx,y or at a stationary point within the given measurement square Kx,y is met if and when the following two conditions are simultaneously met:
a) The data transmission speed ),(,
kKvyxd
reaches a value of mindv in at least 50% of all
measurement samples in the given measurement square Kx,y. The relative coverage
success rate must therefore be 5.0)(
)()(
,
,
, yx
yxOK
yxKN
KNKR , where NOK(Kx,y) is the number of
measurement samples k, for which the condition min,
),(dyxd
vkKv is satisfied, and N(Kx,y) is
the total number of the measurement samples of data transmission speeds in the given
measurement square Kx,y.
b) The average data transmission speed )(,yxd
Kv from all N(Kx,y) measurement samples in the
given measurement square Kx,y must reach at least 75% of the mindv , which means that
min
)(
1
,
,
,75,0),(
)(
1)(
,
d
KN
k
yxd
yx
yxdvkKv
KNKv
yx
must be valid.
(9) In the case of repeated measurements, the number of measurement repetitions being L, the calculation is performed as follows:
a) The relative coverage rate is calculated by summing up the number of samples of the basic measurement with the index 0, and of the repeated measurements, in the given measurement
square Kx,y. The value
L
i
iyxOKyxOKKNKN
0
,,)()( and
L
i
iyxyxKNKN
0
,,)()( is therefore used in item (8)
above.
b) The resultant average data transmission speed )(,yxd
Kv from the basic measurement with the
index 0 and from the repeated measurement, in the given measurement square Kx,y is calculated
as follows: min
0
,,75,0)(
)1(
1)(
d
L
i
iyxdyxdvKv
LKv
.
(10) The number of repetitions L is as follows:
a) For measurement without repetition the L = 0.
b) For measurement with repetition according to Section 4 Subsection 3c) and 4c), the L = 1.
c) For measurement along motorways and speedways by driving twice in one direction and twice in the other direction, as referred to in Section 4 Subsection 5), the L = 3.
(11) The tender requirement for the desired data transmission speed according to Item 5 in the document [3] is met if and when the number of population in the squares meeting the conditions of Item (8) of this Methodical Procedure is at least 95% of the total population of the community or district.
(12) The tender requirement for measurement along motorways and speedways is met if and when the number of squares JOK meeting the conditions of Item (8) represents at least 90% of the total number of
squares Kx,y containing at least 5 measurement samples, i.e., 9.0J
JOK . Squares Kx,y that are crossed
by the road being measured but contain less than 5 measurement samples are a priori excluded from the evaluation.
References and sources:
[1] ČTÚ [Czech Telecommunication Office]: Appendix 1 to the Methodology of Data Speed Measurement according to Item 5 of Annex 3 to the Invitation to Tender for the Award of the Rights to Use Radio Frequencies for Providing a Public Communications Network in the 800 MHz, 1800 MHz and 2600 MHz Bands
[2] IETF: rfc6349 Framework for TCP Throughput Testing
[3] ČTÚ: Annex 3 to the Invitation to Tender for the Award of the Rights to Use Radio Frequencies for Providing a Public Communications Network in the 800 MHz, 1800 MHz and 2600 MHz Bands
Annex 1
to the Methodical Procedure to Measure Data Transmission Speed in Mobile Networks in
Accordance with the LTE Standard
Based on Item 5 of Appendix 3 to the Invitation to Tender for the Award of the
Rights to Use Radio Frequencies for Providing a Public Communications
Network in the 800 MHz, 1800 MHz and 2600 MHz Bands
Contents
1. Terms and acronyms used ............................................................................................................... 9
2. Measurement of data transmission speed ..................................................................................... 13
2.1. Stationary measurement ....................................................................................................... 13
2.2. Drive test measurement ........................................................................................................ 14
3. Detailed description of the measurement method ...................................................................... 16
3.1. Detailed description of TCP protocol setting ........................................................................ 17
3.2. Calculation of data transmission speed ................................................................................ 19
1.Terms and acronyms used
Interval between measurements OM
is a continuous time interval in seconds, during which no network throughput test data are
transmitted. OMk – is the kth interval between measurements within the contemplated series of
measurements, see below.
Measurement sample MV
is a continuous time interval in seconds, during which the volume of transmitted test data is measured
in bytes. MVk – is the kth measurement sample within the contemplated series of measurements.
The instantaneous value of the transmitted data volume w1(tz) is taken at time tz at the beginning of
the MVk measurement sample, and w2(tk) at the end (See Fig. 1). From these data, the transmission
speed sample vp(MVk), is then calculated for the MVk measurement sample.
Fig. 1 – Principle of stationary measurement [see next page for translation of legend].
začátek čtyřhodinové série / beginning of four-hour series čtyřhodinová série / four-hour series konec čtyřhodinové série / end of four-hour series hodinová série / one-hour series PHPR / required value of transmission speed vzorek měření nevyhovuje / measurement sample not satisfactory čas / time
čtyřhodinová série
hodinová série #1 hodinová série #2 hodinová série #3 hodinová série #4
vzorekměření
odstupměření
čtyřhodinová série = 4 x hodinová sériehodinová série = 4 x měřicí periodaměřicí perioda = vzorek_měření + odstup_měření
měřicí perioda
čas
…. ……...
začátek čtyřhodinové série konec čtyřhodinové série
přijatá
dataW(t)[B]
čas [s]
w1
w2
PHPR
vzorek měřenínevyhovuje
čas [s]
Tvz /2 Tvz /2
Tvz
),(),(
),(),(8),(
,,
,1,2
,
kyxzkyxk
kyxkyx
kyxpMVKtMVKt
MVKwMVKwMVKv
),( , kyxp MVKv
kMV
zt kt
12 nebo 4N kde ,.75,0),(1
)(1
,,
PHPRMVKvN
KvN
k
kyxpyxp
1MV 2MV 3MVkMV 2kMV
NMV
PHPRMV .5,05 PHPRMVMV kk .5,0, 43
5,0)(
),( že platí, :}{
,
,
N
MVKR
PHPRMVKvkMVMV
yx
kyxpk
vzorek měření / measurement sample odstup měření / interval between measurements měřicí perioda / measurement period čtyřhodinová série = 4 x hodinová série / four-hour series = four times a one-hour series hodinová série = 4 x měřicí perioda / one-hour series = four times a measurement period měřicí perioda = vzorek měření + odstup měření / measurement period = measurement sample + interval between measurements přijatá data / data received [Equations on the right below]: …, kde N = 4 nebo 12 / …, where N = 4 or 12
…} : k platí, že vp … / …} : k it holds that vp …
--------------------------------
Measurement square yxK ,
The CTO [Czech Telecommunication Office] drew up the coverage of the Czech territory with standardised 100 x 100m
squares with exactly defined GSP location and orientation. The x, y indices are for information only, indicating the notional position
in relation to axes x, y, which are not related to the maps of the Czech Republic. Each square has a unique identifier, denoted as
ID, which consists of the row and column number of the initial graticule [2].
Transmission speed sample ),( , kyxp MVKv
is the transmission speed sample in square yxK , pertaining to the measurement sample in time, kMV .
Initial data volume ),( ,1 kyx MVKw
is the received data volume in bytes, registered at the beginning ),( , kyxz MVKt of measurement sample
MVk in square yxK , .
Final data volume ),( ,2 kyx MVKw
is the received data volume in bytes, registered at the end ),( , kyxk MVKt of measurement samples
MVk in square yxK , .
Measurement period MP
is a time interval in seconds consisting of two immediately succeeding intervals: the measurement
sample (MV) interval and interval between measurements (OM); it holds that kkk OMMVMP ,
where MPk – is the kth measurement period within the contemplated series of measurements.
Series of measurements SM
is generally the sequence of a finite number N of successive measurement periods MPk, k = 1..N. One-
hour series of measurements (HSM) and four-hour series of measurements (ČSM) are determined
according to the measurement type – see below. N=4 for HSM and N=16 for ČSM, unless otherwise
provided.
Measurement time ČOM
is generally a discontinuous period of time defined by the enumeration of the days and time intervals
during which any series of measurements can be performed (HSM, ČSM). Unless specifically provided
otherwise, the ČOM is defined for the purposes hereof as a set of all business days in the year with a
closer definition of the time interval from 7.00 h to 19.00 h for each such day.
One-hour series of measurements HSM
is a time interval in seconds defined as a continuous chain of four measurement periods immediately
succeeding each other; HSM = 4321 MPMPMPMP . An HSM can start at any time within the
defined measurement time ČOM, unless specifically provided otherwise. Where more HSM series are
used in the measurement, see below, the individual HSMs are distinguished by an inferior letter, i.e.,
HSMk, k = 1..N, where N is the number of HSMs in the given series.
Four-hour series of measurements ČSM
is a time interval in seconds defined as a continuous chain of four one-hour measurement periods HSM
immediately succeeding each other; ČSM = 4321 HSMHSMHSMHSM . A ČSM can start at any
time within the defined measurement time ČOM, unless specifically provided otherwise. Where more
ČSM series are used in the measurement, see below, the individual ČSMs are distinguished by an
inferior letter, i.e., ČSMk, k = 1..N, where N is the number of repeatedly performed ČSMs.
Stationary four-hour/one-hour series of measurements StM
is measurement performed during the entire time of any series of measurements (ČSM or HSM) with
the measurement terminal in a fixed position S, determined by means of a fully synchronised GPS
receiver. The StM’s key characteristic is the terminal’s rest position, given by the GPS coordinate with
a 5% maximum probability of error in determining the position and speed. Each measurement point
S is part of one of the defined squares, i.e. yxKSyx ,:, .
For the purposes hereof, the following parameters are determined for the stationary four-hour series
of measurements:
OMk=600 s, MVk=300 s, k = 1..16, i.e., ČSM = 14,400 s.
For the purposes hereof, the following parameters are determined for the stationary one-hour series
of measurements:
OMk=600 s, MVk=300 s, k = 1..4, i.e., HSM = 3,600 s.
Continuous drive test measurement KMzaJ
is measurement performed while in movement, using a mobile data terminal. The measuring antenna
must be located so as to minimise the adverse effects of the measuring vehicle on the measurements
being taken, typically including signal reflection from the roof of the vehicle, EMC interference in the
vicinity of the antenna system, vibrations while driving etc.
For the purposes hereof, the following parameters of the series of measurements are determined for
the continuous drive test measurement:
MVk = 1 s, OMk = 0 s, k = 1 . (number of one-second route samples).
Required value of transmission speed PHPR
For 7 years from the finality of the acquisition of the allocation, it is set at a value of at least 2 Mbit/s
(PHPR = 2 Mbit/s) downlink, and afterwards it is increased to at least 5 Mbit/s (PHPR = 5 Mbit/s)
downlink. This speed applies to one piece of mobile equipment and one SIM .
Measurement route TR
is the route driven with the measuring vehicle. The measurement route has its start point and end
point. The drive direction is denoted as XY, where X is the identifier of the start point and Y is the
identifier of the end point (typically the GPS coordinates). The measurement route is given by the set
of measurement points TR1 = {a1,.. al} for the first drive, TR2 = {b1,.. bm} for the second drive, TR3 = {c1,..
cn} for the third drive and TR4 = {d1,.. do} for the fourth drive. In some case hereunder, only certain route
point sets are used, e.g., only TR1, or TR1 and TR2. With respect to the fact that each drive along the
route has its specific speed profile, it generally holds that onml .
Measurement point aw
is a measurement point, defined by GPS coordinates and falling within the yxK , square, and belonging to
the set of points of the measurement route during the first drive along the route (AB direction).
Measurement point bw
is a measurement point, defined by GPS coordinates and falling within the yxK , square, and belonging to
the set of points of the measurement route during the second drive along the route (BA direction).
Measurement point cw
is a measurement point, defined by GPS coordinates and falling within the yxK , square, and belonging to
the set of points of the measurement route during the third drive along the route (AB direction).
Measurement point dw
is a measurement point, defined by GPS coordinates and falling within the yxK , square, and belonging to
the set of points of the measurement route during the fourth drive along the route (BA direction).
Average speed )( ,yxp Kv
is the average speed for all stationarily measured samples of transmission speeds ),( , kyxp MVKv for
all kMV in the given square yxK , or for all transmission speed samples
},,,:)( ),( ),( ),({ ,yxdcbadpcpbpap Kdcbadvcvbvav
Relative rate of coverage failure )( ,yxKR
is a dimensionless number between 0 and 1 (or percentage between 0 and 100 %) indicating the rate of
failure to cover a given place over time in stationary measurement in the Kx,y square for the defined PHPR
limit, or failure to cover the Kx,y square while driving through the square , for the defined PHPR limit. This
parameter is determined so as to ensure that 5,0)( , yxKR .
Mobile data technology indicator )( kaMDM
is a text chain having an “LTE” value for the LTE technology and a “UMTS” value for the UMTS
technologies at the ak measurement point. Its values may be different for other data modes but this
directive is not intended for other modes.
2. Measurement of data transmission speed
2.1. Stationary measurement In compliance with item 5 of Annex 3, data transfer in mobile networks will be measured as follows:
a. To address a situation where the result of coverage calculation performed by the CTO is
lower than 90% of the population of the district, the results will have to be re-calculated
according to the ITU diffraction model, followed by a four-hour stationary measurement
and the evaluation thereof.
b. To perform measurements for CTO’s purposes for checking the coverage, a one-hour
stationary measurement is performed.
Stationary measurement (StM) is performed in order to verify the required value of transmission
speed (PHPR) for the invariable site S of the data terminal with the given GPS coordinates. The GPS
position identification may involve an error of ±5 m at the maximum, with a 5% maximum
probability of position identification error. The stationary location S lies in a certain Kx,y square
100 m x 100 m in size ( yxKS , ). The conditions surrounding the measurement are specified at
this point by Item 4 of Annex 3.
In stationary measurement, the tender conditions are met when:
a. The transmission speed ),( , kyxp MVKv reaches the PHPR value in at least 50% of all
measurement samples kMV (for 12..1k ) in the given four-hour series of measurements
at the given site S. The relative rate of coverage failure must be 5.0)( , yxKR ; for the
calculation, see Fig. 1.
b. The average data transmission speed )( ,yxp Kv for all measurements within the given four-
hour series of measurements at site S must reach at least 75% of the PHPR; for the
calculation, see Fig. 1.
c. Should the above conditions of speed fail to be met, a second four-hour series of
measurements ČSM2 will be performed for verification. Transmission speeds pertaining to
the respective measurement samples are averaged as follows in this case: MVk=[MVk,1+
MVk,2]/2 (MVk,1 – measurement sample from the first series ČSM1; MVk,2 – measurement
sample from the second series ČSM2, for 12..1k ), and the subsequent evaluation will be
performed according to items a and b above. The second series of measurements will be
repeatedly performed at the same start of day as the first series of measurements.
The condition of maintaining the required value of transmission speed (PHPR) applies to the entire
time interval specified in the PHPR definition.
2.2. Drive test measurement In compliance with item 6b) of Annex 3, measurement of the coverage of a populated area is
performed as continuous drive test measurement (KMzJ) at a drive speed of 40 km/hour (a lower
speed is used where this is impossible due to the conditions prevailing at the time of
measurement – i.e., speed limit and the traffic situation) along the major roads pertaining to the
measurement site. For details see Figs 2, 3, 4. Speed measurement using a GPS receiver may
involve a position identification error of up to ±5 m and a speed determination error of up to 2
km/h, with an up to 5% probability of error in position and speed determination.
The requirement for the desired speed is met under the following conditions:
a. The transmission speed )( wp av reaches the PHPR value in at least 50% of all measurement
points wa , for all )(.. nkkw , where yxnkkk Kaaa ,1 },..,{ square for the first drive
along the TR1 measurement route. The relative rate of coverage failure for the given yxK ,
square must therefore be 5,0)( , yxKR , i.e.,
n
KMVKR
yx
yx
1
)()(
,
, , where
}},..{)( že platí,:{)( ,, yxnkkwwpwyx KaaaPHPRavwaKMV
[platí, že = it holds that]
The above must apply to all yxK , squares that are crossed by the TR1 measurement route, i.e., for yx, it
holds that 0,1 yxKTR
b. For all the wa measurement points, the average data transmission speed )( ,yxp Kv for all
)(.. nkkw , where yxnkk Kaa ,}..{ square for the first drive along the TR1 route, must
be larger than 75% of the PHPR for the given square, i.e.,
75.01
)(
)( ,
n
av
Kv
nk
kw
wp
yxp
c. In the squares where the above speed conditions are not met, the measurement will be
repeated for verification, i.e., the measuring vehicle will drive along the route again under
the same conditions. The average value of the data transmission speed for all the measures
that have been performed will be used as the result. It is admissible in such a case not to
drive along the entire route and to cover only its parts that cross the squares where the
above conditions are not met. If such is the case, the following relations will apply to the
average data transmission speed and relative rate of coverage failure:
PHPRmn
bvav
Kv
ml
lr
rp
nk
kw
wp
yxp .75.02
)()(
)( ,
5.02
)()(
,
,
mn
KMVKR
yx
yx
},)(),( že platí, ,:,{)( ,, yxrwrpwprwyx KbaPHPRbvavrwbaKMV
[platí, že = it holds that]
In this case it holds that measurement points wa pertain to the first measurement along
the route TR1 and points rb to the second measurement along the route TR2. From both
measurements, only those points are included in the above equations that fall within the
same square, i.e., )(.. nkkw , )(.. mllr where yxmllnkk Kbbaa ,},..,,..{ .
d. Major roads should be understood to mean the A roads and B roads that cross the
towns/villages and also the village greens and town squares.
In compliance with item 6d), coverage measurement is performed as continuous drive test
measurement at a drive speed of 60 km/hour (a lower speed is used where this is impossible due
to the conditions prevailing at the time of measurement – i.e., speed limit and the traffic situation)
along the entire length of the motorways and expressways.
Data communication must be available in 90% of the 100m x 100m squares, defined by the Office,
which are crossed by the road. The availability of the given transmission technology in a certain
yxK , square is measured at each of its measurement points yxnkk Kaa ,}..{ as indication, with the
)( kaMDM variable, whether the network supports the LTE or UMTS technology at the given
moment. A yxK , square, crossed by the measured TR1 route of a motorway or expressway, is O.K.,
if the condition )"")(|"")(( UMTSaMDMLTEaMDM kk -> TRUE holds good for all its
measurement points yxnkk Kaa ,}..{ . A motorway and expressway meets the above condition if
the condition is satisfied by 90% of the squares crossed by it. After 7 years elapse from obtaining
the authorisation, the above condition will change to "")( LTEaMDM k -> TRUE.
Traffic is faster on motorways and this is reflected in a smaller number of the ak measurement
points within one yxK , square (see Fig. 4 left above). To make the statistics more credible and to
avoid ‘discrimination’ in the measurement of squares with a smaller number of points, compared
to those with more points, the measurement will be performed four times: twice in one direction
and twice in the other direction along the same route. Four sets of measurement points will thus
be obtained: aa, cc (direction AB) and bb, dd (direction BA).
The desired speed is reached under the following conditions:
a. Data transmission speed pv reaches the PHPR value in at least 50 % of all measurement
points:
aa , for all )(.. nkka , where yxnkk Kaa ,}..{ ,
bb , for all )(.. mllb , where yxmll Kbb ,}..{ ,
cc , for all )(.. hssc , where yxhss Kcc ,}..{ ,
da , for all )(.. prrd , where yxprr Kdd ,}..{ .
Hence, the relative rate of coverage failure for the given square yxK , must be 5.0)( , yxKR
5.04
)()(
,
,
phnm
KMVKR
yx
yx, že platí, ,,,:,,,{)( , dcbadcbaKMV dcbayx
},,,)(),(),(),( ,yxdcbadpcpbpap KdcbaPHPRdvcvbvav [platí, že = it holds that]
b. The average data transmission speed )( ,yxp Kv for all measurement points at item a) above
must be larger than 75% of the PHPR for the given yxK , square, i.e.,
PHPRphmn
dvcvbvav
Kv
pr
rw
wp
hs
sw
wp
ml
lw
wp
nk
kw
wp
yxp .75.04
)()()()(
)( ,
3. Detailed description of the measurement
method Measurement of territory coverage by the data service can be characterised by a number of quality parameters. The monitoring
of the transmitted data volume w(t) as a function of time (see Fig. 4, the red line in the graph) is one of the simplest parameters.
Instantaneous transmission speed can be considered as a derived criterion, defined by the following equation:
dt
tdwtvp
)(8)( [bit/s, bytes, s]
Transmission speed and the transmitted data volume can be examined for both transmission directions; nevertheless, according
to the nature of transmission in the LTE networks, it is only the downlink speed (from the network to the mobile terminal) that can
be considered essential – at least in the initial phase. However, the achieved speed level of data transfer is not the only factor
underlying data transmission quality. Other factors include:
a) the rate of the loss of data units (IP packets) in the network,
b) the mean latency of data unit transmission,
c) the distribution of the latencies of data unit arrival.
The ideal situation is that all data units sent from the source arrive to the receiver with minimum delay and minimum scatter.
However, this can never be perfectly achieved. In this respect it is necessary to find measurement methods that can measure
separately all the above parameters and combine them into one quality parameter in the given transmission direction. The result,
however, may vary with how transmission quality is perceived by the application or even the end user. Generally it holds that the
closer the evaluation to the user, the more relevant it is, although it is at the same time very subjective: for example, each user
may use different criteria to evaluate the delivery of the service.
For evaluation under this directive, the regulator used the method of measuring the transmitted data volume by the TCP transport
protocol from the IETF TCP/IP family of protocols. More detailed information on the TCP protocol testing procedures can be found
in the RFC 6349 [1] recommendation where the method of the TCP protocol measurement and optimisation is described for
different transmission technologies, including instructions for analysing the data obtained from the measurements.
Note on the TCP Protocol
This solution has the following advantages:
a) The TCP protocol is and will be used je as the primary software interface between an
application and the network, and therefore, in this context, it is the closest to the
application (though not yet to the user).
b) The TCP is a protocol, which is sensitive to all the above parameters during data transfer,
including loss of some data units (TCP segments in this case), latency, and, in particular,
latency distribution. This means that if any parameter worsens TCP throughput is reduced
and, thereby, efficient data transfer speed also worsens. The need for simultaneous
monitoring of several parameters is thus avoided, as the TCP integrates them into one
parameter: data transfer throughput.
c) The TCP protocol guarantees the transmission, which means that the data delivered by the
TCP to the application are free of error.
d) Many important applications (WEB, Mail and others) use the TCP protocol and therefore
the measurements taken on its basis are significant for them.
However, the use of the TCP also has certain disadvantages:
a) If data units fail to be delivered, the TCP requests the transmitter to send the data again,
thus causing the data channel to handle data that would not exist if the transmission were
error-free. Although this serves with advantage as a network quality criterion, it may be
without effect on the operation of the applications that are not sensitive to outages: in such
a case, TCP transmission does not respond to the quality of the application – for example,
the transmission of audio or video flows. The volume of the repeatedly transmitted data is
reflected in the statistics of the total amount of transmitted data on the physical layer
(within the PDP context). This will cause a difference between throughput measured within
the TCP protocol and the total throughput measured on the physical layer.
b) The TCP protocol is sensitive to great and rapid fluctuation in network latency. Unnecessary
pauses in transmission or, on the other hand, unnecessary repetition of the data units that
have already been received occur in such cases. This can be used with advantage as a
network quality criterion because significant fluctuation manifests itself in a degradation
of TCP throughput. On the other hand, large fluctuation in latency may not matter at all in
certain applications.
c) The TCP protocol cannot be used to test multipoint links: TCP can only serve for point-to-
point transmissions. Multipoint communication requires the use of an adequate number of
individual point-to-point instances of TCP connection.
d) It may happen as a result of fluctuation in latency and a large number of packet errors that
the TCP protocol stops working – i.e., the data communication is either entirely
disconnected, or cannot be re-established. Such situations must be addressed during
measurement, and if they happen the series of measurements must be carried out again.
This issue will have a minimum effect in LTE networks, but it may occur during handover or
during switching between radios (e.g., LTE to UMTS).
3.1. Detailed description of TCP protocol setting
For transmission speed measurements during TCP transmission, it is necessary to have at one’s disposal the server part of the
measuring application, which must be able, upon establishment of TCP connection, to generate downlink data traffic at a speed
greater than the PHPR. This can be checked by prior measurement, where a fixed monitoring station (client) is connected directly
to the server to verify the maximum flow the server is able to generate on the given TCP. If the server is used to measure several
parallel TCP sessions, this will have to be verified separately for each session.
The key parameter for reaching a maximum throughput per TCP session (connection) is the size of the receiving window
(RCWND), with which the TCP receiver can control the speed at which the segments are sent from the TCP source (server in
our specific case), doing so dynamically within time, (if necessary). If this window is too small or the transfer latency is too big, or
both simultaneously, the TCP connection cannot use the maximum network capacity offered on the third layer (IP, in our specific
case), and the measurement will be distorted. For this reason, it is necessary to set the RCWND size with respect to reaching a
maximum throughput. The estimate for calculating the maximum achievable TCP connection throughput (expressed as
transmission speed here) for the given RCWND size and latency in both directions (Round Trip Time, RTT) is as follows:
RTT
RCWNDvp 8max [bits/s, bytes, s]
For easier understanding, this dependence is plotted in Fig. 2, where the transmission speed limits of 2 and 5 Mbit/s are also
indicated. The blue area in the graph represents an approximate limit for RTT in the networks in the combined LTE/UMTS mode.
In LTE networks alone, if they are not overloaded, the RTT latency may be up to 15 ms for almost 90% of the tests performed –
see, for example, the source in Fig. 3. To be able also to test speeds higher than just 2 or 5 Mbit/s, it is advisable to use optimum
RCWND sizes between 64 kB and 128 kB.
At present, many applications use a number of simultaneously open TCP sessions. To be able to monitor network behaviour
under these conditions, similar for the applications, the tests will be carried out with three TCP sessions running simultaneously
and with the measurement of transmission speed based on the data volume w, added up for all sessions together. In such a case,
a smaller window for one session will suffice to fill the data channel. The RCWND will therefore set at 64 kB for each session. As
a result, only the network, rather than the set parameters of TCP connection, will be a limiting factor.
To eliminate the influence of other ISPs (internet service providers), it is advisable to connect the testing TCP server as close as
possible to the operator’s network internet access starting point. The best solution is to place the testing server in the backbone
infrastructure, which interconnects all the key internet connection providers. We recommend to use the Ethernet connection
technology with a speed of at least 1 Gbit/s.
Fig. 2 – Dependence of maximum TCP connection throughput on the size of the RCWND window and
transmission latency
1
10
100
5 15 25 35 45 55 65 75 85 95 105
Vp[
Mbi
t/s]
RTT [ms]
RCWND=32 KB RCWND=64 KB RCWND=128 KB
RCWND=256 KB RCWND=512 KB
Fig. 3 – LTE network’s RTT latency (Source: http://www.signalsresearch.com/Docs/SRG%20Presentation%20-
%20PDF%20Version.pdf).
3.2. Calculation of data transmission speed
Data transmission speed is the relevant measurement parameter, which has to be calculated by the above equation. For this
purpose, the methodology introduces the concept of measurement sample (MV) for stationary measurement. The MV is the time
interval at the beginning and end of which the instantaneous value of the transmitted data volume is measured: w1(tz) and w2(tk),
respectively – see Fig. 1. Transmission speed is then calculated from these data by the following equation:
),(),(
),(),(8),(
,,
,1,2
,
kyxzkyxk
kyxkyx
kyxpMVKtMVKt
MVKwMVKwMVKv
, for the k th measurement sample in the series for a yxK ,
square.
The above equation shows the average transmission speed within a MVk sample. It is impossible in this case to identify the time
moment to which this speed applies. However, for easier understanding, a moment corresponding to the middle of the MV sample
is assigned to this speed value.
Stationary measurement
For stationary measurement, the MV time interval is set at 5 min. By using this length of measurement, we get the mean value of
transmission speed in the given square. To ensure compatibility of the tests with mobile measurement, measurements are taken
in one-second intervals within the above five-minute time interval, and the software calculates the instantaneous transmission
speed within these one-second measurement intervals. The transmission speed sample for the MV interval is then calculated as
the arithmetic mean for all one-second intervals. In other words, the MV measurement sample consists of shorter one-second
measurement intervals MIl, for l = 1..MVk.
k
MV
l
lyxp
kyxpMV
MIKv
MVKv
k
1
,
,
),(
),( , where
),(),(
),(),(8),(
,,
,1,2
,
lyxzlyxk
lyxlyx
lyxpMIKtMIKt
MIKwMIKwMIKv
Drive test measurement
The situation is more complicated for the drive-test measurement, because it is necessary not only to monitor the time of each
measurement sample but also to map them in the system of CTO squares. This situation is shown in Fig. 4. The network of 100
m x 100 m squares is drawn in the Carthesian coordinate system. Each square is labelled with its x and y indices.
The starting point of the measurement route is denoted as A and the end point as B. The red line represents the path of the
measurement route and connects the measurement points established while first driving in the AB direction. For the drive test
measurement, the first measurement sample is 1 second long and the measurement is continuously repeated with no interval OM
(=0 s) between the measurements. Generally speaking, each square can accommodate a different number of measurement
points, because of the varying speeds of the vehicle drive and the complexity of synchronising the MV measurement sample
origins at the edges of the squares. In addition, the spacing between the points along the route of the drive also varies. In this
measurement, the key issue is to ensure that the numbers of measurement points in all squares are the same or at least very
similar. If this fails to be achieved, the static calculation for cells with a higher number of measurement points will be more precise
than for cells with less measurement points.
To be able to eliminate this problem, it is advisable to repeat the measurement more times from one side and the other side,
possibly with a small change in the speed profile distribution. This is represented by the green line in the graph. In this way the
statistics of the squares are balanced out with respect to the impossibility to keep the speed unchanged during the entire drive.
Fig. 5 shows the principle of mapping the course of measuring the data transmission speed (which is calculated according to Fig
6 – left below) for each square, including the use of the equations indicated above.
Fig. 4 – Principle of the drive test measurement and the distribution of measurement points along the drive test
route.
For easier understanding, only two route measurements are shown here.
Trajektorie dráhy měření / Measurement route path
bod měření v místě čtverce Kx+1, y+1 ve směru trasy A-B (body „a“) / Measurement point at the site of the Kx+1, y+1 square
in the A-
B route direction (points “a”)
Bod měření v místě čtverce Kx+1, y+1 ve směru trasy B-A (body „b“) / Measurement point at the site of the Kx+1, y+1 square
in the B-
A route direction (points “b”)
Vyhodnocení bodu měření ve čtverci / Evaluation of measurement point in the square
Note on the drive test measurement
However, the above solution is of no help in the cases where the route crosses a very narrow tip of a square, which may fail to be
covered by a measurement point or may be covered by just one point – see Fig. 4, above, showing the coverage of a square by
measurement points when driving in another direction and at a different speed.
In these cases it is advisable to monitor whether this square is covered by signal from the same BTS station as the adjacent
squares and to see what their SINR is in such a case. Where the adjacent squares are covered by signal from the same BTS
station as the contemplated square corner and the adjacent squares’ SINR is comparable to that of the square corner, the
measurement point can be calculated as the average between the measurements in the adjacent squares. Where technical
conditions allow, small parts of the square can also be measured by reducing the speed in this part of the route path, thus making
it possible to place more measurement points in the tip of the square. The purpose of driving the route several times or for changing
the speed profile is to ensure that the total number of measurement points for each square does not differ from the average
number of measurement points by more than 20%. To be able to monitor the SINR in correlation (in terms of time) with the
measured throughput, the time bases of both measurements are synchronised.
vyhodnocení bodu
měření ve čtverci Kx+1,y+1
bod měření v místě čtverce Kx+1,y+1 ve směru trasy A-B (body „a“)
A
B
Kx,yy
Kx+1,y+1
Kx+2,y+2
Kx+1,y+2
Kx+2,y+3
Kx+3,y+3
Kx+4,y+3
Kx+5,y+3
Kx+5,y+4
bod měření v místě čtverce Kx+1,y+1 ve směru trasy B-A (body „b“)
Trajektoriedráhy měření A-B
Trajektoriedráhy měření B-A
y+1
y+2
y+3
y+4
x x+1 x+2 x+3 x+4 x+5 x+6
a1
a2
a3
a4
a5
ap
ap+4
aF
b5
b1
bk
bk+1
bM
b8
b9
Fig. 5 – Principle of drive test measurement and allocation of individual MV measurement samples to the specific yxK ,
square.
měření / measurement
patří do čtverce / belongs to square
platí, že / it holds that
měřicí vzorky rychlosti odpovídající bodům … / speed measurement samples corresponding to points …
směr jízdy / drive direction
měř
ení A
-B #
1m
ěřen
í B-A
#1m
ěřen
í A-B
#2m
ěřen
í B-A
#2
t
t
t
t
vp[bit/s]
vp[bit/s]
vp[bit/s]
vp[bit/s]
vp (ak) .. vp (ak+n) patří do čtverce Kx,y
vp (bl) .. vp (bl+m) patří do čtverce Kx,y
vp (cs) .. vp (cs+h) patří do čtverce Kx,y
vp (dr) .. vp (dr+p)patří do čtverce Kx,y
měřicí vzorky
rychlosti odpovídající bodům a- směr jízdy (A-B) #1
měřicí vzorky
rychlosti odpovídající bodům b- směr jízdy (B-A) #1
měřicí vzorky
rychlosti odpovídající bodům c- směr jízdy (A-B) #2
měřicí vzorky
rychlosti odpovídající bodům d- směr jízdy (B-A) #2
Kx,y
PHPRphmn
dvcvbvav
Kv
pr
rw
wp
hs
sw
wp
ml
lw
wp
nk
kw
wp
yxp .75,04
)()()()(
)( ,
5,04
)(
},,,{)(),(),(),(
že platí, ,,,:},,,{
,
,
phnm
MVKR
KdcbaPHPRdvcvbvav
dcbadcbaMV
yx
yxdcbadpcpapap
dcba
Fig. 6 – Calculation of transmission speed patterns from the received data volume (w/t) and their allocation to
measurement points (left); mapping the measurement samples, points in the square (below right); vertical
view of the squares and the allocation of measurement points (below right); distribution of measurement
points for different passages through the square and different speed at a MV length = 1 s (above left)
bajtech / in bytes
měření a – b # 1 (body a) / measurements A – B # 1
(points a)
(pro body b, c, d, je to stejné / for points b, c, d it is the
same)
směr jízdy / drive direction
t(axp) – čas hrany čtverce 1 / t(axp) – edge time of square
1
t(ax p+2) – čas hrany čtverce 2 / t(axp+2) – edge time of
square 2
měření A-B #1 (body a)(pro body b,c,d je to stejné)
t(ap) t(ap+1) t(ap+5)
w(ap)
w(ap+1)
w(ap+2)
w(ap+3)
w(ap+4)
w(ap+5)
t
v p(ap)
v p(ap+1)
v p(ap+2)v p(ap+3)
v p(ap+4)
)()(
)()(8)(
1
1
pp
pp
ppatat
awawav
W(t)[bajtech]
vp(t)[bit/s]
měření A-B #1 (body a)(pro body b,c,d je to stejné)
vp(t)[bit/s]
t(ap) t(ap+5)
w(ap)
w(ap+1)
w(ap+2)
w(ap+3)
w(ap+4)
w(ap+5)
t
v p(ap)
v p(ap+1)
vp(a
p+
2)
v p(ap+3)
v p(ap+4) W(t)[bajtech]
w(axp)
w(axp+2)
t(ap+1) t(ap+2)
t(axp)-čas hranyčtverce 1
t(axp+2)- čas hranyčtverce 2
ap+1
ap+2
t(ap+3)
ap
ap+3
Kx,y
Kx+1,y+1Kx,y+1
ap
ap+1
ap+2
ap+3
ap-1
axp
axp+2
směr jízdy
10
0 m
90 km/h
40 km/h
100 m
10
0 m
40 km/h
100 m
References
[1] RFC 6349 - Framework for TCP Throughput Testing.
[2] Vector map 100x100m used by the CTO – description.