Annex22 Enigma DE2.doc

7/29/2019 Annex22 Enigma DE2.doc

1/13

DOCUMENTTYPE 1 (13)

TypeUnitOrDepartmentHereTypeYourNameHere TypeDateHere

Project Enigma KPIs

Summary

This document is draft and for technical discussion only.

This document outlines the measurement techniques that will be put in place to verify the satisfactoryoperation of those areas of the O2 DE 3G radio network that are affected by the swap of radioequipment from Nortel to Nokia under ro!ect Eni"ma# $t is e%pected that this draft document willunder"o several iterations as the boundaries of responsibility durin" the equipment swap process arefinalised#

The basic principles underlinin" the swap process are& The 3G rf performance in terms of covera"e and interference after the swap shall be equivalent

or better than before the swap took place# The network quality in terms of call completion success rate and customer e%perience of data

services shall be equivalent or better than before the swap took place#

The swap process will involve the up"radin" of some sites from Omni Transmit' (ector )eceive *OT() +known within Nokia as )O,- to (ector Transmit' (ector )eceive *(T() + known within Nokia as ,E,-#The possibility e%ists that it will not be possible to include a limited number of sites as part of the swapevent# The a"reed measurement process will take these circumstances into account and is basedaround the followin" assumptions&

,onfirmation of basic rf function will be carried out in a method analo"ous to that used in a 2Gswap pro"ramme in the ./#

,onfirmation of 3G network quality will follow a process complimentin" the current O2 clusteracceptance procedure#

This document is based upon the O2 ./ ro!ect Eni"ma / $s document' authored by 0drian (harples')adio En"ineerin" O2 ./' with modifications to make it applicable to the O2 DE network#


2/13

DOCUMENTTYPE 2 (13)


TABLE OF O!TE!TS

(cope and Ob!ectives ........................................................................................................................................ 31# $ntroduction .................................................................................................................................................. 32# /ey 0ssumptions ........................................................................................................................................ 33# Drive Testin" ............................................................................................................................................... 5

# ,overa"e erification ................................................................................................................................. 5#1 4asic procedure ................................................................................................................................... 5

#1#1 OT() to (T() ,onversions ..................................................................................................... 6#2 rocessin" Of )esults ........................................................................................................................ 6

#2#1 ,orrected (i"nal 5evel + (ame (erver 0nalysis *3G- ............................................................ 6#2#2 ,umulative Distribution Of 4inned )(, ................................................................................ 6#2#3 alid 4ins ......................................................................................................................................

#3 0cceptance ,riteria .............................................................................................................................6# $nterference 0ssessment ...........................................................................................................................7# Network erformance / $s .......................................................................................................................

Network erformance .................................................................................................................................... !7#1#1 (uccess )ate Definitions ........................................................................................................... !7#1#2 Number Of 8easurements 0nd Tolerance ............................................................................... !7#1#3 E%clusions .................................................................................................................................... "7#1# Data Throu"hput .......................................................................................................................... "7#1#6 3G92G :andover erformance .................................................................................................. "

7#2 0cceptance ,riteria ........................................................................................................................... 1#;# $n9buildin" 8easurements ....................................................................................................................... 1#

;#1 $n9buildin" ,overa"e Tests ............................................................................................................... 1#;#2 $n9buildin" (ervice Tests ................................................................................................................... 1#


3/13

DOCUMENTTYPE 3 (13)


S OPE A!" OB#E TI$ESThis draft of the document defines the procedure for establishin" the satisfactory operation of the O2 DEradio network after the equipment swap events carried out under ro!ect Eni"ma# The ob!ective is todefine an unambi"uous test of acceptability of the network alon" the principle that the networkperformance must be equivalent or better than the performance of the network before the swap tookplace#

%. I!T&O"' TIO!

To verify the network performance followin" the swap of 3G radio equipment in the O2 DE network under ro!ect Eni"ma' metrics must be defined to enable a"reement between Nokia and O2 to be reached

re"ardin" the success of the swap process#

0 similar procedure was carried out as part of a 2G pro"ramme to swap Nokia equipment into the ./network# This measurement pro"ramme concentrated on the use of drive data to confirm covera"elevels and correct installation#

The process in ro!ect Eni"ma is further complicated by the fact that& (ite up"rades from OT() to (T() *)O, to ,E, in Nokia terminolo"y- will take place as part of

the swap# 5ow traffic levels in the O2 3G network make reliance on network performance counters

inappropriate# $nter9vendor handover will not be supported as part of the swap pro"ramme#

The principles employed in this process are that fundamental rf properties *covera"e and interference-will be verified# Network quality will be assessed in a method usin" call / $s *in line with the clusteracceptance testin" currently performed within O2 ./-#

(. KE) ASS'*PTIO!S

The statements in the remainder of this document are dependent&

There is insufficient customer traffic on the O2 DE 3G network for )N, counters to be used as akey measure of network performance before and after a swap event#

8easurements relatin" to the performance of the O2 3G network shall be reliant on drive testin"# The same 3G carrier frequency will be used durin" all swap events# re swap drive testin" will take place before the start of the swap of any ad!acent areas# ost swap drive testin" will take place after the completion of swap events in ad!acent areas# 0ll / $ testin" will be conducted with devices in dual mode# No new site inte"rations will take place between the pre9swap and post9swap periods# $nter9vendor handover will not be supported as part of the ro!ect Eni"ma equipment swap# 0 site within a swap event cluster that cannot form part of the swap event will not transmit durin"

the event verification drive testin"# 0s part of the swap event' some sites will be up"raded from OT() to (T() *)O, to ,E, in

Nokia terminolo"y-# The up"rade from OT() for (T() will remove a 6#7d4 attenuation in the downlink link bud"ets

which must be discounted in the measurement post processin"#


4/13

DOCUMENTTYPE $ (13)


Given that the downlink power reference points differ between Nokia and Nortel equipment' prior

to the pre9swap drive test the Nortel equipment parameter settin"s shall be confi"ured to providethe same top9of rack pilot channel *cpich- power that forms part of the standard Nokia databuild#

0 tolerance factor of 1d4 shall be allowed on si"nal level assessments to take account ofinaccuracies in the measurement process#

/ $s related to 3G downlink throu"hput will fail the principle of pre and post swap performanceequivalence due to known differences between Nokia and Nortel implementations# )esults on theNokia area of O2>s .T)0N network show mobile throu"hput values appro%imately ;?@ of thoseachieved in Nortel areas on the 3< kbit=s bearer# 0s a result' a modification factor will be used for comparisons in the swap process# This modification factor will be reviewed whenever desi"nchan"es are made that improve the Nokia .T)0N throu"hput deficit#

erformance / $ assessments will be made usin" appropriate test equipment#

The measurement process shall be reviewed after swap events to ensure that the need to meetthe overridin" principles is bein" achieved#


5/13

DOCUMENTTYPE 5 (13)


+. "&I$E TESTI!,

The routes within the swap event cluster are to be a"reed between O2 and Nokia# The drive tests willspan a minimum of A workin" days# The measurement period may be varied by a"reement on an eventby event basis# The selection of the drive route shall adhere to the followin" principles

0ll sectors of every site in the cluster are covered to derive the clear footprint of each cell'includin" areas with missin" sites#

3G92G transition areas are crossed# Bocus on residential areas' ma!or roads' and routes around key business centres' shoppin"

areas' tourist attractions and railway stations# 0dditional side streets are travelled in city centres# The entire event area is driven in a uniform fashion such that all areas are driven irrespective of

their distance from a cell site# *i#e# The route did not favour locations close to cell sites-# The event drive route shall be desi"ned to ensure that border areas between the event areas and

into surroundin" 2G areas are driven# The route should travel sufficiently close to the first tier of border cells to ensure that :O to cells outside the swap event is tested#

The border approach is illustrated in the fi"ure below#

-. O$E&A,E $E&IFI ATIO!

-.% Basic rocedure

re and post9swap drive tests shall be carried out alon" the a"reed route that will encompass all sectorsformin" part of the swap event# ,overa"e measurement shall be performed usin" a scanner#

8easurements shall be binned on a "eo"raphic basis into a 16?m % 16?m resolution "rid laid over thedrive route# Each measurement report shall be placed in the appropriate bin indicated by its associatedG ( location sample#


6/13

DOCUMENTTYPE 6 (13)


Bor each bin' two levels shall be reported# The first is the avera"e level *taken to be the avera"e of the

d4 values' not the linear avera"e- of all best server )(, measurement samples taken within the bin*i#e# the scramblin" code with the hi"hest )(, value in any measurement report# The secondmeasurement is the avera"ed )(, level' avera"ed in terms of d4' not linear avera"e' for the dominantserver within the bin# $n this case the dominant server is the scramblin" code with the lar"est number ofbest server samples received within the bin# $f two or more scramblin" codes have the same number ofbest server measurements then the one with the hi"hest avera"e si"nal value shall be selected# Thescramblin" code of the dominant server will also be stored for each bin#

#1#1OT() to (T() ,onversions

Chere an OT() to (T() up"rade has taken place' an additional processed set of data is requiredrelatin" to measurements from the up"raded site# Bor each post9swap measurement report that includes)(, values for scramblin" codes of collocated sectors' the linear sum of these measurements shall bestored to represent the value of the equivalent power measured from the OT() site# $t is this combinedvalue that will be used in power level comparisons for the site#

-.( Processing Of &esults

)aw )(, si"nal level plots shall be presented without binnin" or any of the si"nal level correctionsdescribed in the followin" sections to indicate the e%tent of poor covera"e in the area# These plots will becoloured accordin" to a"reed thresholds#

$n addition' uncorrected )(, level plots will be shown of the chan"e in the avera"e )(, level of binsin the area before and after the swap process# Each bin of these plots shall be coloured accordin" toa"reed thresholds#

lots shall be presented with each bin coloured accordin" to the scramblin" code of the dominant server'with a key provided showin" the mappin" of colour to scramblin" code# These shall be overlaid with adia"ram illustratin" the location of the base stations' the a imuth directions of the antennas and theplanned scramblin" codes to be transmitted from each of the antennas# These plots are required toverify that the scramblin" code plan has been implemented correctly and sectors have not been crossed#

#2#1,orrected (i"nal 5evel + (ame (erver 0nalysis *3G-

These plots will show the difference in )(, for bins that are served by the same dominant server bothpre and post swap# Chere an OT() to (T() conversion has taken place' the data from the sum of thecollocated sectors *as described (ection #1#1- shall be used# 4efore comparison with the pre9swapsi"nal level' the a"reed splitter insertion loss fi"ure shall be subtracted from the si"nal levels from theup"raded site#

The number of "eo"raphical bins covered by this dominant server both pre and post swap will also bepresented#

Chere there has been de"radation in the corrected )(, level in a bin followin" the swap event thisshall be taken to be indicative of a deficiency in the swap process for which corrective action isnecessary#

#2#2,umulative Distribution Of 4inned )(,


7/13

DOCUMENTTYPE (13)


The pre and post swap cumulative distributions shall be produced of the avera"e received level of

measurement samples within each valid bin for which a measurement has been taken both pre and postswap# The definition of a valid binF is "iven below#

Chere an OT() to (T() conversion has taken place' the data from the sum of the collocated sectors*as described (ection #1#1- shall be used# 4efore comparison with the pre9swap si"nal level' the a"reedsplitter insertion loss fi"ure shall be subtracted from the si"nal levels from the up"raded site#

$.2.3 alid 4ins

0 valid bin is one where all measured samples post swap are from sites that have been included in theswap event# 4ins that include a measurement from a site that has not been included in the swap event*and is not transmittin" post9swap- will be e%cluded from both pre and post swap cumulativedistributions# This situation will be reviewed after swap events to ensure that sufficient bins are bein"included to make this measurement a statistically valid approach#

The corrected )(, value at the 1?@ point of the cumulative distribution post swap shall not be lessthan the 1?@ point of the pre9swap cumulative distribution by an amount that is less than the tolerancefactor#

-.+ Acce tance riteria

0cceptance of 3G covera"e maintenance shall be taken to be the satisfaction of the measure in (ections#2#1# and #2#2# The failure to meet other covera"e criteria shall be taken to indicate an individual fault

that shall be rectified by remedial action by Nokia#

/. I!TE&FE&E! E ASSESS*E!T

Cith the up"rade of some sites from OT() to (T()' the direct comparison of the interferencelandscape in terms of Ec=$o is complicated by the introduction of new sectors since this brin"s with itadditional cell boundaries# To provide a pre and post9swap comparison the followin" procedure shall befollowed#

Brom the scanner data presented previously' the sum of the best three )(, values in anymeasurement report shall be summed *linear summation-# This shall be divided by the measured )(($value at that location "ivin" a total equivalent active set Ec=$oF# Bor each valid 16?m % 16?m bin theavera"e equivalent active set Ec=$oF shall be stored# This avera"e shall be the d4 avera"e of all

equivalent active set Ec=$oF values within the bin#

0 valid bin is as defined in (ection #2#3

0 plot of the difference between the pre9swap and post9swap equivalent active set Ec=$oF for each validbin shall be presented# These shall be coloured accordin" to a"reed thresholds#

The intention of this process is to identify a method for ensurin" that the pre and post9swap interferenceenvironment has not de"raded durin" the swap process but which also takes account of the alteration ofthe number of sectors in the area#

0 reduction in the equivalent Ec=$o in any bin that is "reater than the tolerance factor will be taken toindicate a fault in the swap process that shall be rectified by remedial action by Nokia#


8/13

DOCUMENTTYPE ! (13)


0. !ET1O&K PE&FO&*A! E KPIS

!et2or3 Performance

The network performance / $s for assessin" the network pre and post swap are based on the currentcluster acceptance criteria used within O2#

The / $s to be measured are shown in the table below#

Bearer *easurement$oice Overall call completion rate *8O, only- 3G=2G :andover success rate *8O, Only- PS "ata 0-4%(536 s 6earer 0-4+5-36 s 6earer D5 throu"hput D5 throu"hput .5 throu"hput BT session success rate

3G=2G ( handover success rate )ound Trip TimeA )ound Trip TimeA $ideo ideo overall completion rateA

AThese / $ requirements will be included in the first swap events and the requirement to continuethrou"h the swap process reviewed at that time#

7#1#1(uccess )ate Definitions

The definition of overall call completion success rate is *O,,()-

O,,() *,all 0ttempts 9 E%cluded ,alls 9 ,all (etup Bailures 9 ,all Drops- = ,all 0ttempts

This shall be applied for voice' data and videotelephony calls# The definition of e%cluded calls is "iven in(ection 7#1#3# Definitions of call events are "iven in (ection


9/13

DOCUMENTTYPE " (13)


The percenta"e call success rate after the swap shall be at equal to or hi"her than the call success rate

before the swap minus 1#6@# This 1#6@ tolerance is included to allow for statistical uncertaintyassociated with the call samplin" process#

7#1#3E%clusions

oice and data call handlin" shall be e%pected to handover between 3G and 2G networks# 0s a result'the only e%clusion of failed voice calls from the network performance / $s shall be those that can beidentified as due to&

Test equipment failure resultin" in collection file corruption 5ocation update clashes

.E issues *e#"# handset reset-

$n each case the e%clusion of a particular failed call shall be a"reed between Nokia and O2#

Bor videotelephony' the absence of a handover to 2G requires additional allowance for situations whereit has not been possible to include a site in the swap event# E%clusions for failed videotelephony calls willtherefore be allowed if the failure occurs within an invalid 16?m % 16?m "eo"raphical bin# The definitionof a valid bin is "iven in (ection #2#3#

7#1# Data Throu"hput

There are known deficiencies in the current Nokia downlink packet data implementation# This' combinedwith a more reactive 3G to 2G handover process in Nokia' lead to a si"nificant reduction in throu"hputwhen compared to equivalent results on the Nortel part of the O2 network#

$t is e%pected that all swap events would fail an acceptance criteria based on 3G downlink datathrou"hput equivalence# 0s a result the followin" acceptance criteria shall be applied#

Bor the 3< kbit=s downlink data rate' the avera"e throu"hput post swap shall be "reater than theavera"e pre9swap throu"hput multiplied by a modification factor of ?#;# This modification factor shall bereviewed periodically in the li"ht of the results of early swap events# The modification factor shall beviewed as a concession to allow for known deficiencies# The principle of equivalence would require themodification factor to be set at a value of one#

Bor the 12


10/13

DOCUMENTTYPE 1# (13)


n I 6 40

6 I n I 1? 25 1? I n I 2? 12 2? I n I 3? 93? I n I ? 8

? I n I 6? 7 6? I n I 1?? 5

n J 1?? 2

0.( Acce tance riteria

0 swap event will have met the / $ acceptance criteria if each of the followin" conditions is met&

The 3G call completion success rate *voice' data and video- is "reater than or equal to the pre9swap value *less a statistical tolerance factor of 1#6@ and allowin" for e%clusions-#

Due to known deficiencies in the Nokia packet scheduler implementation' the data throu"hputvalues post swap shall be compared to absolute values or be compared to the pre9swap valueswith a modification factor applied *as described in (ection 7#1# -# The post9swap throu"hputvalues shall be "reater than the specified absolute values or modified pre9swap values#

The 3G92G handover success rate *voice and data- is "reater than or equal to the pre9swapvalue *less a statistical tolerance factor described in (ection 7#1# -#

7. I!8B'IL"I!, *EAS'&E*E!TS

0s part of the swap process' the in9buildin" performance of the system pre and post9swap should becompared at a number of locations to be a"reed in advance of the swap event# One day shall be allowedfor in9buildin" measurement in the pre and post swap measurement weeks in which at least twospecified buildin" locations shall be measured#

7.% In86uilding o9erage Tests

Bor each in9buildin" location an area of measurement shall be a"reed between Nokia and O2# Cherethe area of measurement e%ceeds 1?m % 1?m' for e%ample in a shoppin" centre' the total area shall besplit into a number of sub9 ones each of appro%imately 1?m % 1?m# These areas shall be marked on aplan to be submitted with the measurements#

Cithin each sub9 one' )(, measurements shall be taken# The principles for these measurements arethat they should be collected in as uniform a manner as possible across the whole of the sub9 one area#

0t least 500 individual measurements shall be collected in each sub9 one#

The data shall be processed to provide the d4 avera"e and standard deviation of the dominant server for the sub9 one# The dominant server is indicated by the scramblin" code that provides the "reatestnumber of best server measurements across the sub9 one#

$f the dominant server in the post9swap measurements has under"one an OT() to (T() up"rade aspart of the swap event then the resultant avera"e )(, level shall have the a"reed splitter insertion lossvalue subtracted before comparison with the pre9swap si"nal level takes place#


11/13

DOCUMENTTYPE 11 (13)


$f the avera"e dominant server )(, value post9swap *corrected to allow for OT() to (T() conversion

where appropriate- is less than the pre9swap value by a value "reater than the post swap )(,standard deviation for a sub9 one' Nokia shall carry out remedial action to resolve the situation#

7.( In86uilding Ser9ice Tests

Cithin each of the sub9 ones of an in9buildin" area' a number of test service calls shall be carried out#The sample calls shall comprise&

1?% voice calls1?% videotelephony calls1?% 7 .5 = 3< D5 data calls

The success criteria for each call type shall be the same as that applied to the drive testin" described inthe previous sections# The same e%clusions allowed in the drive test cases shall also apply#

$f the number of call failures post9swap is "reater than one more than the number of failure pre9swap'Nokia shall identify the cause of the additional failures and carry out remedial action to resolve thesituation#


12/13



5. ALL E$E!T "EFI!ITIO!S

5.% %(.(3 9oice callBor a 12.2k voice call to be counted as successful it must satisfy all of the criteria in The Tablebelow#

Criteri a

,all sets up successfully *)), connection request to alertin"-

,all does not drop durin" the test call holding time period *disconnect other than normal disconnect-

,all terminates successfully,all hands9over *if necessary' both 3G93G and 3G92G-#

5.( +,4(, hand8o9er for a %(.(3 9oice callBor a 3G=2G hand9over for a 12.2k voice call to be counted as successful it must satisfy all thecriteria in the Table below

Criteri a

Test .E receives a )), :0NDO E) B)O8.T)0N ,O880ND messa"e from the 3G .T)0NError& )eference source not found

.E sends a )) :0NDO E) ,O8 5ETE messa"eto the 2G 4((#

5.+ 0-3 S data callBor a 64k CS data call to be counted as successful it must satisfy all of the criteria in the Tablebelow&

Criteri a

,all sets up successfully *)), connection request toalertin"-,all does not drop durin" test call holding time period#,all terminates successfully#

,all hands9over *if necessary' 3G93G-#

5.- PS data sessionBor a PS data session to be counted as successful it must satisfy all of the criteria in the Tablebelow&


13/13



Criteri

aD conte%t activates successfullyD conte%t or radio bearer does not drop durin" the

test session holding time period#

Data session hands9over *if necessary' 3G93G aand3G92G-

D conte%t deactivates successfully#

5./ +,4(, hand8o9er for a PS "ata SessionBor a 3G=2G hand9over for a PS Data Session to be counted as successful it must satisfy all the

criteria in the Table belowCriteri

a

Test .E receives a ,E55 ,:0NGE O)DE) B)O8.T)0N ,O880ND messa"e from the 3G .T)0N.E receives a )O.T$NG 0)E0 . D0TE ,ompletemessa"e to the 2G 4((#

Annex22 Enigma DE2.doc

Documents

Transcript of Annex22 Enigma DE2.doc