crm

customer relationship management

1

The information contained in this presentation is confidential information as per your terms and conditions with BT. Please do not forward, republish or permit unauthorised access.

The content is accurate at the time of writing and is subject to change.

BT Wholesale Planned Engineering Works (PEW) 90 day overview – May to July 2011.

BTW 2011 Network Changes – 3 month rolling plan

• Whilst BTW has continually worked to enhance its network, we are currently undergoing an exceptional level of change driven by four main drivers:-

– Stability – to improve the reliability of the network, BTW is targeting certain network elements with hardware and software improvements.

– Capacity – the need for additional capacity is driven by an increase in the number of broadband users on the 21C network (through migration from 20C and growth) and also by the ever-increasing bandwidth demands of those users.

– Resilience –to improve the resilience of the network , BTW is actively removing single points of failure from the network

– New Functionality – to meet the needs of the market BTW needs to enhance its network to deliver the services its customers require. There is a current focus on efficient content delivery that will be offered through the Wholesale Content Connect product.

• This presentation outlines the key upgrade programmes, their drivers, customer impact and the number of upgrades per programme per week.

• BTW is keen to deliver all the required improvements as quickly as possible whilst actively managing the risk to customers. For this reason a range of run-rates against each programme has been outlined. The actual run-rates achieved will be determined by this ongoing live analysis of risk.

• Customer outage times for the PEWs start between 00:01 and 02:00 with the aim of completing customer impacting work by 04:00. This allows 2 hours for roll-back should the PEW be unsuccessful.

• Business as usual capacity upgrades and upgrades to resilient structures have not been listed here. All the activities outlined on slides 2, 3 and 4 involve an end user outage.

2

20C Upgrades

Key:-Outage is defined as full loss of service

Planned month for upgrades

Contingency month for upgrades – 20% likelihood

Contingency month for upgrades – 50% likelihood

See diagram on slide 5 for network location

1

3

#

20C BRAS code & card – this upgrade is needed to get the Juniper ERX BRAS ready for HQOS which is required for WCC (content). Rollout (Pt1) of Junos code only is proceeding as this does fix some minor stability issues on the BRAS. Pt 2 will add new cards and HQOS and is currently being tested with a view to intercepting the Pt1 rollout plan - COMPLETE

21C Upgrades

3

5

6

7

4

21C BRAS (Ericsson) code & card upgrades – 2nd Upgrade will see new code, improved processor cards (with greater memory) and Premium Forwarding (HQOS), 2nd Upgrade expected to complete during June. 3rd Upgrade further non-EU impacting to improve capacity capability.

Provider Edge switch separation – these upgrades increase core capacity – directly benefits WBMC Shared and Ethernet - Completed

SVLAN moves - these moves offload SVLAN’s (bundles of Broadband traffic) from one BRAS to another. This is carried out to alleviate capacity issues upgrades (complete).

FER MCLAG & Virtual Switch – two separate upgrades that together improve capacity and resilience of WBMC shared

8

7750 code upgrade – these Multi-service boxes are located at different points of the network. This upgrade indirectly enhances the capacity of these boxes (allowing new hardware to be used) and is a pre-requisite for TV connect

21C Upgrades

9

10

* Not including BAU capacity mgmt moves 5

11

12

EFM Code Upgrades – Required to upgrade code to provide greater service assurance functionality

MPLS Core Router Upgrades required as existing software is soon to become “End of Life”.

Copper MSAN upgrades – Hardware update on single card

Access MCLAG – removes single point of failure between MSAN (mostly Fibre MSAN which could aggregate several Copper MSANs) and metro node to improve overall availability

7750 Minor code upgrade – Required to improve software stability

13

Re-planned PEWs

• The level of network change that BTW is driving through inevitably leads to a need to re-plan a number of PEWs.

• The reasons for these re-plans include:- unforeseen technical issues specific to (or indirectly impacting) that programme, re-prioritisation of work, insufficient time to complete all work on the night and operational decisions.

• BTW have made some improvements to the management of re-planned PEWs and where possible will seek to avoid using Emergency PEWs for this re-planned work. In some cases re-planned work will need to be carried out with less notice.

6

Re-planned PEWs

Specific Programmes with high proportion of replans:-

7

Programme Reasons

20C BRAS Code & Card upgrades

1. Unforeseen technical issue impacting operational mgmt of some BRAS2. Delayed fix meant re-planned PEWs had to be moved again

FER MCLAG upgrades 1. Prioritisation (before Xmas) decision to re-allocate resource to BRAS MCLAG programme

2. Process issues have led to abort decisions on the night before service impacted. This is under review & some shorter-notice PEWs (~ 1week) may be required to avoid capacity issues

7750 Code upgrades 1. Productivity issues (now resolved) meant high proportion not upgraded on the night

2. Additional checks were requested from the vendor to provide better control on the night for the most heavily loaded 7750s. This meant that some of the PEWs were replanned. These are now being carried out.

Broadband Services on 21CN

12

3 4

5

6

7

88

8

8109

8

11

11

13

13

13

21CN Core

EEA/EES

EEA/EES

EEA/EES

F

EEA/EES

KeyETHC Ethernet Circuit

ETHA Ethernet Access (Fibre/Copper)

7750 Edge Router

EFM Router

Customer / CP Site

Ethernet Service on 21CN

EEA/EES

F

EFM

EFM

8

12

8

8

8

13

13

13

13

Descriptions

10

Management of risk

• In addition to the standard risk mitigation that would be in place for any network upgrade project, BT is applying additional rigour as outlined below:-– Co-ordination of PEW activities to avoid co-incident clashes with other

network and systems changes.– Geographical rollout to focus Field Engineering resource and spares for 72

hour post-PEW support following higher-risk PEWs– Daily calls to sanction all PEW activity for following night based on latest

available information– Separate Go / No go calls for all key upgrades – On-the-night efficiencies to reduce risk of failures and overruns.– Combinations of co-incident BRAS upgrades scheduled to minimise impact on

RADIUS servers– Additional resource being mentored to widen the skill sets and spread

available hands– Review of previous night’s PEWs with full root-cause analysis of issues and

over-runs and mitigations put in place to minimise re-occurrences

11

PEW IMPROVEMENTS

You Said Were Working onDelays in the affected features being seen within BBCR.

Carrying out e2e reviews of the PEW process with BT Design, Operate & Wholesale to understand this in more detail.

Different PEW communication processes (20C v 21C) Carrying out e2e reviews of the PEW process with BT Design, Operate & Wholesale to understand this in more detail and align if possible.

Not always notified if a PEW is cancelled or re-planned

Manual process put in place Jan 11, new automated process being introduced Release W 15th May.

12

PEW IMPROVEMENTSYou Said We DidBTW don’t notify us when a PEW has been re-scheduled. From January 2011 we now notify you when a PEW has been re-scheduled.

PEW notifications do not always include the location / exchange details. We now include the location / exchange details within the PEW communications.

Lack of detail & understanding as to why the large increase in BTW PEWs. Set up weekly calls with Industry reviewing a 90 day rolling PEW programme explaining the key upgrade programmes, their drivers & impact.

We now link the PEW communications to the PEW programme of works for CP’s to refer to for further information.

Duplicate PEWs under one PEW reference can last a number of days meaning it’s difficult for CP’s to effectively manage these PEWs with their own customers.

We now issue separate PEW’s for these works so CP’s know the impact and when these works will take place.

We have seen duplicated PEW communications (Network & System) for the same works at the same time and location causing confusion / duplicate handling.

Investigated & resolved the issue by removing a flag that was being generated against both these types of PEW notifications. (Network & System)

Can we notify you if a PEW programme is as a result of a recent MSO to prevent future incidents in the network if common cause found.

From February 2011 we now notify CP’s if the PEW is related to a MSO.

The PEW gadget on My Page is really slow to download the relevant PEW information so not a tool we can utilise.

We have resolved the speed issues meaning the PEW gadget now downloads much quicker for CP’s to utilise & view PEWs & the affected features.

The PEW gadget on My Page doesn’t show all PEW notifications. Resolved some data issues to now show all PEWs however there is a delay in the data uploading to the PEW gadget (approximately a few hours) so not a LIVE gadget.

Receiving PEW notifications for infrastructure not impacted and some cases being received late

New internal process now in place to remove inappropriate notifications and measures taken to ensure minimum notification period is adhered to.

13