Techniques for Restoring Telecom after Disasters & Outages
Transcript of Techniques for Restoring Telecom after Disasters & Outages
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 1/45
M A R K R E Y N O L D S , U N I V E R S I T Y O F N E W M E X I C O
T I M R U F F , T E L E C O M R E C O V E R Y
Techniques for Restoring Telecomafter Disasters & Outages
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 2/45
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 3/45
3
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 4/45
The Problem4
Incoming local DIDs
Incoming toll-free numbers Outbound dial tone
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 5/45
1970 1975 1980 1985 1990 1995 2000 2004
Droughts
Floods
Earthquakes
Epidemics
Slides
Volcanic eruptions
Wild fires
Wind Storms
Insect infestation
Source: EM-DAT : The OFDA/CRED International Disaster Database
http://www.em-dat.net, UCL - Brussels, Belgium
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 6/45
Uh-
6
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 7/45
Common Causes of Phone Outages
Equipment failure Power outage
New Installation
Moving
Water leak
Animals
Cable cut Wiring short
Technician mistake
Winter storm
Accidents
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 8/45
October 17, 1989 4:00 PM
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 9/45
October 17, 1989 6:00 PM
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 10/45
Silicon Valley, April 9, 2009
1:30 AM Vandals sever major fiber backbone withhacksaws
AT&T, Verizon, Sprint down
Landlines, cellular, internet down No 9-1-1 service, no credit cards
Lasted 14 hours
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 11/45
Pandemic Quarantine
United States H1N1 stats as of 10/17/09:
98,000 hospitalizations
3,900 deaths
What if the next strain is more severe? Are you prepared to have some/most/all employees work from home?
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 12/45
The 6 Vulnerabilities of Telecommunications
LocalCarrier
Long Dist.Carrier
12
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 13/45
Vulnerability 1: Equipment Failure
LocalCarrier
Long Dist.Carrier
13
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 14/45
Vulnerability 2: Power
LocalCarrier
Long Dist.Carrier
14
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 15/45
Vulnerability 3: Last Mile (cable cut)
LocalCarrier
Long Dist.Carrier
15
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 16/45
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 17/45
Vulnerability 5: Long Distance Carrier
Long Dist.Carrier
LocalCarrier
17
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 18/45
Vulnerability 6: Evacuation / Quarantine
LocalCarrier
Long Dist.Carrier
18
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 19/45
The 6 Vulnerabilities of Telecommunications19
1. Equipment failure
2. Power outage
3. Last mile (cable cut)
4. Local carrier failure/congestion5. Long distance carrier failure/congestion
6. Evacuation/pandemic quarantine
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 20/45
20
QwestBCR
Qwest
(505) 272-XXXX Hospital
Dept.Cell
Phones
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 21/45
21
Good:
Can divert calls after any failures except carrierfailures and congestion
Not so good: Calls go direct to cell phones, no transfer, no
greeting, no centralized voicemail, no attendant, etc.
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 22/45
Qwest
UNM New Recovery Plan22
Hospital(505) 272-XXXX
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 23/45
Qwest
UNM New Recovery Plan23
Hospital(505) 272-XXXX
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 24/45
Recovery server
Qwest
UNM New Recovery Plan24
Hospital(505) 272-XXXX
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 25/45
Recovery Server
Qwest
Internet
UNM New Recovery Plan25
Hospital(505) 272-XXXX
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 26/45
Recovery Server
Qwest
Internet
UNM New Recovery Plan26
Hospital(505) 272-XXXX
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 27/45
Recovery Server
Qwest
Internet
UNM New Recovery Plan27
Hospital(505) 272-XXXX
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 28/45
UNM New Recovery Plan28
Recovery Server
Qwest
Internet
Hospital(505) 272-XXXX
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 29/45
UNM New Recovery Plan29
Recovery Server
Qwest
BCR
Qwest
Hospital(505) 272-XXXX
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 30/45
UNM New Recovery Plan30
Good:
Incoming calls get back into Hospital PBX
If PBX fails, calls get to same employees on cell, sat,
home, other locations, recordings, etc. Real-time control, unlimited flexibility
Backup dial tone using any broadband connection
Not so good:
Local numbers down if Qwest is down (toll-freenumbers recoverable regardless of Qwest failure)
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 31/45
The 13 Most Critical Mistakes
1. Underestimating the pain of outages
2.
3. Confusing outbound with inbound
4. Confusing disasters with outages5. Neglecting vulnerabilities
6. Inadequate mass communication
7. Inadequate control
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 32/45
The 13 Most Critical Mistakes
8. No satellite phone
9. Insufficient capacity
10. Reliance on hot site
11. Putting backup equipment on-premise12. Lack of testing
13. Spending too much
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 33/45
10 Low-Cost Improvements
1. Twitter/Facebook
2. Skype
3. MagicJacks
4. Create voluntary reimbursement agreements withemployees
5. Gather alternate phone #s, email
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 34/45
10 Low-Cost Improvements
6. Create Emergency section on website
7. Publish a toll-free hotline
8. Retain some POTS (copper) lines, wired outside the
phone system9.
carrier
10. Obtain GETS/WPS cards
(www.ncs.gov/services.html)
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 35/45
FAQs35
What if we already have a backup PBX?
What if we already have a SONET ring, SHARPS,SHINS, etc.?
What if we already have a mass notification system? What if we already have cell phones, radios, satellite
phones, etc.?
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 36/45
FAQs36
calls to you?
1. Neustar/Port DR 2. We need your help in pressuring carriers to adopt
Port DR!
3. Advantages of toll-free over local
4. Still need numbers loaded in a backup carrier
5. Still need a place to send the calls
6. Backup satellite T1 (for delivery & outbound)
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 37/45
Backup Satellite T1
Off-siteControl system
Public Phone
Network
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 38/45
Off-siteControl system
Public Phone
Network
Backup Satellite T1
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 39/45
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 40/45
Off-siteControl system
Public PhoneNetwork
Backup Satellite T1
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 41/45
Off-siteControl system
Public PhoneNetwork
Backup Satellite T1
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 42/45
Off-siteControl system
Public PhoneNetwork
Backup Satellite T1
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 43/45
FAQs43
calls to you?
How much does this cost?
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 44/45
What Next?
Determine what next steps to take.
Identify people involved in making and approving aplan.
Establish a project to implement the plan. Communicate and test the plan.
44
8/6/2019 Techniques for Restoring Telecom after Disasters & Outages
http://slidepdf.com/reader/full/techniques-for-restoring-telecom-after-disasters-outages 45/45
What are the questions?
What are the important numbers at the institutionthat should function during a disaster?
How will calls to the numbers identified above behandled?
Who would authorize and activate the emergency routing of numbers?
Who would administer and maintain the
information?
45