To Hell and Back in Five Days

21
To Hell and Back in Five Days: Lessons in disaster recovery and emergency response. Jon DiPietro Principal, Bridge-Soft [email protected]

description

Between February 7 and March 14, 2009, more than 400 bush fires across the state of Victoria, Australia, scorched over a million acres of land, killing 173 and injuring 414. Engineers at Goulburn Valley Water (GVW), provider of urban water and wastewater services to 54 towns and cities on the outskirts of Melbourne, watched as their telemetry system from the Kilmore Dissolved Air Filtration plant reported an ambient control room temperature of 142 °F before going silent on the afternoon of Saturday, February 7. A site visit on the following day revealed that while the treatment plant survived the fire, its control room was completely incinerated, destroying the electrical switchgear, plant HMI, laboratory, instrumentation and chemical dosing systems. With only five days worth of water stored, an emergency response plan to rebuild the control room and recommission the plant went into action.

Transcript of To Hell and Back in Five Days

Page 1: To Hell and Back in Five Days

To Hell and Back in Five Days:

Lessons in disaster recoveryand emergency response.

Jon DiPietroPrincipal, Bridge-Soft

[email protected]

Page 2: To Hell and Back in Five Days

“Burning Trees” courtesy of Sascha Grant via Creative Commons on Flickr

400 fires

1,000,000+ acres

173 dead

414 injured

Page 3: To Hell and Back in Five Days
Page 4: To Hell and Back in Five Days
Page 5: To Hell and Back in Five Days
Page 6: To Hell and Back in Five Days
Page 7: To Hell and Back in Five Days

Preparation

“If you want to make an apple pie from scratch,

you must first create the universe.”- Carl Sagan

Page 8: To Hell and Back in Five Days

Documentation

Page 9: To Hell and Back in Five Days

Human Resources

Page 10: To Hell and Back in Five Days

Relationships

Page 11: To Hell and Back in Five Days

Planning

“Strategy without tactics is the slowest route to victory.

Tactics without strategy is the noise before defeat.”- Sun Tzu

Page 12: To Hell and Back in Five Days

Assessment

Page 13: To Hell and Back in Five Days

Strategy

Page 14: To Hell and Back in Five Days

Tactics

Page 15: To Hell and Back in Five Days

Execution

“No battle plan ever survives contact with the enemy.”

- Field Marshall Helmuth Carl

Page 16: To Hell and Back in Five Days

Assembly

Page 17: To Hell and Back in Five Days

Installation

Page 18: To Hell and Back in Five Days

Improvisation

Page 19: To Hell and Back in Five Days

Commissioning

Page 20: To Hell and Back in Five Days

Conclusion

Emergency preparednessCross-trained & motivated team

Strong vendor relationshipsSolid planning

LUCK

Page 21: To Hell and Back in Five Days

Thank You

Jon DiPietroPrincipal, Bridge-Soft

[email protected]