Creating a complete recovery solution Adam Backman [email protected] Partner, White Star Software.
-
Upload
easter-jennings -
Category
Documents
-
view
215 -
download
0
Transcript of Creating a complete recovery solution Adam Backman [email protected] Partner, White Star Software.
Creating a complete recovery solution
Adam Backman [email protected], White Star Software
Why backup?
No reason to backup – Our stuff never fails Just takes up resources We already have redundancy I hate changing the tapes I’m tired I’m hungry I don’t feel good Are we there yet?
Cost of Lost Data
Loss of data is expensive− Cost to rekey− Cost of lost business
Can you recover the data− Not everyone has paper trail
Focus on recovery
Building a Good Recovery Strategy
Know your business− Components of business
how people do business with you− Components of systems
Tools (applications and physical) Know your risks (fire, flood, hurricane, …) Be inclusive
− Technical people (network, phones, facilities, …)− Business people (people who own the application functionality)
Build an execution plan with contingencies
Why be inclusive?
Business people know what they need− What they use in the application− What they need to do their job (phone, fax, internet, …)
Stops the finger pointing when a problem occurs Business people generally control the budget Need decision makers
− Authorize spending − Data loss vs. recovery time decisions
Focus on Recovery
Loss of dataRogue program, hacked, poor testing
Loss of diskDisk failure
Loss of machineMachine failure
Loss of machine roomFire, flood,…
Loss of buildingNatural disaster
Creating a plan
Goals (Event-based goals)− Data corruption (Data)− If we lose a disk (DB)− If we have a fire (Machine)− If we have a natural disaster (Facility)
Hardware Software Other stuff
Main Goal
The main goal for any recovery plan is to minimize or eliminate the impact to the customer
Creating a plan - Goals
Acceptable downtime (Generally cost based)Everyone wants zero but it is generally cost prohibitive
Planned outages− Hardware install and maintenance − Software upgrade− O/S upgrade or patch
Notifications (Both before and during outage)− Who− When− What do they do?
Creating a Plan – Other Considerations
What makes your business run?− Phones− Faxes− Business to Business (EDI, XML Feed, …)
Can people work from home? Do you have/need another location? Contact lists in case of major catastrophe
− Kept up-to-date− Kept online and printed in an accessible location
Cover All Sides
Everyone should be running after image journaling Need removable backup periodically (daily)
− Wide scale events (fire, flood, …)− To recover from after image journals
Replication is the new normal− OE Replication− Log-based replication− Hardware replication
Portions of a OpenEdge Database
Database table of contents (.db) The data files (.d*) Before image journals (.b*) After image journals (.a) Modified buffers in memory
How probkup -online works
1. Lock the DB2. Do a pseudo-checkpoint3. Switch AI files (if necessary)4. Backup the BI file5. Unlock the DB6. Backup the database
Other Items to Consider
Application files Data outside the database (GIS, photos, …) User files (reports, output, …) External systems (EDI, Warehouse, …)
Backup’s Impact to Production
Backing up production− Pause during backup of before image journal− Uses I/O capacity of production
Split mirror backup− Use of quiet point keeps pause to a minimum− Pause is non-zero
After Image file backup− Still needs a backup to begin the process− Little impact for backup process− Potentially long recovery time
After Image Management
Use OpenEdge after image management− Automated− Supported− Nice features
Use script-based after image management− Custom− Requires support and testing
Replication options
OpenEdge Replication− Supported by Progress− Real time replication− Allows for reporting against target database− Additional license cost
Log-based replication− Free− Requires self-support− Not real time (greater potential data loss)− No reporting option
Hardware replication
OpenEdge Replication – Single Target
Production (source) Reporting (target)
Shared Memory
Shared Memory
Replication Server
Replication Agent
Source DB
Target DB
OpenEdge Replication
Production (source)
Target 1
Shared Memory
Shared Memor
y
Replication Server
Rep Agent
Source DB
Target DB
Target 2
Shared Memor
y
Rep Agent
Target DB
Multiple Targets
Log-Based Replication
Log-based replication has been used for years Log-based replication provides a vehicle for replication without the licensing costs of OE Replication
Not real-time Code for this type of replication must be
maintained by the user and there is no official support from the vendor
Hardware-Based Replication
Hardware-based replication is a function of the hardware vendors and thus supported directly by them
This method is NOT supported by Progress ALL write operations must be guarantied across
the source and target disk systems
Archiving – Why is it important
Needs to be accessible Needs to be secure Needs to be in a different location Need to have policies
− Physical access− Number of saved archives − Security
Archiving
Who does your archiving (Iron Mountain, third-party, someone’s house, …)
What do you keep− Two weeks of dailies− 5 weeks of weeklies− 1 year of monthlies
How to label you backups− Who did the backup− Command to restore− Date and Time
Pre-archiving
After image files − Move to remote system
Database backups− Move to remote system before moving to removable
media Support files User data
Archiving (continued)
Data Archiving – What data archiving?− Archive/Delete?− Archive/Save historical− Archive/Save aggregates
After Image file archiving− At least 2 backups worth− I recommend a week or more if possible
How about if I am a SAAS user
Who is your provider Verify their recovery plan Run dry run of at least one recovery scenario Have specific service level agreements
− Time to recover− Maximum loss of data− Penalties for missing times (need to cover your costs)
How about if I am a SAAS provider
Build regular recovery plan Unique concerns
− Security− Compliance (HIPAA, SOX, …)
Build achievable SLAs for your users that at a minimum include:− Data loss− Uptime− Maximum recovery time− Maintenance time
Implementing Your Plan
First implementation should be a totally manual process to insure the steps work and allow for documentation
Document the process as you go− Who are you logged in as?− Exactly what you typed− Where you were (console, remote, …)− Can things be done in parallel or sequentially− Where are the logs − What to look for in the logs (good and bad)
Documentation
All recovery documentation should be VERY specific as the person doing the recovery will not be familiar with you or your environment
Create documents for normal maintenance− Backups− Database growth− Modification of OS, Application, printers, …
Create scenario based recovery plans− Lose a disk (or disk pair)− Fire − Flood
Testing Your Plan
Who does the test?− Not the person who wrote it− The backup person for the implementation− Someone who is “always” there regardless of technical
ability How often to test?
− Material data change− Any change in database configuration
New directories New areas
− Do you have a second site or redundant hardware?− Do you have enough disk capacity (space and throughput)
How to test your plan
Fail over to your backup system− You may lose the use of the backup system but that is a good
trade in the vast majority of cases− Maybe not live but at least a restore and test
Fail back to your primary system− After a live failover
Contingency planning for:− Personnel− Physical plant − Equipment − Lead time for resources (hours or days)
Summary: Recovery Planning
You still need to backup Backup your backup not production if possible Be inclusive when building your team Create and maintain a comprehensive plan
Include everything needed to use the application: Hardware Applications Data
Create and maintain physical and online contact lists and store it with your archived backups
Conclusion: Recover planning
Always backup what you have now, however little, before starting to recover
Test your plan periodically (At least annually)
Still have questions?
Please feel free to contact me directly.
Adam BackmanWhite Star Software(603)[email protected]
THANK YOU
Thank you for your time