VMware presentation - VMware Communities
Transcript of VMware presentation - VMware Communities
© 2009 VMware Inc. All rights reserved
Update on VMware vSphere Backup APIs &
Backup Approaches
Pete Marfatia - Systems Engineer, VMware.
VMUG 2010 June 03
Agenda
2
Introduction
• RPO and RTO – a quick explanation
• Where various VMware technologies fit
• Remind me, why are we backing up again?
Backup Methods in a VMware Environment
• Quick recap of VMware Consolidated Backup (VCB) framework
• vStorage API‘s for Data Protection (vSphere 4 )
• vADP & VCB side by side comparison
• vADP ready partner solutions
• Types of Data Deduplication
VMware Data Recovery Appliance Overview
VMware Data Recovery Install, Config and FAQ’s (for reference later)
Introduction
3
• RPO and RTO – a quick explanation
• Where various VMware technologies fit
• Remind me, why are we backing up again?
Disaster Recovery Concepts
SecsMinsHrsDaysWks Secs Mins Hrs Days Wks
Recovery Point Recovery Time
Recovery Point Objective (RPO)
Point in time to which data must be recovered (i.e. how old can the data be?)
Recovery Time Objective (RTO)
Time required to complete recovery of the application and it’s data
Disaster Strikes
Tape
Backup
SecsMinsHrsDaysWks Secs Mins Hrs Days Wks
Recovery Point Recovery Time
Recovery Point Objective (RPO)
Synchronous
Replication
Periodic
Replication
Asynchronous
Replication
Disaster Strikes
Tape Restore
SecsMinsHrsDaysWks Secs Mins Hrs Days Wks
Recovery Point Recovery Time
Recovery Time Objective (RTO)
Recovery Time includes:
Fault detection
Recovering data
Bringing apps back online
Automated
Migration Manual
Migration
Disaster Strikes
vCenter Site
Recovery
Manager
Introduction
7
• RPO and RTO – a quick explanation
• Where various VMware technologies fit
• Remind me, why are we backing up again?
Technology to help maintain availability and reduce downtime
Business Continuity
Maintain High Availability and Reduce Downtime
Planned Outage
Maintenance Tasks
• Hardware
• MM, vMotion & DRS
• Storage vMotion
• Software
• Update Manager
Recovery Tasks
• vCenter Server
• vCenter Heartbeat
• Backup Tape / Image
• VCB vADP
• vCenter DR
Site Level (FB/DC relocation)
• Site Recovery Manager
Unplanned Outage
Site Level (DR)
• Backup Tape / Image
• VCB vADP
• vCenter DR
• Array Replication
• Site Recovery Mgr
Equipment Level
• vCenter Server
• vCenter Heartbeat
• Host Level
• HA, FT
• VM, O/S hang
• HA
• HA,
RPO/RTO = 0 with VMware Technology
What we are going to discuss today
Backup and Restore in a virtual environment
9
Using in-guest agents—why might you be using this?
Existing backup solution does not support VCB / vADP (Really?)
Carry over from P2V—transitioning to solution leveraging VCB / vADP
Support for specific NOS eg: File level back up of Linux (VCB limitation)
Specific application level integration required eg: Domino, SAP
For fine grain restore eg: Individual Exchange mailbox restores
Source based de-duplication ie minimize data transferred over WAN
Potential to restore to other systems (physical) for DR
As with
physical, watch
out for I/O
bottlenecks
Backup and Restore in a virtual environment
10
Why might you increase the percentage of VM backup to a solution
leveraging VCB / vADP vs in guest?
Agent less => reduced operational touch / generic approach to VM backup—80:20
rule. Use in-guest only where specifically required
VM snapshot technology to shrink effective backup window
Offload backup from the LAN to optimized SAN => faster and more efficient
File system-level consistent backup via Sync Driver or MS VSS
Application-level consistent backup via MS VSS
Ability to easily do image level backup for reduced RTO ie DR or change control
Introduction
11
• RPO and RTO – a quick explanation
• Where various VMware technologies fit
• Remind me, why are we backing up again?
Remember why we do backups in the first place…
12
Restore requirements dictate backup methods
RTO is impacted by the speed and ease with which you can recover the
system successfully AND consistently
Will the application be ready to run once restored or do you need to check
consistency/replay transaction logs first? How long?
Backup to/restore from disk vs tape maybe required
Restore over SAN vs over LAN vs over WAN
Remote offices may use a Virtual Storage Appliance locally, which can dedupe
then replicate back to H/O more efficiently across the WAN to meet
requirements of both offsite backup AND RTO
And so on…
What about the ESX host config?
COS based agent install—supported but less than ideal
COS goes in 2011—ESXi is like an appliance, just reload, restore config & go
Other VMware features such as Host Profiles for config remediation &
consistency
Backup Methods in a VMware
Environment
13
• Quick recap of VMware Consolidated Backup (VCB) framework
• vStorage API‘s for Data Protection (vSphere 4 )
• vADP & VCB side by side comparison
• vADP ready partner solutions
• Types of Data Deduplication
14
VCB Framework Topography – The old way of backing up VMs
First available in 2007 with VI3
Required a physical Proxy server with FC or iSCSI HBA‘s (virtual proxy in 1.5)
Support for backup on FC, iSCSI, NFS, LAN mode
Windows file and image level backup
Image level for Linux
VMware Snapshot quiescing options
Nothing (not a good idea) Pre- & Post-quiescing scripts
File system-level consistent via Sync driver VSS* for file system- & application-level
Early partner integration was varied but has improved significantly in
experience, features and the number of participating partners:
IBM Tivoli Storage Manager Symantec NetBackup & Backup Exec
CA BrightStor ARCserve HP Data Protector VizionCore vRanger Pro
EMC NetWorker & Avamar CommVault Galaxy
Some security and retry mechanism improvements
* ESX(i) 3.5 U2 or later required
VCB Components – Old way of backup software integration
15
SAN-based
VMFS datastore
mydata
VCB (since v1.1)
can now reside on
the same host as
Virtual Center
The future for VCB
18
It’s going away with vSphere 4.next (2010 timeframe) per the email
many of you received earlier this year…
This allows us to focus resources on further enhancing vADP--OK?
Still can’t see a way out?
Speak with your VMware Partner or VMware Rep for a Professional Services
engagement to help square this away
Backup Methods in a VMware
Environment
19
• Quick recap of VMware Consolidated Backup (VCB) framework
• vStorage API‘s for Data Protection (vSphere 4 )
• vADP & VCB side by side comparison
• vADP ready partner solutions
• Types of Data Deduplication
vStorage APIs for Data Protection
Next evolution of VMware Data Protection
Part of vSphere 4—no extra install
Enables native integration between vSphere 4 and backup application
Backup server is now proxy equivalent
Supports Windows and Linux file-level backup
Supports MS VSS
Supports FC, iSCSI, NAS or local storage
Improved Functionality
Supports full, differential, and incremental image backup and restore of virtual
machines
Efficient backups
Easy restores
Goal is to make virtual backup better than physical
VADP
VADP
vADP works with Change Block Tracking
21
VMware has introduced ―Changed Block Tracking‖ with vSphere 4.0 where
the ESX(i) host stores the last changed time for each block in a VMDK
Applications (like VMware Data Recovery & Storage VMotion) can access this
information and find out know which blocks have changed and which have not
been touched
vADP APIs may now access this last changed time data and determine if the
block needs to be backed up or not—faster backup, less to transfer
Storage I/O is reduced with this algorithm as we only touch changed blocks for
backup
Included in the API to access storage on VMFS volumes
Changed block tracking is only available with VM‘s running with hardware
version 7
Change Block Tracking (CBT) on virtual machines http://kb.vmware.com/kb/1020128
Cannot enable FT on backed up virtual machine http://kb.vmware.com/kb/1015300
Reverting to a pre-existing snapshot can cause incremental backups based on
Changed Block Tracking to become inconsistent http://kb.vmware.com/kb/1021607
vADP & VCB side by side comparison
22
vADP VCB
Requires additional
download & install
No, built into the data
protection software
Yes
Full VM image backup Yes, single step copy – source
to target
Yes, with two step copy –
source to VCB proxy and VCB
proxy to the target
Incremental VM image
backup
Yes – using change block
tracking
No
File level backup Yes, both Windows and Linux Yes, Windows only
Full VM image restore Yes Yes, by using VMware
Converter
Incremental VM image
restore
Yes No
File level restore Yes, using restore agents Yes, using restore agents
CLI for image backup No Yes
CLI for file backup Yes Yes
Introduction to vStorage APIs for Data Protectionhttp://blogs.vmware.com/storage/2010/02/introduction-to-vstorage-apis-for-data-protection---vstorage-
apis-for-data-protection-were-introduced-in-vsphere-40-to-facil.html
vADP ready Partner Solutions
23
Tivoli TSM v 6.2
EMC Avamar 5.0
CA ArcServe 12.5
Commvault Simpana 8 with
post SP4 update
VEEAM 4.0
Vizioncore vRanger Pro 4.2
vStorage APIs for Data Protection (VADP) FAQ http://kb.vmware.com/kb/1021175
Data Recovery 1.x
Symantec Backup Exec 2010
& Net Backup 7.0
Backup Methods in a VMware
Environment
24
• Quick recap of VMware Consolidated Backup (VCB) framework
• vStorage API‘s for Data Protection (vSphere 4 )
• vADP & VCB side by side comparison
• vADP ready partner solutions
• Types of Data Deduplication
Data Deduplication - Concepts
Deduplication is where only one unique instance of the data is actually
retained on the storage media, such as disk or tape. Redundant data is
replaced with a pointer to the unique data copy. Similar concept to ESX(i)
transparent page sharing!
Deduplication helps reduce the amount of space consumed by full backups
Initially during the first full backup all the blocks on the VM are backed up
(deduplicated accordingly), and subsequent full backups only need to store the
blocks which have changed since the previous backup
25
Types Data Deduplication (source based & target based)
In this example, we are backing up multiple hosts which are all running the
same OS and similar applications
26
Source (agent) based
data deduplication
Reduces data to be
transferred to backup
server / target
Target (backup server) based
data deduplication
Data common across multiple
guests/hosts can be
deduplicated, further reducing
storage required
Overview of VMware Data Recovery Functionality
28
• Backups:
Disk based backup, not file level (fast backup and quick restore)
Any Guest OS can be backed up.
A VM can be backed up even if it is suspended or powered off.
Leverages VM snapshot technology and uses the SYNC driver or Microsoft VSS to
quiesce the Guest OS on a running VM. VMware Tools must be installed and running.
Leverages VM Change Block Tracking (very efficient storage of backup)
• Restores:
Can overwrite the current VM.
Restore the VM if it no longer exists.
Restore a ‗test‘ VM which is identical to the original VM, but does not have networking
enabled – this is called a Restore Rehearsal.
File level restore Technical Note http://www.vmware.com/pdf/vdrFileRestore_Usage.pdf
29
VirtualCenter
Agent-less, disk-based backup
and recovery of VMs on ESX(i)
4.0 hosts
VM or file level restore
Incremental backups plus data
de-dupe and compressed to save
disk space
Quick, simple and complete data
protection for your VMs
Ease of deployment
Centralized Management through
vCenter Server
Cost Effective Storage
Management
1. Backup
2. Restore
1. VM goes down
2. Select VM images/files to recover
3. Fast restore of VM…downtime minimized
1. Schedule backups via VC
2. Snapshots taken
3. Data de-duped and stored
vCenter 4.0
vCenter 4.0
X
De-duplicated
Storage
X
VMware Data Recovery
30
vCenter integration
- vSphere Client Plug-in
- Wizard driven backup and restore job creation
- Automatically import virtual machine inventory
- Awareness of HA/VMotion/DRS
VMware Infrastructure
- VSS support via VMware Tools
- Changed block tracking functionality allows backups to
be more efficient
Destination Storage
- Any VMFS storage: DAS, NFS, iSCSI or Fibre Channel
storage plus CIFS shares as target
- All backed up virtual machines are stored in a
deduplicated datastore
Backup and Recovery Appliance
- Linux appliance in OVF format - leverages vStorage API
for Data Protection to discover, manage backup and
restore
- First backup is full VM, then incremental forever
- VM or file level restore
VMware
vCenter
VMware Data Recovery Key Components
VDR Appliance Hot-Add Mechanism
31
VDR
Appliance
Virtual Infrastructure
VMDK Snapshot
Shared Storage
Changed
Blocks
Snapshot
VDR Appliance Hot-Add Mechanism
32
• Effectively the VDR appliance attaches to a linked-clone disk of the original VM
during a hot-add backup operation to access the read-only base disk for
backup purposes
Base Disk
Delta DiskDelta Disk
VM being
backed up
Appliance
VM
Overview of Data Recovery Components (continued)
33
VMware Data Recovery uses a variation of the new vStorage API called
vcbAPI.
Only works with vSphere vCenter 4.0 & ESX(i) 4.0.
Data Recovery VM must have a store for dedupe purposes. This can be:
Additional local VMDK assigned to the appliance (Recommended for performance
reasons—can leverage SCSI Hot-Add feature vs using the LAN)
RDM (iSCSI or FC) mapped to the appliance
SMB/CIFS shared storage configured via the VDR GUI
NFS storage supportability:
VMDK on an NFS share presented to the ESX(i) is supported.
An NFS share mapped to the appliances is not supported.
VDR does a full backup, only the used blocks on the disk are obtained. If the
disk is lazy zeroed or a thin disk, then not all of the disk needs to be read.
Storage guidelines http://viops.vmware.com/home/docs/DOC-11532
VMware Data Recovery - Licensing
34
Data Recovery is included in the following editions:
vSphere Essentials Plus vSphere Advanced
vSphere Enterprise vSphere Enterprise plus
Existing Enterprise customers are entitled to Data Recovery as part of SnS. VI3
Foundation and VI3/vS4 Standard Edition customers can purchase as an Add-
on. See vSphere Pricing, Packaging and Licensing - VMware vSphere™ 4
The VMs being protected must exist on ESX(i) hosts that have valid licenses.
VDR can protected up to a maximum of 100 Virtual Machines – SMB market.
A host that is licensed can run multiple Data Recovery appliances.
Currently only tested to one VDR appliance per vCenter Server instance.
Does not support vCenter Linked Mode.
VMware Data Recovery - Limitations
35
There is currently no integration with tape. See appendix slides on
Archiving the Backups to backup dedupe destinations to tape or replicate.
VDR cannot backup Fault Tolerant VMs as there is no way to snapshot FT
VM‘s.
VDR appliance cannot reside in a vApp.
8 concurrent jobs running on the appliance at any time (backup & restore).
Maximum of 100 Virtual Machines can be backed up using the appliance
(maximum of 100 backup jobs). This is a hard limit.
An appliance can have at the most two store destinations (up to 1TB max
each) due to the overhead involved in deduping. However this is a soft
limit and is not strictly enforced, and is unsupported if the limit of two is
exceeded.
Multiple backup appliances can be deployed but they do not interact with
each other. Hence it is up to the user to make sure that the same VM is not
used as a source in both the appliances.
There is no support to backup up VM‘s on ESX(i) 3.x hosts.
Installing
• ALWAYS use latest code! Download (see resources section)
• Install vSphere Client plug-in
• Then File \ Deploy OVF Template to deploy VDR appliance
• Add your storage destination VMDK
• Now turn on the appliance, and get ready to do configuration
Configuration Outline
• Open vSphere console to the appliance
Turn on the appliance
IP / TZ
Change root account password from vmw@re
• Use plug-in
Credentials
Destinations – may need to use IP
Create a backup job!
Technical Details
• Know your retention needs, and what you chose!
• Test carefully!
• Actually do a restore! Restore rehearsal is with new datastore and no network!
• File level restore available with experimental utility (command line tool with a
GUI version ‗soon‘)
• VDR is NOT backward compatible – NO ESX 3.5 or VC 2.5!
Technical Details
• Virtual Hardware 7 is needed for best performance! It provides changed block tracking to
shorten backup for VDR,
• First image backup is full, all the rest are incremental, but restores are always from full.
• The de-duplication is block level, not file and it happens within the VMDK even though it
is a file. The de-duplication can happen across VM‘s in the destination de-dupe store so
it is VERY efficient
• Each job runs once per 24 hour period
• If your job is at folder / host / resource pool new VM‘s back up without a configuration
change!
Technical Details – Virtual Hardware
Tip: use vCenter Upgrade Manager achieve this across all your VM‘s,
easily.
Upgrading to Virtual Hardware v.7
Things To Know
• You need to make a copy of your back up destination offsite for the best protection; you
can easily do this with VDR
• You first select your destination and than press Unmount
• No need to backup appliance (jobs, catalog for restores, and back ups all
in appliance)
• Copy it to tape, to another location, or whatever works for you
• You can mount it to another appliance as necessary. Suggest you save the .OVF of the
appliance with the destination in case of version issues in
the future
Archiving the Backups
Things To Know
• In the VS Client, Grow the VMDK file in the VM hardware settings
• Restart the appliance
• Now back in the Data Recovery client, unmount the destination, and
use the Extend option
• You may be prompted for an Integrity Check so let it start.
If not, select it on the menu bar – it is right beside Extend
• When it is ready you can mount it and it will be ready for use!
Extending Your Destination
Best Practices and Guidelines
• Use the ovf to install – is smoother and avoids human error
• Use a static IP that is added to DNS
• At most – two destinations per appliance
• At most 100 VMs backed up per appliance – hard limit
• Suggest Thick destination disk to start, not Thin. Thin works but Thick
provides a small improvement in performance
• Destination can be VMDK or CIFS, but strongly recommend VMDK for
performance
• VM‘s with RDM‘s in virtual compatibility mode are supported. VM snapshots
not possible on RDM physical compatibility, so not supported.
Best Practices and Guidelines
• Must have 5 – 8 GB in destination to start for system use, so start with at least 20 GB
• Most space efficient to have less destinations with more VM‘s rather than opposite
• And group like VMs together – to maximize dedupe!
Resources
• Community - http://www.vmware.com/products/data-recovery/community.html
• Resources - http://www.vmware.com/products/data-recovery/resources.html
• Storage guidelines - http://viops.vmware.com/home/docs/DOC-11532
• File Level Restore Usage - http://communities.vmware.com/docs/DOC-10670
• Eval guide – http://www.vmware.com/resources/techresources/10055
• Docs – http://www.vmware.com/support/pubs/vdr_pubs.html
• Release notes – http://www.vmware.com/support/vdr/doc/vdr_10_releasenotes.html
• Admin guide - http://www.vmware.com/pdf/vdr_10_admin.pdf
• Email alerts - http://communities.vmware.com/thread/252977?tstart=0
• Bits - http://www.vmware.com/download/vsphere/
Technical Resources
• http://www.vmware.com/products/data-recovery/
• http://www.vmware.com/technical-resources/virtual-storage/resources.html
• http://www.vmware.com/technical-resources/high-availability/best-practices.html
• http://www.vmware.com/technical-resources/high-availability/virtualization-backup.html
• http://www.vmware.com/a/webcasts/details/257/
Summary
• Know your retention needs, and what you chose!
• Test carefully!
• Move dedupe store destination off site!
• Keep an eye on the last backup date of the VMs!
• Lots of resources available
• Make sure your VMs are VH7!
Q and A
Questions Please!
• You cannot change the URL port from 5480!
• Change network settings via console instead of web
• VDR will backup a VM that has been VMotioned
• VDR will backup a VM that has been Storage VMotioned
• Version number can be found on the Configuration tab in the client.
• If you appliance doesn‘t keep its name, make sure you have a forward, and reverse DNS
entry for it.
• You do not need to upgrade the VMware Tools in the appliance
• The backups will not run if the host CPU utilization is 90% or higher, or if free space in
destination is less than 5 GB per VM in job
• Only 8 backup or restore jobs can run at the same time
• How do I see the other logs? Use <shift _ logs> when in the Log view.
• You do not have to change the hardware version for the appliance!
Background Material
• Error ―The operation is not allowed on non-connected sockets‖ means IP is good but the
engine in the appliance is not listening on the socket that the UI is using. Install again –
but don‘t delete!
• Yes, you can point a new appliance at old destination – all will be seen
• To be able to use DNS for destinations
/etc/hostname with FQDN
/etc/hosts with IP and FQDN
/etc/sysconfig/network (HOSTNAME=, DOMAIN=)
/etc/resolve.conf with DNS server
/etc/sysconfig/network-scritps/ifcfg-eth0 with IP info
Hostname –F /etc/hostname
Service network restart
• Reset an appliance to defaults
Service datarecovery stop
Rm /var/vmware/datarecovery/Config10.*
Service datarecovery start
Background Material #2
• Are you suspicious of your appliance and want to confirm something? You can turn off
your appliance, and install a new one. It should use a different datastore destination and
you can do a backup. In this way you eliminate your appliance and destinations as
potential issues! Most important, don‘t let a single VM be backed up by both of these
appliances!
• Did you know that when VDR starts, it polls all VMs that are part of its backup jobs, and
removes any stale SCSI Hot-Add associations to the appliance and will delete any VDR
snapshots (_datarecovery_) in any of these VMs. This is in case a crash occurred in the
middle of a backup operation.
• Did you know when VDR starts a backup of a VM it will look for and delete any snapshot
that VDR created. This is in case a crash occurred in the middle of a backup operation.
• How often does the retention policy apply? It is applied once per day if the dedupe store
is 80% full and has less than 250 GB free space, otherwise it is applied once per week.
Background Material #3 – datarecovery.ini
• You can make adjustments to VMware Data Recovery through the use of an ini file.
These require 1.0.1 of VDR.
• You will generally need to create the file and populate it.
• Log into the appliance, and stop Data Recovery (service datarecovery stop)
• Create a file called datarecovery.ini in \var\vmware\datarecovery, and create an [Options]
section in it
• Some of the more useful options include:
• IntegrityCheckInterval=x – is default every day with a once per week ―deep‖ check. If
x=2, then the deep check would be every two days.
• BackupRetryInterval=x – the default is 30 minutes – if possible, and the x value is in
minutes and has no max value.
• RetentionPolicyInterval=x – if there is more than 20% of storage available, or more
than 256 GB available, there will be no retention policy grooming. So if your dedupe
store is 81% full and with less than 256 GB free, and this is set to =3, then every three
days it will perform a groom.