IW308: Windows SharePoint Services and SharePoint Portal Server: Data Protection, Disaster Recovery, Site Archival and Beyond
Joel OlesonJoel OlesonTechnologistTechnologistIT Microsoft CorporationIT Microsoft Corporation
Overview
Microsoft IT Case StudyMicrosoft IT Case StudyMS SharePoint EnvironmentMS SharePoint Environment
IT Pain PointsIT Pain Points
Data Protection / BackupsData Protection / Backups
Disaster RecoveryDisaster Recovery
Data Recovery / RestoreData Recovery / RestoreDemo TextCopyDemo TextCopy
IT ImprovementsIT Improvements
Chofu
Dublin
Singapore
SharePoint Deployment
Redmond
Stats:Stats:10 WSS Farms, 6 SPS Farms10 WSS Farms, 6 SPS Farms7 SQL Clusters7 SQL Clusters2.7 TB of database content2.7 TB of database content~320 SPS 2003 Portals~320 SPS 2003 Portals50,000+ Site Collections50,000+ Site Collections89000 User Profiles89000 User Profiles
Americas; Europe, Middle East, Africa (EMEA); Americas; Europe, Middle East, Africa (EMEA); South Pacific; Far EastSouth Pacific; Far East
4 Regional Intranet Deployments4 Regional Intranet Deployments3 Regional Extranet Deployments3 Regional Extranet Deployments
Americas Intranet Hardware
MSWeb Shar ed Ser vices
2 WFE
2 Sear ch
2 Index
2 WFE
Por tal 100
2 WFE
Por tal 200
2 WFE
Upgr aded WSS Far m
SQL (A/P)
Team SQL Cluster
SharePoint Portal Server 2003 SharePoint Services
2 WFE
Team Far m
2 WFE
Host Header
SQL (A/P)
Por tal SQL Cluster
Regional SharePoint
SQL (A/P)
Consolidated SQL Cluster
Intranet Deployment
SQL 2000
Extranet Deployment
Shar ed Ser vices and Por tal Far m
2 WFE
2 Sear ch
2 Index
2 WFE
Cor p Host ing
2 WFE
Host Header
2 WFE
Extr anet WSS
IT SharePoint Storage4 TB SAN (DB)4 TB SAN (DB)
2 TB per SQL Cluster 2 TB per SQL Cluster (DB) (DB)
Drive RecoveryDrive Recovery
Site Backups Site Backups
OtherOtherTransaction Logs, Quorum Drive &System databases (13%)
Site Dump (29%)
DB Dump/backup (29%)
Databases (29%)
SPReport Results (June 04)Report Tool Available from Microsoft.com
Site Collections: 62000 Sites: 15000Site Collections: 62000 Sites: 15000Total Size: 2.7TB Total Size: 2.7TB Average Team Site Size: 56MB Average Team Site Size: 56MB
Large Sites (250MB+): 1710 (5-250MB): 23875 Large Sites (250MB+): 1710 (5-250MB): 23875 (5MB-): 56049 (5MB-): 56049
Average Per Site StatsAverage Per Site StatsUsed Lists: 4 Used Lists: 4 Alerts: 1 Alerts: 1 Users: 29 Users: 29
Readers: 12 Readers: 12 Contributors: 15 Contributors: 15 Web Designers: 3 Web Designers: 3 Admin: 4 Admin: 4 Customized Home Page: 15% (30% Team)Customized Home Page: 15% (30% Team)
IT Pain Points
TapeTapeTape Backups long runningTape Backups long running300-500MB/min300-500MB/minHigh NIC utilization by Tape backupHigh NIC utilization by Tape backup
DatabaseDatabaseDatabase backups long running Database backups long running
1TB 10hr (100 GB/hr)1TB 10hr (100 GB/hr)
SPSBackup backup duplication of databasesSPSBackup backup duplication of databases
Site CollectionSite CollectionSTSADM doesn’t scale well over the network STSADM doesn’t scale well over the network
~300GB in 24/hr (12MB/min)~300GB in 24/hr (12MB/min)
STSADM full 2-3 days (1TB)STSADM full 2-3 days (1TB)
Restore management painsRestore management pains
Data Protection
What to backupWhat to backup
Database BackupsDatabase Backups
ToolsToolsSPS ToolsSPS Tools
WSS ToolsWSS Tools
Spbackup Reskit demoSpbackup Reskit demo
Third Party Backup VendorsThird Party Backup Vendors
Scripts/Dev IdeasScripts/Dev Ideas
Disk to TapeDisk to Tape
SharePoint Storage Objects
Database
Site Collections
Document Libraries and Lists
Files/Documents and List Items
What to Backup
DatabasesDatabasesWSS/SPSWSS/SPS
CONFIG DBCONFIG DB
STS_W3SVC#_#STS_W3SVC#_#SPSSPS
PROF DBPROF DB
SERVDBSERVDB
SITEDBSITEDB
*External to Db*External to DbSystem StateSystem State
SPS: Indexes & SPS: Indexes & WebStore Doc LibWebStore Doc Lib
IIS Meta baseIIS Meta base
Web Part PackagesWeb Part Packages
Home DirectoriesHome Directories
Database Backups
Database to dump on diskDatabase to dump on disk
Databases TypesSharePoint DatabasesSharePoint Databases
CONFIGDB – CONFIGDB – Config data of Databases, Config data of Databases, Virtual Servers, and Site Collections Virtual Servers, and Site Collections
PROFDB PROFDB -- User profiles databaseUser profiles database
SERVDB SERVDB – Alerts & Search scopes– Alerts & Search scopes
SITEDB SITEDB - Portal's content, areas, site - Portal's content, areas, site collectionscollections
STS_W3SVC#_#STS_W3SVC#_# - Site Collection data - Site Collection data containing documents, lists, surveys, containing documents, lists, surveys, discussions, webs, etc..discussions, webs, etc..
DatabaseBackup Optimization
Database sizesDatabase sizes
25-50GB Database Storage (>500 sites)25-50GB Database Storage (>500 sites)
Number of databases (>300)Number of databases (>300)
Cost SavingsCost SavingsDirect to Media?Direct to Media?
Backup compression/SpeedBackup compression/SpeedSQL LiteSpeed – 25-35% compressionSQL LiteSpeed – 25-35% compression
Snapshots or ClonesSnapshots or Clones
SharePoint Portal ToolsSPSBackup.exeFull Farm BackupFull Farm BackupSpsbackup.exeSpsbackup.exe /all /file <Backup path and file>/all /file <Backup path and file>
Central Portal Index Backup:Central Portal Index Backup:
Spsbackup.exeSpsbackup.exe /portal http://portal /service index /file \\server\share\backup /overwrite /portal http://portal /service index /file \\server\share\backup /overwrite
DatabasesDatabasesBackward-compatible Doc LibBackward-compatible Doc LibSPS IndexesSPS IndexesXML manifestXML manifest
schema versionschema versionportal GUID portal GUID Path to DB backup filePath to DB backup fileResultResult
Windows SharePoint Services ToolsSTSAdm.exe UtilitySTSAdm.exe Utility
stsadm.exe –o backup –url stsadm.exe –o backup –url http://team/sites/site -f file.bakhttp://team/sites/site -f file.bak
Site Collections OnlySite Collections Only
WSS sites – full fidelity w/ securityWSS sites – full fidelity w/ securityCustom assemblies/web parts not copiedCustom assemblies/web parts not copied
MySites and Team Sites on Portals – Not MySites and Team Sites on Portals – Not recommended by Supportrecommended by Support
Windows SharePoint Services Tools (Cont)Smigrate.exe “Smigrate.exe “The SharePoint Migration Tool”The SharePoint Migration Tool”smigrate –w http://vserver/sites/site -f c:\file.fwp smigrate –w http://vserver/sites/site -f c:\file.fwp
Site Collection & Sub Sites SupportedSite Collection & Sub Sites SupportedFiles & Lists copied to disk with manifest Files & Lists copied to disk with manifest Same as FrontPage2003 Client Backup/RestoreSame as FrontPage2003 Client Backup/RestoreSupport for Site Collections on SPS and Sub Support for Site Collections on SPS and Sub sites (not Areas), WSS, and Personal Sitessites (not Areas), WSS, and Personal Sites
** NOTE:** NOTE:My sites - No Personalization (Home Page My sites - No Personalization (Home Page reset), profile changes and Securityreset), profile changes and SecurityOther Sites and Site Collections –Security Lost Other Sites and Site Collections –Security Lost during backupduring backup
SharePoint BackupThird Party VendorsAvePointAvePoint
DocAve – SPS 2003 – Item level backup/restore/archive with DocAve – SPS 2003 – Item level backup/restore/archive with multiple migration options (supports SQL & Webstore)multiple migration options (supports SQL & Webstore)Downloadable Demo AvailableDownloadable Demo Available
Computer Associates (CA)Computer Associates (CA)BrightStor ARCserve Backup BrightStor ARCserve Backup
Backup/Restore SharePoint databases hosted on the various SQL Backup/Restore SharePoint databases hosted on the various SQL servers distributed across the enterprise.servers distributed across the enterprise.Advanced document level backup and restore supportAdvanced document level backup and restore support
CommVaultCommVaultGA, Galaxy- SPS 2003 database level backup and recovery GA, Galaxy- SPS 2003 database level backup and recovery productproduct
Enterprise wide backup and recovery of SPS 2003 data (SQL and Enterprise wide backup and recovery of SPS 2003 data (SQL and indices). Site and sub-site backup and recovery; Webstore backup indices). Site and sub-site backup and recovery; Webstore backup supportsupport
Product in Beta for “Doc Level” backup/restoreProduct in Beta for “Doc Level” backup/restoreVeritasVeritas
Backup Exec 10.0 (Late Q3 04) - SPS Server Level backupBackup Exec 10.0 (Late Q3 04) - SPS Server Level backup
SpbackupSpbackup
Automating Site Backup (Reskit Tool)Automating Site Backup (Reskit Tool)http://www.microsoft.com/technet/prodtechnol/sppt/reskit/c2861881x.mshttp://www.microsoft.com/technet/prodtechnol/sppt/reskit/c2861881x.mspxpx
Site ArchivalCleanup Storage
Auto Notification Auto Notification
Auto DeleteAuto Delete
Usage Logging and ReportingUsage Logging and Reporting
Analyze Site UsageAnalyze Site UsageLast ModifiedLast Modified
IIS usageIIS usage
Identify sites to “cleanup”Identify sites to “cleanup”
Archive/Backup then Delete targeted sitesArchive/Backup then Delete targeted sites
DBCC Shrink DBDBCC Shrink DB
Monitoring BackupsSharePoint Backup Relevant MOM Events
WSS MOM PackWSS MOM PackDatabase Capacity ReachedDatabase Capacity ReachedDatabase Warning ReachedDatabase Warning Reached
SPS MOM PackSPS MOM PackSPSBackup.exeSPSBackup.exe backup success/fail backup success/fail
Indexes or databaseIndexes or database
SQL Server 2000 MOM PackSQL Server 2000 MOM PackSQL Server database and transaction log space SQL Server database and transaction log space warnings and errors.warnings and errors.SQL Server failed backupsSQL Server failed backupsSQL Server failed agent jobsSQL Server failed agent jobs
Backup Media
Disk to TapeDisk to Tape
TapeBackup Optimization
Network Upgrade - Over NetworkNetwork Upgrade - Over NetworkPrivate NetworkPrivate NetworkSAN TapeSAN Tape
1GB/Min (db dump to tape)1GB/Min (db dump to tape)
Max drive or share sizeMax drive or share size100-300 GB for quicker restart100-300 GB for quicker restart
Script/Dev ideas
Incremental Site Backup – Incremental Site Backup – backup changed backup changed sitessites
Document backup – Document backup – Web folder backup of Web folder backup of documents or object model iterating through documents or object model iterating through documentsdocuments
Incremental Document backupIncremental Document backup – Backup – Backup changed documents using Object Modelchanged documents using Object Model
Archive – Archive – Use visits w/ modify date to Use visits w/ modify date to determine what sites should be archived determine what sites should be archived (backed up to disk) then deleted(backed up to disk) then deleted
Data Recovery
Disaster RecoveryDisaster Recovery
Recovery Scenarios & RequestsRecovery Scenarios & Requests
Recovery FlowRecovery FlowDatabaseDatabase
GranularGranular
Demo: Textcopy Demo: Textcopy Doc restore from Content DB Doc restore from Content DB
Portal RestorePortal Restore
Service LevelsService Levels
Disaster Recovery: Server
Web Front End – Web Front End – A. Restore failed drives and system state (metabase) A. Restore failed drives and system state (metabase) B. Reinstall, remove/re-add topology reconfigure, re-add web parts, B. Reinstall, remove/re-add topology reconfigure, re-add web parts, recreate virtual servers, extend and map to existing virtual serversrecreate virtual servers, extend and map to existing virtual servers
Search – Search – A. Restore failed drives and system state (metabase) propegateA. Restore failed drives and system state (metabase) propegateB. reinstall, remove/add back to topology and propagateB. reinstall, remove/add back to topology and propagate
Index – Index – A. Restore failed drives and system state (metabase) restore Index A. Restore failed drives and system state (metabase) restore Index using using spsbackup.exespsbackup.exereinstall, remove/add back to topology, restore indexes using reinstall, remove/add back to topology, restore indexes using spsbackup.exespsbackup.exe, and propagate, and propagate
SQLSQL – Restore failed databases (except Config db) – Restore failed databases (except Config db)Disconnect reconnect content databasesDisconnect reconnect content databases
Backward Compatible Doc LibBackward Compatible Doc Lib – –Disconnect Doc libs, remove from topology & farmDisconnect Doc libs, remove from topology & farmRe-add new server to farm & topologyRe-add new server to farm & topologyRestore using Restore using SPSBackup.exeSPSBackup.exe restore restore
Hot Swappable FarmSQL Log Shipping (See Reskit Chapter 28)
Mirror ServersMirror ServersSet DB Full or Bulk Logged recovery modeSet DB Full or Bulk Logged recovery modeMirror DataMirror Data
Full Backup of FarmFull Backup of FarmFull Restore of FarmFull Restore of Farm
Enable SQL Log shippingEnable SQL Log shippingSITE, SERV, PROF & Content dbs SITE, SERV, PROF & Content dbs (NOT CONFIG)(NOT CONFIG)
Set DBs on Mirror to Standby or No RecoverySet DBs on Mirror to Standby or No RecoveryConfigure FrequencyConfigure FrequencyCreate Farm, Customization/Portal & Virtual Server Create Farm, Customization/Portal & Virtual Server configconfig
Recommended: Periodic execution of “delete portals and Recommended: Periodic execution of “delete portals and detach databases; reconnect them to mirror farm config detach databases; reconnect them to mirror farm config DB” DB”
Recovery of Hot Swap Farm Via SQL Log ShippingSQLSQL
Close Open ConnectionsClose Open Connections
Change role of Secondary SQL to Primary (or Change role of Secondary SQL to Primary (or keep dbs read only until recovery of primary)keep dbs read only until recovery of primary)
SharePointSharePointAs needed Create Portal from DBs (connect)As needed Create Portal from DBs (connect)
Disconnect/Reconnect Content DbsDisconnect/Reconnect Content Dbs
WINS/DNS to point to new Farm VIPWINS/DNS to point to new Farm VIP
-Start a Full Crawl-Start a Full Crawl
Recovery ScenariosDatabase -Database - most reliable disaster recovery most reliable disaster recovery scenarioscenario
Site Collections – Site Collections – STSAdm.exe STSAdm.exe restorerestore
Database restoreDatabase restore
Document Library, List, Doc recovery Document Library, List, Doc recovery – – Database restoreDatabase restore
STSAdm.exeSTSAdm.exe restore restoreDocumentDocument- - Textcopy or other API can make retrieval Textcopy or other API can make retrieval from DB possiblefrom DB possible
Restore Requests
Default 15000 Sites MaxPer Content DB
Avg 20GB
>1% of Restore Requests
Includes Sub sites Workspaces
Avg 30-65 MB
41% Restores68/167 site68/167 site
Doc Libraries, list items, lists, events,Surveys, forms, etc..
Avg 1.5 MB
49% Restores82/167 doc82/167 doc
Database Restore
Tape to Disk to Online dbTape to Disk to Online db
Database Recovery
SITE / STS_W3SVC#_#SITE / STS_W3SVC#_# – restore database, – restore database, remove from config db using admin UI, re-remove from config db using admin UI, re-add using admin UIadd using admin UI
PROF DB – PROF DB – restore profile database, start full restore profile database, start full import, remove objects missing from import import, remove objects missing from import using UIusing UI
SERV DB – SERV DB – restore services database, bring restore services database, bring online (keep in sync w/ SITE DB)online (keep in sync w/ SITE DB)
Granular RecoveryPortal and below
Restore relevant databases (SITE, SERV, PROF, Restore relevant databases (SITE, SERV, PROF, STS_W3SVC#_#)STS_W3SVC#_#)
Restore to standby environment using Restore to standby environment using SPSBackup.exeSPSBackup.exe / /Restore tab GUIRestore tab GUI
STSAdmSTSAdm for site collection migration for site collection migration
SmigrateSmigrate for area or sub site migration (reapply security) for area or sub site migration (reapply security)
TextCopyTextCopy for doc restore for doc restore
Portal Restore From DBDBs Only (no index)
Central AdminCentral AdminClick - Create a portal siteClick - Create a portal site
Choose - Restore a portal siteChoose - Restore a portal site
Select Servers and databasesSelect Servers and databases
Choose Virtual Server and specify URLChoose Virtual Server and specify URL
Extend and MapExtend and Map
Granular RecoverySite/Sub siteSite CollectionSite Collection
Backed up Backed up Stsadm.exe –o restore –url Stsadm.exe –o restore –url http://http://sitenamesitename –f filename –f filename –overwrite–overwriteRequires 2Requires 2ndnd content db content db
No Site BackupNo Site BackupRestore content databaseRestore content databaseAttach database to Attach database to standbystandby virtual server/Lab virtual server/Lab environmentenvironmentBackup site using Backup site using stsadm.exestsadm.exeRestore using Restore using stsadm.exestsadm.exe with overwrite option with overwrite option
Sub site - use Sub site - use smigratesmigrate to move data to production to move data to production with overwritewith overwrite
Document Restore TextCopy
What Content Database to Restore?What Content Database to Restore?USE @WSS_configUSE @WSS_config
SELECT nameSELECT name
FROM sites s, databases dFROM sites s, databases d
WHERE s.databaseid = d.databaseidWHERE s.databaseid = d.databaseid
AND urlpath = @urlpathAND urlpath = @urlpath
VariablesVariables@WSS_config = configuration database name@WSS_config = configuration database name
@URLpath=Site; @leafname = filename @dirname = directory name of @URLpath=Site; @leafname = filename @dirname = directory name of file file
SQL Query to Define DocWhat Site and Document?What Site and Document?USE [X1_SITE]USE [X1_SITE]SELECT content SELECT content FROM docsFROM docsWHERE dirname = @dirnameWHERE dirname = @dirnameAND leafname = @leafnameAND leafname = @leafname
Example Text CopyExample Text Copytextcopy /s server /u @user /P @password /d textcopy /s server /u @user /P @password /d
“@database" /t docs /c content /W "where “@database" /t docs /c content /W "where leafname= ‘@leafname’ and dirname= leafname= ‘@leafname’ and dirname= ‘@dirname’” /F c:\temp\leafname /O /Z‘@dirname’” /F c:\temp\leafname /O /Z
VariablesVariables@WSS_config = configuration database name @database = content database@WSS_config = configuration database name @database = content database@URLpath=Site; @leafname = filename @dirname = directory name of file @URLpath=Site; @leafname = filename @dirname = directory name of file
IT ImprovementsTapeTape
SAN Tape/Fiber Attached BackupSAN Tape/Fiber Attached BackupThroughput 1.2 GB/Min, 2-3xThroughput 1.2 GB/Min, 2-3x
Split drive into ~300GB shares for Tape BackupSplit drive into ~300GB shares for Tape BackupGB Network upgrade on SQL boxesGB Network upgrade on SQL boxes
DatabaseDatabaseReduce database sizes Max 25-50GBReduce database sizes Max 25-50GBReduce DB to >300DB per SQL instanceReduce DB to >300DB per SQL instanceMulti backup and SQL LiteSpeedMulti backup and SQL LiteSpeedSPSBackup.exeSPSBackup.exe to backup indexes only to backup indexes onlyDefragment Drives and databasesDefragment Drives and databases
Site CollectionSite CollectionIncremental Backups (backup changed sites)Incremental Backups (backup changed sites)Cleanup/AutoDelete – ArchiveCleanup/AutoDelete – ArchiveSite Backup storage on Passive Node/File serverSite Backup storage on Passive Node/File serverGB Network upgrade on Web ServersGB Network upgrade on Web Servers
Restore process flow better understoodRestore process flow better understood
+IT SharePoint Best Practices
Limit # of site collections per database to Limit # of site collections per database to 250-500250-500
Limit Large sites at ~2GBLimit Large sites at ~2GB
Use few IIS Virtual ServersUse few IIS Virtual Servers
Consolidate app pools for common appsConsolidate app pools for common apps
Blocked File TypesBlocked File Types
Antivirus on SQL Servers: Exclude FTS, Antivirus on SQL Servers: Exclude FTS, Database & Log drivesDatabase & Log drives
Additional Lessons Learned
4 GB RAM on Web Servers use /3GB in 4 GB RAM on Web Servers use /3GB in boot.iniboot.ini-g512 startup parameter in SQL for -g512 startup parameter in SQL for additional MTL (Mem To Leave)additional MTL (Mem To Leave)What to watch: App Pools & SQL Memory, What to watch: App Pools & SQL Memory, NIC, SQL Server Connections, SQL NIC, SQL Server Connections, SQL Blocking, disk usageBlocking, disk usage# of databases below 600# of databases below 600Isolate Critical Apps in their own app poolIsolate Critical Apps in their own app poolWatch for w3wp.exe memory exceeding Watch for w3wp.exe memory exceeding 800mb800mb
Q&A
© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
Q1: Overall satisfaction with the session
Q2: Usefulness of the information
Q3: Presenter’s knowledge of the subject
Q4: Presenter’s presentation skills
Q5: Effectiveness of the presentation
Please fill out a session evaluation on CommNetPlease fill out a session evaluation on CommNet
Attend a free chat or web casthttp://www.microsoft.com/communities/chats/default.mspx http://www.microsoft.com/usa/webcasts/default.asp
List of newsgroupshttp://communities2.microsoft.com/communities/newsgroups/en-us/default.aspx
MS Community Siteshttp://www.microsoft.com/communities/default.mspx
Locate Local User Groupshttp://www.microsoft.com/communities/usergroups/default.mspx
Community siteshttp://www.microsoft.com/communities/related/default.mspx
Three ways to access Online Evaluation Forms:Three ways to access Online Evaluation Forms:
MGB Evaluation Kiosks MGB Evaluation Kiosks
CommNet CommNet
From your laptop or Pocket PC, From your laptop or Pocket PC, onsite Wireless Access or any wired connection to: onsite Wireless Access or any wired connection to: http://mgb2004.com/evalhttp://mgb2004.com/eval
Please Complete An Evaluation FormYour input is important!
For more information please refer to the Registration GuideFor more information please refer to the Registration Guide
© 2004 Microsoft Corporation. All rights reserved.This presentation is for informational purposes only. Microsoft makes no warranties, express or implied, in this summary.
Top Related