Dealing with large Content Scenarios in SharePoint Server 2007
description
Transcript of Dealing with large Content Scenarios in SharePoint Server 2007
![Page 1: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/1.jpg)
Dealing with large Content Scenarios in SharePoint Server 2007 Architecture, Challenges, and Strategies
Abrar Chisti, Microsoft Corporation
![Page 2: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/2.jpg)
![Page 3: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/3.jpg)
AgendaOverviewManageabilityPlanningAvailabilityCase StudyTakeaway’s
![Page 4: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/4.jpg)
![Page 5: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/5.jpg)
Content Database GrowthUse as Document Repository
Multiple versions of documents70-95% of size is File Stream
Storage of large Multi Media filesLack of Governance/Site Quotas
One Large Site CollectionLack of Planning
![Page 6: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/6.jpg)
Is SharePoint the Right Solution?
SharePoint sites evolve organically.Database Capacity planning is often overlookedLimited or no GovernanceOne or more large content database(s)
Difficulty for IT to maintainIO Throughput and Latency is effected
![Page 7: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/7.jpg)
Manageability
![Page 8: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/8.jpg)
Plan for ManageabilityLimit Content Database Size to <= 100GIf Content DB Size is > 100G
Use Differential/Incremental BackupsSQL Server 2005/2008DPM 2007
Test & Baseline IO Sub-SystemSet DB Auto-growth to Fixed ValueSplit Sites in Content DB to multiple Content DB’s
![Page 9: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/9.jpg)
Backup & Restore Options
![Page 10: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/10.jpg)
How to Manage ContentSplit Content Database
Move Site Collections between DatabasesMove Sites into Site Collections (Re-Parent)
May need to promote sub sites to sitesMay need to move site collections between web applications
Use OOB or 3rd Party ToolsStsadm –o export/importStsadm –o backup/restoreStsadm –o mergecontentdbContent Deployment API (Selective)
![Page 11: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/11.jpg)
How to Limit StorageDocument Libraries
Limit # of Versions.Archive or Delete Old SitesArchive or Delete Unused SitesImpose Site Quotas
Different types of quotas – Small/Med/LargeTake into Consideration Recycle Bin
Manage Lists for Performance
![Page 12: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/12.jpg)
Upgrade Hardware/SoftwareEnsure Latest SP/PatchUse Dedicated SQL ServerUse 64 Bit Architectures and 64 Bit OSUse MS Hardware RecommendationsUse SQL Server connection alias when you configure your farmIncrease Bus Bandwidth
![Page 13: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/13.jpg)
Take Advantage of SQL Server 2008 Capabilities
Performance - Implement database backup compression.Availability - Implement log stream compression.Security – Implement Transparent Data Encryption (TDE).Resource management – Use SQL Server 2008 Resource GovernorBe Aware of DB Migration Considerations
![Page 14: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/14.jpg)
Content Archival/ReductionUse Database SnapshotsUse Records Repository ImplementationExternalize (BLOB) storage
![Page 15: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/15.jpg)
Database SnapshotProvides “snapshot” of
Content DB at given instant.Requires Same DB Server Instance
Refers to the Original DatabaseUses “Copy on write” mechanismNeed to create Separate Web App.
![Page 16: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/16.jpg)
Records Repository
![Page 17: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/17.jpg)
Remote/External Blob Storage
Reduce Storage CostsExternal Blob Storage APIRemote Blob Storage APISQL Server 2008 has support
for RBSCan write BLOB directly using
RBIhttp://blogs.msdn.com/sqlrbs/
![Page 18: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/18.jpg)
External Blob Based Solution-BLOB IO is moved toWeb Front End
-Supports Compression And Encryption Capability
![Page 19: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/19.jpg)
Planning
![Page 20: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/20.jpg)
Plan for Software BoundariesBottom Up Approach
Plan for SQL StorageSharePoint Performance Recommendations
# of Site Collections/Content DB50,000
# of Site Collections/Web Application150,000 Site Collections
100 Content DB’s Per Web ApplicationUse Multiple SQL Servers for Higher Scalability
![Page 21: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/21.jpg)
Storage ArchitectureUse Appropriate Disk and SAN interface
SCSI vs IDE vs SATA vs SASConsideration – Hot Swap, Multiple IO, Speed, Capacity, Protocol
Use Appropriate Disks and RAID Arrays
Faster Disks/ArraysSeparate Disks for TempDB, ContentDB, and Trans LogsMultiple Data Files for Large Content and Search DB’s
Distribute files across Disks
![Page 22: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/22.jpg)
Content Database AllocationSharePoint Allocation of Content DB’s
Pre-Allocate Pool of db’sRound Robin Scheme between DB’s
Based on Delta between Max sites and Current sitesExample
Site Collection Per DatabaseCreate Database with 100G (using ALTER DB Command)Leverage Managed Paths
![Page 23: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/23.jpg)
Availability
![Page 24: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/24.jpg)
ClusteringSAN or Shared Disks
Use Windows/SQL Clustering for HADedicated Disks or DAS
Use SQL Server Mirroring
![Page 25: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/25.jpg)
Redundancy across Data Centers
Log ShippingSynchronous MirroringAsynchronous Mirroring
SQL Server 2008 Log Compression
![Page 26: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/26.jpg)
![Page 27: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/27.jpg)
High Availability Farm
![Page 28: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/28.jpg)
Monitoring
![Page 29: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/29.jpg)
MonitoringProcessor: % Processor Time: _Total. On the computer that is running SQL Server, this counter should be kept between 50 percent and 75 percent. System: Processor Queue Length: (N/A). 2 x #of core CPUs.Memory: Available Mbytes: (N/A). Monitor this counter to ensure that you maintain a level of at least 20 percent of the total physical RAM available.Memory: Pages/sec: (N/A). Monitor this counter to ensure that it remains below 100.
![Page 30: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/30.jpg)
Disk CountersLogical Disk: Disk Transfers/sec Logical Disk:Disk Read Bytes/sec & Disk Write Bytes/secLogical Disk: Average Disk sec/Read (Read Latency)/Avg Disk Sec/WriteLogical Disk: Average Disk Byte/Read/WritePhysical Disk: % Disk TimeLogical Disk: Current Disk Queue LengthLogical Disk: Average Disk Reads/Sec and Logical Disk
![Page 31: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/31.jpg)
Performance MonitoringPerfmon
Analyze Logs using codeplex toolsFavorite Web Monitoring (3rd Party) solution.System Center Operations Manager (SC-OM)
SharePoint Monitoring Toolkithttp://blogs.msdn.com/sharepoint/archive/2007/12/10/announcing-new-system-center-operations-manager-2007-packs-for-wss-3-0-and-moss-2007.aspx
![Page 32: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/32.jpg)
Case Study
Large Automotive Loan Origination Application
![Page 33: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/33.jpg)
Large Storage Scenario (Phase I)
Ability to house 10.5 million content items (1+TB).System input with "normal" input load, defined as 27,000 document per day (1 day = 10 hours). Simulate user load to represent 200 users simultaneously accessing the system to:
Use search to find elements of document metadata.View a document (scanned TIFF image).Update elements of document metadata.
![Page 34: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/34.jpg)
Phase IIAbility to house 50 million content items (5+TB).
35 million TIFF images.15 million Microsoft Office documents
Determine the maximum number of users the solution could support.Users perform the following tasks:
Use search to find elements of document content (full-text) and metadata.View a document (scanned TIFF image or Microsoft Office document).
![Page 35: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/35.jpg)
Architectural OverviewLogical Architecture – Phase I
![Page 36: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/36.jpg)
![Page 37: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/37.jpg)
![Page 38: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/38.jpg)
Architectural OverviewLUN/DBMatrix
![Page 39: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/39.jpg)
Takeaway’sOptimize Performance
Planning & MonitoringPlan for ScalePlan for AvailabilityPlan for Manageability
![Page 40: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/40.jpg)
ReferencesSQL Server Database Optimization
http://technet.microsoft.com/en-us/library/cc263261.aspxPlan for Software Boundaries
http://technet.microsoft.com/en-us/library/cc262787.aspxMove Site Collections to new Content Database
http://technet.microsoft.com/en-us/library/cc825328.aspxEnable SharePoint 2010 to Use Remote BLOB Storage
http://technet.microsoft.com/en-us/library/ee748641(office.14).aspx/Content Deployment API (PRIME)
http://msdn.microsoft.com/en-us/library/cc264073.aspxIntegration of SQL Server 2008 and SharePoint
http://msdn.microsoft.com/en-us/library/cc264073.aspxUse Database Snapshots for Archiving Sites
http://technet.microsoft.com/en-us/library/cc706872.aspxConfigure Availability in SharePoint Farm
http://technet.microsoft.com/en-us/library/dd207311.aspxCase Study for Large Content Scenario
http://technet.microsoft.com/en-us/library/cc262067.aspxScaling Storage Architecture
http://www.knowledgelake.com/whitepaper/Scaling%20SharePoint%202007%20-%20Storage%20Architecture.pdf
![Page 41: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/41.jpg)
Remember to fill out your evaluations for your chance to win a Zune HD and one of 12 copies of
Office 2007 (13 prizes will be awarded)
Evaluation Prizes Sponsored by:
![Page 42: Dealing with large Content Scenarios in SharePoint Server 2007](https://reader035.fdocuments.us/reader035/viewer/2022062501/568165db550346895dd8eeaf/html5/thumbnails/42.jpg)
Tools AvailabilitySPUsed Space InfoSPSiteInfoContent Deployment Wizard
Migrate from other source systems.Other tools in CodePlex3rd Party
Metalogix, Qwest, Tzunami, AvePoint, StoragePoint, Knowledge Lake