© Grant Thornton LLP. All rights reserved. © 2010 Mark Vogt Ltd. All rights reserved or... "What...

© Grant Thornton LLP. All rights reserved.© Grant Thornton LLP. All rights reserved.© 2010 Mark Vogt Ltd. All rights reserved

or...

"What you should be doing to help your mature-as-it's-ever-going-to-be SharePoint 2007 farm through to SP2010“

presenter:

Mark VogtVOGTLAND CONSULTING

2010.03.18

SHAREPOINT 2007 PERFORMANCE TUNING

MUST-HAVES


About The Presenter…

Mark Vogt, Mark Vogt, Ltd – SharePoint Consultant, PM & EA Presenter Bio:- BS: Aerospace Engineering - U of MN IT- MS: Mechanical Engineering & Robotics (ABT) – U of MN IT- MS: Computer Science/AI - IIT- Consulting in KM & portal technologies since 1995- Roles actively include developer, PM, architect & manager - Worked with SharePoint technologies since it came out in 2000- Runs blogs & videos focusing on SharePoint, Consulting & “The Big Picture”... - LOVES SharePoint- Sees SharePoint as one-of-a-kind enabling technology with no equal- SharePoint Purple Belt


Target Audience: SharePoint Green Belts…

• CSPUG Chairs • Presenters • Sponsors • Visitor/Novices • White Belts • Green Belts • Purple Belts • Brown Belts • Black Belts

Most CSPUG Attendees are Green Belts - people experienced in using SharePoint for months/years, possibly experts in certain aspects of SharePoint, but NOT experts in all…


Deliverables: from this Presentation…

• Overview of Situation for you & your colleagues• Definitions for you & your colleagues• Explanations for you & your colleagues• Risks/Benefits for your Bosses• Checklists for your Administrators• Next Steps for everyone


Game Plan: Why the slides are so “dense”…

• The presentation is meant to serve as a take-away• We’ll be moving fast & skimming• You can review the added details later• That way the presentation is still useful without presenter


Overview of Situation…

It's 2010 - SharePoint 2007 has been out for a full 3 years;• Even late adopters now have a fully-mature SharePoint 2007 environment;• "Fully-Mature" is code for "everyone's finally using it"; • "Fully-Mature" is code for "at or beyond limits of original design";• "Fully-Mature" is code for "daily complaints about performance";• "Fully-Mature" is code for "not putting another dime into it";

This presentation will cover some "low hanging fruit" which can significantly improve your SP2007 performance, at least long enough to let you deploy your SP2010 farm to new, fast hardware, migrate content and let your "fully-mature" users into the new environment…

(note: "fully-mature users" is also code for "now they take SharePoint for granted, ignore all your warnings about what NOT to do in your farm, turn on versioning 20 levels deep, use WiFi connections 300 ft from the nearest WAP, upload 100 MB files which have NO value to anyone but themselves, install web parts and javascript they KNOW they're NOT supposed to, coerce your developers into "enhancing" your farm WITHOUT your knowledge… and still complain about YOUR farm"… )


What can you do?...

• We can't talk about everything you could do…• We can talk about a few "must-haves"…

– Techniques that are easy to implement– Techniques that apply to all SharePoint environments– Techniques with high benefit-to-risk ratios


Tip: Be Methodical in evaluating each improvement…

• Devise repeatable tests with: • large & small files (emphasize most common file sizes expected)• varying file types (different file types are affected differently)• Varying operations (typically Page Loads, Downloads, Uploads, Edit/Saves)• short & long distances from The Farm • wide & narrow bandwidth to The Farm

• Re-Test again & again as “stack on” each enhancement


Must-Have: IIS (HTTP) Compression…

Definition:

"The seamless, on-the-fly compression, transmission and decompression of file-based content from web server to web browser";

• It's NOT "IIS" Compression;• It IS (more accurately) "HTTP" compression;• Works on outbound (downloads) content delivery• Works with individual files (e.g. htm, aspx, js, css,) comprising a

request• Disabled OOTB, but enabled per SP web app during provisioning


Explanation: HTTP Compression…


Free Tool: Fiddler lets you see HTTP compression…

http://www.fiddlertool.com


Risks/Benefits: HTTP Compression…

• Low Risks:– OOTB in SP2007;– Single Metabase.xml file requires editing– Manual editing possible & straightforward– Readily/Easily adjusted or even reverted– Some ZIP-related oddities; fix available– Requires WFEs which are below 80% Avg CPU (typical)

• High Benefits:– Outbound content reductions from 15% to 95% !


Checklist: HTTP Compression…

PRE-EDIT STEPS:_ OPEN Start > Administrative Tools > IIS Manager

_ RMC on Local Server Icon > Properties

_ CHECK checkbox property “Enable Direct Metabase Edit”

EDIT STEPS:_ LAUNCH cmd.exe > IISRESET /stop

_ OPEN Windows Explorer

_ NAVIGATE to C:\Windows\System32\inetsrv\

_ LOCATE Metabase.xml file

_ DUPLICATE Metabase.xml file > Metabase_20100318.xml

_ EDIT Metabase.xml

_ SEARCH for text “IISCompressionScheme” (2 elements – gzip & deflate)

_ EDIT <IISCompressionScheme> elements (so both match)

_ CHANGE Compression Levels to "9"

_ INCLUDE all the static & dynamic content YOU have!

POST-EDIT STEPS:_ SAVE & CLOSE Metabase.xml

_ RESTART IIS via cmd.exe > IISRESET /start

_ RE-TEST performance tests &record results

_ CHECK with Users (ultimate judges of success)

SAMPLE EDITS: …

<IIsCompressionScheme Location ="/LM/W3SVC/Filters/Compression/gzip"

HcCompressionDll="%windir%\system32\inetsrv\gzip.dll"

HcCreateFlags="1"

HcDoDynamicCompression="TRUE"

HcDoOnDemandCompression="TRUE"

HcDoStaticCompression="TRUE"

HcDynamicCompressionLevel="9"

HcFileExtensions="htm

html

txt

js

css

htc

doc

docx

xls

xlsx

ppt

pptx

pdf"

HcOnDemandCompLevel="9"

HcPriority="1"

HcScriptFileExtensions="asp

aspx

asmx

dll

exe">

</IIsCompressionScheme>

…

C:\windows\system32\inetsrv\metabase.xml:


Results: HTTP Compression…

TIP: Test new file formats by batch-ZIP-ing a bunch of them, and do some stats…

Extension File Type Low Average High

XLS, XLSX Static 60% 75% 95%

DOC, DOCX Static 15% 25% 40%

PPT, PPTX Static 10% 15% 20%

VSD Static 10% 20% 30%

JS, CSS Static 30% 50% 70%

PDF Static 3% 25% 60%

ASPX, ASMX Dynamic 65% 75% 85%

DLL, EXE Dynamic 55% 60% 65%

TIF Static 2% 15% 20%

JPG, GIF Static - - -


Thought…

Why does HTTP compression only work on outbound traffic?

(save for discussion…)


Uber-Topic: SharePoint (ASP.NET) Web Caching…

Definition: Web Caching – The temporary storing (typically in WFE's RAM or VRAM) and retrieving of already-generated HTTP responses or portions of responses.

This is MUCH (10x) faster than re-requesting the same page or portions of pages (resources) all the way back to the Database Server…


Explanation: SP2007 Web Caching…

Output Caching

BLOB Caching

Object Caching

Browser (private)Caching

RAM

VRAM

Output (page)Caching

CHALLENGE: caching is going on everywhere…



FAST SLOW


Confusion: So much caching-related lingo…

How Mark makes sense of it all:

Caching Data Caching Web Caching Browser-side ("client-side") Caching Server-side Caching Output (Page) Caching Object (Control) Caching BLOB (Resource) Caching Appliance-side Caching (aka "edge" caching)


Table: Summarizing SP2007 Web Caching…

TYPE LOCATION RESOURCES TOOLS SCOPE CACHES TUNING

BrowserCaching

Browser RAMVRAM

browser N/A everything

Output Caching

WFE RAMVRAM

SPCA site collection Web PagesWeb Part HTML

SizeProfiles

BLOB Caching

WFE Disk web.config web application Resource files (gif, jpg, css, js)

LocationSizeFiles

Object Caching

WFE RAMVRAM

SPCA site collection controls, data, fields

Size

Everything Caching

caching appliance

RAMVRAM

Custom Everything Everything Size


Type: Browser("private") Caching…

• IE7 has built-in "private" caching• Limit is 1024 MB (1 GB) • Represents the very fastest

(shortest) caching• Operation can be hard to

administratively control and optimize…


Must-Have: Custom cache-control HTTP Header…

Browser cache usage can be improved for static websites via custom HTTP response headers;

“post-check” is cool trick to load “old” page instantly from browser cache, then update behind-the-scenes as needed…


Risks/Benefits: Custom cache-control HTTP Header…

• Low Risk– Leverages OOTB caching in IE7 IE8– Readily/Easily adjusted via IIS Manager at web app– Tends to benefit mostly-static web pages/sites

• High Benefits– Fastest possible caching scenario– Significant reduction of Page Load times– Clever leveraging of “effective Page Load” phenomena

(“perception is reality…”)


Checklist: Custom cache-control HTTP Header…

SCHEDULE the following tasks for next maintenance window

__ RDC into each WFE in The Farm with admin-level privileges

__ PERFORM IISRESET /stop /noforce

__ OPEN IIS Manager

__ FOR EACH (web application in SharePoint farm)

__ RMC > Properties > HTTP Headers

__ ADD new custom header: “cache-control: max-age=3600, post-check”

__ RESTART IIS via IISRESET /start

__ RE-TEST Performance; record results; compare to previous baseline

__ EXPERIMENT with max-age (vary between 1 min and 1 hour)

__ CHECK with Users (ultimate judges of success)


Type: Server-side (WFE) Caching…

1. Output or "Page" Caching (think "pages & web parts")

2. BLOB or "Disk" Caching (think "resources/external files")

3. Object Caching (think "page fragment/control")

• All exist on each WFE• All use either WFE RAM or VRAM (disk)• Some share the SAME server resources (RAM/VRAM)!• Each type of caching serves a specific purpose…• Not all purposes apply to all environments• => Not all caching will benefit all situations


Must-Have: BLOB (disk) Caching…

• AKA "disk-based" caching• Content really is stored on disk instead of RAM• "BLOBS" include all file content typically stored in and retrieved from

the database (e.g. js, css, gif, jpg, etc); • Think "resources/files that are also part of a requested page“• Typically 20-60 resources for SharePoint page ! (significant)• Part of ASP.NET (built right into native IIS performance counters)• Disabled by default in SP2007; • Enabled on per-web application basis (easy to manage)


Explanation: BLOB (disk) Caching…

Edit each web app's web.config file:


Risks/Benefits: BLOB (disk) Caching…

• Low Risk– OOTB in SP2007;– Single line of web.config file; – Highly configurable wrt location, size, selection;– Location off of WFE further improves performance;– Readily/Easily adjusted or even disabled;– Managed at the Web Application-level (*whew*)

• High Benefits– Reduction of Browser Load times ranging from ½ to 1/7th

of un-cached load times


Checklist: BLOB (disk) Caching…


__ PRE-TEST performance of BLOB retrieval using Fiddler 2 (record download times of largest files)

__ RDC into each WFE in The Farm with admin-level privileges

__ PERFORM IISRESET /stop /noforce

__ EXPLORE C:\inetpub\wwwroot\wss\virtualdirectories\

__ NAVIGATE into each web application sub-folder

__ FOR-EACH (web application) DO

LOCATE the web.config file

COPY web.config file to web.config_2010.03.18 (best practice);

OPEN web.config file in edit mode

FIND "<Blobcache" text string => should yield only a single element

EDIT <Blobkcache string as desired - particularly location, size & file types

SAVE web.config file

__ PERFORM IISRESET /start

__ RE-TEST performance

__ CHECK with Users (ultimate judges of success)


Must-Have: Object Caching…

• Located in the VRAM of each WFE (shares resources with BLOB Cache)• Caches

– Site Navigation Structure– Content in the different fields of a document library item or a list item– Published pages & resources– Draft pages

• Manifested to Users as good/bad navigation, queries and edits/saves• Enabled by default• Controls a single site collection at a time (challenging for some farms)• Defaults to 100 MB; ranges from 100 MB to 300 MB max per site collection• Shares both same memory space & same process as BLOB Cache• Need to be careful about over-allocating memory for this cache• Need to be REALLY careful about under-allocating memory for this cache


Risks/Benefits: Object Caching…

• Moderate Risk– OOTB & on-by-default in SP2007;– Only need to adjust Object Cache Size (MB); – Readily/Easily adjusted via Site Collection Admin Page– DO need to be careful about balancing shared server resources– OVER-allocating immediately & negatively impacts performance– UNDER-allocating can create "cache thrashing" – VERY BAD!– Annoying to manage for large site collection counts– Tuning techniques range from complex to simplistic (go for simple)– Often tuned manually, and only on high-use site collections !

• Moderate Benefits– Already working with default of 100 MB (per site collection)– Varying reduction of resource load times


Checklist: Object Caching…


__ START with all Site Collections all remaining at default Object Cache Size = 100 MB

__ MONITOR (via PerfMonitor Counter Log ) SharePoint Publishing Cache/Number of Compactions

low/zero compactions and good edit/save feedback from users = > probably leave as-is

high compactions and poor edit/save feedback from users => try increasing in 25 MB increments

NEVER reduce without having valid reason (thrashing horror stories abound)

__ DETERMINE (various means) all "high latency" site collections

FOR-EACH (site collection)

__ BROWSE to Site Actions > All Site Settings > Site Collection Admin > Site Collection Object Cache

__ CONFIRM current Object Cache Setting (100 MB default)

__ INCREASE value by 50 MB

__ SAVE & RESTART SERVER !! (or previous counts will not be reset)

__ RE-TEST performance (particularly navigation and edits/saves)

__ CHECK with Users (Ultimate judges of success)

OPTIONAL – leverage Performance Counter to quantify optimal settings (overkill compared to other effort)


Must-Have: Increased Browser-Server Connections…

Both IE7 and IE8 OOTB have fewer max server connections than they could have…

IE7 defaults to 2 server connections

IE8 defaults to 6 server connections

Microsoft provides a fix to increase this value to 10:

http://support.microsoft.com/default.aspx/kb/282402?p=1


Risks/Benefits: Increased Browser-Server Connections…

Low Risk:

Enhancement comes directly from Microsoft

Can be implemented by individual users or via SMS

Can be tuned/adjusted to optimize LAN bandwidth

Daring to go above 10 connections => perception of DNS !

High Benefits:

Increased # concurrent downloads => faster page loads

IE7 (2 connections): 10 sec page loads

IE7 (10 connections): 3 sec page loads


Checklist: Increased Browser-Server Connections…

__ BROWSE to http://support.microsoft.com/default.aspx/kb/282402?p=1

__ CLICK “Fix it” button

__ RE-TEST Performance against current baseline


Must-Have: Ethernet (not WiFi) Connections

Connection Type is not sexy, but in many environments this single apparently insignificant decision can eclipse all other attempts to improve SP2007 performance combined.

Example: 1. Your Ethernet connection (ideally) provides 25 Mb/s of download bandwidth; 2. Your pre-tuning performance gave 20 secs to download a 20 MB file; 3. Your Tunings improve this to 10 secs (a whopping 50% gain); BUT… 4. Your users like the convenience of WiFi, so they ignore your best practices; 5. Your users set up 75 ft from the nearest WAP, because they like the view; 6. At this location, their Effective Bandwidth is now 5 Mb/s for downloads; SO… 7. Downloading that same file now takes 50 secs, AND USERS COMPLAIN.


Also Investigate: Proxy Server Caching…

• Most proxy servers these days also provide static web content (output) caching – investigate this if your WFEs are memory-limited…


Also Investigate: Load Balancer Caching…

• Many current load balancers also provide static web content (output) caching. Investigate this as well if your WFEs are memory limited…


Nice-to-Have: Riverbed’s Steelhead CC Appliance & Client…

• Does Caching & Compression like other technologies• Differentiator is that it handles BOTH outbound AND inbound traffic


• Bolts-on to IIS• Configured per web app (handy)• Post-processes HTTP responses

– Packs js,css– Minifies & gzips resources– Sprite-ifies images

• Improves caching• Remote Page Times improved 65%• Local Page Times improved 37%• Used on SharePoint.microsoft.com

Nice-to-Have: Aptimize ISAPI Filter…


Next Steps…

__ SKETCH out your own environment's Content Delivery Chain, from Database to Browser

__ IDENTIFY all possible points of caching & compression

__ MEASURE baseline performance metrics (Content Download Rate, Content Upload Rate, Page Load Time, Edit/Save Time, others)

__ ASSESS current IIS (HTTP) Compression settings on each WFE

__ INVENTORY all commonly-used file types (e.g. doc, docx, xls, xlsx, ppt, pptx, pdf, etc)

__ PREPARE changes/modifications needed to the HTTP Compression settings

__ INVENTORY Disk Caching Settings on all WFEs

__ INVENTORY BLOB Caching Settings on all WFEs

__ INVENTORY Object Cache in “key” Site Collections (ugh)

__ IMPLEMENT improvements ONE AT A TIME

__ RE-TEST/MEASURE performance improvements & compare to baseline

__ CHECK WITH USERS (ultimate judges of success)


Conclusion…

• HOW TO REQUEST A COPY OF THIS PRESENTATION:– POSTED on CSPUG website

– EMAIL [email protected] or [email protected]

• VISIT MARK’S BLOG: – www.markvogt.us or www.thewiseconsultant.us

• OTHER AVAILABLE PRESENTATIONS: – PowerShell v2.0 for SharePoint 2007 Reporting, Policing & Administration – The Perfect SharePoint Server Farm– Governing SharePoint…with SharePoint– SharePoint-enabled Project Management Methodology

mailto:[email protected]

mailto:[email protected]

http://www.markvogt.us/


Discussion…

Q: WHAT experiences have others had with these techniques?

Q: WHAT other performance-relevant best practices can others share?

Q: WHAT other inbound technologies have others had success with?

Q: WHAT techniques have others developed for quickly/programmatically inventorying various caching settings?

Q: WHAT does Microsoft literature mean when it states that BLOB Caching 100 MB is "per site collection"? What are the best practices for environments with thousands of low-use site collections and users which demand good performance?

Q: What happens when you circumvent Central Admin, browse directly to the Output Cache URL, and set it. Can you get Output Caching for NON-publishing websites?


References/Credits…

• http://blogs.technet.com/wbaer/archive/2008/01/30/http-compression-internet-information-services-6-0-and-sharepoint-products-and-technologies.aspx

• http://blogs.msdn.com/joekr/archive/2008/09/04/enabling-http-compression-in-iis.aspx

• http://technet.microsoft.com/en-us/library/cc298466.aspx

• http://msdn.microsoft.com/en-us/library/bb727371.aspx

• http://blog.thekid.me.uk/archive/2007/07/17/speed-up-your-sharepoint-site-using-cache-control.aspx

© Grant Thornton LLP. All rights reserved. © 2010 Mark Vogt Ltd. All rights reserved or... "What...

Documents

Transcript of © Grant Thornton LLP. All rights reserved. © 2010 Mark Vogt Ltd. All rights reserved or... "What...