Magee Fixing App Performance2008
-
Upload
mrammaddali -
Category
Documents
-
view
9 -
download
0
description
Transcript of Magee Fixing App Performance2008
-
2008 MartinScott Consulting LLC. All rights reserved.
Fixing Application Performance Problems
Jamie MageeMartinScott Consulting
-
Who is Jamie Magee?
Irish Co-founder of MartinScott ConsultingFrequent speaker for Advisor, The View, Sphere
Developing Lotus apps since 1994Designing large apps so they scale wellSpeeding up slow applications
Creator of:NoteMan Toolbar for Developers/AdminsfA more powerful Document Properties and more!
WirelessMail for Domino mail to any device Blog: JamieMagee.com
-
3 things you need to know first
1. All the hardware in the world will not fix a poorly designed application Most performance issues are best resolved in the design of
the applications 2. HTTP server is a Web wrapper around the Domino
server Most Notes application performance tuning concepts still
apply on the Web3. Performance Tuning has trade-offs You can only have 2 out of 3f Good - (functionality)f Inexpensive - (development and maintenance effort)f Fast - (performance)
-
What Well Cover
Coding for performance Managing view indexing activity Web application performance techniques Configuring server settings for application performance Tracing performance problems Monitoring server resources Wrap-up
-
Getting a Handle to a View
NotesDatabase.GetView(viewName) Can be slow (e.g., 1+ second)! Fastest implementation is when parameter is the true view
name not the alias f Using correct case!
Using the view alias is easier to maintain, but slower Dont get the same view inside a loop
Tradeoff
LotusScript and JavaLotusScript and Java
-
Traversing a View or Doc Collection
When writing data to multiple documents, must use LotusScript/Java For traversing a NotesView:
f GetNthDocument(N) a little faster than GetFirstDoc/GetNextDoc
For traversing NotesDocumentCollection:f GetFirstDocument() with GetNextDocument(doc) much
faster than GetNthDocument() When only reading data from documents @DbLookup/@DbColumn is fastest
LotusScript and JavaLotusScript and Java
-
Reading Values on a Doc from a View
WORKSStatus = doc.Status(0)
BETTER!Status = doc.ColumnValues(2)
BETTER!Status = doc.ColumnValues(2)
10%Faster
Status is the third column of the view that was used to get to this document
Status is the third column of the view that was used to get to this document
LotusScript and JavaLotusScript and Java
-
Looping
WORKSDo
z=z+1y=y+1
Loop While y
-
Put the Smaller Bounds on the Outer Loop
WORKSfor q=0 to 5000for y=0 to 2
z=z+x(q,y)next
next
BETTER!for y=0 to 2for q=0 to 5000
z=z+x(q,y)next
next
BETTER!BETTER!for y=0 to 2for y=0 to 2for q=0 to 5000for q=0 to 5000
z=z=z+x(q,yz+x(q,y))nextnext
nextnext
400%Faster
LotusScriptLotusScript
-
Branching with IF Statements
WORKSif a=x and totalPrice()>0
(has to compute totalPriceeven if a x)
BETTER!if a=x then if totalPrice()>0
(only computes totalPrice if a=x)
BETTER!if a=x then if totalPrice()>0
(only computes totalPrice if a=x)
LotusScriptLotusScript
-
When Doc is Open in the Notes UI
WORKS BETTER!
Updates ui doc only once, after all fields are written
BETTER!
Updates ui doc only once, after all fields are writtenLotusScriptLotusScript
5xFaster
-
Demonstration: Autoreload Property
-
Profile Docs for Config Data
Profile Documents are cached Built-in programmatic access: Formula: GetProfileField() LotusScript/Java: NotesDatabase.GetProfileDocument()
But how do you manually access profile docs? End users: Profile Docs (open source) form from Andre
Guirards blog at www.ibm.comf Shows profile docs in the parent db containing the form
Geeks: NoteMan Toolbar (free) shows view of profile docs f Shows profile docs in any database
-
Demonstration: Profile document management
-
(See Designer Help Properties that improve database performance)
Miscellaneous Database Properties
-
Shared Elements in Notes Client
Shared fields/actions, subforms, script libraries stored in separate notes
Domino must make one call to get the document, one call to get the form, and one call for each shared element 10 shared fields = 10 extra design element fetches More than three computed subforms can have an exponential
effect on performance Shared elements are great for design maintenance but
use with caution in Notes Client!
Tradeoff
-
Shared Elements on the Web
In web applications, requesting and assembling of shared elements is done right on the server, without network activity No real performance impact go for it!
-
Quiz
Question: Whats the biggest Domino performance problem?
Answer: View indexing
-
What Well Cover
Coding for performance Managing view indexing activity Web application performance techniques Configuring server settings for application performance Tracing performance problems Monitoring server resources Wrap-up
-
View Indexing
The top performance killer in Domino isView Indexing Every time a document is saved, Domino updates the
indexes of all auto-indexed views which select the document Including Notes form, web submit, and agent updates to a
document Indexing performance is affected by
f Number of documents in viewsf Number of fieldsf Frequency of updatef Number of viewsf Complexity of views
-
View Indexing tactics
Use a separate physical disk for view indexing aid (VIEW_REBUILD_DIR notes.ini variable)
Delete unused viewsf Determine which views are really being used
-
How to know which views are being used?
Use the Admin Clienta. Purge suspect indexesb. Allow user activity to resume for a week/monthc. Later, look for re-created indexes (they are active views)d. Remove unused views (Size=0) from design
-
Views: Time @Functions
Avoid @Today, @Now in view formulas Forces the calculation to be done every time the view
is opened H o w m a n y s e c o n d s d o e s it ta k e to In d e x a V ie w ?
T he se d a ta ba se s co n ta in o n ly 1 V ie w a nd 1 F o rmT he V ie w ha s o n ly o ne co lum n , a nd the S e le c tio n F o rm u la is @ A ll
1 0 1 0 0 1 0 0 00
5
1 0
1 5
2 0
2 5
3 0
3 5
N u m b e r o f D o c u m e n ts
S e c o n d s
F ie ld N a m e@ To d a y@ N o w
-
Views: Time @Functions (continued)
To show a view of documents more than N days old: A simple daily scheduled agent moves docs in/out of the old
documents folder (do not use a view)f Putting a document in a folder does not modify it --
minimizes replication activity and conflicts Or, daily agent modifies hard-coded date in view formula
f NotesView.SelectionFormula Or, reduce view index refresh interval
-
View Complexity
A p p ly in g D iffe re n t A ttr ib u te s to V ie w C o lu m n sN u m b e r o f M in u te s to B u ild a V ie w w ith 5 C o lu m n s
1 0 0 F ie ld s1 0 0 ,00 0 D o cu m e n ts
0
5
1 0
1 5
2 0
2 5M in u te s
P L A INF O R M U L A S (5 )S O R T (1 )S O R T (4 )C AT E G O R IZ E (1 )C AT E G O R IZ E (4 )
-
Views: User Sortable Columns
Count as viewsEach sort option has its own view index
WarningWarning
-
Views: Readers Restrictions
Readers restrictions can cause slow view display if ... User has Read access to only a small portion of many docs Domino must scan until it finds a pages worth (usually 30) of
accessible docs, or all the way until the end of the view!
WarningWarning
-
What Well Cover
Coding for performance Managing view indexing activity Web application performance techniques Configuring server settings for application performance Tracing performance problems Monitoring server resources Wrap-up
-
Should I use WebQueryOpen?
R6, R7 and R8 have shown tremendous improvement regarding Open Agent performance. The horror stories have come from pre R6 days. Use R7 Agent Profiler to probe these agents.fMore on this later
WebQueryOpen LotusScript agents not quite as fast as, but offer more flexible programming functionality than, formulas.
-
WindowOnload Problem
Developer uses the onload event to start Web apps on page loads The problem: Onload does not fire until all page content has
been loaded, including images If you have any scripts that depend only on the DOM to be
loaded, then use a domReady.js as the kick offf Get your copy of js_domready.js on
DominoPerformance.com thanks to Dwight Wilbanks
function FormOnLoad( ) {~ your processes }
var FormLoader = new domFunction( FormOnLoad,"")
-
Use Cascading Style Sheets (CSS)
Separates your style and data Smaller HTML pages = faster loading CSS, JS files are cached in the browser Make your CSS efficient and standardized
-
Domino and Doctypes
By default domino throws out an incomplete doctype 6.5.3 notes.ini parameter sets the doc type server wide:fDominoCompleteDoctype=n 0 = Domino default 1 = Standard 2 = Strict R7 - a field $$HTMLFrontMatter sets the doctype per page
-
Why Doctypes are important
With the default or incomplete Doctype the page runs in Quirks Mode This has extra parsing for older HTML This slows down the document parsing which slows the
page display The right choice is to run in Standards Mode. This expects well formed HTML and XHTML This displays the page fasterfNot noticed on the server, but faster appearance to
end user
-
Use Server Web Site Rule to maximize file caching
Drastically improve your users Web Cache The users browser will cache many files, but it still needs
to check to see if it has the current version each timefThis can cause 2-3 extra seconds to each page load.
To fix this we need to tell the users browser that certain files are good for N days
Admin needs to set it but Developer needs to code it
-
Add Web Site Rule
Items in:
Domain//cachePath/
Will be cached
If-Modified-Since header variable
-
GZip Will Improve Performance
GZip (GNU Zip) is a common compression found on many Web Servers such as Apache, IIS,..
First discovered on Domino in R6.5x but only in DWA There have been many attempts to use this built in
GZip functionality but most have been reported as highly unstable.
Which brings us to
-
Use Web Site Rules to use GZip
Because browsers know about GZip, we can feed them GZip files We can do this for any external files (CSS, JS, etc.) To do this, make a web site rule to handle these items
and place them in a gz directory path How to use GZip
1. Make the Web Site Rule2. Compress your files in GZip format (www.gnu.org)3. Add these files in shared resources with a gz/ prefix4. Reference these files when needed
-
Add Web Site Rule for GZip
Items in:
Domain//gz/
Will be served as GZip
-
What Well Cover
Coding for performance Managing view indexing activity Web application performance techniques Configuring server settings for application performance Tracing performance problems Monitoring server resources Wrap-up
-
Server Document: Maximum Concurrent Agents
For agent-intensive applications Allow more than one agent to run at the same time
Monitor your resource utilization after such a change to ensure your system has the CPU and memory to deal with increased activity
-
Server Document: Web Maximum Concurrent Agents
Allow more than one agent to run at the same time on the Web
Enabled To allow more than one agent to run on the Web server at the same time (asynchronously) Disabled (default) To run only one agent at a time (serially)
-
NOTES.INI Application Performance Variables
VIEW_REBUILD_DIR= Use a separate physical disk for view indexing
f Greatly improves overall performance on index-heavy servers
Updaters = [number] Run multiple update tasks to keep view indexes updated Set [number] = number of processors
NSF DbCache_Maxentries = [number] Number of databases that can be cached at one time Default is 25 or NSF_Buffer_Pool_Size divided by 300 KB If ratio of Database.DbCache.Hits to InitialDbOpen is low, then
consider increasing [number] Console command Dbcache disable closes cached databases
-
Monitoring Transactions vs. Users
Server_Show_Performance=1 in notes.ini Post to the server console every 60 seconds Number of transactions the server performs per minute and the
current number of active users on the system Helpful in tracking system performance over a period of time Transactions include router, agent manager, replication, virus
scanner, and user actions
[01A8:0021-0254] 04/10/2008 04:30:47 PM 6 Transactions/Minute, 0 Notes Users[01A8:0021-0254] 04/10/2008 04:31:47 PM 0 Transactions/Minute, 0 Notes Users[01A8:0021-0254] 04/10/2008 04:32:47 PM 4 Transactions/Minute, 0 Notes Users
-
Configuring Server Settings
Administrators own these settings, but developers must be aware of them
Both parties need to collaborate to determine best settings for each server
Make sure to monitor resource utilization before and after making such changes so that you can determine if you have enough hardware to benefit (and not make it worse!)
-
What Well Cover
Coding for performance Managing view indexing activity Web application performance techniques Configuring server settings for application performance Tracing performance problems Monitoring server resources Wrap-up
-
Performance Debugging: Users perspective indications
Large, continuous tables on formsExcessive DbLookup/DbColumn operationsInefficient WebQueryOpen/WebQueryCloseLotusScript events
Creating/opening/saving document
Time functions in view formulasLarge active views with broad selection formulasExcessive view categorization
Saving a document (submitting from a browser, agent-based saves of other docs)
Time functions in view formulasReaders restrictions on documentsIndex property not set to Automatic
Opening a view
Possible ProblemPerformance Symptom
-
DDM (R7 and later)
Domino Domain Monitor contains super agent statistical tools What agent is causing problems? Time to run, Memory used
Wrapped in a Domino DB for easy reporting
-
Agent Profiler (R7 and later)
Profile an Agent to get detailed performance information Step 1 mark the agent to be profiled Step 2 Run the agent Step 3 View
-
Agent Profiler Sample Output
-
Fiddler Example Output
-
Undocumented Notes Feature: Watching NRPCs
Watch behind the scenes querying of forms, subforms, views (not just code) Remote Procedure Calls (RPCs) = Notes-to-Domino talk See http://MartinScott.com Resources Technical Articles
Notes Client Domino Server
-
Demo: Watching Notes RPCs
-
Undocumented Notes Feature: Uncover Performance Bottlenecks
Enable RPC watching on Notes client NOTES.INI file on Notes client/Designer
f Client_Clock=1f Debug_Console=1f Debug_Outfile= (optional)
Restart Notes client Remove from NOTES.INI and restart to disable RPC watching
-
(1-81) OPEN_DB:(2-83) OPEN_COLLECTION: 429 ms. [32+626=658](3-83) READ_ENTRIES: 8354 ms. [46+64432=64478](4-91) CLOSE_COLLECTION: 0 ms. [12+0=12](5-92) DB_REPLINFO_GET: 438 ms. [14+32=46](6-92) DB_REPLINFO_GET: 169 ms. [14+32=46]...
Watching Notes RPCs Opening my mailWatching Notes RPCs Opening my mail
-
Understanding RPCs
Common RPCs and What They Indicate
(20-121) OPEN_NOTE: 1045 ms. [28+3906=3934]
(Seq. #), RPC_NAME, time, [bytes_sent+bytes_received=total]
RPC_Name DescriptionOpen_Session Authenticate with the server and establish a sessionOpen_Database Find and open a database
Open_Note Get the contents of a note (data document, design element, or ACL info)
Update_Note Save a document Open_Collection Open a viewRead_Entries Get a list of document information from a view or search
Find_By_Key Find a document via DBLookup or LotusScript GetDocumentByKey
Get_Special_Note_ID Get ACL informationClose_dB Close database session
-
(1-14 [1]) OPEN_DB(CN=Vienna01/O=MSC!!mail\jmagee2.nsf): (Connect to Vienna01/MSC: 0 ms) (Exchnames: 0 ms) (OPEN_SESSION: 561 ms)(2-16 [3]) GET_UNREAD_NOTE_TABLE: 571 ms. [134+290=424](2-17 [4]) DBGETREPLICAMATCHES: 1692 ms. [290+17452=17742](3-17 [5]) OPEN_NOTE(REP85257246:005DD81C-NTFFFF0010,03000400): 841 ms. [176+780=956](3-18 [6]) OPEN_DB(CN=Vienna01/O=MSC!!mail\jmagee2.nsf): 621 ms. [48+1886=1934](4-18 [7]) GET_NAMED_OBJECT_ID($profile_015calendarprofile_): 641 ms. [134+290=424](4-18 [8]) READ REPLICATION HISTORY: 541 ms. [54+24=78](5-18 [9]) OPEN_NOTE(REP85257246:005DD81C-NT00002FE6,00400020): (6-19 [10]) POLL_DEL_SEQNUM: 571 ms. [60+64=124] (Cache entry not found)(5-19 [10]) DB_REPLINFO_GET: 821 ms. [14+32=46]1342 ms. [48+9050=9098](7-20 [11]) GET_NAMED_OBJECT_ID($profile_024archive database profile_): (6-20 [12]) SEARCH: (Connect to Vienna01/MSC: 1382 ms) 581 ms. [64+24=88](8-20 [13]) OPEN_NOTE(REP85257246:005DD81C-NT000010F2,00400020): (OPEN_SESSION: 531 ms)520 ms. [48+214=262](9-21 [15]) OPEN_COLLECTION(REP85257246:005DD81C-NT0000073E,0040,4008): 1292 ms. [70+782=852](7-21 [16]) OPEN_NOTE(REP85257246:005DD81C-NTFFFF0040,03000400): 551 ms. [110+28=138]1061 ms. [48+1534=1582]
Watching Notes RPCsWatching Notes RPCs
-
How to see what the NoteIDs are
Use a free tool like NoteMan to look up the NoteID00002FE6
This item is a Calendar Profile doc, and took .571 seconds of my DB open time.
(5-18 [9]) OPEN_NOTE(REP85257246:005DD81C-NT00002FE6,00400020): (6-19 [10]) POLL_DEL_SEQNUM: 571 ms.
-
NRPC Parser open source tool
Runs on NRPC output files to translate IDs and commandsf Much easier to find and understand problems
Developed by Jamie Magee and Andrew Magerman Download from www.JamieMagee.com or www.Magerman.com
-
What Well Cover
Coding for performance Managing view indexing activity Web application performance techniques Configuring server settings for application performance Tracing performance problems Monitoring server resources Wrap-up
-
Resource Monitoring
Use operating system tools PerfMon (Windows)
f Start \ Control Panel \ Administrative Tools \ Performance PerfMeter (UNIX) PerfMon / PEX (AS/400)
Use during live use to see bottlenecks
-
Resource Monitoring
Example: agent is bound by CPU, more RAM would not helpLimited by Processor
-
Resource Monitoring (continued)
Example: View index rebuild showing that more RAM and faster drive would help, but more CPU would not.
-
Resource Monitoring: Identifying areas to improve
HighMedium
Increase server RAMOptimize .INI file memory settings
Memory
HighMedium
Increase network capacityTake advantage of browser caching
Network
HighHighMediumMediumMedium
Reduce view complexity, # and sizeOptimize code implementationOptimize server .INI file settingsDisable server screen saverTake advantage of browser caching
Processor
HighMediumLow
Reduce view complexity, # and size Optimize cache settings in .INI fileTake advantage of browser caching
Disk Access
Improvement Potential
Area of FocusLimiting Resource
-
What Well Cover
Coding for performance Managing view indexing activity Web application performance techniques Configuring server settings for application performance Tracing performance problems Monitoring server resources Wrap-up
-
Resources References
Lotus developerWorks performance zone: www.ibm.com/developerworks/lotus/performance
f Index to all IBM performance articles www.redbooks.ibm.com Any Domino book with Performance in the name Several other good ones
MartinScott.com 30-page performance article: http://MartinScott.com Resources Technical Articles
NoteMan Toolbar www.NoteMan.com Andre Guirards blog: Best Practice Makes Perfect http://www-10.lotus.com/ldd/bpmpblog.nsf
Some ideas from Kevin Marshall and Andrew Magerman www.FastNSF.com Server performance tips
-
Key Points to Take Home
Beware of code and design gotchas! Refine and minimize view indexing, remove unused
views Employ web standards and take advantage of fast web
service techniques Developers and admins should collaborate on server
configuration settings DDM, Agent Profiling, and NRPC monitoring can tell you
a lot about your application code Resource monitoring reveals the pulse of the server,
and indicates hardware needs
-
Your Turn!
Questions?
How to contact me:JMagee at MartinScott.com
/ColorImageDict > /JPEG2000ColorACSImageDict > /JPEG2000ColorImageDict > /AntiAliasGrayImages false /DownsampleGrayImages true /GrayImageDownsampleType /Bicubic /GrayImageResolution 300 /GrayImageDepth -1 /GrayImageDownsampleThreshold 1.50000 /EncodeGrayImages true /GrayImageFilter /DCTEncode /AutoFilterGrayImages true /GrayImageAutoFilterStrategy /JPEG /GrayACSImageDict > /GrayImageDict > /JPEG2000GrayACSImageDict > /JPEG2000GrayImageDict > /AntiAliasMonoImages false /DownsampleMonoImages true /MonoImageDownsampleType /Bicubic /MonoImageResolution 1200 /MonoImageDepth -1 /MonoImageDownsampleThreshold 1.50000 /EncodeMonoImages true /MonoImageFilter /CCITTFaxEncode /MonoImageDict > /AllowPSXObjects false /PDFX1aCheck false /PDFX3Check false /PDFXCompliantPDFOnly false /PDFXNoTrimBoxError true /PDFXTrimBoxToMediaBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXSetBleedBoxToMediaBox true /PDFXBleedBoxToTrimBoxOffset [ 0.00000 0.00000 0.00000 0.00000 ] /PDFXOutputIntentProfile () /PDFXOutputCondition () /PDFXRegistryName (http://www.color.org) /PDFXTrapped /Unknown
/Description >>> setdistillerparams> setpagedevice