ArcGIS Server Performance and Scalability—Performance Factors … · –Storage type: ESRI...
Transcript of ArcGIS Server Performance and Scalability—Performance Factors … · –Storage type: ESRI...
ArcGISArcGIS Server Performance and Server Performance and ScalabilityScalability——Performance Factors Performance Factors and Optimizationand Optimizationand Optimizationand Optimization
David PlemonsDavid PlemonsRandall WilliamsRandall WilliamsRandall WilliamsRandall Williams
SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop 11
IntroductionsIntroductions•• Who are we?Who are we?
–– ESRI Support Services, Charlotte Support CenterESRI Support Services, Charlotte Support CenterS I l t ti S US I l t ti S U–– Server Implementation, Server Usage Server Implementation, Server Usage
•• Assumptions about you, our audienceAssumptions about you, our audience–– Familiar with Familiar with ArcGISArcGIS DesktopDesktop–– Basic knowledge of ArcGIS Server architecture and conceptsBasic knowledge of ArcGIS Server architecture and concepts–– Comfortable with Web architecturesComfortable with Web architectures
SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop 22
Performance Factors in Performance Factors in ArcGISArcGIS Server System ComponentsServer System Components
M hi hit tM hi hit t•• Machine architectureMachine architecture
•• Data sourcesData sources
•• ArcGIS Server frameworkArcGIS Server framework
•• ArcGISArcGIS Server servicesServer services
Cli t li ti d th ADFCli t li ti d th ADF•• Client applications and the ADFClient applications and the ADF
•• GIS architectureGIS architecture
•• Question and answer periodQuestion and answer period
G l Sh h t ti i th t t d liG l Sh h t ti i th t t d li
33
Goal: Show how to optimize these components to deliver Goal: Show how to optimize these components to deliver highly performing highly performing ArcGISArcGIS Server deployments.Server deployments.
SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Machine ArchitecturePerformance Factors: Machine ArchitectureSelect adequate hardware to support desired performance and load.Select adequate hardware to support desired performance and load.q pp pq pp p
•• PerformancePerformance–– CPU speedCPU speed
•• Published benchmarks: http://www spec org/cpu2006/results/cint2006 htmlPublished benchmarks: http://www spec org/cpu2006/results/cint2006 html•• Published benchmarks: http://www.spec.org/cpu2006/results/cint2006.htmlPublished benchmarks: http://www.spec.org/cpu2006/results/cint2006.html•• Mapping: Select CPU with the highest baseline CINT rate/coreMapping: Select CPU with the highest baseline CINT rate/core•• GP: Select CPU with highest baseline CFP rate/coreGP: Select CPU with highest baseline CFP rate/core
–– Disk speedDisk speedDisk speedDisk speed•• 15K RPM SAS disks15K RPM SAS disks
–– Network latencyNetwork latency•• Fast switchesFast switches
–– Memory bus speedMemory bus speed
•• CapacityCapacityp yp y–– # CPU cores# CPU cores–– Network bandwidthNetwork bandwidth
•• Gigabit switches and NICs in Full Duplex modeGigabit switches and NICs in Full Duplex modeg pg p–– # Disks and controllers# Disks and controllers–– Amount of RAMAmount of RAM
44SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Data SourcesPerformance Factors: Data SourcesS l t d t th t id ti l fS l t d t th t id ti l fSelect data sources that provide optimal performance.Select data sources that provide optimal performance.
•• VectorVector–– Data provider: DBMS, File GDB, Shapefile, SDCData provider: DBMS, File GDB, Shapefile, SDC–– Storage type: ESRI Storage type: ESRI ST_GeometryST_Geometry, Oracle SDO, SQL Server Geometry, Oracle SDO, SQL Server Geometry
•• RasterRaster–– Data provider: Image Server, File GDB, DBMSData provider: Image Server, File GDB, DBMS–– Compression: LZ77, JPEG, JPEG2000Compression: LZ77, JPEG, JPEG2000
55SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
MIME vs. URL (Service Configuration)MIME vs. URL (Service Configuration)( g )( g )
•• MIME responses are streamed directly to the client (push).MIME responses are streamed directly to the client (push).
•• URLURL--based responses generate data on the server machine based responses generate data on the server machine and are accessed by the client via URL (pull).and are accessed by the client via URL (pull).and are accessed by the client via URL (pull).and are accessed by the client via URL (pull).
UC2009 Technical WorkshopUC2009 Technical Workshop 66
MIME vs. URL (Service Configuration)MIME vs. URL (Service Configuration)
–– For serving maps, MIME can scale better than URL.For serving maps, MIME can scale better than URL.•• Disk and UNC shares are often bottlenecks before network bandwidthDisk and UNC shares are often bottlenecks before network bandwidth•• Disk and UNC shares are often bottlenecks before network bandwidth.Disk and UNC shares are often bottlenecks before network bandwidth.
–– MIME performs better with smaller images.MIME performs better with smaller images.
MIME URL
140000160000180000
Hr)
MIME vs. URL
80000100000120000140000
hput
(Tr/H
High Complexity Map (large images)
200004000060000
Thro
ug Low Complexity Map (small images)
77
0FGDB_Local_MIME FGDB_Local_URL
SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Data SourcesPerformance Factors: Data SourcesRepresentative comparison of throughput variances based on service Representative comparison of throughput variances based on service source datasource data
Low Complexity Map: Throughput vs. data source
100000
120000
140000
)
60000
80000
100000
hrou
ghpu
t (Tr
/Hr)
0
20000
40000Th
FGDB_Local_URL SHP_Local_URL Ora11g_AS_URL SQLSvr_AS_URL Postgres_AS_URL
Machine Specs: Four 2.5 GHz dual-core CPUs, 8 GB RAM
88SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Data SourcesPerformance Factors: Data Sources
•• Local vs. UNCLocal vs. UNC–– Locally sourced data always results in better throughput.Locally sourced data always results in better throughput.–– Inherent latency with UNC protocolInherent latency with UNC protocolInherent latency with UNC protocol.Inherent latency with UNC protocol.
99SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Data SourcesPerformance Factors: Data SourcesOptimize DBMS configuration and conduct maintenance.Optimize DBMS configuration and conduct maintenance.
•• DBMS configurationDBMS configuration
•• Create and maintain (rebuild) attribute indexesCreate and maintain (rebuild) attribute indexes
•• Updating DBMS statisticsUpdating DBMS statistics
•• Versioning managementVersioning management–– Reconcile and postReconcile and post–– CompressCompress
1010
NonoptimalNonoptimal DBMS may be a source of significant performance DBMS may be a source of significant performance degradation.degradation.SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkSOMSOM
•• 165 map draw requests/second per core165 map draw requests/second per core•• 60% CPU utilization on SOM machine60% CPU utilization on SOM machine•• 0.027 seconds average response time0.027 seconds average response time
1111SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkWeb Service ComponentsWeb Service Components
•• Web service handlers (MIME)Web service handlers (MIME)–– Easily overloaded by MIME data (REST/SOAP returning MIME)Easily overloaded by MIME data (REST/SOAP returning MIME)
•• Network saturationNetwork saturation–– 72 cores can approach 40% gigabit utilization72 cores can approach 40% gigabit utilization
N t k I/O (f llN t k I/O (f ll d l )d l )–– Network I/O (fullNetwork I/O (full--duplex)duplex)•• CPU utilizationCPU utilization
–– Use multiple Web instancesUse multiple Web instancesUse multiple Web instances.Use multiple Web instances.
Vi t l di t i (URL)Vi t l di t i (URL)•• Virtual directories (URL)Virtual directories (URL)–– Virtual output directory bottlenecksVirtual output directory bottlenecks
•• Network saturationNetwork saturationDisk performanceDisk performance•• Disk performanceDisk performance
–– Use multiple output directories on different machines.Use multiple output directories on different machines.1212SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Performance Factors: ArcGISArcGIS Server FrameworkServer FrameworkWeb ServicesWeb Services
•• Security Security –– Each mechanism contributes to CPU overhead.Each mechanism contributes to CPU overhead.–– SSLSSL
–– LSASSLSASS
Problem: Problem: On Windows 2003 Server, the Local Security Authority On Windows 2003 Server, the Local Security Authority Subsystem Service (lsass.exe) grows in CPU usage and Subsystem Service (lsass.exe) grows in CPU usage and memory utilization under heavy load.memory utilization under heavy load.
•• http://resources.arcgis.com/content/kbase?fa=articleShow&d=32620http://resources.arcgis.com/content/kbase?fa=articleShow&d=32620–– Web Tier SecurityWeb Tier Security
U /R l StU /R l St•• User/Role StoresUser/Role Stores•• Token ServerToken Server
1313SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server FrameworkPerformance Factors: ArcGIS Server Framework•• Optimal number of instances does vary significantly by complexity of Optimal number of instances does vary significantly by complexity of
service (complex map or GP model) and choice of source data type.service (complex map or GP model) and choice of source data type.•• Varies in practice from 1 to 4 highVaries in practice from 1 to 4 high--isolation instances per CPU coreisolation instances per CPU coreVaries in practice from 1 to 4 highVaries in practice from 1 to 4 high isolation instances per CPU coreisolation instances per CPU core
3.5
ut
Variance of Service Instances by Source Data Type
2
2.5
3
at M
ax T
hrou
ghpu
1
1.5
2
s pe
r CPU
cor
e a
0
0.5
FGDB_Local FGDB_UNC SHP_Local SHP_UNC Ora11g_AS Ora11g_DC SQLSvr_AS SQLSvr_DC
Inst
ance
s
1414SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server ServicesPerformance Factors: ArcGIS Server ServicesMap ServiceMap Service
MXDMXD–– Optimizing map symbols for performanceOptimizing map symbols for performancep g p y pp g p y p
–– Spatial index Spatial index
–– Avoid projecting on the fly.Avoid projecting on the fly.
–– Optimizing map text and labels for performanceOptimizing map text and labels for performancep g p pp g p p•• Use annotations.Use annotations.•• Avoid using the Maplex Engine with ArcGIS Server.Avoid using the Maplex Engine with ArcGIS Server.
–– Use spatial joins or maintain data in a single DB (no crossUse spatial joins or maintain data in a single DB (no cross--DB joins).DB joins).
–– Avoid wavelet compressionAvoid wavelet compression--based raster types (based raster types (MrSidMrSid, JPEG 2000)., JPEG 2000).
1515SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server ServicesPerformance Factors: ArcGIS Server ServicesMap ServiceMap Service
•• MXDMXD–– Simplify dataSimplify data
–– Set scale dependencySet scale dependency
•• Tuning ToolsTuning Tools–– ArcMap 9.3.1 ArcMap 9.3.1 AnalyzeAnalyzerr (see next two slides (see next two slides –– “Analyze” or “Analyzer” “Analyze” or “Analyzer”
?)?) Tool Tool –– Derived from MXDPERFSTAT (ArcScripts)Derived from MXDPERFSTAT (ArcScripts)
1616SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server Services Performance Factors: ArcGIS Server Services Map Service Map Service
Optimizing mxd with new ArcMap 9.3.1 Optimizing mxd with new ArcMap 9.3.1 AnalyzeAnalyze (Analyzer?) (Analyzer?) tooltoolp g pp g p yy ( y )( y )
1717SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server Services Performance Factors: ArcGIS Server Services Map ServiceMap Service
•• Optimizing MXDs with the ArcMap 9.3.1 Optimizing MXDs with the ArcMap 9.3.1 Analyze (or Analyze (or Analyzer?) Analyzer?) Tool: DemoTool: Demo
1818SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server Services (Map Service)Performance Factors: ArcGIS Server Services (Map Service)UnoptimizedUnoptimized vs. Optimized Source vs. Optimized Source ArcMapArcMap DocumentDocument
ions
/Hr)
65800
ut (T
rans
acti Optimized
(Unloaded TT: 0.34 sec)(2.1 Instances/core)
2520Thro
ughp
u
Unoptimized(Unloaded TT: 11.97 sec)
(1.6 Instances/core)
13 17
1919
Number of InstancesSERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server ServicesPerformance Factors: ArcGIS Server ServicesMap Service Map Service
•• Cache basemap.Cache basemap.
•• Display the operational layer as a cached map service.Display the operational layer as a cached map service.–– Even if your operational dataset changes frequentlyEven if your operational dataset changes frequentlyEven if your operational dataset changes frequentlyEven if your operational dataset changes frequently
•• Display the operational layer as clientDisplay the operational layer as client--side graphics.side graphics.Do not fetch large numbers of featuresDo not fetch large numbers of features–– Do not fetch large numbers of features.Do not fetch large numbers of features.
•• Can take a long time to stream features to clientCan take a long time to stream features to client•• Can take a long time to render in clientCan take a long time to render in client
2020SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server Services Performance Factors: ArcGIS Server Services GP ServiceGP Service
•• Use Local Jobs Directory Use Local Jobs Directory ––Greatest single performance factorGreatest single performance factor––9.3.1 allows simple deployment9.3.1 allows simple deployment
•• Subset very large Subset very large rastersrasters
•• Use native types like GRID.Use native types like GRID.
U l f d t i t d f d t f DBU l f d t i t d f d t f DB•• Use layers from a map document instead of raw data from DB.Use layers from a map document instead of raw data from DB.
•• Write intermediary data to memory instead of to disk.Write intermediary data to memory instead of to disk.
2121SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: ArcGIS Server Services Performance Factors: ArcGIS Server Services GP ServiceGP Service
•• PrecomputePrecompute intermediate steps when possible.intermediate steps when possible.
Use local paths to data and resourcesUse local paths to data and resources•• Use local paths to data and resources.Use local paths to data and resources.
•• Avoid unneeded coordinate transformations.Avoid unneeded coordinate transformations.
•• Add attribute indexes if selecting by attribute queries (add Add attribute indexes if selecting by attribute queries (add attribute index tool).attribute index tool).
•• Simplify data.Simplify data.
•• Consider a Geometry service (ArcGIS Online) or an SOE.Consider a Geometry service (ArcGIS Online) or an SOE.Detailed instructions on the Detailed instructions on the Resource CenterResource Center atathttp://webhelp esri com/arcgisdesktop/9 3/index cfm?id=1061&pid=1044&tohttp://webhelp esri com/arcgisdesktop/9 3/index cfm?id=1061&pid=1044&tohttp://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?id=1061&pid=1044&tohttp://webhelp.esri.com/arcgisdesktop/9.3/index.cfm?id=1061&pid=1044&to
picname=Performance_tips_for_geoprocessing_servicespicname=Performance_tips_for_geoprocessing_services2222SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services Image ServiceImage Service
•• JPEGJPEG--compressed TIFF the best performer by far (10compressed TIFF the best performer by far (10--100x 100x faster)faster)
•• TiledTiled rastersrasters even faster than single TIFFeven faster than single TIFFTiledTiled rastersrasters even faster than single TIFFeven faster than single TIFF
2323SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services GeocodeGeocode and Globe Servicesand Globe Services
•• GeocodeGeocode ServicesServices–– Use local locator files instead of UNC locator files for single Use local locator files instead of UNC locator files for single
address or batch address or batch geocodinggeocoding..–– Locators based on ArcSDE are OK, but local is fasterLocators based on ArcSDE are OK, but local is faster
•• Globe ServicesGlobe Services–– Use Use billboardedbillboarded labelslabels–– PrecachePrecache the globe servicethe globe service
2424SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services Mobile ServiceMobile Service
•• Document PreparationDocument Preparation–– Minimize operational layers.Minimize operational layers.–– Cache basemap layers.Cache basemap layers.
•• Service ConfigurationService Configuration–– Try to keep total service cache size under 250 MB to avoid network Try to keep total service cache size under 250 MB to avoid network
swapping. swapping. –– IIS 6 upload/download size limits (200 KB upload/4 MB download) IIS 6 upload/download size limits (200 KB upload/4 MB download)
http://www.banmanpro.com/support2/File_Upload_limits.asphttp://www.banmanpro.com/support2/File_Upload_limits.asp
•• Usage ConsiderationsUsage Considerations•• Usage ConsiderationsUsage Considerations–– Avoid batch postings in favor of frequent updates.Avoid batch postings in favor of frequent updates.
2525SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services GeodataGeodata ServiceService
•• Data PreparationData Preparation–– Perform regular version maintenance (keep versioning tree Perform regular version maintenance (keep versioning tree
small, compress, schedule synchronizations, etc.) small, compress, schedule synchronizations, etc.) p y )p y )http://blogs.esri.com/Dev/blogs/geodatabase/archive/2008/11/25/Geodahttp://blogs.esri.com/Dev/blogs/geodatabase/archive/2008/11/25/Geodatabasetabase--replicationreplication--andand--compress.aspxcompress.aspx
–– Use 1 way replicas over 2 way replicas when possible.Use 1 way replicas over 2 way replicas when possible.
–– Consider 2 way replicas instead of checkout replicas.Consider 2 way replicas instead of checkout replicas.
–– WellWell--defined data modeldefined data model
2626SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Performance Factors: ArcGISArcGIS Server Services Server Services GeodataGeodata ServiceServiceGeodataGeodata ServiceService
•• Service ConfigurationService Configuration–– Server object usage timeout (set larger than 10 min default)Server object usage timeout (set larger than 10 min default)U l d/D l d d f lt IIS i li it (200 KB l d/4 MB d l d)U l d/D l d d f lt IIS i li it (200 KB l d/4 MB d l d)Upload/Download default IIS size limits (200 KB upload/4 MB download)Upload/Download default IIS size limits (200 KB upload/4 MB download)
•• Usage ConsiderationsUsage Considerations–– Client HTTP timeout Client HTTP timeout Error: Creating or synchronizing a replica with a geodata service over Error: Creating or synchronizing a replica with a geodata service over
the Internet generates errors when there are many changes or large the Internet generates errors when there are many changes or large volumes of datavolumes of datavolumes of data.volumes of data.
–– http://resources.arcgis.com/content/kbase?fa=articleShow&d=35971http://resources.arcgis.com/content/kbase?fa=articleShow&d=35971
Error: Create Replica failed The server took too long to answer TheError: Create Replica failed The server took too long to answer TheError: Create Replica failed. The server took too long to answer. The Error: Create Replica failed. The server took too long to answer. The client has timed out.client has timed out.
–– http://resources.arcgis.com/content/kbase?fa=articleShow&d=35972http://resources.arcgis.com/content/kbase?fa=articleShow&d=35972
2727SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Performance Factors: Client Applications and the Performance Factors: Client Applications and the ADFADF
•• Meet functional requirementsMeet functional requirements•• Ensure highest performance and scalabilityEnsure highest performance and scalability•• Leverage existing skills and standardsLeverage existing skills and standards
•• Different APIs have different performance and functionality Different APIs have different performance and functionality profiles, e.g.profiles, e.g.•• .NET ADF.NET ADF
J ADFJ ADFADF vs. Flex Application
Si l D i M•• Java ADFJava ADF•• JavaScriptJavaScript•• Flex APIFlex API•• SilverlightSilverlight 1.05
1.11.15
e (s
ec)
Simple Dynamic Map Rendering Comparison
SilverlightSilverlight
0.80.85
0.90.95
11.05
NET ADF Flex
Res
pons
e Ti
me
2828
.NET ADF Flex
SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: MIMEServer Architectures: MIMENLB + SOM + N SOC + M Web Servers (WS)NLB + SOM + N SOC + M Web Servers (WS)
NLBNLB
WSWS WSWS WSWS AdvantagesAdvantagesWSWS11 WSWS22 WSWSmm
SOMSOM
……gg
•• Best performingBest performing•• Easy maintenanceEasy maintenance
DisadvantagesDisadvantages•• SOM is a single point of failure.SOM is a single point of failure.
SOMSOM
g pg pSOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33 ……
2929SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: MIMEServer Architectures: MIMEPerformance CharacteristicsPerformance Characteristics NLBNLB
1800WSWS11 WSWS22 WSWSmm
SOMSOM
……ArcGIS Server Scalability (High Complexity)
ParametersParameters1200
1400
1600
ed U
sers
SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……
•• Document Complexity: Document Complexity: HighHigh•• Return Type: Return Type: MIMEMIME•• Web Servers: Web Servers: 11--22
SOM Machines:SOM Machines: 11800
1000
1200
f Sim
ulat
e
•• SOM Machines: SOM Machines: 11•• SOC Machines: SOC Machines: 11--9 (72 cores)9 (72 cores)
ObservationsObservations400
600
Num
ber o
f
Obse at o sObse at o s•• Linearity breaks down at Linearity breaks down at 77 SOC SOC
machines (machines (425,000 425,000 TrTr/Hr/Hr) () (49 49 corescores).).
•• Second Web server restoresSecond Web server restores
0
200
1 2 3 4 5 6 7 8 9
N
•• Second Web server restores Second Web server restores linearity.linearity.
3030
Number of SOC Machines
Actual: 1 WS Actual: 2 WS Ideal SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: MIMEServer Architectures: MIMEPerformance CharacteristicsPerformance Characteristics NLBNLB
WSWS11 WSWS22 WSWSmm
SOMSOM
……ArcGIS Server Scalability (Low Complexity)
3500
ParametersParameters
SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……
2500
3000
ed U
sers
•• Document Complexity: Document Complexity: LowLow•• Return Type: Return Type: MIMEMIME•• Web Servers: Web Servers: 11--22
SOM Machines:SOM Machines: 111500
2000
Sim
ulat
e
•• SOM Machines: SOM Machines: 11•• SOC Machines: SOC Machines: 11--9 (72 cores)9 (72 cores)
ObservationsObservations500
1000
umbe
r of
Obse at o sObse at o s•• Linearity breaks down at Linearity breaks down at 88 SOC SOC
machines (machines (1,150,000 1,150,000 TrTr/Hr/Hr) () (56 56 corescores).).
•• Second Web server restoresSecond Web server restores
0
500
1 2 3 4 5 6 7 8 9
N
•• Second Web server restores Second Web server restores linearity.linearity.
3131
Number of SOC Machines
Actual: 1 WS Actual: 2 WS Ideal SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: MIMEServer Architectures: MIMEPerformance CharacteristicsPerformance Characteristics
WSWS11 WSWS22 WSWSmm……
NLBNLB
Observations (continued)Observations (continued)
WSWS11 WSWS22 WSWSmm
SOMSOM
SOCSOC SOCSOC SOCSOCSOCSOC ……
•• Note increase in SOM CPU Note increase in SOM CPU utilization between tests utilization between tests run with 1 Web serverrun with 1 Web server
SOM CPU UtilizationSOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33
run with 1 Web server run with 1 Web server ((blueblue) and 2 Web servers ) and 2 Web servers ((greengreen).).
IIS CPU Utilization
UC2009 Technical WorkshopUC2009 Technical Workshop 3232
ArcGIS Server Architectures: “Classic” URLArcGIS Server Architectures: “Classic” URLNLB + SOM + N SOC + M Web Servers (WS) + X File Servers NLB + SOM + N SOC + M Web Servers (WS) + X File Servers (FS)(FS)
NLB(w/Affinity)NLB(w/Affinity)
WSWS WSWS WSWS
( y)( y)
AdvantagesAdvantages•• “Classic” users are familiar“Classic” users are familiar
WSWS11 WSWS22 WSWSmm
SOMSOM
……
FSFS FSFSFSFS•• Best performing (of URL Best performing (of URL cfgscfgs))•• Easy maintenanceEasy maintenance
SOMSOMFSFS FSFSFSFS ……
DisadvantagesDisadvantages•• SOM Single Point of FailureSOM Single Point of Failure•• FS Single Point of FailureFS Single Point of Failure
SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33 ……
3333
S S g e o t o a u eS S g e o t o a u e•• FS UNC PenaltyFS UNC Penalty
SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: “Classic” URLServer Architectures: “Classic” URLPerformance CharacteristicsPerformance Characteristics
WSWS WSWS WSWS……
NLB(w/Affinity)NLB(w/Affinity)
ArcGIS Server Scalability (High Complexity)
1000
WSWS11 WSWS22 WSWSmm
SOMSOM
……
FSFS FSFSFSFS ……
ParametersParameters700
800
900
ed Users
SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……
•• Document Complexity: Document Complexity: HighHigh•• Return Type: Return Type: URLURL•• Web Servers: Web Servers: 11--22
SOM Machines:SOM Machines: 11400
500
600
of Sim
ulate
•• SOM Machines: SOM Machines: 11•• SOC Machines: SOC Machines: 11--5 (45 cores)5 (45 cores)
ObservationsObservations100
200
300
400
Num
ber o
Obse at o sObse at o s•• Linearity breaks down at Linearity breaks down at 44 SOC SOC
machines (machines (275,000 275,000 TrTr/Hr/Hr) () (36 36 corescores).).
•• Second Web serverSecond Web server does notdoes not
0
100
1 2 3 4 5•• Second Web server Second Web server does notdoes not
restore linearity.restore linearity.
3434
Number of SOC Machines
Actual IdealSERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: “Classic” URLServer Architectures: “Classic” URLPerformance CharacteristicsPerformance Characteristics
WSWS WSWS WSWS……
NLB(w/Affinity)NLB(w/Affinity)
ArcGIS Server Scalability (Low Complexity)
1800
WSWS11 WSWS22 WSWSmm
SOMSOM
……
FSFS FSFSFSFS ……
ParametersParameters1200
1400
1600
d Users
SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……
•• Document Complexity: Document Complexity: LowLow•• Return Type: Return Type: URLURL•• Web Servers: Web Servers: 11--22
SOM Machines:SOM Machines: 11800
1000
1200
of Sim
ulated
•• SOM Machines: SOM Machines: 11•• SOC Machines: SOC Machines: 11--5 (45 cores)5 (45 cores)
ObservationsObservations400
600
Num
ber o
Obse at o sObse at o s•• Linearity breaks down at Linearity breaks down at 33 SOC SOC
machines (machines (350,000 350,000 TrTr/Hr/Hr) () (27 27 corescores).).
•• Second Web serverSecond Web server does notdoes not
0
200
1 2 3 4 5•• Second Web server Second Web server does notdoes not
restore linearity.restore linearity.
3535
Number of SOC Machines
Actual Ideal SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: “Classic” URLServer Architectures: “Classic” URLPerformance CharacteristicsPerformance Characteristics
WSWS WSWS WSWS……
NLB(w/Affinity)NLB(w/Affinity)
WSWS11 WSWS22 WSWSmm
SOMSOM
……
FSFS FSFSFSFS ……
Observations (continued)Observations (continued)•• Disk I/O on UNC share is not excessive.Disk I/O on UNC share is not excessive.
SOCSOC11 SOCSOC22 SOCSOCnnSOCSOC33……
•• Network traffic to UNC share does not even approach peak disk Network traffic to UNC share does not even approach peak disk I/O transfer rates. I/O transfer rates.
ConclusionsConclusions•• Inherent latency in UNC/CIFS share is the bottleneck.Inherent latency in UNC/CIFS share is the bottleneck.
•• A single service must use a single file server. If it becomes A single service must use a single file server. If it becomes popular, it doesn’t scale very well.popular, it doesn’t scale very well.
•• If many services exist, consider mapping each service to its own If many services exist, consider mapping each service to its own UNC share to improve performance.UNC share to improve performance.
3636SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures (MIME): Active/Active Server Architectures (MIME): Active/Active ( )( )SOMsSOMs2 SOM + M SOC + N WS2 SOM + M SOC + N WS
NLBNLB
WSWS11 WSWS22
SOMSOM SOMSOM22
WSWS33 WSWSnn
AdvantagesAdvantages•• Highly availableHighly available
SOMSOM11 SOMSOM22
•• Web Servers well utilizedWeb Servers well utilized
DisadvantagesDisadvantages•• Duplication of serviceDuplication of service
SOCSOC11 SOCSOC22 SOCSOCmm……SOCSOC33
•• Duplication of service Duplication of service configurationsconfigurations
•• SOMs don’t know how SOCs SOMs don’t know how SOCs are utilized by other SOMs.are utilized by other SOMs.
3737SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: Active/Passive SOMsServer Architectures: Active/Passive SOMs2 SOM + M SOC + N WS2 SOM + M SOC + N WS
NLBNLB
AdvantagesAdvantages•• Highly availableHighly available
WSWS11
AA PP
WSWS22
AA PP
WSWS33
AA PP
WSWSnn
AA PP •• Independent SOMsIndependent SOMs•• All Web servers utilizedAll Web servers utilized•• All SOC machines utilizedAll SOC machines utilizedSOMSOM11 SOMSOM22
AA PP AA PP AA PP AA PP
DisadvantagesDisadvantages•• MaintenanceMaintenance•• Only available in Java at Only available in Java at
9.3.1 (will be available in 9.3.1 (will be available in .NET at .NET at 10)10)
SOCSOC11 SOCSOC22 SOCSOCmm……SOCSOC33
3838SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGISArcGIS Server Architectures: “AllServer Architectures: “All--inin--ones” (MIME ones” (MIME ((or URL)or URL)N WS/SOM/SOC machinesN WS/SOM/SOC machines
NLB (w/affinity)NLB (w/affinity)
AdvantagesAdvantages•• Highly availableHighly availableWSWS WSWS WSWS
( y)( y)
•• Highly availableHighly available•• Some ease of configuration (no Some ease of configuration (no
distributed installation)distributed installation)•• Easily imaged and virtualizedEasily imaged and virtualized
WSWS11 WSWS22 WSWSnn
SOMSOM11
SOCSOC11 SOCSOC22 SOCSOCnn
……SOMSOM22 SOMSOMnn
•• Supports URL or MIME return Supports URL or MIME return type with same linear scalabilitytype with same linear scalability
DisadvantagesDisadvantages•• Duplication of service Duplication of service
configurationsconfigurations
3939
•• Reduced performance (SOCs Reduced performance (SOCs compete with SOM and WS)compete with SOM and WS)
SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
ArcGIS Server Architectures: MIMEArcGIS Server Architectures: MIMEVirtualization Penalty with VMwareVirtualization Penalty with VMware
1V_4C_16R 4V_1C_2R
Physical Machine: 4 CPU/16GB RAM
VM: 4 CPU/16GB RAM
SOM SOC WS
Physical Machine: 4 CPU/16GB RAM
VM: 1 CPU/2GB RAM
SOM SOC WS
VM: 1 CPU/2GB RAM
SOC
VM: 1 CPU/2GB RAM
SOC
VM: 1 CPU/2GB RAM
SOC
Physical Machine: 4 CPU/16GB RAM
2V_2C_8R
Physical Machine: 4 CPU/16GB RAM
4V_1C_4R
VM: 2 CPU/8GB RAM
SOM SOC WS
VM: 2 CPU/8GB RAM
SOC
VM: 1 CPU/4GB RAM
SOM SOC WS
VM: 1 CPU/4GB RAM
SOC
VM 1 CPU/4GB RAM VM 1 CPU/4GB RAM
UC2009 Technical WorkshopUC2009 Technical Workshop 4040
VM: 1 CPU/4GB RAM
SOC
VM: 1 CPU/4GB RAM
SOC
ArcGIS Server Architectures: MIMEArcGIS Server Architectures: MIMEVirtualization Penalty with VMwareVirtualization Penalty with VMware
4141SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
SummarySummaryyy•• TodayToday
–– Reviewed key performance factors inReviewed key performance factors in ArcGISArcGIS Server system componentsServer system components–– Reviewed key performance factors in Reviewed key performance factors in ArcGISArcGIS Server system componentsServer system components
–– Described strategies for component optimizationDescribed strategies for component optimization
–– Recommended deployment architecturesRecommended deployment architectures
ESRI System Design Strategies wikiESRI System Design Strategies wiki–– ESRI System Design Strategies wikiESRI System Design Strategies wikihttp://www.wiki.gis.com/wiki/index.php/System_Design_Strategieshttp://www.wiki.gis.com/wiki/index.php/System_Design_Strategies
4242
Still have questions?Still have questions?SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Additional ResourcesAdditional ResourcesQuestions, answers, and informationQuestions, answers, and information
•• Tech Talk Tech Talk •• ESRI Resource CentersESRI Resource Centers–– Outside this room right now!Outside this room right now!
•• Other sessionsOther sessions
–– PPTs, code, and videoPPTs, code, and video
resources.arcgis.comresources.arcgis.com
4343SERUG 2010 Technical SERUG 2010 Technical WorkshopWorkshop
Testing Testing ArcGISArcGIS Server Architectures (Data/Maps)Server Architectures (Data/Maps)
Map Service DescriptionMap Service Description
•• High ComplexityHigh Complexity
High Complexity
High ComplexityHigh Complexity–– 20 vector layers20 vector layers–– 3 raster layers3 raster layers–– Return image size ~ 100Return image size ~ 100--300 K300 K–– Long transaction times (~0.3 sec)Long transaction times (~0.3 sec)
•• Low ComplexityLow Complexity1 t l1 t l–– 1 vector layer1 vector layer
–– Return image size ~ 5Return image size ~ 5--50 K50 K–– Short transaction times (0.06 sec)Short transaction times (0.06 sec)
Low Complexity
•• BothBoth–– 9.3.1 fast “optimized” map service9.3.1 fast “optimized” map service–– Data local on SOC machinesData local on SOC machines–– FGDB dataFGDB data
4444UC2009 Technical WorkshopUC2009 Technical Workshop
Testing Testing ArcGISArcGIS Server Architectures (Methodology)Server Architectures (Methodology)•• SoftwareSoftware
Mi ft T Editi f S ft T tMi ft T Editi f S ft T t–– Microsoft Team Edition for Software Testers Microsoft Team Edition for Software Testers –– Microsoft Load Test Agent/ControllerMicrosoft Load Test Agent/Controller
•• Unit TestUnit Test–– Export dynamic maps via SOAP serviceExport dynamic maps via SOAP service–– 800 x 600 pixels800 x 600 pixels–– PNG 8PNG 8–– Random extents and scale levels that are reused between test runsRandom extents and scale levels that are reused between test runs
•• Load TestsLoad Tests–– Unit test is executed repeatedly by increasing numbers of test clients. Unit test is executed repeatedly by increasing numbers of test clients.
•• Hardware Hardware –– 4 dual4 dual--core core 2.5 GHz CPU, 16 GB RAM, 15K RPM local disks2.5 GHz CPU, 16 GB RAM, 15K RPM local disks
4545UC2009 Technical WorkshopUC2009 Technical Workshop