ArcGIS for Server: Administrative Scripting and Automation€¦ · -SSL/certificate management...

Esri UC 2014 | Technical Workshop |

ArcGIS for Server: Administrative Scripting and Automation Shreyas Shinde Ranjit Iyer


• Introduction to server administration • Command line tools • ArcGIS Server Manager • Overview of the administrative API • Automation • Popular scripting workflows

- Publishing of services - Setting up permissions - Extracting popular extents - Monitoring services and databases - Scaling and statistics

Agenda

ArcGIS for Server: Administrative Scripting and Automation


Introduction to server administration


ArcGIS Server Installation Media Provision hardware Select a deployment pattern

ArcGIS Server Setup Software Authorization

Keep it running…


The administrator’s toolbox


ArcGIS Server Manager

ArcGIS Server Administrator Directory

Console tools


• Located under /arcgis/server/tools • A collection of most popular tools • Categories

- Administrative credentials - For resetting administrative accounts

- SSL/certificate management - Backup and restore - Managing cache

• Lock the tools down using OS file permissions!

Command line tools


Esri UC 2014 | Technical Workshop | Esri UC 2014 | Technical Workshop |

Command line tools

Demo



• Browser based interactive application • Located at: http://server.domain.com:6080/arcgis/manager • Simple, easy-to-use • Ability to manage your entire server site

- Even though you are accessing Manager on a specific server node

• Great for common workflows!

ArcGIS Server Manager


http://server.domain.com:6080/arcgis/manager


le Tour de Manager

Demo



ArcGIS Server Manager – under the hood


Simple HTTP requests to the administrative API


• Entire server administration is an API - If ArcGIS Server Manager can use it, so can you!

• RESTful - GET and POST of JSON data

• Well organized and well documented • Lots of samples • Scriptable • Designed for automation!

Overview of the administrative API



The administrative API


Admin API

Services

Create and manage your services. Start or stop them as wanted

Machines +

Clusters

Add new nodes to grow capacity. Remove them to shrink. Organize machines into clusters.

Security

Configure GIS tier or web tier authentication. Manage accounts, roles and permissions on services. Configure SSL.

Statistics (new at 10.3)

Get fine grained statistics on your GIS services.

Logs

Harvest logs for auditing, understanding usage patterns and troubleshooting.

Databases Manage all your database connections through a single interface.

System environment

Manage system properties, web adaptors, server directories.


• Common repeated workflows - Plot popular extents for the data

• Scheduled actions - Restarting services when data is updated - Build map caches during low demand by moving machines to caching cluster

• Reactive actions - Adding capacity (server nodes) to a cluster when demand increases - Send email notifications when errors are logged

• Works with all programming/scripting languages that support HTTP!

Why automate?



Where should you begin?


http://localhost:6080/arcgis/help/en/0154/0154000002np000000.htm

http://localhost:6080/arcgis/admin/www/doc/index.html


Publish Services

Demo



• Publishing many services • Moving services between Dev, Staging, Production environments

- Backup/restore is an option but sometimes you just want to republish services

• Can publish from Desktop and Manager • Tedious if you have lots of services • Automation is your friend!

Motivation



• Help system has a script that uses ArcPy • ArcPy is an python library that complements the Admin REST API • Provides many convenience functions

Task : Publish a list of services listed in a file



• Works great, just need a few lines of code • However, you need ArcPy on the machine • Script publishes service definitions sequentially

- For every SD - Upload - Publish

• With many SDs to publish this can be a very slow process • Publishing ~100 services took hours

Caveats



• Run the script from any machine - No ArcPy dependency

• Exploit concurrency

- Parallel uploads - Parallel publishing

• If your script is amenable to concurrency, exploit it by all means

We can do better



• Two step process

- Upload the SD to the Server - QUICK

- Invoke the publishing tool - INTENSIVE

Publishing a service with an SD



Script design


Upload

Upload

Upload

Publish

Publish

SD SD SD UpldID UpldID UpldID SD


Comparison results


Speed up of ~3

ArcPy publishing script DIY multi-threaded publishing script


Starting a service

Demo



• Manager

• Admin

• Script

Options


Start Geometry Service

http://resources.arcgis.com/en/help/main/10.2/#/Example_Start_the_geometry_service/0154000005qq000000/


• Main function - Drives the logic

• Internal utility functions

- Get Token - Make HTTP requests - Parse JSON responses

Anatomy of an Admin script



Apply Permissions

Demo



• Permissions are the way to secure your service • Do it in Manager

• Tedious for lots of services. Must automate!

Task : Apply permissions to services



• Show them previously published services in Manager • Show them roles • Run the script • Discuss the script

Demo



Health check for services

Demo


Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation

Get all the folders Using the report API, get all the service

instances.

Using Geo-services REST API to test

availability.

Print report


Health check for databases

Demo



Use findItems to search for all

enterprise geodatabases Use validateDataItem

to test database connectivity.

Print output


Popular extents

Demo



Elastic scaling

Demo



• Add machines to reduce latency and increase throughput • Dynamically add them when a latency threshold is breached • De-allocate machines when traffic is less

Task : Dynamically add machines to your site



• Start with a single node site • Make service requests and watch latency (we’ll use JMeter) • Run a script that adds a machine to a cluster • Watch the latency drop

Demo



Statistics in 10.3 (Sneak peak)



• Server is always collecting statistics - Avg response - Number of requests - Number of timeouts - . . .

• Report defines the resource, metric and interval - Previous day, last 7 days, last 30 days - Custom time window

• Server chooses intelligent data point intervals in the reports • REST API to harvest all this information

Statistics at 10.3



Python is great!!



• Most modern languages (like Python) support Threads • Some languages provide concurrency as a language feature

- Go - Erlang - Clojure - Scala

• Suited for concurrent programming

Short detour…



• Invented at Google • About concurrency • Lightweight threads (goroutines) • Typed thread-safe communication and synchronization (channels) • Readable concurrent code

Go



• Invented at Ericsson • Functional language • Light weight ‘processes’ • Can spawn 100s of thousands of them • Concurrency via message passing • Actors model

Erlang



Python is great!!


Don’t get us in trouble, we don’t want to start a rebellion J


Thank you…

• Please fill out the session survey:

First Offering ID: 1146 Second Offering ID: 1344

Online – www.esri.com/ucsessionsurveys Paper – pick up and put in drop box


http://www.esri.com/ucsessionsurveys


Questions?



Resources


GIS Tutorial for Python Scripting Esri Press, 2014

Paperback and e-book Just released! Offers several hands-on tutorial exercises.

Python Scripting for ArcGIS Esri Press, 2013

Paperback and e-book Good reference text

http://esripress.esri.com/display/index.cfm?fuseaction=display&websiteID=256&moduleID=0




ArcGIS for Server: Administrative Scripting and Automation€¦ · -SSL/certificate management...

Documents

Transcript of ArcGIS for Server: Administrative Scripting and Automation€¦ · -SSL/certificate management...