Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation...

Esri UC 2014 | Technical Workshop |

ArcGIS for Server: Administrative Scripting and AutomationShreyas Shinde

Ranjit Iyer


• Introduction to server administration

• Command line tools

• ArcGIS Server Manager

• Overview of the administrative API

• Automation

• Popular scripting workflows- Publishing of services

- Setting up permissions

- Extracting popular extents

- Monitoring services and databases

- Scaling and statistics

Agenda

ArcGIS for Server: Administrative Scripting and Automation


Introduction to server administration


ArcGIS Server Installation MediaProvision hardware Select a deployment pattern

ArcGIS Server SetupSoftware Authorization

Keep it running…


The administrator’s toolbox


ArcGIS Server Manager

ArcGIS Server Administrator Directory

Console tools


• Located under /arcgis/server/tools

• A collection of most popular tools

• Categories- Administrative credentials

- For resetting administrative accounts

- SSL/certificate management

- Backup and restore

- Managing cache

• Lock the tools down using OS file permissions!

Command line tools


Esri UC 2014 | Technical Workshop |Esri UC 2014 | Technical Workshop |

Command line tools

Demo



• Browser based interactive application

• Located at: http://server.domain.com:6080/arcgis/manager

• Simple, easy-to-use

• Ability to manage your entire server site- Even though you are accessing Manager on a specific server node

• Great for common workflows!

ArcGIS Server Manager



le Tour de Manager

Demo



ArcGIS Server Manager – under the hood


Simple HTTP requests to the administrative API


• Entire server administration is an API- If ArcGIS Server Manager can use it, so can you!

• RESTful- GET and POST of JSON data

• Well organized and well documented

• Lots of samples

• Scriptable

• Designed for automation!

Overview of the administrative API



The administrative API


Admin APIAdmin API

ServicesServices

Create and manage your services. Start or stop them as wanted

Machines+

Clusters

Machines+

Clusters

Add new nodes to grow capacity. Remove them to shrink. Organize machines into clusters.

SecuritySecurity

Configure GIS tier or web tier authentication. Manage accounts, roles and permissions on services. Configure SSL.

Statistics(new at 10.3)

Statistics(new at 10.3)

Get fine grained statistics on your GIS services.

LogsLogs

Harvest logs for auditing, understanding usage patterns and troubleshooting.

DatabasesDatabasesManage all your database connections through a single interface.

System environment

System environment

Manage system properties, web adaptors, server directories.


• Common repeated workflows- Plot popular extents for the data

• Scheduled actions- Restarting services when data is updated

- Build map caches during low demand by moving machines to caching cluster

• Reactive actions- Adding capacity (server nodes) to a cluster when demand increases

- Send email notifications when errors are logged

• Works with all programming/scripting languages that support HTTP!

Why automate?



Where should you begin?



Publish Services

Demo



• Publishing many services

• Moving services between Dev, Staging, Production environments- Backup/restore is an option but sometimes you just want to republish services

• Can publish from Desktop and Manager

• Tedious if you have lots of services

• Automation is your friend!

Motivation



• Help system has a script that uses ArcPy

• ArcPy is an python library that complements the Admin REST API

• Provides many convenience functions

Task : Publish a list of services listed in a file



• Works great, just need a few lines of code

• However, you need ArcPy on the machine

• Script publishes service definitions sequentially- For every SD

- Upload

- Publish

• With many SDs to publish this can be a very slow process

• Publishing ~100 services took hours

Caveats



• Run the script from any machine- No ArcPy dependency

• Exploit concurrency- Parallel uploads

- Parallel publishing

• If your script is amenable to concurrency, exploit it by all means

We can do better



• Two step process

- Upload the SD to the Server- QUICK

- Invoke the publishing tool- INTENSIVE

Publishing a service with an SD



Script design


UploadUpload

UploadUpload

UploadUpload

PublishPublish

PublishPublish

SDSD SDSD SDSD UpldIDUpldID UpldIDUpldID UpldIDUpldIDSDSD


Comparison results


Speed up of ~3

ArcPy publishing script DIY multi-threaded publishing script


Starting a service

Demo



• Manager

• Admin

• Script

Options


Start Geometry Service


• Main function- Drives the logic

• Internal utility functions- Get Token

- Make HTTP requests

- Parse JSON responses

Anatomy of an Admin script



Apply Permissions

Demo



• Permissions are the way to secure your service

• Do it in Manager

• Tedious for lots of services. Must automate!

Task : Apply permissions to services



• Show them previously published services in Manager

• Show them roles

• Run the script

• Discuss the script

Demo



Health check for services

Demo


Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation

Get all the foldersGet all the foldersUsing the report API, get all the service

instances.

Using the report API, get all the service

instances.

Using Geo-services REST API to test

availability.

Using Geo-services REST API to test

availability.

Print reportPrint report


Health check for databases

Demo



Use findItems to search for all

enterprise geodatabases

Use findItems to search for all

enterprise geodatabases Use validateDataItem

to test database connectivity.

Use validateDataItem to test database

connectivity.

Print outputPrint output


Popular extents

Demo



Elastic scaling

Demo



• Add machines to reduce latency and increase throughput

• Dynamically add them when a latency threshold is breached

• De-allocate machines when traffic is less

Task : Dynamically add machines to your site



• Start with a single node site

• Make service requests and watch latency (we’ll use JMeter)

• Run a script that adds a machine to a cluster

• Watch the latency drop

Demo



Statistics in 10.3 (Sneak peak)



• Server is always collecting statistics- Avg response

- Number of requests

- Number of timeouts

- . . .

• Report defines the resource, metric and interval- Previous day, last 7 days, last 30 days

- Custom time window

• Server chooses intelligent data point intervals in the reports

• REST API to harvest all this information

Statistics at 10.3



Python is great!!



• Most modern languages (like Python) support Threads

• Some languages provide concurrency as a language feature- Go

- Erlang

- Clojure

- Scala

• Suited for concurrent programming

Short detour…



• Invented at Google

• About concurrency

• Lightweight threads (goroutines)

• Typed thread-safe communication and

synchronization (channels)

• Readable concurrent code

Go



• Invented at Ericsson

• Functional language

• Light weight ‘processes’

• Can spawn 100s of thousands

of them

• Concurrency via message passing

• Actors model

Erlang



Python is great!!


Don’t get us in trouble, we don’t want to start a rebellion


Thank you…

• Please fill out the session survey:

First Offering ID: 1146

Second Offering ID: 1344

Online – www.esri.com/ucsessionsurveys

Paper – pick up and put in drop box



Questions?



Resources


GIS Tutorial for Python ScriptingEsri Press, 2014

Paperback and e-book Just released! Offers several hands-on tutorial exercises.

Python Scripting for ArcGISEsri Press, 2013

Paperback and e-book Good reference text

Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation...

Documents

Transcript of Esri UC 2014 | Technical Workshop | ArcGIS for Server: Administrative Scripting and Automation...