James Sill, Lu Zhang | Federal Science & Civilian Team

Jsill@esri.com | L.zhang@esri.com

Data Science Using ArcGIS Notebooks

Key Takeaways

• How Esri supports data science workflows

• How notebooks fit into the ArcGIS ecosystem

• Use cases of ArcGIS Notebooks

Agenda

• Introduction

• Notebooks within the ArcGIS Platform

- Notebooks in ArcGIS Online

- ArcGIS Notebook Server (Enterprise)

- Notebooks in ArcGIS Pro

• Key Workflows

• Q&A

Introduction

Spatial Analysis Data Science

Spatial Analysis Data Science

How some people describe it

Data Science

Spatial Analysis Data Science

How other people describe it

Spatial

Analysis

Spatial Analysis

Data Science

What it is

• Method > Interface

• Involves spatial data or

location

• A map is not always the

answer, but it usually

helps

• It is about the problem you are solving, not

about job titles

The overlap between spatial analysis and

data science

Analytical Tools in ArcGIS

• Maximum Likelihood

Classification

• Random Trees

• Support Vector Machine

Clustering

• Empirical Bayesian Kriging

• Areal Interpolation

• EBK Regression Prediction

• Ordinary Least Squares

Regression and

Exploratory Regression

• Geographically Weighted

Regression

• Forest Based Prediction• Spatially Constrained

Multivariate Clustering

• Multivariate Clustering

• Density-based Clustering

• Image Segmentation

• Hot Spot Analysis

• Cluster and Outlier Analysis

• Space Time Pattern Mining

Classification Prediction

• Python is one of the most widely used programming languages.

• Contains a wide set of libraries for conducting Data Science and Analysis

Python

Web GIS

Administration

Content

Management

Analysis and

Data Science

Python + GIS use cases

ArcPy

• Tools

• Functions

• Classes

• Modules

- Mapping – arcpy.mp

- Manipulate contents of ArcGIS Pro projects and layer

files

- Upgraded from arcpy.mapping

- Leverages Python 3.x

Desktop GIS Functionality

ArcGIS API for Python

• On your Web GIS perform:

- GIS visualization

- Analysis

- Spatial data management

• Leverage SciPy ecosystem for

- Automating workflows

- Performing repetitive tasks

• Integrate with notebooks for interactive

scripting and visualization

Web GIS Python Functionality

Python Notebook / Jupyter

• Jupyter is open source software designed to support interactive data science and

scientific computing.

• It is browser based and lets you write, run, and document your code all in one place.

• Esri has included Jupyter alongside ArcGIS Pro for several years and the ArcGIS

API for Python was developed to make the most of Jupyter’s functionality.

Why Use Notebooks

• Storytelling: transparency and reproducibility

• Interactive / Provide Feedback on a cell by cell basis

• Access to our Python libraries: ArcPy and the ArcGIS API for Python

• Integration with ArcGIS, to promote collaboration and sharing

Expanding your workflow…

ArcGIS

Open SourcePython & Data Science

• ArcGIS API for Python

• ArcPy• Analytics

• Geoprocessing

• Web GIS analysis• GeoAnalytics*

• Raster Analytics*

Apps

DesktopAPIs

ArcGIS Pro User Managed Environment

ArcPy

ArcGIS API

for Python

Python IDE

ArcGIS Notebooks

ArcPy

ArcGIS API

for Python

ArcGIS Enterprise

ArcGIS Online

How does ArcGIS Support Notebooks?

Integrated experience

within the Web GIS

ArcGIS Notebooks

ArcGIS API

for Python

ArcGIS Notebook Server

(ArcGIS Online / Enterprise)

Licensing – ArcGIS Notebook Server

Standard Advanced

ArcGIS API for Python + 3rd

party Python libraries Standard + ArcPy libraries

Windows

Prerequisites

• HyperV

• Docker Desktop

- Docker must use Linux

containers

• Docker

- The specific version of

Docker will depend on the

OS selected

• Window Server 2016 (Standard or higher)

• Windows Server 2019

• Red Hat Enterprise

Linux Server 7

• Ubuntu LTS Server

16.04 or 18.04

Operating

Systems

• 8 GB RAM

• 50 GB disk space

• 8 GB RAM

• 50 GB on /var directory

Hardware

Requirements

• Chrome

• Firefox

• Edge

• Chrome

• Firefox

• Edge

Browsers

Linux

Conceptual Architecture

Portal

for ArcGIS

Hosting

Server

ArcGIS

Data Store

(relational + tile cache)

ArcGIS

Web Adaptor

ArcGIS

Web Adaptor

ArcGIS Notebook

Server

ArcGIS

Web Adaptor

User 1

User 2

User 3

One container, multiple open Notebooks

Base Deployment

Install and

configure Docker

on the same

machine as your

Notebook Server.

End to End Workflow

ArcGIS Notebooks

Demo

Notebooks in ArcGIS ProNew in ArcGIS Pro 2.5

ArcGIS Notebooks in ArcGIS Pro

• New to ArcGIS Pro 2.5 | All Editions

• Leverages ArcGIS Pro Python Environment

• Import / execute existing Python Notebooks files

ArcGIS Notebooks in ArcGIS Pro

• Most Existing Notebook Functionality (Markdown, Autocompletion, Documentation)

• Magic (%) and shell (!) commands are not supported in notebooks run from ArcGIS

Pro at ArcGIS Pro 2.5.

• ArcGIS Notebooks hosted with ArcGIS Enterprise are not supported directly

in ArcGIS Pro 2.5.

What’s Next / Conclusion

Now

• Notebook Server Manager

• ArcGIS Notebooks in Online

- Beta now open!

• Notebooks in Pro

Mid to long term

• Versioning

• Notebooks as web tools

• Integration with ArcGIS Monitor

• More samples & tools

• Scheduling

Common Questions & Answers

Q. Installation / Architecture Related Questions..

A. Product Island @ Exhibit Hall

Q. Credit Consumption Rates for Notebooks in ArcGIS

Online

A. Basic: 3 credits an hour | GPU advanced ~ 20 credits an hour

- See Shannon Kalisky @ Product Island

Q. What Version of Python is supported within ArcGIS Notebooks?

A. Python 3.x

Print Your Certificate of Attendance

Print Stations Located in 150 Concourse Lobby

Tuesday12:30 pm – 6:30 pm

Expo

Hall B

5:15 pm – 6:30 pm

Expo Social

Hall B

Wednesday10:45 am – 5:15 pm

Expo

Hall B

6:30 pm – 9:30 pm

Networking Reception

Smithsonian National Museum

of Natural History

Download the Esri

Events app and find your event

Select the session

you attended

Scroll down to

“Survey”

Log in to access the

survey

Complete the survey

and select “Submit”

Please Share Your Feedback in the App