Download - ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Transcript
Page 1: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

ArcGIS API for Python for

ANALYSTS and DATA SCIENTISTSROHIT SINGH

ALBERTO NIETO

ANDREW CHAPKOWSKI

Page 2: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Introduction

Page 3: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Analysts, Data Scientists and Developers

Source: PyCon India 2018 Python And Data: Past, Present And Future By Peter

Wang

Data Scientist

Page 4: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Why Python?

• Popularity

• Productivity

• Interoperability

• Solves the “two-language” problem

• Scientific Python ecosystem

• Community

Page 5: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Source: Gateway Data Sciences Courses Reach Enrollment Milestones

Page 6: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Analysis

Page 7: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

geometry

geo

processing

network

schematics

featuresraster

realtime

widgets

mapping

env

geo

analytics

gisgeocoding

geo

enrichment

Page 8: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

ArcGIS + Jupyter = ♥

Text goes here

Page 9: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 10: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 11: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 12: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Discover relationships, patterns and trends in data

arcgis.feature submodules

Spatial Analysis

Page 13: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Spatial Analysis Tools

• Summarize Data

• Find Locations

• Data Enrichment

• Analyze Patterns

• Use Proximity

• Manage Data

Page 14: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Summarize DataCalculate summary statistics for features and attributes

Page 15: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Enrich LayerAdd detailed demographic data and statistics to your analysis

Page 16: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Use Proximity“What is near what?”

Page 17: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Use Proximity“What is near what?”

Page 18: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Manage DataManage geographic data, overlay layers

Page 19: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Identify, quantify, and visualize spatial patterns in your data.

Page 20: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Geocoding, Batch geocoding, reverse geocodingarcgis.geocoding module

Geocoding

Page 21: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

GeocodeSingle line or multi field addresses

Page 22: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

GeocodePoints of Interest

• Cultural or geographic landmarks

• Businesses by name or category

• Administrative divisions

Page 23: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Batch GeocodingGeocode an entire list of single line or multi field addresses

Page 24: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Reverse GeocodeDetermines address at a particular x/y location

Page 25: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Routing and directions, location allocation, service areas…

arcgis.network.analysis module

Network Analysis

Page 26: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Network analysis capabilities and toolsCommercial grade, traffic aware routing and directions for multiple travel modes

Page 27: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Network analysis – Service Areas

Page 28: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Enrich your analysis with demographic and business data

arcgis.geoenrichment module

GeoEnrichment

Page 29: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

GeoEnrichment

• Get facts about a location or area

- Street addresses

- Points, lines and polygon geometries

- Within a drive time or service area

- Named geographical areas

- Counties or block groups in California

- Districts and subdistricts in India

• Create charts and choropleth maps

Page 30: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

GeoEnrichment – create reports

Page 31: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Fast distributed spatio-temporal analysis of large vector and tabular data

arcgis.geoanalytics module

GeoAnalytics

Page 32: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

GeoAnalytics Tools

• Summarize Data

• Manage Data

• Use Proximity

• Find Locations

• Analyze Patterns

Page 33: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

DemoAnalysis of crime patterns in Houston

Page 34: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 35: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 36: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 37: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 38: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

On-the-fly and distributed batch analysis of raster data

arcgis.raster module

Imagery and Raster Analysis

Page 39: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 40: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 41: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 42: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 43: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 44: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 45: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 46: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 47: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 48: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 49: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 50: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 51: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 52: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 53: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 54: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 55: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 56: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 57: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 58: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 59: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 60: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 61: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 62: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 63: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 64: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 65: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Feature layers as Pandas’ dataframe

arcgis.features.SpatialDataFrame class

Spatial Data Frame

Page 66: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

What is a Spatial Data Frame?

• Built on Pandas

• Easy to Update, Query and Add New Content

• Quickly visualize data

Page 67: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Built to do Work!

FIONA

Shapely Pyshp

Page 68: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Benefits of Spatial Data Frame

• Packs large datasets into memory efficiently

• Spatial Indexing

• Consumes multiple data format

• Small footprint with BIG results

• Using Pandas v.23

Page 69: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Geometry Engine Support

• ArcPy

• Shapely

Data Reader Support

• ArcPy

• Fiona

• pyshp

Page 70: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Demo Flight Paths

Page 71: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Demo

Page 72: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Find the patterns hidden in data

Data Science

Page 73: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Data Science with ArcGIS - Data

• Esri curated content – Living Atlas

- Multi-spectral, temporal, dynamic imagery layers

- Landsat, NAIP, MODIS, Elevation

- Basemaps, Imagery, Demographics, Transport

- Boundaries & places, Landscape, Oceans

- Earth Observations, Urban Systems, Historical Maps, …

• Your data, org’s data, data shared with you

- Shapefiles, File geodatabase, CSV, Excel, HTML, …

- File shares, cloud share

- HDFS, Hive and databases

• Public data

- Maps, layers and datasets shared by users worldwide

Page 74: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Data Science with ArcGIS - Visualization

• Visualize with ArcGIS

- Map widget in Jupyter notebook

- Web Maps and Web Scene

- Feature layers

- Raster and imagery layer

- Smart mapping

- Pythonic renderers and symbology

• Visualize with Python

- Matplotlib, Seaborn, Bokeh, Plotly, …

- Datashader, Holoviews, Mayavi, …

Page 75: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Data Science with ArcGIS - Analysis

• Analysis with ArcGIS

- Geoprocessing in Web GIS

- Spatial analysis, Routing and directions

- Network analysis, Geocoding, Geoenrichment…

- Imagery and Raster Analysis

- On the fly dynamic image processing

- Distributed raster analysis

- GeoAnalytics – large tabular and vector data

Page 76: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Data Science with ArcGIS - Analysis

• Analysis with Python libraries

- Data wrangling

- Pandas, numpy, scipy

- Machine learning

- Scikit-learn, tensorfly, keras, pytorch, fastai

- Geospatial analysis

- PySAL, GDAL, Shapely, Fiona, …

- Image processing and computer vision

- PIL, OpenCV, scikit-image

Page 77: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Data Science workflows - Deployment

• Deploy as information products

- WebMaps

- Web scenes

- Layers

• Deploy as web tools

- Geoprocessing script tools

- Binder projects

• Deploy as dashboards

- ArcGIS Operations Dashboard

- Jupyter dashboard

- Plotly dashboard

Page 78: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Questions?

Page 79: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Please Take Our Survey on the App

Download the Esri Events

app and find your event

Select the session

you attended

Scroll down to find the

feedback section

Complete answers

and select “Submit”

Page 80: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 81: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

Presentation TitlePresenter Title

Page 82: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 83: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 84: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,
Page 85: ArcGIS API for Python for Analysts and Data Scientists · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine learning-Scikit-learn,

See Us Here

• Title

WORKSHOP LOCATION

• Where

TIME FRAME

• 12:00 – 1:00