ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS...

75
ArcGIS API for Python for ANALYSTS and DATA SCIENTISTS Rohit Singh

Transcript of ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS...

Page 1: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

ArcGIS API for Python for

ANALYSTS and DATA SCIENTISTSRohit Singh

Page 2: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Thank You to Our Sponsors

Page 3: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Introduction

Page 4: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Analysts, Data Scientists and Developers

Source: PyCon India 2018 Python And Data: Past, Present And Future By Peter

Wang

Data Scientist

Page 5: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Why Python?

• Popularity

• Productivity

• Interoperability

• Solves the “two-language” problem

• Scientific Python ecosystem

• Community

Page 6: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Source: Gateway Data Sciences Courses Reach Enrollment Milestones

Page 7: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

ArcGIS API for Python

Page 8: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

geometry

geo

processing

network

schematics

featuresraster

realtime

widgets

mapping

env

geo

analytics

gisgeocoding

geo

enrichment

apps

Page 9: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

ArcGIS + Jupyter = ♥

Text goes here

Page 10: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 11: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 12: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 13: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Discover relationships, patterns and trends in data

arcgis.feature submodules

Spatial Analysis

Page 14: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Spatial Analysis Tools

• Summarize Data

• Find Locations

• Data Enrichment

• Analyze Patterns

• Use Proximity

• Manage Data

Page 15: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Summarize DataCalculate summary statistics for features and attributes

Page 16: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Enrich LayerAdd detailed demographic data and statistics to your analysis

Page 17: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Use Proximity“What is near what?”

Page 18: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Use Proximity“What is near what?”

Page 19: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Manage DataManage geographic data, overlay layers

Page 20: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Identify, quantify, and visualize spatial patterns in your data.

Page 21: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Geocoding, Batch geocoding, reverse geocodingarcgis.geocoding module

Geocoding

Page 22: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

GeocodeSingle line or multi field addresses

Page 23: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

GeocodePoints of Interest

• Cultural or geographic landmarks

• Businesses by name or category

• Administrative divisions

Page 24: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Batch GeocodingGeocode an entire list of single line or multi field addresses

Page 25: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Reverse GeocodeDetermines address at a particular x/y location

Page 26: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Routing and directions, location allocation, service areas…

arcgis.network.analysis module

Network Analysis

Page 27: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Network analysis capabilities and toolsCommercial grade, traffic aware routing and directions for multiple travel modes

Page 28: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Network analysis – Service Areas

Page 29: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Enrich your analysis with demographic and business data

arcgis.geoenrichment module

GeoEnrichment

Page 30: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

GeoEnrichment

• Get facts about a location or area

- Street addresses

- Points, lines and polygon geometries

- Within a drive time or service area

- Named geographical areas

- Counties or block groups in California

- Districts and subdistricts in India

• Create charts and choropleth maps

Page 31: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

GeoEnrichment – create reports

Page 32: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Fast distributed spatio-temporal analysis of large vector and tabular data

arcgis.geoanalytics module

GeoAnalytics

Page 33: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

GeoAnalytics Tools

• Summarize Data

• Manage Data

• Use Proximity

• Find Locations

• Analyze Patterns

Page 34: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

DemoAnalysis of crime patterns in Houston

Page 35: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 36: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 37: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 38: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 39: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

On-the-fly and distributed batch analysis of raster data

arcgis.raster module

Imagery and Raster Analysis

Page 40: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 41: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 42: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 43: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 44: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 45: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 46: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 47: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 48: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 49: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 50: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 51: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 52: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 53: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 54: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 55: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 56: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 57: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 58: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 59: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 60: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 61: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 62: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 63: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 64: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 65: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine
Page 66: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Feature layers as Pandas’ dataframe

Spatially enabled DataFrame

Page 67: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Spatially enabled Pandas DataFrame

• Interchange format between ArcGIS feature data and scientific Python libraries

• Pandorable spatial and attribute operations

• Read and write feature layers, shapefiles,

file geodatabases, feature classes

• Visualize features using matplotlib-like plot() method, with support for renderers and

symbology

• Support for spatial index and spatial joins

Page 68: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Find the patterns hidden in data

Data Science

Page 69: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Data Science with ArcGIS - Data

• Esri curated content – Living Atlas

- Multi-spectral, temporal, dynamic imagery layers

- Landsat, NAIP, MODIS, Elevation

- Basemaps, Imagery, Demographics, Transport

- Boundaries & places, Landscape, Oceans

- Earth Observations, Urban Systems, Historical Maps, …

• Your data, org’s data, data shared with you

- Shapefiles, File geodatabase, CSV, Excel, HTML, …

- File shares, cloud share

- HDFS, Hive and databases

• Public data

- Maps, layers and datasets shared by users worldwide

Page 70: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Data Science with ArcGIS - Visualization

• Visualize with ArcGIS

- Map widget in Jupyter notebook

- Web Maps and Web Scene

- Feature layers

- Raster and imagery layer

- Smart mapping

- Pythonic renderers and symbology

• Visualize with Python

- Matplotlib, Seaborn, Bokeh, Plotly, …

- Datashader, Holoviews, Mayavi, …

Page 71: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Data Science with ArcGIS - Analysis

• Analysis with ArcGIS

- Geoprocessing in Web GIS

- Spatial analysis, Routing and directions

- Network analysis, Geocoding, Geoenrichment…

- Imagery and Raster Analysis

- On the fly dynamic image processing

- Distributed raster analysis

- GeoAnalytics – large tabular and vector data

Page 72: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Data Science with ArcGIS - Analysis

• Analysis with Python libraries

- Data wrangling

- Pandas, numpy, scipy

- Machine learning

- Scikit-learn, tensorfly, keras, pytorch, fastai

- Geospatial analysis

- PySAL, GDAL, Shapely, Fiona, …

- Image processing and computer vision

- PIL, OpenCV, scikit-image

Page 73: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Data Science workflows - Deployment

• Deploy as information products

- WebMaps

- Web scenes

- Layers

• Deploy as web tools

- Geoprocessing script tools

- Binder projects

• Deploy as dashboards

- ArcGIS Operations Dashboard

- Jupyter dashboard

- Plotly dashboard

Page 74: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine

Questions?

Page 75: ArcGIS API for Python: for Analysts and Data Scientists · 2018-11-07 · Data Science with ArcGIS - Analysis • Analysis with Python libraries-Data wrangling-Pandas, numpy, scipy-Machine