HarnessingthePowerofPythoninArcGISUsingtheCondaDistributionShaunWalbridge

https://github.com/scw/conda-uc-2016-demo

HighQualityPDF(2MB)

https://github.com/scw/conda-uc-2016-demohttps://4326.us/esri/conda-uc/uc-2016-conda-conda-demo-full.pdf

Python

WhyPython?Accessiblefornew-comers,andthe

Extensivepackagecollection(56thousandon ),broaduser-baseStronggluelanguageusedtobindtogethermanyenvironments,bothopensourceandcommercialOpensourcewithliberallicense—dowhatyouwant

mosttaughtfirstlanguageinUSuniversites

PyPI

http://cacm.acm.org/blogs/blog-cacm/176450-python-is-now-the-most-popular-introductory-teaching-language-at-top-us-universities/fulltexthttps://pypi.python.org/pypi

WhyPython?Inthebox:

TheSciPyStack(NumPy,SciPy,Pandas,matplotlib,sympy)

xlrd,netCDF4,requests,PyPDF,pytz

ScientificProgrammingwiththeSciPyStack

https://4326.us/esri/scipy/#/

WhyPython?Beyondthebox:

IntegratingOpen-SourceStatisticalPackageswithArcGISusingPythonandR—Tomorrowat10:15am,Ballroom6D

Python:ExtendingwithOtherLibraries—Todayat4:00pm,TechTheater16

DeeperDiveintoCondainDevSummitTechSessionVideo

http://video.esri.com/watch/5072/harnessing-the-power-of-python-in-arcgis-using-the-conda-distribution

PackageManagementforPython

Whynotpip,wheels,virtualenvs?Don’thandletheharderproblemofsystemdependencies,consideredoutofscopebyPythonpackagers—doesitendupinsite-packages?Packagedevs:OnOSXandLinux,‘easy’togetthedeps!Useasystempackagemanager(e.g.apt,brew,yum)andtheincludedcompiler(e.g.clang,gcc).It’sstillnoteasytomakereproduciblebuilds,andwhataboutWindows?

WhataboutWindows?WeareparticularlystuckonWindowswhichlacksbroadlyusedpackagemanagementOnlydevelopershaveaCcompilerontheirmachineAhardproblem

WhataboutWindows?WeareparticularlystuckonWindowswhichlacksbroadlyusedpackagemanagementOnlydevelopershaveaCcompilerontheirmachineAhardproblem

EnterConda

WhyConda?

ScientificPythoncommunityidentifiedthattherewasagapnotbeingaddressedbythecorePythoninfrastructure,limitingtheirabilitytogetpackagesintothehandsofusers

Industrystandardbuiltbypeoplewhocareaboutthisspace—ContinuumAnalytics

WhyConda?

Itsolvesahardproblem:

Handlesdependenciesformanylanguages(C,C++,RandofcoursePython)

BuiltforPythonfirst,butitreallysolvesamuchbroaderinfrastructuralissue.

Conda

CondaCross-platform:simplydeveloprecipesforbuildingandinstallingsoftwareonLinux,OSXandWindows.Opensource:Esriisusingit,youcanuseitinyourownprojectsforothercontexts

Whatcanitinstall?Notjustscientificpackages.Itcanhelpwith:

GUItoolkits(PyQt,TKinter)C++Libraries(Boost)IDEs(Spyder,Juptyer)

CondaEnvironments:CanisolateaPythonenvironment,flexiblymakechangeswithotaffectinginstalledsoftware.Requirements—includeexplicitstateinformation,notjustthepackagename.Namesaren’tenough!AlsohandlesplatformsandJupyternotebooks

HowDoesitWork?Condapackagescancomefromavarietyoflocations:

Ondisk(file://)Publicrepositories(AnacondaCloud,self-hosted)Privaterepositoriesanaconda.org

https://anaconda.org/

CondaBasicsCommandlineinterface,fornow

CondaCheatsheet

http://conda.pydata.org/docs/_downloads/conda-cheatsheet.pdf

CondaBasicsDemo

CondaBasicsActivatingenvironments,acoupleways:

UsetheshortcutsManuallyactivatetheenvironment:

cd C:\ArcGIS\bin\Python\Scripts activate arcgispro-py3

CondaBasicsTostart:

AcollectionofpackagesandPythoninstalliscalledanenvironmentorenv,thebuildingblockformanagingPythonwithCondaCanhavemultipleenvironmentsandseamlesslyswitchbetweenthem

conda --help

CondaBasicsOnceyou’reinanenvironmentgetdetailswithinfo:

Condainfoisthestartingpoint—ittellsyouthestateoftheenvironment.

conda info

CondaBasicsconda info

Current conda install:

platform : win-64 conda version : 4.0.6 conda-build version : not installed python version : 3.5.1.final.0 requests version : 2.9.1 root environment : C:\ArcGIS\bin\Python (writable) default environment : C:\ArcGIS\bin\Python\envs\arcgispro-py3 envs directories : C:\ArcGIS\bin\Python\envs package cache : C:\ArcGIS\bin\Python\pkgs channel URLs : https://conda.anaconda.org/esri/win-64/ https://conda.anaconda.org/esri/noarch/ https://repo.continuum.io/pkgs/free/win-64/ https://repo.continuum.io/pkgs/free/noarch/ config file : C:\ArcGIS\bin\Python\.condarc

CondaBasicsconda list

# packages in environment at C:\ArcGIS\bin\Python\envs\arcgispro-py3:#arcgispro 1.3 0 esricolorama 0.3.6 py34_0 defaultsfuture 0.15.2 py34_0 defaultsmatplotlib 1.4.3 np19py34_0 defaultsmsvc_runtime 1.0.1 vc10_0 [vc10] defaultsnose 1.3.7 py34_0 defaultsnumpy 1.9.3 py34_0e [arcgispro] esriopenssl 1.0.2h vc10_0 [vc10] defaultspandas 0.17.1 np19py34_0 esripip 8.1.1 py34_1 defaultspy 1.4.31 py34_0 defaultspyparsing 2.1.1 py34_0 defaultspypdf2 1.25.1 py_0 esripytest 2.9.1 py34_0 defaultspython 3.4.4 4 defaultspython-dateutil 2.5.3 py34_0 defaultspytz 2016.4 py34_0 defaultsrequests 2.9.1 py34_0 defaultsscipy 0.16.1 np19py34_0e [arcgispro] esrisetuptools 20.7.0 py34_0 defaultssix 1.10.0 py34_0 defaultssympy 0.7.6.1 py34_0 defaultsvs2010_runtime 10.00.40219.1 0 defaults

CondaBasicsCreatingnewenvironments:

Afewdifferentways.Canmanuallyspecifythedependencies:

Canalsouseafilewhichincludesallthedependencies:

Thesecancontainexplcitinformationaboutchannels,toensurethatthenewenvironmentpreciselymatchestherequirements.

conda create --name my_env python=3.4 numpy flask dask

conda create --name my_env --file my_sweet_depends.txt

Condavs…Name Means Will

Ship?

Conda Thecommanditself ✓Miniconda AminimumsetofPythonpackagesto

buildandrunConda.✓

Anaconda Adistribution200+packagesbuiltwithConda

AnacondaServer

Hostthefullinfrastructureinternally

SkikitLearnDemoHavetweetsaboutaniOSappreleasedat#SXSWWhattopeoplethinkofit?Useanaivebayesclassifiertodeterminesentiment

SkikitLearnDemo# Scikit learn model based Lukas Biewald's Scikit Learn class# https://github.com/lukas/scikit-class

import arcpyimport pandas as pdfrom sklearn.feature_extraction.text import CountVectorizerfrom sklearn.naive_bayes import MultinomialNB

input_csv = arcpy.GetParameterAsText(0)test_string = arcpy.GetParameterAsText(1)

df = pd.read_csv(input_csv)target = df['is_there_an_emotion_directed_at_a_brand_or_product']text = df['tweet_text']

SkikitLearnDemofixed_text = text[pd.notnull(text)]fixed_target = target[pd.notnull(text)]

count_vect = CountVectorizer()count_vect.fit(fixed_text)counts = count_vect.transform(fixed_text)

nb = MultinomialNB()nb.fit(counts, fixed_target)

# print out our predictionarcpy.AddMessage(nb.predict(count_vect.transform([test_string][0])))

DeeperDive

MultiplePythonsCurrently:

Platform Pythonversion

Desktop Python2.7.x(2.7.10)

Pro Python3.4.x(3.4.3)

MultiplePythonsUpgradecode?

Doit!Youcansupport2+3withoutthatmuchworkStillneedtochangearcpy.mappingtoarcpy.mpwhenmovingfromDesktoptoPro,butnoPythonlanguagelevelchangesneeded....

But…thiscanbecostly.Formanyorganizations,asignificantburden,evenifthelanguagechangesarerelativelysmall.MultiplePythonsis

asolutiontothis.

PythonmigrationforArcGISPro

http://pro.arcgis.com/en/pro-app/arcpy/get-started/python-migration-for-arcgis-pro.htm

ChallengesHavetomakesureyou’rerunningtherightPython(whathappens

whenyoutypepythonatthecommandline?)WorkingtomakethiseasyaspossibleIt’llbeeasytotellinappIsolatedinstallationfixesavarietyofissues

Requiressomeusereducationoverthe“onlyonePythononthebox”model

WhatDoIGetOutoftheBox?

CondacommandandaCondarootPythoninstallNewmodules(e.g.requests)CondaenvironmentwithalloftheArcGISProdependenciesasCondapackages

HowcanIusethis?WealreadyshipyoutheSciPystack—powerfulandoutofthebox,canusetoday(Proand10.4)Canstartusingcondatoday.Minicondaisfullystand-alone,won’taffectyourglobalPython(unlessyoutellitto)Packageyourwork:thisisanopportunitytodistributeit,possiblyincludingcommercialsideaswell.

WhereCanIRunThis?

ArcGISPro1.3WillbethePythoninstall.

Future:UIforinteractionTakeadvantageofmorefeaturesIntegrationwithplatform

fromfutureimport*EffectivelymanagecomplexsoftwaredependencieswithConda.Thousandsofpackagesexisttoday,canintegrateitintoyour

organization’sneeds.

Closing

ThanksEsriCondaTeam:

ContinuumAnalyticsforcreatingandopensourcingConda

RateThisSessionPleasetakeoursurvey,findsessioninappandprovidereview