Harnessing the Power of Python in ArcGIS Using the Conda...
38
Harnessing the Power of Python in ArcGIS Using the Conda Distribution Shaun Walbridge Clinton Dow
Transcript of Harnessing the Power of Python in ArcGIS Using the Conda...
HarnessingthePowerofPythoninArcGISUsingtheCondaDistributionShaunWalbridge
ClintonDow
https://github.com/scw/conda-devsummit-2017-talk
HighQualityPDF
Conda
GettingtoPackages
WhyPackages?Softwareiscomposedofmanysmallercomponents,oftencalledpackagesorlibraries.It’softenbettertoreusecodethatsolvesaproblemwellratherthanrecreatingitBut,sharingcodeisahardproblem.Doyouhavethesamepackagesofthesameversionsasthedeveloperdid?
PackageManagementforPython
Whynotpip,wheels,virtualenvs?
Don’thandletheharderproblemofsystemdependencies,consideredoutofscopebyPythonpackagers–doesitendupinsite-packages?Packagedevs:OnOSXandLinux,‘easy’togetthedeps!Useasystempackagemanager(e.g.apt,brew,yum)andtheincludedcompiler(e.g.clang,gcc).It’sstillnoteasytomakereproduciblebuilds,andwhataboutthatotherplatform?
WhyConda?
ScientificPythoncommunityidentifiedthattherewasagapnotbeingaddressedbythecorePythoninfrastructure,limitingtheirabilitytogetpackagesintothehandsofusers
Industrystandardbuiltbypeoplewhocareaboutthisspace—ContinuumAnalytics
WhyConda?
Itsolvesthehardproblem:
HandlesdependenciesformanylanguagesBuiltforPythonfirst,butitreallysolvesamuchbroaderinfrastructuralissue.
Gatewaytodatascience—scientific,analytics,integratedsoftwareecosystemfororganizations
Interlude:ReviewingReviews
ReviewingReviews
ReviewingReviewsscore=(ratings.sum(axis=1)+inverse_ratings.sum(axis=1)*5)"""104130.0103430.0103629.0103927.0103826.0103726.0104025.0104224.0103515.0"""#1035,let'sseeiftheyleftusacommentdf_conda.loc[1035]['Comments']
Therewasnorealbasicexplanationofwhatcondadoes…justapackageandenvmanager.
ReviewingReviewsIfyoudon’tagreewithsomeone,trywalkingamile
intheirshoes…
…becausethenyou’llbeamileawayfromthem,andyou’llhavetheirshoes.—TerryPratchett
Fundamentals
FundamentalsConsistentbuildingofpackages(Windows,Linux,MacOS),publicandprivatesharingCrossplatform,andcross-langauge—handlesC/C++,R,Java,Scala,JavascriptandmanymoreUltimatelyrealizedasacollectionoffilesinanarchive,andruleswhichdictatepackagedependenciesOpensource:Esriisusingit,youcanuseitinyourownprojectsforothercontexts
Environments—AcollectionofpackagesandPythoninstalliscalledanenvironment,thebuildingblockformanagingPythonwithCondaFlexiblymakechangeswithoutaffectinginstalledsoftwareCancreatemultipleenvironmentsandswitchseamlessly
Requirements—includeexplicitstateinformation,notjustthepackagename.
WheredoIgetpackages?Condapackagescancomefromavarietyoflocations:
PublicrepositorieshostedonPublicrepositoriesself-hostedPrivaterepositories
OndiskChannels—
AcollectionofpackagesownedbyauserororganizationConfigureCondatolookattheselocations(.condarcfile)
AnacondaCloud
AnacondaEnterprise
Demo:UsingPackages
CondaBasicsActivatingenvironments,acoupleways:
UsetheshortcutsincludedinProManuallyactivatetheenvironment:
cdC:\ArcGIS\bin\Python\Scriptsactivatearcgispro-py3
CondaBasics
Condainfoisthestartingpoint—ittellsyouthestateoftheenvironment.
conda--help
condainfo
CondaBasicscondalist
#packagesinenvironmentatC:\ArcGIS\bin\Python\envs\arcgispro-py3:#colorama0.3.7py35_0defaultscycler0.10.0py35_0defaultsfuture0.15.2py35_0defaultsmatplotlib1.5.3np111py35_0e[arcgispro]esrimpmath0.19py35_1defaultsnetcdf41.2.4py35_0e[arcgispro]esrinose1.3.7py35_1defaultsnumexpr2.6.1np111py35_0e[arcgispro]esrinumpy1.11.2py35_0e[arcgispro]esripandas0.19.0np111py35_0defaultspip8.1.2py35_0defaultspy1.4.31py35_0defaultspyparsing2.1.4py35_0defaultspypdf21.26.0py_0esripytest2.9.2py35_0defaultspython3.5.20defaultspython-dateutil2.5.3py35_0defaultspytz2016.6.1py35_0defaultsrequests2.11.1py35_0defaultsscipy0.18.1np111py35_0e[arcgispro]esrisetuptools27.2.0py35_1defaultssympy1.0py35_0defaultswheel0.29.0py35_0defaults
DeeperDive
CondaPackagingOK,sohowdowemakeanewpackage?
whichdescribestheinstructionstobuildthesoftware
WheredoIgetthecode?Whatarewebuilding,andwhatdoesitdependon?Runcondabuildtocreateapackagefromthisrecipe
Createarecipe
CondaPackagingmeta.yaml
package:name:conda-devsummit-2017-talkversion:"1.0"
source:git_url:https://github.com/scw/conda-devsummit-2017-talk.git
requirements:run:-python-scikit-learn
CondaPackagingBuildthepackage:
Uploadandshare:
condabuildc:\example\mypackage
anacondauploadmypackage
Demo:CondaPackaging
PythonPackageManager
HowcanIusethis?WealreadyshipyoutheSciPystack—powerfulandoutoftheboxinallproductsCondacommandandaCondarootPythoninstallNewmodules(e.g.requests),environmentwithPro
PythonPackageManagerinProGetpackages,expandyourpossibilityspacePackageyourwork:thisisanopportunitytodistributeit
WhereCanIRunThis?
ArcGISPro1.3CondaisthePythoninstall,includedforall
ArcGISPro1.4PythonPackageManagerPython3.5withcurrentpackageset
ArcGISEnterprise10.5ArcGISAPIforPython
fromfutureimport*ArcGISPro2.0:
ManageenvironmentsandchannelsResettableenvironment“μConda”
Resources
OtherSessionsWeds4:00PM,SanJacinto
Weds4:00,DemoTheater1
Weds4:30PM,DemoTheater1
Thurs10:30AM,MesquiteG-H
GettingDataSciencewithRandArcGIS
DeployingYourGeoprocessingToolsasPythonModules
PythonPackageManagementUsingConda
ContinuumAnalytics:ExploringContinuumAnalytics’Open-SourceOfferings
Condavs…Name Means Included?
Thecommanditself ✓AminimumsetofPythonpackagestobuildandrunConda.
✓
Adistribution200+packagesandrunConda
Self-hosted,distributedandHPCadditions
Conda
Miniconda
Anaconda
AnacondaWorkgroup
ResourcesCondaCheatsheetAnaconda.orgCondaRecipesHarnessingthePowerofPythoninArcGISUsingtheCondaDistributionUnderstandingCondabyJakeVanderplasSecurityupdatesforPythondepdendencies
Closing
ThanksEsriCondaTeam:
ContinuumAnalyticsforcreatingandopensourcingConda
RateThisSessioniOS,Android:Feedbackfromwithintheapp
Bewarnedthatwemayincorporatefeedbackintonextyear’ssession
