HarnessingthePowerofPythoninArcGISUsingtheCondaDistributionShaunWalbridge

ClintonDow

https://github.com/scw/conda-devsummit-2017-talk

HighQualityPDF

Conda

GettingtoPackages

WhyPackages?Softwareiscomposedofmanysmallercomponents,oftencalledpackagesorlibraries.It’softenbettertoreusecodethatsolvesaproblemwellratherthanrecreatingitBut,sharingcodeisahardproblem.Doyouhavethesamepackagesofthesameversionsasthedeveloperdid?

PackageManagementforPython

Whynotpip,wheels,virtualenvs?

Don’thandletheharderproblemofsystemdependencies,consideredoutofscopebyPythonpackagers–doesitendupinsite-packages?Packagedevs:OnOSXandLinux,‘easy’togetthedeps!Useasystempackagemanager(e.g.apt,brew,yum)andtheincludedcompiler(e.g.clang,gcc).It’sstillnoteasytomakereproduciblebuilds,andwhataboutthatotherplatform?

WhyConda?

ScientificPythoncommunityidentifiedthattherewasagapnotbeingaddressedbythecorePythoninfrastructure,limitingtheirabilitytogetpackagesintothehandsofusers

Industrystandardbuiltbypeoplewhocareaboutthisspace—ContinuumAnalytics

WhyConda?

Itsolvesthehardproblem:

HandlesdependenciesformanylanguagesBuiltforPythonfirst,butitreallysolvesamuchbroaderinfrastructuralissue.

Gatewaytodatascience—scientific,analytics,integratedsoftwareecosystemfororganizations

Interlude:ReviewingReviews

ReviewingReviews

ReviewingReviewsscore=(ratings.sum(axis=1)+inverse_ratings.sum(axis=1)*5)"""104130.0103430.0103629.0103927.0103826.0103726.0104025.0104224.0103515.0"""#1035,let'sseeiftheyleftusacommentdf_conda.loc[1035]['Comments']

Therewasnorealbasicexplanationofwhatcondadoes…justapackageandenvmanager.

ReviewingReviewsIfyoudon’tagreewithsomeone,trywalkingamile

intheirshoes…

…becausethenyou’llbeamileawayfromthem,andyou’llhavetheirshoes.—TerryPratchett

Fundamentals

FundamentalsConsistentbuildingofpackages(Windows,Linux,MacOS),publicandprivatesharingCrossplatform,andcross-langauge—handlesC/C++,R,Java,Scala,JavascriptandmanymoreUltimatelyrealizedasacollectionoffilesinanarchive,andruleswhichdictatepackagedependenciesOpensource:Esriisusingit,youcanuseitinyourownprojectsforothercontexts

Environments—AcollectionofpackagesandPythoninstalliscalledanenvironment,thebuildingblockformanagingPythonwithCondaFlexiblymakechangeswithoutaffectinginstalledsoftwareCancreatemultipleenvironmentsandswitchseamlessly

Requirements—includeexplicitstateinformation,notjustthepackagename.

WheredoIgetpackages?Condapackagescancomefromavarietyoflocations:

PublicrepositorieshostedonPublicrepositoriesself-hostedPrivaterepositories

OndiskChannels—

AcollectionofpackagesownedbyauserororganizationConfigureCondatolookattheselocations(.condarcfile)

AnacondaCloud

AnacondaEnterprise

Demo:UsingPackages

CondaBasicsActivatingenvironments,acoupleways:

UsetheshortcutsincludedinProManuallyactivatetheenvironment:

cdC:\ArcGIS\bin\Python\Scriptsactivatearcgispro-py3

CondaBasics

Condainfoisthestartingpoint—ittellsyouthestateoftheenvironment.

conda--help

condainfo

CondaBasicscondalist

#packagesinenvironmentatC:\ArcGIS\bin\Python\envs\arcgispro-py3:#colorama0.3.7py35_0defaultscycler0.10.0py35_0defaultsfuture0.15.2py35_0defaultsmatplotlib1.5.3np111py35_0e[arcgispro]esrimpmath0.19py35_1defaultsnetcdf41.2.4py35_0e[arcgispro]esrinose1.3.7py35_1defaultsnumexpr2.6.1np111py35_0e[arcgispro]esrinumpy1.11.2py35_0e[arcgispro]esripandas0.19.0np111py35_0defaultspip8.1.2py35_0defaultspy1.4.31py35_0defaultspyparsing2.1.4py35_0defaultspypdf21.26.0py_0esripytest2.9.2py35_0defaultspython3.5.20defaultspython-dateutil2.5.3py35_0defaultspytz2016.6.1py35_0defaultsrequests2.11.1py35_0defaultsscipy0.18.1np111py35_0e[arcgispro]esrisetuptools27.2.0py35_1defaultssympy1.0py35_0defaultswheel0.29.0py35_0defaults

DeeperDive

CondaPackagingOK,sohowdowemakeanewpackage?

whichdescribestheinstructionstobuildthesoftware

WheredoIgetthecode?Whatarewebuilding,andwhatdoesitdependon?Runcondabuildtocreateapackagefromthisrecipe

Createarecipe

CondaPackagingmeta.yaml

package:name:conda-devsummit-2017-talkversion:"1.0"

source:git_url:https://github.com/scw/conda-devsummit-2017-talk.git

requirements:run:-python-scikit-learn

CondaPackagingBuildthepackage:

Uploadandshare:

condabuildc:\example\mypackage

anacondauploadmypackage

Demo:CondaPackaging

PythonPackageManager

HowcanIusethis?WealreadyshipyoutheSciPystack—powerfulandoutoftheboxinallproductsCondacommandandaCondarootPythoninstallNewmodules(e.g.requests),environmentwithPro

PythonPackageManagerinProGetpackages,expandyourpossibilityspacePackageyourwork:thisisanopportunitytodistributeit

WhereCanIRunThis?

ArcGISPro1.3CondaisthePythoninstall,includedforall

ArcGISPro1.4PythonPackageManagerPython3.5withcurrentpackageset

ArcGISEnterprise10.5ArcGISAPIforPython

fromfutureimport*ArcGISPro2.0:

ManageenvironmentsandchannelsResettableenvironment“μConda”

Resources

OtherSessionsWeds4:00PM,SanJacinto

Weds4:00,DemoTheater1

Weds4:30PM,DemoTheater1

Thurs10:30AM,MesquiteG-H

GettingDataSciencewithRandArcGIS

DeployingYourGeoprocessingToolsasPythonModules

PythonPackageManagementUsingConda

ContinuumAnalytics:ExploringContinuumAnalytics’Open-SourceOfferings

Condavs…Name Means Included?

Thecommanditself ✓AminimumsetofPythonpackagestobuildandrunConda.

✓

Adistribution200+packagesandrunConda

Self-hosted,distributedandHPCadditions

Conda

Miniconda

Anaconda

AnacondaWorkgroup

ResourcesCondaCheatsheetAnaconda.orgCondaRecipesHarnessingthePowerofPythoninArcGISUsingtheCondaDistributionUnderstandingCondabyJakeVanderplasSecurityupdatesforPythondepdendencies

Closing

ThanksEsriCondaTeam:

ContinuumAnalyticsforcreatingandopensourcingConda

RateThisSessioniOS,Android:Feedbackfromwithintheapp

Bewarnedthatwemayincorporatefeedbackintonextyear’ssession