Installing soware on HPC - USC · 4/14/17 1 Before we begin § Best to have X display forwarding...

4/14/17

1

Before we begin

§  BesttohaveXdisplayforwardingenabled–  OnMac,downloadandinstallXQuartzfromwww.quartz.org–  OnPC,checkthatXforwardingisenabledinyourclient

§  Logintohpc-login2orhpc-login3with-XopKon–  Gotoyour/stagingdirectoryandcreateworkshopspace$  cd/staging/<project>/<user>$  mkdirworkshop$  cdworkshop

§  Slidesandfilesusedarehere--downloadslidestofollowalong–  /home/rcf-proj/workshop/handouts/–  /home/rcf-proj/workshop/installing_soQware/

Installingso+wareonHPC(Handoutsavailableat/home/rcf-proj/workshop/handouts)

ErinShawandCesarSul

AdvancedCyberinfrastructureResearchandEducaFonFacilitaFon

USCCenterforHigh-PerformanceCompuFng

4/14/17

2

HPC Facilitation

§  Requestassistance–  [email protected](emailagain!)–  Drop-intoofficehours•  UPC:everyTuesday,2:30p(LVL3M)•  HSC:AJerworkshops,4:30(NML203)

–  RequestaconsultaKon(requestlinkisonwebsite)

§  Learnmore!–  Webpagesonlineath[ps://hpcc.usc.edu–  A[endaworkshop(whenscheduled)•  UPC:Fridays,2:30p(VPD2016)•  HSC:Wednesdays,1:30(MCA249)

Outline

IntroducKonSe_ngyourenvironmentPrecompiledbinaryCompilingsourcecodeCompilingsourcecodewithdependenciesInstallingfromGitHubInstallingPythonpackagesInstallingRlibrariesLicensedsoQwareOpKmizedcompiling

4/14/17

3

Introduction

§  InstallingandrunningsoQwaretypicallyrequiresknowledgeofthecomputeritwillrunon,atsomelevel(varies)

§  Computersdifferinsomeofthefollowingways:–  Computerarchitecture(instrucKonset,logic)–  OperaKngsystem(Linux,Windows,MacOSX)

§  Forexample,HPCcomputersfeature:–  Intel64bitprocessors(x86_64)–  CentOS7.xoperaKngsystems•  CentOSisaderivaTveofRedHatEnterpriseLinux•  LinuxisaderivaTveofUnixforpersonalcomputers

Introduction

§  InstallingsoQwarecanbequickandpainless–  WithprecompiledbinariesforyourspecificoperaKngsystems,for

example,itcanbeaseasyasunzippingafile§  Orneither

–  IfyouhavetocompilethesoQwareyourselfusingcompilers,linkers,Makefiles,externallibraries,etc.

§  Orevenworse...–  It'sfromanacademiclabfrom1999andrequiresoldversionsof

mulKplelibrarieswhichhavemulKpledependencies!

4/14/17

4

Introduction

§  SoQwarecanbeinstalledgloballyorlocally§  HPCuserswillperformlocal,or"user”,installs

–  SoQwareinstalledto'local'folders•  /home/rcf-proj/<project>/<user>•  Alwaysworkinprojectdirectory!

–  Requireswriteprivileges,whichyouhaveinyourowndirectories–  SoQwarewillbeaccessiblebyyou,evenoncomputenodes

Introduction

§  Globallymeanssystem-wide,wheresoQwareisinstalledtosystemlocaKonslike/usr/binor/usr/local

–  Globalinstallsrequirerootprivileges,andonlysystemsadministratorshaverootprivilegesonHPC

§  Sosystem-wideinstallaKonswillnotworkonHPCOperaKngSystem Command

GNU/Linux

Architecture YouwillseetheseinthedocumentaKon

Debian/Ubuntu apt install

Fedora yum install

Gentoo emerge install

MacHomebrew brew install

MacPorts port install

BSDFreeBSD pkg install

OpenBSD pkg_add

4/14/17

5

Outline


Setting your environment

§  Environmentvariables!–  Arenamedstringvariables,e.g.•  HOME=/home/rcf-proj/erinshaw,USER=erinshaw•  mydata=/home/rcf-proj/erinshaw/ess/data

–  ByconvenKon,system-setenvsareallUPPERCASEanduser-setenvsareatleastparKallylowercase

–  Thevalueisaccessedbyprecedingthevariablewitha”$”•  Type:echo$HOME,echo$SHELL,echo$mydata•  Type:env(toprintallenvs)

§  Envscanbesetbythesystem,programs,andusers;andareavailablesystem-wide,oncommandline,andinsidecode

4/14/17

6


§  $PATHcontainsalistofdirectories(locaKons)–  ThelocaKonsaresearchedsequenKallybytheshell,

whenyoutypeacommand,tofindthespecifiedcommand•  Thefirstmatchwillbeexecuted•  Whenthereisnomatch,theshellwillproduceanerror

$ python $ which python /usr/bin/python $ echo $PATH /usr/lib64/qt-3.3/bin:/opt/moab/bin:/opt/mam/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin $ python3 -bash: python3: command not found


§  Changingthe$PATHvariable,by‘sourcing’(running/execuKng)HPC’ssetup.shscriptchanges,willchangewhichversionofpythonisexecuted

§  Whyhasthe/usr/uscversionofpythonbeenselected?

$ source /usr/usc/python/2.7.8/setup.sh $ echo $PATH /usr/usc/python/2.7.8/bin:/usr/lib64/qt-3.3/bin:/opt/moab/bin:/opt/mam/bin:/usr/local/bin:/usr/bin:/usr/local/sbin:/usr/sbin

$ which python /usr/usc/python/2.7.8/bin/python

4/14/17

7


§  Don'taddenvironmentvariablestoshellfileslike.bashrc,.bash_profile,.login,or.profile!

–  ThewrongenvironmentvariablescouldleadtoconflicKngse_ngs–  You'llneedtobeableto'turnoff'environmentvariables/se_ngs–  ThishappensalltheKme!


§  BesttocreateenvironmentsetupscriptsthatiniKalizethese_ngsyouneed

–  e.g.,env.sh(wewillbecreaKngthese)

§  EspeciallyifyouaredoingworkformulKpleprojects§  KeeptrackofhowsoQwareisinstalled

–  Youmighthavetodoitagain–  Youmighthavetoreconfigureit–  Environmentse_ngscouldinfluenceyourresults

§  Youwantyourresultstobereproducible

4/14/17

8

Outline


Precompiled binary

Example:XaoSStep1:AcquiretarballStep2:Extract(x=extract,v=verbose,f=file)Step3:Setyourenvironment(addsanewlocaKontoyourpath)Step4:TestinstallaKon(requiresthatX11forwardingbeenabled)

$ cp /home/rcf-proj/workshop/installing_software/xaos-3.6-bin.tar.gz .

$ tar xvf xaos-3.6-bin.tar.gz

$ export PATH=/staging/<prj>/<usr>/workshop/xaos/bin:${PATH}

$ xaos

4/14/17

9

Outline


Compiling (or not)

§  Scriptscanbeeditedandrunwithoutformallycompiling–  Someprograms(bash,perl)interpretthetextinthefiles*–  TheinterpretersthemselvesmustbecompiledforeachOS

§  AllHPCresearcherswriteshellscripts(pbsfiles)§  Manysystemadministratorsuseperlscripts

myprogram.shmyprogram.plmyprogram.js

bashmyprogram.shperlmyprogram.pljavascript myprogram.js

4/14/17

10

Compiling to bytecode

§  Codecanbecompiledto“bytecode”andrunbyavirtualmachine(theVMsthemselvesarecompiledforeachOS)

§  GATKtools,forexample,arewri[eninjava§  ManyHPCresearchersareusingpython

myprogram.javamyprogram.py

javamyprogram[.class]pythonmyprogram[.pyc]

myprogram.classmyprogram.pyc

Compile

Compiling to machine code

§  C/C++andFortranprogramsarecompiledandassembled–  Theoutputismachine-executableobjectcode

§  ManyHPCuserswritetheirowncodeinC,C++,orFortran

myprogram.cmyprogram.f(sourcecode)

myprogram(executableprogram)

myprogram.o(objectfile)

Preprocess&Compile

Link&Assemble

x86_lib.somath_lib.so

(sharedobjectfiles/libraries)

4/14/17

11

HPC Compilers

§  HPCsupportsopensourceandcommercialcompilersforC,C++andFortranprogramminglanguages

Vendor C/C++ Fortran Path

GNU(opensource) gcc/g++ gfortran /usr/bin

/usr/usc/gnu/gcc

Intel* icc/icpc ifort /usr/usc/intel

PGI* pgcc/pgc++ pgfortran /usr/usc/pgi

OpenMPI** mpicc/mpicxx mpifort /usr/usc/openmpi

*Commercialcompilers**MPIcompilersare"wrappers"thatcanuseGNU/Intel/PGI

HPC Compilers

§  AtypicalcompilecommandforCcodelookslike

§  Wheretheenvswerepredefinedlike$ gcc ${CCFLAGS} source.c ${CPPFLAGS} ${LDFLAGS} -o binary_name

$ CCFLAGS='-g -Wall -O3' $ CPPFLAGS='-I/path/to/include' $ LDFLAGS='-L/path/to/lib -lgsl -lgslcblas –lm’

VariableName Whatisit?CCFLAGS FlagstopasstheCcompiler

CPPFLAGS WheretheCpreprocessorcanfindinclude(.h)files

LDFLAGS Whichlibraries(.so,.afiles)touseandwherethelinkercanfindthem

4/14/17

12

With configure/make

§  ApplicaKonsmayhavehundredsoffilesandsomanuallytypingcompileandlinkcommandsisnotfeasible

§  SoQwarebuilduKliKes,e.g.gnu’sautotools,areusedforthis

Makefile(finalcompileandlinkrules)

myprogram(executable)

run‘make’

run‘makeinstall’

myprogram(installed)

configure(pathstoincludefilesandlibrariesareaddedbyyouorsearchedfor)

run‘configure’

Makefile.in(iniKalMakefile)

HPC Libraries

§  ScienKficsoQwareusuallyrequireslowlevelmathlibraries–  BLAS(BasicLinearAlgebraSubprograms)isthede-factostandardfor

linearalgebralibrariesforCandFortran–  Setoflow-levelrouKnesforperformingcommonlinearalgebra

operaKons,suchasvectoraddiKonandmatrixmulKplicaKon

§  HPCmaintainsopKmizedBLASlibrariesforuserinstalls–  OpenBLAS,IntelMKL–MathKernelLibrary

§  HPC-installedsoQwarein/usr/usc/iscompiledwithOpenBLAS–  NumPy,Matlab,R,MathemaKca

4/14/17

13

Compiling source code

Example:XaoSStep1:AcquiretarballStep2:ReadinstalldirecKonsStep3:RunconfigurescriptStep4:Runmakefile

$ wget /home/rcf-proj/workshop/installing_software/xaos-3.6.tar.gz . $ tar xvf xaos-3.6.tar.gz $ cd xaos

$ ./configure --prefix=/staging/<prj>/<usr>/workshop/XaoS

$ make $ make install

$ less INSTALL


Step5:Setyourenvironment(Putthisinascript)•  edit(nano),save(^o)andexit(^x)

$ nano env.sh #!/bin/bash PATH=/staging/<prj>/<usr>/workshop/XaoS/bin:${PATH} MANPATH=/staging/<prj>/<usr>/workshop/Xaos/share/man:${MANPATH} export PATH MANPATH $ source env.sh

4/14/17

14


§  Step6:TestinstallaKon–  RunXaoS(NeedX11forwardingenabled)–  Clickonpartoftheimagetozoomin

$ xaos

TroubleshootingTypeofproblem Whatdoesitmean Howtofixit

PermissiondeniedRead-onlyfilesystem

Youaretryingtoinstallsomewhereyoudon’thaveaccessto.

InstalltoalocaKonwhereyouhave“writepermission”ie,./configure--prefix=path/to/dir

Library(.soor.a)filenotfoundUndefinedreferenceto...Cannotfindsharedobjectfile

Yourprogramuseslibraries(3rdpartycode)thattheinstallercan’tfind.

SetyourlibrarysearchpathLDFLAGS=-L/path/to/lib./configure

Header(.h)filesnotfoundUndefinedreferenceto...

Yourprogramusesheaders(3rdpartycode)thattheinstallercan’tfind

Setyour“Include”pathCPPFLAGS=-I/path/to/include./configure

4/14/17

15

Outline


Compiling source with dependencies

§  Example:NetCDFrequiresHDF5Step0:InstallHDF5(It'salreadyin/usr/usc/hdf5)Step1:DownloadtarballStep2:untar,cdtodirectoryandreadinstalldirecKonsStep3:Runconfigurescript

$ wget https://github.com/Unidata/netcdf-c/archive/v4.4.1.tar.gz

$ 'CC=mpicc CPPFLAGS=-I/usr/usc/hdf5/1.8.18/parallel/include LDFLAGS=-L/usr/usc/hdf5/1.8.18/parallel/lib $./configure --prefix=/path/to/install/dir'

4/14/17

16

Step4:RunmakefileStep5:Createanenvironmentscript

$ make $ make install

$ nano env.sh #!/bin/bash source /usr/usc/hdf5/default/setup.sh PATH=/path/to/install/dir/bin:${PATH} LD_LIBRARY_PATH=/path/to/install/dir/lib:${LD_LIBRARY_PATH} export PATH LD_LIBRARY_PATH

Outline


4/14/17

17

Installing from GitHub

Installing from GitHub

Step1:DownloadsourcecodeStep2:CompilesourceStep3:Setenvironment

$ git clone <paste_project_url_here>

$ ./configure --prefix=/path/to/install/dir $ make $ make install

4/14/17

18

Outline


Installing Python packages

§  Don'tforgettosourcetheversionofpythonyouwanttouse

§  Tocheckwhatpackagesareavailableusethecommand§  Python’sdefaultinstallaKonlocaKonis${HOME}/.local

–  Youhaveonly1GBdiskquotaand100,000filequotainyourhomedirectoryandapythonpackageinstallcaneasilybelarger

–  SoluKonistocreateasymboliclink,or'symlink’,whichisapointer(named.local)toanotherdirectorywithmorediskspace

$ source /usr/usc/python/2.7.8/setup.sh

$ pip freeze

4/14/17

19

Initial Python package

§  Createadirectoryoutsideofyourhomedirectory,forstoringpythonpackages,thencreateasymboliclinktoitcalled.local

§  Installpackage(bashshell)§  SomeKmesyou’llneedtoinstallthelatestversionofapackage

thatisalreadyinstalled

$ source /usr/usc/python/<version>/setup.sh $ pip install <package_name> --user

$ cd ~ $ mkdir -p /staging/<prj>/<usr>/python_packages $ ln -s /staging/<prj>/<usr>/python_packages .local

$ pip freeze $ pip install <package_name> --upgrade --user

Dependencies for Python packages

§  SomepackagesarePythonwrappersforC/C++libraries§  Theinstallerneedstoknowwheretheselibrariesare§  Theh5pypackageisoneexample

§  Youmighthavetodownloadthepackagetarballandeditsomefileslikesetup.py

$ HDF5_DIR=/path/to/hdf5 $ HDF5_VERSION=X.Y.Z $ CC="mpicc" $ pip install h5py

4/14/17

20

Outline


Installing R packages

§  R’sdefaultinstallaKonlocaKonis${HOME}/R–  Youhaveonly1GBdiskquotaand100,000filequotainyourhome

directoryandapythonpackageinstallcaneasilybelarger–  SoluKonistocreateasymboliclink,or'symlink’,whichisapointer

(namedR)toanotherdirectorywithmorediskspace

§  Createadirectoryoutsideofyourhomedirectory,forstoringpythonpackages,thencreateasymboliclinktoitcalled“R”

$ cd ~ $ mkdir -p /staging/<project>/<user>/R_packages $ ln -s /staging/<project>/<user>/R_packages R

4/14/17

21

Installing R packags

§  SourcetheversionofRyouwanttouseandstartR

§  Installpackagesyntax(youmayhavetospecifypath)

§  Thenloadthelibrarywhenyouwanttouse

> install.packages('<package_name>') > install.packages('<package_name>", lib="/path/to/packages") > library (<package_name>, lib.loc="/path/to/packages")

$ source /usr/usc/R/default/setup.sh $ R

> library('<package_name>’) > library('<package_name>’, lib.loc="/path/to/packages")


§  Example

§  Displayslist,orwindowonrightif(-Xenabled)

–  SelectUSA(CA1)asdownloadsite

> install.packages("gplots") Warning in install.packages("gplots") : 'lib = "/auto/usc/R/3.3.1/lib64/R/library"' is not writable Would you like to use a personal library instead? (y/n) y Would you like to create a personal library ~/R/x86_64-pc-linux-gnu-library/3.3 to install packages into? (y/n) y --- Please select a CRAN mirror for use in this session --- also installing the dependencies ‘bitops’, ‘gtools’, ‘gdata’, ‘caTools’

4/14/17

22


§  YoucandisplaypackagesthatareinstalledintheUSCversionsofR,orthatyouhaveinstalled.

–  ThisisnotanopKmalwaytoviewpackages–  Be[erifyoucancheckonepackageataKme

§  Don’tforgettoloadthelibrary!

> installed.packages(lib.loc=”/usr/usc/R/default/lib64/R/library”) > installed.packages(lib.loc=”~/R/x86_64-pc-linux-gnu-library/3.3”)

> system.file(package=”gplots”)

> library(“gplots”) > library("gplots", lib.loc="~/R/x86_64-pc-linux-gnu-library/3.3”)


§  DisplayyourlibrarypathsinRwith.libPaths()

§  YoucanalsosetR’slibrarypathwiththeR_LIBS_USERenvironmentvariableorin.Rprofile

–  e.g.,addlinetoyourenv.shorPBSscript

–  e.g.createan.Rprofilefileinyourhomedirectoryandaddpath

$ export R_LIBS_USER=~/R/x86_64-pc-linux-gnu-library/3.3:${R_LIBS_USER}

$ touch .Rprofile $ nano .Rprofile libPaths( c(.libPaths(), "~/R/x86_64-pc-linux-gnu-library/3.3”))

>.libPaths() [1] "/auto/rcf-proj/ess/erinshaw/homeR/x86_64-pc-linux-gnu-library/3.3" [2] "/auto/usc/R/3.3.1/lib64/R/library"

4/14/17

23

Dependencies for R packages

§  SomepackagesareRwrappersforC/C++libraries–  Theinstallerneedstoknowwheretheselibrariesare–  Youmighthavetodownloadthepackagetarballandeditsomefiles

§  YoucansetcompilaKonenvironmentvariableslikeLDFLAGSinthefile${HOME}/.R/Makevars

Outline


4/14/17

24

Licensed software

§  HPCcomputenodesliveinaprivatenetwork–  Thereisnopublicinternetaccess

§  WerecommendyougetafloaKnglicense–  HPCwillhostthelicenseserver–  Computenodeswillhaveaccesstothisserver–  HPCcansetrestricKonssoonlyyouoryourusergroupcanuseit–  Youwillbeinchargeofmaintainingthelicense

§  TherearesituaKonswhenlicensesmustresideoutsideHPC–  MixeduselicensesfordesktopsandHPC–  HPCcanaccommodatethesesituaKons,too

Outline


4/14/17

25

Optimized Compiling

§  It'spossibletoincreaseperformancethroughcompilaKon–  UseopKmizedlibrarieslikeFFTW,IntelMKL,OpenBLAS...–  Useacommercialcompiler–  Intelallowsyoutocompileforspecificmachinearchitecture–  ThebinarydetectsthebestinstrucKonsetbasedonwhichnodeit's

runningon

§  HPCmaintainedsoQwareiscompiledthisway§  Formoredetails,[email protected]

Thank you for attending!

Questions? ([email protected])

Installing soware on HPC - USC · 4/14/17 1 Before we begin § Best to have X display forwarding...

Documents

Transcript of Installing soware on HPC - USC · 4/14/17 1 Before we begin § Best to have X display forwarding...