Using$&$Installing$So:ware$ onOdyssey$ · FAS$Research$Compu1ng$ Using$&$Installing$So:ware$...
Transcript of Using$&$Installing$So:ware$ onOdyssey$ · FAS$Research$Compu1ng$ Using$&$Installing$So:ware$...
FAS Research Compu1ng
Using & Installing So:ware on Odyssey
Bob Freeman, PhD Dir. Research Technology Opera1ons (HBS)
formerly RC Facilitator (FASRC)
[email protected] @DevBizInfoGuy
Slide 1
FAS Research Compu1ng
Goals
Objec&ves • Feel knowledgeable about compute & so:ware environment • Understand and create Lmod module files • Know how to handle different types of programs & scripts • Customize libraries for common scrip1ng languages • Understand basics on so:ware development • Know common prac1ces for compiling so:ware Strategically • “Work smarter, beVer, faster” • ... and to Think Differently • Enable you to be successful with your research!
Slide 2
FAS Research Compu1ng
Overview
1. Environment basics – Login scripts & environment variables
2. Module system – How they work, what they look like, and what they do for you
3. Crea1ng your own modules 4. Java, Python, & Perl applica1ons 5. Upda1ng local modules & packages 6. Version control 7. Using FASRC / precompiled libraries 8. Compiling So:ware
– Methods – Flags, op1ons, and environment variables
– Troubleshoo1ng
Slide 3
FAS Research Compu1ng
Environment basics When you login, Unix executes certain steps for your interac1ve sessions • Startup files are read • Command prompts are set up • Aliases expanded ….
Startup files set up default values for your environment • /etc/profile
• .bash_profile | .bash_login | .profile
• .bashrc
The only things that really need to be in .bash_profile are • environment variables and • their exports and commands • these aren’t defini1ons but actually run or produce out-‐ put when you log in.
Op1on and alias defini1ons should go into the environment file (.bashrc)
Slide 4
FAS Research Compu1ng
Environment basics [bfreeman@sa01 ~]$ cat ~/.bash_profile # .bash_profile # Get the aliases and functions if [ -f ~/.bashrc ]; then
. ~/.bashrc fi # User specific environment and startup programs How shells behave:
Slide 5
This is the Title of the Book, eMatter EditionCopyright © 2009 O’Reilly & Associates, Inc. All rights reserved.
Shell Scripts and Functions | 83
There are many ramifications to using subshells. An important one is that the exportedenvironment variables that we saw in the last chapter (e.g., TERM, EDITOR, PWD)are known in subshells, whereas other shell variables (such as any that you define inyour .bash_profile without an export statement) are not.
Other issues involving subshells are too complex to go into now; see Chapter 7 andChapter 8 for more details about subshell I/O and process characteristics, respec-tively. For now, just bear in mind that a script normally runs in a subshell.
Functionsbash’s function feature is an expanded version of a similar facility in the System VBourne shell and a few other shells. A function is sort of a script-within-a-script; youuse it to define some shell code by name and store it in the shell’s memory, to beinvoked and run later.
Functions improve the shell’s programmability significantly, for two main reasons.First, when you invoke a function, it is already in the shell’s memory; therefore afunction runs faster. Modern computers have plenty of memory, so there is no needto worry about the amount of space a typical function takes up. For this reason, mostpeople define as many commonly used functions as possible rather than keep lots ofscripts around.
The other advantage of functions is that they are ideal for organizing long shellscripts into modular “chunks” of code that are easier to develop and maintain. If youaren’t a programmer, ask one what life would be like without functions (also calledprocedures or subroutines in other languages) and you’ll probably get an earful.
Figure 4-1. Ways to run a shell script
a
source aliceShell: hatter gryphon
c
Shell:
Subshell:
b
Shell:
Subshell:
alice
alice &
hatter gryphon
hatter gryphon
source .bash_profile & more, keeps all sekngs in same context
main shell waits; subshell gets .bashrc only, knows about exports; all is lost a:er exit
same as b, but main shell can keep doing work with other in bkg
Chapter 3: Learning the bash Shell (O'Reilly)
FAS Research Compu1ng
[bfreeman@sa01 ~]$ cat ~/.bashrc # .bashrc # Source global definitions if [ -f /etc/bashrc ]; then
. /etc/bashrc fi # User specific aliases and functions alias l.='ls -d .*' alias lt='ls -alt' alias lthead='ls -alt | head' alias llhead='ll | head' alias which='alias | /usr/bin/which --tty-only --read-alias --show-dot --show-tilde' alias get_interact1='srun --pty --x11=first --mem=1000 -p interact -t 0-6:00 /bin/bash' alias ssqueue='/usr/bin/squeue -o "%.18i %.9P %.8j %.8u %.2t %.10M %.6D %R %C"' alias sacct_ram="sacct --format=JobID,JobName,ReqMem,MaxRSS,Elapsed" alias cd_regal='cd /n/regal/rc_admin/bfreeman' export PATH=$PATH:$HOME/bin export PATH=$PATH:$HOME/apps/parafly/bin source new-modules.sh module load fasrc export EDITOR=nano # setup local Perl environment export LOCALPERL=$HOME/apps/perl export PERL5LIB=$LOCALPERL:$LOCALPERL/lib/perl5:$PERL5LIB export PERL_MM_OPT="INSTALL_BASE=$LOCALPERL" export PERL_MP_OPT="--install_base $LOCALPERL" export PATH="$LOCALPERL/bin:$PATH" module load perl-modules # setup for local Python environment
Environment Basics
Slide 6
command aliases
adding executables to where systems looks
op1ng in to new modules
sekng default editor
personal perl modules
FAS Research Compu1ng
Common Environment Variables
Slide 7
You may run across these in your Unix & cluster endeavors... • For variables that include filesystem loca1on (paths), these are colon (':')
delimited, and usually do not contain a trailing '/'. • Order is important! Pre-‐pending your local paths will supersede system
executables of the same name
Variable Explana&on
PATH Ordered loca1ons that the system looks for programs you wish to execute
HOME Your home folder. Usually not a good idea to change
TEMP TMP TMPDIR
Local directory used for temporary file crea1on
DISPLAY Used to direct X11 GUI data to the appropriate drawing screen
EDITOR The name of the user's preferred text editor. Used by command-‐line programs such as git
LIBRARY_PATH LD_LIBRARY_PATH
List of directories in which library files should be searched for
CLASSPATH List of directories for Java jar files / class files
PWD Your current directory
USER Your account / login name
PAGER Name of the u1lity used to display long text by commands such as man
FAS Research Compu1ng
Overview
1. Environment basics – Login scripts & environment variables
2. Module system – How they work, what they look like, and what they do for you
3. Crea1ng your own modules 4. Java, Python, & Perl applica1ons 5. Upda1ng local modules & packages 6. Version control 7. Using FASRC / precompiled libraries 8. Compiling So:ware
– Methods – Flags, op1ons, and environment variables
– Troubleshoo1ng
Slide 8
FAS Research Compu1ng
Lmod Module System
Slide 9 hVp://fasrc.us/rclmod
So:ware is loaded incrementally using modules, to set up your shell environment (e.g. PATH, BLASTDB, and other environment variables)
Using the Harvard-‐modified, TACC module system Lmod: • New system is s1ll opt-‐in. Strongly suggested reading: hVp://fasrc.us/rclmod
source new-modules.sh # for opt-in folks module load fastqc/1.0.0-fasrc01 # recommended!!
module load fastqc # most recent version module-query fastqc # recommended!! module-query --full-text fastqc # gives more detail
module spider fastqc # find details on software module avail 2>&1 | grep -i fastqc # find titles/defaults
So:ware search capabili1es similar to module-query are also available on the RC Portal!
Module loads best placed in SLURM batch scripts: • Keeps your interac1ve working environment simple • Is a record of your research workflow (reproducible research!) • Keep .bashrc module loads sparse, lest you run into so:ware and library conflicts
FAS Research Compu1ng
Lmod Module System
Slide 10 hVp://fasrc.us/rclmod
LMod: Environmental Modules System
from hVps://www.tacc.utexas.edu/research-‐development/tacc-‐projects/lmod
Provide a convenient way to dynamically change the users' environment through modulefiles (Lua-‐based scrip1ng file). This includes easily adding or removing directories to the PATH environment variable.
A modulefile • contains the necessary informa1on to allow a user to run a par1cular applica1on or provide
access to a par1cular library. dynamically changes environment without logging out and back in. • applica1ons modify the user's path to make access easy. • Library packages provide environment variables that specify where the library and header files
can be found.
Packages can be loaded and unloaded cleanly through the module system. • All the popular shells are supported: bash, ksh, csh, tcsh, zsh. • Also available for perl and python. • It is also very easy to switch between different versions of a package or remove it.
FAS Research Compu1ng
[bfreeman@sa01 ~]$ module load trinity/2.0.6-fasrc01 [bfreeman@sa01 ~]$ which Trinity /n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/Trinity [bfreeman@sa01 ~]$ ll /n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01 total 974 drwxr-xr-x 12 root root 514 May 5 2015 . drwxr-xr-x 7 root root 163 Oct 30 12:51 .. drwxr-xr-x 5 root root 118 May 5 2015 Analysis drwxr-xr-x 4 root root 171 May 5 2015 Butterfly drwxr-xr-x 9 root root 1488 May 5 2015 Chrysalis drwxr-xr-x 2 root root 695 May 5 2015 galaxy-plugin drwxr-xr-x 3 root root 540 May 5 2015 hpc_conf drwxr-xr-x 4 root root 501 May 5 2015 Inchworm -rw-r--r-- 1 root root 1490 May 5 2015 LICENSE -rw-r--r-- 1 root root 1492 May 5 2015 LICENSE.txt -rw-r--r-- 1 root root 1737 May 5 2015 Makefile -rw-r--r-- 1 root root 2113 May 5 2015 modulefile.lua -rw-r--r-- 1 root root 87 May 5 2015 notes drwxr-xr-x 7 root root 1384 May 5 2015 PerlLib -rw-r--r-- 1 root root 105 May 5 2015 README -rw-r--r-- 1 root root 76 May 5 2015 README.md -rw-r--r-- 1 root root 51881 May 5 2015 Release.Notes drwxr-xr-x 12 root root 444 May 5 2015 sample_data -rwxr-xr-x 1 root root 104354 May 5 2015 Trinity drwxr-xr-x 11 root root 620 May 5 2015 trinity-plugins drwxr-xr-x 7 root root 634 May 5 2015 util [bfreeman@sa01 ~]$ pushd !$ # cool: save current dir and use last param of last cmd pushd /n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01 /n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01 ~/source_tarballs ~
Modules: How do they work?(1)
Slide 11
FAS Research Compu1ng
[bfreeman@sa01 2.0.6-fasrc01]$ cat modulefile.lua local helpstr = [[ trinityrnaseq-2.0.6-fasrc01 De-novo assembler from RNA-Seq from Broad Inst. et al. ]] help(helpstr,"\n") whatis("Name: trinityrnaseq") whatis("Version: 2.0.6-fasrc01") whatis("Description: De-novo assembler from RNA-Seq from Broad Inst. et al.") ---- prerequisite apps (uncomment and tweak if necessary) for i in string.gmatch("bowtie/1.1.1-fasrc01 samtools/1.1-fasrc02 perl-modules/5.10.1-fasrc11","%S+") do if mode()=="load" then a = string.match(i,"^[^/]+") if not isloaded(a) then load(i) end end end ---- environment changes (uncomment what is relevant) prepend_path("PATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01") setenv("TRINITY_HOME", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01") --prepend_path("CPATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/include") --prepend_path("FPATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/include") --prepend_path("INFOPATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/info") --prepend_path("LD_LIBRARY_PATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/lib") --prepend_path("LIBRARY_PATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/lib") --prepend_path("LD_LIBRARY_PATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/lib64") --prepend_path("LIBRARY_PATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/lib64") --prepend_path("MANPATH", "/n/sw/fasrcsw/apps/Core/trinityrnaseq/2.0.6-fasrc01/man") ...
Modules: How do they work? (2)
Slide 12
program metadata
we load dependencies for you
info for module-‐query & help
Magic! we change your environment as needed
FAS Research Compu1ng
Modules: Why Use Hierarchies? Use of groupings is important for proper func1oning programs! • Libraries built with one compiler need to be linked with applica1ons with the same compiler
version. • For HPC there are libraries called Message Passing Interface (MPI) that allow for efficient
communica1ng between tasks on a distributed memory computer with many processors. • Parallel libraries and applica1ons must be built with a matching MPI library and compiler.
Instead of using a flat namespace, we can use module hierarchies! • Simple technique because once users choses a compiler and MPI implementa1on, they can
only load modules that match that compiler and MPI implementa1on. • FASRC follow's TACC's conven1on:
$MODULEPATH_ROOT/{Core,Comp,MPI} # /n/sw/fasrcsw/modulefiles
Core: compiled so:warelibraries that should work in all environments Comp: so:ware/libraries 1ed to a specific compiler and version MPI: compiler-‐specific & MPI-‐specific flavors of so:ware/libraries
Slide 13
FAS Research Compu1ng
[bfreeman@sa01 modulefiles]$ module-query raxml ---------------------------------------------------------------------------------------------------- raxml ---------------------------------------------------------------------------------------------------- Description: RAxML (Randomized A(x)ccelerated Maximum Likelihood) is a program for sequential and parallel Maximum-Likelihood-based inference of large phylogenetic trees. It can also be used for postanalyses of sets of phylogenetic trees, analyses of alignments and, evolutionary placement of short reads. It has originally been derived from fastDNAml which in turn was derived from Joe Felsentein’s dnaml which is part of the PHYLIP package. Versions: raxml/8.1.5-fasrc03..................... Comp raxml/8.1.5-fasrc02..................... MPI raxml/7.7.6.bib-fasrc02................. Core raxml/7.3.0-fasrc01..................... Core To find detailed information about a module, enter the full name. For example, module-query raxml/7.3.0-fasrc01
Modules: Hierarchies
Slide 14
FAS Research Compu1ng
[bfreeman@sa01 modulefiles]$ module-query raxml/8.1.5-fasrc02 ------------------------------------------------------------------------------------------------------------ raxml : raxml/8.1.5-fasrc02 ------------------------------------------------------------------------------------------------------------ Description: RAxML (Randomized A(x)ccelerated Maximum Likelihood) is a program for sequential and parallel Maximum-Likelihood-based inference of large phylogenetic trees. It can also be used for postanalyses of sets of phylogenetic trees, analyses of alignments and, evolutionary placement of short reads. It has originally been derived from fastDNAml which in turn was derived from Joe Felsentein’s dnaml which is part of the PHYLIP package. Build comments: Building with SSE3 instructions. Some, but not all compute nodes have AVX. Spec file can be used properly as either Comp or MPI. When building as Comp, the CC line is removed from the Makefile so that the environment value is picked up. This module an be loaded as follows: module load gcc/4.8.2-fasrc01 openmpi/1.8.1-fasrc04 raxml/8.1.5-fasrc02 module load gcc/4.8.2-fasrc01 openmpi/1.10.0-fasrc01 raxml/8.1.5-fasrc02 module load gcc/4.8.2-fasrc01 openmpi/1.8.3-fasrc02 raxml/8.1.5-fasrc02 module load gcc/4.8.2-fasrc01 mvapich2/2.0-fasrc03 raxml/8.1.5-fasrc02 module load gcc/4.8.2-fasrc01 mvapich2/2.0-fasrc02 raxml/8.1.5-fasrc02 module load intel/15.0.0-fasrc01 openmpi/1.8.1-fasrc04 raxml/8.1.5-fasrc02 module load intel/15.0.0-fasrc01 openmpi/1.10.0-fasrc01 raxml/8.1.5-fasrc02 module load intel/15.0.0-fasrc01 openmpi/1.8.3-fasrc02 raxml/8.1.5-fasrc02 module load intel/15.0.0-fasrc01 mvapich2/2.0-fasrc03 raxml/8.1.5-fasrc02 module load intel/15.0.0-fasrc01 mvapich2/2.0-fasrc02 raxml/8.1.5-fasrc02 [bfreeman@sa01 modulefiles]$ module load gcc/4.8.2-fasrc01 openmpi/1.10.0-fasrc01 raxml/8.1.5-fasrc02 [bfreeman@sa01 modulefiles]$ which raxml /n/sw/fasrcsw/apps/MPI/gcc/4.8.2-fasrc01/openmpi/1.10.0-fasrc01/raxml/8.1.5-fasrc02/bin/raxml
Modules: Hierarchies
Slide 15
FAS Research Compu1ng
Modules: Make your own! Why? • Install newer version of open source so:ware than is currently available in central loca1ons • Easily change version of applica1ons or libraries while being developed • BeVer documenta1on for what so:ware is available. It is easy to install a new version of some soDware and place it in your personal PATH and forget about it. At least when it is in a module, it be listed in the loaded modules. Can be done in 3 easy steps 1. Add loca1on of local module files to module path 2. Create the module file with lua code for the appropriate program/package 3. Do the new module load (Voila!)
4. Include the op1onal modules path in your .bashrc login script
Slide 16 hVps://www.tacc.utexas.edu/research-‐development/tacc-‐projects/lmod/advanced-‐user-‐guide
FAS Research Compu1ng
# assumes that program & packages are in $HOME/apps [bfreeman@sa01 ~]$ ll apps/thin_fastq-1.0.0/ total 232 drwxr-xr-x 4 bfreeman rc_admin 46 Jan 26 08:41 . drwxr-xr-x 17 bfreeman rc_admin 932 Jan 27 11:18 .. drwxr-xr-x 3 bfreeman rc_admin 905 Jan 29 10:03 bin drwxr-xr-x 3 bfreeman rc_admin 32 Jan 26 08:41 include drwxr-xr-x 3 bfreeman rc_admin 905 Jan 29 10:03 lib # prepends your path to MODULEPATH environment variable. Can locate these anywhere. $ mkdir $HOME/modulefiles $ module use $HOME/modulefiles # assuming a program thin_fastq at $HOME/apps/thin_fastq-1.0.0/bin $ cd ~/modulefiles $ mkdir thin_fastq $ cd !$ # module file gets home directory from the environment # then uses Lmod 'pathJoin' function to join strings together # to pre-pends this program's location to the user's PATH $ cat > 1.0.0.lua local home = os.getenv("HOME") local pkg = pathJoin(home,"apps/thin_fastq-1.0.0/bin") prepend_path("PATH", pkg) ^D # now load in the new module $ module load thin_fastq # this line will add the local (user) module environment for future logins $ echo 'module use $HOME/modulefiles' >> ~/.bashrc
Modules: Make Your Own!
Slide 17
FAS Research Compu1ng
Overview
1. Environment basics – Login scripts & environment variables
2. Module system – How they work, what they look like, and what they do for you
3. Crea1ng your own modules 4. Java, Python, & Perl applica1ons 5. Upda1ng local modules & packages 6. Version control 7. Using FASRC / precompiled libraries 8. Compiling So:ware
– Methods – Flags, op1ons, and environment variables
– Troubleshoo1ng
Slide 18
FAS Research Compu1ng
Java Programs
Slide 19
FASRC will no longer install Java apps and Python scripts • they will probably not work correctly in read-‐only sec1ons of the cluster • Python scripts use pip-install or python setup.py & doesn’t work with some setups
For Java… • Download the *.jar files or the install files into a home or lab apps/ or bin/ directory • Include the java CLASSPATH statement in your .bashrc, OR • Set up a bash environment variable in your .bashrc • Call the so:ware using the java command, poin1ng to the appropriate rou1ne
cd ~ mkdir –p apps; cd apps wget http://…longURL…/Trimmomatic-0.36.zip unzip Trimmomatic-0.36.zip ln –s Trimmomatic-0.36 trimmomatic export TRIMMOMATIC=$HOME/apps/trimmomatic echo 'export TRIMMOMATIC=$HOME/apps/trimmomatic' >> ~/.bashrc
# in SLURM script or on command line… module load java/1.8.0_45-fasrc01 cd ~/myFASTQdirectory; mkdir trimmed
# minHeap (-Xms) and maxHeap (-Xmx) options are optional but useful in some cases!! java -Xms128m –Xmx4g -jar $TRIMMOMATIC/trimmomatic-0.32.jar SE -threads 1 \ PSG177_TGACCA.fastq.gz trimmed/PSG177_TGACCA.fastq ILLUMINACLIP:TruSeq3-PE.fa:2:40:15 LEADING:3 TRAILING:3 \ SLIDINGWINDOW:4:20 MINLEN:25
hVp://fasrc.us/installsw
FAS Research Compu1ng
Python Programs
Slide 20 hVp://fasrc.us/installsw
For Python we recommend:
• Use the standard module load python/2.7.6-fasrc01 for pulling in default modules • Use the Anaconda environment for customizing modules & versions • Mul1ple custom environments can be set up for home or lab folders (e.g. development or
produc1on code). Check conda op1ons for 'non-‐standard' loca1ons module load python/2.7.6-fasrc01 # for any python version conda create -n ENV_NAME --clone="$PYTHON_HOME” # created at ~/env/ENV_NAME # load in environment source activate ENV_NAME # local environment is default # load in new package conda install MYPACKAGE # replace MYPACKAGE # update a specific package # 'remove' first if you hit update problems conda update MYPACKAGE # other methods after source activate pip install MYPACKAGE python setup.py install MYPACKAGE
FAS Research Compu1ng
R / Python / Perl Libraries
Slide 21 hVp://fasrc.us/installsw
We've already covered Python libraries. R and Perl are much simpler!
R: Use the R_LIBS_USER environment variable… # load R, default packages, & set local install dir (must already exist!) module load R_packages/3.2.0-fasrc01 ... # can put this in .bashrc, but best to do this after R module load export R_LIBS_USER=$HOME/apps/R:$R_LIBS_USER # R install.packages('deplyr') # inside R
For Perl, we can do the same thing! # load Perl, default modules, & set local install dir (must already exist!) module load perl-modules/5.10.1-fasrc11
# can put these in your .bashrc export LOCALPERL=$HOME/apps/perl export PERL5LIB=$LOCALPERL:$LOCALPERL/lib/perl5:$PERL5LIB export PERL_MM_OPT="INSTALL_BASE=$LOCALPERL" export PERL_MB_OPT="--install_base $LOCALPERL" export PATH="$LOCALPERL/bin:$PATH"
# and now do easy, local installs with cpan cpan FASTAParse
FAS Research Compu1ng
Overview
1. Environment basics – Login scripts & environment variables
2. Module system – How they work, what they look like, and what they do for you
3. Crea1ng your own modules 4. Java, Python, & Perl applica1ons 5. Upda1ng local modules & packages 6. Version control 7. Using FASRC / precompiled libraries 8. Compiling So:ware
– Methods – Flags, op1ons, and environment variables
– Troubleshoo1ng
Slide 22
FAS Research Compu1ng
Version Control Version control is a system that records changes to a file or set of files over 1me so that you can recall specific versions later. • Typically used for source code files • In reality you can do this with nearly any type of file on a computer.
Slide 23 from Git SCM book
FAS Research Compu1ng
Installing Your Own Software
Slide 24 hVp://fasrc.us/installsw
• Users can compile so:ware in their /home or /groups directory, where they have permission
• Binaries just require “unzipping” (ie tar -zxvf *.tgz)
• Common compiling libraries are found as modules: gcc/4.8.2-fasrc01 intel/15.0.0-fasrc01 boost/1.59.0-fasrc01
gcc/4.8.2-fasrc01 llvm/3.5.1-fasrc02
gcc/4.8.2-fasrc01 openmpi/1.8.3-fasrc02 gcc/4.8.2-fasrc01 mvapich2/2.0-fasrc03
• Common math libraries are also present but may be compiler-‐specific
gcc/4.8.2-fasrc01 intel-mkl/11.0.0.079-fasrc01
FAS Research Compu1ng
Installing A Binary (Pre-compiled)
Slide 25 hVp://fasrc.us/installsw
• Users can place so:ware in their /home or /groups directory, where they have permission
• Binaries just require “unzipping” (ie tar -‐zxvf .tgz) cd ~ mkdir –p apps; cd apps wget http://…longURL…/myBinarySoftware-1.0.tgz tar -xvf myBinarySoftware-1.0.tgz ls myBinarySoftware-1.0 ln –s myBinarySoftware-1.0 myBinarySoftware # add to .bashrc; assumes all binaries are in top level of this folder echo 'export PATH=$HOME/apps/myBinarySoftware:$PATH' >> ~/.bashrc export PATH=$HOME/apps/myBinarySoftware:$PATH # and now run it! myBinary
FAS Research Compu1ng
Installing From Source Code
Slide 26
• Users can compile so:ware in their /home or /groups directories, where they have permission
• Source requires untaring and (o:en) selec1ng a compiler cd ~ mkdir –p apps; cd apps wget http://…longURL…/myBinarySoftware-1.0.tgz
tar -xvf myBinarySoftware-1.0.tgz ls myBinarySoftware-1.0 ln –s myBinarySoftware-1.0 myBinarySoftware
cd myBinarySoftware less README
module load gcc/4.8.2-fasrc01 ./configure --prefix=$HOME/apps/myBinarySoftware make make install
# add to .bashrc; assumes all binaries are in top level of this folder echo 'export PATH=$HOME/apps/myBinarySoftware:$PATH' >> ~/.bashrc # NB! 's export PATH=$HOME/apps/myBinarySoftware:$PATH
# and now run it! myBinary
FAS Research Compu1ng
Understanding Compiling Code
Slide 27
The GNU Build System, also known as the Autotools, is a suite of programming tools designed to assist in making source code packages portable to many Unix-‐like systems. The GNU Build System makes it possible to build many programs using a two-‐step process: configure followed by make: module load gcc/4.8.2-fasrc01
./configure \
--prefix=$HOME/apps/myBinarySoftware
make
make install
FAS Research Compu1ng
Understanding Compiling Code
Slide 28
FAS Research Compu1ng
Source Compile Considerations
Slide 29
Source Code (.f .F .f95 .f03, .c ) • main code or driver • variables • func1on calls • subrou1ne
Includes / headers (.i or .h) CPATH, C_INCLUDE Libraries • files
• object files ( .o) LIBRARY_PATH • shared object files ( .so) LD_LIBRARY_PATH • archive files ( .a)
• sekngs • ldconfig • LD_LIBRARY_PATH
Compilers • Intel: ifort, icc, icp, mkl, mpif90, ... • GNU: gfortran, gcc • PGI: pgf90, … • Abaso: configure scripts make: Makefiles
FAS Research Compu1ng
Compiling: Configure
Slide 30
./configure sets up compile sekngs & target op1ons based on your system environment -‐-‐prefix= place to store architecture-‐independent binaries ($HOME/apps) -‐-‐exec-‐prefix= place to store architecture-‐dependent binaries -‐-‐bindir= where the executables go ( bin/) -‐-‐sbindir= system admin executables ( /sbin/) -‐-‐includedir= C header files ( include/) -‐-‐libdir= object code libraries ( lib/) ./configure -‐-‐prefix=$HOME/apps/program will have program/bin program/lib etc ./configure -‐-‐prefix=$HOME/apps will have bin/ lib/ include/ etc
Some influen1al environment variables: PYTHON python program CXX C++ compiler command CXXFLAGS C++ compiler flags LDFLAGS linker flags if libraries in a nonstd directory <lib dir>: -‐L$HOME/apps/lib CPPFLAGS C/C++ preprocessor flags if headers in a nonstd directory <include dir>: -‐I$HOME/apps/include CC C compiler command: gcc CFLAGS C compiler flags: -‐O3 CXXCPP C++ preprocessor
FAS Research Compu1ng
Compiling: Make
Slide 31
make is a Unix tool to simplify building program executables from many modules. • reads in rules (list of target entries) from a user created Makefile • run the commands for that given rule • will only re-‐build things that need to be re-‐built
A very simple Makefile all: g++ main.cpp hello.cpp factorial.cpp -o hello install:
cp hello $(PREFIX)/bin
So…
make # runs the first (default) command all make install # runs the install rule
Influen1al environment variables from previous page affect how the Makefile runs
FAS Research Compu1ng
[bfreeman@sa01 source_tarballs]$ tar xvf tophat-2.0.13.tgz [bfreeman@sa01 source_tarballs]$ ls -al tophat-2.0.13 total 1574 drwxr-xr-x 3 bfreeman rc_admin 596 Oct 1 2014 . drwxr-xr-x 9 bfreeman rc_admin 1677 Apr 27 17:20 .. -rw-r--r-- 1 bfreeman rc_admin 38873 Sep 29 2014 aclocal.m4 -rw-r--r-- 1 bfreeman rc_admin 1011 Sep 25 2014 AUTHORS -rw-r--r-- 1 bfreeman rc_admin 7621 Sep 29 2014 ax_boost_base.m4 -rw-r--r-- 1 bfreeman rc_admin 3404 Sep 29 2014 ax_boost_thread.m4 -rw-r--r-- 1 bfreeman rc_admin 0 May 29 2012 ChangeLog -rwxr-xr-x 1 bfreeman rc_admin 43420 Sep 29 2014 config.guess -rw-r--r-- 1 bfreeman rc_admin 3079 Sep 29 2014 config.h.in -rwxr-xr-x 1 bfreeman rc_admin 31743 Sep 29 2014 config.sub -rwxr-xr-x 1 bfreeman rc_admin 274317 Sep 29 2014 configure -rw-r--r-- 1 bfreeman rc_admin 3888 Sep 29 2014 configure.ac -rw-r--r-- 1 bfreeman rc_admin 5206 Sep 25 2014 COPYING -rwxr-xr-x 1 bfreeman rc_admin 15936 Sep 29 2014 depcomp -rw-r--r-- 1 bfreeman rc_admin 9478 May 29 2012 INSTALL -rwxr-xr-x 1 bfreeman rc_admin 9233 Sep 29 2014 install-sh -rw-r--r-- 1 bfreeman rc_admin 1469 May 29 2012 LICENSE -rw-r--r-- 1 bfreeman rc_admin 114 Sep 24 2014 Makefile.am -rw-r--r-- 1 bfreeman rc_admin 19100 Sep 29 2014 Makefile.in -rwxr-xr-x 1 bfreeman rc_admin 11014 Sep 29 2014 missing -rw-r--r-- 1 bfreeman rc_admin 6612 May 29 2012 NEWS -rw-r--r-- 1 bfreeman rc_admin 848 Sep 25 2014 README drwxr-xr-x 4 bfreeman rc_admin 2249 Oct 1 2014 src -rw-r--r-- 1 bfreeman rc_admin 2140 Sep 26 2013 THANKS [bfreeman@sa01 source_tarballs]$ cd !$ # cool trick! last option on prev line [bfreeman@sa01 source_tarballs]$ ./configure # OPTIONS HERE ARE IMPORTANT
Install from Source Code
Slide 32
INSTALL and README present. Make sure you look thru these! No Makefile here, so have to run ./configure
FAS Research Compu1ng
Compiling: Using Libraries Libraries allow you to pull in pre-‐compiled func1ons and code to your programs • Many are already installed on Odyssey: MKL, NetCDF, LAPACK, etc • Should be able to pull in with a simple module load
module load beagle/2.1.trunk-fasrc01
But also libraries that may be part of the OS • /lib, /lib64 • can be pull in via environment variables [bfreeman@sa01 tophat-2.0.13]$ module display beagle/2.1.trunk-fasrc03 ----------------------------------------------------------------------------------------------- /n/sw/fasrcsw/modulefiles/Core/beagle/2.1.trunk-fasrc03.lua: ----------------------------------------------------------------------------------------------- help([[beagle-2.1.trunk-fasrc03 A general purpose library for evaluating the likelihood of sequence evolution on trees ]], [[ ]]) whatis("Name: beagle") whatis("Version: 2.1.trunk-fasrc03") whatis("Description: A general purpose library for evaluating the likelihood of sequence evolution on trees") prepend_path("CPATH","/n/sw/fasrcsw/apps/Core/beagle/2.1.trunk-fasrc03/include") prepend_path("FPATH","/n/sw/fasrcsw/apps/Core/beagle/2.1.trunk-fasrc03/include") prepend_path("LD_LIBRARY_PATH","/n/sw/fasrcsw/apps/Core/beagle/2.1.trunk-fasrc03/lib") prepend_path("LIBRARY_PATH","/n/sw/fasrcsw/apps/Core/beagle/2.1.trunk-fasrc03/lib") prepend_path("BEAGLE_PATH","/n/sw/fasrcsw/apps/Core/beagle/2.1.trunk-fasrc03") prepend_path("PKG_CONFIG_PATH","/n/sw/fasrcsw/apps/Core/beagle/2.1.trunk-fasrc03/lib/pkgconfig")
• or -L and -l flags gcc -ltrhead
gcc -L/lib64
Slide 33
FAS Research Compu1ng
Compiling: CMAKE Tools CMake is an open-‐source, cross-‐pla~orm family of tools designed to build, test and package so:ware. CMake is used to control the so:ware compila1on process using simple pla~orm and compiler independent configura1on files, and generate na1ve makefiles and workspaces that can be used in the compiler environment of your choice. Tips: • it's best to build out of the so:ware tree (ie mkdir build; cd build; cmake ..) • -DCMAKE_INSTALL_PREFIX="$HOME/apps/thisapp" • -DCMAKE_INCLUDE_PATH:STRING="$BOOST_INCLUDE;$ZLIB_INCLUDE" for include paths • -DCMAKE_LIBRARY_PATH:STRING="$BOOST_LIB;$ZLIB_LIB" for lib paths There is a newer version of cmake available as a module • seems to be necessary for most these days. • module load cmake/2.8.12.2-fasrc01
Slide 34
FAS Research Compu1ng
Compiling: BOOST Libraries Boost provides free peer-‐reviewed portable C++ source libraries that work well with the C++ Standard Library. • intended to be widely useful • usable across a broad spectrum of applica1ons • license encourages both commercial and non-‐commercial use.
We aim to establish "exis1ng prac1ce" and provide reference implementa1ons so that Boost libraries are suitable for eventual standardiza1on. Ten Boost libraries are included in the C++ Standards CommiVee's Library Technical Report (TR1) and in the new C++11 Standard. C++11 also includes several more Boost libraries in addi1on to those from TR1. More Boost libraries are proposed for standardiza1on in C++17.
FASRC this as a module that is constantly updated: • module load boost/1.59.0-fasrc01
setenv("BOOST_HOME","/n/sw/fasrcsw/apps/Core/boost/1.59.0-fasrc01") setenv("BOOST_INCLUDE","/n/sw/fasrcsw/apps/Core/boost/1.59.0-fasrc01/include") setenv("BOOST_LIB","/n/sw/fasrcsw/apps/Core/boost/1.59.0-fasrc01/lib") prepend_path("CPATH","/n/sw/fasrcsw/apps/Core/boost/1.59.0-fasrc01/include") prepend_path("LD_LIBRARY_PATH","/n/sw/fasrcsw/apps/Core/boost/1.59.0-fasrc01/lib") prepend_path("LIBRARY_PATH","/n/sw/fasrcsw/apps/Core/boost/1.59.0-fasrc01/lib")
If an autotools build does not give you an opportunity to specify non-‐standard loca1ons for libraries (i.e. there is no -‐-‐boost-‐home or -‐-‐boost-‐lib op1on) • can force it by overriding the CC or CXX variable • e.g. export CC="gcc -I$BOOST_INCLUDE -L$BOOST_LIB"
Slide 35
FAS Research Compu1ng
Compiling: Troubleshooting
Slide 36
FAS Research Compu1ng
Compiling: Take-home Messages • Use your .bashrc file to include aliases and environment variables • Keep it clean and sparse • Organize your home folder as best as possible • Use environment variables where possible • Pay aVen1on to the order of paths in your environment variables • Read the README and INSTALL files!! • Compiles may go faster on /tmp or /scratch on compute nodes • Don't use sudo • Watch the compile logs very carefully • Be sure to use the module/compiler hierarchy consistently • Google is your best friend
– Next in line is Stack Overflow • And then your friendly FASRC person
Slide 37
FAS Research Compu1ng
Gekng Help RC Website & Documenta1on -‐-‐ only authorita1ve source
hVps://rc.fas.harvard.edu/ Submit a 1cket on the portal hVps://portal.rc.fas.harvard.edu/ Best way to help us to help you? Give us...
Descrip1on of problem Addi1onal info (login/batch? par11on? JobIDs?) Steps to Reproduce (1., 2., 3...) Actual results Expected results
OdyBot, for quick-‐fix problems hVp://odybot.org/
38
FAS Research Compu1ng
Research Compu1ng
• Please talk to your peers, and … • We wish you success in your research!
• hVp://rc.fas.harvard.edu • hVps://portal.rc.fas.harvard.edu • @fasrc
• Harvard Informa1cs • @harvardifx
39