Using theEaStCHEM Research

Computing Facility

An Introduction to Linux, SLURM, CYGWINand many other cryptic acronyms

Herbert Früchtl

OverviewThe EaStCHEM Research Computing FacilityConnect to a Linux computer from Windows using

CYGWIN Installation UseIntroduction to Linux Basic Linux commands Editing with Emacs and vi Transfering files between desktop and Linux serverRunning calculations through SLURM or SGE A typical job script How to submit and monitor jobsRules of useSecurity and privacy

Current St Andrews Clusters

wardlawwith Maths & Astronomy

290 x 12-core150 TB disk space

Accessible to allSt Andrews researchers

obelix

7 x 12 + 1 x 16 cores18 TB disk space

computationalchemistry group

knox

18 x 8 cores3 TB disk space

Bühl group andcollaborators

chpc-hf63-3

6 x 8 cores2 TB disk space

Früchtl group andcollaborators

kennedywith Astronomy

54 x 32-core250 TB disk space

Accessible to allSt Andrews researchersPaid-for access possible

The St Andrews Cluster Wardlaw

Molecular QuantumChemistry

GaussianNWChemGAMESS-UKGAMESS-USORCADALTONMOLPROTurbomole

Application Areas and Programs

Plane Wave DFTCASTEPVASPCPMDNWChemPWscf

Molecular MechanicsGULPNWChemDL_POLY

Other Periodic MethodsWien2kCRYSTALSIESTACP2K

Connecting from Windows:XMing and PuTTY

InstallationXMing:

Download fromhttps://sourceforge.net/projects/xming/and install(in case of graphics problems later, get Xming-mesa fromhttp://www.straightrunning.com/XmingNotes/)

PuTTY

Download Windows installer fromhttp://www.chiark.greenend.org.uk/~sgtatham/putty/download.htmland execute it

Starting PuTTY Terminal

Start Xming (runs in

background)

Start PuTTY

Make sure X11 forwardingis enabled

Can save session settingsand recall

Specify host in Sessionwindow

Start terminal

File TransferUsing cygwin (or Linux) command line:

sftp <user>@wardlaw10.mcs.st-andrews.ac.uklogin, password…put <file> PC -> wardlawget <file> wardlaw -> PCexit

Alternative: WinSCP

http://winscp.net/eng/index.php

graphical user interface (drag and drop)

Convert text files (newline character differs between UNIX and DOS/Windows)

dos2unix <file> Windows -> Linux both commands should workunix2dos <file> Linux -> Windows on cluster and in CYGWIN

Not doing this can lead to weird errors!

Creating and Using ssh KeysOn Linux, Mac or Cygwin:

Enter commandssh-keygen

Accept all default filenames

Choose (and remember) passphrase when prompted

Email file ~/.ssh/id_rsa.pub to system administrator.(In Cygwin, this is C:\cygwin64\home\<username>\.ssh\id_rsa.pub)

Do NOT send the private key (id_rsa without suffix)

Creating ssh Keys with PuTTYgen

PuTTYgen • Start PuttyGEN (in PuTTYfolder in start menu).

• Select "Generate"

• Move mouse randomlythrough grey area in windowuntil finished.

• Save private key. You willneed this to connect.

• Copy long string in upperwindow into text file.

• Email this file to sysadmin.

Using ssh Key with PuTTY

PuTTY

• Select Auth tab underConnection -> SSH

• Use browser menu to findprivate key (.ppk)

You can save settings (key location and X11 enabling in theSession tab.

Using ssh Key with WinSCP

• Select Advanced settings

• SSH -> Authentication tab

• Use browser menu to find private key (.ppk)

Linux commandsman <command>

manual page for <command>

exit

quit current shell (log out if this is the login shell)

passwd

change password

use a cryptic one!

Not necessary on wardlaw, kennedy or knox(authentication is via ssh keys).

Linux commands

ls

list current directory

ls -l

list directory in detail

ls -lrt (same as ls -l -t -r)

list current directory in detail,ordered by time, in reverse order

ls <dir>

list directory <dir>

man ls ...

more Linux commands

cd <directory>

“change directory”: change working directory (folder)

.. parent directory (one up in the tree)

. current directory

/ between parent and child (not \ as in Windows)

pwd“print working directory”: show where we are

mkdir <directory>

create directoryrmdir <directory>

delete (empty) directory

Still more Linux commands

cp <file1> <file2>

copy

mv <file1> <file2>

rename (“move”)

mv <file1> <dir>

move <file1> into directory <dir> (works also with cp)

rm <file>

delete <file>

rm -r <dir>

delete <dir> and all subdirectories!

A few more useful ones...head <file>

show first 10 lines of file

head -20 <file>

show first 20 lines (works for any number)

tail <file>

show last 10 lines (e.g. check if output file is complete)

tail –20 <file>

guess what…

tail –f <file>

“follow”: keep showing newly added lines. Exit with Ctrl-c

And the last one for now

grep <pattern> <file>

show all lines in <file> containing <pattern>

Wildcards Most file/directory commands accept “wildcard

characters”. Most commonly used ones are

? Any single character

* Any number of characters (including 0)

Editing with Emacs

emacs <file> &

Opens Emacs window on your desktop

& executes (any) program in background, soyou can go on working in the same window

Using Emacs

Menus are self-

explanatory

Try to remembersome keyboardshortcuts, in case youneed to use it withoutX client (emacs -nw)

Ctrl-x-s save

Ctrl-x-c exit

Ctrl-x-w save underdifferentname

Editing with vi Faster over a network

No separate window start with

A bit cryptic at first… vi <file>

Two modes

Command Mode: cursor keysand 1-2 letter commands

x delete character

dd delete line

D delete rest of line

i insert before cursor

a append after cursor enter

o new line below input

O new line above mode

:w save

:x save and exit

Input Mode

Write text

Delete with backspace key

[ESC] key: back to command mode

Many tutorials and “cheatsheets” available

Google “vi editor”

Running SLURM Batch Jobs

Submit script to batch queuefor execution when computenodes are available

Job script with SLURMdirectives at the start:

Name of job

Request one nodeUse 12 coresRun for 48h maxUse complete nodesScript to run Gaussian

SLURM directives

Commands toexecute when jobstarts

Managing jobs

sbatch <jobscript>submit job

squeueshow running or queued jobs

scancel <jobnumber>delete job

Other Queueing Systems

Other clusters (knox, chpc-hf63):SGE or PBS queueing system

sbatch => qsubsqueue => qstatscancel => qdel

Get an example job from your supervisor!

Privacy and Data Protection

UNIX File permissions:

user (=owner)

group (same group as owner)

others (anybody else)

read

write

execute (file) or access (dir)

To change:

chmod go-rwx

(remove all permissions from yourgroup and others)

chmod g+r

(give your group read permission)

man chmod …

Root user can read (or write) anything!

Security

Passwords

• Don’t share. Get your ownaccount

• Make it difficult to guess(no names, dictionarywords, birth dates; usespecial characters)

• Change if compromised

Don’t run unknown programs

Don’t log in from a PC thatmay have viruses

Rules of Use

Wardlaw and kennedy open to any St Andrewschemist

Higher priority access available for £500/y

Be considerate: don’t flood the machines whenthey are already full.

Other clusters: ask your supervisor

How to get an Account or Advice

Contact me!

hf63@st-andrews.ac.uk

Wardlaw website

http://www-solar.mcs.st-and.ac.uk/~herbert/cluster/

Go down to “Software and usage”.

EaStCHEM/SUPA postgrad course Introduction to Computational Chemistry

9 lectures starting October

Email your name and supervisor to Tanja van Mourik

tvm22@st-andrews.ac.uk

Other Lectures and SeminarsLectures on

Molecular Quantum Chemistry

DFT on Periodic Systems

Computation in Chemistry Seminars

See Chemistry Newsletter

Sign up to mailing list with Tanja van Mourik

EaStCHEM PgCert course Computational Chemistry and Modellinghttp://www.ed.ac.uk/studying/postgraduate/degrees/index.php

Free for St Andrews PG students