Unix tutorial Basics

www.msi.umn.edu

Unix Basics

Presented by: Patton FastUser Support Grouphelp-line: [email protected] 612-626-0802personal: [email protected] 612-625-6573

Research.We make it happen.

www.msi.umn.edu

I. Warnings!

II. Basic Concepts

III. Basic Commands

IV. The Fun Stuff

V. Resources

Please Ask Questions!

Outline

www.msi.umn.edu

• If you delete a file, it is GONE! The file can only beretrieved from a backup.

• If you overwrite a file it has been changed forever.The original file can only be restored from a backup.

• Home directories are backed up nightly.

• The scratch spaces are NOT backed up!

Bottom line? Be Careful With Your Files!

Warnings!

www.msi.umn.edu

Scratch Space Time Limits!

• Regatta (/regscratch1 and /regscratch2) – 7 days, 50 Gb

• SP (/scratch1 and /scratch2) – 7 days, 50 Gb

• Altix (/scratch1 and /scratch2) – 14 days

• Netfinity (/scratch1 and /scratch2) – 14 days

• SDVL (/scratch) – 14 days

• BSCL (/wrk) – 14 days

Bottom line? Be Careful With Your Files!

Warnings!– scratch space

www.msi.umn.edu

Core Resource Service Units

• IBM Power4 – 3.5 hours / su

• SP Winterhawk+ – 5 hours / su

• SP Nighthawk – 10 hours / su

• Altix – 3.5 hours / su

• Netfinity – 12 hours / su

Warnings!– service units

www.msi.umn.edu

IBM Power4364 Power4 processors603 Gb of memory

6 Tb of storage

SGI Altix432 Itanium2 processors876 Gb of memory

4 Tb of storage

IBM Linux Cluster160 Intel processors226 Gb of memory2.5 Tb of storage

IBM SP400 Power3 processors675 Gb of memory

4 Tb of storage

Basic Concepts– core resources

www.msi.umn.edu

Computational Genetics Lab16 CPU SunFire 68004 Dell PCs1 Mac G51 SGI Fuel1 SGI Tezro1 IBM IntelliStation

Basic Sciences Computing Lab48 CPU SGI Altix16 CPU SGI Onyx49 SGI Octanes8 CPU Sun UltraSparcIII2 Linux Workstations1 SunBlade 2000

10 Tb of storage64” x 80” stereo system

Scientific Dev. & Vis. Lab9 SunBlade 2000s7 SGI Octanes1 SGI Onyx

Basic Concepts– laboratory resources

Med. Chem./Supercomputing Inst.Vis. Workstation Lab

3 SGI Octanes3 SGI O2’s2 SunBlade 2000s

http://www.msi.umn.edu/bsclhttp://www.msi.umn.edu/vwl

http://www.msi.umn.edu/sdvlhttp://www.msi.umn.edu/cgl

www.msi.umn.edu

Basic Concepts– laboratory resources

Basic Sciences Computing Lab• Proteomics• Structural Biology

Computational Genetics Lab• Bioinformatics• Genomics

Lab for Large Scale Data Analysis• Data Mining• Database Modeling

Med. Chem./MSI Visualization Workstation Lab• Drug Design• Molecular Modeling

Scientific Development & Visualization Lab• Computational Fluid Dynamics• Scientific Computation

www.msi.umn.edu

After your first login you should change your initial passwordto something that is easy for you to remember, but not easyfor someone to guess.

%> yppasswd (Institute machines)

%> passwd (general)

You should NEVER give your password to anyone!

If you forget your password, send a message [email protected] and your password will be reset.

Note: Never share accounts or passwords!

Basic Concepts– changing passwords

www.msi.umn.edu

Initial PasswordsYour initial password should be the same for all machines within a givenlab, regardless of operating system.

Unix, Linux, Mac OS XAll of the Unix based machines share the same password file. When youchange your password on one of the machines within a given lab, it ischanged for all of the machines.

WindowsAll of the Windows bases machines share the same password file acrossall labs. When you change your password on one machine it is changedfor all machines regardless of the lab.

Basic Concepts– changing passwords

www.msi.umn.edu

Basic Concepts– console

Console Login:

• Enter username

• Enter password

Using the keyboard and mouse physically attached to themachine.

Console Logout:

Logout procedures vary between consoles. Make sure youlogout! Otherwise your files and account are open to thenext person. You could be logged out by an administrator.

www.msi.umn.edu

ssh [options] machine secure shell; a program for logging into a remotemachine providing encrypted communicationsbetween machines

options:-l login name-X sets environment variables for porting X-display

example:

type: ssh -l pfast sp open a secure connection for the user pfast onthe machine sp.msi.umn.edu

type: ssh bi2 open a secure connection to bi2.msi.umn.eduwith current username

SSH clients:

http://www.msi.umn.edu/user_support/ssh

Basic Concepts– remote login

www.msi.umn.edu

Why?

• So that you get all of your email sent to one account.

• Makes it easier for sys admins to communicate with you.

How?

1) Create the file .forward in your home directory

2) On the first line:

[email protected]

3) Create this same file in all of your accounts that you wantto forward.

Basic Concepts– forwarding email

www.msi.umn.edu

man [options] command displays manual pages about command-k keyword search for a command

info command displays Info document of command

When in doubt, check the man/info pages!

Need extra details? Check the man/info pages!

Basic Concepts– command line help

www.msi.umn.edu

Basic Concepts– file structure overview

/

bin@ home/ lib@ scratch/ usr/

user1/ user2/

file1

mail/ source/ work/

proj1/ proj2/

.cshrc

input output

bin/ lib/ local/

user1/ user2/

dir1/ dir2/

file1/ file2/directory ( / )executable ( * )link ( @ )

more*

www.msi.umn.edu

Basic Concepts– metacharacters

? single character wild card

* wild card, arbitrary number of characters

~ home directory of current user

~name home directory of user name

. current directory

.. parent directory (back one)

www.msi.umn.edu

command [options] argument

example:

ls -al /scratch

command name

[options] (usually preceded by a dash)

argument (quite often a filename or directory)

Note: UNIX commands, options, and arguments are allCasE seNsiTive!

Basic Concepts– command anatomy

www.msi.umn.eduCommand Description

ls [options] dirname list the contents of dirname

pwd display full pathname

mkdir dirname create the directory dirname

cd dirname change to the directory dirname

mv [options] source destination move a file or directory

cp [options] source destination copy a file or directory

rm [options] source remove a file or directory

ssh [options] hostname remote login

scp [options] user@host1:file user@host2:file remote copy; file transfer

Basic Commands

skip basic commands

www.msi.umn.eduls [options] dirname list the contents of dirname

options:

-a list all files including hidden files[hidden files are preceded by a ‘.’; eg .cshrc]

-l long listing showing ownership, permissions,and links

examples:

type: ls /home/user/temp view the contents of a directory with absolutepathname /home/user/temp

type: ls ../../temp list the contents of a directory using a relativepath.

Listing Contents

www.msi.umn.edu

mkdir dirname create the directory dirname

examples:

type: mkdir work create the directory work/ in the current workingdirectory

type: mkdir work/proj1 create the directory proj1/ in the work/ directory

type: mkdir /wrk/user2 create the directory user2/ in the /wrk/ directory

Make Directory

www.msi.umn.edu

cd change to your home directory

cd dirname change to the directory dirname

examples:

type: cd ~tom change to tom’s home directory

type: cd /wrk/user2 change to the directory /wrk/user2

Change Directory

www.msi.umn.edu

mv [options] presname newname rename a file

mv [options] srcfile destdir move a file to anotherdirectory

options:

-i confirm overwrites

example:

type: mv ~user1/file ./outputfile moves file from user1’s homedirectory to the current workingdirectory and renames it outputfile

Note: Be careful when overwriting files!

Moving or Renaming

www.msi.umn.edu

cp [options] srcfile destfile copy a file to another filename

cp [options] srcfile destdir copy a file to another directory

options:

-i confirm overwrites-R recursively copy a directory and its

contents, copies symbolic links

example:

type: cp -R proj1 proj2 copy the directory proj1/ and name itproj2/


Copying

www.msi.umn.edu

cp [options] srcfile destfile copy a file to another filename

cp [options] srcfile destdir copy a file to another directory

options:

-i confirm overwrites-R recursively copy a directory and its

contents, copies symbolic links

example:

type: cp -R proj1 proj2 copy the directory proj1/ and name itproj2/


Copying

Tip:If you have large files tomove from the scratchdirectories,copy (cp) themrather than moving (mv)them.

www.msi.umn.edu

rm [options] filename remove a file

options:

-f remove without prompting-i confirm removal-r recursively remove a directory and its contents

example:

type: rm -rf /scratch/user2/dir1 remove the directory /scratch2/user2/dir1and its contents

Note: Be careful when removing files!Especially with wild cards (*,?)!

Removing

www.msi.umn.edu

scp [options] host:file1 host:file2 secure copy; copy files betweenhosts on a network using the sshprotocol

options:-r recursive (used for directories)

example:type: scp pfast@wind:”output/*.out” . copy the output files from host wind

with username pfast to current directory

type: scp -r mail origin:mail_copy copy the mail directory to the originand rename the directory mail_copy

Note: You can use sftp if scp is not available.

Remote Copy

www.msi.umn.edu

scp [options] host:file1 host:file2 secure copy; copy files betweenhosts on a network using the sshprotocol

options:-r recursive (used for directories)

example:type: scp pfast@wind:”output/*.out” . copy the output files from host wind

with username pfast to current directory

type: scp -r mail origin:mail_copy copy the mail directory to the originand rename the directory mail_copy

Note: You can use sftp if scp is not available.

Remote Copy

Tip:If you have large files totransfer, use sftp insteadof scp.

www.msi.umn.educhmod [options] who ops permission filename change file permissions

use ls -l to view permissions

options

–R recursively change permissions

who can be any combination of:u (user) g (group)o (other/world) a (all or ugo)

ops adds or takes away permission, and can be:+ (add permission) – (remove permission)

permission can be any combination of:r (read) w (write) x (execute) X (check user first)

Changing Permissions

www.msi.umn.eduhistory display list of most recent commands

!string repeat command beginning with string

up/down arrows scroll up and down through mostrecent commands; csh and tcsh

left/right arrows use for editing current line

Control-E end of lineControl-A beginning of line

examples:

type: !ls repeat last ls command

Command Repetition

www.msi.umn.edu

TAB finish the current command, filename,or or directory if possible, or show theControl-D possible completers

(works with c and tc shells)

Command Completion

www.msi.umn.edu

tar [options] files collect or extract files or directoriesinto or from an archive

options:c create a new archivex extract files from archivef store files in archivev print function letter and name of files– store to standard output

example:type: tar xvf file.tar extract the files and or directories from the

archive, file.tar

Tarring

www.msi.umn.edu

compress file compress file and rename it file.Z(the program automaticallyrenames the file)

uncompress file.Z uncompress file.Z and rename it file

gzip file compress file and rename it file.gz(the program automaticallyrenames the file)

gunzip file.gz uncompress file.gz and rename itfile

Compression

www.msi.umn.edu

Process Control

Control-C cancel a foreground job

Control-Z stop (interrupt) a foreground job

jobs list of background jobs

bg run stopped job in the background

fg run stopped job in the foreground

& appended to the end of a command will placethat job in the background

example:type: blastp swissprot test.seq > test.out & run WU-BLAST job in the

background

www.msi.umn.edu

ps [options] display the status of the current processes andthe process id-number

options:-a all processes-u display processes owned by a particular user

jobs shows any jobs that are currently running inthe background or suspended

Process Control

www.msi.umn.edu

kill –9 id-number terminate a process owned by you; id-number (process id) can be found withthe ps command

or

kill %job-number terminate a process owned by you; job-number can be found with the jobscommand

Process Control

www.msi.umn.edu

du -sk file display total disk usage of file ordirectory in kilobytes

df display the current disk usage forall connected file systems

stat file view attributes of a file or directory

quota -v list your disk quota

Other Commands

www.msi.umn.edu

cat [options] files dumps the contents of files to thescreen

more [options] files a pager, that lets you view one pageat a time

less file similar to more, but more

Viewers

www.msi.umn.edu

Graphical:

nedit filename if not available, ask to have itinstalled

xemacs filename

Command-line:

pico filename full-screen text editor available onmost UNIX systems

vi filename full-screen text editor available onvirtually every UNIX system

Editors

www.msi.umn.edu

The Fun Stuff– redirect

> command output redirection (create new,overwrites existing file)

>> command output redirection (append)

< command input redirection (from file)

examples:

how to: combine individual sequence files into one file

type: cat seq1 seq2 > seq

how to: email a file to a collaborator

type: mail -s “results” [email protected] < new_blast_results

www.msi.umn.edu

The Fun Stuff– piping

command1 | command2 | … directs standard output of one commandinto standard input for the next command

examples:

type: ls -al | more look at the ls output one page at a time

type: qstat -a | grep user1 check the queue on the origin for anyprocesses being run by user1

www.msi.umn.edu

grep [options] string file find and print out lines containing the string ina file or multiple files

options:

-i ignore case

examples:

how to: extract sequence information from protein structure files

type: grep SEQRES pdbfile | cut -c20-70 > seqs

The Fun Stuff– pattern searching

www.msi.umn.edu.cshrc shell startup file

Some things to add to your .cshrc file:

alias alias-string com-string abbreviate a command string with analias string

examples:

type: alias la ls -a create a shortcut for ls -a

type: alias q qstat -u pfast check if you have any jobs running inthe queue (origin)

module load(unload) software load or unload the environment settingsfor a particular software

source /usr/local/gcg/gcgstartup runs the gcgstartup script to set thenecessary env. variables

The Fun Stuff– .cshrc file

www.msi.umn.edu

UNIX in a Nutshell by O’Reilly

Developing Bioinformatics Computer Skills by O’Reilly

http://www.iats.missouri.edu/servlets/knowledgebase/article/422

http://www.geek-girl.com/unix.html

http://riceinfo.rice.edu/Computer/Documents/Unix/unix1.01.pdf

Resources

www.msi.umn.edu

Questions?

email: [email protected]: 612-626-0802

Unix tutorial Basics

Documents

Transcript of Unix tutorial Basics