Shell Basics

CS465 - Unix

Shell Basics

• Shells provide:– Command interpretation– Multiple commands on a single line– Expansion of wildcard filenames– Redirection of input and output– Pipelines– Background execution– Script-programming facility

Running Multiple Commands; Causes commands to run in sequence.

$ cmd1 ; cmd2

( ) Causes sequenced commands to be grouped $ (cmd1 ; cmd2)

&& Runs the next command only if the current command succeeds.

$ cmd1 && cmd2

|| Runs the next command only if the current command fails.

$ cmd1 || cmd2

Sequence ExampleCommands can be sequenced (run in sequence) using the

semicolon (;):

$ ls ; who ; date ; pwd file1 file2 file3 newfilesmall000 pts/1 Jun 4 14:41Tue Jun 4 14:42:51 MDT 2003/export/home/small000$

If any one command fails, the others will still execute.

NOTE: To make the command line easier to read, separate commands from the semicolon (;) with blanks.

Grouped Commands Example

• Commands can be grouped together using parentheses

– To create a “single command” out of a group of commands (especially useful before a pipe):

$ (cat letter1; head -2 names) | sort

&& Example

To run the second command only if the first command succeeds (i.e. short-circuit “and” operator):

BAD: $ cd /somedir; rm -r *

GOOD: $ cd /somedir && rm -r *

|| Example

To run the second command only if the first command fails (i.e. short-circuit “or” operator):

Example:

$ ls file1 || cp fileX file1

Shell Environment• Your environment defines such characteristics as

your user identity, where you are working on the system, and what commands you are running.

• Your working environment is defined by both environment variables and shell variables.

– Your default login shell uses environment variables and passes them to all processes and subshells that you create.

– Shell variables are valid only for your current shell and are not passed to subshells.

Environment Variables

HOME Name of your login directory. Set by login program.

HOST Name of host machine running your shell program.

LOGNAME Your login name. Set by login program.

MAIL Pathname of the file used by the mail system to detect the arrival of new mail. Set by login program.

Environment Variables

PATH Directory list and order that your system uses to search for, find, and execute commands. Set by login scripts.

SHELL Your default shell. Set using the shell specified in your entry in the /etc/passwd file.

TERM Type of terminal you are using. Set by your login script.

TZ Your current time zone. Set by the system login script.

Environment Variables• To see the value of a defined variable, place a $ in front of the

variable name:

$ echo $HOME/home/small000$ echo $TZMDT$

• Note the difference if you leave off the "$":

$ echo HOMEHOME

Metacharacters

• Metacharacters are used to create patterns to match filenames and command names.

– For example, we often wish to apply a command to a selection of files or directories

– The shell wildcard (often known as globbing) feature allows this.

• Each shell program recognizes a set of special characters called metacharacters:; & ( ) | \ ^ ~ % { } $ # ´ “ ‘ @ * ? [ ] !

Wildcard Metacharacters? matches any character

* matches any number of characters

[abc] matches any one of the characters in the brackets

[a-z] matches any one character in the range of characters in the brackets

[!a-z] matches any character except the range of characters in the brackets

Wildcard Examples

$ ls abc?def

– matches abcXdef, abc.def, abc3def but not abcdef

$ ls abc*

– matches abc., abcd, abcdef, abc.another.part

$ ls *.*

– matches any name with a “.” in it

$ ls *– matches any name (except names beginning with “.”)

$ ls [a-z]*– matches any name beginning with a lower-case letter

Wildcard Exercise• Assume we have the following files under the

current directory:

120 235 305

a.out c22 cap

Doc.1 Doc.2 Doc.3

one.c two.c three.c

• What would match for:ls *.c ls ?2?ls D* ls [a-z]*[!0-9]ls ??? ls [23]*ls * ls .*

Avoiding Wildcard Expansion

• Sometimes you may NOT want wildcard characters expanded.

• The Unix glob utility translates wildcard characters on the command line.

• To turn off ALL wildcard expansion:$

set noglob

• To turn ALL wildcard expansion back on:

$ set glob

Avoiding Wildcard Expansion

• Sometimes you need to selectively enter a wildcard character on a command line, and not have it expanded.

• Method 1: Double quote the argument containing the character:

$ ls “abc*def”abc*def

ls the file whose name is abc*def (if the system allows such a name)

Avoiding Wildcard Expansion• Method 2: “Escape” the character

$ ls abc\*def

– again lists the file whose name is abc*def

• Can use these methods in combination$ echo “Show asterisks *” and \*Show asterisks * and *$

– * inside the double quotes is not expanded

– * after the backslash is not expanded

Double Quotes vs Single Quotes

• Within double quotes, the reserved characters:

• dollar sign ($)

• grave accent (`)

• backslash (\)

keep their special meanings

• All other characters are taken displayed literally.

Double vs Single Quote Example

Displaying a variable name without it being interpreted by the shell:

$ echo “Value of $USER is “ $USER

Value of small000 is small000

$

$ echo 'Value of $USER is ' $USER

Value of $USER is small000

$

Grave Accent/Backquotes

• Within backquotes (graves), command substitution is forced:

• Example:$ echo I am user whoami

I am user whoami

$ echo I am user `whoami`

I am user small000

$

Summary of Quoting Metacharacters

• Single quotes– Contents are used verbatim

• No wildcard expansion• No variable interpolation• No command execution

• Double quotes– Only turns off wildcard expansion

• Backquotes– Forces execution of command in quotes– Output from command replaces quoted string

Redirection of I/OEvery program run under Unix is provided with three

standard I/O streams: stdin, stdout and stderr

commandstdin (0)

stdout (1)

stderr (2)

By default:

- the standard input (stdin) is the keyboard

- the standard output (stdout) and standard error (stderr) are the display screen

Your shell allows you to divert any or all of these streams.

Output Redirection

• Use the “>” character to redirect the standard output of a command to a file:

$ ls > myfiles$ cat myfilesfile1 file2 file3$

$ echo hello $USER > greeting$ cat greetinghello small000$ echo hi there > greeting$ cat greetinghi there$

Output Redirection

• Use the “>>” character after a command to redirect output to APPEND to a file:

$ cat greetinghi there$ echo “How are you?” >> greeting$ cat greetinghi thereHow are you?$

Output Redirection

$ ls x*

ls: x*: No such file or directory

$ ls x* 1>xfiles 2>errs

$ cat xfiles

$ cat errs

ls: x*: No such file or directory

$ echo roses are red 1> rhyme 2> errs$ echo violets are blue 1>> rhyme 2>> errs$ cat errs$ cat rhymeroses are redviolets are blue$

• Can separate stdout (1) and stderr (2):

Output Redirection

$ cat testtype line 1type line 2$

$ cat > testtype line 1type line 2

<ctrl-d> to end your input$

Redirecting cat’s output can create a file (no input file name given, so gets input from stdin):

Output Redirection

Typing to append to the end of a file

$ cat >> testtype line 3type line 4<ctrl-d>

$ cat testtype line 1type line 2type line 3type line 4

Appending file1 to file2

$ cat file1 >> file2

Overwriting Files?• By default, the use of ‘>’ will overwrite any existing file, without

warning. Use “noclobber” to prevent this: $ set –o noclobber(+o to revoke)

• Gives error if you try to overwrite an existing file.

• Can still use >| to force overwriting of an existing file.$ lsfile1 file2$ set -o noclobber$ cat file1 > file2ksh: cannot create file2: File exists$ cat file >| file2$

Input Redirection

• Use the “<” character to redirect input from a file to a command:

$ sort < names

$ mail user < letter.txt

• Can use input and output redirection together:

$ sort < names > names.sort

Filters

• Most UNIX utilities (commands) are filters

• A filter is a program which:

– reads input (text) from standard input only

– writes output and errors to standard output only

• Filters can be combined to work together using pipes

• A pipe takes the stdout of one command and uses it as the stdin of another command

Pipes• The standard output of one program is “piped” to the

standard input of another:$ who | sort

• Several pipes can be connected:$ ls | sort | more

• No limit on number of processes that can be chained by pipes

Pipes

• Pipes and I/O redirection can be used together:

$ ls | sort > files.sorted

$ sort < names | more

Why Pipes?

• Could chain commands together

$ ls > temp; sort < temp; rm temp

• Works, but requires temporary intermediate file, which must be manually removed

• Solution – Use a pipe:$ ls | sort

– No intermediate file, processes run in parallel

Multiple Outputs

• What if you want the output of a command to be saved in a file AND displayed on the screen?

• Use the tee command

• Example: $ ls –la | tee filelist

Displays directory listing to the screen, AND saves it to the file filelist.

Summary of Shell Command Interpretation

The shell reads a line from a script file or from a user. Then:

1. Meta-characters are "handled.“

2. Executable is found (via PATH).

3. Arguments are passed to the program.

4. File redirection is setup.

5. The program is executed.