Bourne Shell Programming Topics Covered · Bourne Shell Programming – Topics Covered Shell...
Transcript of Bourne Shell Programming Topics Covered · Bourne Shell Programming – Topics Covered Shell...
1
Bourne Shell Programming – Topics
Covered
Shell variables
Using Quotes
Arithmetic On Shell
Passing Arguments
Testing conditions
Branching
if-else, if-elif, case
Looping
while, for, until
break and continue
Using options using getopts
Reading Data
Environment and subshells
Parameter substitution
Using set
I/O redirection
Shell archives
The eval, trap, wait commands
2
Part 1
Some Regular Expression Characters
. (dot) – any character
^ - beginning of line
$ - end of line
* - zero or more occurences of previous regular expression
[chars] – any character in the set of chars
[^chars] – any character not in chars.
\{min,max\} – at least min occurences and at most max occurences of
the previous regular expression.
3 2
Shell Scripts
Bourne Shell/Korn Shell
Invoking a shell script
$shell_script_file or $sh -options shell_script_file
the script file must have execute-permission.
Shell Variables
mydir=/usr/jsmith/bin
count= #assign a null value to the variable
echo $mydir #display the contents of mvdir
x=*
echo $x #substitutes the names of the files in the directory
#name of a command, options and arguments can be stored inside variables
command=ls
option=-l
filename=namesFile
$command $option $filename #shell performs the variable substitution before it
# executes the command.
4
Quotes
The Single Quote
The white-space characters
enclosed between the single
quotes are preserved by the
shell.
The special characters are
ignored.
Example:
filename=/usr/jsmith/bin/prog1
echo $filename
echo ‘$filename’
echo ‘<> | ; () {} ` & “‘
The Double Quote
The special characters, $, back
quotes (`) and back slashes (\) are
not ignored.
Example;
x=*
echo $x #filenames are substituted
echo ‘$x’ #$x is displayed
echo “$x” # * is displayed, variable
substitution is done inside the double quotes,
no file name substitution is done and * is
passed to the shell.
5
Quotes
The Back Quote
purpose is to tell the shell to
execute the enclosed command
and to insert the standard output
from the command at that point
on the command line.
Example:
echo The date and time is: `date`
echo There are `who | wc -l` users
logged on
filelist=`ls`
echo $filelist (#what is the output)
mail `sort -u names` < memo
#-u option removes the duplicate
# entries from the file
The Back Slash
Is same as putting single quotes around a single character.
Quotes the single character that immediately follows it.
X=*
echo \$x # $x is displayed
Is interpreted inside the double quotes.
Use backslash inside the double quotes to remove the meaning of characters that otherwise would be interpreted.
Examples:
echo “\$x” #$x is displayed
echo “The value of x is \”$x\””
#The value of x is “5” is displayed
6
Arithmetic On Shell
A variable is just considered a string of characters.
Example:
x=1
x=$x+1
echo $x #will display 1+1
A unix program expr evaluates an expression given to it on the command line.
Each operator and operand given to expr must be a separate argument. The operators, +, -, *, /, % are recognized.
Example:
i=1
i=`expr $i + 1`
Evaluates only integer arithmetic expressions.
awk may be used for floating point calculations.
expr 10 * 2 # what is the problem with this?
7
Passing Arguments
$#: Number of arguments passed to the program from the command line.
$* : references all the arguments
Example:
%cat showArgs
echo $# arguments passed.
echo they are :$*:
%showArgs a b c d 1 #output -
%showArgs “a b c d 1” #output -
% showArgs `cat names` #output -
% showArgs x* #output -
%cat lookup
grep $1 phonebook
lookup “Mary Jones”
What is the result?
8
Positional Parameters
Positional parameters
set shell script arguments. e.g.
$my_script a b xy “1 2 3”
positional parameters have the values
$0 -- my_script
$1 -- a
$2 -- b
$3 -- xy
$4 -- 1 2 3
$* - references all the variables passed as arguments
9
The shift command
shift
left-shifts the positional
parameters.
If more than 9 arguments
are supplied, arguments 10
and up cannot be
referenced.
use shift to access these
arguments.
shift assigns value in $2 into
$1, $3 into $2 etc.
The number of arguments
($#) gets decremented by
one on each shift.
%cat testshift
echo $# $*
shift
echo $# $*
shift
echo $# $*
% cat testshift 1 2 3 4 5 6 7 8 9 10
What is the output?
10
Testing Conditions
if statement: allows to test a condition and branch on the exit status of the condition tested.
An exit status of 0 indicates the program executed successfully.
An exit status of non-zero indicates a failure.
$?: contains the exit status of the last command executed.
Operators for integer comparisons
eq (equal), -ge (greater than or equal), -gt (greater than), le (less than or equal), -lt (less than) and –ne (not equal)
Operators for string comparisons
= , !=, -n (string is not null) and –z (string is null)
File operators
-d file file is a directory
-f file file is an ordinary file
-r file file is readable by the process
-s file is of non-zero length
11
Testing Conditions
Examples
user=$1
who | grep “^$user” > /dev/null
- the exit status of the last command in the pipe line is returned.
12
The test command
The test command is used to test one or more conditions in an if statement.
y=ABC
test "$y" = ABC
echo $? # displays 0
x=
test -n x #checks if x is not null
echo $? #displays 1
test -z x #checks if string is null
echo $?
x=ABC
[ "$x" = ABC ] #[] same as using test
[ ! "$x" = ABC ]
x=5
# -a for logical and -o for logical or
[ "$x" -ge 0 -a "$x" -lt 10 ]
[ -f “$file1” -a -r “$file1” ]
13
Branching
%cat isUserOn
#checks if a user is logged on
User=$1
if who | grep “$user” #what is the problem with matching a username in the output of
who?
then
echo “$user is logged on”
fi
========================================
if [ “$#” -ne 1 ] #checking for the correct number of arguments
then
echo “Incorrect number of args”
exit 1 #terminates the program with the exit status
fi
if [“$NAME” = “John Doe” -a “$BAL” -gt 5000] then echo “ok” else echo “not ok” fi
14
Using case
case: allows a comparison of a single character against other values and execute a command if a match is found.
%cat ctype
x=A
case "$x“ #The value in x is compared with each of the cases
#until a match is found. When a match is found, the
#commands up to the double colons are executed.
in
[0-9] ) echo digit;;
[A-Z] ) echo uppercase;;
[a-z ) echo lowercase;;
* ) echo special character;;
esac
Exercise: Can you rewrite the script passing the value to be tested as an argument?
15
The && and || constructs
command1 && command2
if the exit status of command 1 is 0 then command 2 is executed.
Example
EDITOR=[ -z "$EDITOR" ] && EDITOR=/bin/ed
echo "$EDITOR"
command1 || command2
If the exit status of command 1 is not zero, then command 2 is
executed.
Example:
grep “$name” phonebook || echo “Couldn’t find name”
Debugging with a -x option
Trace the execution of any
program by typing in sh -x
followed by the name of the
program and its arguments.
Starts up a new shell to execute
the program with the debug
option.
Commands are printed at the
terminal as they are executed
preceded by a + sign.
sh -x ctype A
+ [ 1 -eq 1 ]
+ x=A
+ + echo A
+ wc -c
num=2
+ [ 2 -ne 1 ]
+ echo Enter a single character
Enter a single character
+ exit 1
Looping
The for loop is executed for as many words as are included after in
for var in listofwords
do
commands
done
for i in 1 2 3
do
echo $i
done
for file in * #substitutes all the files in the # directory
do
processCmd $file
done
18
The for loop
for var #uses all the arguments given to the program on the command line
do
command
command
done
for file in $* # Replaces $1, $2 as $1, $2 etc
do
x=`wc -l $file`
echo There are `echo $x |cut -f1 -d’ ‘` lines in $file
done
for file in “$@” #Replaces $1, $2 as “$1”, “$2” etc. Should be included in
double quotes
do
echo $file
done
Looping
while command
do
command1
command2
done
command1 and 2 are executed until
command returns a nonzero exit
status
# Print command line arguments
while [ “$#” -ne 0 ]
do
echo “$1”
shift
done
until command
do
command1
command2
done
command1 and command2 are
executed as long as command
returns a non-zero exit status.
until who | grep “^$user “
do
sleep 60
done
Break and continue
break: to break out of a loop.
break n: to break out of n inner most loops
for file
do
#variable error can be set to a value
count=1
while [ “$count” -le 5 ]
do
#process file
if [ -n “$error” ] #contains a value
then
break 2
fi
count=`expr $count + 1`
done
done
continue: the remaining commands
in the loop are skipped.
for file
do
if [ ! -f “$file” ]
then
echo “$file not found”
continue
fi
cat $file
done
11
The getopts command
The built-in command, getopts processes the command line
arguments.
Format: getopts options variable
Used in designing programs which take options.
Example:
isLoggedOn mary
isLoggedOn -m mary
isLoggedOn -m -t 120 mary
isLoggedOn -t 120 -m mary
The getopts command is designed to be executed inside a loop.
An example using getopts
#wait until a specified user logs on -- version with
options
mailopt=FALSE
interval=60
#process command options using getopts
while getopts mt: option
do
case "$option"
in
m) mailopt=TRUE;;
t)
interval=$OPTARG;;
\?) echo "Usage : mailTo [-m] [-t n] user"
exit 1;;
esac
done
An example using getopts (cont)
#Make sure a user name is specified
if [ "$OPTIND" -gt "$#" ]
then
echo "Missing user name"
exit 2
fi
#get the user name into $1
shiftcount=`expr $OPTIND - 1`
shift $shiftcount
#Check for user logging on
# If the user is logged on, send mail (a reminder) if the mail
option
# is true.
user=$1
24
Reading Data
read variables – shell reads a line from stdin and assigns the first word to the first variable, second word to the second variable etc.If there are more words than variables, the excess words are assigned to the last variable.
Can redirect the input from a file.
Examples:
read x y #reads the input from the stdin
read x y < names # reads the first line in names file and assigns the first word into x and the rest into y
Example:
while read val1 val2
do
total=`expr $val1 + $val2`
echo $total
done
25
Copying a file using mycopy
#check if two arguments are supplied. if not exit with an error message and usage
fromFile="$1"
toFile="$2"
#check if the destination is a directory
if [ -d "$toFile" ] then
#extract the file name from the source file name and
#concatenate it to the destination directory
toFile=$toFile/`basename $fromFile`
fi
#check if a file with the destination file name exists
if [ -f "$toFile" ] then
echo "The file $toFile already exists. Do you want to
overwrite it?"
read $answer
if [ "$answer" != "yes" ] then
exit 0
fi
cp $fromFile $toFile
fi
26
SubShells
$cat shelltest
x=200
echo $x
$shelltest #output is 200. The program shelltest executed in a
# subshell.
x=100
echo $x #output is 100
A subshell runs in its own environment with its own local variables.
A subshell has no knowledge of the local variables in the parent shell,
therefore cannot change the value of a variable in the parent shell.
The command export is used to make the variables in a shell known
to all subshells executed from that point.
27
$cat one
echo “x = $x y = $y “
$ one #x = y = are displayed since x and y have null values
$ x=100
$ y=200
$ export y
$ one #y=200 is displayed
Once a variable is exported, it remains exported to all the subshells that
are subsequently executed.
Exported variables and their values are copied into a subshell’s
environment, where they may be accessed. Any changes made to them
have no effect on the variables in the parent shell.
A variable can be exported before or after it is assigned a value.
28
Subshells
$cat one
echo "x = $x"
echo "y = $y"
$cat two
x=1
y=2
z=3
export z
three
$cat three
echo "x = $x"
echo "y = $y"
echo "z = $z"
$one #What is the output
$y=100
$export y
$one #What is the output
$two # output
# x =
# y = #what is the output?
# z = 3 why?
$three
#output
# x =
# y = 100
# z = Why?
29
Some Environment Variables
PS1- stores the command prompt
PS2 – stores the secondary command prompt
HOME – Stores the home directory
PATH – contains a list of directories which the shell searches whenever you
type in the name of the program to be executed.
The PATH specifies the directories to be searched for programs to be
executed and not for any other types of files.
$ echo $PATH – displays output which may look like this
/bin:/usr/bin::
CDPATH
30
The set command
Is used to set various shell options.
Is used to reassign the positional parameters.
Set with no arguments will give a list of all the variables that exist in your
environment.
HOME, PATH, SHELL are some of the variables that are displayed.
Set can be used to reassign the positional parameters – can be used to
parse the data from a file or the terminal.
Example:
$cat testSet
set x y z
echo $1: $2: $3
echo $#
echo $*
$testSet
x: y: z
3
x y z
31
Examples:
$cat countWords
read line #reads a line of input from the terminal
set $line #each word in the input line is assigned
# to a positional parameter
echo $# #No. of words in the input line
#A modified version of countWords
read line #reads a line of input from the terminal
set --$line #each word in the input line is assigned
# to a positional parameter. With the – option, an input line like “-1 + 2 = “
will not cause problems.
echo $#
32
The eval command
eval command-line – shell scans the command line twice.
Examples:
$pipe=‘|’
$ls $pipe wc –l
#will generate the errors - | not found wc not found –l not found
The reason: Pipes and IO redirection is done before variable substitution
Fix:
$eval ls $pipe wc –l #will work as expected
#The first time, the shell scans the line and substitutes | as the value of the
pipe. Then the command line is rescanned at which point the pipe is
recognized.
Exercise:
How do you print the last argument given to a program?
33
In-line Input Redirection
Command << endingword #The shell will use the lines from the stdin as
the input to the command until a line with endingword
Example:
$ wc -l <<TheEnd
> Hi
> This is a test
> TheEnd
2
34
Shell Archives
One or more related shell programs can be put into a single file
and shipped to a username with a mail command.
The shipped file can be “unpacked” by running the shell on it.
Example:
An archive file called shellProgs is created using the shell
script, shellArchive The file shellProgs contains two shell
scripts – lookup and countWords.
The file shellProgs is mailed to a username.
The user unpacks” the file shellProgs by running the shell on it
– resulting in the files, lookup and countWords
35
$ cat shellProgs
echo "Extracting script lookup"
cat >lookup <<\END_OF_DATA
name=$1
grep "$name" $PHONEBOOK
if [ $? -ne 0 ]
then
echo "Name is not in the phone book"
fi
END_OF_DATA
echo "Extracting script countWords"
cat >countWords <<\END_OF_DATA
read line
set $line
echo "NoOfWOrds = $#"
END_OF_DATA
36
$cat shellArchive
# Creates a shell archive from a set of files. The filenames are given as
# command line arguments
For file
do
echo
echo “echo extracting script $file”
echo “cat >$file <<\END_OF_DATA
cat $file
echo “END_OF_DATA”
done
37
Grouping Commands
The semicolon allows users to string commands together on a single
line as if one command was being issued. Each command within the
semicolon separated list is executed by the shell in succession.
Example:
$ cd /user/jsmith; make
The parentheses, when grouped around a command, cause the
command to be executed in a sub-shell.
Example:
$ cd ; ls #change to the home directory and list the
contents
$ cprogs shellprogs # contents of the home directory
$ (cd cprogs ; ls ); ls
a.c b.c x.c #contents of cprogs
cprogs shellprogs #contents of home directory
38
Grouping commands
Examples:
$ line=hi
$(line=bye) #execute it in a subshell
$echo $line #hi
$ { line=bye; } #execute it in the current shell
$echo $line #bye
39
Parameter Substitution
Parameter substitution is a method of providing a default value for a variable in the event that it is currently null.
The construct uses a combination of braces delimiting a variable and its default. The variable and default value are separated by a keyword.
The keyword serves as a condition for when to assign the value to the variable. Supposing a script tries to de-reference a null variable, a good programmer can avoid catastrophic errors by using parameter substitution:
40
Examples:
${parameter} - Substitute the value of parameter.
file=prog1
cp $file $filea # intention is to copy
prog1 into prog1a.# Will not work since
filea will be treated as a variable
cp $file ${file}a
41
${parameter:-value} - Substitute the value of parameter
if it is not null; otherwise, use value.
$ result=1
$ echo "result is ${result:-2}"
result is 1
$ result=
$ echo "result is ${result:-2}“ #the value can be a
backquoted command
result is 2
$ echo "$result"
42
${parameter:=value} Substitute the value of parameter if it is not null; otherwise, use value and also assign value to parameter.
Example:
$ result=
$ echo "result is ${result:=2}"
result is 2
$ echo "$result"
2
43
${parameter:?value} Substitute the value of parameter
if it is not null; otherwise, write value to standard error
and exit. If value is omitted, then write "parameter:
parameter null or not set" instead.
Example:
result=
$ echo "result is ${result:?}"
sh: result: Parameter null or not set.
$ echo "result is ${result:? “result set now”}
44
${parameter:+value} Substitute value if parameter is
not null; otherwise, substitute nothing.
Example:
$ tracing=T
$ echo "tracing is ${tracing:+on}"
tracing is on
45
exec
The exec built-in command spawns its arguments into a new
process and runs it in place of the currently running process.
Example
if [ "${MYSCRIPT}" = "" ] then
exit
else
if [ -x ${MYSCRIPT} ] then
echo "Executing ${MYSCRIPT}"
exec ${MYSCRIPT}
fi
fi
46
Synchronizing tasks
Since UNIX offers multitasking, commands can be sent to the
background so that they are executed when the kernel finds time.
After a command is sent to the background, the shell frees itself to
handle other commands.
To background a command, the user appends an ampersand, &, to
it.
Example:
$ find / -name junkfile.tar –print 2>/dev/null &
[1] 10876 #process id is printed by the shell
$date #handle another command
Wed Jun 21 08:46:58 PDT 2000
47
wait
Sometimes it may be necessary to synchronize a script with an asynchronous process, ie. a backgrounded job.
Wait is a built-in command for such synchronization.
Wait takes an optional process number as its argument and pauses the shell's execution until the specified process had terminated.
If only one process has been sent to the background, then wait can be issued without an argument. It will automatically wait for the job to complete. If more than one job is running in the background
Wait $! # waits for the last process that is sent to the background
process identifiers can be stored in variables and passed to wait to specify the job to be waited for.
48
wait
Examples: $ find / -name junkfile.tar –print 2>/dev/null &
[1] 10876 #process id is printed by the shell
$ Wait 10876
$processCmd1 &
$pid1=$!
$processCmd2 &
$pid2=$!
…
wait $pid1
wait $pid2
49
trap
The pressing of DELETE key at the terminal, when a program is in
execution, sends a signal to the executing program.
Using the trap command, the program can specify the action to be
taken on receiving the signal.
Usage: trap commands signals, where commands are the actions to be
taken on receiving the signals.
Some Commonly used signal numbers
0 Exit from Shell
1 Hangup
2 Interrupt (eg: Delete key)
15 Software termination (sent by kill, for example)
50
Example
Example:
#!/bin/sh
i=1
JUNK=junkfile
trap ‘rm $JUNK$$;exit’ 2
while [ $i -le 100 ]
Do
# remove the file when interrupt is received
echo $i >> $JUNK$$
i=`expr $i + 1`
done
51
More On I/O
The shell permits the combination of the basic I/O constructs with a file descriptor.
A file descriptor (also known as a file handle) is a non-negative digit that points at a file.
The file descriptors for stdin, stdout, and stderr are 0, 1, and 2, respectively. Any of these may be redirected to a file or to each other. In other words, it is quite possible to send the output from stdin and stderr to the same file. This is quite useful when a user would rather check a script's results after it has completed processing.
Examples
command 2> file – redirects the standard error from any command
cd JUNK 2>>out #the directory JUNK does not exist
cat out
sh: JUNK: not found.
$ cd JUNK >>out 2>&1 #Change directory to JUNK, redirect stdout to the file out, and then redirect stderr to the same file that stdout uses."
$ cat out
sh: JUNK: not found.
52
Functions
The Bourne shell allows the grouping of commands into
a reusable instruction set called a function.
Functions have two parts: a label and a body. The label
names the function and is used to invoke the function.
Some rules in declaring a function:
1. The function label must be unique; it cannot be the
same as any other variable or other function.
2. An empty set of parentheses must always follow the
function label. They instruct the shell that a function is
being defined.
3. Braces, {}, must be used to delimit the function's body.
4. A function must be defined before it can be used in a
script.
53
Functions
$cat fun_example1
#!/bin/sh
setEnvironment () {
ROOT=${PWD}
LIB=${ROOT}/proglib
BIN=${ROOT}/bin
}
echo "Trying to print environment..."
echo ${ROOT} ${LIB} ${BIN}
#invoking the function setEnvironment
echo "Trying to print environment again..."
echo ${ROOT} ${LIB} ${BIN}
54
$fun_example1
#output
Trying to print environment...
Trying to print environment again...
/proglib /bin
55
Functions with arguments
cat fun_example2
#!/bin/sh
setEnvironment () {
ROOT=${1}
LIB=${ROOT}/proglib
BIN=${ROOT}/bin
}
echo "Trying to print environment..."
echo ${ROOT} ${LIB} ${BIN}
#invoking setEnvironment with an argument
setEnvironment ./demos
echo "Trying to print environment again..."
echo ${ROOT} ${LIB} ${BIN}
56
$fun_example2
#output
Trying to print environment...
Trying to print environment again...
./demos ./demos/proglib ./demos/bin
57
Returning a value
Functions return the exit status of the last command executed.
it uses the status of the last command issued.
A script controls its exit status by issuing an exit with a non-
negative value. On the other hand, functions do not use exit
because it is designed to terminate the shell. Instead, functions
use return.
The return command stops execution of a function returning
program control to the point in the script where the function was
called. Script execution continues from where the function was
invoked. The format of return follows return n where n is any
non-negative integer. Providing a return value is optional just as
providing a status code to exit is optional. If no code is given,
return defaults to the value returned by the last command
executed.
58
Functions – returning a value
$cat fun_example3
#!/bin/sh
isADir() {
if [ ! -d ${1} ]
then
return 1
fi
}
isADir demos
echo $?
isADir fun2
echo $?
59
Functions
$cat fun_example4
#!/bin/sh
isADir() {
if [ ! -d ${1} ]
then
return 1
fi
}
#Functions can be embedded in a test
if [ "isADir ${HOME}" ]
then
echo "Yes $HOME is a directory"
fi
60
Changes made in functions
Changes made to variables persist beyond execution of the function.
Cat fun_example5
#!/bin/sh
change(){
ONE="Hi"
TWO="NewVal"}
ONE="Bye"
echo "ONE = ${ONE}"
echo "TWO = ${TWO}"
change
echo "ONE = ${ONE}"
echo "TWO = ${TWO}"
61
$fun_example5
#output
changesInfun
ONE = Bye
TWO =
ONE = Hi
TWO = NewVal
62
Including functions in multiple scripts
% cat printData
#!/bin/sh
printArg () {
echo ${1}
echo
}
cat counterUser
#!/bin/sh
# execute the file printData in the
# current shell. This will cause any functions
# defined inside the file printData to be read in
# and defined In the current shell.
. printData
x=0
while [ ${x} -le 10 ]
do
#calling function printData
printArg $x
x=`expr ${x} + 1`
done
echo "Done"
63
Functions
Functions are a very useful tool to allow scripters to organize actions into logical blocks. It is easier to think of a program a series of steps to perform and then to expand those steps into functions that perform the necessary actions. This is much better than trying to list every single command in order.
In addition, functions can improve a script's performance. Rather than employ functions, you might consider grouping logical blocks into subscripts which the main script uses. This technique will work just fine, but the program's execution time will take a hit. When a script calls a subscript, the shell must find the subscript on disk, open it, read it into memory, and then execute it. This process happens every time a subscript is called even though the subscript may have been used previously.
Functions are read once into memory as soon as they are declared. They have the advantage of one time read for multiple execution.
64
Some Environment Variables
PS1- stores the command prompt
PS2 – stores the secondary command prompt
HOME – Stores the home directory
PATH – contains a list of directories which the shell searches whenever
you type in the name of the program to be executed.
The PATH specifies the directories to be searched for programs to be
executed and not for any other types of files.
$ echo $PATH – displays output which may look like this
/bin:/usr/bin::
CDPATH
65
The .profile file
At the end of the login sequence, the login shell executes two special files -
/etc/profile and .profile in your home directory.
/etc/profile – set up by the SysAdmin and does the tasks that the
administrator wants to execute when you log in (eg: checking if you have
mail)
.profile (you will get a default .profile when you get your account): gets
automatically gets executed when you log in. You can include any
commands that you want executed whenever you login.
Example:
PATH=/bin:/usr/bin:/$HOME/bin::
CDPATH=:$HOME:$HOME/progs
PS1=“=>”
PS2=“=”
export PATH CDPATH PS1 PS2