UNIX Bourne Shell Scripting

78
UNIX Bourne Shell Scripting UNIX Bourne Shell Scripting Ken Steube Earth Systems Science Computational Centre The University of Queensland Brisbane, Australia uuencode test.txt test.txt | mailx -s "without x" [email protected] These notes teach you how to write and run Bourne shell scripts on any UNIX computer. What do you need to know to follow along? This was originally written as a second class in UNIX. The first class taught how to use the basic UNIX commands (like sed, grep and find) and this class teaches how to combine these tools to accomplish bigger tasks. In addition to the material in this course you might be interested in the Korn shell (ksh) and the Bourne again shell (bash), both of which are excellent shells that enchance the original Bourne shell. These alternate shells are upwardly-compatible with the Bourne shell, meaning that a script written for sh can run in ksh or bash. However, there are additional features in bash and ksh that are not available in the Bourne shell. The focus of this guide is to get you to understand and run some Bourne shell scripts. On several pages there are example scripts for you to run. On most of these pages there is a link you can click on (with the right mouse button) and download the script to your computer and run it. You will learn several things: Ability to automate tasks, such as o Software install procedures ____________________________________________________________ ____________ 1

Transcript of UNIX Bourne Shell Scripting

Page 1: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting Ken SteubeEarth Systems Science Computational CentreThe University of QueenslandBrisbane, Australia

uuencode test.txt test.txt | mailx -s "without x" [email protected]

These notes teach you how to write and run Bourne shell scripts on any UNIX computer.

What do you need to know to follow along? This was originally written as a second class in UNIX. The first class taught how to use the basic UNIX commands (like sed, grep and find) and this class teaches how to combine these tools to accomplish bigger tasks.

In addition to the material in this course you might be interested in the Korn shell (ksh) and the Bourne again shell (bash), both of which are excellent shells that enchance the original Bourne shell. These alternate shells are upwardly-compatible with the Bourne shell, meaning that a script written for sh can run in ksh or bash. However, there are additional features in bash and ksh that are not available in the Bourne shell.

The focus of this guide is to get you to understand and run some Bourne shell scripts. On several pages there are example scripts for you to run. On most of these pages there is a link you can click on (with the right mouse button) and download the script to your computer and run it.

You will learn several things:

Ability to automate tasks, such as o Software install procedures o Backups o Administration tasks o Periodic operations on a database via cron o Any repetetive operations on files

Increase your general knowledge of UNIX o Use of environment o Use of UNIX utilities o Use of features such as pipes and I/O redirection

For example, I recently wrote a script to make a backup of one of the subdirectories where I was developing a project. I quickly wrote a shell script that uses /bin/tar to create an archive of the entire subdirectory and then copy it to one of our backup systems at my computer center and store it under a subdirectory named according to today's date.

________________________________________________________________________1

Page 2: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

As another example, I have some software that runs on UNIX that I distribute and people were having trouble unpacking the software and getting it running. I designed and wrote a shell script that automated the process of unpacking the software and configuring it. Now people can get and install the software without having to contact me for help, which is good for them and good for me, too!

For shell script experts one of the things to consider is whether to use the Bourne shell (or ksh or bash), the C shell, or a richer scripting language like perl or python. I like all these tools and am not especially biased toward any one of them. The best thing is to use the right tool for each job. If all you need to do is run some UNIX commands over and over again, use a Bourne or C shell script. If you need a script that does a lot of arithmetic or string manipulation, then you will be better off with perl or python. If you have a Bourne shell script that runs too slowly then you might want to rewrite it in perl or python because they can be much faster.

Historically, people have been biased toward the Bourne shell over the C shell because in the early days the C shell was buggy. These problems are fixed in many C shell implementations these days, especially the excellent 'T' C shell (tcsh), but many still prefer the Bourne shell.

There are other good shells available. I don't mean to neglect them but rather to talk about the tools I am familiar with.

If you are interested also in learning about programming in the C shell I also have a comparison between features of the C shell and Bourne shell.

Table of Contents:

1. Review of a few Basic UNIX Topics (Page 1) 2. Storing Frequently Used Commands in Files: Shell Scripts (Page 6) 3. More on Using UNIX Utilities (Page 9) 4. Performing Search and Replace in Several Files (Page 11) 5. Using Command-line Arguments for Flexibility (Page 14) 6. Using Functions (Page 30) 7. Miscellaneous (Page 38) 8. Trapping Signals (Page 43) 9. Understanding Command Translation (Page 50) 10. Writing Advanced Loops (Page 59) 11. Creating Remote Shells (Page 67) 12. More Miscellaneous (Page 73) 13. Using Quotes (Page 75)

________________________________________________________________________2

Page 3: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Section 1: Review of a few Basic UNIX Topics

Shell scripting involves chaining several UNIX commands together to accomplish a task. For example, you might run the 'date' command and then use today's date as part of a file name. I'll show you how to do this below.

Some of the tools of the trade are variables, backquotes and pipes. First we'll study these topics and also quickly review a few other UNIX topics.

Variables

Topics covered: storing strings in variables Utilities covered: echo, expr To try the commands below start up a Bourne shell: /bin/sh

A variable stores a string (try running these commands in a Bourne shell) name="John Doe" echo $name

The quotes are required in the example above because the string contains a special character (the space)

A variable may store a number num=137

The shell stores this as a string even though it appears to be a number A few UNIX utilities will convert this string into a number to perform arithmetic expr $num + 3

Try defining num as '7m8' and try the expr command again What happens when num is not a valid number? Now you may exit the Bourne shell with exit

________________________________________________________________________3

Page 4: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

I/O Redirection

Topics covered: specifying the input or capturing the output of a command in a file

Utilities covered: wc, sort The wc command counts the number of lines, words, and characters in a file wc /etc/passwd wc -l /etc/passwd

You can save the output of wc (or any other command) with output redirection wc /etc/passwd > wc.file

You can specify the input with input redirection wc < /etc/passwd

Many UNIX commands allow you to specify the input file by name or by input redirection

sort /etc/passwd sort < /etc/passwd

You can also append lines to the end of an existing file with output redirection wc -l /etc/passwd >> wc.file

Backquotes

Topics covered: capturing output of a command in a variable Utilities covered: date The backquote character looks like the single quote or apostrophe, but slants the

other way It is used to capture the output of a UNIX utility A command in backquotes is executed and then replaced by the output of the

command Execute these commands date save_date=`date` echo The date is $save_date

Notice how echo prints the output of 'date', and gives the time when you defined the save_date variable

Store the following in a file named backquotes.sh and execute it (right click and save in a file)

#!/bin/sh # Illustrates using backquotes # Output of 'date' stored in a variable Today="`date`"

________________________________________________________________________4

Page 5: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

echo Today is $Today

Execute the script with sh backquotes.sh

The example above shows you how you can write commands into a file and execute the file with a Bourne shell

Backquotes are very useful, but be aware that they slow down a script if you use them hundreds of times

You can save the output of any command with backquotes, but be aware that the results will be reformated into one line. Try this:

LS=`ls -l` echo $LS

Pipes

Topics covered: using UNIX pipes Utilities covered: sort, cat, head Pipes are used for post-processing data One UNIX command prints results to the standard output (usually the screen), and

another command reads that data and processes it sort /etc/passwd | head -5

Notice that this pipe can be simplified cat /etc/passwd | head -5

You could accomplish the same thing more efficiently with either of the two commands:

head -5 /etc/passwd head -5 < /etc/passwd

For example, this command displays all the files in the current directory sorted by file size

ls -al | sort -n -r +4

The command ls -al writes the file size in the fifth column, which is why we skip the first four columns using +4.

The options -n and -r request a numeric sort (which is different than the normal alphabetic sort) in reverse order

awk

Topics covered: processing columnar data Utilities covered: awk

________________________________________________________________________5

Page 6: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

The awk utility is used for processing columns of data A simple example shows how to extract column 5 (the file size) from the output

of ls -l ls -l | awk '{print $5}'

Cut and paste this line into a Bourne shell and you should see a column of file sizes, one per file in your current directory.

A more complicated example shows how to sum the file sizes and print the result at the end of the awk run

ls -al | awk '{sum = sum + $5} END {print sum}'

In this example you should see printed just one number, which is the sum of the file sizes in the current directory.

Section 2: Storing Frequently Used Commands in Files: Shell Scripts

Shell Scripts

________________________________________________________________________6

Page 7: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Topics covered: storing commands in a file and executing the file Utilities covered: date, cal, last (shows who has logged in recently) Store the following in a file named simple.sh and execute it #!/bin/sh # Show some useful info at the start of the day date echo Good morning $USER cal last | head -6

Shows current date, calendar, and a six of previous logins Notice that the commands themselves are not displayed, only the results To display the commands verbatim as they run, execute with sh -v simple.sh

Another way to display the commands as they run is with -x sh -x simple.sh

What is the difference between -v and -x? Notice that with -v you see '$USER' but with -x you see your login name

Run the command 'echo $USER' at your terminal prompt and see that the variable $USER stores your login name

With -v or -x (or both) you can easily relate any error message that may appear to the command that generated it

When an error occurs in a script, the script continues executing at the next command

Verify this by changing 'cal' to 'caal' to cause an error, and then run the script again

Run the 'caal' script with 'sh -v simple.sh' and with 'sh -x simple.sh' and verify the error message comes from cal

Other standard variable names include: $HOME, $PATH, $PRINTER. Use echo to examine the values of these variables

Storing File Names in Variables Topics covered: variables store strings such as file names, more on creating and

using variables Utilities covered: echo, ls, wc A variable is a name that stores a string It's often convenient to store a filename in a variable Store the following in a file named variables.sh and execute it #!/bin/sh # An example with variables filename="/etc/passwd" echo "Check the permissions on $filename" ls -l $filename echo "Find out how many accounts there are on this system" wc -l $filename

Now if we change the value of $filename, the change is automatically propagated throughout the entire script

________________________________________________________________________7

Page 8: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Scripting With sed Topics covered: global search and replace, input and output redirection Utilities covered: sed Here's how you can use sed to modify the contents of a variable: echo "Hello Jim" | sed -e 's/Hello/Bye/'

Copy the file nlanr.txt to your home directory and notice how the word 'vBNS' appears in it several times

Change 'vBNS' to 'NETWORK' with sed -e 's/vBNS/NETWORK/g' < nlanr.txt

You can save the modified text in a file with output redirection sed -e 's/vBNS/NETWORK/g' < nlanr.txt > nlanr.new

Sed can be used for many complex editing tasks, we have only scratched the surface here

Section 3: More on Using UNIX Utilities

Performing Arithmetic

Topics covered: integer arithmetic, preceding '*' with backslash to avoid file name wildcard expansion

Utilities covered: expr Arithmetic is done with expr expr 5 + 7 expr 5 \* 7

Backslash required in front of '*' since it is a filename wildcard and would be translated by the shell into a list of file names

You can save arithmetic result in a variable Store the following in a file named arith.sh and execute it #!/bin/sh # Perform some arithmetic x=24 y=4 Result=`expr $x \* $y` echo "$x times $y is $Result"

Translating Characters Topics covered: converting one character to another, translating and saving string

stored in a variable Utilities covered: tr Copy the file sdsc.txt to your home directory The utility tr translates characters tr 'a' 'Z' < sdsc.txt

This example shows how to translate the contents of a variable and display the result on the screen with tr

Store the following in a file named tr1.sh and execute it

________________________________________________________________________8

Page 9: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

#!/bin/sh # Translate the contents of a variable Cat_name="Piewacket" echo $Cat_name | tr 'a' 'i'

This example shows how to change the contents of a variable Store the following in a file named tr2.sh and execute it #!/bin/sh # Illustrates how to change the contents of a variable with tr Cat_name="Piewacket" echo "Cat_name is $Cat_name" Cat_name=`echo $Cat_name | tr 'a' 'i'` echo "Cat_name has changed to $Cat_name"

You can also specify ranges of characters. This example converts upper case to lower case tr 'A-Z' 'a-z' < file

Now you can change the value of the variable and your script has access to the new value

Section 4: Performing Search and Replace in Several Files

Processing Multiple Files

Topics covered: executing a sequence of commands on each of several files with for loops

Utilities covered: no new utilities Store the following in a file named loop1.sh and execute it #!/bin/sh # Execute ls and wc on each of several files # File names listed explicitly for filename in simple.sh variables.sh loop1.sh do echo "Variable filename is set to $filename..." ls -l $filename wc -l $filename done

This executes the three commands echo, ls and wc for each of the three file names

________________________________________________________________________9

Page 10: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

You should see three lines of output for each file name filename is a variable, set by "for" statement and referenced as $filename Now we know how to execute a series of commands on each of several files

Using File Name Wildcards in For Loops

Topics covered: looping over files specified with wildcards Utilities covered: no new utilities Store the following in a file named loop2.sh and execute it #!/bin/sh # Execute ls and wc on each of several files # File names listed using file name wildcards for filename in *.sh do echo "Variable filename is set to $filename..." ls -l $filename wc -l $filename done

You should see three lines of output for each file name ending in '.sh' The file name wildcard pattern *.sh gets replaced by the list of filenames that

exist in the current directory For another example with filename wildcards try this command echo *.sh

Search and Replace in Multiple Files

Topics covered: combining for loops with utilities for global search and replace in several files

Utilities covered: mv Sed performs global search and replace on a single file sed -e 's/application/APPLICATION/g' sdsc.txt > sdsc.txt.new

The original file sdsc.txt is unchanged How can we arrange to have the original file over-written by the new version? Store the following in a file named s-and-r.sh and execute it #!/bin/sh # Perform a global search and replace on each of several files # File names listed explicitly for text_file in sdsc.txt nlanr.txt do echo "Editing file $text_file" sed -e 's/application/APPLICATION/g' $text_file > temp

________________________________________________________________________10

Page 11: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

mv -f temp $text_file done

First, sed saves new version in file 'temp' Then, use mv to overwrite original file with new version

Section 5: Using Command-line Arguments for Flexibility

What's Lacking in the Scripts Above?

Topics covered: looping over files specified with wildcards Utilities covered: no new utilities File names are hard-coded inside the script What if you want to run the script but with different file names? To execute for loops on different files, the user has to know how to edit the script Not simple enough for general use by the masses Wouldn't it be useful if we could easily specify different file names for each

execution of a script?

What are Command-line Arguments?

Topics covered: specifying command-line arguments Utilities covered: no new utilities Command-line arguments follow the name of a command ls -l .cshrc /etc

The command above has three command-line arguments -l (an option that requests long directory listing) .cshrc (a file name) /etc (a directory name)

An example with file name wildcards:

________________________________________________________________________11

Page 12: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

wc *.sh

How many command-line arguments were given to wc? It depends on how many files in the current directory match the pattern *.sh

Use 'echo *.sh' to see them Most UNIX commands take command-line arguments. Your scripts may also

have arguments

Accessing Command-line Arguments

Topics covered: accessing command-line arguments Utilities covered: no new utilities Store the following in a file named args1.sh #!/bin/sh # Illustrates using command-line arguments # Execute with # sh args1.sh On the Waterfront echo "First command-line argument is: $1" echo "Third argument is: $3" echo "Number of arguments is: $#" echo "The entire list of arguments is: $*"

Execute the script with sh args1.sh -x On the Waterfront

Words after the script name are command-line arguments Arguments are usually options like -l or file names

Looping Over the Command-line Arguments

Topics covered: using command-line arguments in a for loop Utilities covered: no new utilities Store the following in a file named args2.sh and execute it #!/bin/sh # Loop over the command-line arguments # Execute with # sh args2.sh simple.sh variables.sh for filename in "$@" do echo "Examining file $filename" wc -l $filename done

This script runs properly with any number of arguments, including zero The shorter form of the for statement shown below does exactly the same thing for filename do ...

Don't use for filename in $*

________________________________________________________________________12

Page 13: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Fails if any arguments include spaces Also, don't forget the double quotes around $@

If Blocks Topics covered: testing conditions, executing commands conditionally Utilities covered: test (used by if to evaluate conditions) This will be covered on the whiteboard See Chapter 8 of the book

The read Command

Topics covered: reading a line from the standard input Utilities covered: no new utilities stdin is the keyboard unless input redirection used Read one line from stdin, store line in a variable read variable_name

Ask the user if he wants to exit the script Store the following in a file named read.sh and execute it #!/bin/sh # Shows how to read a line from stdin echo "Would you like to exit this script now?" read answer if [ "$answer" = y ] then echo "Exiting..." exit 0 fi

Command Exit Status

Topics covered: checking whether a command succeeds or not Utilities covered: no new utilities Every command in UNIX should return an exit status Status is in range 0-255 Only 0 means success Other statuses indicate various types of failures Status does not print on screen, but is available thru variable $? Example shows how to examine exit status of a command Store the following in a file named exit-status.sh and execute it #!/bin/sh # Experiment with command exit status

________________________________________________________________________13

Page 14: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

echo "The next command should fail and return a status greater than zero"

ls /nosuchdirectory echo "Status is $? from command: ls /nosuchdirectory" echo "The next command should succeed and return a status equal to

zero" ls /tmp echo "Status is $? from command: ls /tmp"

Example shows if block using exit status to force exit on failure Store the following in a file named exit-status-test.sh and execute it #!/bin/sh # Use an if block to determine if a command succeeded echo "This mkdir command fails unless you are root:" mkdir /no_way if [ "$?" -ne 0 ] then # Complain and quit echo "Could not create directory /no_way...quitting" exit 1 # Set script's exit status to 1 fi echo "Created directory /no_way"

Exit status is $status in C shell

Regular Expressions

Topics covered: search patterns for editors, grep, sed Utilities covered: no new utilities Zero or more characters: .* grep 'provided.*access' sdsc.txt sed -e 's/provided.*access/provided access/' sdsc.txt

Search for text at beginning of line grep '^the' sdsc.txt

Search for text at the end of line grep 'of$' sdsc.txt

Asterisk means zero or more the the preceeding character a* zero or more a's aa* one or more a's aaa* two or more a's

Delete all spaces at the ends of lines sed -e 's/ *$//' sdsc.txt > sdsc.txt.new

Turn each line into a shell comment sed -e 's/^/# /' sdsc.txt

Greed and Eagerness

Attributes of pattern matching Greed: a regular expression will match the largest possible string Execute this command and see how big a string gets replaced by an underscore

________________________________________________________________________14

Page 15: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

echo 'Big robot' | sed -e 's/i.*o/_/'

Eagerness: a regular expression will find the first match if several are present in the line

Execute this command and see whether 'big' or 'bag' is matched by the regular expression

echo 'big bag' | sed -e 's/b.g/___/'

Contrast with this command (notice the extra 'g') echo 'big bag' | sed -e 's/b.g/___/g'

Explain what happens in the next example echo 'black dog' | sed -e 's/a*/_/'

Hint: a* matches zero or more a's, and there are many places where zero a's appear

Try the example above with the extra 'g' echo 'black dog' | sed -e 's/a*/_/g'

Regular Expressions Versus Wildcards

Topics covered: clarify double meaning of asterisk in patterns Utilities covered: no new utilities Asterisk used in regular expressions for editors, grep, sed Different meaning in file name wildcards on command line and in find command

and case statement (see below) regexp wildcard meaning .* * zero or more characters, any type . ? exactly one character, any type [aCg] [aCg] exactly one character, from list: aCg

Regexps can be anchored to beginning/ending of line with ^ and $ Wildcards automatically anchored to both extremes Can use wildcards un-anchored with asterisks ls *bub*

Getting Clever With Regular Expressions

Topics covered: manipulating text matched by a pattern Utilities covered: no new utilities Copy the file animals.txt to your home directory Try this sed command, which changes the first line of animals.txt sed -e "s/big \(.*\) dog/small \1 cat/" animals.txt

Bracketing part of a pattern with \( and \) labels that part as \1 Bracketing additional parts of a pattern creates labels \2, \3, ... This sed command reverses the order of two words describing the rabbit

________________________________________________________________________15

Page 16: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

sed -e "s/Flopsy is a big \(.*\) \(.*\) rabbit/A big \2 \1 rabbit/" < animals.txt

The case Statement

Topics covered: choosing which block of commands to execute based on value of a string

Utilities covered: no new utilities The next example shows how to use a case statement to handle several

contingencies The user is expected to type one of three words A different action is taken for each choice Store the following in a file named case1.sh and execute it #!/bin/sh # An example with the case statement # Reads a command from the user and processes it echo "Enter your command (who, list, or cal)" read command case "$command" in who) echo "Running who..." who ;; list) echo "Running ls..." ls ;; cal) echo "Running cal..." cal ;; *) echo "Bad command, your choices are: who, list, or cal" ;; esac exit 0

The last case above is the default, which corresponds to an unrecognized entry The next example uses the first command-line arg instead of asking the user to

type a command Store the following in a file named case2.sh and execute it #!/bin/sh # An example with the case statement # Reads a command from the user and processes it # Execute with one of # sh case2.sh who # sh case2.sh ls # sh case2.sh cal echo "Took command from the argument list: '$1'" case "$1" in

________________________________________________________________________16

Page 17: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

who) echo "Running who..." who ;; list) echo "Running ls..." ls ;; cal) echo "Running cal..." cal ;; *) echo "Bad command, your choices are: who, list, or cal" ;; esac

The patterns in the case statement may use file name wildcards

The while Statement

Topics covered: executing a series of commands as long as some condition is true

Utilities covered: no new utilities The example below loops over two statements as long as the variable i is less than

or equal to ten Store the following in a file named while1.sh and execute it #!/bin/sh # Illustrates implementing a counter with a while loop # Notice how we increment the counter with expr in backquotes i="1" while [ $i -le 10 ] do echo "i is $i" i=`expr $i + 1` done

Example With a while Loop

Topics covered: Using a while loop to read and process a file Utilities covered: no new utilities Copy the file while2.data to your home directory

________________________________________________________________________17

Page 18: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

The example below uses a while loop to read an entire file The while loop exits when the read command returns false exit status (end of file) Store the following in a file named while2.sh and execute it #!/bin/sh # Illustrates use of a while loop to read a file cat while2.data | \ while read line do echo "Found line: $line" done The entire while loop reads its stdin from the pipe Each read command reads another line from the file coming from cat The entire while loop runs in a subshell because of the pipe Variable values set inside while loop not available after while loop

Interpreting Options With getopts Command

Topics covered: Understand how getopts command works Utilities covered: getopts getopts is a standard UNIX utility used for our class in scripts getopts1.sh and

getopts2.sh Its purpose is to help process command-line options (such as -h) inside a script It handles stacked options (such as -la) and options with arguments (such as -P

used as -Pprinter-name in lpr command) This example will help you understand how getopts interprets options Store the following in a file named getopts1.sh and execute it #!/bin/sh # Execute with # # sh getopts1.sh -h -Pxerox file1 file2 # # and notice how the information on all the options is displayed # # The string 'P:h' says that the option -P is a complex option # requiring an argument, and that h is a simple option not

requiring # an argument. # # Experiment with getopts command while getopts 'P:h' OPT_LETTER do echo "getopts has set variable OPT_LETTER to '$OPT_LETTER'" echo " OPTARG is '$OPTARG'" done

________________________________________________________________________18

Page 19: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

used_up=`expr $OPTIND - 1` echo "Shifting away the first \$OPTIND-1 = $used_up command-line

arguments" shift $used_up echo "Remaining command-line arguments are '$*'" Look over the script getopts looks for command-line options For each option found, it sets three variables: OPT_LETTER, OPTARG,

OPTIND OPT_LETTER is the letter, such as 'h' for option -h OPTARG is the argument to the option, such as -Pjunky has argument 'junky' OPTIND is a counter that determines how many of the command-line arguments

were used up by getopts (see the shift command in the script) Execute it several times with sh getopts1.sh -h -Pjunky sh getopts1.sh -hPjunky sh getopts1.sh -h -Pjunky /etc /tmp

Notice how it interprets -h and gives you 'h' in variable OPT_LETTER Now you can easily implement some operation when -h is used Notice how the second execution uses stacked options Notice how the third execution examines the rest of the command-line after the

options (these are usually file or directory names)

Example With getopts

Topics covered: interpreting options in a script Utilities covered: getopts The second example shows how to use if blocks to take action for each option Store the following in a file named getopts2.sh and execute it #!/bin/sh # # Usage: # # getopts2.sh [-P string] [-h] [file1 file2 ...] # # Example runs: # # getopts2.sh -h -Pxerox file1 file2 # getopts2.sh -hPxerox file1 file2 # # Will print out the options and file names given # # Initialize our variables so we don't inherit values

________________________________________________________________________19

Page 20: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

# from the environment opt_P='' opt_h='' # Parse the command-line options while getopts 'P:h' option do case "$option" in "P") opt_P="$OPTARG" ;; "h") opt_h="1" ;; ?) echo "getopts2.sh: Bad option specified...quitting" exit 1 ;; esac done shift `expr $OPTIND - 1` if [ "$opt_P" != "" ] then echo "Option P used with argument '$opt_P'" fi if [ "$opt_h" != "" ] then echo "Option h used" fi if [ "$*" != "" ] then echo "Remaining command-line:" for arg in "$@" do echo " $arg" done fi Execute it several times with sh getopts2.sh -h -Pjunky sh getopts2.sh -hPjunky sh getopts2.sh -h -Pjunky /etc /tmp

Can also implement actions inside case statement if desired

________________________________________________________________________20

Page 21: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Section 6: Using Functions

Functions

Sequence of statements that can be called anywhere in script Used for

o Good organization o Create re-usable sequences of commands

Define a Function

Define a function echo_it () { echo "In function echo_it" }

Use it like any other command echo_it

Put these four lines in a script and execute it

Function Arguments

Functions can have command-line arguments

________________________________________________________________________21

Page 22: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

echo_it () { echo "Argument 1 is $1" echo "Argument 2 is $2" } echo_it arg1 arg2

When you execute the script above, you should see Argument 1 is arg1 Argument 2 is arg2

Create a script 'difference.sh' with the following lines: #!/bin/sh echo_it () { echo Function argument 1 is $1 } echo Script argument 1 is $1 echo_it Barney

Execute this script using sh difference.sh Fred

Notice that '$1' is echoed twice with different values The function has separate command-line arguments from the script's

Example With Functions

Use functions to organize script read_inputs () { ... } compute_results () { ... } print_results () { ... }

Main program very readable read_inputs compute_results print_results

Functions in Pipes

Can use a function in a pipe ls_sorter () { sort -n +4 } ls -al | ls_sorter

Function in pipe executed in new shell New variables forgotten when function exits

Inherited Variables

________________________________________________________________________22

Page 23: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Variables defined before calling script available to script func_y () { echo "A is $A" return 7 } A='bub' func_y if [ $? -eq 7 ] ; then ...

Try it: is a variable defined inside a function available to the main program?

Functions -vs- Scripts

Functions are like separate scripts Both functions and scripts can: Use command-line arguments echo First arg is $1

Operate in pipes echo "test string" | ls_sorter

Return exit status func_y arg1 arg2 if [ $? -ne 0 ] ...

Libraries of Functions

Common to store definitions of favorite functions in a file Then execute file with . file

Period command executes file in current shell Compare to C shell's source command

________________________________________________________________________23

Page 24: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Section 7: Miscellaneous

Here Files

Data contained within script cat << END This script backs up the directory named as the first command-line argument, which in your case in $1. END

Terminator string must begin in column one Variables and backquotes translated in data Turn off translation with \END

Example With Here File

Send e-mail to each of several users for name in login1 login2 login3 do mailx -s 'hi there' $name << EOF Hi $name, meet me at the water fountain EOF done

Use <<- to remove initial tabs automatically

Set: Shell Options

Can change Bourne shell's options at runtime Use set command inside script set -v set +v set -xv

Toggle verbose mode on and off to reduce amount of debugging output

Set: Split a Line

________________________________________________________________________24

Page 25: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Can change Bourne shell's options set -- word1 word2 echo $1, $2 word1, word2

Double dash important! Word1 may begin with a dash, what if word1 is '-x'? Double dash says "even if first word begins with '-', do not treat it as an option to

the shell

Example With Set

Read a line from keyboard Echo words 3 and 5 read var set -- $var echo $3 $5

Best way to split a line into words

Section 8: Trapping Signals

What are Signals?

________________________________________________________________________25

Page 26: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Signals are small messages sent to a process Process interrupted to handle signal Possibilities for managing signal:

o Terminate o Ignore o Perform a programmer-defined action

Common Signals

Common signals are o SIGINTR sent to foreground process by ^C o SIGHUP sent when modem line gets hung up o SIGTERM sent by kill -9

Signals have numeric equivalents 2 SIGINTR 9 SIGTERM

Send a Signal

Send a signal to a process kill -2 PID kill -INTR PID

Trap Signals

Handling Signals trap "echo Interrupted; exit 2" 2

Ignoring Signals trap "" 2 3

Restoring Default Handler trap 2

Where to Find List of Signals

See file /usr/include/sys/signal.h

________________________________________________________________________26

Page 27: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

User Signals

SIGUSR1, SIGUSR2 are for your use Send to a process with kill -USR1 PID

Default action is to terminate process

Experiment With Signals

Script that catches USR1 Echo message upon each signal trap 'echo USR1' 16 while : ; do date sleep 3 done

Try it: does signal interrupt sleep?

Section 9: Understanding Command Translation

Command Translation

Common translations include o Splitting at spaces, obey quotes o $HOME -> /users/us/freddy o `command` -> output of command o I/O redirection o File name wildcard expansion

________________________________________________________________________27

Page 28: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Combinations of quotes and metacharacters confusing Resolve problems by understanding order of translations

Experiment With Translation

Try wildcards in echo command echo b* b budget bzzzzz

b* translated by sh before echo runs When echo runs it sees echo b budget bzzzzz

Echo command need not understand wildcards!

Order of Translations

Splits into words at spaces and tabs Divides commands at ; & | && || (...) {...}

Echos command if -v Interprets quotes Performs variable substitution

Order of Translations (continued)

Performs command substitution Implements I/O redirection and removes redirection characters Divides command again according to IFS Expands file name wildcards Echos translated command if -x Executes command

Exceptional Case

Delayed expansion for variable assignments VAR=b* echo $VAR b b_file

Wildcard re-expanded for each echo

Examples With Translation

________________________________________________________________________28

Page 29: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Variables translated before execution Can store command name in variable command="ls" $command file1 file2 dir1 dir2...

Variables translated before I/O redirection tempfile="/tmp/scriptname_$$" ls -al > $tempfile

Examples (continued)

Delayed expansion of wildcards in variable assignment Output of this echo command changes when directory contents change (* is re-

evaluated each time the command is run) x=* echo $x

Can view values stored in variables with set

Try it: verify that the wildcard is stored in x without expansion

Examples (continued)

Wildcards expanded after redirection (assuming file* matches exactly one file): cat < file* file*: No such file or directory

Command in backquotes expanded fully (and before I/O redirection) cat < `echo file*` (contents of file sent to screen)

Eval Command

Forces an extra evaluation of command eval cat \< file* (contents of matching file)

Backslash delays translation of < until second translation

________________________________________________________________________29

Page 30: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Section 10: Writing Advanced Loops

While loops

Execute statements while a condition is true i=0 while [ $i -lt 10 ] do echo I is $i i=`expr $i + 1` done

Until loops

Execute statements as long as a condition is false until grep "sort" dbase_log > /dev/null do sleep 10 done echo "Database has been sorted"

Example executes until grep is unsuccessful

________________________________________________________________________30

Page 31: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Redirection of Loops

Can redirect output of a loop for f in *.c do wc -l $f done > loop.out

Loop runs in separate shell New variables forgotten after loop Backgrounding OK, too

Continue Command

Used in for, while, and until loops Skip remaining statements Return to top of loop for name in * do if [ ! -f $name ] ; then continue fi echo "Found file $name" done

Example loops over files, skips directories

Break Command

Used in for, while, and until loops Skip remaining statements Exit loop for name in * do if [ ! -r $name ] ; then echo "Cannot read $name, quitting loop" break fi echo "Found file or directory $name" done

Example loops over files and directories, quits if one is not readable

________________________________________________________________________31

Page 32: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Case Command

Execute one of several blocks of commands case "string" in pattern1) commands ;; pattern2) commands ;; *) # Default case commands ;; esac

Patterns specified with file name wildcards quit) ... qu*) ...

Example With Case

Read commands from keyboard and interpret Enter this script 'case.sh' echo Enter a command while read cmd do case "$cmd" in list) ls -al ;; freespace) df . ;; quit|Quit) break ;; *) echo "$cmd: No such command" ;; esac done echo "All done"

When you run it, the script waits for you to type one of: list freespace quit Quit

Try it: modify the example so any command beginning with characters "free" runs df

Infinite Loops

Infinite loop with while while : do ... done

________________________________________________________________________32

Page 33: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

: is no-op, always returns success status Must use break or exit inside loop for it to terminate

Section 11: Forking Remote Shells

Remote Shells

Rsh command rsh hostname "commands"

Runs commands on remote system Must have .rhosts set up Can specify different login name rsh -l name hostname "commands"

Examples With rsh

Check who's logged on rsh spooky "finger"

Run several remote commands rsh spooky "uname -a; time"

Executes .cshrc on remote system Be sure to set path in .cshrc instead of .login

Access Control with .Rhosts

May get "permission denied" error from rsh Fix this with ~/.rhosts on remote system Example: provide for remote shell from spunky to spooky spunky % rlogin spooky spooky % vi ~/.rhosts (insert "spunky login-name") spooky % chmod 600 ~/.rhosts spooky % logout spunky % rsh spooky uname -a spooky 5.5 sparc SUNW,Ultra-1

May also rlogin without password: security problem!

________________________________________________________________________33

Page 34: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Remote Shell I/O

Standard output sent to local host rsh spooky finger > finger.spooky

Standard input sent to remote host cat local-file | rsh spooky lpr -

Return Status

Get return status of rsh rsh mayer "uname -a" echo $?

Returns 0 if rsh managed to connect to remote host Returns 1 otherwise

o Invalid hostname o Permission denied

Remote Return Status

What about exit status of remote command? Have to determine success or failure from stdout or stderr

________________________________________________________________________34

Page 35: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Section 12: More Miscellaneous

Temporary Files

Use unique names to avoid clashes tempfile=$HOME/Weq_$$ command > $tempfile

$$ is PID of current shell Avoids conflict with concurrent executions of script Do not use /tmp!

Wait Command

Wait for termination of background job command & pid=$! (other processing) wait $pid

Allows overlap of two or more operations

Section 13: Using Quotes

________________________________________________________________________35

Page 36: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Quotes

Provide control of collapsing of spaces and translation of variables Try it: run three examples No quotes (variables translated, spaces collapsed) echo Home: $HOME Home: /users/us/freddy

Double quotes (no collapsing) echo "Home: $HOME" Home: /users/us/freddy

Single quotes (no translation or collapsing) echo 'Home: $HOME' Home: $HOME

Try it: single quotes within double quotes echo "Home directory '$HOME' is full..."

Metacharacters

Characters with special meaning to shell " ' ` $ * [ ] ? ; > < & ( ) \

Avoid special meaning with quoting echo 'You have $20'

Backslash like single quotes Applies only to next character echo You have \$20

Examples With Quotes

Bad command line: grep dog.*cat file

Shell tries to expand dot.*cat as file name wildcard Use quotes to avoid translation grep 'dog.*cat' file

Single quotes OK in this case because we don't need variable translation

More Examples With Quotes

Read name and search file for name read name

________________________________________________________________________36

Page 37: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

grep "$name" dbase

Single quotes not OK because we need variable translation

Searching for Metacharacters

Bad command line: search for dollar sign grep "Gimme.*$20" file

Problem: shell translates variable $20 Solution: use single quotes grep 'Gimme.*$20' file

The “Vi” Editor  This document attempts to list all commands that are to be used for Vi editor in Unix. This can be used by starters as well as by experienced persons who would like to carry this command list along with them in a portable format.  THE ‘VI’ Editor:

• •      There are primarily two types of editors available under UNIX:

– –   ed is a line editor comparable to EDLIN IN DOS.

________________________________________________________________________37

Page 38: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

– –   vi is the full screen editor - vi stands for visual. vi : cautions:

• •      Very user unfriendly - no help.

• •      Same keystrokes can have more than on meaning depending on current mode. Also, the key combinations are in no way mnemonic.

• •      Fanatically case-sensitive vi : Salient features :

• •      vi is omnipresent - in all UNIX systems.

• •      vi is fast.

• •      vi’s got powerful UNDO features.

• •      vi’s got lots of support . vi : The basics :

• •      vi & ex commands compatible with each other but ex is a line editor only.

• •      All editing done in a buffer i.e. , on a copy of original until save.

• •      vi works in 3 different modes :

– –   Command mode - All keys are interpreted as commands.

– –   Insert mode - Keys echoed in edit buffer.

– –   The ex Escape or The Esc colon (:) mode - Keys interpreted as commands & echoed at the command line at the bottom of vi screen.

vi : The basics :...

• •      Bottom most line in vi is the command line where all commands in Esc : mode & messages are displayed.

• •      No error massages in vi - only beeps,beeps & beeps. (PC speaker vital !).

• •      Never press any control or special keys - no arrow keys for cursor movement on most terminals !

vi : Invoking :

• •      vi - - Starts an empty edit buffer.

• •      vi name

– –   Edits the named file - creates named file if not exist.

• •      vi +3 name

– –   Opens & goes to third line.

• •      vi + /bye name

– –   Searches for the first occurrence of “bye” in the file.vi : Invoking : ...

• •      NOTE : The TERM variable to be set right & exported ...

• •      Press “i” now to enter “insert” mode from initial cmd mode & to enter text.

• •      After, press Esc repeatedly (F11 on Micro vax II !) till beep - this is command mode again.

• •      The period “.” is the repeat insert command.

________________________________________________________________________38

Page 39: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

• •      Come to cmd mode like before , type “u” to undo.vi : cursor movements (navigating in a file):More cursor control in vi :

• •      #w– –   Moves word forward

• •      #W– –   Moves word forward till next space

• •      #e– –   Goes to last character in word

• •      #E– –   Same as #e but ignores punctuation

• •      f[character]– –   Upto specified character in current line - right

• •      F[character]– –   Same as above but - left

More cursor control : ...

• •      t[character]– –   Same as f[c] but one c before

• •      T[character]– –   Same as F[c] but one c before

• •      #b– –   back a word

• •      #B– –   Same as #b , ignores punctuation’s

• •      0 or ^– –   beginning of a line

• •      #$– –   end of line

• •      #G

– –   Goto – –   2G to 2 nd line of file

• •      # indicates repeat count.vi : screen commands (Scrolling):

• •      Ctrl-F

– –   Screen forward - except last two lines

• •      Ctrl-B

– –   Screen backwards

• •      Ctrl U

– –   Forwards - half a screen

• •      Ctrl-D

– –   Backwards - half a screen

________________________________________________________________________39

Page 40: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

• •      Ctrl R or Ctrl L

– –   Re-draw & clear messages

• •      Ctrl-G

– –   Status of editvi : deleting data:

• •      #dd– –   delete current line where cursor

• •      #dw– –   delete word from cursor till punctuation

• •      #dW– –   Same as #dw but include (i.e., ignore ) punctuation

• •      #x– –   delete c at cursor

• •      #X– –   delete c before cursor

• •      d$ or D– –   delete current line from where cursor

• •      d0– –   delete current line from where cursor - upto start of line

vi : text insertion:

• •      i

– –   invokes insert mode & inserts before character

• •      I

– –   Same as i but insertion at beginning of line

• •      O

– –   opens blank line above current line

• •      o

– –   Same as O but below

• •      a

– –   Same as i but insert after cursor

• •      A

– –   Same as a but append at end of linevi : text insertion:...

• •      JOINING :

– –   The J command causes the lines of text below the current line to get joined with the current line.

• •      LINE NUMBERING :

– –   In ex Esc mode the command : set nu sets line numbers & : set nonu removes them.

vi : Block moves (colon : commands)-

________________________________________________________________________40

Page 41: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

• •      Copy 2 nd line after 5 th -

– –   :2co5 Destination (5) to be greater than source (2)

• •      Multiple lines copy-

– –   :1,3co$ $ for last line, 1,3 is range

• •      Move to line 6-

– –   :6 compare 6G

• •      Delete three consecutive lines now (#dd) -

– –   3dd alternative is :6,9dvi : Block moves :...

• •      Yet another way -

• •      Symbol Means“.” Current line where cursor“+” Line after current line“-”Line before current line

• •      Delete 3 lines after current line-

– –   : .+3d

• •      Delete 3 lines below current line including current line-

– –   :.,+3d , denotes range

• •      Move line 1 to after line 3 -

– –   : 1mo3

• •      Lines 1,2 & 3 after line 5 -

– –   : 1,3mo5vi : Saving & Exiting -

• •      “:q!”

– –   Quit & abandon changes

• •      “:wq”

– –   Save & quit

• •      “:x”

– –   Replace old copy with new & quit

• •      “ZZ” Save & Quit (from command mode) vi : Replacing existing text:

• •      “r”

– –   Replace a single c at current cursor position

• •      “R”

– –   Invoke insert mode & go on replacing till Esc

• •      “s”

– –   Replace a single c with text - insert mode - continues till Esc - $ appears after c to be replaced

• •      “S”

________________________________________________________________________41

Page 42: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

– –   Replace current line with new text - blanks entire current line - invokes insertion mode

vi : Replacing existing text:

• •      “r”

– –   Replace a single c at current cursor position

• •      “R”

– –   Invoke insert mode & go on replacing till Esc

• •      “s”

– –   Replace a single c with text - insert mode - continues till Esc - $ appears after c to be replaced

• •      “S”

– –   Replace current line with new text - blanks entire current line - invokes insertion mode

vi : Replacing existing text: ...

• •      “cw”

– –   Replaces a word - $ at end of word - replacement continues after end of $ if word size greater than current word

• •      “C”

– –   Same as cw but acts on rest of line from current cursor position

• •      “cc”

– –   Same as C but entire line

• •      NOTE : Flipping case : Command mode - cursor under c - Press “ ~” on key-board - c -> C.

vi : cut & paste:

• •      “#yy”

– –   Yank (cut !) specified number of lines into yank buffer

• •      “p”

– –   Paste from yank buffer below current line

• •      “P”

– –   Same as p but above current line

• •      NOTE : dd also copies to yank buffers. Imperative for paste to immediately follow cut.

vi : 9 lives !-

• •      vi offers undo of upto 9 line deletions (each deletion may be a group of lines ) apart from the usual undo “u” .

– –   This follows the LIFO (Last In First Out ) rule -

• •      “#p or “#P

– –   # is 1 to 9vi : User Defined Registers:

________________________________________________________________________42

Page 43: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

• •      vi offers 26 registers (buffers) named “a” to “z” to the user into which text may be yanked & put anywhere else in the file using the “put” commands. These assignments are active during the entire current session

• •      “a5yy

– –   yank 5 whole lines including current & place in user defined buffer “a”

• •      “ap

– –   put contents of “a” below current linevi : Marking & Yanking:

• •      Marking the current cursor position-

– –   mk - “m” for mark , “k” is block name (In command mode)

– –   ‘k - flips cursor to mark “k” from anywhere else in file

– –   d’k - deletes block from mark “k” upto current cursor position

– –   “ay’k - yanks into register “a”

– –   “ap - puts contents of “a”

vi : Pattern Searching:

• •      Basic commands are :

• •      /g

– –   search for pattern “g” forwards

• •      ?g

– –   search for pattern “g” backwards

• •      n

– –   repeat the last / or ? commandvi : Global & Substitute:

• •      These are : (colon) commands.

• •      To replace “string” by “string1” on the current line -

– –   :s/string/string1/ replaces only first occurrence

– –   :s/string/string1/greplaces all occurrences

vi : Global & Substitute: ...

• •      Substitution can occur on multiple lines if range is specified -

– –   :1,9s/string/string1/g

• •      For substituting throughout the file -

– –   :1,$s/string/string1/g

– –   OR

– –   :g/string/s//string1/g

________________________________________________________________________43

Page 44: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

• •   note the double slash.vi : Macros:

• •      vi macros are created using the “abbr” command -

• •      :abbr uo “UNIX operating system”

• •      In insert mode press “u” “o” & press “Spc bar” or “Enter”

– –   the macro string is inserted at current cursor position.

• •      :abbr

– –   lists the macros defined

• •      Macros can be unabbreviated or deleted -

• •      :una uo

– –   deletes macro uovi : Mapping Keys to Commands:

• •      The “map” command maps a user defined key to a specific function -

• •      The cursor movement keys j,k,l,h may be mapped by default to their respective cursor movement keys -

• •      To map the key “ctrl-X” to the “wq” command -

• •      :map ^X :wq^M -

– –   note that ^M signifies <CR> , in Vi “^” is typed by typing “Ctrl-v”vi : Mapping Keys to Commands:...

• •      An “unmap” command is available to cancel assignments:

• •      :unmap ^X

• •      Mapping in insert mode done using the “map!” command -

• •      :map! ^x Sam fox

– –   In insert mode “Ctrl-x” will insert “Sam fox” in the editor

• •      NOTE :

– –   An insert mode map cannot be unmapped unlike abbr

– –   :map displays only keys mapped to command mode functionsvi : Running shell commands from within :

• •      “:sh” temporarily “shells out” to $ and “ctrl-d” returns back from the shell

• •      “: !<command>“ “!” stands for run which will directly run <command>vi : Reading command output from within vi :

• •      This is done using “:r!”

• •      :r! date

– –   reads in output of “date” at current cursor position

• •      :r! banner Hi therevi : Editing Multiple files Simultaneously:

• •      vi can take multiple file arguments -

– –   $ vi file1 file2 .....

________________________________________________________________________44

Page 45: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

• •      OR metacharacters too -

– –   $ vi f*

• •      :args

– –   [file1] file2 file3

• •   - The bracketed file taken for display/edit

• •      :n

– –   Switches to next file in list

• •      :args

– –   file1 [file2] file3vi : Editing Multiple files Simultaneously:...

• •      :rew

• •      :args

– –   [file1] file2 file3-i.e. “rew” rewinds to file1

• •      :rew!

– –   discards changes & coercively rewinds

• •      :e file

– –   edits file outside chosen list but :args will not show this file in listvi : Editing Multiple files Simultaneously:...

• •      :rew

• •      :args

– –   [file1] file2 file3-i.e. “rew” rewinds to file1

• •      :rew!

– –   discards changes & coercively rewinds

• •      :e file

– –   edits file outside chosen list but :args will not show this file in list vi : Read & Write across files -

• •      Write first five lines of text from file1 to file2-:1,4 w! file2

• •      Read same text from file2 to file1-:1,4 r file2

vi : The “Set” Options :

• •      The “set” option helps customize the vi environment -

• •      : set all - displays all “set” options available

• •      The more common options are :

– –   SET COMMAND DEFAULT

• •   DESCRIPTION

– –   :set number,nu nonumber

• •   Line numbers

________________________________________________________________________45

Page 46: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

– –   :set showmode noshowmode

• •   Display vi mode vi : The “Set” Options :...

– –   :set autowrite,aw noaw

• •   Save on n, rew, !

– –   :set autoindent,ai noai

• •   Indent as first line

– –   :set beautify,bf nobeautify

• •   Control characters omitted

– –   :set directory,dir dir-/tmp

• •   Sets temporary directory

– –   :set errorbells,eb noeb

• •   No error beep vi : The “Set” Options :...

– –   :set ignorecase,ic noic

• •   Uppercase as lowercase

– –   :set magic magic

• •   Allow all metacharacters

– –   :set mesg nomesg

• •   Inhibit Write messages

– –   :set redraw noredraw

• •   Act as intelligent terminal

– –   :set scroll scroll=half window

• •   No. of lines to scroll vi : The “Set” Options :...

– –   :set shell sh=/bin/sh

• •   Set which shell

– –   :set term value of TERM

• •   Set which terminal

– –   :set terse noterse

• •   Error messages terse

– –   :set warn warn

• •   Warn save on Shell Esc vi : The “Set” Options :...

– –   :set window dependentwindow speed

• •   No of lines in text window

– –   :set wrapscan,ws ws

• •   Wrap pattern on find

– –   :set warpmargin,wm wm=0

• •   Blank screen margin right

– –   :set writeany, wa nowa

• •   Allow writes freelyvi : The “.exrc” file -

________________________________________________________________________46

Page 47: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

• •      vi reads .exrc before loading and all options like map,abbr & set can be entered here so that they are permanent for that vi session -

• •      $ vi .exrc

– –   set nu

– –   set showmode

– –   abbr uo UNIX operating system

– –   map ^k :r! date^M

UNIX – Basic Commands

Note: This file is part of Linux Shell Scripting Tutorial, and contains many Linux/Unix definition, miscellaneous concepts and answer to many shell scripts exercise section.

Free

Linux is free. First ,It's available free of cost (You don't have to pay to use this OS, other OSes like MS-Windows or Commercial version of Unix may cost you money) 

Second free means freedom to use Linux, i.e. when you get Linux you will also get source code of Linux, so you can modify OS (Yes OS! Linux OS!!) according to your taste. 

It also offers many Free Software applications, programming languages, and development tools etc. Most of the Program/Software/OS are under GNU General Public License (GPL).

Unix Like 

Unix is almost 35 year old Os. 

In 1964 OS called MULTICS (Multiplexed Information and Computing System) was developed by Bell Labs, MIT & General Electric. But this OS was not the successful one.

Then Ken Thompson (System programmer of Bell Labs) thinks he could do better (In 1991, Linus Torvalds felt he could do better than Minix - History repeats itself.). So Ken Thompson wrote OS on PDP - 7 Computer, assembler and few utilities, this is know as Unix (1969). But this version of Unix is not portable. Then Unix was rewrote in C.

________________________________________________________________________47

Page 48: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Because Unix written in 'C', it is portable. It means Unix can run on verity of Hardware platform (1970-71). 

At the same time Unix was started to distribute to Universities. There students and professor started more experiments on Unix. Because of this Unix gain more popularity, also several new features are added to Unix. Then US govt. & military uses Unix for there inter-network (now it is know as INTERNET).

So Unix is Multi-user, Multitasking, Internet-aware Network OS.  Linux almost had same Unix Like feature for e.g.

Like Unix, Linux is also written is C. Like Unix, Linux is also the Multi-user/Multitasking/32 or 64 bit Network OS. Like Unix, Linux is rich in Development/Programming environment. Like Unix, Linux runs on different hardware platform; for e.g.

o Intel x86 processor (Celeron/PII/PIII/PIV/Old-Pentiums/80386/80486) o Macintosh PC's  o Cyrix processor  o AMD processor  o Sun Microsystems Sparc processor o Alpha Processor (Compaq)

Open Source

Linux is developed under the GNU Public License. This is sometimes referred to as a "copyleft", to distinguish it from a copyright.

Under GPL the source code is available to anyone who wants it, and can be freely modified, developed, and so forth. There are only a few restrictions on the use of the code. If you make changes to the programs , you have to make those changes available to everyone. This basically means you can't take the Linux source code, make a few changes, and then sell your modified version without making the source code available.

Common vi editor command list

For this Purpose Use this vi Command Syntax

To insert new textesc + i ( You have to press 'escape' key then 'i')

To save fileesc + : + w (Press 'escape' key  then 'colon' and finally 'w')

To save file with file name (save as) esc + : + w  "filename"

To quit the vi editor esc + : + q

To quit without saving esc + : + q!

________________________________________________________________________48

Page 49: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

To save and quit vi editor esc + : + wq

To search for specified word in forward direction

esc + /word (Press 'escape' key, type /word-to-find, for e.g. to find word 'shri', type as/shri)

To continue with search  n

To search for specified word in backward direction

esc + ?word (Press 'escape' key, type word-to-find)

To copy the line where cursor is located esc + yy

To paste the text just deleted or copied at the cursor

esc + p

To delete entire line where cursor is located esc + dd

To delete word from cursor position esc + dw

To Find all occurrence of given word and Replace then globally without confirmation 

esc + :$s/word-to-find/word-to-replace/g

For. e.g. :$s/mumbai/pune/gHere word "mumbai" is replace with "pune"

 

To Find all occurrence of given word and Replace then globally with confirmation

esc + :$s/word-to-find/word-to-replace/cg

To run shell command like ls, cp or date etc within vi

esc + :!shell-command

For e.g. :!pwd

How Shell Locates the file

To run script, you need to have in the same directory where you created your script, if you are in different directory your script will not run (because of path settings), For e.g.. Your home directory is ( use $ pwd to see current working directory) /home/vivek. Then you created one script called 'first', after creation of this script you moved to some other directory lets say /home/vivek/Letters/Personal, Now if you try to execute your script it will not run, since script 'first' is in /home/vivek directory, to overcome this problem there are two ways first, specify complete path of your script when ever you want to run it from other directories like giving following command$ /bin/sh   /home/vivek/first

________________________________________________________________________49

Page 50: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Now every time you have to give all this detailed as you work in other directory, this take time and you have to remember complete path. 

There is another way, if you notice that all of our programs (in form of executable files) are marked as executable and can be directly executed from prompt from any directory. (To see executables of our normal program give command $ ls -l /bin ) By typing commands like$ bc$ cc myprg.c$ caletc, How its possible? All our executables files are installed in directory called /bin and /bin directory is set in your PATH setting, Now when you type name of any command at $ prompt, what shell do is it first look that command in its internal part (called as internal command, which is part of Shell itself, and always available to execute), if found as internal command shell will execute it, If not found It will look for current directory, if found shell will execute command from current directory, if not found, then Shell will Look PATH setting, and try to find our requested commands executable file in all of the directories mentioned in PATH settings, if found it will execute it, otherwise it will give message "bash: xxxx :command not found", Still there is one question remain can I run my shell script same as these executables?, Yes you can, for this purpose create bin directory in your home directory and then copy your tested version of shell script to this bin directory. After this you can run you script as executable file without using command like$ /bin/sh   /home/vivek/first Command to create you own bin directory.

$ cd$ mkdir bin$ cp first ~/bin$ first

Each of above commands can be explained as follows:

Each of above command Explanation

$ cd Go to your home directory

$ mkdir bin

Now created bin directory, to install your own shell script, so that script can be run as independent program or can be accessed from any directory

$ cp   first ~/bin copy your script 'first' to your bin directory

$ firstTest whether script is running or not (It will run)

________________________________________________________________________50

Page 51: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Answer to Variable sections exercise

Q.1.How to Define variable x with value 10 and print it on screen.$ x=10$ echo $x

Q.2.How to Define variable xn with value Rani and print it on screenFor Ans. Click here$ xn=Rani$ echo $xn

Q.3.How to print sum of two numbers, let's say 6 and 3$ echo 6 + 3This will print 6 + 3, not the sum 9, To do sum or math operations in shell use expr, syntax is as follows  Syntax: expr   op1   operator   op2Where, op1 and op2 are any Integer Number (Number without decimal point) and operator can be+ Addition- Subtraction/ Division% Modular, to find remainder For e.g. 20 / 3 = 6 , to find remainder 20 % 3 = 2, (Remember its integer calculation)\* Multiplication$ expr 6 + 3 Now It will print sum as 9 , But$ expr 6+3will not work because space is required between number and operator (See Shell Arithmetic)

Q.4.How to define two variable x=20, y=5 and then to print division of x and y (i.e. x/y)For Ans. Click here$x=20$ y=5$ expr x / y

Q.5.Modify above and store division of x and y to variable called zFor Ans. Click here$ x=20$ y=5$ z=`expr x / y`$ echo $z

Q.6.Point out error if any in following script

________________________________________________________________________51

Page 52: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

$ vi   variscript### Script to test MY knolwdge about variables!#myname=Vivekmyos   =  TroubleOS    -----> ERROR 1myno=5echo "My name is $myname"echo "My os is $myos"echo "My number is   myno,   can you see this number"  ----> ERROR 2

Following script should work now, after bug fix!

$ vi   variscript### Script to test MY knolwdge about variables!#myname=Vivekmyos=TroubleOSmyno=5echo "My name is $myname"echo "My os is $myos"echo "My number is   $myno,   can you see this number"

Parameter substitution.

Now consider following command$($ echo 'expr 6 + 3')

The command ($ echo 'expr 6 + 3')  is know as Parameter substitution. When a command is enclosed in backquotes, the command get executed and we will get output. Mostly this is used in conjunction with other commands. For e.g.

$pwd$cp /mnt/cdrom/lsoft/samba*.rmp `pwd`

Now suppose we are working in directory called "/home/vivek/soft/artical/linux/lsst" and I want to copy some samba files from "/mnt/cdrom/lsoft" to my current working directory, then my command will be something like

$cp   /mnt/cdrom/lsoft/samba*.rmp    /home/vivek/soft/artical/linux/lsst

________________________________________________________________________52

Page 53: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Instead of giving above command I can give command as follows

$cp  /mnt/cdrom/lsoft/samba*.rmp  `pwd`

Here file is copied to your working directory. See the last Parameter substitution of `pwd` command, expand it self to /home/vivek/soft/artical/linux/lsst. This will save my time.$cp  /mnt/cdrom/lsoft/samba*.rmp  `pwd`

Future Point: What is difference between following two command?$cp  /mnt/cdrom/lsoft/samba*.rmp  `pwd`

                        A N D

$cp  /mnt/cdrom/lsoft/samba*.rmp  .

Try to note down output of following Parameter substitution.

$echo "Today date is `date`"$cal > menuchoice.temp.$$$dialog --backtitle "Linux Shell Tutorial"  --title "Calender"  --infobox  "`cat  menuchoice.temp.$$`"  9 25 ; read

Answer to if command. 

A) There is file called foo, on your disk and you give command, $ ./trmfi   foo what will be output.Ans.: foo file will be deleted, and message "foo file deleted" on screen will be printed.

B) If bar file not present on your disk and you give command, $ ./trmfi   bar what will be output.Ans.: Message "rm: cannot remove `bar': No such file or directory" will be printed because bar file does not exist on disk and we have called rm command, so error from rm commad

C) And if you type $ ./trmfi, What will be output.Ans.:  Following message will be shown by rm command, because rm is called from script without any parameters.rm: too few argumentsTry `rm --help' for more information.

Answer to Variables in Linux.

________________________________________________________________________53

Page 54: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

1) If you want to print your home directory location then you give command:     (a) $ echo $HOME

                    or

     (b) $ echo HOME

Which of the above command is correct & why?

Ans.: (a) command is correct, since we have to print the contains of variable (HOME) and not the HOME. You must use $ followed by variable name to print variables cotaines.

Answer to Process Section.

1) Is it example of Multitasking?Ans.: Yes, since you are running two process simultaneously.

2) How you will you find out the both running process (MP3 Playing & Letter typing)?Ans.: Try $ ps aux or $ ps ax | grep  process-you-want-to-search

3) "Currently only two Process are running in your Linux/PC environment", Is it True or False?, And how you will verify this?Ans.: No its not true, when you start Linux Os, various process start in background for different purpose. To verify this simply use top or ps aux command.

4) You don't want to listen music (MP3 Files) but want to continue with other work on PC, you will take any of the following action:

1. Turn off Speakers 2. Turn off Computer / Shutdown Linux Os 3. Kill the MP3 playing process 4. None of the above

Ans.: Use action no. 3 i.e. kill the MP3 process.Tip: First find the PID of MP3 playing process by issuing command:$ ps ax | grep mp3-process-name Then in the first column you will get PID of process. Kill this PID to end the process as:$ kill  PID

Or you can try killall command to kill process by name as follows:$ killall  mp3-process-name

________________________________________________________________________54

Page 55: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Linux Console (Screen) 

How can I write colorful message on Linux Console? , mostly this kind of question is asked by newcomers (Specially those who are learning shell programming!). As you know in Linux everything is considered as a file, our console is one of such special file. You can write special character sequences to console, which control every aspects of the console like Colors on screen, Bold or Blinking text effects, clearing the screen, showing text boxes etc. For this purpose we have to use special code called escape sequence code.  Our Linux console is based on the DEC VT100 serial terminals which support ANSI escape sequence code.

What is special character sequence and how to write it to Console?

By default what ever you send to console it is printed as its. For e.g. consider following echo statement,$ echo "Hello World"Hello WorldAbove echo statement prints sequence of character on screen, but if there is any special escape sequence (control character) in sequence , then first some action is taken according to escape sequence (or control character) and then normal character is printed on console. For e.g. following echo command prints message in Blue color on console$ echo -e "\033[34m   Hello Colorful  World!"Hello Colorful  World!

Above echo statement uses ANSI escape sequence (\033[34m), above entire string ( i.e.  "\033[34m   Hello Colorful  World!" ) is process as follows

1) First \033, is escape character, which causes to take some action2) Here it set screen foreground color to Blue using [34m escape code.3) Then it prints our normal message Hello Colorful  World! in blue color.

Note that ANSI escape sequence begins with \033 (Octal value) which is represented as ^[ in termcap and terminfo files of terminals and documentation.

You can use echo statement to print message, to use ANSI escape sequence you must use -e option (switch) with echo statement, general syntax is as followsSyntaxecho   -e  "\033[escape-code    your-message"

In above syntax you have to use\033[ as its with different escape-code for different operations. As soon as console receives the message it start to process/read it, and if it found escape character (\033) it moves to escape mode, then it read "[" character and moves into Command Sequence Introduction (CSI) mode. In CSI mode console reads a series of ASCII-coded decimal numbers (know as parameter) which are separated by semicolon (;) . This numbers are read until console action letter or character is not found (which determines what action to take). In above example

________________________________________________________________________55

Page 56: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

\033 Escape character

[ Start of CSI

34 34 is parameter

m m is letter (specifies action)

Following table show important list of such escape-code/action letter or character

Character or letter

Use in CSI Examples

h Set the ANSI mode echo -e "\033[h"

l Clears the ANSI mode echo -e "\033[l"

m

Useful to show characters in different colors or effects such as BOLD and Blink, see below for parameter taken by m.

echo -e  "\033[35m Hello World"

qTurns keyboard num lock, caps lock, scroll lock LED on or off, see below.

echo -e "\033[2q"

sStores the current cursor x,y position (col , row position) and attributes

echo -e "\033[7s"

uRestores cursor position and attributes

echo -e "\033[8u"

m understand following parameters

Parameter Meaning Example

0

Sets default color scheme (White foreground and Black background), normal intensity, no blinking etc.

 

1 Set BOLD intensity

$ echo -e "I am \033[1m BOLD \033[0m Person"I am BOLD PersonPrints BOLD word in bold intensity and next ANSI Sequence remove bold effect (\033[0m)

2 Set dim intensity$ echo -e "\033[1m  BOLD \033[2m DIM  \033[0m"

5 Blink Effect $ echo -e "\033[5m Flash!  \033[0m"

________________________________________________________________________56

Page 57: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

7

Reverse video effect i.e. Black foreground and white background in default color scheme

$ echo -e "\033[7m Linux OS! Best OS!! \033[0m"

11

Shows special control character as graphics character. For e.g. Before issuing this command press alt key (hold down it) from numeric key pad press 178 and leave both key; nothing will be printed. Now give --> command shown in example and try the above, it works. (Hey you must know extended ASCII Character for this!!!)

$ press alt + 178$ echo -e "\033[11m"$ press alt + 178$ echo -e "\033[0m"$ press alt + 178

25Removes/disables blink effect

 

27Removes/disables reverse effect

 

30 - 37

Set foreground color31 - RED32 - Greenxx - Try to find yourself this left as exercise for you :-)

$ echo -e "\033[31m I am in Red"

40 - 47

Set background colorxx - Try to find yourself this left as exercise for you :-)

$ echo -e "\033[44m Wow!!!"

q understand following parameters

Parameters Meaning

0 Turns off all LEDs on Keyboard

1 Scroll lock LED on and others off

2 Num lock LED on and others off

3 Caps lock LED on and others off

This is just quick introduction about Linux Console and what you can do using this Escape sequence. Above table does not contains entire CSI sequences. My up-coming

________________________________________________________________________57

Page 58: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

tutorial series on C Programming Language will defiantly have entire story with S-Lang and curses (?). What ever knowledge you gain here will defiantly first step towards the serious programming using c. This much knowledge is sufficient for  Shell Programming, now try the following exercise :-) I am Hungry give me More Programming Exercise & challenges! :-)

1) Write function box(),  that will draw box on screen (In shell Script)    box (left, top, height, width)    For e.g. box (20,5,7,40)   

   

Hint: Use ANSI Escape sequence1) Use of 11 parameter to m2) Use following for cursor movement   row;col H      or   rowl;col f    For e.g.  $ echo   -e "\033[5;10H Hello"  $ echo   -e "\033[6;10f Hi"

In Above example prints Hello message at row 5 and column 6 and Hi at 6th row and 10th Column.

Shell Built in Variables

Shell Built in Meaning

________________________________________________________________________58

Page 59: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

Variables

$#Number of command line arguments. Useful to test no. of command line args in shell script.

$* All arguments to shell

$@ Same as above

$- Option supplied to shell

$$ PID of shell

$! PID of last started background process (started with &)

echo CommandUse echo command to display text or value of variable.

echo [options] [string, variables...]Displays text or variables value on screen.Options-n Do not output the trailing new line.-e Enable interpretation of the following backslash escaped characters in the strings:\a alert (bell)\b backspace\c suppress trailing new line\n new line\r carriage return\t horizontal tab\\ backslash

For e.g. $ echo -e "An apple a day keeps away \a\t\tdoctor\n"

Shell ArithmeticUse to perform arithmetic operations.

Syntax:expr op1 math-operator op2

Examples: $ expr 1 + 3

________________________________________________________________________59

Page 60: UNIX Bourne Shell Scripting

UNIX Bourne Shell Scripting

$ expr 2 - 1$ expr 10 / 2$ expr 20 % 3$ expr 10 \* 3$ echo `expr 6 + 3`

Note:expr 20 %3 - Remainder read as 20 mod 3 and remainder is 2.expr 10 \* 3 - Multiplication use \* and not * since its wild card.

For the last statement not the following points

(1) First, before expr keyword we used ` (back quote) sign not the (single quote i.e. ') sign. Back quote is generally found on the key under tilde (~) on PC keyboard OR to the above of TAB key.

(2) Second, expr is also end with ` i.e. back quote.

(3) Here expr 6 + 3 is evaluated to 9, then echo command prints 9 as sum

(4) Here if you use double quote or single quote, it will NOT workFor e.g.$ echo "expr 6 + 3" # It will print expr 6 + 3$ echo 'expr 6 + 3' # It will print expr 6 + 3

________________________________________________________________________60