23494612 the Unix Shell Guide

8/9/2019 23494612 the Unix Shell Guide

1/63

THE UNIX SHELL GUIDE

Norman J. Buchanan

Douglas M. Gingrich

The UNIX Shell Guideby Norman J. Buchanan & Douglas M. GingrichCopyright 1996 University of Alberta, Department of Physics. All rights reserved.UNIX is a registered trademark of Novell.Many of the designations used by manufactures and sellers to distinguish their productsare claimed as trademarks. Where those designations appear in this book, we are aware ofa trademark claim.While every precaution has been taken in the preparation of this book, the authors assumeno responsibility for errors or omissions, or for damages resulting from the use of theinformation contained here in.

THE UNIX SHELL GUIDE ...............................................................................................1Preface ..................................................................................................................................2What is a Shell? ................................................................................................................... 3UNIX Commands versus Built in Shell Commands ............................................................4Interactive and Sub Shells ....................................................................................................5Command Line Parsing ........................................................................................................5Redirection ...........................................................................................................................6The Bourne Shell ................................................................................................................. 7A Bit of History ................................................................................................................... 8Getting to Know the Bourne Shell .......................................................................................8

Multiple Commands Per Line ..............................................................................................9Redirection and Pipelines .................................................................................................. 10Redirection and Pipelines .................................................................................................. 11Special Characters ..............................................................................................................12Variables ............................................................................................................................ 13

Variable Substitutions .................................................................................................... 14Parameters ......................................................................................................................15Environmental Variables ............................................................................................... 15Special Variables ........................................................................................................... 16

Bourne Shell Programming ................................................................................................17Testing and Branching ................................................................................................... 18String Comparisons ........................................................................................................20Testing Files ...................................................................................................................20Loop Control .................................................................................................................. 25

Exiting Loops Early ...................................................................................................26Input and Output (I/O) ................................................................................................... 27Advanced Programming Topics .................................................................................... 28

Functions ....................................................................................................................29Trapping Errors .......................................................................................................... 30


2/63

Dot Files .........................................................................................................................31The C Shell ........................................................................................................................32A Bit of History .................................................................................................................32Getting Started with the C Shell ........................................................................................33Command Execution ..........................................................................................................33

Redirection and Pipelines .................................................................................................. 35Filename Substitution ........................................................................................................36Filename Completion .........................................................................................................37Special Characters ..............................................................................................................38Command History .............................................................................................................. 39Aliases ................................................................................................................................41Variables ............................................................................................................................ 42Variables ............................................................................................................................ 44

Array Variables ..............................................................................................................45Global or Environment Variables .................................................................................. 46Parameters ......................................................................................................................47

Special Read Only Variables .........................................................................................48Variable Modifiers .........................................................................................................48C Shell Programming .........................................................................................................49

Testing and Branching ................................................................................................... 50Signal Handling .............................................................................................................57

Job Control .........................................................................................................................58Special Files .......................................................................................................................60

Introduction to UNIX Security ......................................................................................61

Preface

This book is intended to assist UNIX users in understanding and dealing with five of themost popular UNIX shells: the Bourne shell (sh); the C shell (csh); the Korn shell (ksh);the TC shell (tcsh); and the Z shell (zsh). The idea came mainly out of frustration intrying to get understandable information on shell usage. Much has been written on UNIXas a whole but rarely is more than a chapter on Bourne shell or C shell included, andthese usually prove to be little more than a loose translation of the online manual (man)pages. Shells are certainly sophisticated and powerful enough to warrant a detailedtreatment on their own. The man pages, as any UNIX user is well aware, are fine forlisting commands and associated flags but that is where the usefulness ends. Examplesare seldom found and some of the features are covered in one or two lines of text, leavingthe user to overlook their importance. The man pages are, after all, meant to act as areference more than an actual instruction manual and should be treated as such.


3/63

Hopefully this book will achieve two goals. First, to give a clear and concise look at eachof five shells. Each shell will be examined from the point of view of interactive workfeatures, and then from a shell programming point of view. Examples will be used asmuch as possible to lend illustration to the concepts presented in the text, especially in thearea of shell programming. Hopefully, the range of examples will be such that there will

be something that everyone can use. New UNIX users will then be able to pick up onsome of the powerful features of the particular shell(s) of interest and develop a feel forshell programming, which is probably the most flexible and exciting feature of UNIXshells in general. The second goal of the book is to allow users the opportunity toexamine the features of the various shells covered and determine which shell(s) might beright for them. Each shell will be contrasted with the others to demonstrate the strengthsand weaknesses compared with the rest. This we hope will make this book truly unique.As with most other things in life, choice can make things more difficult than expected.While it is nice to have choice, making a choice can require a level of research that mostpeople just do not have the time to invest. Sometimes the number of choices is not evenknown, further complicating the decision process - how can you make a choice if you do

not know what the choices are? In this book enough information will be presented so theusers can make educated decisions without spending the time trying to gather theinformation on their own, or having to make sense of it afterwards.Users new to UNIX will be able to become acquainted with the shell they are providedwith, learning the details of shell usage until they are able to decide if a different shellmay better suit their needs. Even users who have spent many years mastering a particularshell may find that some of the more recent shells provide powerful features that they cantailor to their specific needs. If any of the shells mentioned in this book are not availableon your machine, most system administrators can be persuaded to load them on to yourmachine for use. To further assist in making the various shells available, FTP sites wherethey can be found are included.This book will be of great interest to users of personal computers who are consideringmoving or have just moved to the new UNIX-compatible operating system for thepersonal computer - LINUX. LINUX is distributed freely in electronic form and for lowcost from many vendors. The standard software distribution includes many of the shellsdescribed in this book.Norman J. BuchananDouglas M. Gingrich

What is a Shell?

What is a shell? A shell is a command interpreter. While this is certainly true it likelydoesn't enlighten the reader any further. A shell is an entity that takes input from the userand deals with the computer rather than have the user deal directly with the computer. Ifthe user had to deal directly with the computer he would not get much done as thecomputer only understands strings of 1's and 0's. While this is a bit of a misrepresentationof what the shell actually does (the idea of an operating system is neglected) it provides arough idea that should cause the reader to be grateful that there is such a thing as a shell.A good way to view a shell is as follows. When a person drives a car, that person doesn't


4/63

have to actually adjust every detail that goes along with making the engine run, or theelectronic system controlling all of the engine timing and so on. All the user (or driver inthis example) needs to know is that D means drive and that pressing accelerator pedalwill make the car go faster or slower. The dashboard would also be considered part of thethe shell since pertinent information relating to the user's involvement in operating the car

is displayed there. In fact any part of the car that the user has control of during operationof the car would be considered part of the shell. I think the idea of what a shell is comingclear now. It is a program that allows the user to use the computer without him having todeal directly with it. It is in a sense a protective shell that prevents the user and computerfrom coming into contact with one another.

Basic UNIX primero The Man Pages

UNIX Commands versus Built in Shell Commands Interactive and Sub Shells

Command Line Parsing Redirection

UNIX Commands versus Built in Shell

Commands

The UNIX operating system comes with many commands that the user can use to interactwith computer. UNIX commands are simply programs (usually written in the Cprogramming language) that are executed when called for. The usual place for the storage

of these commands is the /usr/bin directory. The commands that are available on aparticular machine will vary. There is a set number of standard commands that come witha UNIX system, but there is no limit to the commands that may be available. An exampleof this is the more command. Typing more followed by a filename (preferably a text file)will cause the filename to be presented to the default output device a page at a time.Pressing the space-bar will display the next page, typing the enter-key will display thenext line, and pressing the q will exit the program. This is how the man pages aredisplayed. Many users felt that this was too inflexible and along came another commandfound on many UNIX systems called less. The less command is essentially the same asthe more command with the exception that it allows the up and down arrow keys to beused to scroll around a document. Many of the UNIX commands may be well known tothe user while others may not. Some examples are ls, cd, grep, find and chmod to namejust a select few. It is important to realize that while these commands might vary insyntax and usage somewhat from one platform of UNIX to another, they are shellindependent. Whether in the C shell or the Z shell, the grep command behaves the same.Remember to see how a particular command is used on a particular platform, the user canuse the man command. Now each shell comes with its own set of built-in\ commands.These are commands that are local to the particular shell. Some examples of these are thehistory command in the C shell, and the export in the Bourne shell. Built-in commands
http://csr.phys.ualberta.ca/~gingrich/research/shells/node5.html#SECTION00410000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node6.html#SECTION00411000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node7.html#SECTION00420000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node8.html#SECTION00430000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node9.html#SECTION00440000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node10.html#SECTION00450000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node5.html#SECTION00410000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node6.html#SECTION00411000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node7.html#SECTION00420000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node8.html#SECTION00430000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node9.html#SECTION00440000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node10.html#SECTION00450000000000000000


5/63

can be taken as platform independent. It is important to keep in mind having said this thatthere is a possibility of slight variation between different versions of the shellsthemselves. This is inevitable, but for the purpose of this book a standarized shell is againassumed. If there is any discrepency between this book and any particular shell, the manpages can again be referenced.

Interactive and Sub Shells

When a user logs onto a UNIX account, a shell is immediately set running. The shell thatis started depends on some predetermined settings in special files that are executed duringthe logon process. Ultimately then, the shell started during logon is determined by thesystem administrator. As will be seen in later chapters, a different shell can be set to startat logon using the chsh command. This shell is called a login (or root) shell. This is ashell that runs at all times during a users session. When the user exits the session the shellis terminated by the operating system. The login shell is a special kind of interactive

shell. The purpose of an interactive shell is to let the user interact with the computerduring a session. The interactive shell (as discussed briefly in the begining of thischapter) examines each command line entered by the user and then executes thecommand if it is correct syntactically (in usage). One must be careful when using theterm correct however, as by correct it is meant that the proper usage of the command(outlined in the man page for that command) is used. The user could still enter acommand that makes absolutely no sense in terms of what it accomplishes, yet issyntactically correct. The shell has no way of checking for this. A new interacive shellcan be started within a login shell at any time.A subshell is a shell that runs underneath of an interactive shell. The user cannot interactdirectly with a subshell as subshells only take input from commands or another shell. A

subshell is really just where a command is executed. When a command line is entered inan interactive shell, the command(s) may be passed to a subshell to execute. This wouldoccur for exemple if a group of commads is enclosed in parentheses. Programs written inthe shells will be automatically executed in subshells. It is important to understand thatvariables declared in an interactive shell are not automatically passed to a subshell. Thiswill be covered in more detail in the chapters pertaining to the shells themselves.

Command Line Parsing

To parse a command line means to look at each part of the command line and convert itinto something that the computer can execute. Since there variations in how differentshells parse a command line, it can be assumed for the rest of this section, that the shell inquestion is generic. For the purpose of this book the small differences between shells aswell as the exact precise details of parsing can be ignored. When a user enters acommand line at the prompt, the shell begins by analyzing the command line. The shellwill break the command line down into small indivisible pieces called tokens (sometimesthey are refered to as atomic). Each token is then analyzed in terms of its relationshipwith the other tokens. This is similar to the human examination of an english sentence. If


6/63

a noun is present, but no verb, the sentence is deemed incomplete. The shell behaves inmuch the same matter. It doesn't only check for missing bits, it also makes sure that whatis there is in correct order. The shell may have to examine a command line more thanonce to collect all of the tokens. Each examination is called a pass. The reason formultiple passes is that command lines can be quite complicated, there can be all kinds of

substitutions that need to be made. On each pass the shell will make a requiredsubstitution and then collect the available tokens. Since the substitutions can be nested(substitutions containing substitutions), the shell may require several passes to collect allof the tokens. As stated above, if at this point in the process the shell determines that thegrammar of the command line is incorrect, an error is displayed to the user, or else thecommand is executed. While the actual order in which the tokens are gathered isinteresting, it is beyond the scope of this book. Where required (such as aliases) the orderof some of the parsing procedure will be presented.

Redirection

Throught this book the topic of redirection will be visited many times. Redirection iswhere data is sent to or from when interacting with a command. Fo instance, when a userlogs onto a terminal on the local network, a group of messages may be displayed, orperhaps just a prompt. The messages or prompt have been sent to the terminal windowfor the user to read, in the case of the messages, or interact with, as would be the case ofthe prompt. This output stream, as it is called, is sent to what is called the standard output(STDOUT). This is usually automatically set to be the screen of the workstation orterminal, although this hasn't always been the case. In the very early days of UNIX,STDOUT may well have been a teletype machine, long since extinct. When a program(or equivaently, a command) is executed, the output can be redirected to file by using the

> operator. The general syntax for all of the shells covered in this book (and any othershells for that matter) is the following:PROMPT> command name >filename

it would not be an error to have a space between the > and the filename, it is just a matterof taste. When a program requires input, like the program which runs the login procedure,accepts data via the input stream called standard input (STDIN). This is almost always setto the keyboard, for obvious reasons. A command will often take data from either a file orSTDIN, as is the case with the cat (concatenate command):PROMPT> cat filename

which would send the contents from the file filename to the screen. The following wouldrepeat to echo whatever the user had entered to the screen, after pressing the key, until the end of file (EOF) character (usually Control D) had been entered:

PROMPT> catHello There Hello ThereHow are you? How are you?PROMPT>

The last important data stream is the standard error. When a program (or command) isexecuted, it might encounter problems completing its task for whatever reason. When this


7/63

happens, the shell will, in most cases, echo an error message to STDERR. By default,STDERR is directed to the screen along with STDOUT, but it doesn't have to be.Situations may very well arise where the error messages might be sent to a file, orsomewhere else, while the standard output would be sent to the screen. The shells handlethis situation in different ways so examples demonstrating this procedure will be left until

the individual chapters.In UNIX these data streams can also be referred to by numbers called file descriptors.This provides another way to represent redirections but is again shell dependent and willthus left to the appropriate sections for examples. The following table associates each filedescriptor with the corresponding data stream:

Table 1.1: File descriptors.

The Bourne Shell

A Bit of History Getting to Know the Bourne Shell Multiple Commands Per Line Redirection and Pipelines Filename Substitution Special Characters Variables

o Variable Substitutionso Parameters

o Environmental Variables

o Special Variables

Bourne Shell Programmingo Testing and Branching

o String Comparisons

o Testing Files

o Loop Control

Exiting Loops Earlyo Input and Output (I/O)

o Advanced Programming Topics

Functions Trapping Errors

o Dot Files
http://csr.phys.ualberta.ca/~gingrich/research/shells/node12.html#SECTION00510000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node13.html#SECTION00520000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node14.html#SECTION00530000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node15.html#SECTION00540000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node16.html#SECTION00550000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node17.html#SECTION00560000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node18.html#SECTION00570000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node19.html#SECTION00571000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node20.html#SECTION00572000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node21.html#SECTION00573000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node22.html#SECTION00574000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node23.html#SECTION00580000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node24.html#SECTION00581000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node25.html#SECTION00582000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node26.html#SECTION00583000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node27.html#SECTION00584000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node28.html#SECTION00584100000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node29.html#SECTION00585000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node30.html#SECTION00586000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node31.html#SECTION00586100000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node32.html#SECTION00586200000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node33.html#SECTION00587000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node12.html#SECTION00510000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node13.html#SECTION00520000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node14.html#SECTION00530000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node15.html#SECTION00540000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node16.html#SECTION00550000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node17.html#SECTION00560000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node18.html#SECTION00570000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node19.html#SECTION00571000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node20.html#SECTION00572000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node21.html#SECTION00573000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node22.html#SECTION00574000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node23.html#SECTION00580000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node24.html#SECTION00581000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node25.html#SECTION00582000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node26.html#SECTION00583000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node27.html#SECTION00584000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node28.html#SECTION00584100000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node29.html#SECTION00585000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node30.html#SECTION00586000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node31.html#SECTION00586100000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node32.html#SECTION00586200000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node33.html#SECTION00587000000000000000


8/63

A Bit of History

The Bourne shell is the obvious shell to examine first for two reasons. First and foremost,it was the first shell written. It was was written at Bell Laboratories, by Stephen Bourne.

This makes the Bourne shell the foundation which all other shells are (at least in part)built on. In fact it will be seen that the shells covered in this book fall basically into twocategories, or families, the Bourne family and the C shell family. The second reason tostart with the Bourne shell is that it comes with all UNIX systems, making it a goodintroductory shell purely on the basis of availability.

Getting to Know the Bourne Shell

If the user sits down and starts typing a few UNIX commands he will be utilizing the

shell that happens to be installed. A good place to start is to confirm that the resident shellis, in this case, the Bourne shell. To check if this is true, type the following command:echo $SHELL

and the output should be/bin/sh

indicating that the shell being used is indeed the Bourne shell.If this is not the case, and another shell is installed, the chsh command can be used tochange the shell, or for the purpose of learning the shell, an interactive shell can beinvoked using the sh command. While the default prompt for a non-root user in theBourne shell is a $, this is not really a good way to determine the current active shellsince in any shell the prompt can be changed. For the rest of the examples in this sectiona dollar sign will lead the text to emulate the prompt.

Now that the current working shell is a Bourne shell, it can be examined in some detail. Itis instructional to look at what exactly happens when a command is entered. The usertypes in any command on a line followed by the enter-key or return-key, for example,$ cp *.txt text_files

This is a common enough command, it moves all of the files with a .txt extension fromthe current working directory to a subdirectory called text_files. The shell is set intoaction by the enter-key. It begins immediately by searching the UNIX command foranything foreign to the command. For example it looks for variables, pipelines, otherredirection characters and command separators - all of which will be covered in due time.The above example contains nothing but a simple UNIX command so the shell passes thecommand to the operating system which then processes it. If any of the special characters

mentioned above had been contained in the command, the shell would have handled thembefore passing the command off to the operating system. What if the command was toolong for the terminal window? No problem. The input may look different to differentusers since some terminals will wrap longer commands onto the next line, and others willadjust in such a way that the command will keep going to the right past the terminalboundary. In either case, the shell waits until the enter-key has been pressed before takingany action. There is a way however to enter a long command such that it will be broken


9/63

at the end of the top line and continued on the next. This can be accomplished by typing abackslash (\) character before pressing return at the breakpoint, as follows:$ echo This is a long command so why not break it here \> and start on the next line.

which gives as output:This is a long command so why not break it here and start on the next line.

The > is the shell's way of letting the user know that the current line is a continuation ofthe previous line. Note however that the output is printed as though it was never broken.

Multiple Commands Per Line

One of the things that the Bourne shell looks for when a command is entered, is if there ismore than one command entered. The most common way to have multiple commandsentered on one line is to use the semicolon separator. This separator tells the shell to sendeach command to be processed in the order in which they appear, like the following:$ cd docs; mkdir old_docs; mv *.* old_docs

which is the same as$ cd docs $ mkdir old_docs $ mv *.* old_docs

Now suppose there is a situation where a user wants to have some commands carried outat the same time without having to wait for each to start. A particular situation that comesto mind is copying a several megabyte file (which can take minutes) while trying to doanything else. The following could be entered:$ cp big_file new_dir& rm *.o& who

which is equivalent to$ cp big_file new_dir& $ rm *.o&

$ who where the shell puts the command (plus arguments) before an ampersand into thebackground. In the above case it copies the big_file (in the background), it deletes all ofthe object files (in the background) and finally it runs the who command (in theforeground) .The Bourne shell also allows command grouping. Command grouping treats a group ofcommands as a unit and executes them as such in a subshell. To group commands enclosethem in round parentheses () and separate the commands by semicolons. The groupedcommands are executed in a subshell. For example,$ MY_NAME='Norm Buchanan'$ (MY_NAME='George Bush'; echo $MY_NAME)George Bush

$ echo $MY_NAMENorm Buchanan

This example was chosen to demonstrate how command grouping works, but also to givea glimpse of variable scope or visibility (which will be covered in greater detail insection 2.7, variables). User defined variables are local to the current shell which meansthat they can not be accessed or altered in a subshell. This is why the variable MY_NAMEreturns to its original value as soon as the commands enclosed by the parentheses havebeen executed.
http://csr.phys.ualberta.ca/~gingrich/research/shells/node18.html#varhttp://csr.phys.ualberta.ca/~gingrich/research/shells/footnode.html#56http://csr.phys.ualberta.ca/~gingrich/research/shells/node18.html#var


10/63

These features lead quite naturally into more powerful features of the Bourne shell, orshells in general.

Redirection and Pipelines

The situation often arises where the user desires that the output of a command go to a filerather than to the screen, or that the output of a command go directly to anothercommand. The Bourne shell provides a group of features to handle both situations. Theshell recognizes the following redirection characters: the output (>), the input (>), and the input from standard in (filename

which will send the long listing of the current directory to the file called filename. Thedifference between the (>) and the (>>) is that the append (>>) symbol causes theoutput of the command to be directed to the end of the file if it already exists or else itcreates the file. The standard output redirection operator (>) will create the file in orderto write the output and if this file exists it will overwrite it. The input operator (&D redirects output to D,$ command &- closes standard output, and$ command err.filewhich sends the error messages from the command to a file called err.file. Probably themost common use of file descriptors comes when the user's desire to send stdout andstderr to a file. This would be accomplished by the following:$ command out.file 2>&1

This will send both the output and error messages of the command to a file called out.file.While this is a slightly complicated command to remember, it is well worth the effort.
http://csr.phys.ualberta.ca/~gingrich/research/shells/node10.html#bournedeschttp://csr.phys.ualberta.ca/~gingrich/research/shells/node10.html#bournedeschttp://csr.phys.ualberta.ca/~gingrich/research/shells/node10.html#bournedesc


11/63

Pipelines are used to pass the output from a command and feed it into another commandto be processed. The symbol for a pipeline is |. A good illustration of using a pipelinemight be to count the number of users logged onto the user's system which could beaccomplished by$ who | wc -l

Some command pipelines can be accomplished in more than one way. For example,$ cat text_file | wc -l

gives the same result as$ wc -l ), the input (>), and the input from standard in (filename

which will send the long listing of the current directory to the file called filename. Thedifference between the (>) and the (>>) is that the append (>>) symbol causes theoutput of the command to be directed to the end of the file if it already exists or else itcreates the file. The standard output redirection operator (>) will create the file in orderto write the output and if this file exists it will overwrite it. The input operator (&D redirects output to D,$ command &- closes standard output, and$ command


12/63

where D is one of the three file descriptors. Slightly more complicated I/O redirectionscan be constructed using file descriptors such as$ command 2>err.file

which sends the error messages from the command to a file called err.file. Probably themost common use of file descriptors comes when the user's desire to send stdout andstderr to a file. This would be accomplished by the following:$ command out.file 2>&1

This will send both the output and error messages of the command to a file called out.file.While this is a slightly complicated command to remember, it is well worth the effort.Pipelines are used to pass the output from a command and feed it into another commandto be processed. The symbol for a pipeline is |. A good illustration of using a pipelinemight be to count the number of users logged onto the user's system which could beaccomplished by$ who | wc -l

Some command pipelines can be accomplished in more than one way. For example,$ cat text_file | wc -l

gives the same result as

$ wc -l


13/63

are taken to be string characters. An advantage of quotation marks over escape sequencesis that if there are many special characters in a string, quotes will save keystrokes andthus time. Another advantage is that they recognize white space. An echo statementwithout quotation marks will not recognize spaces or tabs causing the following:$ echo \*\*Warning\*\* Disk Space is Low

which will give as output:**Warning** Disk Space is Low

A quick and easy fix would be$ echo '**Warning** Disk Space is Low'

and this would give the appropriate output. While both single and double quotes solve theabove problems, they are different. The difference comes in handling variables, whichwill be covered shortly. The single quote will not substitute a variable into an expressionwhereas the double quote will. When deciding which of the methods to use, thecomplexity of the expression (i.e. how many special characters are used) should belooked at, as well as variable usage.Command substitution is a topic that ties up these last few sections. Commandsubstitution allows the output of a command to be substituted into another command. For

example,$ echo "There are `who | wc -l` users on the system"

may give as outputThere are 13 users on the system

The user now has the tools and flexibility to handle even the most specific or complicatedcommand in the Bourne shell.

Variables

This is where the shell gets interesting. Variables add a level of generality to theenvironment. A variable is simply a name that acts as a placeholder for a value or set ofvalues (an array which will be covered in later sections). In the Bourne shell there arefour different types of variables:

User Defined - local (only accessible by the current shell), Parameters - command-line arguments, Environmental - special variables for the shell environment and Special - defined by shell.

User defined variables are fairly straightforward. In the Bourne shell they take thefollowing form:$ SIZE=1024$ [email protected]$ greeting='Welcome to the Bourne shell'

To access a variable, a $ must be placed in front of the variable name or else the shellwill not realize that what follows is not a command. The shell will then attempt toexecute the command and return an error message. An example of accessing a userdefined variable is then


14/63

$ echo You can e-mail me at $MY_ADDRESSYou can e-mail me at [email protected]

The above can be used to demonstrate the difference between single- and double-quotehandling of strings. If single quotes are used to enclose the string, the shell will not makethe variable substitution as it treats the $ as a text character rather than a signal that avariable is coming:$ echo 'You can e-mail me at $MY_ADDRESS'

You can e-mail me at $MY_ADDRESS

Double quotes however, will allow the shell to recognize the variable and pass its valueto the echo command:$ echo "You can e-mail me at $MY_ADDRESS"

You can e-mail me at [email protected]

Variable names can be nested as well. This is another way to say that one variable can beset equal to another variable which contains another, etcetera.$ today=Tuesday$ day=$today$ echo The day of the week is $day

The day of the week is Tuesday

When assigning a value to a variable, it is important to leave no white space. This isbecause the assignment is terminated by white space, unless the appropriate quotationcharacters enclose the string value. This allows more than one variable assignment to bemade on a single line:$ a=cat b=dog c=elephant

It is important however to realize that the assignments are processed from right to left andtherefore if the following assignments were made:$ VAR1=$VAR2 VAR2=hello

the value ofVAR1 would be hello whereas if the the order was reversed:$ VAR1=hello VAR2=$VAR1

the value ofVAR2 would be undefined. This is because VAR2 is assigned the value ofVAR1 before VAR1 has been given a value.Variable assignments can also be removed using the unset command. For example,$ VAR="Hello"$ echo $VARHello$ unset VAR$ echo $VAR

$

Variable Substitutions Parameters Environmental Variables Special Variables

Variable Substitutions
http://csr.phys.ualberta.ca/~gingrich/research/shells/node19.html#SECTION00571000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node20.html#SECTION00572000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node21.html#SECTION00573000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node22.html#SECTION00574000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node19.html#SECTION00571000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node20.html#SECTION00572000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node21.html#SECTION00573000000000000000http://csr.phys.ualberta.ca/~gingrich/research/shells/node22.html#SECTION00574000000000000000


15/63

Once a variable has been assigned a value, it can be easily referenced by the use of adollar sign in front of the variable name as shown above. A user may wish to append astring directly to a string variable such as$ VAR=Tues$ echo $VARday

which will return no value since no value has been assigned to VARday. The shell can notdetermine that the user meant to treat $VAR as a separate entity and hence it treated thethe string as one complete variable name. This is because the shell uses white space wheninterpreting variable 22substitutions. However, if curly braces containing the variablename are placed within a string, the shell will handle the entire string as desired:$ VAR=Tues$ echo ${VAR}day

Tuesday

The Bourne shell also provides variations on the above variable substitution that allowalternative substitutions under certain conditions. There are four such conditionalsubstitutions in the Bourne shell which are listed in table 2.2:

Table 2.2: Conditional variable substitutions.For example, if the user wants to have an error message printed when a variable getsassigned a value, he could do the following:$ ERROR=TRUE

$ echo ${ERROR:+"An error has been detected"}An error has been detected

Parameters

Parameters are another type of shell variable that carry information from the command-line. Any command can be broken up into parts. The first element of the command-line isalways a program (either an executable or a shell script, which will be covered shortly).The following elements can be looked at as special values to be passed to the program.These values are called positional parameters and are stored in the variables $1 through

$9. The parameter$0 is a special variable that always holds the name of the program.Positional parameters will be covered in more detail in the section on shell programming.

Environmental Variables
http://csr.phys.ualberta.ca/~gingrich/research/shells/node19.html#bournecondhttp://csr.phys.ualberta.ca/~gingrich/research/shells/node19.html#bournecond


16/63

Environmental variables are variables that have special meaning to the shell. These are tobe used to customize the environment for a particular user. These variables should bedefined in the special program that is executed during login called the .profile (dotprofile) file. Any defined variables would then be set until the end of the session unlessexplicitly unset or redefined. An example of a .profile file is included in the section on

programming. Table 2.3 contains an alphabetical list of the Bourne shell environmentalvariables, a brief description of what each is used for, and the default setting.

Table 2.3: Bourne shell environmental variables.To make these variables global (so that they maintain their value in any subshell) theexport command must be used. For example if you use ---> for a prompt in the currentshell and then start up a subshell, the prompt will be $ again. To remedy this situation usethe export command:$ PS1="--->"$ export PS1

To see a list of environmental variables currently set and their settings, one can use theenv command. A typical listing might be as follows:$ envCOLUMNS=80HOME=/users/smithLINES=24LOGNAME=smithMAIL=/usr/mail/smithPATH=/bin:/usr/bin/:/users/smith/binSHELL=/bin/sh

TERM=vt100

Special Variables

Special or builtin variables are define by the shell for use at any time. They are mostlyused for shell programming and are covered in more detail in the next section. Table2.4contains descriptions of the Bourne shell special variables:
http://csr.phys.ualberta.ca/~gingrich/research/shells/node21.html#bourneenvhttp://csr.phys.ualberta.ca/~gingrich/research/shells/node22.html#bournespechttp://csr.phys.ualberta.ca/~gingrich/research/shells/node22.html#bournespechttp://csr.phys.ualberta.ca/~gingrich/research/shells/node21.html#bourneenvhttp://csr.phys.ualberta.ca/~gingrich/research/shells/node22.html#bournespec


17/63

Table 2.4: Bourne shell special variables.

Bourne Shell Programming

Shell programming gives the shell a tremendous amount of power. Not only can groupsof commands be used together in a shell program (or script), but the Bourne shell comeswith a full programming language that allows such things as arithmetic calculations onvariables, decision making or branching, and controlled looping. Unlike languages suchas FORTRAN, PASCAL and C, shell scripts do not require compiling or linking. Thismeans that there is no conversion of the source code down into machine language beforeexecution. A script is really nothing more than a sequential list of instructions (some ofwhich are UNIX commands) that the shell interprets and executes. Each shell script iswritten using the particular syntax, or language, given in the following sections. After thescript has been written to the users satisfaction, it is made to be an exectabule file usingthe chmod command. If, for example, a user had a file called my_script that contained ashell program he would do the following:chmod u+x my_script

Notice that there is no special extension or any other feature to distinguish the script as aspecial program. A shell script can be given any name with any extension, and thereforeit is convenient to give the scripts meaningful names to help the user remember theirpurpose. A script can also be run in a subshell by using the command sh script_name.The most simple shell script is one which contains a group of UNIX commands to carryout a particular task. The following simple example is a script to backup all the files inthe subdirectories above a directory named data. It first archives all of the subdirectoriesand files using the tar utility, it then compresses the archive file to conserve space, andfinally copies the compressed archive to a remote disk for storage:#! /bin/sh

# Backup is a sh script to back up all of my datatar cfv mydata.tar data/compress mydata.tarcp mydata.tar.Z /dsk2/storage/

All but the top two lines of the shell script are ordinary UNIX utilities. The second linedoes nothing, it is just a comment statement. In UNIX shell scripts, the hash charactersignifies a comment statement, allowing text documentation to be left for futurereference. This particular script is short and straightforward enough that documentation


18/63

may not be required, but in larger scripts it can be essential in order to understand whatthe program does. Very few people can remember what exactly a program does morethan a week after writing it. A well documented script is much nicer to debug, or modify,at a later date than a seemingly endless string of commands and other instructions. Thefirst line is a special line that should be present at the beginning of any shell script written

in any shell scripting language. The first two characters tell the shell that what follows isthe path of the shell that the script was written under. The current shell will then start asubshell of the type contained in the first line (a Bourne shell in this example). After thescript has been executed, control is returned to the original shell. After a glance over theabove script, the reader may wonder what the difference is between a shell script and agroup of commands, say a pipeline. Well, in this simple example there is not muchdifference, except for the fact that the same group of commands can be carried out bytyping the name of the script rather than re-entering them each time. This is the mainreason for putting a small group of commands into a script.

Testing and Branching String Comparisons Testing Files Loop Control

o Exiting Loops Early

Input and Output (I/O) Advanced Programming Topics

o Functions

o Trapping Errors

Dot Files

Testing and BranchingThe power of programming lies in the ability to have the computer make a decision basedon one or more choices (either embedded right in the program or provided by the userwhen the program is executed). This decision-making process is called branching,because of the obvious analog of a tree-branch system. The tool provided by the Bourneshell to build these branching systems is the if:then:else construct, which has thefollowing form:if statement_Athen

statement_Bstatement_C

elsestatement_D

fistatement_E

What happens here is that statement_A (or more correctly condition_A) is attempted. If atrue value is returned, statement_B and then statement_C are carried out, followed bystatement_E. If a false value is returned from statement_A, statement_D is executed


19/63

followed by statement_E. The fi statement signals the end of the construct and allowssequential execution of the following commands to resume.The interesting part of the branching process is how the decision is made. Theprogrammer sets up a test (statement_A above). Statement_A can be a specific condition(is A bigger than B) or it can be an expression. When the shell executes a command or an

expression it returns a value (called the exit status); if the command executes withoutproblems, a true value is returned, otherwise a false value is returned. It may seemstrange to have a command act as a test condition, but this is actually a very convenientway to handle errors and other conditions. For example a user may want to automaticallyprint a file, and have an error returned if the file does not exist:if ls plot.psthen

lpr plot.pselse

echo ":::: ERROR FILE DOESN'T EXIST ::::"fi

If the file does not exist the shell will produce an error message, separate from the

message in the echo statement. This error message is what the shell returns automaticallyif the argument (filename) for the list command does not exist. The stderr can beredirected form the screen to a file which would keep the error messages from coming upon the screen, but it would also leave a file containing the message in the user's account.UNIX provides a tool called the null device to handle situations like this. The null deviceacts like a file that is emptied as soon as anything is placed in it. Any unwanted outputcan be redirected to the null device and it will then be immediately disposed of. Caremust be taken when sending anything to the null device as nothing can be recovered onceit has been sent there. The following modification to the line containing the ls commandwill send stderr to the null device:if ls plot.ps 2>/dev/null

The above demonstrates how a return value, or exit status, can be used in a test, but whatabout other types of tests? Suppose two strings are to be compared, and the result of thecomparison determines the next step, or if the status of a file determines the branch taken;what then? The Bourne shell comes with two ways to test conditions, the explicit method:the test statement; and the shorthand method: square brackets [] . The full general testexpression works as follows:$ string1="APPLE"$ string2="ORANGE"$ test "$string1" = "$string2"$ echo $?1

The 1 that is returned is a false value. In the Bourne shell an exit value of zero is

translated as a true value whereas a non-zero value is translated as a false value . It isalso important to note that the white space between the = sign and the values on eitherside are required. In assignment expressions there would be no white space around theequals sign. When used in a decision branch the above example is a lengthy way to get tothe result of the test. A more direct test would take the following form:if [ "APPLE" = "ORANGE" ]then

(A)else
http://csr.phys.ualberta.ca/~gingrich/research/shells/footnode.html#119http://csr.phys.ualberta.ca/~gingrich/research/shells/footnode.html#118


20/63

(B)fi

where statement(s) B will be executed, as the strings are clearly not equivalent. Again, thewhite space around the brackets is necessary. This is a much more elegant method oftesting and will be the preferred choice for the remainder of this chapter.The three conditions that will need to be tested are: string comparisons, file status andinteger comparisons.

String Comparisons

Strings are considered to be equal if each character in one is matched exactly in the other,and they have the same length. For example the following strings are not equal: String isnot equal to String. While the strings contain the same letters, the second string containsan extra space, and they are therefore not equal. All of the string tests supported by testare listed in table 2.5.

Table 2.5: String tests supported by test.

Testing Files

A very important test is often the status of a file. Perhaps one test would be to test if a fileexists before creating another with the same name (which would of course destroy andreplace the pre-existing file). The test command will test files for such attributes asexistence, writability, readability and file type. Table 2.6summarizes the file test formatsand return values.
http://csr.phys.ualberta.ca/~gingrich/research/shells/node25.html#borunestringhttp://csr.phys.ualberta.ca/~gingrich/research/shells/node25.html#borunestringhttp://csr.phys.ualberta.ca/~gingrich/research/shells/node26.html#bournefilehttp://csr.phys.ualberta.ca/~gingrich/research/shells/node26.html#bournefilehttp://csr.phys.ualberta.ca/~gingrich/research/shells/node25.html#borunestringhttp://csr.phys.ualberta.ca/~gingrich/research/shells/node26.html#bournefile


21/63

Table 2.6: File test formats and return codes.

To illustrate the use of file testing, the following is a script that acts as a revised versionof the UNIX move (mv) command. Not only does this script check for overwrites, it alsodeletes any file that is of zero size.#! /bin/sh## Move is a script that moves a file if no overwrite will occur# and will delete the file to be moved if it is empty## Usage: move [file] [destination]#if [ ! -s $1 ]then

if rm $1 2>/dev/null

thenecho "$1 was empty and thus removed"

elseecho "$1 does not exist"

fielse

mv -i $1 $2fi

This example uses the if:then:else construct but also uses a few things not discussed inmuch detail. The first new idea was briefly covered in the section on variables -parameters. The $1 passes the name of the file being moved (the first name entered onthe command-line aftermove), and the $2 is the destination or place that the file is

moving to. The Bourne shell allows nine such parameters, each separated by white space,and therefore nine pieces of information can be passed to a shell program from thecommand-line.While the Bourne shell only accommodates direct access to nine parameters (i.e. $1through $9), nothing stops the user from entering more. The shell comes with a tool thatpermits access to more than nine parameters. More generally, this tool permits the use ofan undetermined, at run time, number of parameters. This means that the script can let theuser enter as many parameters as he wishes on the command line and the program will


22/63

handle any or all of them as required. This tool is the shift command. The shift commandslides the parameter list by a specified number of places to the left and has the followingform:shift n

where n is the an integer that specifies how many places the list will be adjusted. Ifn isnot explicitly given, the parameter list is moved one place. The following illustrates howthe shift command works on a parameter list of size n. Before the shift command hasbeen used, the list would look like$1 $2 $3 $4 ... $n

and after using shift:$2 $3 $4 $5 ... $(n-1)

where after the shift command was used, the first parameter is no longer accessible. Thenumber of parameters in the parameter list drops by the number of places that the list wasshifted as well. For example, if a user inputed 5 parameters and then the script shifted thelist:echo $#5

shiftecho $#4

where the 4 and 5 would be the output of the shell script. This presents a clever way ofaccessing any number of parameters in a script. To ensure that all parameters have beenprocessed, a simple test command can be incorporated into the script as follows:while [ $# -ge 1 ]

doecho $1...shift

doneThis script piece will print the parameters, carry out some unspecified tasks, and repeatuntil all of the parameters passed have been exhausted.If a script is written to expect a certain number of parameters, and the user places toomany on the command line, the program will only use as many as were required. Thismeans that if the script expects 2 parameters, and the user enters 3, only the first twoentered will be used. If, on the other hand, the user enters fewer parameters than thescript was written to expect, it will fill the un-entered parameters with a null value (i.e.they will be empty).Another new idea (from the previous example script) was that of having one ifstatementwithin another, which is called a nested ifstatement. This allows tests and decisions to be

made as a result of earlier tests. Nested decisions allow very specific testing with lesscoding than would otherwise be the case. Notice that the first ifstatement uses a !character, which negates the test, or turns a TRUE value to a FALSE value and viceversa.The comments have been included to give some description of the task the script willcarry out. One commented line in particular gives the expected usage of the script whichallows the user to have knowledge of which order to enter the parameters on the


23/63

command-line. This is not required but is good practice - especially if the script is to beused by people other than the programmer.There is a third type of test that can be made. This is the integer comparison test. Thistype of test is done when comparing the values of two integers. The first integer, the oneon the left, is compared with the one on the right. For instance, the following tests to see

whetherInteger1 is greater than Integer2:$ test Integer1 -gt Integer2There are in total six comparisons that can be made between two integers: if they areequal (-eq); if the are not equal (-ne); ifInteger1 is greater than Integer2(-gt); ifInteger1 is greater than or equal to Integer2(-ge); ifInteger1 is less than Integer2(-lt);and finally, ifInteger1 is less than or equal to Integer2(-le).In the Bourne shell, mathematical expressions cannot be simply assigned to a variable.This means that the expression a=b+c is not valid. To get the result of an arithmeticexpression, the expr command must be used. This command evaluates the arithmeticexpression, and then returns the result. There are five arithmetic operators that can beused in the Bourne shell:

Table 2.7: Arithmetic operators in the Bourne shell.The multiplication and division operators have higher presidence than do the addition andsubtraction operators. The order can be arranged in any fashion however, with the use ofback quotations (`). The Bourne shell does not recognize parenthesese. For example,$ expr 3 + 2 \* 819

is different from

$ expr `3 + 2` \* 840

Note also, that the multiplication operator (*) was escaped in the expression. This isbecause the shell tries to use the charater as a filename expansion if it is not escaped. Onelast important note on arithmetic expressions is that the division operator is integerdivision. This means that when one integer is divided by another integer and the result isnot itself an integer, the remainder is dropped and only the integer portion of the divisionis returned. The modulus operator can be used however to return the remainder of theoperation.It may be neccessary to use more sophisticated compound tests in a decision, andtherefore tests can be combined using logical AND and OR operators. A compound test

is a fancy way of saying that two simple tests are combined into a single test using logicaloperators. The logical AND operator is written -a, while the logical OR operator iswritten -o. To illustrate the use of a compound test, consider the situation where a file istested for readability as well as for writability:if [ -r prog_name -a -w prog_name ]then

command listfi

A test can also be negated by placing the ! character before the test:


24/63

if [ ! -x prog_name ]then

lpr prog_nameelse

echo ``Are you sure you want to print an executable file?''fi

The if:then:else construct works well for handling one or more decisions in the flow of ascript, but an even better construct is available for handling decisions where manychoices are available - the case construct. In these situations, the case construct issimplier to impliment than the if:then:else construct. The case statement takes a value, orpattern, and compares it with a list of patterns. The number of patterns is up to theprogrammer's discression, but there is no imposed limit. Along with each pattern in thelist is a list of commands. The first pattern that matches the initial value has its commandlist executed. The case construct has the following formatcase value in

pattern1)command list;;

pattern2)

command list;;...

patternN)command list

esac

The double semicolons are used to signify the end of a command list. The last group ofcommands does not require the double semicolons as the esac statement signifies the endof the construct, which implies the end of the last group of commands. More than onepattern can be entered on a line if they are separated by the logical OR symbol (|). Whenmultiple patterns are used in this way, the first line that has any one of its patterns

matched has its command list executed. Character substitutions, such as * and [], can alsobe used in a pattern. Since the patterns in the comparison list are compared from top tobottom it is always good practice to use the meta-character (*) as the last pattern choice tohandle any unexpected choices. The case construct is especially useful in menus ofchoices such as the following:echo ``Do you wish to delete $FILE ?''echo ' 'echo ``q or quit Quits''read ANSWERcase $ANSWER in

y|Y|yes|YES)echo `Removing $FILE'';

rm $FILE;;n|N|no|NO)echo ``$FILE was not removed'';;

q|Q|quit|QUIT)exit 1;;

*)echo ``$ANSWER was not an understood option'';exit 2

esac


25/63

The only command used here which is yet to be covered is the read command which getsinput from the user. This is covered in the section on input-output.

Loop Control

While branching is an integral part of shell programming, it is only one part of programcontrol. Looping in a program allows a portion of a program to be repeated as long as theprogrammer wishes. This can be for a specified number of iterations (or loops), or it canbe until a particular condition is met. For instance, a programmer might want to repeat aparticular operation on every file in a particular directory. Rather than rewrite the sectionof the program that carries out the operation over and over for each file, the operation canbe written once and iterated as many times as required. Loop control also allows forprograms that are more general. Rather than having to pre-specify how many iterationsare required for a particular task, the programmer uses conditions to control the iterations.The Bourne shell provides a rich variety of loop control constructs. Depending on what isneeded the programer can chose the construct that best suit his needs.

The for:in:do construct is used to repeat a group of commands once for each item in aprovided list. The construct has the following form:for VARIABLE in LIST

doCOMMAND LISTdone

where VARIABLE is a variable name assigned each item in LIST during the execution ofCOMMAND LIST. What happens is as follows: the variable takes the value of the first itemin the list and then executes the command list; after the command list has been passedthrough, the variable is assigned the value of the second item in the list, and so on, untilthe list has been exhausted. Searching for the occurence of a string in a file could be donelike the following:#! /bin/sh## A script to look for the occurence of a string in a file# Usage: match [string] [file]#for word in `cat $2`

doif [ ``$word'' = ``$1'' ]then

echo ``Found $1 in file $2''else

:fidone

where the first parameter passed to the script is string and the second is the file thought tocontain the string. Notice that after the else statement a colon has been placed. This is thenull command which tells the computer to do nothing. This is clearly an unnecessarysection of the script and was only added to demonstrate the use of the null command.If the list is omitted from the for statement, each parameter in the command line will bepassed to word.


26/63

A second type of loop control construct is the while loop. This construct, unlike thefor:in:do construct, checks the TRUE or FALSE value of a condition before proceeding.It has the following form:while condition

docommands

donewhere the condition can be obtained in the usual fashion using one of the forms of test.The done statement signifies the end of the construct. The following script segmentcounts backwards from 10 to 1:number=10while [ $number -ge 1 ]

doecho $numbernumber=`expr $number - 1`done

Another construct which is very similar to the while loop is the until construct. Theconstruct works in precisely the same manner with the one exception that it repeats a

series of commands until a condition is met. The until loop looks likeuntil condition

docommandsdone

To count backwards, as in the above example, the until loop would be used as follows:number=10until [ $number -lt 1 ]

doecho $numbernumber=`expr $number - 1`done

Exiting Loops Early

In the three looping constructs examined above, the loop would continue to execute untila specific condition was met (a boolean condition in the while and until loops, orcompletion of a certain number of loops in the for loop). There are times however when itwould be beneficial to exit a loop early. The Bourne shell provides two methods forexiting from a loop early, the break command, and the continue command.The break command will exit from the current loop and program control will be resumeddirectly after the loop construct exited from. The break command takes an integerparametern which determines the number of levels to jump out of. For example,until cond1

doCommand_Auntil cond2

doCommand_Bif ! $?

break 2fi


27/63

doneCommand_Cdone

Command_D

In this example, cond1 is evaluated and if it does not have a TRUE value Command_A isexecuted. Ifcond2 has a FALSE value Command_B is then executed and the following if

statement checks to determine if the return value was TRUE. If the return value is TRUEcond2 is again tested. If the return value ofCommand_B is not TRUE (i.e. non-zero), thebreak command is executed. Since a parameter value of 2 has been passed to thecommand, it jumps out two levels and Command_D is executed. Notice thatCommand_C can only be executed ifcond2 becomes TRUE.The second method for exiting a loop early is the continue command. This commandbehaves very much like the break command with the exception that it jumps out of thecurrent loop and places control at the next iteration of the same loop structure. Thecontinue command also takes an integer parameter which determines the number oflevels to jump back. Looking at the above example, with the continue commandreplacing the break command.

until cond1doCommand_Auntil cond2

doCommand_Bif ! $?

continue 2fidone

Command_Cdone

Command_D

In this example, Command_A is executed as before followed by the test of conditioncond2. IfCommand_B returns a FALSE value, the continue command jumps out of theloop and condition cond1 is again evaluated, and if not TRUE, Command_A is againexecuted. In this example Command_C will only be evaluated if a test ofcond2 returns aTRUE value, as above, but Command_D will only be executed if the test ofcond1returns a TRUE value. The continue command will not pass program control directly toCommand_D as it did in the first example.When used in case structures, the break command is a pleasant alternative to the exitcommand for handling unwanted choices as it allows for control to be passed to anothersection of the program rather than exiting the program entirely.

Input and Output (I/O)

Now that the programming structure has been covered, it would be nice to be able tointeract with the program while it runs. As has been shown above, the program can writeout to the screen, a file or a device, and can even read in information from the user. Whileall of the above examples have used some form of output, in the form of an echostatement, they have not utilized any of the special escape characters provided. Thefollowing is a list of these special characters and their purpose.


28/63

Table 2.8: Escape characters in the Bourne shell.A couple of these characters are extremely useful for interacting with the user. Forexample, the \c character can be used when getting input from the user to prevent thecursor from dropping to the next line:$ echo ``Do you wish to continue? \c''Do you wish to continue? $

Until the newline character is entered, any text entered will be displayed on the same lineas the input prompt. Another useful special output character is the system, or alert, beep \

007 which can be used when alerting the user that a problem or error has arisen:if [ ``$password'' = ``'' ]then

echo ``You must not have a null password \007''exit 1

fi

Shell scripts can also take input from stdin. User input allows shell programs to be fullyinteractive, which will add to their generality. The read statement is used for thispurpose. The read statement will cause the shell to accept an input string to be read inuntil a newline character is encountered. The shell removes all white space (with theobvious exception of newline characters) but does not interpret filename or charactersubstitutions. The shell takes the input string and breaks it up into substrings which are

surrounded by white space. The read statement has the following form:$ read Variable1 Variable2 Variable3 ... VariableN

where each variable is an expected substring. If, for example, the user is expected to inputtwo filenames, the read statement would be$ read file1 file2

If the user accidentally enters three files rather than two, the second variable will beassigned the last two file names. If on the other hand, there are more variables thansubstrings entered, the unmatched variables will be null, or empty.

Advanced Programming Topics

Functions Trapping Errors


29/63

Functions

As a shell programmer becomes more experienced, he will undoubtably want to writeincreasingly more complex shell scripts to handle more interesting tasks. With morecomplex tasks come larger more complex programs. As in any computer language, large

programs can become cumbersome and difficult to understand. This can be a realproblem when attempting to maintain scripts. One way that programmers have managedto keep complicated programs clear and hence easily maintainable is to modularize theprograms. This means to write programs in blocks or modules, where each modulecontains a group of commands specific to a certain subtask of the whole program. Anexample of this would be a program written in any generic language to calculate the sumof areas of a circle, square, and a triangle. While it would be possible to write theprogram sequentially, using modules gives the program a certain clarity that can noteasily be achieved any other way:begin areasum

circ_area ()

area = Pi * rad * radreturn area

squar_area ()area = side * sidereturn area

tri_area ()area = 1/2 base * heightreturn area

sum = circ_area + squar_area + tri_area

endThese modules are often called subroutines or functions. The difference between themeanings is language dependent. In Bourne shell programming, these modules are calledfunctions. Another reason, even more valuable than maintenance, for using functions isthe ability to repeat a task many times without having to re-enter the same code wheneverit is needed. A good example of this is a sine function. A program which tracks a sattelitemay have to calculate a sine function many times. If the list of commands to calculate thesine function had to be written in to the program for each time it was to be used, thelength of the program would be ridiculous. Instead, a function which calculates the sineof a number would be written once in a function and called when needed. In shellprogramming it may become necessary to repeat a task many times in the context of a

larger task. For example, consider a function which might examine a file and print outinformation pertaining to a particular string contained in the file. The actual programmight examine many directories containing many different files, but only want toexamine a certain type of file for this string. The main shell script could then call thisfunction only when it is needed rather than have it operate on all of the files examined bythe main program. This could save the user much time and make his entire program moreefficient. The following example of a function is an enhancement of the long listing inUNIX (ls -l) in that it displays a title over each of the columns of information diplayed:


30/63

$ list () {> echo ``Permission Ln Owner Group Size Modified Name''> echo ``---------- -- ----- ----- ---- -------- ----''> ls -l $*;>}

which could be used as follows:

$ listPermission Ln Owner Group Size Modified Name---------- -- ----- ----- ---- -------- ----total 86-rwx-r---- 1 normb users 60041 Dec 15 19:48 Bourne.texdrwx------ 2 normb users 1024 Sep 18 3:13 mail/drwxr-xr-x 2 normb users 1024 Sep 30 11:01 tmp/$

Because of the use of the $* variable this long listing function will also take wildcards (orfilename meta-characters).

Trapping Errors

This last topic on Bourne shell programming deals with the way programs can handleinterruptions. An interruption can be anything from a terminal being disconnected to thecomputer running out of memory. What happens if an interruption occurs duringexecution of a script? This is not an easy question to answer as the answer is basicallythat it depends on what kind of interruption has occured. If, for example, the power to thesystem was lost, there is really nothing that can happen since the computer will no longerbe operational. If however, the user presses an interrupt (or break) key, the question isstill not answered. Will the program stop immediately or will it finish executing thecurrent loop, or perhaps nothing will happen. What actually happens depends on acommand called trap. The trap command allows the user to have the program carry out a

command string of one or more commands prior to exiting the script. If more than onecommand is contained in the string, the commands should be contained in quotes asdescribed in the section on shell special characters, and command execution. The syntaxof the trap command is as follows:trap command(s) signal(s)

One situation where this command is extremely useful is where information is beingwritten to a file (to be removed at normal exit of the program) and an interruption (orusing correct UNIX terminology, a signal) is sent which would by default cause theprogram to terminate. If program termination occurred prior to normal exit from theprogram, the default action would terminate the program and leave the mess behind.Adding the following to the script would allow the clean-up to occur before terminationdue to a user break (control-C):trap `rm tmp/*; exit 1;`

Care must be taken for a couple of reasons when trapping signals. First, the signals whichmay be trapped vary from machine to machine, and second, the definite kill signal (9)cannot be trapped on any machine. What follows is a typical table of signals and theevent which causes them:


31/63

Table 2.9: Signals and the event which causes them.The details of this list are not really important as there are only a few which will be

trapped in ordinary day-to-day activities: 0, 1, 2 and 15. One should also note that thedefault (ie. not using trap) is always immediate termination of the script.

Dot Files

Throughout this chapter there have been mentions of the environment, such asenvironmental variables. For instance it was shown that the environmental variable$PROMPT allowed the user to change the appearance of his prompt during an interactivesession. What if however, a user would like to have that prompt setting as the default forevery session? As was mentioned earlier, the prompt setting can be placed in a file called.profile in his home directory which is executed at each login. This file is simply a script

that the shell looks for at login and will execute at that time. This is where anyenviromental variables should be set, such as $PROMPT and $PATH. There will likely bea file /etc/profile which will also execute during login. This is a file that the systemadministrator will have set as a default so that users do not need a .profile in their owndirectory. It also allows the system administrator to set some things like the path so thatthe users on the system can have access to all of the executables without having to try andfigure out the paths for them. Depending on the user's level of access, the /etc/profile isoften a good file to copy into his directory as a skeleton to start with. As well asenviromental settings, function definitions can also be placed in this file. This means thatthe list function which was described in the section on functions can be used during everysession without having to be re-entered. It is always good practice to keep the functions

in their own file to prevent cluttering up the .profile file. As will be seen in later chapters,the newer shells can have several of these startup files which can be quite confusing if notorganized in some fashion. The functions can all be put into a file called .functions whichwould then be executed by the .profile file. The .function file can be executed using thesource command or the . command. The .profile script would then have one of thefollowing two lines (which are equivalent):source .functions

or


32/63

. .functions

The C Shell

A Bit of History Getting Started with the C Shell Command Execution Redirection and Pipelines Filename Substitution Filename Completion Special Characters Command History Aliases Variables

o User Defined Variables

o Array Variables

o Global or Environment Variables

o Parameters

o Special Read Only Variables

o Variable Modifiers

C Shell Programmingo Testing and Branching

o Signal Handling

Job Control Special Files

o Introduction to UNIX Security

A Bit of History

The shells in this book are grouped into two main families, the Bourne shell family, andthe C shell family. Since we have already covered the Bourne shell, it would thus be a

logical step to examine the C shell next to outline the other major family.The C shell was developed by Bill Joy, the creator of the vi text editor, at the Universityof California, Berkeley. He improved on the Bourne shell in many areas by adding somenew features, and altering some of the original features. He based parts of the syntax onthe C programming language, but the C shell and C programming languages are two verydifferent entities.


33/63

Getting Started with the C Shell

As with the Bourne shell, a good place to start is by checking that the current shell is infact the C Shell. The echo command can be used to see which shell is running:

echo $SHELLwhich should give/bin/csh

if the C shell is the current operating shell. If the C shell is not the current shell, it can beinvoked by typing the following command:chsh /bin/csh

will change the login shell (the shell that is started everytime you login). An interactiveshell can be temporarily invoked, until the end of the session, for learning or using the Cshell by typing csh. The default prompt in the C shell is the percent symbol (%) whichwill be used as the default prompt for the rest of this chapter. The following is the typicalappearance and execution of a command line in the C shell:% ls -F

Mail/ News/ bin/ message.txt slide.ps%

This is essentially the same as the output from the same command in the Bourne shellwith the exception of the different prompt. All shells handle command lines in the samemanner. They first accept the user input and then they break it down into individualcomponents such as the command to be executed and any flags that go with it.The C shell allows long command lines in the same manner that the Bourne shell did. If acommand line is too long for the screen (usually 80 characters on most monitors), simplybreak the line with a backslash, or in other words, escape the return character. Thisbackslash character tells the shell to ignore the enter-key when it is pressed. A use of thismight be when copying a list of files into another directory:% cp file1 file2 file3 file4 file5 file6 \> file7 file8 file9 file10 file 11 file12 \> ~jones\data\storage\old_files%

which would allow all of the files to be moved in a single command. The > charactersignifies the continuation of a command line onto the next terminal line. The ~ character,or tilde (pronounced til-duh), is a special character in the C shell that is understood by theshell to mean the current user's home directory. This and other special characters will beexamined in the section on filename substitution.

Command Execution

The C shell allows commands to be entered in different ways. Grouping multiplecommands into a single command line, running commands in the background, andbuilding complicated commands from commands inside of commands can all be handledby the C shell.To execute multiple commands on a single command line, simply place a semicolonbetween each of the commands. The commands will be executed from left to right witheach successive command being executed after the previous one. For example, the


34/63

following command line changes to a directory containing documents which need to bemoved. A new directory is created for the documents, which are then all moved into it.% cd docs; mkdir old_docs; mv *.* old_docs

This task could have been accomplished by executing each of the commands separatelyon a separate line:% cd docs% mkdir old_docs% mv *.* old_docs

If all of these commands execute quickly, this is a good method for carrying out tasks. Ifhowever, one or more of the tasks, takes a significant amount of time to execute, the userwill be left waiting for its completion - wasting time. To get around this problem, timeconsuming commands can be run in the background. This means that the shell handlesthe execution and waits for its completion without the user having to wait to do otherthings. For example, if the user needs to carry out a few routine disk management tasks,including archiving all of his directories and subdirectories, he might wish to have thearchiving done in the background.% rm ~/tmp/*; tar cfv my_dirs.tar ~

Sometimes a situation might arise where the user would like to have commands executedsuch that one command is the argument (or input) of another command. For example, theuser may wish to count the number of users logged into the system at the present time.One way to do this would be to type who and count the number of users, but a better waywould be the following:% wc -l `who`

In this example, the left single quotes tell the shell to execute what is inside first (the whocommand), and use the result as the input to the wc command. The user would only see anumber as the result of this command. While the who command generates a list of userscurrently logged onto the system, this output is sent to the wc command and thereforeonly the number of lines that the wc command counts is output to the screen.

A group of commands can also be executed in a subshell. This means that a group ofcommands can be executed in a shell other than the interactive shell currently being used.To do this, the user would enter a list of commands separated by semicolons inside ofparantheses (). The following example will illustrate the syntax of this type of commandwhile also demonstrating the way in which variables are affected by use in subshells.% set MY_NAME = 'N Buchanan'% (set MY_NAME = 'Joe Blow'; echo $MY_NAME)

Joe Blow% echo $MY_NAMEN Buchanan

It can be seen that the same variable is used in

23494612 the Unix Shell Guide

Documents

Transcript of 23494612 the Unix Shell Guide