8/3/2019 Unix Mod 2

1/42

1

Problem-Solving Approaches in UNIX

When u use a computer, usually have a goal in mind ?

To learn more about the system, play a game,write a program ,edit a letter, communicate with e-mail, chat etc An operating system offers you a work environment for satisfying your goals.UNIX has dozens of command that are not standard equipment on most operating systems.

Example

There are commands that count words, search for strings, sorting etc. UNIX system offers fourapproaches to problem solving. They range from the very simple to the complex.

1. Simplest approach is to find a UNIX command that directly solves a problem

Eg:to sort a file use sort command

2. If u cant find a UNIX command that solves the problem directly look for an appropriate

combination of commands.

E.g.: If u wish to extract from the file lmcst those lines containing mca and havearrange those lines alphabetically ,

grep mca lmcst | sort

3. Shell programming (shell scripts) this is a file of unix commands that is used as a programto tell the shell what to do. Also UNIX allows you to use loops etc.

4. sometimes shell script wont solve a problem or else will solve too slowly. The obvious choiceon a UNIX system is the C language. Since UNIX is written in C and the UNIX system calls

and library functions are available to C programs as ordinary C functions.

4 -problem solving approaches.

Single command solutions

combined command solutions

shell scripts solutions

C program solutions

8/3/2019 Unix Mod 2

2/42

2

Using single UNIX command to solve problems

Find a command that solves your problem and use it.

You may not know about command and how to use it.

One solution to these problem is to browse through UNIX manual.

The UNIX commandline

ls -l -s /usr/black/flag

Command name options arguments

The simplest command lines consists just of a single command name,

such as date or who.

You will follow the command name with one or more arguments. Thesearguments provide additional information for the command.

These arguments are filenames or directories.

cd /usr/include

another type of argument is the option which modifies the behavior of a

command.

Sort lmcst // will sort the file contents alphabetically

sort -r lmcst // sort in reverse order

the options are marked with a hyphen .

User can use several options like ls -lsa

if you want to extend the command to next line use backslash and hitreturn.

rajesh@ubuntu:~$ ls -i ff \

> sort294680 ff 296300 sort

The symbol > indicates that the command is being continued to the nextline

8/3/2019 Unix Mod 2

3/42

3

Standard input, Standard output, Standard error

In a UNIX system, a program, including many UNIX commands, is normally connectedautomatically to three file called standard input, standard output, standard error.

If the program does not explicitly request other files, it gets its input from the

standard input, sends output to standard output and sends any error message to the

standard error. Unless other wise specified all three files are default to your terminal.(devices are treated as files).

ie, it takes input from your key board and sends output to your screen.

Unix allows you to change the standard input and output temporarily by using pipesand redirection.

Unixcommand

Standard input(S.I.)

Standard o/p (S.O)

Standard error (S.E)

Input comes

in here

Error message sent

out here

o/p sent out

here

COMPOUND UNIX COMMANDS

When a single UNIX command does not suffice to solve a problem or do a task, try joining

commands together. The chief tools are for this are redirection and the pipe.

Redirection

Redirection changes the assignments for standard input and standard output. The > operator

makes the filename following the operator become the new standard output, and the cat < f

lmcst

mcet

cet

mit

Both produce the same result.

cat like many file-reading programs can read the standard input and a file input during the execution of

the command.

The standard method is to use a hyphen instead of a filename to stand for standard input.

rajesh@linux-1dzg:~> cat -

Haihai

rajesh@linux-1dzg:~> cat f -

lmcst

mcet

cetmit

unix programming

unix programming

cat concatenated the two input. Since the file name came first, it was printed first. Then thekeyboard input, represented by the hyphen was printed.$ sort < f > f3

input taken from file f and the out put to be routed to f3

$ cat f3

cetlmcst

mcet

mit

Another redirection operator is append operator >>cat f f1 f2 f3 >> f4

8/3/2019 Unix Mod 2

5/42

5

Pipes

Redirection lets us connect programs to files.

Pipe facility lets us connect programs to other programs

A pipe | makes the output of one program the input of another.

grep fresnobillsdue | sort

the output of grep becomes the input to sort.

grep

sort And fresno

but fresno

billsdue

|

SI

Keyboard

SO

SI

SO

SE SE

Input from pipe can be combined with input from files. Use a hypen symbol , so that thecommand recognize that there will be a standard input.

Grep fresno billsdue | sort olddue > newdue

The output from grep becomes the standard input to sort, mean while sort opens the file olddue.The contents of this file are sorted together with the grep output and the sort output is redirectedto the file newdue. Redirection routes output to files while pipes route output to other programs.

The TEE

The tee commands reads the standard input and sends it onto the standard output. It alsoredirects a copy of what it has read into the file of your choice

cat part1 part2 | tee total | lpr

The o/p of cat becomes the standard input of tee It is sent on through another pipe to the lpr lineprinter command. Meanwhile a copy is stored in the file total.

Append to a file using tee (use -a option) : cat part3 part4 | tee -a total | lpr

Filters

A filter is a program that can take a flow of data form standard input, process (filter) it and sendthe result on to standard output.

Examples

8/3/2019 Unix Mod 2

6/42

6

cat, sort, grep

How can you tell if a command is a filter?

Read its description , if it can take input from the standard input and if it send its output to thestandard output, then it is filter.

SHELL SCRIPTS

If user (you) wants to use a sequence of command , he can commit them permanently to a file.Then the shell read the file and follow the script of commands in it, such a file is called a shellscript.

Example

echo Here is the date and time

date

To run this script, give the filename as argument to the sh (for shell) command.

$sh example

Here is the date and time

Tue Mar 3 19:43:52 PST 2009

Here sh command creates a new child shell process. This new shell reads the file example,

executes the command it finds and dies, returning control to the original shell.

User can make the example file executable, by giving executable (e) permission to the file

example.

$Chmod u+x example

$example

Here is the date and time

Tue Mar 3 19:43:52 PST 2009

Features of shell scripts

1. shell scripts can take arguments, just like most commands. These arguments (parameters) canbe symbolically represented within the script. So we can pass on values, filenames.

2. The Shell allows you to create shell variables and to assign values to them. They can be usedby shell scripts for many purpose.

3. the shell has a collection of metacharacters (?,*,[]), characters assigned special meanings thatgreatly expand the applicability of shell scripts.

8/3/2019 Unix Mod 2

7/42

7

4. the shell provides control structures such as if .....else constructs and for loops that let youimplement programming language approaches.

A script built up on standard UNIX commands is highly portable from one UNIX system toanother. Shell scripts take up a little storage. The drawback of shell script is that it is relativelyslow .

C programs for solving problems

Why C ?

Because it is the language that most (90%) of UNIX is written in, so the interface between C andUNIX is straightforward and natural. System calls, the subroutines used by kernel they are Cfunctions. UNIX has about 60 such system functions and they can be used in C programs justlike any other C functions.

C is a compiled language. Means you use one of the system editors to write a program. Then usubmit the program to the C compiler, which converts it to an executable program in machinelanguage. Then u can run the program.

Procedures

1. use an editor, such as vi, ex or ed to write the program. File name should end in .C (to identifyit is a c program to the compiler

example (mca.c)

main(){printf (mca @ lmcst \n);

}2. submit the file to cc, the C compiler

cc mca.c // if ur program is ok, the compiled version is placed in a file called a.out (bydefault)

3. to run the program, type a.out

$ a.out

mca @ lmcst

$

4. to change the name of the a.out file

mv a.out mca

now just type mca to run the program

8/3/2019 Unix Mod 2

8/42

8

Building your own command library of programs

.profile this file is created for you when ur UNIX account is set up.

When u login UNIX scans this file for instructions on how to handle your account.

(BSD versions use .login)

one line in the file is

PATH=.: /bin: /usr/bin (PATH is one of the buitl-in shell variable)

this provides the list of directories the shell uses to find command names.

: is used to separate one directory name from the next.

Change the PATH definition

PATH=.: /usr/raj/rvs: /bin: /usr/bin

once you created this setup you can use your commands the same as you use UNIX commands.

Any changes made to .profile do not go into effect until the system reads the file again.

Working with the Bourne shell

shell is the interface between you and the system. Acts as a command interpreter.

There are two shells in widespread use

the most widespread shell program is the Bourne shell, this is developed by Stephen bourne of

bell labs for bell release UNIX. Another shell is csh it is developed by berkely , developed forBSD version of UNIX. Applications developed on Bourne shell can be used for all UNIXsystems.

Shell's filename expansion capabilities

File name expansion is the ability to expand a shorthand notation for a group of filenames to acomplete set of explicit filenames.

Mr. Mister

Shell metacharacters

The shell uses a set of special symbols called metacharacters to let you produce pattern ortemplates to match various classes of filenames.

1.The ? - stands for any single character.

d?g would match three-character filename starting with d and

ending with g.

8/3/2019 Unix Mod 2

9/42

9

possible matches include dig,dog,d4g, d_8

2. The * - stands for any combination of zero or more characters.

(this will not match a period at the beginning of a filename , to protect .profile)

s*n match any filenames, regardless of length, that starts with s and ends with n.

possible matches sn,sun,soon,sudden,s747n,s1.bin

3. The [] - lets u represent a restricted range of single characters

enclosed within the brackets.

You can use [acdAC] or indicate a range by using a hyphen

[a-m].

d[io]g dig , dog (diog is worng)

each bracket pair represents just one character place in a name

Metacharacters can be used in conjunction with one another.

[A-Z]*[0-9] matches any filename starting with a capital letter and ending with a digit.

Example :

rajesh@ubuntu:~$ ls [A-Z]*[0-9]

B2 C8 CZ29 f1 f3 f4

rajesh@ubuntu:~$ ls [A-Z][0-9]

B2 C8 f1 f3 f4

*[0-9][0-9] matches any filename ending with two digits.

*[10-99] wont work because for a single bracket pair matches only one bracket.

Shell variable

A variable is a name associated with a value. Shell variable comes in two varieties

1. those created and maintained by the UNIX

2. those created by the user

User-Created shell variable

name=value (name of shell variable = value of shell variable)

8/3/2019 Unix Mod 2

10/42

10

pet=bulldog (pet is a shell variable name = its value is the string bulldog

shell variables are string variables

weight=156

the value of weight is a three-character string consisting of the digit characters 1 5 6.

the value of weight is not the number 156. the shell can if necessary interpret a digit-stringnumerically.

One special string called null string, it is a string consists of no characters.

Vacuous= ' '

notmuch=

idea=

No space are used when we assign a value to a variable.

You can use letters, digits and underscore character in variable names.

Example

figaro, HO, name2, go_where (LEGAL VARIABLES)

big-foot=squasher (illegal)

squasher=big-foot ( legal)

To find out the value of a shell variable, we can use echo command

$echo pet

pet

a variable name preceded by a $ as an argument to echo, the value of the variable is echoed.

$echo $pet

bulldog

// pet is a variable name

// $pet is the variable's value.

$echo the value of pet is $pet

the value of pet is bulldog.

rajesh@ubuntu:~$ pet=

rajesh@ubuntu:~$ echo the name of pet is $pet

8/3/2019 Unix Mod 2

11/42

11

the name of pet is

System shell variables

UNIX maintains its own set of shell variables.

Typesetto see what your system is using.

The set command makes the shell print out the variables it knows, (system and user variables).

HOME=/usr/mca08PATH=.:/bin: /usr/bin

Left of equal represents the variable name and to the right is the variables value. System shellvariables are in upper case.Some standard shell variable

EXINIT initialization instructions for the ex and vi editors

HOME this is set to the pathname of your home directory

IFS (internal field separator) this is set to a list of the characters that are used to separate wordsin a command line. This list consists of the space character, the tab character and new linecharacter.

LOGNAME The users login name

MAIL this variables value is the name of the directory in which electronic mail addressed toyou is placed. The shell checks the contents of the directory often.

PATH this names the directories which the shell will search to find commands that you use. Acolon is used to separate the directory names. The directories are searched in the order given.

PS1 (Prompt String 1) : this is the symbol used as your prompt ( normally it is set to $, we canredefine it by assigning a new value.

PS1=#

PS2 (Prompt String 2) : this prompt is used when UNIX thinks you have started a new line without finishing a command.

Example

you can continue a line by using a backslash before hitting the return key.

rajesh@rajesh-laptop:~$ echo hello ho\> w are youhello how are yourajesh@rajesh-laptop:~$

TERM this identifies the kind of terminal you use.

8/3/2019 Unix Mod 2

12/42

12

Many UNIX command use built-in shell variable.

cd $HOME

we can use system shell variables in programs also

Local and Global shell Variables : export

ordinarily a shell variable is a local variable. That is , it is known only to the shell that created it.When u start a new shell that shell is born ignorant of the old shell's variables.so each shellvariable are private (local) to itself.

rajesh@rajesh-laptop:~$ cat=katerajesh@rajesh-laptop:~$ echo $catkaterajesh@rajesh-laptop:~$ sh$ echo $cat$ rajesh@rajesh-laptop:~$ echo $cat

kate

If we want the new shell to know the old's shells variable use the export command. Any shellvariable used as an argument for this command will have copies of the variable and its valuepresented to all shells descending from it.this kind of variable is termed global.

rajesh@rajesh-laptop:~$ export catrajesh@rajesh-laptop:~$ cat=lmcstrajesh@rajesh-laptop:~$ echo $catlmcstrajesh@rajesh-laptop:~$ sh

$ echo $catlmcst$ rajesh@rajesh-laptop:~$ echo $catlmcstrajesh@rajesh-laptop:~$ sh$ export j$ j=john$ echo $jjohn$ rajesh@rajesh-laptop:~$ echo $j

Variables can be exported to subshells, but they cannot be exported back to parent shells.Exported shells are not completely global variables

SHELL SCRIPTS

two types of approaches

8/3/2019 Unix Mod 2

13/42

13

1. write an interactive script that requests the information it needs from the user. Prompts theuser fro input, perhaps asking a question, and then reads the users answer. (shell is an interactiveprogram it prompts with $ and reads the commands you type)

2. the script accepts arguments from from the command line.

cat sortfile

this method is more powerful and flexible.

Interactive Shell Scripts

this is based on using UNIX shell commands read and echo.

echo Dear User , what is ur name\?read nameecho ur name is $name

here we use the backslash to remove the special pattern-matching property of the question markcharacter. (this is called escaping or quoting the character)

place this program in a file glad and execute.

Echo enter three nosread a b c

the first variable gets the first value u type and so onif there are more words than variable,all the left over words get assigned to the last variable.

echo enter 3 nos

read a b cecho $aecho $becho $cecho $a - $c

enter 3 nos1 2 3 4 5 6123 4 5 6

1 - 3 4 5 6

If there are more variables than words, the unused variables are not assigned values. Read is abuilt in shell command., it is part of the sh program

program for copying a file

8/3/2019 Unix Mod 2

14/42

14

echo what file to copyread orgecho what name u desire for copyread copycp $org $copy

echo $org is copied into $copyShell scripts with argumentspositional parameters

convert pounds yen pesos

(shell script - $0) $1 $2 $3

refer the first argument as $1 , the second argument as $2, the special symbol $0 stands for thename of the command itself. $* stands for all the arguments from $1 on up ($1,$2,$3 ) thenumbered arguments are referred to as positional parameters $#, which represents the number ofarguments .

rajesh@rajesh-laptop:~/Desktop$ sh show a b c

I used show commandthses are my arguments a b cthe total arguments are 3the first arg is athe second arg is bthe third arg is c

cp $1 $HOME/Music/$1.svrajesh@rajesh-laptop:~/Desktop$ sh new\ file hh

rajesh@rajesh-laptop:~/Desktop$ ls /home/rajesh/Music/

hh.sv

we can use $1 more than once.

Write a shell script to make a shell script into an execuitable.

chmod u+x $1 // save this in a file execfile

rajesh@rajesh-laptop:~/Desktop$ sh execfile hh

copy this execfile in /usr/bin then u just type execfile hh

using set

set is used to assign values to positional parameters.

rajesh@rajesh-laptop:~/Desktop$ set hai how r u

rajesh@rajesh-laptop:~/Desktop$ echo $1 $2 $3 $4

8/3/2019 Unix Mod 2

15/42

15

hai how r u

Set is used in conjunction with the metacharacter ` (backquote) when a command is enclosed inbackquotes, the command is replaced by the output it produces, this process is called commandsubstitution.

rajesh@rajesh-laptop:~/Desktop$ echo date

date

rajesh@rajesh-laptop:~/Desktop$ echo `date`

Thu Mar 12 02:51:04 IST 2009

rajesh@rajesh-laptop:~/Desktop$ who am irajesh pts/0 2009-03-12 01:43 (:0.0)

rajesh@rajesh-laptop:~/Desktop$ set who am i

rajesh@rajesh-laptop:~/Desktop$ echo $1 $2 $3who am i

rajesh@rajesh-laptop:~/Desktop$ set `who am i`

rajesh@rajesh-laptop:~/Desktop$ echo $1 $2 $3 $4 $5rajesh pts/0 2009-03-12 01:43 (:0.0)

Write a shell script that print outs date information in this order : time , day of week, day

number, month, year.rajesh@rajesh-laptop:~/Desktop$ set `date`

rajesh@rajesh-laptop:~/Desktop$ echo `date`Thu Mar 12 03:28:25 IST 2009rajesh@rajesh-laptop:~/Desktop$ echo $4 $1 $3 $2 $603:28:32 Thu 12 Mar 2009

Eg)echo do not panic! All is wellname=$1echo you know $name, that are one of my favourite users.set `date`echo It is $1 $3 th $4 , $6echo relax $name

echo the time is now $4 $5

do not panic! All is wellyou know rajesh, that are one of my favourite users.It is Thu 12 th 03:38:07 , 2009relax rajeshthe time is now 03:38:07 IST

8/3/2019 Unix Mod 2

16/42

16

eg)set `who`who | grep -i $1echo $1

if [ $1 = rajesh ]then

echo the user $1 logged infi

rajesh@rajesh-laptop:~/Desktop$ sh userrajesh tty7 2009-03-12 03:59 (:0)rajesh pts/0 2009-03-12 04:03 (:0.0)rajeshthe user rajesh logged in

Looping and making choice

what truly make the shell a programming language is its ability to control the flow of a program

for loopcasewhileuntilif constructions

THE for LOOPa loop is a programming device that lets you cycle through the same steps

several times.

The general form of the Bourne shell for loop

for variable-name in value1 value2

do

commands

done

---------------------------------------------------------------------------------------------------

Eg)for i in 1 2 3 4doecho $i

done1234

8/3/2019 Unix Mod 2

17/42

17

Eg)for i in 1 2 3 4; do echo $i; done

Eg)for i in 1 2 3 4> do> echo $i> done

Eg)for i in fly spider frog> do> echo i know $i> echo swallowed a $i> donei know flyswallowed a flyi know spiderswallowed a spider

i know frogswallowed a frog

Printing file contents one word per line

for i in `cat glad`do

echo $idone

For loop with command line argument

for i in $*doecho $idonerajesh@rajesh-laptop:~$ sh hh 12 34 5 6123456

Eg)for idoecho $idone

eg)for i in $@doecho $i

8/3/2019 Unix Mod 2

18/42

18

done

Generating Values for aforLoop

1. for i in2. for i in $*

for i3. for file in *docat $filedone

4. take from shell variable // interactive program

echo Enter ur nameread name

for i in $namedo

echo $idone5. take value from the output of a command

rajesh@rajesh-laptop:~$ for i in `cat jj`doecho $idoneoutputecho

Enterurnamereadnameforecho$idone

Some times for loop is invaluable when you want to process several files in some manner.

To do a word count on several files

wc ex hh jj

for loop is invaluable at this time (wc command has a loop mechanism built in).

-m, --chars

8/3/2019 Unix Mod 2

19/42

19

print the character counts

-l, --lines

print the newline counts

-w, --words

print the word counts

To sort multiplte files

for i in $*do

sort $idone

Word-seeking program

#wordseek-searches files for words (to add comments)source=$1shift //drops the original $1 from the argument list, leaving $* equalto the rest of the argumentsfor i in cat $source` //getting words from source filedogrep $i $* // Si is the word , S* are the filenamesdonesh wordseek music ch[1-3]

CHOICE-MAKING : THE case STATEMENT

Syntax

case value inchoice1) commands;choice2) commands;....esac

Choice1, choice2 are labels that identify potential choices of action.One use for the case statement is to set up menus

Exampleecho " BASIC UNIX COMMANDS "echo "--------------------------"echo "command option"echo "__________________________"

8/3/2019 Unix Mod 2

20/42

20

echo "LIST 1"echo "DATE 2"echo "CURRENT DIRECTORY 3"echo "EXIT 4"echo "--------------------------"

echo "Enter your option.."read optioncase $option in

1)ls ;;2) who am i ;;

p|3) pwd ;;quit|4) exit ;;*) echo invalid choice ;;

esac

We can use command-line argument for case statement

echo " BASIC UNIX COMMANDS "echo "--------------------------"echo "command option"echo "__________________________"echo "LIST 1"echo "DATE 2"echo "CURRENT DIRECTORY 3"echo "EXIT 4"echo "--------------------------"echo "Enter your option.."read option

case $1 in1)ls ;;2) who am i ;;

p|3) pwd ;;quit|4) exit ;;*) echo invalid choice ;;

esac

PatternMatchingwe can use ? And [] metacharacters.

case $1 in[aeiouAEIOU]*) echo the given word starting with vowel ;;??) echo a two letter word ;;*) echo I dont know ;;esac

$ sh vowels earsthe given word starting with vowel

8/3/2019 Unix Mod 2

21/42

21

$ sh vowels EYEthe given word starting with vowel$ sh vowels ZZa two letter word$ sh vowels 1323

I dont know

Make the file execuitable and run$vowels eye

We can use case `pwd` in

using for and case together

for file in *

docase $file in*.c) echo it is a c file $file;;*.cc) echo it is a c++ file $file;;*.txt) echo it is a text file $file;;esacdone

# sh forcaseit is a c file a.cit is a c++ file a.cc

it is a text file a.txtit is a c file b.cit is a text file new file.txt

CONDITIONALLOOPING : while AND untilUNIX offers three loop structures : for, while and until.The general form of while loop is

while control command

do

commands

done

echo enter the choice //while loop like C or PASCAL iswrong cannt use < > == (conditions)

read iwhile [ $i -gt 10 ]doecho $i no greater than 10done

8/3/2019 Unix Mod 2

22/42

22

If the control command returns a zero exit status , the command between do and done areexecuted.

Word=$1 //whilegrepshift

while grep $word $1doshiftdoneecho first file not containing $word

First variable word gets the value arrahman. The shift command shfits ch1 to $1 the whilestatement become

while grep arrahman ch1do

shiftdone

If the grep command reports success, then the instructions between do and done are executed., ifit reports failure the loop is terminated.Suppose arrahman is found in ch1, then the grep reports success, and the following shiftcommand is executed , this shifts ch2 to $1 and the next loop becomes

while grep arrahman ch2doshiftdone

if the grep fails there is no arrahman in ch2 the loop terminatesthe script move onto next lineecho first file not containing $word

The Until loop

The until loop is executed until the test condition succeeds. A while loop runs until a controlcommand fails The until loop runs until a control command succeeds

until control commanddo

commandsdone

until ls | grep $1 > /dev/nulldo

sleep 30 //loop continues until the file is created.

8/3/2019 Unix Mod 2

23/42

23

doneecho file $1 is here

THEifSTATEMENTif statement is concerned with the status of commands

if control commandthen

commandsfi

first the command following the if is executed. If the command is successful (exit status 0), thenthe commands between the then and the fi are performed.

if cmp -s $1 $2 // -s suppresses the usual output only thestatus of the command (0 if the file match, 1 if they dont (fail)

thenrm $2fi

The if .....then.....else structureif control commandthen

commandselse

commandsfi

eg)if cmp -s $1 $2thenrm $2

echo the $2 was duplicate and removedelseecho the files $1 and $2 are not the samefi

If then .....elif then elseif control command 1then

commandselif control command2then

commandselse

commandsfi

8/3/2019 Unix Mod 2

24/42

24

eg)if [ $leap -eq 1 -a $date -gt 29 ]then

echo "date not valid"elif [ $leap -eq 0 -a $date -gt 28 ]

then echo "date not valid"echo "enter correct date and try"

elseecho "Date valid"

fi

THE test COMMAND$ dog=nikkit //set dog variable to nikkitest $dog=nikki // test for equality

the test command produces no o/p, but it produces a 0 exit status if the test is successful. And a 1exit status if it fails.

$? check status of test command

Eg)$dog=nikki$test $dog = nikki$echo $?0$test $dog = pluto$echo $?

1Test string1 = string2 (There is a space between two strings)

Eg)Echo what is the best operating system

if test $1 = UNIX

then

echo I agree

else

echo I never heard of $1

fi

eg) Echo i am thinking of a number between 1 and 50

read guessuntil test $guess = 33

do

echo nope guess again

read guess

done

echo well done // test

8/3/2019 Unix Mod 2

25/42

25

File checking

the test command has several options for checking the status of a file

-r file - true if the file exists and readable

-w file - true if the file exists and writable

-f file - true if the file exists and is not a directory

-d file - true if the file exists and is a directory

-s file - true if the file exists and has a size greater than zero

-x for identifying executable files

$ ls -l ex

-rw-r--r-- 1 rajesh rajesh 8 2009-03-15 20:17 ex

$ test -r ex

$ echo $?

0

$ test -x ex

$ echo $?

1

String Checkingcomparing two strings

string1 = string2 // true if the strings are samestring1 != string2 // true if the strings differ

the space are obligatorytest $ans = hoop //valid formtest $ans= hoop //invalid form

three options deal with the size of a string-n string // true if the string has nonzero lenght-z string // true if the string has zero lengthstring // true if the string is not the null string

An undefined variable , appears to be a null string$ frog=bumpy$ test $frog$ echo $?0

$ test $frag //frag not defined$ echo $?1

We can use test to make a while loop act like a foor loopwhile test $1do.....

8/3/2019 Unix Mod 2

26/42

26

shiftdone

Numerical Comparisonsthses options compare integers

n1 -eq n2 // true if the integers are equaln1 -ne n2 // true if the integers are not equaln1 -gt n2 // true if n1 is greater than n2n1 -lt n2 // true if n1 is less than n2n1 -ge n2 // true if n1 is greater than or equal to n2n1 -le n2 // true if n1 is less than or equal to n2

eg)if test $# -ne 2thenecho this pgm needs two args

$ sh numeric 1this pgm needs two args$ sh numeric 1 2

5 = 5 // true , the character string 5 is same as 55 = 05 // false , the character string '5' is different from '05'5 -eq 05 // true , 5 and 05 have the same numerical value

$ test 5 = 5$ echo $?0

$ test 5 = 05$ echo $?1

$ test 5 -eq 05$ echo $?0

Logical operations

test -rw flees is it possible ?

!-a binary and operator-o binary or operatortest ! -r film // true if the filem is not an existing, readable filetest -r film -a -w film // true if the file film is readable and writable

8/3/2019 Unix Mod 2

27/42

27

test -r film -o -w film // true if the file film is readable or writable

Using parentheses for groupingtest \( -r film -a -w film \) -o \( -r flam -a -w flam \)parentheses are shell metacharacters so use the backslash

Test with [ ]echo what is ur ageread ageif [ $age -le 6 ]thenecho child---------------------------------------------------if [ $age -gt 9 -a $age -lt 20 ]

// write a shell script to check whether the given file is exists or not

if test ! -f $1thenecho file $1 not foundelse echo file foundfi

UNIX Tools : grep, sed, tr, and awkgrep a pattern matching utilitysed a stream editortr a character translation programawk a pattern matching and processing labguage

Grepgrep stands for global regular expression printer

grep pattern filenameexamplegrep mca lmcstgrep seraches the string mca in the file lmcst and print the line containing the string

$ ls | grep aExamplesgladha

Templates

Common options for grep-v prints those lines that do not match-c prints only a count of the number of matching lines-l prints only the names of files with matching lines-n prints the line number with each output line-s prints no o/p, just return status

8/3/2019 Unix Mod 2

28/42

28

$ grep -v mca lmcstmc$ grep -c mca lmcst3

$ grep -l mca lmcstlmcst

$ cat lmcstmca is a pg coursemcamaster of computer applications - mcamc

Grep accepts multiple filenamegrep mca lmcst lmcst1 lmcst2

If the pattern contains a space or tab character in it, it is necessary toput them on in a single or double quotes.grep 'san francisco' hotelsorgrep san francisco hotels

Greps pattern-matching$ grep ar c?c1:haiarrahmanc2:arrahman oscar

grep 'm.a' lmcst // will count the space alsoa period represents any one character

eg)rajesh@rajesh-laptop:~$ cat > grepspatespotrespitesputterthe asp took

rajesh@rajesh-laptop:~$ grep 'sp.t' grepspatespotrespitesputterthe asp took

rajesh@rajesh-laptop:~$ grep 'sp..t' grep

8/3/2019 Unix Mod 2

29/42

29

sputter

Examples

h.t // matches hit, hst, h8th[ea]t // het , hat (heat is not matched)

[A-Z][^ A-Z] // two character first is a capital letter second is not a capitalletter (examples B4,Me,S!)^The // matches any line beginning with The , caret symbol is different

inside and outside the bracket\.$ // matches any line ending with periodho*t // matches ht(zero occurrences of the preceding

pattern),hot,hoot,hooot and so on

Some typical pattern-matching problem1. counting blank lines to count the number of blank lines in a file, first we use greps -coption, which counts the number of matche and Second we need a pattern to match a blank line

^$ // matches an empty line$ grep -c ^$ lmcst2when we write a file , we start the first line by hitting the space bar. In this case there will besome space character present but invisibleto find the spaces we can use *$a blank line can also contain a tab character , to look for lines containing just tabs, we can usethis pattern:*$ //matches blank line with any number of tabsIn C programs we can find the tab character by using the pattern^\t*$

we can combine these patterns to catch lines that may have blank lines,tabs or both^[ \t] * $to count the blank lines (with or without spaces or tabs) in the file text isgrep -c '^[ \t]*$' text

2. finding two patterns in proper sequenceto find all lines containing Mickey and Donald in that order, but which may have additionalcharacter in between them.

then Mickey tossed the gem to Donald

grep 'Mickey.*Donald' hero

rajesh@rajesh-laptop:~$ cat > spthe Mickey tossed the gem to DonaldMickey and Donald are friends

rajesh@rajesh-laptop:~$ grep 'Mickey.*Donald' spthe Mickey tossed the gem to Donald

8/3/2019 Unix Mod 2

30/42

8/3/2019 Unix Mod 2

31/42

31

Two other additions to egrep are the special + and ?, both are similiar to *, which stood for zero or more occurrences of the preceding character.

The + stands for 1 or more occurrences and the ? Stands for zero or one occurrence.

ho*t matches ht,hot,hoot etcho+t matches hot,hoot etcho?t matches ht and hotparentheses can be used with *, + , ?(ho)*t matches t,hot,hohot etcpattern using the egrep scheme of special characters are known asfull regualrexpressions.

CHARACTERTRANSLITERATION : tr

tr transliterates characters$cat philoI compute, therefore I am.

$ tr eiou ~#$% < philoI c$mp%t~, th~r~f$r~ I am.

tr is a pure filter, it takes input from the standard input. Small letters (i) and capital letters (I) aredifferent, above example I is not affected by I$ tr eiou EU < philoI cUmpUtE, thErEfUrE I am

Special Notations

you can indicate ranges with tr by using a hyphentr A-Z a-z //converts upper case to lower caseortr [a-z].[A-Z]

Optionstr command has three options-d uses just one list of characters, and it causes them to delete$tr -d eiou < philoI cmpt, thrfr I am-c option causes the substitutions to affect every character but those in the first list.

$ tr eiou * < philoI c*mp*t*, th*r*f*r* I am

$ tr -c eiou '*' < philo***o**u*e****e*e*o*e******$

-c option replaces all the space,punctuation with * andnew line character at the end

8/3/2019 Unix Mod 2

32/42

32

finally -s option compresses strings of repeated ouput characters at the end of the line, ie strings**** become *.$ tr -cs eiou '*' < philo*o*u*e*e*e*o*e*$

rajesh@rajesh-laptop:~$ cat > philoI compute , thereforerajesh@rajesh-laptop:~$ cat philo | tr "[a-z]" "[A-Z]" > ph1rajesh@rajesh-laptop:~$ cat ph1I COMPUTE , THEREFORE

$ tr eiou EU < philoI cUmpUtE, thErEfUrE I am

the last character from the second list is used for all the left over characters from the first list.

THE STREAMEDITOR: sed

to edit a file called philo u typeed philothe editor then creates a temporary copy of philo in the /tmp directory, all the changes made tothe copy, when we give the w command, the temporary file is copied into the original, the netresult is that the editor alters the original file.Sed is a UNIX filter, it can take input from a file, but its output is sent to the standard output, theoriginal file is not altered.The o/p from sed can be redirected to a new file if u wish to save it.Or it can be piped along to some other program as input.

What exactly does sed do?It takes a line from input, applies an editing command to it, and prints the result. Then it takes thenext line and repeats the process until the whole file has flowed through sed.Basic usage of sed issed 'editing command' filenameif no filename is given, then sed reads the standard input, this means it can receive input from apipe.If more than one filename is given, then they are processed in turn.

Editing commands for sed

it has two parts : a line specifier and an editing instructionif there are no line specifier then the command is applied to every line.There are two approaches to specifying lines.One way to specify a line is to give a line number or a range of line numbers.

3 // third line5,8 // lines 5 through 810, $ // lines 10 through the last line

8/3/2019 Unix Mod 2

33/42

33

second way to specify a line for sed is to provide a matching pattern enclosed in slashes.(patterns are the same as those used by grep)

Examples/huge/ all lines containing the string huge

/[Dd]o/ all lines containing strings Do or do/^[ \t]*$/ all blank lines.The two forms of line identification (line numbering and pattern matching) can be combined.1,/gossip/ From line 1 to the first mention of gossip! - to indicate the command is to be applied to lines that dont match the line specifier.

The editing instructions for seds substitutep printd deleteq quit

editing instructions immediately follows the line specifiersed '10q' text // quit after reading line 10sed prints each line as it is processed, this command prints the first 10 lines of a file.Sed '$p' text //prints the last linesed '/^[ \t]*$/d' text // delete all blank lines from textSed '/value/!d' text // delete every line in text not containing the string

valueThe sed substitution command (s)the general form is s/oldstring/newstringthis replaces the first occurrence of oldstring on a line with newstring.If u want to replace all occurrences on a given line we tack on a g (for global) at the end of thecommand.Sed 's/Michael/Michele/g' text

$ cat > textjohn is coming from tvmjohn and his friend john

$ sed 's/john/hary/' texthary is coming from tvmhary and his friend john

$ sed 's/john/hary/g' texthary is coming from tvmhary and his friend hary

Sed '1, /Jessup/s/Michael/Michele/g' text // replaces Michael with Michele up through thefirst line containing Jessup

8/3/2019 Unix Mod 2

34/42

34

special characters also can be used with sed

$ sed 's/john/"hary"/g' text"hary" is coming from tvm"hary" and his friend "hary"

$ sed 's/john/"john"/g' text"john" is coming from tvm"john" and his friend "john"

$ sed 's/john/"&"/g' text"john" is coming from tvm"john" and his friend "john"

$ sed 's/john/&! &!/g' textjohn! john! is coming from tvm

john! john! and his friend john! john!

Sed tags

you can enclose part of oldstring between \( and \)the enclosed part is said to be tagged.U can then refer in newstring to the tagged part as \1if u tag more than one part in old string, u can call the second tagged part \2, and so ons/ \ (john\) and \(hary\) / \2 and \1/ - replaces the phrase

john and hary with hary and john.\1 stands for john (first tagged string)\2 stands for hary (second tagged string)$ sed 's/ \(john\) is \(coming\) / \2 is \1/g' textcoming is johnfrom tvmjohn and his friend john

To change all occurrences ofoccurence and Occurence to occurence and Occurence s/ [Oo]ccurence/ [Oo]ccurence/g - this dont workthe pattern [Oo] can only be used in the left-hand pattern (oldstring)the only special symbols usable on the right side (new string) are & and \1 family. So patterns using [], . , * cant be used on the right side.S/ \ ([Oo]ccur \) ence / \ 1rence / g

$ sed 's/john/hary/g> s/hary/mary/g> s/mary/bony/g' textbony is coming from tvmbony and his friend bony-------------------------------------------------------$ sed 's/john/hary/gs/is/was/g' text

8/3/2019 Unix Mod 2

35/42

35

hary was coming from tvmhary and hwas friend hary

Options for sed

-n , -f , -e

-n option suppresses the automatic printing of each processed line,Sed -n '$p' listit print just the last line of the file list

the -f option followed by a space, then a filename, This option causessed to take its editing commands from the named filesed -f edit sermonsed to look in the file edit for commands to apply to the file sermon.

Sed -f name -f form memothis command first use the editing command from name and then from memo

the -e optionthe -e option lets you mix a command-line editing command with commands in a file.Sed -e 's/Boston/Wasco/g' -f form trav.txtBoston will replaced by Wasco then the editing command in form to be applied, and then on tothe next line.

A pattern-scanning and processing language : awk

The awk command (named after and by its creators : Aho, Weinberger, and Kernighan)awk command combines pattern matching, comparison making, numerical operations and c-like

programming features into one program.A basic command line looks likeawk awk-command filenameeach awk command has a line-specifier (a pattern) part followed by an instruction (an action).Awk lets you identify particular fields within a linea field is a squence of characters containing no blanks & separated by blanks from other fields.

Each word on this line is a separate field.Example of an awk command line.awk '/poultry/ { print $1, $3, $6}' suppliesthe pattern part is /poultry/.

The action part is enclosed in braces; it requests that the first, third and sixth fields of thematching line be printed.The $1 stands for first field, $3 stands for third fieldif there is no pattern portion the action is applied to every line of input.The following prints the first & second fields of the table supplies in reversed order.awk ' { print $2, $1}' supplies

8/3/2019 Unix Mod 2

36/42

36

If there is no action portion, the default action is to print the entire line which matches thepatternawk '/poultry/ ' suppliesmore than one command can be included by starting a new line for each command$ awk '/scandel/ { print $0}

/rumour/ { print $0} ' rag$0 stands for entire line

Eg)$ awk '/mca/ {print $0}> /john/ {print $0} ' lmcstmca hai haimcahai john mcahai john mca

eg)$ awk '/mca/ {print $1}

/john/ {print $0} ' lmcstmcamcahaihai john mca

If u have many commands, it is usually more convenient to place them in a file and the awk getits commands from the file.This is done by using the -f option.awk -f cawk ragthis command tells awk to apply the command in the cawk file to the input from the rag file.Fields and Recordsan awk program processes its input a record at a time.By default a record is just a line.The new line character is the default record separator.When ever awk encounters a newline character it starts a new record.

Within a record, the input is divided into fields.A field is a series of characters delineated by a fields separator, by default the field separator is ablank, that is a space or tab.

Examplemc |mcstBtech CSE lmcstlmcst is the second field in the first recordBtech is the first field in the second record.Awk has some built-in variables dealing with the fields and records.NR represents the current record no. NF represents the number of fields in the current record.

In the above example after reading first line NR =1 and NF = 2

8/3/2019 Unix Mod 2

37/42

37

the variable FS stands for field separator, by default this is blank (ie a space or tab character)we can reset this field separator to another value.This can be done by using -Fawk -F: '{print $1, $2}' lmcstthis prints the first two fields of the lmcst file, using a colon as a field separator for the input.

The variable RS represents the record separator

Pattern-Matching Metacharacters for awk

\ Match following character literally^ the beginning of a line$ the end of a line. any one character[list] any one character in list[^list] any one character not in list* zero or more occurrences of the preceding character

or group

+ one or more occurrences of the preceding characteror group? zero or more occurrences of the preceding character

or group(exp) is expexp1 \ exp2 means exp1 or exp2

Patterns in awk

the awk command accepts several forms of pattern-matchingFull Regular Expressionsawk recognizes the same full regular expressions that grep does.Regular Expressions and Tablesthe entire line is called $0.the ~ symbols indicate a matchthe !~ symbols indicate no matchto find all lines in which the second field matches flubbo$2 ~ /Flubbo/

$2 ~ /^flubbo$/this matches only lines in which the second field is flubbo.$2 !~ /^Noxin$/matches all lines for which the second field is not NoxinRelational Expressionsthis compare two expressionsawk command recognizes the same relational operators that C does.

$ cat > compare12 34 4412 55 23$ awk '$2 > $3' compare

8/3/2019 Unix Mod 2

38/42

38

12 55 23 // prints the line $2 greater than $3$ awk '$2 < $3' compare12 34 44 // // prints the line $2 less than $3awk also recognizes the arithmetic operators +, - , * , / and %.$ awk '$2 >=2 *$3' compare

12 55 23 // print line for which $2 greater than or equal to two times of $3

Relational operators for awk

Operator Meaning< less than greater than>= greater than or equal to== equal to!= not equal to

L

ogical operators for awk&& and|| or! not

$ awk '$2 > $1 && $3 > $1' compare12 33 4411 23 45!($2 > $1 && $3 > $1) // selects records for which the condition is not true.Rangesawk allows you to specify a range of lines.NR == 20, NR == 40 // the records 20 through 40 NR == 1, /[D]dismay/ // record 1 through the first appearance of

Dismay or dismay

Special Patternsawk has two special patterns BEGIN and ENDBEGIN is used to label actions to be done before any records are read.BEGIN {RS = :; FS = /}this command establishes the colon as the record separator and sets up slash as the fieldseparator.The END label is used to identify actions that are done after all the input has been read.

Actionsthe awk actions can print, it also has built in functions, ability to create variables , arrays,arithmetic operations and flow control statements.Printingto use print statement$ awk '{print $1}' lmcstmca

8/3/2019 Unix Mod 2

39/42

39

mcahaihai

eg)$ awk '{print $1, $2}' lmcst

mca haimcahaihai john

eg)$ awk '{print $1 + $2}' compare4534

Eg)$ awk '{print}' compare12 33 44

11 23 45

Eg)$ awk '{print $0, "total is: " , $1 + $2 + $3}' compare12 33 44 total is: 8911 23 45 total is: 79

Eg)$ awk '{printf "first val is: %6d second val is %d: total is: $%d\n", $1, $2, $1 + $2}'comparefirst val is: 12 second val is 33: total is: $45first val is: 11 second val is 23: total is: $34

Print Redirectionawk can redirect print output to files$ awk '{print $1 >> "col1"; print $2 >> "col2" }' comparethe first field in each line is appended to a file named col1the second field will go to the file named col2Variablesawk lets us create a variable, the value of a variable can be numeric or a string.We want to count the number of lines that begin with mca or with hai$ awk '$1 == "mca" || $1 == "hai" { count = count + 1 } END { print count, "lines"}' lmcst4 lines

We can use { count +=1} and {count++}+= operator adds whatever is to its right to whatever is to its left++ operator increments its variable by 1you have a table of numbers and wish to total the third column$ cat compare12 33 4411 23 45$ awk '{sum += $3}

8/3/2019 Unix Mod 2

40/42

40

> END {print sum}' compare89

Arrays$ awk '{sum[1] += $1; sum[2] += $2; sum[3] = sum[1] + sum[2]}

END {print sum[1], sum[2], sum[3]}' compare23 56 79this creates an array sum with three elements sim[1], sum[2], sum[3]the array index can be a non-null value (including a string)we can use tom and dick instead of 1 and 2to find how often the words philodox and snoach occur in a file/philodox/ { count[philodox]++}/snoach/ { count[snoach]++}END { print count[philidox], count[snoach]}

Flow Control

awk uses the if , if-else, for and while loops of CTheifstatement{if ( $1 < $2)print $1, $2elseprint $2, $1 }$ awk '{ if ($1 < $2)print $1, $2elseprint $2, $1 }' compare12 3411 552 3

Eg)$ awk '{ if ($1 < $2)print $1, $2}' compare12 34

2 3

The forstatementthe for statement has this form :for (initialize;test; update)

commandthe initialize expression is performed once, when the loop starts up.Then the test expression is examined, if it is true, the command is executed, if it is false the loopterminated.Each time loop is executed the update statement gets executed.

8/3/2019 Unix Mod 2

41/42

41

A non-C form of for loop

for ( i in array)

command

array is an array name , the command is performed for each array

element.

{ count[$1]++}

END { for (i in count)

print i, length(i), count[i] }

$ awk -f awkex lmcst

hai 3 2

mca 3 2

lmcst 5 1lm 2 1

awkex

$ cat lmcstmca hai haimca

haihai john mca

lmlmcst

The while loop

the while loop has the form:

while ( test )command // the command get executed as long as the test is true.

Prgm this prints out the fields in a line until it hits the end of the line orthe word private.

cat > samplejohns hobby is swimmingben's hobbies are fishingprivate sewing toe-pickingjillians hobbies are volleyball

{ field = 1while ( field

8/3/2019 Unix Mod 2

42/42

42

awk -f awkex sample

johns hobby is swimmingben's hobbies are fishingjillians hobbies are volleyball

the program uses the variable field to keep track of the field number.The while loop ends when it reaches the last field number (field == NF)or when the value of the field ($field) is private.

Jumps : continue, break, next and exit

break causes the loop to break off and terminatecontinue causes loop to skip to the beginning of the next iterationnext causes awk to skip to the next record, restarting the action there.exit takes you to the end of the input, but does not skip over an END statement.

Sprintf

sprintf function lets you format various expressions into a string.It uses the formatting conventions of C's printf()Ex:$1 has the value Dan and $3 has the value 65 thenlabel = sprintf(%s of the %d kg class, $1,$3)Results in the variable label having the valueDan of the 65 kg class