Shell Script Programming 2 - Santa Monica College
Transcript of Shell Script Programming 2 - Santa Monica College
1
© David Morgan 2011-13
Shell Script Programming 2Shell Script Programming 2
David Morgan
© David Morgan 2011-13
Useful capabilitiesUseful capabilities� parameter processing
� validation– usage checking
– user input
� custom functions
� filenames – decomposition, tempnames, random names
� action hinged on compound condition evaluation (ifless if)
� sourcing– execute commands from a file in current shell
– “#include” functions in such a file, by reference
� debugging
2
© David Morgan 2011-13
Supplemental supporting utilitiesSupplemental supporting utilities
� find – generate qualified file name lists
� xargs – build command lines from stdin
� sed – dynamic, programatic, in-place editing
� awk – text manipulation
� bc – arbitrary precision math
� capabilities lacking or weak in shell
� often utilized in scripts
© David Morgan 2011-13
Special shell variablesSpecial shell variables
� $0 – script name from command line
� $1, $2, etc – command line parameters
� $* - command line parameters collectively
� $$ - process ID (PID) of current process
� $? – exit status of last command
3
© David Morgan 2011-13
shift shift –– positional parameter promoterpositional parameter promoter
© David Morgan 2011-13
getoptsgetopts
� getopts vs getopt
– a shell built-in
– a stand-alone binary
– not identical
� processes command-line options
4
© David Morgan 2011-13
CommandCommand--line formatline format
command [ - options ] [ arguments ]
ls -l m*
tokens
© David Morgan 2011-13
Some options take a value,Some options take a value,
some commands take argumentssome commands take arguments
myprog [ -x value ] [-y] [ aaa ] [ bbb ] [ ccc ]
command [ - options ] [ arguments ]
an option (-d ) taking a value ( : )
a command ( cut ) taking an argument ( /etc/passwd )
5
© David Morgan 2011-13
““getoptsgetopts”” commandcommand
� resides in a script
� knows the script’s allowable options
� parses the command line that called the script
� executed successively, finds/returns the options in the line one at a time
� used in loops to read/detect them one after the other
� detect phase normally followed by programmed response phase
© David Morgan 2011-13
““getoptsgetopts”” documentationdocumentation (from (from ““man bashman bash””))
getopts optstring name [args]
getopts is used by shell procedures to parse positional parameters. optstring contains the option characters to be
recognized; if a character is followed by a colon, the option is expected to have an argument, which should be separated
from it by white space. The colon and question mark characters may not be used as option characters. Each time it is
invoked, getopts places the next option in the shell variable name, ini- tializing name if it does not exist, and the index of
the next argument to be processed into the variable OPTIND. OPTIND is initialized to 1 each time the shell or a shell
script is invoked. When an option requires an argument, getopts places that argument into the variable OPTARG. The
shell does not reset OPTIND automatically; it must be manually reset between multiple calls to getopts within the same
shell invocation if a new set of parameters is to be used.
When the end of options is encountered, getopts exits with a return value greater than zero. OPTIND is set to the index of
the first non-option argument, and name is set to ?.
getopts normally parses the positional parameters, but if more arguments are given in args, getopts parses those instead.
getopts can report errors in two ways. If the first character of optstring is a colon, silent error reporting is used. In
normal operation diagnostic messages are printed when invalid options or missing option arguments are encountered. If
the variable OPTERR is set to 0, no error messages will be displayed, even if the first character of optstring is not a colon.
If an invalid option is seen, getopts places ? into name and, if not silent, prints an error message and unsets OPTARG. If
getopts is silent, the option character found is placed in OPTARG and no diagnostic message is printed.
If a required argument is not found, and getopts is not silent, a question mark (?) is placed in name, OPTARG is unset,
and a diagnostic message is printed. If getopts is silent, then a colon (:) is placed in name and OPTARG is set to the option
character found.
getopts returns true if an option, specified or unspecified, is found. It returns false if the end of options is encountered or
an error occurs.
6
© David Morgan 2011-13
functionsfunctions
� install runnable code unit in memory
� under a callable name
“A shell function… stores a series of commands for later execution.
When the name of a shell function is used as a simple command name,
the list of commands associated with that function name is executed.
Functions are executed in the context of the current shell; no new process
is created to interpret them (contrast this with the execution of a shell script).
bash man page
© David Morgan 2011-13
functionsfunctions
7
© David Morgan 2011-13
filenamesfilenames
� decomposition
– basename
– dirname
� temporary names
� random names
© David Morgan 2011-13
Filename decompositionFilename decomposition
8
© David Morgan 2011-13
Generating working/temp filenames Generating working/temp filenames
““datedate”” basedbased
© David Morgan 2011-13
Generating working/temp filenamesGenerating working/temp filenames
RANDOM variable, RANDOM variable, mktempmktemp commandcommand
in /tmp
in some other
directory
9
© David Morgan 2011-13
ifif--less ifless if
if [ . . . ]; then
xxx
fi
[ . . . ] && xxx
� seen often
� understood seldom
� a (too?) slick way to do conditionals
© David Morgan 2011-13
ifif--less ifless if
Some examples found in rc.sysinit
AND-based righthand action only if lefthand condition true:
[ -x /usr/bin/plymouth ] && PLYMOUTH=yes
[ ! -f "$file" ] && continue
[ "$READONLY" = "yes" ] && return 1
[ -f /var/log/dmesg ] && mv -f /var/log/dmesg /var/log/dmesg.old
OR-based righthand action only if lefthand condition not true:
[ -f /.autorelabel ] || touch /.autorelabel
10
© David Morgan 2011-13
sourcesource
� execute code from a file in current shell
� often used on a file containing functions
� effectively similar to c language #include
“source filename [arguments]
Read and execute commands from filename in the current shell environment”
bash man page
© David Morgan 2011-13
findfind
� searches for files in a directory tree
� described by an expression
� expression consists of elements
– options
– tests
– actions
� each element returns boolean result
� find evaluates as many elements of its expression as needed to know expression’s outcome
11
© David Morgan 2011-13
Most common useMost common use
for <all files in a set of files>
if <something about the file> do <something with the file>
next
but the operation details are more complex than that
© David Morgan 2011-13
find examplefind example
find . -maxdepth 1 -size +1000000c -print
expression
a testan option an action
find files 1) in the current directory (no subdirectory search)
2) bigger than a million bytes
3) and print their names
12
© David Morgan 2011-13
Some example elementsSome example elements
find . options* tests actions
maxdepth name print
mount atime +n ls
etc size +n exec
executable ok
type etc
empty
false
etc
* find’s options, not shell command options
© David Morgan 2011-13
Some example elementsSome example elements
find . options tests actions
what it returns: true always true or false true or false
what it does: influence nothing their particular
find behavior action
13
© David Morgan 2011-13
Operational logicOperational logic
"[evaluates] the given expression from left to right...
until the outcome is known (the left hand side is false for
and operations, true for or), at which point find moves on
to the next file name.“
- “find” man page
© David Morgan 2011-13
Operational logicOperational logic
because -name “A*” is false for B* files
because printing happens before -name “A*” evaluation
from the 2nd -print
(2nd print doesn’t happen for B* files)
14
© David Morgan 2011-13
exec action exec action ––
arbitrary response for qualifying filesarbitrary response for qualifying files
a “finder” script command:
find . -type f –exec grep –l “$1” {} \;
print names of all files in current directory containing
a given string
� needs to be terminated with ;
� uses {} as placeholder for current file
� need to escape these from shell
© David Morgan 2011-13
String manipulationString manipulation
15
© David Morgan 2011-13
sedsed –– stream editorstream editor
� used for search and replace
� filters standard input to standard output
© David Morgan 2011-13
sedsed –– stream editorstream editor
prints 2 lines replace 1st o with *, each line
all o’s
replace any h or e
replace any vowel
replace any letter or numeral
replace anything neither h nor e
replace anything neither letter nor numeral
16
© David Morgan 2011-13
awkawk ((oror gawk)gawk)
� a pattern scanning and processing language
� better text processing facilities than shell’s
� often used in scripts to break text into fields
© David Morgan 2011-13
gawkgawk
gawk ‘ <program> ’ <input files>
{ <pattern> <action> }
gawk processes all lines in the input, comparing each to the pattern
and, for those that match, performing the action
17
© David Morgan 2011-13
gawkgawk
the program
the pattern the action
pattern only: default action is to print whole line
action only: default pattern selects all lines
© David Morgan 2011-13
bcbc
� arbitrary precision calculator– strong computation features
– operates internally in decimal
� interactive– offers command prompt
� programmable– weak programming features
– complements shell (weak comp, strong prog)
� use with shell– pipe commands to bc from shell
– result returns to shell from bc
18
© David Morgan 2011-13
bcbcexpressions
high integer precision
hig
h in
teg
er
pre
cis
ion
hig
h in
teg
er
pre
cis
ion
© David Morgan 2011-13
bcbc how shell does it
(no decimals)
how bc does it
high decimal precision
decimal periodicity = 1
decimal periodicity = 2
decimal periodicity = 6
hig
h d
ecim
al p
recis
ion
hig
h d
ecim
al p
recis
ion
19
© David Morgan 2011-13
bcbc7, 97, 983 and other primes have special properties (see “cyclic numbers”)
periodicity = 7 – 1 = 6
periodicity = 97 – 1 = 96
periodicity = 983 – 1 = 982
hig
h in
teg
er
pre
cis
ion
hig
h in
teg
er
pre
cis
ion
© David Morgan 2011-13
bcbc -- functionsfunctions
enter function code at bc prompt
utilize it thereafter
20
© David Morgan 2011-13
bcbc –– function function ““librarylibrary”” filesfiles
mod( ) – no such function
but here’s a file that defines
such a function (plus others)
file’s contained functions are now available
© David Morgan 2011-13
bcbc –– code from standard inputcode from standard input
things you do in bcthings you do in bc
you can have done by bc in the shell
21
© David Morgan 2011-13
quick RSA tutorialquick RSA tutorial
� with suitably chosen values for e, n, and d*
� an integer m encrypts to: c= me mod n
� and can be recovered by: cd mod n
� a set of suitable values (ones that work):
e – 15941n – 48863
d – 17741
* pair {e,n} is called “public key” and pair {d,n} “private key”
© David Morgan 2011-13
encrypt and recover 1001 in encrypt and recover 1001 in bcbc
22
© David Morgan 2011-13
encrypt and recover 1001 in shell scriptencrypt and recover 1001 in shell scriptuse command substitution where the command is use command substitution where the command is bcbc
© David Morgan 2011-13
use here doc to drop in entire use here doc to drop in entire bcbc progprog
any bc code that returns a
desired calculated value
23
© David Morgan 2011-13
Looping helpersLooping helpers
© David Morgan 2011-13
debugging scriptsdebugging scripts
� sectional debugging
– turn on with “set –x”, off with “set +x” within
script
� whole-script debugging
– “set –x” at command line before running
script
– shabang line in script: #!/bin/bash -x
24
© David Morgan 2011-13
ResourcesResources� Advanced Bash-Scripting Guide, An in-depth exploration of the art of shell scripting;
Mendel Cooper
� GNU manuals:find: http://www.gnu.org/software/findutils/manual/find.htmlgrep: http://www.gnu.org/software/grep/manual/sed: http://www.gnu.org/software/sed/manual/sed.htmlbc: http://www.gnu.org/software/bc/manual/bc.html
� Classic Shell Scripting Arnold Robbins and Nelson A. F. Beebe, O'Reilly & Associates, 2005
� UNIX Shells by Example Ellie Quigley; Prentice Hall, 4th edition, 2005
� Learning the bash Shell Cameron Newham and Bill Rosenblatt, O'Reily & Associates, 3rd edition, 2005
� Introducing Regular Expressions Michael Fitzgerald, O'Reilly & Associates, 2012
� Mastering Regular Expressions Jeffrey E. F. Friedl, O'Reilly & Associates, 3rd edition, 2006
� Sed and Awk Dale Dougherty, O'Reilly & Associates, 1997
� The AWK Programming Language, Alfred Aho, Brian Kernighan, Peter Weinberger, Addison-Wesly Publishing Company, 1988