New Shell Programming - LISUG Programming.pdf · 2018. 10. 29. · •Unlike CL there are a number...

1© 2018 Rogue Wave Software, Inc. All Rights Reserved. 1

Shell Programming

Erwin Earley

([email protected])

Sr. Professional Services Consultant@erwinephp@RougeWaveInc@Zend


Agenda

• What is a Shell?

• Working with the shell

• Programming Constructs


What is a Shell?• The command line used on “Unix™” systems (as well as Unix-like systems)

• Like CL it can be used interactively, or run as a program

• Like CL most commands are actually programs that get called– There are some “built in” commands

• Unlike CL there are a number of varieties of shell– sh bourne shell

– csh c shell

– ksh korn shell

– bash bourne again shell

– qsh Q shell

• There are some similarities and some differences

• Most of the discussion here is not operating system specific– Will work on AIX, Linux, QSH in OS/400, other nasty Unix variants, etc


Why do we care about the shell?

• All system configuration operations can be done through the shell – often more quickly then through a GUI

• It may be the only environment available in the case of a system crash

• Shell scripts can automate most routine tasks such as backups, scheduled emails, etc.

• GUI can be used for a great amount of admin activities

– However, the shell tends to be a comfort zone providing ability to fix things in case something goes wrong


Different Types of Shells• A number of shells are available each providing function/usability customized to a

particular type of user:

• Popular shells include:– BASH (Bourne Again Shell)– PDKsh (Public Domain Korn Shell)– csh (C shell)– mc (Midnight Commander)– QSHELL (PASE shell)– ksh (Korn shell, default on AIX)

• Difference tends to be in scripting capabilities and user interface– Items such as command recall and file name completion are typically different


Login Shell vs. Non-Login Shell • Login Shell: Shell executed when user logs in to the system.

– Typically performs general setup – initializes the terminal, sets some variables, etc.

• Non-login shell: Either subshells (started from the login shell), shells started by the GUI desktop, or disconnected shells started by a command

– Remember that shell is simply another command on the system

• Login shells unlike Non-login shells read a series of configuration files:

Login Shell:

On login

if "/etc/profile”" source it

if "~/.bash_profile" exists, source it

else if "~/.bash_login" exists, source it

if "~/.profile" exists, source it

On exit

if "~/.bash_logout" exists, source it

Non Login Shell:

On startup

if "~/.bashrc" exists, source it


Starting with bash

• bash stands for Bourne Again Shell

– Developed by Brian Fox in 1987

– One of the most popular shells available in Linux

• Bash incorporates features of the Korn and C shell (ksh and csh)

• Bash configuration files:

/bin/bash – Bash executable

/etc/profile – System wide initialization file for login shells

~/.bash_profile – Personal initialization file for login shells

~/.bashrc – Personal per-interactive-shell startup file

~/.bash_logout – Login shell clean file that executes when shell exits

Available in PASE, more on that coming up!


A little more on BASH

• Default Linux shell – This can be changed in a variety of ways

• /etc/profile – login shell• $HOME/.profile • Magic string within the script (more on this in a moment)

• Very powerful as a command line shell– Recall previous commands– Command and file completion with the key

• Many programming features– Loops and conditionals


Shell Environment• The shell is an environment where commands can be entered and the Operations

system can respond to them

• A key concept to the environment is environment variables

– There are a large number of environment variables

– HISTFILE: points to file containing the shell history, defaults to ~/.bash_history

– HISTFILESIZE: how man last commands you wish to have in history

– HOME: points to your home directory– PATH: set of directories to search when trying to execute a command

– PS1: Prompt variable

– USER: usernameNOTE: All of these environment variables are available when running bash in the PASE environment.


Shell metacharactersSymbol Meaning> Output redirection>> Output redirection (append)< Input redirection* File substitution wildcard; zero or more characters? File substitution wildcard; one character[ ] File substitution wildcard; any character between brackets`cmd` Command substitution$(cmd) Command substitution| The pipe (connect output of command on right to input on

command on left); Command Sequence|| OR conditional execution&& AND conditional execution


Shell metacharacters (continued)

Symbol Meaning( ) Group commands

& Run command in the background

# Comment

$ Expand the value of a variable

\ Prevent or escape interpretation of the next character


Useful Shell Constructs

• Arrow Up & Down: Scroll through recent commands used

• &&: command is only executed if preceding command was successful:command1 && command2

• alias: sets a command alias or prints defined aliasesalias wrklnk=ls

• bg[jobid]: Resumes the suspended job in the background

• cd: changes current directory to directory indicatedcd /home


Useful Shell Constructs (continued)• echo: Outputs the arguments

echo “hello world”

• find [path][expression]: searches the directory indicating looking for files that match expression:

find / -name passwd –print

• pwd – Prints the absolute pathname of the current working directory

• unalias – Removes an alias

• history – displays command history with line numbers

• umask – User file creation mask

• logout: exits the shell environment

• exit [n]: exits shell environment with exit status n


More information on bash

• Just enter the command

– $ man bash

• Manual page for bash is about 5,000 lines!!

NOTE: the manual pages are not yet available in the PASE environment.


"bash"ing PASE – yes I went there ;-)• The bash shell is available for PASE

– It is part of the RPM pile

• Step 1: Install the RPM pile bootstrap

• Step 2: Install bash

yum install bash

• Step 3: Create a home directory:

mkdir /home/

• Step 4: Create a .profile

• Step 5: Install nano

yum install nano

PATH=/QOpenSys/pkgs/bin:$PATHexport PATH

bash


qsh

• A version of a shell for System i

• Has many of the same features as bash

• RTFM

This is an excellent and comprehensive book!


Programming Constructs


The First Shell Program#!/bin/bash echo Hello World

• This is called a shell script• The first line "#!/bin/bash" is a magic string• When a file is loaded if the first two characters are “#!” (called shebang) then

the remainder of the line is used as the program to run the file• "#!/home/erwin/my_nice_program" is a valid shell script• echo is a built-in command that just displays the rest of the line

NOTE: on IBM i the path to bash is /QOpenSys/pkgs/bin/bashHowever, symbolic links have been created to /bin/bash and /usr/bin/bashThis means that many existing scripts can be brought over from Linux environments and run unchanged in PASE under the bash shell


Creating a shell program

• Use a text editor

–Traditional Unix editors include vi and emacs

–More user friendly editors include “nano”

• On IBM i

– Install and use nano from a PASE session


Writing Your Shell ScriptUsing nano


Running Your Shell Script

• We have a "test.sh" program in our directory• Lets try and run it• Doesn’t work! Why not!?!?

earley@testbed:~$ ls –l test.sh-rw-r—r– 1 earley earley 32 2006-09-11 08:54 test.sh

earley@testbed:~$ test.sh-bash: test.sh: command not found

earley@testbed:~$



• The program is not in the “path”– The current directory is not in the path

• Can run a shell script by typing “.”–Means load the file into the current shell

earley@testbed:~$ echo $PATH/home/earley/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11

earley@testbed:~$ . test.shHello World

earley@testbed:~$



earley@testbed:~$ mv test.sh ~/bin/earley@testbed:~$ echo $PATH/home/earley/bin:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/bin/X11

earley@testbed:~$ test.sh-bash: /home/earley/bin/test.sh: Permission denied

earley@testbed:~$

• Move the shell script into a directory in the path

• Try and execute it again

• Now we get “Permission denied”! Why!?!?


Running Your Shell Scriptearley@testbed:~$ ls –l ~/bin/test.sh-rw-r--r-- 1 earley earley 32 2006-09-11 08:54 /home/earley/bin/test.sh

earley@testbed:~$ chmod u+x ~/bin/test.sh

earley@testbed:~$ ls –l ~/bin/test.sh

-rwx--r-- 1 earley earley 32 2006-09-11 08:54 /home/earley/bin/test.sh

earley@testbed:~$ test.shHello World

earley@testbed:~$

• The file is not marked with "executable" permissions

• "chmod u+x …" says "make it executable by the user who owns it"

• NOW it works


Running a shell script in QSH



Keep in mind that you could also simply ssh into your IBM i system (assuming the sshdaemon is running) and run your scripts

Can also get into qsh via 'STRQSH'


When good scripts go bad

• If you getCommand not found

the path to the shell is probably wrong

• Another error occurs if you move a file from a Windows machine to Linux and the line endings get messed up

– If this happens, you can use the following command:tr –d ‘\r’ < erwin.sh > erwin2.shwe will understand that line more later


Input and Output• *nix programs start with three open files

– Input (called stdin) (#0)

– Output (called stdout) (#1)

– Error output (called stderr) (#2)

• Our shell program uses the "echo" command to write to stdout

• We can redirect the output to go to a file by using ">"

ls –l > output.txt

This will take the output of the "ls –l" command and write it into a file called "output.txt"

STDOUT (1)program

STDIN (0)

STDERR (2)

STDIN (0)


Input and Output

• You can also specify which output goes to a file

ls –l 1> output.txtthis is the same as before.

• Redirecting only error output

grep fred * 2> grep.err

this will redirect only the error output to the file grep.err


Input and Output

• Use the "


Common *nix commands

• Unix is built around the idea of lots of little programs that all do one thing well– lslists files

– sed an editor

– grep a searcher

– cat a file outputer

– find a file locator

– sort sorts files

– tr translates characters

– ps list processes

– seq print a sequence of numbers

– … a thousand others

• Help for all these is in the “man” (for “manual”) command– e.g. man sed

• Shell programs generally involve stringing lots of these together– Which leads us to “pipes”

All of these are available in the

IFS and can be run from an IBM

i shell (i.e., qsh, ssh session)

Currently 230 available in

PASE

Not currently available in

PASE


Pipes

• Standard output (STDOUT) of one command/program is used as the standard input (STDIN) for the next command/program

ps –x | grep java

• List processes, search for any involving java

• Only the 'STDOUT' from the last command is actually output to the screen (unless re-directed)

– NOTE: Any output to STDERR by the commands will be output to the screen – again unless re-directed.

STDIN (0)STDOUT (1)

NOTE: pipes are not limited to two commands… any number

of pipes can be used to build a pipeline:

cmd1 | cmd2 | cmd3 | …. | cmdx


Variables• Bash (both interactively and in programs) has variables• Variables do not need to be declared, and don’t have types• Variable assignment occurs via a reference to the variable followed by an equal

sign and the value to assign. NOTE: There is no spacing:STR="Hello World!"

• Variable reference occurs by preceding the variable with an dollar sign:echo $STR

#!/bin/bashSTR="Hello World!"echo $STR


Special Variables

Variable Description/Usage$1 The first argument/parameter to a program/command, $2 is

the second argument, etc

$* All of the arguments to a program/command represented as a single string.

$@ All of the arguments to a program/command represented as a set of strings.

$# The number of arguments/parameters that the program/command was called with. Useful for testing that the program/command was called correctly as well as looping through the arguments/parameters

$? The exit status of the last program/command.

$$ The process id of the current process.


Command substitution (sometimes called Command variables)

• Command substitution uses the output of a command to replace the command itself.

– The shell performs the expansion by executing the command and replacing the command substitution with the standard output of the command.

• $(command) can be used like a variable, but executes the command

– FILES=$(ls)– Sets the variable $FILES to the output of the "ls" command


Math – not what you would expect

• The 'let' built-in command is used to evaluate arithmetic expressions:

• Consider the following

COUNTER=0COUNTER=$COUNTER+1echo $COUNTER # This prints “0+1”

COUNTER=0let COUNTER=COUNTER+1echo $COUNTER # This prints 1

NOTE: more then just simple addition can be performed with the 'let' command.


Doing Math – Another Approach

• Another choice for doing math is $(( expression ))

counter=1counter=$((counter+1))echo $counter //output will be 2


Interacting with a User – Enter Your Name

#!/bin/bashecho –n "Enter Your Name: " read NAMEecho "Your name is $NAME"exit 0

Shell script gets a name from user input and echoes it to the screen

NOTE: The '-n' option to the echo command indicates that the should be surprised – this results in the prompt for input to be immediately after the echoed output


The select command

• Provides a menu based on a word list

• Enables the user to enter a number instead of a word

– The user enters the sequence number corresponding to the word

#!/bin/bashselect var in steak chicken porkdo

echo "I like eating $var"done

1) Steak2) Chicken3) Pork#? 2I like eating chicken

NOTE: The prompt for the user input can be controlled with the PS3 environment variable:PS3="select entry: "Will replace "#?" With "select entry: "


if Statements• Standard “if/then/else”• Note the weird syntax involving [ … ] and ;

– The [ ] indicates a test• In fact there is a 'test' command that can be used in place of [ ]

– Space following the [ and preceding the ] are required– The ';' is used to separate two commands

• In this case it allows us to include the 'then' on the same line as the 'if'

#!/bin/bash if [ "foo" = "foo" ] ; then

echo expression evaluated as true else

echo expression evaluated as false fi

Not a typo


Conditionals [ … ]

• [ ] is a synonym for the "test" command– "man test" gives the options

• [ -z "$x" ] the length of the string is zero• [ STRING1 = STRING2 ] the strings are equal• [ STRING1 != STRING2 ] the strings are not equal• [ INTEGER1 -eq INTEGER2 ] INTEGER1 is equal to INTEGER2• [ INTEGER1 -ge INTEGER2 ] INTEGER1 is greater than or equal to INTEGER2• [ INTEGER1 -gt INTEGER2 ] INTEGER1 is greater than INTEGER2• [ INTEGER1 -le INTEGER2 ] INTEGER1 is less than or equal to INTEGER2• [ INTEGER1 -lt INTEGER2 ] INTEGER1 is less than INTEGER2• [ INTEGER1 -ne INTEGER2 ] INTEGER1 is not equal to INTEGER2

• [ -d FILE ] FILE exists and is a directory• [ -e FILE ] FILE exists• [ -f FILE ] FILE exists and is a regular file


if statements and variables

• The previous example makes more sense with a variable:

#!/bin/bash if [ "$myvar" = "foo" ]; then

echo expression evaluated as true else

echo expression evaluated as false fi


Example Script

#!/bin/bash#Name: Copy Fileecho "Enter a Filename to copy:"read SRCecho "Enter the filename to copy to:"read DEST

if cp $SRC $DEST 2>/dev/nullthen

echo "Task Complete"else

echo "Copy Failed!" > &2exit 1

fi

exit 0

Simple file copy script with a if/then/else loop


Loops

• for: very different from other programming languages. Loops over a series of 'words' within a string.

• while: loops while a condition is true

• until: almost identical to the while loop, except that the code is executed while the control expression evaluates to false.

• case: Alternative to if/then/else statements

#!/bin/bash for i in $( ls ); do

echo item: $i done


Loop Examples

#!/bin/bashfor i in `seq 1 10`; do

echo $idone

#!/bin/bash COUNTER=0while [ $COUNTER -lt 10 ]; do

echo The counter is $COUNTERlet COUNTER=$COUNTER+1

done

Note: The use of accent marks in the above example is not a typo. The shell

will execute the command and replace it with the output. The statement

could have been represented as:

for i in "1 2 3 4 5 6 7 8 9 10" ; do

These two scripts will NOT

output the same sequence

of numbers – what's the

difference?


case Statement

The case statement can be used to replace nested if/then/else statements

case EXRESSIOON in pattern1)

statements ;; pattern2)

statements ;; ...

esac

• NOTES:– EXPRESSION is expanded and checks are made against each pattern (in order)– When a match is found all associated statements are executed (execution

terminates with ;;)– After first match, case terminates – exit status is status of last command executed– If no match, exit status of case is zero

Not a typo


case statement – simple example#!/bin/bash echo -n "Do you agree with this? [yes or no]: " read yno

case $yno in [yY] | [yY][Ee][Ss] )

echo "Agreed" ;;

[nN] | [n|N][O|o] ) echo "Not agreed, you can't proceed with the installation"; exit 1 ;;

*) echo "Invalid input"

;; esac


Functions

• A shell function is similar to a shell script in that it:

– Stores a series of commands for later execution

– Functions are stored in memory

– The shell executes a shell function in the same shell that called it

• Functions can be defined:

– In a user's .profile

– In a script

– On the command line

• The built-in function 'unset' can be used to remove a function

• Functions must be defined before they can be referenced

– Functions are usually placed at the begging of the script


Functions - example

#!/bin/bash function quit {

exit}

function hello {echo Hello!

}

helloquitecho foo Never gets executed, why?


Functions with arguments

Arguments use the same $1, $2 syntax as the main shell script

#!/bin/bash function quit {

exit} function mysay {

echo $1 } mysay Hellomysay Worldquitecho foo

HelloWorldOutput


Putting text inline; or "here" documents

• Can redirect input from inside a shell script use the "


Setting exit status• Shell scripts return a status value

exit 0…exit -1

• You can also test the status of a command you call using the special $? variable or by using the command in an if statement directly

#!/bin/bash cp $1 /tmp/backup/

if [ $? == 0 ]then echo "Copy of $1 succeeded"else echo "Copy of $1 failed"

fi


Using exit status

#!/bin/bash

if cp $1 /tmp/backup/then echo "Copy of $1 succeeded"else echo "Copy of $1 failed"

fi


A sample backup script#!/bin/bash if [ -z "$1" ]; then

echo usage: $0 directoryexit

fi

SRCD=$1TGTD="/var/backups/"OF=home-$(date +%Y%m%d).tgztar -cZf $TGTD$OF $SRCD


How did I get back on my i?

• Currently in beta, but…. Check out:

– https://www.ibm.com/developerworks/community/wikis/home?lang=en#!/wiki/IBM%20i%20Technology%20Updates/page/RPMs%20-

%20Getting%20Started

https://www.ibm.com/developerworks/community/wikis/home?lang=en


More information

• qsh

–Read the book

• bash

–man bash

–http://bashprogramming.com

• Other useful links

–http://www.linuxcommand.org/lc3_writing_shell_scripts.php

http://bashprogramming.com/http://www.linuxcommand.org/lc3_writing_shell_scripts.php

New Shell Programming - LISUG Programming.pdf · 2018. 10. 29. · •Unlike CL there are a number...

Documents

Transcript of New Shell Programming - LISUG Programming.pdf · 2018. 10. 29. · •Unlike CL there are a number...