Tips and Tricks in Unix Shell scripting DB2 Database ...
Transcript of Tips and Tricks in Unix Shell scripting DB2 Database ...
Tips and Tricks in Unix Shell scripting DB2 Database Administration
Speaker: James SobieskiFourth Millennium Technologies
Session Code: D14 -May 26, 2016 8:00 AM – 9 AM| Platform: DB2 for Linux, UNIX and Windows
Photo by Steve from Austin, TX, USA
Agenda
• Scripting Uses
• Template Scripts
• Building Shell Functions
• Common Unix Commands used in scripts
• Awk usage
• Examples
2
Shell Scripting Uses
• Automating DBA Activities
• Scheduling Complex activities via Cron
• Developing a library of “How I Did Something” scripts
• If you are going to do something 2 or more times, script it!
• General Advice• Ensure Scripts are Documented
• Be Consistent – this encourages reuse
• Write re-usable code snippets
• “Borrow generously” from others’ scripts
• This presentation will deal with mainly Korn Shell techniques
• We will review many powerful Unix commands used in shells
3
Templates Script
• Recommend that you create a template script
• Include• Heading Comments
• Name, Description, Usage, Author, Create Date, Mod Date + Description
• Standard Functions
• Help or Usage
• Command Line Parsing
• Exit Cleanup
• DB2 Command post processing
• Common Debugging Code
• Standard Variables containing names of directories for TEMP files, outputs, scripts, etc
• Common scripting structures
4
Temporary Files
• Many scripts use temporary files
• Recommend putting filenames in a variable
• Include the PROGRAM name in the filename
• Include some clue to its use in the filename
• Use “$$” to include the scripts’ Process Id in the file name
TEMP_SQL_OUT=“$TEMP_DIR/$PROGNAME.sql_outfile.$$”
db2 –tvf $MySqlFile >$TEMP_SQL_OUT
RC=$?
Run_SQL_Post_Processing_Function $TEMP_SQL_OUT
rm $TEMP_SQL_OUT
5
Common Shell Variables :
• $0 - the program name, fully qualified
• $* - all the arguments passed on the invoking scripts
• $# - the number of arguments provided on the command line
• $n - “n” is an integer – each successive arg on command line
• $$ - the variable containing the Process Id of the script
6
Common Shell Variables :
• When referencing variables, one can • Test if variable is null and assign a default value. Use the :- construct
DBNAME=${1:-”Sample”}
• Test if the variable is null and print an error message and exitDBNAME=${1?”$usage”}
• Invoking a script with a dot “.” in front of it, runs the script in the caller’s shell. Note that all variables and changes made in the script remain in effect when it returns control. ./.profile
7
Building a Shell Script Funtion:
• Functions have two declaration syntaxes:• function <funcname> {
stmts}
• <funcname> ( parameter_vars) {stmts}
• Both allow parameters passed in• Parameters are provided the same way a script is provided
parms• Variables are global – available to calling script• Local variables can be declared
• <xxx>
• Inside function, $ variables are local
8
A Usage Function:
• Functions need to be defined prior to invoking them
• Usage (){echo "USAGE: ${0} -I instancename" echo "Example: ${0} -I db2lsd" exit 1 }
if [ $# != 2 ]
then Usage
fi
9
Typical Template Variables:
• Variables in many Template Scripts
# Get start time in db2 format
START_TIME=$( date '+%Y-%-m-%d-%:H:%M:%S )
# Scripts, Temp and Output directories: SCRIPT_DIR=/dbamaint/scripts
TEMP_DIR=/TMP/script_temp
OUTPUTS_DIR=$HOME/work/outputs
10
A Usage Function:
• Functions need to be defined prior to invoking them
• Usage (){echo "USAGE: ${0} -I instancename" echo "Example: ${0} -I db2lsd" exit 1 }
if [ $# != 2 ]
then Usage
fi
11
Processing Command Line Options:
• GETOPTS is a function that will search the command line for a character preceded by a minus or plus and allow you to process the next argument following that character:
while getopts i: c
do
case $c in
i) instnme=$OPTARG;;
\?) Usage
exit;;
esac
done
shift `expr $OPTIND - 1`
12
DB2 CLP Command
• The db2 command, a.k.a, Command Line Processor is powerful
• Commonly used options
• adfsf
13
Option Descriptiont Terminators in SQL/Statements so they can be multi-linedf Filename follows it - reads SQL/Statements from the file
m Returns the number of rows modified by the last SQL stmta Returns the SQLCA areas Stops the execution if an Error or Severe Error occursp Toggles prompting for input +p turns off promptc Toggles Autocommit. -c Auto commits after each Stmt. +c does notv Verbose - echos statements to std out as they are executed
The DB2 SQLCA
• When –a is provided on the db2 command the SQLCA is displayed after every statement executed
• Fields in the SQLCA include
14
Option DescriptionSQLCAIDSQLCABCSQLCODE
SQLERRMLSQLERRMC
SQLERRP
SQLERRD1SQLERRD2SQLERRD3SQLERRD4SQLERRD5SQLERRD6SQLWARNSQLSTATE
Post Processing DB2 SQLCA example
15
Function to run a DB2 SQL Statement
• Run_db2 is a function which will run a db2 command and set standard variables for subsequent interrogation
16
AWK Programs and common uses
• AWK is a programming language well suited for parsing, math and command processing
• Common uses include piping output into an Awk program to:• Select the Nth token from the command line:
grep “$( hostname –s ) db2nodes.cfg | awk ‘{ print $1}’
• Find the line number of a string in a fileawk ‘/James Sobieski/ {print NR }’ my_contacts
• db2 get dbm cfg | grep ‘DIAGPATH’ | awk -F"=" '{print $2}'
17
Common Simple GREP usage
• GREP (General Regular Expression Processor) is powerful tool to post-process output from a command or a file
• Common grep options
18
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Common SORT command usage
• SORT is powerful tool to post-process output from a command or a file and deduplicate output
• Common sort options
19
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Common HEAD unix command usage
• Head is powerful tool to pull lines of data from a file or stdoutstarting at the beginning of the file
• Common head options
20
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Common TAIL unix command usage
• Tail is powerful tool to pull lines of data from a file or stdoutreading backwards through the file
• Common tail options
21
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Common XARGS unix command usage
• XARGS is powerful tool which will execute a command substituting variables in positions of the command
• XARGS options
22
Option Description
Common Simple GREP usage
• GREP (General Regular Expression Processor) is powerful tool to post-process output from a command or a file
• Common grep options
23
Option Descriptionc counts occurrences of a string in StdIn or a filen includes the line number where a string was matchedf includes the filename of the file in which the string was matched
Example Snippets – Archiving the DB2 Diag log
• Archiving the db2 diag.log
24
Example Snippets – Post Processing db2pd -hadr
25
Example Snippets – Examining an Application Snapshot
• Snapu script:
26
Example Snippets – making list applications readable
• listu
27
Example Snippets – Displaying and Summarizing Tablespace status
• Show_ts.ksh
28
Example Snippets – Combining System Catalog Tables
• Show_tab_ix
29
Example Snippets – Counting SQL messages in a file
• show_all_sql_msgs.ksh
30
Example Snippets – using AWK to summarize vmstat
• vmstatt.ksh
• iostatt.ksh
31
Example Snippets – Capture Single SQL Execution Stats
• Stmt_execute.ksh
32
Example Snippets – Run an explain on SQL statement
• Jexpln.ksh
33
Example Snippets – Capture runaway Applheapsz
• Capture_applheapsz_too_large.ksh
34
Example Snippets – Add Partition to RP table
• Add_next_partition.ksh
35
Example Snippets – Convert Snapshot cmd output to script
• Gen_all_ss_variables.ksh
36
Example Snippets – Generate Views for Catalog Tables
• make_one_shortcat.ksh
37
Example Snippets – Generate Views for Catalog Tables
• make_all_shortcats.ksh
38
Example Snippets – Report on backups
• Report_all_backups_taken.ksh
39
Insert Name HereInsert Company Name HereInsert E-mail here
SessionTitle
Please fill out your session
evaluation before leaving!
Photo by Steve from Austin, TX, USA