Shell Scripting for MySQL Administration: An Introduction
-
Upload
best-tech-videos -
Category
Documents
-
view
822 -
download
0
description
Transcript of Shell Scripting for MySQL Administration: An Introduction
![Page 1: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/1.jpg)
Shell Scripting
for MySQL Administration
An Introduction
Bob Burgess
radian6 Technologies
MySQL User Conference 2009
![Page 2: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/2.jpg)
Who Are You?
Programmed before, at least a little
Starting out with Unix/Linux
Minimal exposure to "shell"
Probably moving from Windows
Windows "cmd" Bash shell
![Page 3: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/3.jpg)
Scope
Linux Bash shell – Beginner Level
◦ Variables
◦ Parameters
◦ Tests
◦ Flow Control
◦ File Redirection
![Page 4: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/4.jpg)
Scope
Common Utilities/Commands
◦ hostname
◦ tr
◦ cut
◦ ls
◦ read
◦ nail
◦ sed
![Page 5: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/5.jpg)
Scope
Script Interface to MySQL
◦ command-line parameters
◦ retrieving a single value
◦ iterating through a result set
◦ backup control
◦ security
![Page 6: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/6.jpg)
Scope
Examples
◦ Monitoring database status variables
◦ Monitoring/controlling replication
◦ Monitoring performance
◦ Alarming and alarm emails/pages
◦ Controlling backup
◦ Monitoring environment metrics such as
disk space
![Page 7: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/7.jpg)
Scope – NOT
Complete Course on Shell
Advanced Shell Techniques
Amazing Tricks
Under The Hood (MySQL / Linux)
![Page 8: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/8.jpg)
The Shell
![Page 9: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/9.jpg)
The Linux Shell
Command Processor
Bash
Bourne Shell (Steve Bourne, 1975)
Bourne-Again Shell
BASh
![Page 10: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/10.jpg)
Variables
Named with letters, numbers, “_”
Names start with letters or “_”
Holds strings
Math
◦ built-in does 64-bit signed ints
◦ utility “bc” does floating-point
Example:TIMEOUT=60
![Page 11: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/11.jpg)
Variables
Referenced with “$”$TIMEOUT
Surround name with { } when
confusion is possible ${TIMEOUT}
Can represent programs tooMYSQL=/usr/local/mysql/bin/mysql
$MYSQL -e”select 1”
![Page 12: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/12.jpg)
Variable Scope
Default: within current shell or script
Use export to make variable visible within commands launched from this shell or script
No way* to make variables visible to “parent” shell
* unless you “source” the script which sets the variables
![Page 13: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/13.jpg)
Script Parameters
$n
$0 – script name itself
$1 – first parameter
$2 – second parameter
${10} – tenth parameter
etc.
![Page 14: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/14.jpg)
Built-In “Variables”
$# – number of parameters
$? – return value from last command
$$ – Process ID of this shell
![Page 15: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/15.jpg)
Default I/O Channels
Standard In – 0
Standard Out – 1
Standard Error – 2
![Page 16: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/16.jpg)
File Redirection
Use non-default sources or
destinations for program input and
output
$ echo "hello"
hello
$ echo "hello" >some_file
$ cat some_file
hello
![Page 17: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/17.jpg)
Pipes
Daisy-chain commands
first_cmd | second_cmd
Connects stdout of one command to
stdin of the next
first_cmd second_cmd
![Page 18: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/18.jpg)
File Redirection
Use a file instead of keyboard input
some_command <input_file
Append the output to an existing file
echo "new line" >>my_file
Save the error output
mycmd 2>err_file >log_file
Save error and normal together
mycmd >log_file 2>&1
![Page 19: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/19.jpg)
Tests
Two types: String or Numeric
Useful in if/then and while
![Page 20: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/20.jpg)
Tests: String
[ "$ANSWER" = "NO" ]
[ "$ANSWER" \> "A" ]
[ "$ANSWER" \< "A" ]
[ -z "$ANSWER" ] (null)
[ -n "$ANSWER" ] (not null)
![Page 21: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/21.jpg)
Tests: Numeric
[ $VAL -eq 1 ]
[ $VAL -ne 1 ]
-le, -lt, -ge, -gt
[ 0$VAL -eq 1 ]
![Page 22: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/22.jpg)
Tests: File
File exists:[ -e "$MYFILE" ]
File does not exist:
[ ! -e "$MYFILE" ]
File has >0 bytes:[ -s "$MYFILE" ]
File exists and is a directory:[ -d "$DIR" ]
![Page 23: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/23.jpg)
Flow Control!
if / then
while
for
![Page 24: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/24.jpg)
IF / THEN
if [ $VAL -eq 1 ]; then
statements
else
statements
fi
![Page 25: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/25.jpg)
IF / THEN
if command; then
statements if command succeeds
else
statements if command fails
fi
![Page 26: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/26.jpg)
While
while [ $VAL -eq 1 ]; do
statements
done
![Page 27: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/27.jpg)
For
ALLPORTS="3306 3308 3319"
for PORT in $ALLPORTS; do
echo $PORT
statements
done
![Page 28: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/28.jpg)
For
cd /var/log/mysql
for LOGFILE in mysql-bin.[0-9]*
do
echo $LOGFILE
statements
done
![Page 29: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/29.jpg)
Two handy shell features
Run this and insert the result here
`command` (backticks)$ echo “Script started at `date`” >$LOGFILE$ cat $LOGFILE
Script started at Mon Apr 6 15:37:48 ADT 2009
Run a script and come back
. other_script.sh (source with a dot)
![Page 30: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/30.jpg)
Writing a Bash Script
First line: which executable to use:
#!/bin/bash
![Page 31: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/31.jpg)
Writing a Bash Script
Suggested Comments...
# script_name.sh [{maxrows}]
# Send an email if „queue‟ table
# exceeds {maxrows} rows.
# Parameters:
# Input: {maxrows}(defaults to 100)
# Output: email is sent to [email protected]
![Page 32: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/32.jpg)
Error Reporting
An Error Log
ERRLOG=/opt/mysql/log/script.err
if [ $OHNO -eq 1 ]; then
echo “script failed” >>$ERRLOG
exit 1
fi
![Page 33: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/33.jpg)
Error Reporting
Output to stderr
if [ $OHCRAP -eq 1 ]; then
echo “script failed” >&2
exit 1
fi
![Page 34: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/34.jpg)
Exit Status
0 = “good”
exit 0
Not 0 = “Not good”
ERR_DBDOWN=900
exit $ERR_DBDOWN
- or -
exit 1
![Page 35: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/35.jpg)
Running the Script
Turn on the “execute” bitchmod +x script.sh
dot slash or complete path./script.sh
See under the hood:bash -x script.sh
![Page 36: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/36.jpg)
Handy Utilities
![Page 37: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/37.jpg)
Utilities: man
The “M” in RTFM
![Page 38: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/38.jpg)
Utilities: info
A newer man
![Page 39: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/39.jpg)
Utilities: hostname
“Who Am I ?”
Handy within backticks:
HOST=`hostname -s`
$MYSQL -h${HOST}
![Page 40: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/40.jpg)
Utilities: tr
Translate characters
Delete characters
one,two,three one two three
one,two,three ONE,TWO,THREE
one,two,three on,two,thr
tr "," " "
tr a-z A-Z
tr –d "e"
![Page 41: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/41.jpg)
Utilities: cut
Return sections from a line
one,two,three two,three
datadir=/store /store
values (10,20,30)
10
cut –f2,3 –d","
cut –f2 –d=
cut –f2 "(" | cut –f1 -d","
![Page 42: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/42.jpg)
Utilities: ls
Options make it powerful
List files, one on each linels -1
List files with details, latest modify
time at the endls -ltr
Example: latest binlog file numberls -1tr /var/log/mysql/mysql-
bin.[0-9]*|tail -1|cut -f2 -d.
![Page 43: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/43.jpg)
Utilities: nail
Email!
Assumes the server is configured to
send emails to the outside world
Example: email contents of one
variable:echo $MSG |
nail -s "Subject" [email protected]
![Page 44: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/44.jpg)
Utilities: nail
Email contents of a file in the body and attach another filenail -a ${ATT_FILE} -s "$SUBJ"
$SENDTO <$MSGBODY
Email a file in the body and alias the From addressnail -s "The List" -S
from="[email protected]" $SENDTO
<list.txt
![Page 45: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/45.jpg)
Utilities: sed
Very powerful
Many commands
Well documented
![Page 46: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/46.jpg)
Utilities: sed
Translations too complex for „tr‟
echo $VX |
sed 's/Error://' >$LFILE
Delete 1st three lines from input
cat $LF | sed '1,3d'
![Page 47: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/47.jpg)
Utilities: date
Writing to log filesecho “Starting at `date`” >$LOGFILE
Starting at Mon Apr 6 23:07:07 ADT 2009
Dated filenamesLOGFILE=/logs/myscript.`date +%Y%m%d`.log
myscript.20090416.log
Rotating filenamesLOGFILE=/logs/myscript.`date +%a`.log
myscript.Wed.log
![Page 48: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/48.jpg)
Utilities: date
Timing
T_START=`date +%s`
do stuff
T_END=`date +%s`
T_ELAPSED=$((T_END-T_START))
![Page 49: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/49.jpg)
Utilities: date
When was $x seconds ago?
UPTIME=$((`date +%s` - $x))
UP_AT=`date -d "1970-01-01 UTC $UPTIME seconds"`
![Page 50: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/50.jpg)
Utilities: read
Accepts a line, puts it into variables
Useful with While loopswhile read LINE; do
echo $LINE
done <$FILE
$MYSQL -e"select a from T" | \
while read ROW ; do
echo $ROW
done
![Page 51: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/51.jpg)
Utilities: grep
Returns lines matching a pattern
From a filegrep "SUMMARY" $TMPFILE
From stdinsome_cmd | grep $PATTERN
![Page 52: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/52.jpg)
Utilities: grep
Just count the matchesgrep -c "Error" $LOGFILE
Put the count in a variableCNT=`grep -c "Error" $LOGFILE`
Show lines that don‟t matchps -ef|grep mysqld|grep -v grep
![Page 53: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/53.jpg)
Utilities: grep
Other Options
Case-insensitive matchgrep -i
Match only n lines then stopgrep –m n
Show matching line plus n lines aftergrep –A n
Quiet (only output is $? ) – good for if
grep –q
![Page 54: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/54.jpg)
Utilities: grep
Match Patterns
Any character: .
Range of characters: [2-8]
Set of characters: [E,H,Q]
Character repeated: *
Limit pattern to start of line: ^
Limit pattern to end of line: $
![Page 55: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/55.jpg)
Utilities: awk
Powerful pattern-matching language
Common use: getting values from a
line
Similar to cut
Can specify the field separator (normally “whitespace”) with -f
![Page 56: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/56.jpg)
Utilities: awk
Pluck data from randomly-spaced
columns
Example: get status and time from all
mysql sessions:mysql -e"show processlist" |
awk '{print $5,$6}'19516545 theuser server1:41083 sphere Sleep 0 NULL
Sleep,0
awk
![Page 57: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/57.jpg)
Utilities: awk
Pluck data from randomly-spaced
columns, with a condition
Example: get status and time from all
mysql sessions:mysql -e"show processlist" \
| awk '{print $5,$6}' \
| grep -v Sleep \
| grep -v Connect \
| awk "{if (\\$2 > $MAX_TIME) print $2}"
![Page 58: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/58.jpg)
mysql Command Line
mysql> _
![Page 59: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/59.jpg)
mysql Command Line
Relevant options
connection info: -u -p -h –P
execute a command: -e
change output format: -s -ss -B
![Page 60: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/60.jpg)
mysql Command Line
Default options in my.cnf
[client]
port = 3307
socket = /var/run/mysqld/mysqld.sock
[mysql]
user = theuser
password = xyzxyz
database = mydb
default-character-set = utf8
![Page 61: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/61.jpg)
mysql Command Line
Output Format: Interactive
mysql> select * from util.sample;+-------+-------+| digit | name |+-------+-------+| 1 | one || 2 | two || 3 | three |+-------+-------+3 rows in set (0.00 sec)
![Page 62: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/62.jpg)
mysql Command Line
Output Format: SQL from stdin
$ echo "select * from util.sample"|mysql
digit name1 one2 two3 three
\t \n
![Page 63: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/63.jpg)
mysql Command Line
Output Format: SQL as -e option, output to console
$ mysql -e"select * from util.sample"
+-------+-------+
| digit | name |
+-------+-------+
| 1 | one |
| 2 | two |
| 3 | three |
+-------+-------+
![Page 64: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/64.jpg)
mysql Command Line
Output Format: SQL as -e option, output to another program
$ mysql -e"select * from util.sample"|grep ".*"
digit name1 one2 two3 three
No table lines when stdin/stdout are redirected.
![Page 65: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/65.jpg)
mysql Command Line
Output Format: -s adds silence
+-------+-------+
| digit | name |
+-------+-------+
| 1 | one |
| 2 | two |
| 3 | three |
+-------+-------+
digit name
1 one
2 two
3 three
1 one
2 two
3 three
-s
-s
-ss
![Page 66: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/66.jpg)
mysql command line
Retrieving one value
V=`$MYSQL -e"select min(digit)
from util.sample"`
echo $V
min(digit) 1
![Page 67: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/67.jpg)
mysql command line
Retrieving one value
V=`$MYSQL -s -e"select min(digit)
from util.sample"`
echo $V
1
![Page 68: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/68.jpg)
mysql command line
Retrieving several values from one line
L=`$MYSQL -s -e"select min(digit),
max(digit), min(name) from
util.sample"`
echo $L
1 3 one
set - $L
echo "Highest digit is $2“
Highest digit is 3
![Page 69: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/69.jpg)
mysql command line
Iterating through several lines
L=`$MYSQL -s -e"select digit,name
from util.sample"`
echo $L
1 one 2 two 3 three
Nope.
![Page 70: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/70.jpg)
mysql command line
Iterating through several lines
$MYSQL -s -e"select digit,name
from util.sample" \
| while read L; do
echo $L
done
1 one
2 two
3 three
![Page 71: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/71.jpg)
mysql command line
Iterating through several lines
$MYSQL -s -e"select digit,name
from util.sample" \
| while read D N; do
echo "Digit $D is called $N"
done
Digit 1 is called one
Digit 2 is called two
Digit 3 is called three
![Page 72: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/72.jpg)
mysql command line
Hiding the password
File mysql_access:
MYSQL_USER="root"
MYSQL_PASSWORD="secret"
chmod 400 mysql_access# ls -l mysql_access
-r-------- 1 root root 44 Nov 17 2007 mysql_access
![Page 73: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/73.jpg)
mysql command line
Hiding the password
In your script:
. mysql_access
MYSQL="/usr/local/mysql/bin/mysql
–u$MYSQL_USER –p$MYSQL_PASSWORD"
![Page 74: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/74.jpg)
cron
Wakes up every minute to run stuff
crontab -e to edit the to-do list
crontab -l to list the list
Has its own execution path!
![Page 75: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/75.jpg)
cron
Schedule format
minute 0-59
hour 0-23
day of month 1-31
month 1-12
day of week 0-7 (0 or 7 is Sunday)
![Page 76: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/76.jpg)
cron
Schedule examples
Every day at 1:15pm:15 13 * * *
Every 5 minutes:*/5 * * * *
Every Monday at 7 am:0 7 * * 1
Every 10 minutes, 2-3am, 8-9am, and 2-7pm, on weekends in February:*/15 2,8,14-18 * 2 6,7
![Page 77: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/77.jpg)
cron
Run SQL and discard output:* * * * * /usr/local/mysql/bin/mysql –
e"update tbl set col=col*1.2"
>/dev/null 2>&1
Run SQL. Log output and errors:* * * * * /usr/local/mysql/bin/mysql –
e"update tbl set col=col*1.2"
>>/log/update.log 2>&1
![Page 78: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/78.jpg)
cron
Run a script from cron:
* * * * * /opt/mysql/script/myscript.sh
Don‟t rely on cron‟s $PATH! Use
absolute paths for everything!
![Page 79: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/79.jpg)
Example:
MySQL Status variablesSend mail when Threads_connected
reaches 1000:TC=`$MYSQL -s -e"show status like
'Threads_connected' "|cut -f2`
if [ $TC –ge 1000 ]; then
echo "Thread count at $TC" | \
nail -s"Thread emergency"
fi
![Page 80: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/80.jpg)
Example:
ReplicationSee how far behind or if we‟re stopped:$MYSQL -e‟show slave status\G‟ >$TMP
SB=`grep Seconds_Behind_Master $TMP |
cut –f2 –d: | tr –d " "`
SBNUM=$SB
if [ "$SB" = "NULL" ]; then
SBNUM=99999999
fi
if [ $SBNUM –le $MAXBEHIND ]; then
exit 0
fi
![Page 81: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/81.jpg)
Example:
ReplicationCheck IO and SQL Running:
IO_R=`grep Slave_IO_Running
$TMPFILE |cut -f2 -d: |tr -d " "`
SQL_R=`grep Slave_SQL_Running
$TMPFILE | cut -f2 -d: |tr -d " "`
![Page 82: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/82.jpg)
Example:
ReplicationTest and Alarm:ERRMSG=""
if [ "$SB" = "NULL" ]; then
ERRMSG="Replication Down."
elsif [ $SBNUM –ge $MAXBEHIND ]; then
ERRMSG="Replication behind ${SBNUM}s."
fi
if [ -n "$ERRMSG" ]; then
echo "$ERRMSG IO:${IO_R}, SQL:${SQL_R}"\
| nail –s"Replication" [email protected]
fi
![Page 83: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/83.jpg)
Example:
Performance MonitorT0=`date +%s`
$MYSQL -e"select avg(x) from
bigtable" >/dev/null
T1=`date +%s`
T=$((T1-T0))
if [ $T -gt $TMAX ]; then
echo "Test query took ${T}s."\
| nail –s "Slow DB"
fi
![Page 84: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/84.jpg)
Alarming
Email directly from script.
echo "Error message" | nail
Easy to write, but hard to maintain.
![Page 85: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/85.jpg)
Alarming
Use an alarm log.echo "WARNING:monitor_repldb:Prod:
${port}:${host}:`date +%Y%m%d-%H%M%S`:
Error message." >>$ALARM_LOG
One alarm log file per server
(or one for many – NFS)
Record of all alarm events
Read / Email with a separate script
![Page 86: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/86.jpg)
Backup – Components
Shutdown
Copy (tar / gzip)
Startup
Generate config files
Call ibbackup (test for success)
Call mysqldump (test for success)
![Page 87: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/87.jpg)
Backup – Shutdown# Start shutdown:echo "Starting shutdown." >$BACKUP_LOG$MYSQLADMIN shutdown 2>>$BACKUP_LOG >>$BACKUP_LOG
# Wait for shutdown to complete:STOP_WAITING_TIME=$((`date +%s` + ($SHUTDOWN_TIMEOUT * 60) ))MYSQLD_RUNNING=`ps -ef|grep "$PS_PATTERN"|grep -v grep|wc –l`while [ $MYSQLD_RUNNING -eq 1 -a `date +%s` -lt$STOP_WAITING_TIME ]; dosleep 3MYSQLD_RUNNING=`ps -ef|grep "$PS_PATTERN"|grep -v grep|wc –l`
done
# Error if it didn't shut down:if [ $MYSQLD_RUNNING -eq 1 ]; thenecho "Shutdown did not finish in time. Ending." >>$BACKUP_LOGecho "WARNING:cold_backup:`date +%Y%m%d-%H%M%S`:Cold Backup of
$HOST:$DB_PORT Failed: Shutdown timed out." >>$ALARM_LOGexit 1
elseecho "Shutdown Complete at `date`" >>$BACKUP_LOG
fi
![Page 88: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/88.jpg)
Backup – Cold Copy
# echo "Start backup to $BACKUP_FILE at `date`" >>$LOG
if tar czf $BACKUP_FILE ${SOURCE_DIR}/* >>$LOG 2>&1
then
echo "File copy (Tar) complete at `date`" >>$LOG
else
echo "Error from Tar at `date`" >>$LOG
echo "WARNING:cold_backup:`date +%Y%m%d-%H%M%S`:Cold
Backup of $HOST:$DB_PORT Failed: Tar error."
>>$ALARM_LOG
exit 1
fi
![Page 89: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/89.jpg)
Backup – Startupecho "Restarting database." >>$BACKUP_LOG
/etc/init.d/mysql start
STOP_WAITING_TIME=$((`date +%s` + ($STARTUP_TIMEOUT * 60) ))
$MYSQLADMIN ping >/dev/null 2>&1
DB_UP=$?
while [ $DB_UP -ne 0 \
-a `date +%s` -lt $STOP_WAITING_TIME ]
do
sleep 3
$MYSQLADMIN ping >/dev/null 2>&1
DB_UP=$?
done
![Page 90: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/90.jpg)
Backup – Generate Config# Write the source config
echo "[mysqld]" >$SOURCE_CNF
$MYSQL -e"show variables like 'innodb%';show variables like
'datadir'" \
| awk '{print $1"="$2}' >>$SOURCE_CNF
# Write the destination config
echo "datadir=$DESTINATION" > $DEST_CNF
echo "innodb_data_home_dir=$DESTINATION" >>$DEST_CNF
echo "innodb_log_group_home_dir=$DESTINATION" >>$DEST_CNF
grep "innodb_data_file_path" $SOURCE_CNF >>$DEST_CNF
grep "innodb_log_files_in_group" $SOURCE_CNF >>$DEST_CNF
grep "innodb_log_file_size" $SOURCE_CNF >>$DEST_CNF
![Page 91: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/91.jpg)
Backup – Call backup or dump
if $IBBACKUP $SOURCE_CNF $DEST_CNF >>$LOG 2>&1; then
echo "# EDIT THIS FILE: CHANGE DIRECTORIES TO RECOVERY
LOCATION" >$DESTINATION/recover.cnf
cat $DEST_CNF >>$DESTINATION/recover.cnf
echo "* ibbackup finished successfully. " >>$LOG
echo "INFO:hot_backup:`date +%Y%m%d-%H%M%S`: Successful
finish." >>$ALARM_LOG
EXITCODE=0
else
echo "* ibbackup: nonzero return code. *" >>$LOG
echo "WARNING:hot_backup:`date +%Y%m%d-%H%M%S`: ERROR from
ibbackup." >>$ALARM_LOG
EXITCODE=1
fi
Calling mysqldump is basically the same.
![Page 92: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/92.jpg)
Environment Monitoring
Important one: disk spaceDFTEMP=/tmp/monitor_disk.$$
df -HP >$DFTEMP &
sleep $DF_WAIT
if [ ! -s $DFTEMP ]
then # df is blocked
echo "WARNING:monitor_disk:$HOST:`date
+%Y%m%d-%H%M%S`: Disk Space cannot be
measured. (Usual cause: NFS error.)"
>> $ALARM_LOG
else # df is not blocked...
![Page 93: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/93.jpg)
Environment Monitoringcat $DFTEMP | awk '{ print $5 " " $1 " " $6}' | \
while read output; do
used=$(echo $output | awk '{ print $1}' | cut -d'%' -f1 )
partition=$(echo $output | awk '{ print $3 }' )
if [ $used -ge $DISK_ERROR ]; then
echo "ERROR:monitor_disk:$HOST:`date +%Y%m%d-%H%M%S`:
Disk Space on $partition at $used% (Threshold=
$DISK_ERROR%)" >> $ALARM_LOG
else
if [ $used -ge $DISK_WARNING ]; then
echo "WARNING:monitor_disk:$HOST:`date +%Y%m%d-
%H%M%S`:Disk Space on $partition at $used%
(Threshold=$DISK_WARNING%)" >> $ALARM_LOG
fi
fi
done
fi
![Page 94: Shell Scripting for MySQL Administration: An Introduction](https://reader033.fdocuments.us/reader033/viewer/2022051413/55281de955034684588b4679/html5/thumbnails/94.jpg)
Resources
man pages
info pages
MySQL Docs
Advanced Bash-Scripting GuideMendel Cooper www.tldp.org/LDP/abs