Www.ischool.drexel.edu INFO 320 Server Technology I Week 9 Unix/Linux tools 1INFO 320 week 9.
-
Upload
grace-robertson -
Category
Documents
-
view
221 -
download
2
Transcript of Www.ischool.drexel.edu INFO 320 Server Technology I Week 9 Unix/Linux tools 1INFO 320 week 9.
www.ischool.drexel.edu
INFO 320Server Technology I
Week 9
Unix/Linux tools
1INFO 320 week 9
www.ischool.drexel.edu
Overview
• This wraps up the course with a handful of other tools and technologies of interest– Linux Utilities– RAID
2INFO 320 week 9
www.ischool.drexel.edu
Linux Utilities
3INFO 320 week 9
From notes by Dr. Randy M. Kaplan, and (Petersen, 2009)
www.ischool.drexel.edu
Linux Utilities
• We’ll look at a few more Unix applications of common interest for server administration– uniq, diff– lpr, cups– which, tar– ntp, cron– emacs
4INFO 320 week 9
www.ischool.drexel.edu
uniq
• uniq removes duplicate lines in a file•uniq filename
• Example– uniq lastnames– Any duplicate names in the file named lastnames will be eliminated
– Make sure to sort the file first!
5INFO 320 week 9
www.ischool.drexel.edu
diff
• diff compares the contents of one file with another file– diff filename1 filename2
• Example– diff colorsa colorsb– Compares the contents of colorsa with the
contents of colorb
6INFO 320 week 9
www.ischool.drexel.edu
diff
• The diff utility bears some further examination
• Let’s assume that colorsa contains –– Red
– Blue
– Green
– Yellow
• Let’s assume that colorsb contains –– Red
– Blue
– Green
7INFO 320 week 9
www.ischool.drexel.edu
diff
• The diff utility operates under the assumption the first file is to be converted into the second file
• diff output is a series of lines that define how the first file can be modified so that its content matches the second file– The output for colorsa and colorsb would be
4d3 which specifies that the 4th line of the colorsa file should be deleted in order that the colorsa file match colorsb
8INFO 320 week 9
www.ischool.drexel.edu
lpr
• lpr prints a file•lpr filename
• Example– lpr areport– lpr –PofficePrinter areport (prints
the file areport on the printer named officePrinter)
9INFO 320 week 9
www.ischool.drexel.edu
cups
• The Common Unix Printing System (CUPS) is a GNU print server application
• Cups is managed using a web-based tool at http://localhost:631– There you can see print jobs, manage
printers, etc.– But first you need to define the printers
to be managed
10INFO 320 week 9
www.ischool.drexel.edu
cups
• It config file is /etc/cups/printers.conf– That contains the device file for each printer
and configuration entries for it– Devices can be identified by parallel name
(lp0, lp1, …), serial name (tty0, tty1), or URI
• Each device gets a print spool directory under /var/spool/cups/devicename
• The cups daemon is cupsd
11INFO 320 week 9
www.ischool.drexel.edu
which
• which shows the location of the file related to the command (like whereis)
•which command
– Example•which info•/usr/local/bin/info
• Useful when a command is not doing what is expected
12INFO 320 week 9
www.ischool.drexel.edu
tar
• tar is short for ‘tape archive’– Original purpose was to create and read
magnetic tape archives – Today it’s used to extract a directory with
subdirectories and files beneath it
• tar files are also frequently compressed– The extension of a tar file that has been
compressed will be .tar.gz or .tgz
13INFO 320 week 9
www.ischool.drexel.edu
tar
• tar syntax is– tar options filename
• Example– tar –xvzf myTarfile.tgz– Decompress and restore (extract) all files in
the archive named myTarfile.tgz
14INFO 320 week 9
www.ischool.drexel.edu
ntp
• The Network Time Protocol (ntp) coordinates time settings across your network
• It and its documentation (ntp-doc) are installed with apt-get or aptitude like most packages– Docs are likely under /usr/share/doc/ntp-doc/html/index.html
15INFO 320 week 9
www.ischool.drexel.edu
ntp
• The ntp config file is /etc/ntp.conf• Start ntp with
– sudo /etc/init.d/ntp start
• Check status of ntp with– ntpq –p
• Or use an external ntp server with the ntpdate command (add to startup or cron)– sudo ntpdate ntp.ubuntu.com
16INFO 320 week 9
www.ischool.drexel.edu
cron
• Addressed in more detail in connection with backups, cron allows you to schedule scripts to execute
• cron is started automatically from /etc/init.d on entering multi-user runlevels– crontab files can be in its spool area
(/var/spool/cron/crontabs) and more important (/etc/crontab) this is where you define cron jobs!
17INFO 320 week 9
www.ischool.drexel.edu
cron
• cron logs its action to the syslog facility ’cron’, and logging may be controlled using the standard syslogd facility
• The heart of setting a service to run, is defining a crontab entry for it– You can specify when a file runs by minute,
hour, day of month, month, or day of week– Each of those five parameters correspond to
numbers or *s at the start of each entry18INFO 320 week 9
www.ischool.drexel.edu
cron
• # use /bin/sh to run commands, no matter what /etc/passwd says
• SHELL=/bin/sh • # mail any output to ‘[email protected]’, no matter
whose crontab this is • [email protected] • # run five minutes after midnight, every day • 5 0 * * * $HOME/bin/daily.job >> $HOME/tmp/out 2>&1
19INFO 320 week 9
www.ischool.drexel.edu
cron
• # • # run at 2:15pm on the first of every month -- output
mailed to bruce (above) • 15 14 1 * * $HOME/bin/monthly • 23 0-23/2 * * * echo "run 23 minutes after midn, 2am,
4am ..., everyday" • 5 4 * * sun echo "run at 5 after 4 every sunday"
20INFO 320 week 9
www.ischool.drexel.edu
Editors
• Before there were word processors there were programs called editors
• Like a word processor, an editor is used to create text files and manipulate them
• The original use for a text editor was to create program files and modify them– A venerable text editor, designed for Lisp
programming, is Emacs
21INFO 320 week 9
www.ischool.drexel.edu
Emacs
• Not developed in the UNIX environment– Therefore – not a special purpose utility
• More than a text editor– Built-in programming language, Lisp– Designed to do everything
• Emacs is a language sensitive editor with specific features for editing programming languages
22INFO 320 week 9
www.ischool.drexel.edu
The big pattern!
• Notice the pattern for most Unix applications outside of the kernel– App names are all lower case, e.g. appname– Most apps can be installed with apt-get or aptitude, e.g. •sudo apt-get install appname
– Every app has a man page; man appname– Most apps have a text configuration file, often
something like /etc/appname.conf
23INFO 320 week 9
www.ischool.drexel.edu
The big pattern!
– Many apps can be started with sudo /etc/init.d/appname start
– If the app needs a spool directory, it’s under /var/spool/appname
– The daemon for an app is usually appnamed – If there’s separate documentation for the app,
it’s probably under /usr/share/doc/appname
• See? UNIX is easy!
24INFO 320 week 9
www.ischool.drexel.edu
RAID
25INFO 320 week 9
Mostly from (Bytepile, 2009) and (Rankin, 2009)Above image from www.fordcortina.net/pix/raid.jpg
www.ischool.drexel.edu
RAID
• RAID (was Redundant Array of Inexpensive Disks, now often Independent) was developed circa 1988 by Patterson, Gibson & Katz; Berkeley CA– A RAID array is a collection of drives which
collectively act as a single storage system, which can tolerate the failure of a drive without losing data, and which can operate independently of each other.
26INFO 320 week 9
www.ischool.drexel.edu
RAID
• RAID is defined by various levels a set of disks (a RAID array) is designed to achieve – RAID 0, RAID 5, etc.
• They differ mainly in terms of – Cost – Speed– Redundancy, and efficiency to do so– How many disks are needed at a minimum
27INFO 320 week 9
www.ischool.drexel.edu
RAID
• RAID can be implemented in software alone, or with dedicated hardware (e.g. cards or built into a motherboard)– Hardware-based RAID is faster and more
expensive
• RAID typically requires all drives in an array to be identical (same make and model, not just same size and capacity)
28INFO 320 week 9
www.ischool.drexel.edu
RAID 0
• RAID 0 is known as striping (watch spelling, it’s not stripping!)
• It offers no redundancy, but improves performance (speed) by reading alternately between disks– No speed improvement for writing data!
• Two or more drives can do RAID 0
29INFO 320 week 9
www.ischool.drexel.edu
RAID 1
• RAID 1 is called mirroring
• It does nothing for performance, but provides redundancy by maintaining a duplicate copy of data– Hence there is 100% duplication of data, the
least efficient possible RAID– Using at least two disks in matched pairs, it
simply makes two copies of everything on separate disks
30INFO 320 week 9
www.ischool.drexel.edu
RAID 0+1
• So if we do both mirroring and striping at once, we have RAID 0+1– RAID 0+1 is implemented as a mirrored array
whose segments are RAID 0 (striped) arrays
• Hence we need at least four disks, and get the read speed boost of striping, and the redundancy of mirroring
31INFO 320 week 9
www.ischool.drexel.edu
RAID 3
• RAID 3 does striping, plus adds a parity drive; 3 drive minimum– The last drive has a parity bit from all of the
other drives, used to reconstruct one drive if it fails
• Very high read and write speeds, high efficiency, good for multimedia data, but expensive and complex
32INFO 320 week 9
www.ischool.drexel.edu
RAID 5
• RAID 5 is similar to RAID 3, but the parity information is distributed across all the drives; 3 drive minimum– Hence if any one drive fails, it can be
recreated from the other drives
• High read rate and efficiency, medium write rate– Very commonly used for dB, web, file, and
other servers33INFO 320 week 9
www.ischool.drexel.edu
RAID 10
• Notice that RAID 10 is not the same as RAID 0+1
• RAID 10 is a striped array whose segments are RAID 1 arrays (mirrors) instead of single drives– Minimum of 4 disks needed– Good when you need higher performance
than RAID 1; but very expensive
34INFO 320 week 9
www.ischool.drexel.edu
RAID 10
• RAID 10 can handle multiple drive failures, as long as you don’t lose both halves of a mirrored pair
35INFO 320 week 9
www.ischool.drexel.edu
RAID 50
• RAID 50 or 05 is striping two sets of RAID 5 arrays– Minimum of six
disks!– Better redundancy,
faster reads
36INFO 320 week 9
www.ischool.drexel.edu
RAID and Ubuntu
• A RAID 5 array can not be used as the boot partition in Ubuntu – GRUB can’t read it– A boot partition can be RAID 1
• RAID is configured during the partitioning process of installation– Each partition is formatted to type ‘K raid’, and
they are joined into an array as a multidisk (MD) device, then partitioned to /, /boot, swap
37INFO 320 week 9
www.ischool.drexel.edu
RAID and Ubuntu
• The main tools for administering RAID in Linux are /proc/mdstat and mdadm
• The former is a file with your current arrays– cat /proc/mdstat
• Linux recognizes RAID 0, 1, 4, 5, 6 and 10– This is software-only RAID
38INFO 320 week 9
www.ischool.drexel.edu
RAID Capacity
• The usable capacity of common RAID levels is– RAID 0 = # Drives x Cap Drive– RAID 1 = (# Drives / 2) x Cap Drive– RAID 3 or 5 = (# Drives -1) x Cap Drive
• RAID 3 and 5 ‘lose’ one drive’s capacity for parity information from the other drives, so larger arrays are more efficient
39INFO 320 week 9
From http://www.kintronics.com/raidwpaper.htm
www.ischool.drexel.edu
Other RAIDs
• There are other RAID approaches (RAID 2, 4, 6, 7, 53, 1E, etc.) but combinations of RAID 0, 1, and 5 are most common
• Other storage technologies include– JBOD– DAS vs. NAS – SAN
40INFO 320 week 9
www.ischool.drexel.edu
JBOD
• JBOD is Just a Bunch Of Disks– RAID systems require identical drives– JBOD allows combining physical drives into
large logical volumes without requiring identical drives
– Cheap, easy, but no redundancy
41INFO 320 week 9
www.ischool.drexel.edu
DAS vs. NAS
• DAS is direct-attached storage, a fancy name for disks which are part of a server
• NAS (network-attached storage) is 100% dedicated to serving files over a network– It’s one or more hard drives, with enough
computer attached to let it communicate directly over the network
– Simple and cheap
42INFO 320 week 9
www.ischool.drexel.edu
SAN
• A Storage Area Network is a dedicated, high performance storage network – Transfers data among servers and storage
devices, separate from the local area network– Used when high speed large transfers are
common, often using fiber optic connections– Often does load balancing and has high
reliability– Often used for transaction processing
43INFO 320 week 9
www.ischool.drexel.edu
SAN
44INFO 320 week 9
From http://www.rad-direct.com/Product-Whitepaper-IP-SAN-3.htm
www.ischool.drexel.edu
Hot-swappable
• Finally, storage systems can have hot-swappable drives or not– Hot-swappable means they can be safely
removed from the system while it’s running
• Most RAID systems are hot-swappable, so that failed drives can be removed without reducing availability
45INFO 320 week 9
www.ischool.drexel.edu
References
• Bytepile, 2009. “RAID Types - Classifications ” from http://bytepile.com/raid_class.php
• The RAID Advisory Board (RAB) does not seem to exist any longer
46INFO 320 week 9