Post on 04-Jan-2016
Xuan Guo CSc 3320 1
Chapter 3-2
Graham Glass and King Ables, UNIX for Programmers and Users,
Third Edition, Pearson Prentice Hall, 2003.
Xuan Guo CSc 3320 2
uniq
• fed with a text file, output the file with adjacent identical lines collapsed to one
• uniq -c -number [inputfile [outputfile]]• -c: causes each line to be preceded by the
number of occurrences that were found• -number: number fields of each line are
ignored
Xuan Guo CSc 3320 3
Happy Birthday to You!Happy Birthday to You!Happy Birthday Dear Tux!Happy Birthday to You!
uniq uniq.dat
Happy Birthday to You!Happy Birthday Dear Tux!Happy Birthday to You!
Xuan Guo CSc 3320 4
Happy Birthday to You!Happy Birthday to You!Happy Birthday Dear Tux!Happy Birthday to You!
uniq -c uniq.dat
2 Happy Birthday to You! 1 Happy Birthday Dear Tux! 1 Happy Birthday to You!
Xuan Guo CSc 3320 5
Sort
• sorts a file in ascending or descending order based on one or more fields.
• Individual fields are ordered lexicographically, which means that corresponding characters are compared based on their ASCII value.
Xuan Guo CSc 3320 6
Sort
• sort -tc -r [+POS1 [-POS2]] {sortField -bfMn}* {fileName}*– -tc separator is c instead of blank– -r descending instead of ascending– -b ignore leading blanks – -f ignore case– -M month sort (3 letter month abbreviation)– -n numeric sort– +POS1 [-POS2] key positions start [up to end]
Xuan Guo CSc 3320 7
Sort Examples
$ cat sort.dat John Smith 1222 20 Apr 1956 Tony Jones 1012 20 Mar 1950 John Duncan 1111 20 Jan 1966 Larry Jones 1223 20 Dec 1946
$ sort +0 -2 sort.dat John Duncan 1111 20 Jan 1966 John Smith 1222 20 Apr 1956 Larry Jones 1223 20 Dec 1946 Tony Jones 1012 20 Mar 1950
Xuan Guo CSc 3320 8
Sort Examples
$ sort +4 -5 -M sort.dat John Duncan 1111 20 Jan 1966 Tony Jones 1012 20 Mar 1950 John Smith 1222 20 Apr 1956 Larry Jones 1223 20 Dec 1946
$ sort +4 -5 sort.dat John Smith 1222 20 Apr 1956 Larry Jones 1223 20 Dec 1946 John Duncan 1111 20 Jan 1966 Tony Jones 1012 20 Mar 1950
Xuan Guo CSc 3320 9
Sort Examples
$ sort -k 6,6 -k 5,5 sort.datLarry Jones 1223 20 Dec 1946Tony Jones 1012 20 Mar 1950John Smith 1222 20 Apr 1956John Duncan 1111 20 Jan 1966
Xuan Guo CSc 3320 10
Archiving
• tar -cfrtuvx tarFileName fileList– creates a “tape archive”-format file from the
fileList– Typically, use either -c or -x, but not both!
• -c for Create• -x for Extracts
– -t option generates a table of contents– -r option unconditionally appends listed files to tar
formatted file
Xuan Guo CSc 3320 11
Archiving
• tar -cfrtuvx tarFileName fileList– -u option appends only files that are more recent
than those already archived– -f option enables you to give a tar file name
• Default name is /dev/rmt0– -v verbose
• If fileList contains directory, its contents are appended/extracted recursively.
Xuan Guo CSc 3320 12
Archive Example
$ tar -cvf ch6.tar ch6 ch6/ ch6/menu.csh ch6/junk/ ch6/junk/junk.csh ch6/junk.csh ch6/menu2.csh ch6/multi.csh ch6/expr1.csh ch6/expr3.csh ch6/expr4.csh ch6/if.csh ch6/menu3.csh
Xuan Guo CSc 3320 13
Archive Example
$ ls -l ch6.tar -rw-rw-r-- 1 raj raj 20480 Jun 26 20:08 ch6.tar
$ tar -tvf ch6.tar drwxr-xr-x raj/raj 0 2007-06-03 09:57 ch6/ -rwxr-xr-x raj/raj 403 2007-06-02 14:50 ch6/menu.csh drwxr-xr-x raj/raj 0 2007-06-03 09:57 ch6/junk/ -rwxr-xr-x raj/raj 1475 2007-06-03 09:57 ch6/junk/junk.csh -rwxr-xr-x raj/raj 1475 2007-06-03 09:56 ch6/junk.csh -rw-r--r-- raj/raj 744 2007-06-02 15:59 ch6/menu2.csh -rwxr-xr-x raj/raj 445 2007-06-02 15:26 ch6/multi.csh -rwxr-xr-x raj/raj 279 2007-06-02 15:18 ch6/expr1.csh -rwxr-xr-x raj/raj 98 2007-06-02 15:20 ch6/expr3.csh -rwxr-xr-x raj/raj 262 2007-06-02 15:21 ch6/expr4.csh -rwxr-xr-x raj/raj 204 2007-06-02 15:22 ch6/if.csh -rw-r--r-- raj/raj 744 2007-06-02 16:01 ch6/menu3.csh
Xuan Guo CSc 3320 14
Archive Example
$ rm -fr ch6
$ tar -rvf ch6.tar date.txt date.txt
$ tar -tvf ch6.tar drwxr-xr-x raj/raj 0 2007-06-03 09:57 ch6/ -rwxr-xr-x raj/raj 403 2007-06-02 14:50 ch6/menu.csh drwxr-xr-x raj/raj 0 2007-06-03 09:57 ch6/junk/ -rwxr-xr-x raj/raj 1475 2007-06-03 09:57 ch6/junk/junk.csh -rwxr-xr-x raj/raj 1475 2007-06-03 09:56 ch6/junk.csh -rw-r--r-- raj/raj 744 2007-06-02 15:59 ch6/menu2.csh -rwxr-xr-x raj/raj 445 2007-06-02 15:26 ch6/multi.csh -rwxr-xr-x raj/raj 279 2007-06-02 15:18 ch6/expr1.csh -rwxr-xr-x raj/raj 98 2007-06-02 15:20 ch6/expr3.csh -rwxr-xr-x raj/raj 262 2007-06-02 15:21 ch6/expr4.csh -rwxr-xr-x raj/raj 204 2007-06-02 15:22 ch6/if.csh -rw-r--r-- raj/raj 744 2007-06-02 16:01 ch6/menu3.csh -rw-rw-r-- raj/raj 29 2007-06-21 11:06 date.txt
Xuan Guo CSc 3320 15
Archive Example
$ tar -xvf ch6.tar ch6/ ch6/menu.csh ch6/junk/ ch6/junk/junk.csh ch6/junk.csh ch6/menu2.csh ch6/multi.csh ch6/expr1.csh ch6/expr3.csh ch6/expr4.csh ch6/if.csh ch6/menu3.csh date.txt
Xuan Guo CSc 3320 16
Find Utility
find pathList expression• finds files starting at pathList• finds files descending from there• Allows you to perform certain actions
– e.g. deleting the files
Xuan Guo CSc 3320 17
Find Utility
-name pattern true if the file name matches pattern
-perm octtrue if the octal description of file's permission equals oct
-type chtrue if the type of the file is ch (b=block, c=char ..)
-user userId true if the owner of the file is userId
-group groupId true if the group of the file is groupId
Xuan Guo CSc 3320 18
Find Utility
-atime counttrue if the file has been accessed within the count-th day
-ctime counttrue if the contents of the file have been modified within the count-th day or any of its file attributes have been modified
-exec command true if the exit code = 0 from executing the command.
command must be terminated by \;If {} is specified as a command line argument it is replaced by the file name currently matched
Xuan Guo CSc 3320 19
Find Utility
-printprints out the name of the current file and returns true
-lsdisplays the current file's attributes and returns true
!expression negation of expression
expr1 [-a] expr2 short circuit and
expr1 -o expr2short circuit or
Xuan Guo CSc 3320 20
Find Examples
• $ find / -name x.c– searches for file x.c in the entire file system
• $ find . -mtime 14 -ls – lists files modified in the 14th day of the past
• $ find . -name '*.bak' -ls -exec rm {} \; – ls and then remove all files that end with .bak
Xuan Guo CSc 3320 21
Find Examples
$ find ch6 -name ‘j*’ ch6/junk ch6/junk/junk.csh ch6/junk.csh
Xuan Guo CSc 3320 22
Scheduling Command (cron)
• $ crontab cronTabName• $ crontab -ler [useName]
– Allows you to schedule a series of jobs– Executed on a periodic basis– Uses a file with the following line format:
minute hour day month weekday command(0-59) (0-23) (1-31) (1-12) (1-7; 1=Monday) (any command)
Xuan Guo CSc 3320 23
Scheduling Example
$ cat crontab.cron0 8 * * 1 echo Happy Monday Morning* * * * * echo One Minute passed > /dev/pts/0
minute hour day month weekday command(0-59) (0-23) (1-31) (1-12) (1-7; 1=Monday) (any command)
Xuan Guo CSc 3320 24
Scheduling Example
$ crontab crontab.cron
$ crontab -l# DO NOT EDIT THIS FILE - edit the master and reinstall.# (crontab.cron installed on Sat Jun 26 23:33:35 2007)# (Cron version -- $Id: crontab.c,v 2.13 1994/01/17 03:20:37 vixie Exp $)0 8 * * 1 echo Happy Monday Morning* * * * * echo One Minute passed > /dev/pts/0
$ crontab -runregister crontab file
Xuan Guo CSc 3320 25
Scheduling Command (at)
• $ at command – allows you to schedule one-time
commands/scripts• $ at -csm time [date [, year]] [+increment] [-f script]
– -c option : use C-shell – -s option : use Bourne shell – -m option : send mail
Xuan Guo CSc 3320 26
Scheduling Command (at)
• $ at -l – -l option list all entries in at queue
• $ atrm [jobId]+– remove entries in at queue
Xuan Guo CSc 3320 27
Scheduling Command (at)
• $ at -csm time [date [, year]] [+increment] [script]– time is specified as HH or HHMM followed by an
optional AM/PM– date is spelled out using first 3 letters of day
and/or month– keyword now can be used in place of time
sequence; today and tomorrow can be used in place of date
– stated time may be augmented by an increment (number followed by minutes/hours/days/weeks/months/years)
Xuan Guo CSc 3320 28
Scheduling Example
$ cat at.csh #!/bin/csh echo at done > /dev/tty1
$ at now + 2 minutes -f at.csh
$ at -l lists the job here
You may program the script to reschedule itself as follows:
#!/bin/csh date > /dev/tty1 at now + 2 minutes -f at.csh
Xuan Guo CSc 3320 29
Hard Links
• $ ln original newLink– Creates a hard link to original file called newLink– Both labels will refer to the same file– File will be deleted only when both labels are
removed– If newLink is a directory then links are made within
the directory
Xuan Guo CSc 3320 30
Hard Link Example
$ cat > oneThis is an example file.
$ ln one two$ cat twoThis is an example file.
$ vi two # changed “an” to “a great”$ cat twoThis is a great example file.
$ cat oneThis is a great example file.
$ ls -l two-rw-r--r-- 2 mweeks mweeks 31 Sep 14 03:26 two
Xuan Guo CSc 3320 31
Soft (Symbolic) Links
• $ ln -s original newLink– Symbolic/soft links can be created from one file
system into another file system– Hard links are restricted to one file system
• Use ls -lL to view link details – which file it refers to
• ls -l displays the contents of the symbolic link
$ ls -llrwxrwxrwx 1 root root 23 2006-09-22 20:26 javac -> j2sdk1.4.2_05/bin/javac$ ls -lL-rwxrwxr-x 1 10016 users 66156 2004-06-04 00:32 javac$ ls -l j2sdk1.4.2_05/bin/javac-rwxrwxr-x 1 10016 users 66156 2004-06-04 00:32 j2sdk1.4.2_05/bin/javac
Xuan Guo CSc 3320 32
In Depth
• Unix files consist of two parts: the data part and the filename part.
• The data part is associated with something called an 'inode'. The inode carries the map of where the data is, the file permissions, etc. for the data.
Xuan Guo CSc 3320 33
In Depth
• The filename part carries a name and an associated inode number.
• More than one filename can reference the same inode number; these files are said to be 'hard linked' together.
Xuan Guo CSc 3320 34
In Depth
• a 'soft link' or a 'symbolic link' is a special file type whose data part carries a path to another file.
Xuan Guo CSc 3320 35
Substituting User
• $ su [-] userName– If userName is not specified, root is assumed– Need access privileges for this– Requires password
• $ sudo command– User can execute command as superuser– Requires password
Xuan Guo CSc 3320 36
Compress/Uncompress
• gzip– $ gzip fileName (.gz)– $ gunzip filename
• tar– $ tar –zcvf filename {filename}*
Xuan Guo CSc 3320 37
Translate Utility (tr)
• $ tr set1 set2• maps all characters in std. input from character
set set1 to the corresponding character in set2• If length of set2 is less, the last character is
repeated.
Xuan Guo CSc 3320 38
Translate Utility (tr)
• $ tr -cds set1 set2– -c option causes string1 to be complemented
(every character not in string1 now is in string1!)– -d option causes every character in string1 to be
deleted from std. input– -s option causes every repeated character in
string1 to be condensed to one occurrence
Xuan Guo CSc 3320 39
Examples of tr
• $ tr a-z A-Z < file1 > file2– causes lower-case to upper-case conversion
• $ tr -c a X < file1 > file2– causes every non-a character to be replaced by X
(including newline)• $ tr -c a-z '\012' < file1 > file2
– causes all non alphabetic characters to be replaced by ASCII 12 (newline)
• $ tr -d a-c < file1 > file2– causes all a, b, c to be deleted from file1
Xuan Guo CSc 3320 40
Comparing Files
• What if you have two files with the same name, but different sizes?
• $ cmp file1 file2 offset1 offset2– compares two files for equality– reports the first byte where there is a mismatch– if one file is a prefix of the other, EOF message is
displayed
Xuan Guo CSc 3320 41
Comparing Files
• cmp -ls file1 file2 offset1 offset2• Optional values offset1 and offset2 are the
offsets into the files where comparison begins.– -l option displays the line number and byte offset
of all mismatched bytes– -s option suppresses all output, return exit status
only (0, 1, >1)
Xuan Guo CSc 3320 42
Comparing Files
• $ diff -i file1 file2– compares two files – outputs a description of their differences– -i flag ignores case
– (a for add, c for change, or d for delete)