More Unix Utilities(find, diff/cmp, at/crontab, tr)

CS465

The find command

• The find command walks down a subtree of the file system looking for files which match certain criteria:– wildcards– file type (e.g. file, directory, special, link)– file’s size, owner, access time / modification

time– etc..

Format of the find command

$ find [start-dir] [criteria] [what-to-do]

• start-dir defines the directory to start from.

– Each subdirectory of this directory will be searched

• criteria is what criteria to use to find the files

– Most common – by filename

• what-to-do tells what to do with the files found. Can be any Unix command.

– Most common: - print to a file

- display to the screen (default)

Starting Directories

• Most often used starting directories are:

$HOME users home directory

. current directory

/ root directory (searches whole

system)

Some find criterion options

-name pattern by filename (can use wildcards)

-perm oct by octal permission settings

-user/-group by owner or group

-size/-type by size or type

-atime/-mtime by last date accessed or modified

-links by number of links to the file

! by negated criteria

find examples

$ find ~ -name projX -print– Starting in my home directory, find any file whose

name is projX and print their full pathnames

$ find /etc -user root -ls– Find all files under /etc owned by root and

display in a long listing format

$ find $HOME ! -type l- Starting in my home directory, display (default)

all files that are NOT link-type files

find examples$ ls -Fpattern.txt pattern2.txt regisdir/$ ls regisdirpattern.txt p.txt

$ find . –name pattern.txt./pattern.txt./regisdir/pattern.txt

$ find . –type f./pattern.txt./pattern2.txt./regisdir/p.txt./regisdir/pattern.txt

$ find . –type d../regisdir

find examples

$ ls -l-rw-rw-r-- 1 jones design 199191 May 7 2004 pattern.txt

-rwxrwxr-x 1 jones play 0 Feb 8 13:52 pattern2.txt

drwxrwxr-x 2 smith design 2048 Feb 8 13:42 regisdir$ ls -l regisdir-rwxrwxr-x 1 smith design 120 Feb 1 10:44 p.txt

$ find . –user jones

./pattern.txt

./pattern2.txt

$ find . –group design

./pattern.txt

./regisdir

./regisdir/p.txt

find examples$ ls -l-rw-rw-r-- 1 jones design 199191 May 7 2004 pattern.txt

-rwxrwxr-x 1 jones play 0 Feb 8 13:52 pattern2.txt

drwxrwxr-x 2 smith design 2048 Feb 8 13:42 regisdir$ ls -l regisdir-rwxrwxr-x 1 smith design 120 Feb 1 10:44 p.txt-rw-rw-r-- 1 smith design 120 Feb 1 10:45 p2.txt

$ find . –perm 775

./pattern2.txt

./regisdir

./regisdir/p.txt

$ find . –perm 664

./pattern.txt

./regisdir/p2.txt

Searching the Entire System

• When you search from root, redirect the error messages to a file

– Prevents seeing error message for every directory you don't have permission to view

Root search Example$ find / -name test -printfind: cannot read dir /lost+found: Permission deniedfind: cannot read dir /usr/lost+found: Permission deniedfind: cannot read dir /usr/dodd/progs Permission deniedfind: cannot read dir /usr/jmsmith Permission denied/usr/kcjones/new/test/usr/lhlester/scripts/testfind: cannot read dir /usr/mkfox Permission deniedfind: cannot read dir /usr/pjsmith Permission denied/usr/wsfrank/test$$ find / -name test -print 2> errmsgs/usr/kcjones/new/test/usr/lhlester/scripts/test/usr/wsfrank/test$

find Filename Wildcards

• You can use filename wildcards with the name option. But if you do, you need to ADD either:

– a backslash in front of the wildcard -OR-

– single quotes around the filename containing the wildcard

to prevent wildcard expansion by the shell (you want the find command to do the expansion of the wildcard, not the shell).

find Wildcarding Example

$ find $HOME –name 'p*.c' –print

/usr/home/jmsmith/prog1.c

/usr/home/jmsmith/hmwk/pass.c

$

$ echo $HOME

/usr/home/jmsmith

$

More find examples$ find $HOME -atime +30 -print

– Display all files within your directories that have not been accessed within the last 30 days

NOTE: 30 accessed exactly 30 days ago-30 accessed less than 30 days ago+30 accessed more than 30 days ago

$ find /usr/jsmith -name '*.ksh'

- Display all Korn shell files in jsmith’s directory structure.

Remember, -print is the default action if none is specified

Combining find criteria

Criteria can be combined using –a and –o and parentheses (backslashed)

-a Logical AND

-oLogical OR

Example: Display all files that have a .ksh extension and have been accessed within the last 30 days.

$ find $HOME \( -name '*.ksh' -a -atime \-30 \) -print

find examples

$ touch soccer.doc

$ ls -F

apples apples.c oranges oranges.c

pattern.txt pattern2.txt regisdir/ soccer.doc$ ls regisdirOther pattern.txt whole.doc

$ find . –name '*.doc' –o –name '*.txt'

./regisdir/pattern.txt

./regisdir/whole.doc

./pattern.txt

./pattern2.txt

./soccer.doc

Additional find actions

If you want to perform an action that is not in the list of find options, you can use find’s -exec option to execute ANY Unix command:

-exec command {} \;

Notes: {} (curly braces)represent the files found

\; (backslash semicolon)ends the exec command

-exec Examples

Find all file ending in .c and move them to the cprogs subdirectory.

$ find $HOME -name '*.c' \ -exec mv {} $HOME/cprogs \;

Find all files within your directories that have not been accessed within the last 30 days and delete them

$ find $HOME -atime +30 –exec rm {} \;

find examples$ ls -F

pattern.txt pattern2.txt regisdir/

$ ls regisdir

$

$ find . –name 'pattern*' –exec cp {} regisdir \;

$ ls regisdir

pattern.txt pattern2.txt

$ find regisdir –name '*2.txt' –exec rm {} \;

$ ls regisdir

pattern.txt

find Exercise Starting in your home directory, search for all files that have

been accessed within the last 3 days and print their full pathnames.

$ find $HOME -atime -3 -print

Expanding the previous exercise, search for all files that have been accessed within the last 3 days and that have a .ksh extension and copy them to subdirectory “current”, under $HOME.

$ find $HOME \( -name '*.ksh' -a -atime -3 \) –exec cp {} $HOME/current \;

The diff command• diff compares two files• Format:

$ diff [options] file1 file2• Options:

-b ignore repeating blanks

-i ignore case

-w ignore ALL spaces and tabs

• Output indicates which lines need be added (a), deleted (d) or changed (c).

• Lines in file1 are identified with a (<) symbol and lines in file2 with a (>) symbol.

diff command output format• The output from diff is an ed editor script that would convert the

first file to the second file if run with the first file as input.

AdditionsfirstStart a secondStart, secondStop

> lines from the second file to add to the first fileDeletions

firstStart, firstStop d lineCount< lines from the first file to delete

ChangesfirstStart, firstStop c secondStart, secondStop

< lines in the first file to be replaced

> lines in the second file to be used for the replacement

diff Example

$ cat file1hellohello$ cat file2hello$$ diff file1 file22d1< hello$$ diff file2 file11a2> hello$

Using diff output with ed

$ cat file1

hello

hello

$$ diff file1 file2 | ed file1

$ cat file1

hello

$$ diff file1 file2

$

diff examples$ cat team1 $cat team2

Mike Mike

John Scott

Kathy Kathy

Martin Bob

$ diff team1 team2

2c2

< John

---

> Scott

4c4

< Martin

---

> Bob

$ cat team3 $ cat team4

Scott Mike

Scott

$ diff team3 team40a1

> Mike

$ diff team4 team3

1d0

< Mike

Using diff with directories

$ ls sub*sub1:file1 prog.c testsub2:file1 memo prog.c$

Shows which files are unique to each directory

$ diff sub1 sub2Only in sub2: memoOnly in sub1: test$

The cmp command

• cmp also compares two files

• Instead of line by line, like diff, compares files character by character until it finds a character mismatch.

• Outputs the character number and line number where the FIRST mismatch was discovered.

• Example:

$ cmp filev1 filev2filev1 filev2 differ: char 60, line 7$

cmp command• Format:

$ cmp [options] file1 file2• Option:

-s work silently; print nothing, but return exit codes, as follows:

0 Files are identical

1 Files are different

2 Files are inaccessible

cmp examples$ cat team1

Mike

John

$ cat team2

Stephen

Scott

$ cmp team1 team2

team1 team2 differ: char 1, line 1

$ cp team1 team3

$ cmp team1 team3

$

$ cmp –s team1 team3 && echo "identical"

identical

cmp command option

• Can also create a listing of ALL character differences in the file, using -l option.

– Column 1: character number (byte number) within the file

– Column 2: ASCII code of character within first file

– Column 3: ASCII code of character within second file

cmp Example

$ ls file1

Hello

Today is Monday

$ ls file2

Hello

Today is Friday

$ ls file3

Hello

$

$ cmp file1 file2 16 115 106 17 157 162 18 156 151$$ cmp file1 file3cmp: EOF on f3$

Process Scheduling

• Two Unix utilities let you schedule commands to run at specific times.

–crontab - lets you schedule commands that need to run on a periodic basis.

–at - lets you schedule a command to run only once.

The crontab file

• The crontab command uses a crontab file, which contains a list of commands and when to run them.

• Commands in your crontab file can be Unix utilities, shell scripts, or executable programs.

– If the command generates output, the output should be re-directed to a file or terminal

• Each user may register ONE crontab file (any name).

– Once the file is registered, the system runs the commands at the scheduled times.

Crontab file specs

• A crontab file contains 6 columns:– Column 1 contains: minute (0-59)– Column 2 contains: hour (0-23, 24-hour clock)– Column 3 contains: day of month (1-31)– Column 4 contains: month (1-12)– Column 5 contains: weekday (1-7, where 1 = Monday)– Column 6 contains: command to run

 • Columns 2 to 5 may also contain a range of hours,

days, or months, or the asterisk symbol. The asterisk says to match ALL hours, days, or months.

Sample crontab file

• First line will run a the script file called monthly_cleanup.ksh at 0300 (3 am) on the 1st of every month.

• Second line will display "Lunchtime!" to terminal 5, Monday through Friday, at 1145 (11:45 am)

$ cat crontab.def

0 3 1 * * $HOME/monthly_cleanup.ksh

45 11 * * 1-5 echo Lunchtime!! > /dev/tty5

$

Invoking crontab

• After you create a crontab file, you must register it using the crontab command, so the system will begin running your commands.

• Once your file is registered, you can list its contents using the crontab -l command.

• You can stop the system from running the commands by un-registering the crontab file using the crontab -r

crontab Example 1

Remove .tmp files from home directory every morning at 6 am:

$ cat crontab.def0 6 * * * rm ${HOME}/*.tmp$ crontab crontab.def$

crontab Example 2$ cat cronfile

0 3 1 * * $HOME/monthly_cleanup.ksh

45 11 * * 1-5 mail $USER < lunchmsg

$ crontab -l0 3 1 * * $HOME/monthly_cleanup.ksh45 11 * * 1-5 mail $USER < lunchmsg

$ crontab cronfile

$ crontab -r

$ crontab -l$

crontab Example 3

# crontab –l jsmith0 3 1 * * $HOME/removeTmps# crontab –l mjones30 11 * * 1-5 mail $USER < lunchmsg#

• Root can view ANY user’s registered crontab file:

cron Output

• cron usually mails output and errors to the cron user. Other methods:

• Method 1 - Discard output

10 * * * * cronscript > /dev/null 2> &1

• Method 2 - Mail output to someone else

10 * * * * cronscript | mail jsmith

The at utility• If you only want to run a command once (instead of on

a regular schedule), but at a specific time, you can use the at command.

• Syntax:$ at time < scriptfile

• Time field can contain:– specific date and time– date/time relative to the current time– month names, day names, and keywords like now, tomorrow, next

at options• Options when running at:

-c, -k and -s tell shell to run the command under a specific shell (C, Korn, or Bourne)

NOTE: Not available on acadunix -- all automatically run under Bourne shell

-m tells the system to send you mail when the command has finished running

• Options after a job has been scheduled: 

-l displays scheduled jobs

-r job# de-schedules the specifed job

at Examples• Run script1 at 2:00 am next Saturday

$ at 0200 sat < script1

• Run script2 an hour from now

$ at now + 1 hour < script2

• Display scheduled jobs

$ at –l

990259200.a Sat May 19 02:00:00 2001

990231999.a Fri May 18 18:26:39 2001

$

More at Examples• Run script3 at 5:00 pm tomorrow

$ at 5pm tomorrow < script3

• Run script4 at 10:00 am on October 1st

$ at 1000 Oct 1 < script4

• Cancel job scheduled for Saturday$ at –l990259200.a Sat May 19 02:00:00 2003990231999.a Fri May 18 18:26:39 2003$ at -r 990259200.a$ at –l990231999.a Fri May 18 18:26:39 2003$

Re-scheduling at

• You can make a script re-schedule itself by making the last command in the script be another at command.

• Example:$ cat deltmp.ksh#! /bin/kshrm ${HOME}/*.tmpat now +1 day < rmtmp.ksh$ at 6am tomorrow < rmtmp.ksh$

Longer Re-scheduling Script

• Check to see if a specific user is logged on, and pipe the output to the checkfile file (if the user is NOT logged on, no file will be created).

• Check if the checkfile file exists.

• If the file does NOT exist, the script will re-schedule itself to run again in 10 minutes.

• If the file does exist, the script will display a message on terminal 5 (assumed to be your terminal) and delete checkfile.

Script Contents$ cat check#!/bin/ksh# $1 = userid of person to check for#who | fgrep $1 > checkfileif [ -s checkfile ]then echo $1 is logged on now > mailmsg mail small000 < mailmsg rm checkfile mailmsgelse at now +10 minutes < $HOME/check $1fiexit 0$

Permissions for at and crontab

• The system administrator can allow or deny both at and crontab use for each user on the system.

• The files that control access are located in the directory /usr/lib/cron

– crontab is controlled by files cron.allow and cron.deny

– at is controlled by files at.allow and at.deny.

• If no allow files exist, then only the users listed in the deny files are denied access.

tr utility

• Translates characters in the standard input, via substitution or deletion of characters

• Format:

tr [options] [string1] [string2]

• Meaning:

– Match characters in string1 and substitute characters from string2.

tr Example #1

• Using tr for case conversion:

$ cat namesJoe SmithMary Jones$ tr '[A-Z]' '[a-z]' < namesjoe smithmary jones$

tr Example #2

• Using tr to convert spaces to newlines:

$ cat namesJoe SmithMary Jones$ tr ' ' '\012' < namesJoeSmithMaryJones$

tr –d option• The –d option deletes all characters listed in string1

• Using tr to delete spaces and lowercase vowels:

$ cat namesJoe SmithMary Jones$ tr –d 'aeiou ' < names > names.tr$ cat names.trJSmthMryJns$

tr –s option• The –s option strips out repeated characters listed in string1

• Using tr to strip out blank lines:

$ cat namesJoe

Mary$ tr –s '\012' < names JoeMary$