240-491 Adv. UNIX: Filters/4

62
0-491 Adv. UNIX: Filters/4 Advanced UNIX Advanced UNIX Objectives Objectives to discuss five useful filters: to discuss five useful filters: tr tr , , grep grep , , awk awk , , sed sed , and , and find find 240-491 Special Topics in Comp. Eng. 1 Semester 2, 2000-2001 4. Filters (Part II, Sobell)

description

 

Transcript of 240-491 Adv. UNIX: Filters/4

Page 1: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 1

Advanced UNIXAdvanced UNIX

ObjectivesObjectives– to discuss five useful filters:to discuss five useful filters:trtr, , grepgrep, , awkawk, , sedsed, and , and findfind

240-491 Special Topics in Comp. Eng. 1Semester 2, 2000-2001

4. Filters(Part II, Sobell)

Page 2: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 2

1. tr1. tr

format:format:tr [options] string1 [string2]tr [options] string1 [string2]

trtr reads its standard input and translates reads its standard input and translates each character in each character in string1string1 to the to the corresponding character in corresponding character in string2string2

Page 3: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 3

ExamplesExamples

$ echo 12abc3def4 | tr ’abcdef’ ’xyzabc’$ echo 12abc3def4 | tr ’abcdef’ ’xyzabc’12xyz3abc412xyz3abc4

$ echo 12abc3de4 | $ echo 12abc3de4 | tr ’[a-c][d-f]’ ’[x-z][a-tr ’[a-c][d-f]’ ’[x-z][a-

c]’c]’12xyz3abc412xyz3abc4

$ cat foo.txt | tr ’[A-Z]’ ’[a-z]’$ cat foo.txt | tr ’[A-Z]’ ’[a-z]’

Page 4: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 4

$ tr ’\015’ ’ ’ < file1 > file2$ tr ’\015’ ’ ’ < file1 > file2– \015\015 is carriage return is carriage return

$ cat mail.txt | $ cat mail.txt | tr -s ’tr -s ’ปป’ ’ ’ ’ ’ ’ > new-mail.txt> new-mail.txt

– ปป represents tab; could write \011represents tab; could write \011– -s-s means remove duplicates of means remove duplicates of string2string2 in outputin output

$ echo Can you read this? | $ echo Can you read this? | tr -d ’aeiou’tr -d ’aeiou’

Cn y rd ths?Cn y rd ths?

Page 5: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 5

““rot13” Textrot13” Text

$ echo Gur chapuyvar bs gur wbxr vf ... |$ echo Gur chapuyvar bs gur wbxr vf ... |tr ’[N-Z][A-M][n-z][a-m]’ tr ’[N-Z][A-M][n-z][a-m]’

’[A-M][N-Z][a-m][n-z]’’[A-M][N-Z][a-m][n-z]’

The punchline of the joke is ...The punchline of the joke is ...

Popular in 1970-1980’s.

Page 6: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 6

2. 2. grepgrep

Format:Format:grep [options] pattern [file-list]grep [options] pattern [file-list]

Search one or more files, line by line, for Search one or more files, line by line, for a pattern (a regular expression). Actions a pattern (a regular expression). Actions taken depend on options.taken depend on options.

Page 7: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 7

Variants of Variants of grepgrep

grepgrep Uses basic RE patternUses basic RE pattern

ffgrepgrep FastFast grep. Pattern can only be grep. Pattern can only bean ordinary string.an ordinary string.

eegrepgrep ExtendedExtended grep. Pattern can use grep. Pattern can usefull REs.full REs.

Page 8: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 8

grep optionsgrep options

-c-c print a print a countcount of matching lines of matching lines -i-i ignoreignore case in pattern during search case in pattern during search -l-l listlist filenames with match filenames with match -n-n precede each matching line by precede each matching line by

a line a line numbernumber -v-v print lines that do print lines that do notnot match pattern match pattern

Page 9: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 9

ExamplesExamples

File testaFile testa File testbFile testb File testcFile testc

aaabbaaabb aaaaaaaaaa AAAAAAAAAAbbbccbbbcc bbbbbbbbbb BBBBBBBBBBff-ffff-ff cccccccccc CCCCCCCCCCcccddcccdd dddddddddd DDDDDDDDDDdddaadddaa

continued

Page 10: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 10

$ grep bb testa$ grep bb testaaaabbaaabbbbbccbbbcc

$ grep -v bb testa$ grep -v bb testaff-ffff-ffcccddcccdddddaadddaa

$ grep -n bb testa$ grep -n bb testa1: aaabb1: aaabb2: bbbcc2: bbbcc

continued

Page 11: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 11

$ grep bb *$ grep bb *testa: aaabbtesta: aaabbtesta: bbbcctesta: bbbcctestb: bbbbbtestb: bbbbb

$ grep -i bb *$ grep -i bb * $ grep -i BB * $ grep -i BB * testa: aaabbtesta: aaabb testa: aaabb testa: aaabb testa: bbbcctesta: bbbcc testa: bbbcc testa: bbbcc testb: bbbbbtestb: bbbbb testb: bbbbb testb: bbbbb testc: BBBBBtestc: BBBBB testc: BBBBBtestc: BBBBB

Page 12: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 12

Fancier PatternsFancier Patterns

$ grep ’fun..ion’ file$ grep ’fun..ion’ file

$ grep -n ’^#define’ file$ grep -n ’^#define’ file

$ grep ’^#de[a-z]*’ file$ grep ’^#de[a-z]*’ file

$ egrep ’while|if’ *.c$ egrep ’while|if’ *.c

$ egrep ’[0-9]+’ *.c$ egrep ’[0-9]+’ *.c

Page 13: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 13

3. awk3. awk

format:format:awk program file-listawk program file-list

awk -f program-file file-listawk -f program-file file-list

awkawk is a pattern scanning and action processing is a pattern scanning and action processing languagelanguage

The action language is very like C.The action language is very like C.

Page 14: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 14

OverviewOverview

3.1. 3.1. Patterns & ActionsPatterns & Actions

3.2. 3.2. awkawk Processing CycleProcessing Cycle

3.3. 3.3. How How awkawk Sees a LineSees a Line

3.4. 3.4. Pattern ExpressionsPattern Expressions

3.5. 3.5. ‘,’ Range Operator‘,’ Range Operator

continued

Page 15: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 15

3.6. 3.6. Many Built-in FunctionsMany Built-in Functions

3.7. 3.7. BEGINBEGIN and and ENDEND

3.8. 3.8. First First awkawk Program File: Program File: pre_headerpre_header

3.9. 3.9. Action LanguageAction Language

3.10. Associative Arrays3.10. Associative Arrays

Page 16: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 16

3.1. Patterns & Actions3.1. Patterns & Actions

An An awkawk program consists of:program consists of:

pattern {action}pattern {action}pattern {action}pattern {action}

::

Page 17: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 17

3.2. awk Processing Cycle3.2. awk Processing Cycle

1. 1. Read next input line.Read next input line.

2. 2. Apply all Apply all awkawk patterns sequentially.patterns sequentially.

3. 3. If a pattern matches, do its action.If a pattern matches, do its action.

4. 4. Go to step (1).Go to step (1).

Page 18: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 18

ExampleExample

$ cat cars$ cat carsplymplym furyfury 7777 7373 25002500chevychevy novanova 7979 6060 30003000fordford mustangmustang 6565 4545 1000010000volvovolvo glgl 7878 102102 98509850fordford ltdltd 8383 1515 1050010500chevychevy novanova 8080 5050 35003500fiatfiat 600600 6565 115115 450450hondahonda accordaccord 8181 3030 60006000fordford thundbdthundbd 8484 1010 1700017000toyotatoyota terceltercel 8282 180180 750750chevychevy impalaimpala 6565 8585 15501550fordford broncobronco 8383 2525 95009500

continued

Page 19: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 19

$ awk ’/chevy/ {print}’ cars$ awk ’/chevy/ {print}’ carschevychevy novanova 7979 6060 30003000chevychevy novanova 8080 5050 35003500chevychevy impalaimpala 6565 8585 15501550

$ awk ’/chevy/’ cars$ awk ’/chevy/’ carschevychevy novanova 7979 6060 30003000chevychevy novanova 8080 5050 35003500chevychevy impalaimpala 6565 8585 15501550

$ awk ’/^h/’ cars$ awk ’/^h/’ carshondahonda accordaccord 8181 3030 60006000

Page 20: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 20

3.3. How 3.3. How awkawk Sees a LineSees a Line

awkawk views each line as a record consisting of views each line as a record consisting of fields separated by spaces.fields separated by spaces.

Each field is referred to by a variable called Each field is referred to by a variable called $<number>$<number>::– $1, $2, $3$1, $2, $3, etc., etc.– $0$0 refers to the whole line (record) refers to the whole line (record)

The current line number is stored in The current line number is stored in NRNR

continued

Page 21: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 21

$ awk ’{print $3, $1}’ cars$ awk ’{print $3, $1}’ cars77 plym77 plym79 chevy79 chevy65 ford65 ford : :83 ford83 ford

$ awk ’/chevy/ {print $3, $1}’ cars$ awk ’/chevy/ {print $3, $1}’ cars79 chevy79 chevy80 chevy80 chevy65 chevy65 chevy

Page 22: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 22

3.4. Pattern Expressions3.4. Pattern Expressions

Format:Format:variable OP patternvariable OP pattern

OPOP forms:forms:– matching:matching: ~~ !~!~– ariithmetic:ariithmetic: < <= == != >= >< <= == != >= >– boolean:boolean: &&&& |||| !!

continued

Page 23: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 23

$ awk ’$1 ~ /h/’ cars$ awk ’$1 ~ /h/’ carschevychevy novanova 7979 6060 30003000chevychevy novanova 8080 5050 35003500hondahonda accordaccord 8181 3030 60006000chevychevy impalaimpala 6565 8585 15501550

$ awk ’$1 ~ /^h/’ cars$ awk ’$1 ~ /^h/’ carshondahonda accordaccord 8181 3030 60006000

continued

Page 24: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 24

$ awk ’$2 ~ /^[tm]/ $ awk ’$2 ~ /^[tm]/ {print $3, $2, “$” $5}’ cars{print $3, $2, “$” $5}’ cars

65 mustang $1000065 mustang $1000084 thundbd $1700084 thundbd $1700082 tercel $75082 tercel $750

$ awk ’$3 ~ /5$/ $ awk ’$3 ~ /5$/ {print $3, $1, “$” $5}’ cars{print $3, $1, “$” $5}’ cars

65 ford $1000065 ford $1000065 fiat $45065 fiat $45065 chevy $155065 chevy $1550

continued

Page 25: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 25

$ awk ’$3 == 65’ cars$ awk ’$3 == 65’ carsfordford mustangmustang 6565 4545 1000010000fiatfiat 600600 6565 115115 450450chevychevy impalaimpala 6565 8585 15501550

$ awk ’$5 <= 3000’ cars$ awk ’$5 <= 3000’ carsplymplym furyfury 7777 7373 25002500chevychevy novanova 7979 6060 30003000fiatfiat 600600 6565 115115 450450toyotatoyota terceltercel 8282 180180 750750chevychevy impalaimpala 6565 8585 15501550

continued

Page 26: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 26

$ awk ’$5 >= “2000” && $ awk ’$5 >= “2000” && $5 < “9000”’ cars$5 < “9000”’ cars

plymplym furyfury 7777 7373 25002500chevychevy novanova 7979 6060 30003000chevychevy novanova 8080 5050 35003500fiatfiat 600600 6565 115115 450450hondahonda accordaccord 8181 3030 60006000toyotatoyota terceltercel 8282 180180 750750

$ awk ’$5 >= 2000 && $5 < 9000’ cars$ awk ’$5 >= 2000 && $5 < 9000’ carsplymplym furyfury 7777 7373 25002500chevychevy novanova 7979 6060 30003000chevychevy novanova 8080 5050 35003500hondahonda accordaccord 8181 3030 6000 6000

Page 27: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 27

3.5. ‘,’ Range Operator3.5. ‘,’ Range Operator

Format:Format:pattern1 , pattern2pattern1 , pattern2

Select a range of lines. Select a range of lines. – the the firstfirst line of the range matches line of the range matches pattern1pattern1– the the lastlast line of the range matches line of the range matches pattern2pattern2

May return several groups of linesMay return several groups of lines

continued

Page 28: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 28

$ awk ’/volvo/ , /fiat/’ cars$ awk ’/volvo/ , /fiat/’ carsvolvovolvo glgl 7878 102102 98509850fordford ltdltd 8383 1515 1050010500chevychevy novanova 8080 5050 35003500fiatfiat 600600 6565 115115 450450

$ awk ’NR == 2 , NR ==4’ cars$ awk ’NR == 2 , NR ==4’ carschevychevy novanova 7979 6060 30003000fordford mustangmustang 6565 4545 1000010000volvovolvo glgl 7878 102102 98509850

continued

Page 29: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 29

$ awk ’/chevy/ , /ford/’ cars$ awk ’/chevy/ , /ford/’ carschevychevynovanova 7979 6060 30003000fordford mustangmustang 6565 4545 1000010000chevychevynovanova 8080 5050 35003500fiatfiat 600600 6565 115115 450450hondahondaaccordaccord 8181 3030 60006000fordford thundbdthundbd 8484 1010 1700017000chevychevyimpalaimpala 6565 8585 15501550fordford broncobronco 8383 2525 95009500

threegroups

Page 30: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 30

3.6. Many Built-in Functions3.6. Many Built-in Functions

length(str)length(str) length of string length of string strstr

lengthlength length of current linelength of current line

split(strings, array, delimitor)split(strings, array, delimitor)split split stringstring into parts based on into parts based on

the the delimitordelimitor, and place in , and place in arrayarray

– split(“a bcd ef g1”, arr, “ “)split(“a bcd ef g1”, arr, “ “)continued

Page 31: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 31

$ awk ’length > 23 {print NR}’ cars$ awk ’length > 23 {print NR}’ cars33991010

Page 32: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 32

3.7. 3.7. BEGINBEGIN and and ENDEND

BEGIN {action}BEGIN {action} executed before first lineexecuted before first lineis processedis processed

END {action}END {action} executed after last lineexecuted after last lineis processedis processed

$ awk ’END {print NR, “cars for sale.”}’ cars

12 cars for sale

Page 33: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 33

3.8. First awk Program File3.8. First awk Program File

$ cat pr_header$ cat pr_header### pr_header# pr_header##BEGIN {BEGIN {print “Make Model Year Miles Price”print “Make Model Year Miles Price”print “---------------------------------”print “---------------------------------” } } {print} {print}

continued

Page 34: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 34

$ awk -f pr_header cars$ awk -f pr_header carsMake Model Year Miles PriceMake Model Year Miles Price------------------------------------------------------------------ plymplym furyfury 77 73 77 73 25002500chevychevy novanova 79 60 79 60 30003000

::::

chevychevy impala 65 85impala 65 85 15501550fordford bronco 83 25bronco 83 25 95009500

Page 35: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 35

redirect_outredirect_out

$ cat redirect_out$ cat redirect_out/chevy/ {print > “chev.txt”}/chevy/ {print > “chev.txt”}/ford/ {print > “ford.txt”}/ford/ {print > “ford.txt”}END {print “done.”}END {print “done.”}

$ awk -f redirect_out cars$ awk -f redirect_out carsdone.done.$ cat chev.txt$ cat chev.txtchevychevy novanova 7979 6060 30003000chevychevy novanova 8080 5050 35003500chevychevy impalaimpala 6565 8585 15501550

Page 36: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 36

3.9. Action Language3.9. Action Language Very C like:Very C like:

– var = exprvar = expr– if (cond) stat1 else stat2if (cond) stat1 else stat2– while (cond) statwhile (cond) stat– for (expr1; cond; expr2) statfor (expr1; cond; expr2) stat– printf “format” expr1, expr2, ...printf “format” expr1, expr2, ...– { stat1 ; stat2; ... ; statN }{ stat1 ; stat2; ... ; statN }

User-defined variables do not need to be declaredUser-defined variables do not need to be declared

continued

Page 37: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 37

Long statements, conditions, expressions Long statements, conditions, expressions may need to be typed over several lines. may need to be typed over several lines.

Use ‘\’ to hide newline:Use ‘\’ to hide newline:if ($3 > 2000 && \if ($3 > 2000 && \ $3 < 3000) print $3 $3 < 3000) print $3

Page 38: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 38

price_rangeprice_range

$ cat price_range$ cat price_range{{if ($5 <= 5000) $5 = “inexpensive”if ($5 <= 5000) $5 = “inexpensive”else if ($5 > 5000 && $5 < 10000) \else if ($5 > 5000 && $5 < 10000) \

$5 = “please ask”$5 = “please ask”else if ($5 >= 10000) $5 = “expensive”else if ($5 >= 10000) $5 = “expensive”printf “%-10s %-8s 19%2d %5d %-12s\n”, \printf “%-10s %-8s 19%2d %5d %-12s\n”, \

$1, $2, $3, $4, $5$1, $2, $3, $4, $5}}

continued

Page 39: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 39

$ awk -f price_range cars$ awk -f price_range carsplymplym furyfury 19771977 7373 inexpensiveinexpensivechevychevy novanova 19791979 6060inexpensiveinexpensive

::::

fordford broncobronco 19831983 2525 please askplease ask

Page 40: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 40

summarysummary

$ cat summary$ cat summaryBEGIN { yearsum = 0 ; costsum = 0BEGIN { yearsum = 0 ; costsum = 0 newcostsum = 0 ; newcnt = 0 } newcostsum = 0 ; newcnt = 0 } { yearsum += $3 ; costsum += $5 } { yearsum += $3 ; costsum += $5 }$3 > 80 { newcostsum += $5 ; newcnt++ }$3 > 80 { newcostsum += $5 ; newcnt++ }END {END { printf “Avg. car age: %3.1f yrs\n”, \ printf “Avg. car age: %3.1f yrs\n”, \

90 - (yearsum/NR)90 - (yearsum/NR) printf “Avg. car cost: $%7.2f\n”, \ printf “Avg. car cost: $%7.2f\n”, \

costsum/NRcostsum/NR printf “Avg. newer car cost: $7.2f\n”, \ printf “Avg. newer car cost: $7.2f\n”, \

newcostsum/newcntnewcostsum/newcnt } }

continued

Page 41: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 41

$ awk -f summary cars$ awk -f summary carsAvg. car age: 13.2 yrsAvg. car age: 13.2 yrsAvg. car cost: $6216.67Avg. car cost: $6216.67Avg. newer car cost: $8750.00Avg. newer car cost: $8750.00

Page 42: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 42

3.10. Associative Arrays3.10. Associative Arrays

Arrays that use strings as indexes:Arrays that use strings as indexes:– array[string] = valuearray[string] = value

Special for-loop for Special for-loop for awkawk arrays:arrays:– for (elem in array) actionfor (elem in array) action

continued

Page 43: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 43

manufmanuf

$ cat manuf$ cat manuf {manuf[$1]++} {manuf[$1]++}END {END { for (name in manuf) \ for (name in manuf) \

print name, manuf[name]print name, manuf[name] } }

continued

Page 44: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 44

$ awk -f manuf cars$ awk -f manuf carshonda 1 honda 1 fiat 1 fiat 1 volvo 1 volvo 1 ford 4 ford 4 plym 1plym 1chevy 3chevy 3toyota 1toyota 1

Page 45: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 45

Sorted OutputSorted Output

Sort by Sort by firstfirst column (i.e. by name): column (i.e. by name):$ awk -f manuf cars | sort$ awk -f manuf cars | sort

Sort by Sort by secondsecond column (i.e. by number): column (i.e. by number):$ awk -f manuf cars | sort +1$ awk -f manuf cars | sort +1

Page 46: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 46

4. sed4. sed Format:Format:

sed ’list of ed commands’ filesed ’list of ed commands’ file

Read lines one at a time from the input fileRead lines one at a time from the input file– apply apply eded commands in order to each line commands in order to each line– write edited line to stdoutwrite edited line to stdout

eded is an old UNIX editor is an old UNIX editor– vivi without full-screen mode without full-screen mode– did you think vi was tough :)did you think vi was tough :)

Page 47: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 47

4.1. Search and Replace 4.1. Search and Replace

The ‘The ‘ss’ command searches for a pattern ’ command searches for a pattern (a regular expression), and replaces it (a regular expression), and replaces it with the new string:with the new string:

’’s/pattern/new-string/g’s/pattern/new-string/g’

– ‘‘gg’ means global (everywhere on line)’ means global (everywhere on line)

Page 48: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 48

ExamplesExamples

$ sed ’s/UNIX/UNIX(TM)/g’ file > new-file$ sed ’s/UNIX/UNIX(TM)/g’ file > new-file

$ sed ’s/^/$ sed ’s/^/ /’ file > new-file/’ file > new-file

– put a tab at the start of every line (no put a tab at the start of every line (no gg needed) needed)

$ sed ’s/[ $ sed ’s/[ ][ ][ ]*/\/g’ file > new-file ]*/\/g’ file > new-file

– replace every sequence of blanks or tabs with a newlinereplace every sequence of blanks or tabs with a newline

– this splits the input into 1 word/linethis splits the input into 1 word/line

continued

Page 49: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 49

$who$whoad tty1 Sep 29 07:14ad tty1 Sep 29 07:14ron tty3 Sep 29 10:31ron tty3 Sep 29 10:31td tty4 Sep 29 08:36td tty4 Sep 29 08:36

$ who | sed ’s/ .* / /’$ who | sed ’s/ .* / /’ad 07:14ad 07:14ron 10:31ron 10:31td 08:36td 08:36

$$

replace a blank and everything that follows it (as much as possible, including more blanks) up to the last blank

Page 50: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 50

More InformationMore Information

sedsed can use most can use most eded commands, not just commands, not just ss

See the entry on See the entry on sedsed in Sobell, p.680-691 in Sobell, p.680-691

Page 51: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 51

5. find5. find

Format:Format:find starting-directory find starting-directory

matching-conditions-and-matching-conditions-and-actionsactions

findfind searches all the directories below the searches all the directories below the starting directory.starting directory.– it carries out the specified actions on the files that it carries out the specified actions on the files that

match the specified conditions match the specified conditions

Page 52: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 52

Assume we are in my home directory, and Assume we are in my home directory, and want to find the want to find the carscars file (used in the file (used in the awkawk examples):examples):

$ find . -name cars -print$ find . -name cars -print./teach/adv-unix/filters/cars./teach/adv-unix/filters/cars$$

Basic ExampleBasic Example

startingpoint

-namecondition

-printaction

Page 53: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 53

-name nm-name nm the filename is the filename is nmnm

-type ty-type ty tyty is a file type: is a file type: ff = file, = file,dd = directory, etc. = directory, etc.

-user usr-user usr the file’s owner is the file’s owner is usrusr

-group grp-group grp the file’s group owner is the file’s group owner is grpgrp

continued

5.1. Some Matching Conditions5.1. Some Matching Conditions

Page 54: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 54

-atime n-atime n file was last accessedfile was last accessedexactly exactly nn days ago days ago

-mtime n-mtime n file was last modifiedfile was last modifiedexactly exactly nn days ago days ago

-size n-size n file is exactly file is exactly nn 512-byte 512-byte blocks longblocks long

Can use + or - to mean more or less.Can use + or - to mean more or less.

Page 55: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 55

5.2. Example Conditions5.2. Example Conditions

-mtime +7-mtime +7 last modified more thanlast modified more than7 days ago7 days ago

-size +100-size +100 larger than 50Klarger than 50K

““And”ing conditions:And”ing conditions:-atime +60 -mtime +120-atime +60 -mtime +120

– files last accessed more than 2 months ago files last accessed more than 2 months ago andand last last modified more than 4 months agomodified more than 4 months ago

continued

Page 56: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 56

““Or”ing Conditions:Or”ing Conditions:\( -mtime +7 -o -atime +30 \)\( -mtime +7 -o -atime +30 \)

– files last modified more than 7 days ago files last modified more than 7 days ago oror last last accedded more than 30 days agoaccedded more than 30 days ago

““Not”Not”-name \*.dat \! -name gold.dat-name \*.dat \! -name gold.dat

– all “.all “.datdat” files except ” files except gold.datgold.dat

Page 57: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 57

5.3. Some Actions5.3. Some Actions

-print-print display pathname of matching filedisplay pathname of matching file

-exec cmd-exec cmd execute execute cmdcmd on file on file -ok cmd-ok cmd prompt before executing prompt before executing

cmdcmd on file on file

Commands must end with Commands must end with \;\; and use and use {} {} to mean to mean the matching file, e.g.:the matching file, e.g.:

-ok rm {} \;-ok rm {} \;

Page 58: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 58

5.4. Examples5.4. Examples

$ find . -name $ find . -name \*.c\*.c -print -print

– Starting from the current directory, display the Starting from the current directory, display the pathnames of all the files ending in “pathnames of all the files ending in “.c.c””

$ find . \( -name core -o -name junk \)$ find . \( -name core -o -name junk \) -print -ok rm {} \; -print -ok rm {} \;

– Print the pathnames of all the Print the pathnames of all the corecore and and junkjunk files in the current directory and below, and files in the current directory and below, and prompt to remove them.prompt to remove them.

continued

Page 59: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 59

$ find /usr -size +100 -mtime +30$ find /usr -size +100 -mtime +30-exec ls -l {} \;-exec ls -l {} \;

– Display a long list of all the files underDisplay a long list of all the files under /usr /usr larger than about 500K that have not been larger than about 500K that have not been modified in a month.modified in a month.

Page 60: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 60

5.5. Problems with Permissions5.5. Problems with Permissions

A A findfind over the entire filesystem will print over the entire filesystem will print many error messages when access is denied many error messages when access is denied to other user’s directories.to other user’s directories.

These error messages (sent to These error messages (sent to stderrstderr) can ) can be be redirectedredirected to to /dev/null/dev/null (a UNIX (a UNIX “black hole”).“black hole”).

Page 61: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 61

ExampleExample

Search for a file/directory called Search for a file/directory called zipzip anywhere anywhere below the root directory:below the root directory:

$ find / -name zip -print$ find / -name zip -printfind: /exports/tmp/code/4210341: Permission deniedfind: /exports/tmp/code/4210341: Permission deniedfind: /exports/tmp/code/4210389: Permission deniedfind: /exports/tmp/code/4210389: Permission deniedfind: /exports/home/suthon/private: Permission find: /exports/home/suthon/private: Permission denieddeniedfind: /exports/home/cj/mail: Permission deniedfind: /exports/home/cj/mail: Permission denied

::::

continued

Page 62: 240-491 Adv. UNIX: Filters/4

240-491 Adv. UNIX: Filters/4 62

Redirect standard errors to the black hole Redirect standard errors to the black hole using using 2>2>

$ find / -name zip -print $ find / -name zip -print 2>2> /dev/null /dev/null/exports/home/s4110068/project/zip/exports/home/s4110068/project/zip/exports/home/s4110316/project/zip/exports/home/s4110316/project/zip/exports/home/s4110316/zip/exports/home/s4110316/zip/exports/home/s4110316/zip/zip/exports/home/s4110316/zip/zip$$