Shell Scripting Practicum

James Rising

March 10, 2011

1 Getting Linux

Columbia Remote ssh [uni]@cunix.columbia.edu

Personal Server DreamHost, Hosting Zoom, existenciaInstall It Ubuntu, openSUSE, gNewSense, GentooMac OS X Done. (Terminal)Windows Install http://www.cygwin.com/.

2 Navigating Around

$$ l smbox pub l i c html ssp$ l s −a. . Xauthority . cy ru s t t . matlab . s ta ta9 pub l i c html. . . ba sh h i s t o ry . forward . p r o f i l e mbox ssp$ l s −alFt o t a l 100

1

drwx−−s−−x 6 jar2234 user 4096 Feb 16 15 :56 . /drwxr−sr−x 745 root s t a f f 61440 Feb 11 13 :00 . . /−rw−−−−−−− 1 jar2234 user 51 Feb 3 14 :13 . Xauthority−rw−−−−−−− 1 jar2234 user 107 Feb 3 15 :13 . ba sh h i s t o ry−rw−−−−−−− 1 jar2234 user 92 May 23 2010 . cy ru s t t−rw−−−−−−− 1 jar2234 user 32 Apr 13 2010 . forwarddrwx−−S−−− 3 jar2234 user 4096 Nov 12 17 :20 . matlab/−r−xr−xr−x 1 jar2234 user 172 Apr 13 2010 . p r o f i l e ∗drwxr−sr−x 5 jar2234 user 4096 Feb 3 14 :13 . s ta ta9 /−rw−−−−−−− 1 jar2234 user 1485 Apr 13 2010 mboxdrwxr−sr−x 2 jar2234 user 4096 Apr 13 2010 pub l i c html /drwx−−S−−− 2 jar2234 user 4096 Feb 16 15 :56 ssp /$ cd ssp /$ l sreadme . txt$ cp readme . txt donotread . txt$ l sdonotread . txt readme . txt$ rm donotread . txtrm : remove r e gu l a r f i l e ‘ donotread . txt ’ ? y$ mv readme . txt donotread . txt$ mkdir f i l e s$ l sdonotread . txt f i l e s$ cd f i l e s$ l s −a. . .$ cd . .$ mv donotread . txt f i l e s /$ l s f i l e s /donotread . txt$ rmdir f i l e s /rmdir : ‘ f i l e s / ’ : F i l e e x i s t s$ mv f i l e s / donotread . txt .$ rmdir f i l e s /$ l sdonotread . txt$ pwd/u/9/ j / jar2234 / ssp$ cd ˜$ pwd/u/9/ j / jar2234

Fundamental Symbols:Spaces are manditory between arguments

. The current directory

.. The parent directoryYour home directory

/ The directory separator (e.g. /projects/scripting)- By convention, introduces optional arguments* Expands to all files in directory (also use as wildcard, e.g. ls *.txt)\ Escape the previous special character (e.g. , (, ), ", ’)

Navigation Commands:

2

ls <dir, ...> List directory -a (all files), -l (long format), -F (classifier char), -1 (one-per-line)cd [dir] Change directory (use TAB!)cp [from, ...] [to] Copy file -r (recursive)mv [from, ...] [to] Move/Rename file (works great on directories)rm [file, ...] Delete file -r (recursive), -f (force)mkdir [dir] Create directoryrmdir [dir] Delete directory (only if empty; otherwise use rm -r)pwd Print current directory

Keystrokes:TAB Auto-complete within directory/pathC-a / C-e Go to beginning/end of commandC-k / C-y ”Kill” to end of line, ”Yank” from killC-c Kill current commandC-z Sleep current command (fg to resume)↑ Show previous commands

3 The Linux Environment

$ env. . .$ echo $PATH/opt/ local /bin : / usr / bin : / usr /ucb : / usr /openwin/bin :$ which pwd/opt/ local /bin /pwd$ cp /opt/ local /bin /pwd l s$ PATH=.:$PATH$ echo $PATH. : / opt/ local /bin : / usr / bin : / usr /ucb : / usr /openwin/bin :$ l s/u/9/ j / jar2234 / ssp$ PATH=/opt/ local /bin : / usr / bin : / usr /ucb : / usr /openwin/bin : .$ l sdonotread . txt l s$ . / l s/u/9/ j / jar2234 / ssp$ man l sUser Commands LS(1 )

NAMEl s − l i s t d i r e c t o r y contents

3

SYNOPSISl s [OPTION ] . . . [ FILE ] . . .

. . .$ man bash. . .

Common Variables:PATH /opt/local/bin:/usr/bin:/usr/ucb:... Path for finding commands, separated by :

LD LIBRARY PATH /usr/openwin/lib:/opt/local/lib Path for finding libraries when compilingMANPATH /opt/local/man:/usr/share/man:... Path to man filesUSER jar2234 Current user’s usernameHOME /u/9/j/jar2234 Current user’s home directory (same as ~)PWD /u/9/j/jar2234/ssp Current directory (same as pwd)TZ US/Eastern Timezone (a rough region)PS1 \$ Command-line promptSHELL /bin/bash path to sh, csh, ksh, bash, tcsh, zshHISTFILE /u/9/j/jar2234/.bash history Previous sessions’ command history

Environment Commands:env List variables and their values[VAR]=[VALUE] Set a variable$[VAR] “Expand” to a variable’s valueecho ... Print whatever you say; useful to see variable values (e.g. echo $PATH)which [cmd] Where is the command you would call?whereis [cmd] Where *might* a command be?man [cmd] Display help for a command (the man page)

4 Input and Output

4

$ cat donotread . txtLine 1 : This i s a test .Line 2 : That was a test .$ head −n 1 donotread . txtLine 1 : This i s a test .$ cat donotread . txt | head −n 1Line 1 : This i s a test .$ l s −a ˜ | head −n 5.. .. Xauthority. ba sh h i s t o ry. cy ru s t t$ l s / usr / bin | t a i l −n 5z ipgrepz i p i n f oz ipnotez i p s p l i tzsh$ l s / usr / bin | more. . .$ l s / usr / bin > usrb in . txt$ l sdonotread . txt usrb in . txt$ more usrb in . txt. . .$ cat donotread . txt usrb in . txt | head −n 5 > dnrub . txt$ cat dnrub . txtLine 1 : This i s a test .Line 2 : That was a test .7 z7za7 zr$ cat donotread . txt >> dnrub . txt$ cat dnrub . txtLine 1 : This i s a test .Line 2 : That was a test .7 z7za7 zrLine 1 : This i s a test .Line 2 : That was a test .$ grep ”Line 1” dnrub . txtLine 1 : This i s a test .Line 1 : This i s a test .$ grep ”Line 1” dnrub . txt > l i n e 1 s . txt$ d i f f donotread . txt l i n e 1 s . txt2c2< Line 2 : That was a test .−−−> Line 1 : This i s a test .$ l s / usr / bin | grep ” t e s t ”n i s t e s ttest

Pipes:

5

[cmd] | [cmd] Send output from one program to another (may be used multiple times)[cmd] > [file] Write output to a file[cmd] >> [file] Append output to the end of a file[cmd] < [file] Read input from a file[cmd] < [file] > [file] Both input and output from files[cmd] 2> [file] Write errors to a file

You also need to know how to install new software. Go to http://www.gnu.org/software/

wget/.

$ mkdir i n s t a l l$ cd i n s t a l l /$ wget http :// f tp . gnu . org /gnu/wget/wget−1.12. ta r . gz−−2011−02−17 19:26:47−− http :// f tp . gnu . org /gnu/wget/wget−1.12. ta r . gzReso lv ing f tp . gnu . org . . . 1 40 . 1 86 . 70 . 20Connecting to f tp . gnu . org | 1 4 0 . 1 8 6 . 7 0 . 2 0 | : 8 0 . . . connected .HTTP reques t sent , awai t ing response . . . 200 OKLength : 2464747 ( 2 . 3M) [ app l i c a t i o n /x−gz ip ]Saving to : ‘ wget−1.12. ta r . gz ’

100%[======================================>] 2 ,464 ,747 3 .05M/ s in 0 .8 s

2011−02−17 19 : 26 : 48 (3 . 05 MB/ s ) − ‘ wget−1.12. ta r . gz ’ saved [2464747/2464747 ]

$ gunzip wget−1.12. ta r . gz$ l swget−1.12. ta r$ ta r xvf wget−1.12. ta rwget−1.12/wget−1.12/INSTALL. . .$ l swget−1.12 wget−1.12. ta r$ cd wget−1.12$ l sABOUT−NLS GNUmakefile NEWS con f i gu r e m4 s r cAUTHORS INSTALL README con f i gu r e . ac maint .mk t e s t sCOPYING MAILING−LIST a c l o c a l .m4 con f i gu r e . bat md5 u t i lChangeLog Make f i l e .am autogen . sh doc msdos windowsChangeLog .README Makef i l e . in bui ld−aux l i b po$ more INSTALL. . .$ . / c on f i gu r e −−p r e f i x=$HOMEcon f i gu r e : c on f i gu r i n g for GNU Wget 1 .12check ing for a BSD−compatible i n s t a l l . . . / opt/ local /bin / i n s t a l l −c. . .$ make. . .$ make check. . .$ make i n s t a l l. . .

Now waste your time on http://uni.xkcd.com/.

6

5 Basic Scripts

5.1 How to Write a Script

do.sh

#!/ bin / bash# Do nothing , and say i t ’ s done .

echo ”Done ! ”

$ emacs do . sh$ chmod a+x do . sh$ . /do . shDone !

5.2 Conditionals

doif.sh

#!/ bin / bash# Do nothing , but on ly i f we say so .

i f [ $1 == ”go” ]then

echo ”Went ! ”else

echo ”Nope ! ”f i

$ . / d o i f . sh aaNope !$ . / d o i f . sh goWent !$ . / d o i f . sh. / d o i f . sh : [ : ==: unary operator expectedNope !

7

doif2.sh

#!/ bin / bash# Do nothing , but on ly i f we say so .

i f [ $# −g t 0 ]then

i f [ $1 == ”go” ]then

echo ”Went ! ”else

echo ”Nope ! ”f i

e lseecho ”What?”

f i

Logical Operations:[num] -eq [num], -ne Numeric equal to, not-equal to[num] -lt [num], -le, -gt, -ge Numeric <, ≤, >, ≥[str] == [str], [str] != [str] String equal to, not-equal to-f [file], -d [dir] Check if a file/directory exists[exp] -a [exp], [exp] -o [exp] logical AND or OR! [exp] logical NOT( [exp] ) Group together expressions

5.3 Loops

$ dateThu Feb 18 15 : 21 : 37 EST 2011$ date +%Y2011$ date +%s1298053381

wait.sh

#!/ bin / bash# Wait f o r one hour

now=‘date +%s ‘l a t e r =‘expr 3600 + $now ‘

while [ ‘ date +%s ‘ − l t $ l a t e r ]do

now=‘date +%s ‘echo ”Waiting” ‘expr $ l a t e r − $now ‘ ” seconds ”

done

myls.sh

#!/ bin / bash# Check what kind each f i l e i s

i f [ $# −eq 0 ]

8

thenf i l e s =∗

elsef i l e s=$∗

f i

for f i l e in $ f i l e sdo

i f [ −f $ f i l e ]then

echo ”FILE” $ f i l ee l i f [ −d $ f i l e ]then

echo ”DIR” $ f i l eelse

echo ”Unknown”f i

done

$ FILES=∗$ echo $FILESbomb cat s readme . txt

Control Structures:‘[expr]‘ Execute a command and return the result like you typed it there.i f [ [ cond ] ]then

do somethingf i

Conditional execution

i f [ [ cond ] ]then

do somethinge l i f [ [ cond ] ]then

do something. . .else

do somethingf i

Full form of an if-expression

while [ [ cond ] ]do

do somethingdone

Repeat something until a condition is false

until [ [ cond ] ]do

do somethingdone

Repeat something until a condition is true

for [ var ] in [ l i s t ]do

do somethingdone

Set [var] to each element of the list and run loop

for ( ( i =1; $i <10; i ++)); dodo something

doneUse C-style syntax for loops

9

6 Regular Expressions

Relevant Terminology: RegExp; Basic, Extended, Perl; Subject, Pattern, Match; Modifiers; [Cap-turing] Groups; Substitute; Greedy

Available RegEx Engines:perl, ruby, bash (v3), javascript str = /pattern/, [[ str = pattern ]], str.match(/pattern/)php, Java, .NET, scheme preg match, java.util.regex, System.Text.RegularExpressions, pregexp.scmgrep, awk, sed, expr basic/extended regular expressionsonline http://regex.powertoy.org/

Using grep: grep -E [-o] ”regexp” files...

$ grep ”<meta” example . html<meta http−equiv=”Content−Type” content=” text /html ; cha r s e t=utf−8” /><meta http−equiv=”Content−Language” content=”en” />

10

. . .$ grep −o −E ”[0−9]+” example . html72008. . .$ grep −o −E ’ property=” [ ˆ ”]+” ’ example . htmlproperty=”og : u r l ”property=”og : l ong i tude ”

Using perl:

$ p e r l −n −e ’/ the \ s+(\w+?)\ s / i && pr in t ”$1\n” ’ example . htmlp r o f e s s o r sNew. . .$ p e r l −n −e ’/(\w+)\ s+−−\s+(\w+)/ && pr in t ”$1 $2\n” ’ example . htmlin f i n a n c i a l199. . .$ p e r l −npe ’ s /\w+/why/ ’ example . html<!why HTML>

<why>Columbia Un ive r s i ty − Morningside Heights − New York , NY</ t i t l e >. . .

Basic Elements:Syntax Engine Explanation

x Most letters match themselves

. . matches any character

[xyz] Match any character in the set [...]

[a-z] A dash makes a span of characters; To match -, put at end

[^xyz] Match anything except for the characters in the set

(...) Group expressions together-- good for using +, n, etc.

xyz|zyx Matches either xyz or zyx

\ ‘‘escapes’’ next character, so it stands for itself (+ ? . * ^ $ ( ) [ ] { } | \)

Duplication:Syntax Engine Explanation

* Match an expression 0 or more times

+ Extended Match an expression 1 or more times

x? Possibly match (0 or 1 times) an expression

n Match an expression exactly n times

n,m Match an expression between n and m times

n, Match an expression at least n times

,m Match an expression between n and m times

Modifications:Syntax Engine Explanation

*? Extended Perform a multiple-match non-greedily

/.../g Perl Global replacements

/.../i Perl Case-insensitive matching

/.../m Perl Multi-line mode (each line matches ^ and $

Boundaries:

11

Syntax Engine Explanation

^ Matches the beginning of a line

$ Matches the end of a line

\<, \> Basic; Perl: \b Match beginning/end of word

Character Classes (use capital letter to mean “anything but ...”):Syntax Engine Explanation

\w [A-Za-z0-9_], [:alnum:] Matches ‘‘word’’ characters

\s [\t\n\r\f], [:space:] Matches whitespace

\d [0-9], [:digit:] Matches digits

Some Examples:es tests

..s tests

[se] tests

[0-9]+ A 32nd

[^aeiou] aloha

t.*t tests

.+s tests

^$

\<the\> There was the word.

<.*> vs. <.*?> <a><img /></a>

\d\D+ vs. (\d\D)+ 1 2 3, go!

[-+]?[0-9]*\.?[0-9]+ A 32.56\% increase

[A-Z0-9._%+-]+@[A-Z0-9.-]+\.[A-Z]{2,4} Contact jar2234@columbia.edu

[0-9]{4}[-/](0[1-9]|1[012])[-/](0[1-9]|[12][0-9]|3[01]) For 2010-02-18

<[^>]+>(.*?)<[^>]+> <td>blue</td>

<([\w+]+).*?</\1> <td><blink>red</blink></td>

<a href="([^"])">(.*?)</a> Click <a href=’’link.html’’>here</a>!

^(?n:(?(\d{1,5}(\ 1\/[234])?(\x20[A-Z]([a-z])+)+ )|(P\.O\.\ Box\ \d{1,5}))\s{1,2}

(?i:(?(((APT|B LDG|DEPT|FL|HNGR|LOT|PIER|RM|S(LIP|PC|T(E|OP))|TRLR|UNIT)\x20\w{1,5})|

(BSMT|FRNT|LBBY|LOWR|OFC|PH|REAR|SIDE|UPPR)\.?)\s{1,2})?)(?[A-Z]([a-z])+(\.?)(\x20[A-Z]([a-z])+){0,2})

\, \x20(?A[LKSZRAP]|C[AOT]|D[EC]|F[LM]|G[AU]|HI|I[ADL N]|K[SY]|LA|M[ADEHINOPST]|N[CDEHJMVY]|

O[HKR]|P[ARW]|RI|S[CD] |T[NX]|UT|V[AIT]|W[AIVY])\x20(?(?!0{5})\d{5}(-\d {4})?))$

7 Examples

7.1 Extracting Data

Using wget: wget -O [output] [url]

$ wget −O l i n k s . html http :// dev . cards . e x i s t e n c i a . org / cards−−2011−02−17 11:51:52−− http :// dev . cards . e x i s t e n c i a . org / cardsReso lv ing dev . cards . e x i s t e n c i a . org . . . 6 9 . 163 . 203 . 182Connecting to dev . cards . e x i s t e n c i a . org | 6 9 . 1 6 3 . 2 0 3 . 1 8 2 | : 8 0 . . . connected .HTTP reques t sent , awai t ing response . . . 200 OKLength : 632279 (617K) [ t ex t /html ]Saving to : ‘ l i n k s . html ’

12

100%[======================================>] 632 ,279 401K/ s in 1 .5 s

2011−02−17 11 : 52 : 05 (401 KB/ s ) − ‘ l i n k s . html ’ saved [632279/632279 ]

$ more l i n k s . html. . .$ grep ”/ cards /show/” l i n k s . html

<td><a h r e f=”/ cards /show/1”>Show</a></td><td><a h r e f=”/ cards /show/2”>Show</a></td><td><a h r e f=”/ cards /show/3”>Show</a></td>

. . .$ grep ”/ cards /show/” l i n k s . html | head −n 1

<td><a h r e f=”/ cards /show/1”>Show</a></td>$ grep −o −E ”/ cards /show/[0−9]+” l i n k s . html/ cards /show/1/ cards /show/2/ cards /show/3/ cards /show/4. . .$ grep −o −E ”/ cards /show/[0−9]+” l i n k s . html | head −n 10 | t a i l −n 1/ cards /show/10

7.2 Data from the Web

$ wget ”http ://www. wunderground . com/ h i s t o r y / a i r p o r t /OMDB/1996/1/1/CustomHistory . html?dayend=1&monthend=1&yearend=2011&format=1”. . .$ cd . . / dlop /$ . / dlop . sh OMDB 1996. . .$ l s data/OMDB/page1996 . csv page1999 . csv page2002 . csv page2005 . csv page2008 . csvpage1997 . csv page2000 . csv page2003 . csv page2006 . csv page2009 . csvpage1998 . csv page2001 . csv page2004 . csv page2007 . csv page2010 . csv$ more data/OMDB/page1996 . csv

GST,Max TemperatureF ,Mean TemperatureF ,Min TemperatureF ,Max Dew PointF ,MeanDew PointF ,Min DewpointF ,Max Humidity , Mean Humidity , Min Humidity , Max Sea Level Pressure In , Mean Sea Leve l PressureIn , Min Sea Leve l PressureIn , Max V i s i b i l i t yM i l es , Mean V i s i b i l i t yM i l e s , Min V i s i b i l i t yM i l e s , Max Wind SpeedMPH, Mean Wind SpeedMPH, Max Gust SpeedMPH, Pr e c i p i t a t i on In , CloudCover , Events , WindDirDegrees<br />1996 −7 −1 ,109 ,96 ,84 ,73 ,59 ,34 ,52 ,31 ,10 ,29 .53 ,29 .51 ,29 .47 ,6 ,6 ,5 ,17 ,7 , , 0 .00 ,1 , , 321 <br />1996 −7 −2 ,111 ,96 ,82 ,79 ,62 ,43 ,56 ,33 ,13 ,29 .62 ,29 .56 ,29 .50 ,6 ,6 ,6 ,15 ,6 , , 0 .00 ,1 , , 337 <br />. . .$ grep −o −E ”ˆ[0−9 ,.−]+” data/OMDB/page1996 . csv | more1996 −7 −1 ,109 ,96 ,84 ,73 ,59 ,34 ,52 ,31 ,10 ,29 .53 ,29 .51 ,29 .47 ,6 ,6 ,5 ,17 ,7 , , 0 . 00 ,1 , , 3211996 −7 −2 ,111 ,96 ,82 ,79 ,62 ,43 ,56 ,33 ,13 ,29 .62 ,29 .56 ,29 .50 ,6 ,6 ,6 ,15 ,6 , , 0 . 00 ,1 , , 337. . .$ grep −h −o −E ”ˆ[0−9 ,.−]+” data/OMDB/∗ > omdb . csv$ l s − lt o t a l 396drwx−−S−−− 3 jar2234 user 4096 Feb 17 17 :02 data−rwx−−x−−x 1 jar2234 user 344 Feb 17 17 :02 dlop . sh−rwx−−x−−x 1 jar2234 user 305 Feb 17 17 :02 dlop . sh˜−rw−−−−−−− 1 jar2234 user 386229 Feb 17 17 :11 omdb . csv

13

$ head −n 2 omdb . csv1996 −7 −1 ,109 ,96 ,84 ,73 ,59 ,34 ,52 ,31 ,10 ,29 .53 ,29 .51 ,29 .47 ,6 ,6 ,5 ,17 ,7 , , 0 . 00 ,1 , , 3211996 −7 −2 ,111 ,96 ,82 ,79 ,62 ,43 ,56 ,33 ,13 ,29 .62 ,29 .56 ,29 .50 ,6 ,6 ,6 ,15 ,6 , , 0 . 00 ,1 , , 337$ t a i l −n 2 omdb . csv2010 −12 −30 ,77 ,67 ,57 ,43 ,35 ,25 ,48 ,32 ,17 ,30 .21 ,30 .13 ,30 .06 ,1 ,1 ,1 ,12 ,5 , , 0 . 00 , , , 672010 −12 −31 ,78 ,66 ,55 ,43 ,31 ,12 ,42 ,28 ,8 ,30 .15 ,30 .06 ,30 .01 , , , , 9 , 5 , , 0 . 00 , , , 84

dlop.sh

#!/ bin / bash# Download many f i l e s , by parameter

i f [ ! −d data ]then

mkdir dataf imkdir data/$1

year=$2

while [ $year − l t ‘ date +%Y‘ ]do

next=‘expr $year + 1 ‘wget −O data/$1/page$year . csv ”http ://www. wunderground . com/ h i s t o r y / a i r p o r t /$1

/ $year /1/1/CustomHistory . html?dayend=1&monthend=1&yearend=$next&format=1”year=$next

done

dcol.sh

#!/ bin / bash# Co l l a t e the r e s u l t s in var ious f i l e s

rm −r f datesmkdir dates

master=$1

lnum=1while [ $lnum − l e ‘wc − l omdb . csv | grep −o −E ”[0−9]+” ‘ ]do

l i n e =‘head −n $lnum $master | t a i l −n 1 ‘date=‘echo $ l i n e | grep −o −E ”ˆ[0−9−]+” ‘echo $datedate comma=”$date , ”

for d i r in data /∗do

grep −h $date comma $d i r /∗ >> ” dates / $date . csv ”done

lnum=‘expr $lnum + 1 ‘done

$ . / dco l . sh omdb . csv1996−7−11996−7−2. . .

14

$ l s dates1996−7−1. csv 1996−7−15. csv 1996−7−20. csv 1996−7−26. csv 1996−7−6. csv1996−7−10. csv 1996−7−16. csv 1996−7−21. csv 1996−7−27. csv 1996−7−7. csv$ cat dates /1996−7−1. csv1996 −7 −1 ,91 ,84 ,77 ,77 ,73 ,72 ,94 ,72 ,52 ,30 .07 ,30 .02 ,29 .95 ,15 ,12 ,8 ,8 ,2 , , 0 . 00 ,3 , Fog ,175<br />1996 −7 −1 ,109 ,96 ,84 ,73 ,59 ,34 ,52 ,31 ,10 ,29 .53 ,29 .51 ,29 .47 ,6 ,6 ,5 ,17 ,7 , , 0 .00 ,1 , , 321 < br />

7.3 Google Bombing

$ wget http ://www. goog le . com/ search ?q=sdds−−2011−02−16 21:38:38−− http ://www. goog l e . com/ search ?q=sddsReso lv ing www. goog l e . com . . . 7 4 . 1 25 . 226 . 114 , 74 . 125 . 226 . 115 , 74 . 125 . 226 . 116 , . . .Connecting to www. goog l e . com | 7 4 . 1 2 5 . 2 2 6 . 1 1 4 | : 8 0 . . . connected .HTTP reques t sent , awai t ing response . . . 403 Forbidden2011−02−16 21 : 38 : 39 ERROR 403 : Forbidden .

$ wget −U f i r e f o x −e robots=o f f http ://www. goog l e . com/ search ?q=sdds−−2011−02−16 21:40:50−− http ://www. goog l e . com/ search ?q=sddsReso lv ing www. goog l e . com . . . 7 4 . 1 25 . 226 . 112 , 74 . 125 . 226 . 113 , 74 . 125 . 226 . 114 , . . .Connecting to www. goog l e . com | 7 4 . 1 2 5 . 2 2 6 . 1 1 2 | : 8 0 . . . connected .HTTP reques t sent , awai t ing response . . . 200 OKLength : un sp e c i f i e d [ t ex t /html ]Saving to : ‘ s earch ?q=sdds ’

[ <=> ] 39 ,684 −−.−K/s in 0 .07 s

2011−02−16 21 : 40 : 50 (519 KB/ s ) − ‘ s earch ?q=sdds ’ saved [ 3 9684 ]

$ grep ” Sus ta inab l e Development” search \?q\=sdds$$ p e r l −p − i −e ” s / h r e f=\” [ˆ\ ”]+\”/ h r e f=\”http :\/\/ b logs . c u i t . columbia . edu\/ sdds \/\”/g” search \?q\=sdds$ more search \?q\=sdds. . .

8 Odds and Ends

j r i s i n g $ scp handout . tex jar2234@cunix . columbia . edu :˜/ ssp / rexp/ test . texKerberosV Password :handout . tex 100% 22KB 21 .7KB/ s 00 :00j r i s i n g $ i f [ [ ” t e s t ” =˜ t ( .+) t ]]> then> echo ${BASHREMATCH[ 1 ] }> f ies

$ sudo su −j a r2234 i s not in the sudoers f i l e . This i n c i d en t w i l l be reported .$ f i nd . −name ” ∗ . csv ”. / dlop /omdb . csv. / dlop / dates /1996−7−1. csv. . .$ f i nd . −name ” ∗ . csv ” −exec grep ”2010−1−1,” {} \ ; −pr in t2010 −1 −1 ,77 ,70 ,62 ,61 ,53 ,43 ,83 ,62 ,29 ,29 .98 ,29 .94 ,29 .89 ,6 ,6 ,5 ,14 ,5 , , 0 . 00 ,2 ,. / dlop /omdb . csv

15

2010 −1 −1 ,77 ,70 ,62 ,61 ,53 ,43 ,83 ,62 ,29 ,29 .98 ,29 .94 ,29 .89 ,6 ,6 ,5 ,14 ,5 , , 0 . 00 ,2 , Rain−Thunderstorm ,189<br />. / dlop /data/OMDB/page2010 . csv. . .

16