Introduction to Unix – CS 21

Lecture 5

Lecture Overview Lab Review Useful commands that will

illustrate today’s lecture Streams of input and output File redirection Piping between different programs Putting it all together for some

powerful tools

Important Points From Lab Get used to both modes of chmod Understand how directory

permissions affect access Write permission only doesn’t let you

do anything Start thinking about using

wildcards to speed up tasks

Good Commands To Know: wc Stands for word count

Will report the number of lines, words, and characters in a file

Works with all sorts of files, but only makes sense with text files

Useful for all sorts of applications Either file direction or piping is

required…

Example Of wc

sort Will sort the lines of a file into

alphabetical order Useful on data files or output from

programs Usage: sort [OPTIONS] [FILE]

Example Of sort

uniq Will go through a file and remove

duplicate copies of a line Will only remove duplicate copies of

lines that are adjacent Useful on files that might have

repeated unnecessary data Usage: uniq [OPTIONS] [FILE]

Example Of uniq

Wouldn’t It Be Nice If We Could sort And uniq A File? No command exists that does both By the end of the lecture, we

should see if this is possible Do you think this will be possible? Hint: Remember Unix was designed

so that tasks could be easily automated and so that tasks could work together

What Exactly Is Input And Output? Input

Everything a Unix program reads Usually from the keyboard

What you type Output

Everything a Unix program writes Usually to the screen

What you see

Streams Of Input And Output stdin

Input from the keyboard stdout

Output to the screen stderr

Error messages that show up on the screen

stdin Everything you type into the

terminal The programs mostly expect all of

their input from the user (you) typing at the keyboard

stdout The results of a running program Goes to the terminal you ran the

program from

stderr Appears the same location as

stdout Used for error messages

Allows for the error messages to be separated from normal output

Redirecting These Streams Deciphering “echo ‘hi’ > helloFile” >

Sends all output that normally would go to stdout into a file

Completely overwrites whatever file was there or creates a new file if none existed

Example Of >

A Tricky Catch To Remember You must be careful because ‘>’

will overwrite files

Redirecting stdin <

Makes all of the input for a file come from a file as opposed to the keyboard

Allows you to input a lot of data without a lot of typing

Exercise: Go to lab and run cat without any parameters

Example Of <

Redirection Of Output Without File Overwrite What if you want to add information

to a file instead of overwriting it? >> Appends all output of a program to a

file Will create a file if it doesn’t exist, but

won’t overwrite any information currently in a file

Example Of Append Redirection

One More Stream Don’t forget about stderr 2>

Not some weird smiley This says redirect stderr to a file Important to note why this notation exists

0 = stdin 1 = stdout 2 = stderr

Example Of Redirecting stderr

Example Of The Difference Between stdout And stderr

Is This Ever Useful? Short answer: Yes! Longer answer:

Whenever you want to save the output of a program, redirect it

Whenever you want to avoid typing input, redirect it

Whenever you want to isolate any error messages, redirect it

Possible Uses For File Redirection Saving data generated by one

program for use by another If you continually type the same

input into different programs, place that data into a file and redirect it into the different programs

How To Remember Which Arrow Does What cat < inputFile

Think of this as the inputFile dumping all of its contents into cat

In other words, inputFile is going into cat cat > outputFile

In this case, cat is dumping its results into outputFile

In other words, cat is going into outputFile

What About Other Command Flags? The input or output file must come directly

after the redirection operator Example:

Right: ls –l * > listingOutput Wrong: ls > -l * listingOutput Question: How do you think this line is interpreted?

As a general rule of thumb, redirection should come at the end of a line Redirection should come after all arguments to

the program to avoid problems like the above

Combining These Flags Entirely possible to have both <

and > on the same line Each redirection operator assumes

the next argument is what is being redirected

Question: Can you have > and >> on the same line?

Answer

What Does This Command Do? ls /bin/doesntExist 2>> errorFile Appends stderr to the end of

errorFile

What About These Commands?

Piping: Tying Programs Together We can combine some of the

previous commands together with a pipe

Pipes feed the output of one program into another

| Yes, that’s a vertical bar

What Does A Pipe Look Like?

sort And uniq Answering our previous question,

we see that it should be possible to both sort and uniq a file at once using pipes

How exactly would we do it?

sort And uniq With Redirection

Same Example With Piping

How To Read Pipes You can have more than one pipe

per line if you’d like Always read left to right, and

output flows left to right

Putting It All Together What if we want to have both file

redirection as well as piping? Well, things start to look ugly, but

this is entirely doable

Complex Example

Really Complex Example

Is All This Stuff Worth It? Well, let’s compare this with how you do

things in Windows With piping, several tasks can be combined

and run in one single step What if I wanted to sort and uniq a Word

file? Open the file and highlight all of the text Select Table->Sort (not intuitive) Specify how to sort and click o.k. Word doesn’t have a uniq feature…

Slowly, Your Power Grows With this information, you are

slowly becoming a Unix power user Maybe you’ll save the world from

dinosaurs yet…

Next Time More thorough explanation of

wildcards Regular expression overview grep – what exactly is it? Quiz #1