Standard I/O and Pipes

File Descriptors• Linux architecture relays on files as the primary mean of

communication between the different processes and the user interface.

• Each process has it’s own list of open files. Every open file is represented as a ‘file descriptor’ - a numerical ID related to a specific file opened by this process.

• The file descriptor keeps track of the location in the file where the reading or writing took place and advances it as with every character read or written.

File Descriptors and Streams• The first three file descriptor for each process has a

conventional meaning and default binding to the terminal. Standard Input: designated “STDIN” and/or the number “0”; this

descriptor is responsible for accepting data Standard Output: designated “STDOUT” and/or the number “1”; this

descriptor is responsible for sending out normal data Standard Error: designated “STDERR” and/or the number “2”; this

descriptor is responsible for sending out error data

Note: Most processed use standard text as the default data type, but this is not always true. Try this running this command: cat /bin/ls

Note: Terminal is considered a device by the system. It is represented as a device file and can be read from and written to.

Redirecting STDOUT• We can use the “>” or “1>”mark to redirect STDOUT to a file.• “>” is a meta-character which tells the shell to create the

redirection before the program starts, that way when it does – the STDOUT is already set to be a file, hence the output will go there and not be displayed on-screen.

• When redirecting output with “>”, any existing content in the file we redirect to will be completely overwritten.

• Redirection of STDOUT does not affect STDIN or STDERR; errors will keep showing up on the screen by default.

# ls -l file? > file_list# cat file_list -rw-rw-r-- 1 nir nir 135 Jul 19 13:42 file1-rwxrwxr-- 1 nir nir 35 Jul 19 13:42 file2-rw-rw-r-- 1 nir nir 200 Jul 19 13:42 file3

# ls -l file? > file_list# cat file_list -rw-rw-r-- 1 nir nir 135 Jul 19 13:42 file1-rwxrwxr-- 1 nir nir 35 Jul 19 13:42 file2-rw-rw-r-- 1 nir nir 200 Jul 19 13:42 file3

Appending and Overwrite protection

• In order to append the redirected output to an existing file, rather than overwriting it completely, we’d use the “>>” marks:

• We can use a BASH option named “noclobber” to prevent files from being overwritten during redirection by running: set –o noclobber

# ls -l [dk]* >> file_list# cat file_list -rw-rw-r-- 1 nir nir 135 Jul 19 13:42 file1-rwxrwxr-- 1 nir nir 35 Jul 19 13:42 file2-rw-rw-r-- 1 nir nir 200 Jul 19 13:42 file3-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 dfile1-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 dfile2-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 kfile9

# ls -l [dk]* >> file_list# cat file_list -rw-rw-r-- 1 nir nir 135 Jul 19 13:42 file1-rwxrwxr-- 1 nir nir 35 Jul 19 13:42 file2-rw-rw-r-- 1 nir nir 200 Jul 19 13:42 file3-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 dfile1-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 dfile2-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 kfile9

Redirecting STDERR• As discussed before, STDERR is also represented by the

number “2”.• In order to redirect STDERR to a file, we’d use “2>”:

• Redirection of STDERR is not affected by redirections of STDOUT, each is a completely separate stream.

• At times we do not want the STDOUT or STDERR to be redirected anywhere, whether it’s the display or a file, we can redirect either of the streams or both of them into a device file named /dev/null

• Anything redirected to /dev/null will be discarded for good.

# sdtrre 2> error_file# cat error_file -bash: sdtrre: command not found

# sdtrre 2> error_file# cat error_file -bash: sdtrre: command not found

Redirecting STDIN• Input redirections are much less common than STDOUT or STDERR

redirections because many applications already take their input by default from the keyboard or a file.

• The “<“ mark is used to redirect input to a command from a file rather than from the keyboard

# cat input_redirect /tmp/test/# ls -l < input_redirect -rw-rw-r-- 1 nir nir 0 Jul 19 15:11 dfile1-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 dfile2drwxrwxr-x 2 nir nir 4096 Jul 19 13:58 directory-rw-rw-r-- 1 nir nir 33 Jul 20 10:53 error_file-rw-rw-r-- 1 nir nir 135 Jul 19 13:42 file1-rwxrwxr-- 1 nir nir 35 Jul 19 13:42 file2-rw-rw-r-- 1 nir nir 200 Jul 19 13:42 file3-rw-rw-r-- 1 nir nir 336 Jul 20 10:47 file_list-rw-rw-r-- 1 nir nir 11 Jul 20 11:10 input_redirect-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 kfile9-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 mfile1

# cat input_redirect /tmp/test/# ls -l < input_redirect -rw-rw-r-- 1 nir nir 0 Jul 19 15:11 dfile1-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 dfile2drwxrwxr-x 2 nir nir 4096 Jul 19 13:58 directory-rw-rw-r-- 1 nir nir 33 Jul 20 10:53 error_file-rw-rw-r-- 1 nir nir 135 Jul 19 13:42 file1-rwxrwxr-- 1 nir nir 35 Jul 19 13:42 file2-rw-rw-r-- 1 nir nir 200 Jul 19 13:42 file3-rw-rw-r-- 1 nir nir 336 Jul 20 10:47 file_list-rw-rw-r-- 1 nir nir 11 Jul 20 11:10 input_redirect-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 kfile9-rw-rw-r-- 1 nir nir 0 Jul 19 15:11 mfile1

Multiple Redirection• Redirections can be merged so that we can redirect multiple

streams in a single action.• In the following example, we will see how to merge both

STDOUT and STDERR and send them to /dev/null to be discarded

• The above example redirects STDOUT (>) to /dev/null and then binds STDERR to the same fate of STDOUT (2>&1)

Note: ‘/dev/null’ is a special device file. Data that is written into this file gets discarded immediately

# date > /dev/null 2>&1# date > /dev/null 2>&1

Pipes• The Pipe “|” meta-character, indicated to Bash that the

STDOUT of one command should be automatically passed as STDIN to another command

command | command | command …