Introduction to UNIX Part 3

Redirection and plumbing

Overview

Standard streams

From Wikipedia :

In computer programming, standard streams are preconnected input and 
output communication channels between a computer program and its 
environment when it begins execution. The three I/O connections are 
called standard input (stdin), standard output (stdout) and standard 
error (stderr).

Why does it matter how things are for computer programming? UNIX has a very incestuous relationship with programming. The C language was written to make porting UNIX to other platforms easier. Many aspects of UNIX are influenced, or related to programming.

Redirection

We have already seen one use of the cat(1) command to write the contents of a file to the screen. Now type cat(1) without specifing a file to be read

1% cat

Now type a few words and press the [Return] key. Finally hold the [Ctrl] key down and press d (written as ^D) to end the input.

What happened? If you execute the cat(1) command without specifing a file to read, it reads from stdin (standard input) - in this case, the keyboard, and on receiving the ’end of file’ (^D), copies it to stdout (standard output) - in this case, the screen.

Because UNIX is so flexible, we can redirect both the input and the output of commands, even at the same time.

Redirecting the Output

We use the > symbol to redirect the output of a command. For example, to create a file called list1 containing a list of fruit, type:

1% cat > list1

Then type in the names of some fruit. Press [Return] after each one.

1pear
2banana
3apple
4^D (Recall that this means press [Ctrl] and [d] to stop)

What happens is the cat command reads the standard input (the keyboard) and the > redirects the output, which normally goes to the screen, into a file called list1

To read the contents of the file, type:

1% cat list1

Exercise 3a

Using cat(1), create another file called list2 containing the following fruit: apple, cherry, bananna, apricot. Display the contents of list2

Appending to a file

The symbol >> appends standard output to a file. We can use it to add more items to the file list1, type:

1% cat >> list1

Now type in the names of more fruit

1lime
2guava
3passion fruit
4^D

To read the contents of the file, type:

1% cat list1

You should now have two files. One contains six fruit, the other contains four fruit.

We will now use the cat command to join (concatenate, this is where cat(1) gets it name) list1 and list2 into a new file called big_list. Type:

1% cat list1 list2 > big_list

This tells cat(1) to read the contents of list1 and list2 in turn, then output all of the text to the file ‘big_list’

To read the contents of the new file, type:

1% cat big_list

Redirecting the Input

We use the < symbol to redirect the input of a command. By default, input is read from the keyboard. Using < we can collect input from a file, or another commands output.

The command sort(1) alphabetically or numerically sorts a list. Type:

1% sort

Then type in the names of some animals. Press [Return] after each one.

1jackal
2tiger
3osprey
4monkey
5^D

The output will be

1jackal
2monkey
3osprey
4tiger

Using < you can redirect the input to come from a file rather than the keyboard. For example, to sort(1) the list of fruit, type:

1% sort < big_list

and the sorted list will be output to the screen.

To output the sorted list to a file, type:

1% sort < big_list > sorted_list

Use cat(1) to read the contents of the file slist

Pipes

To see who is on the system with you, type:

1% who

One method to get a sorted list of names is to type:

1% who > names.txt
2% sort < names.txt

This is a bit slow and you have to remember to remove the temporary file called names.txt when you have finished. What you really want to do is connect the output of the who command directly to the input of the sort command. This is exactly what pipes do. The symbol for a pipe is the vertical bar |

For example, type:

1% who | sort

You will get the same result as above, but quicker and cleaner (you won’t have to remember to remove any files, and you only have a “single” command to type).

To find out how many users are logged on, type:

1% who | wc -l

Exercise 3b

Using pipes, display all lines of list1 and list2 containing the letter ‘p’, and sort the result.

1% cat list1 list2 | grep p | sort