Pipelines and Filters in Unix

Posted On 2005-11-1 by FortyPoundHead
Keywords: Unix Tutorial Part 07
Tags: Tutorial Linux
Views: 1523

Section 7: Pipelines and Filters

CONCEPT: UNIX allows you to connect processes, by letting the
standard output of one process feed into the standard input of another
process. That mechanism is called a pipe.

Connecting simple processes in a pipeline allows you to perform
complex tasks without writing complex programs.

EXAMPLE: Using the more command, and a pipe, send the contents
of your .profile and .shrc files to the screen by typing
.profile .shrc | more
to the shell.

EXERCISE: How could you use head and tail in a pipeline to
display lines 25 through 75 of a file?

ANSWER: The command
cat file | head -75 | tail
would work. The cat command feeds the file into the
pipeline. The head command gets the first 75 lines of the file, and
passes them down the pipeline to tail. The tail command then filters
out all but the last 50 lines of the input it received from head. It
is important to note that in the above example, tail never sees the
original file, but only sees the part of the file that was passed to
it by the head command.

It is easy for beginners to confuse the usage of the input/output
redirection symbols < and >, with the usage of the pipe.
Remember that input/output redirection connects processes with
while the pipe connects processes with other


The grep utility is one of the most useful filters in UNIX. Grep
searches line-by-line for a specified pattern, and outputs any line
that matches the pattern. The basic syntax for the grep command is
grep [-options] pattern [file]. If the file argument is
omitted, grep will read from standard input. It is always best to
enclose the pattern within single quotes, to prevent the shell from
misinterpreting the command.

The grep utility recognizes a variety of patterns, and the pattern
specification syntax was taken from the vi editor. Here are some of
the characters you can use to build grep expressions:

  • The carat (^) matches the beginning of a line.

  • The dollar sign ($) matches the end of a line.

  • The period (.) matches any single character.

  • The asterisk (*) matches zero or more occurrences of the previous

  • The expression [a-b] matches any characters that are
    lexically between a and b.

EXAMPLE: Type the command
grep 'jon'
to search the /etc/passwd file for any lines
containing the string "jon".

EXAMPLE: Type the command
grep '^jon'
to see the lines in /etc/passwd that begin with
the character string "jon".

EXERCISE:List all the files in the /tmp directory owned by the
user root.

EXPLANATION: The command
ls -l /tmp | grep
would show all processes with the word "root"
somewhere in the line. That doesn't necessarily mean that all the
process would be owned by root, but using the grep filter can cut
the down the number of processes you will have to look at.

About the Author

FortyPoundHead has posted a total of 1974 articles.

Comments On This Post

No comments on this post yet!

Do you have a thought relating to this post? You can post your comment here. If you have an unrelated question, you can use the Q&A section to ask it.

Or you can drop a note to the administrators if you're not sure where you should post.

Your IP address is:

Before you can post, you need to prove you are human. If you log in, this test goes away.

Code Links