How to Use Named Pipes and Process Substitution Safely

An anonymous pipeline connects processes launched together. A named pipe, or FIFO, gives the kernel pipe a filesystem name so separately started processes can open the same stream.

Create a private FIFO

workdir=$(mktemp -d) || exit 1
fifo=$workdir/events
trap 'rm -rf -- "$workdir"' EXIT HUP INT TERM
mkfifo "$fifo" || exit 1

Using a private directory prevents another user from substituting a different object at a predictable path. The FIFO contains no stored file payload; writers and readers exchange bytes through the kernel while both ends are open.

Start both sides deliberately

Opening only the read end normally blocks until a writer opens the FIFO, and vice versa. Arrange startup so neither side waits forever:

consume_events <"$fifo" &
consumer_pid=$!

produce_events >"$fifo"
producer_status=$?
wait "$consumer_pid"
consumer_status=$?

Check both statuses. If a consumer exits early, the producer may receive SIGPIPE. If a producer never closes the FIFO, a consumer waiting for EOF will never finish. Multiple writers can interleave output beyond the system’s atomic pipe-write limit, so a FIFO is not automatically a record-safe message queue.

Use process substitution for one command graph

In Bash, Zsh, or Ksh, a command that insists on a filename can consume a stream with <(...):

diff <(sort left.txt) <(sort right.txt)

The shell supplies a /dev/fd path or manages a temporary FIFO. This is concise for processes launched within one shell command, but it is not POSIX sh syntax and its child statuses may require explicit handling.

Know when to choose another mechanism

FIFOs provide no durable queue, authentication, replay, schema, or backpressure policy beyond blocking writes. Use Unix-domain sockets for bidirectional local protocols, ordinary files for durable handoff, and a real message broker when delivery guarantees matter. Named pipes are excellent for bounded streaming—not a substitute for every IPC system.

Where the pipe concept itself came from, and why FIFOs came later

The underlying pipe concept — the idea of connecting one program’s output directly to another’s input without an intermediate file — is credited to Douglas McIlroy, who proposed it in the early 1970s; anonymous pipes reached Unix around Version 3 in 1973. Named pipes, giving that same mechanism a persistent filesystem name so unrelated processes (not just a chain launched together on one command line) could rendezvous through it, followed shortly after, appearing in Version 5 by 1974. That’s the direct lineage behind the distinction this guide draws: an anonymous pipe (| in a shell pipeline) connects processes started together in one breath, while a FIFO’s filesystem name is precisely what lets processes started independently, potentially by different users or at different times, find and connect to the same stream.

Setting permissions deliberately, not by accident

mkfifo respects the calling process’s umask exactly the way file creation does — a FIFO created under a permissive default umask can end up readable or writable by users you didn’t intend, especially relevant since anyone able to open the FIFO for reading can potentially see data meant only for a specific consumer. Passing an explicit mode directly (mkfifo -m 600 "$fifo") rather than relying on whatever umask happens to be active is the more deliberate, auditable choice, particularly for a FIFO carrying anything sensitive.

Confirming a stuck FIFO isn’t the actual bug before debugging further

If a script or command using a FIFO appears to simply hang with no output at all, confirm first whether it’s genuinely stuck waiting for the other end to open the pipe (the expected, documented blocking behavior covered above) rather than a separate bug entirely — lsof or fuser against the FIFO’s path shows which processes currently have it open, immediately confirming whether a writer or reader is actually missing versus both ends being present but one side stalled for an unrelated reason.

Fanning one stream out to several independent readers

A single FIFO connects exactly one writer’s bytes to however many readers happen to have it open concurrently, but the kernel doesn’t duplicate the data per reader — bytes are consumed once, by whichever reader happens to read them, splitting the stream unpredictably across multiple simultaneous readers rather than giving each one a full independent copy. To genuinely fan one source out to multiple independent consumers, pair tee with several separate FIFOs instead of trying to share one:

mkfifo "$workdir/a" "$workdir/b"
producer | tee "$workdir/a" >"$workdir/b" &
consumer_a <"$workdir/a" &
consumer_b <"$workdir/b" &

tee writes its standard input to every named destination in full, plus its own standard output, which is what actually gives each downstream reader a complete, independent copy of the stream rather than a fragment of it.

mkfifo versus the older, lower-level mknod

mkfifo is the portable, POSIX-specified way to create a named pipe; the older mknod utility can also create one (mknod path p) but is a considerably lower-level tool historically used for device nodes as well, with a less consistent interface across systems and often requiring elevated privileges for its other, non-FIFO uses. Unless a script specifically needs mknod’s broader device-node capabilities, mkfifo is the clearer, more portable, and more narrowly-scoped choice for the FIFO-creation task this guide covers — it does exactly one thing, and its argument syntax doesn’t vary by what kind of node you’re asking for.

Related:

Sources: