Are you looking for Embedded Systems Course – Contact ETDA Experts

Embedded Tech Development Academy

Facebook Instagram Linkedin

9845870904

Linux IPC: Pipes, FIFOs and Data Flow Control

Explore Linux IPC using pipes and FIFOs, including kernel buffers, blocking I/O, atomic writes, synchronization, file descriptors, and practical examples. Embedded Tech Development Academy (ETDA).

Table of Contents

Inter-Process Communication in Linux: Pipes, FIFOs, and Data Flow Control

Introduction to Linux Inter-Process Communication

Modern Linux systems run multiple processes that must exchange information, coordinate execution, and share the results of computations. Examples include shell commands connected through pipelines, embedded Linux services communicating with monitoring tools, and background processes transferring sensor data to logging applications. Inter-Process Communication (IPC) provides mechanisms for this exchange, including pipes, FIFOs, signals, sockets, shared memory, and message queues.

Among these mechanisms, anonymous pipes and named pipes (FIFOs) are particularly useful because they integrate directly with Linux file descriptors and stream-based input/output. They allow one process to produce data while another consumes it, without requiring a shared file for intermediate results. Understanding pipe buffers, blocking and non-blocking I/O, EOF handling, atomic writes, file descriptor management, and I/O multiplexing is essential for building reliable Linux applications.

These concepts are especially relevant to embedded Linux development, where lightweight daemons, device-monitoring tools, shell pipelines, and control applications often communicate under resource constraints. At Embedded Tech Development Academy (ETDA), Linux system programming can form part of practical embedded technology learning. Learners researching the Top Embedded Training Institute in Bangalore can prioritize hands-on experience with Linux commands, system calls, and process synchronization, while assured placement support can help them prepare for embedded software development opportunities.

1. Overview of IPC Mechanisms in Linux

Linux provides several IPC mechanisms, each designed for different communication requirements.

Common IPC Types

  • Pipes: Transfer byte streams between processes, commonly related processes.

  • FIFOs: Named pipe objects that allow independently started processes to exchange data.

  • Signals: Notify a process that an event has occurred.

  • Shared memory: Allows processes to access a common memory region.

  • Message queues: Transfer structured messages through kernel-managed queues.

  • Unix domain and network sockets: Support bidirectional communication between processes.

Pipes and FIFOs are byte-stream mechanisms. They do not automatically preserve application-level message boundaries, so programs must define their own data framing when transferring structured records.

2. Anonymous Pipes: Architecture and Behavior

Creating a Pipe

An anonymous pipe is commonly created with the pipe() system call. It returns two file descriptors: one for reading and one for writing.

#include <unistd.h>

int fd[2];

if (pipe(fd) == -1) {
    /* Handle the error */
}

/* fd[0] is the read end; fd[1] is the write end. */

A parent process can create a pipe and then use fork() to let a child process inherit the file descriptors. The processes can close the unused ends and exchange data through read() and write().

Kernel-Level Data Flow

The kernel maintains the pipe’s data buffer and associated state. When a writer places bytes into the pipe, a reader can retrieve those bytes in sequence. The buffer has a finite capacity, so the producer and consumer must coordinate their data rates.

Blocking and Non-Blocking I/O

By default, reading from an empty pipe blocks while a writer remains open. Writing to a full pipe blocks until sufficient space becomes available. These behaviors provide automatic backpressure and avoid busy-waiting.

A process can use fcntl() to enable non-blocking operation. In that mode, an operation that cannot proceed immediately may return an error such as EAGAIN or EWOULDBLOCK, allowing the application to retry or wait for readiness.

End-of-File and File Descriptor Management

A read() call returns zero when the pipe has no remaining data and all file descriptors referring to its write end have been closed. If a process accidentally keeps an unused write descriptor open, the reader may wait indefinitely instead of receiving EOF.

Correctly closing unused descriptors is therefore a core requirement for reliable pipe communication.

3. Atomic Writes and Data Integrity

Linux guarantees that writes to a pipe or FIFO of up to PIPE_BUF bytes are atomic with respect to other writers. This means data from separate writes within that limit will not be interleaved with data from competing writers.

However, PIPE_BUF is not the same as total pipe capacity. Larger writes may be split or interleaved with other writers’ data, depending on conditions and implementation behavior. Atomicity also does not create message boundaries for readers: a single read() may return part of a write or data from multiple writes.

Applications exchanging records should implement explicit framing, such as fixed-size records or length-prefixed messages, and handle partial reads and writes.

4. FIFOs: Named Pipes for Independent Processes

Creating a FIFO

A FIFO is a special filesystem object created using mkfifo().

#include <sys/stat.h>

if (mkfifo("/tmp/app_fifo", 0660) == -1) {
    /* Handle errors such as EEXIST */
}

After creation, separate programs can open the FIFO by its path and exchange data. Access permissions determine which users or processes can access the object.

FIFO Open Semantics

Opening a FIFO for reading normally blocks until a writer opens it. Opening it for writing normally blocks until a reader opens it. These default behaviors create a form of synchronization between the two processes.

Non-blocking opens behave differently, and the application must handle cases where no communication partner is available.

Multiple Readers and Writers

Multiple processes can open the same FIFO, but the kernel does not provide application-level message framing or guarantee that each reader receives a complete logical record. With multiple readers, bytes are consumed from the shared stream and are not automatically broadcast to every reader.

For reliable structured communication, define a protocol, coordinate writers, and consider whether a socket or message queue is more appropriate.

5. Data Flow Control and Signal Handling

Managing Slow Consumers

When a producer writes faster than a consumer reads, the pipe or FIFO buffer eventually fills. A blocking writer waits until space becomes available, naturally slowing the producer. This behavior is called backpressure.

Non-blocking applications can monitor descriptors using poll(), select(), or epoll() where appropriate. Readiness notification helps applications handle multiple communication channels without continuously checking each one.

Broken Pipes and SIGPIPE

If a process writes to a pipe or FIFO after all read ends have been closed, the kernel normally generates SIGPIPE. If the signal is ignored or handled, the write operation can fail with EPIPE.

Robust applications should handle this situation rather than assuming that a receiver is always available. Error handling should also cover interrupted system calls, closed descriptors, and partial transfers.

6. Design Considerations and Best Practices

Selecting the Right IPC Mechanism

Use anonymous pipes for simple stream communication, especially between related processes. Use FIFOs when independently started processes need a named communication endpoint. For bidirectional communication, network-style connections, or more complex peer management, consider Unix domain sockets.

Reliability and Performance

  • Close unused file descriptors promptly.
  • Check return values from pipe(), mkfifo(), open(), read(), and write().
  • Handle partial reads and writes.
  • Define record boundaries for structured data.
  • Use appropriate file permissions for FIFOs.
  • Apply poll() or related mechanisms when monitoring multiple descriptors.
  • Avoid busy-wait loops and design clear shutdown behavior.

These practices are important in embedded Linux daemons, where descriptor leaks or blocked processes can affect long-running services.

7. Real-World Applications

Pipes and FIFOs support shell pipelines, log processing, command-line tools, monitoring services, and communication between embedded Linux processes. For example, one process can collect device status while another formats and stores the output. A FIFO can also provide a simple command channel between a control utility and a background service.

Embedded Tech Development Academy (ETDA) can help learners build practical skills in Linux system programming, Embedded C, process management, and debugging. Students exploring the Top Embedded Training Institute in Bangalore should look for hands-on practice with system calls and IPC-based projects. Embedded Tech Development Academy (ETDA)‘s assured placement support can help learners prepare for roles involving embedded software and Linux-based systems.

Frequently Asked Questions

What are pipes and FIFOs in Linux?

Both are byte-stream IPC mechanisms. Pipes are commonly used between related processes, while FIFOs are named filesystem objects that can connect independently started processes.

No. Pipes and FIFOs transfer byte streams. Applications must implement framing when they need to identify separate messages.

A blocking writer waits until the reader consumes data and space becomes available. A non-blocking write may instead fail with EAGAIN or EWOULDBLOCK.

PIPE_BUF defines the maximum write size guaranteed to be atomic relative to competing writers. It does not specify the pipe’s total buffer capacity or guarantee matching read boundaries.

The process normally receives SIGPIPE. If the signal is ignored or handled, the write can fail with EPIPE. Applications should handle this condition safely.

Conclusion

Pipes and FIFOs are fundamental Linux IPC mechanisms for transferring data between processes. Anonymous pipes are especially useful for related processes and command pipelines, while FIFOs provide named communication endpoints for independent programs. Their behavior depends on kernel-managed buffering, blocking and non-blocking operations, descriptor ownership, EOF handling, and signal delivery. Correct handling of atomic writes and partial reads is essential when multiple processes exchange data.

For dependable Linux applications, developers should define communication protocols, close unused descriptors, handle broken pipes, and use I/O multiplexing when managing several channels. These skills are valuable in embedded Linux services, monitoring systems, and device-control applications. Embedded Tech Development Academy (ETDA) supports practical learning in embedded technologies, while learners comparing the Top Embedded Training Institute in Bangalore can prioritize Linux system programming and real IPC projects. With assured placement support, Embedded Tech Development Academy (ETDA) aims to help aspiring engineers prepare for embedded software and Linux development opportunities.

Author: ETDA Trainers
Experience: 10+ Years of Industry Experience in Embedded Systems, IoT, and Embedded C Programming

Home