Capsicum Beyond the Basics: Building Capability-Mode Services From Scratch

Most sandboxing writeups treat Capsicum as a checkbox: call cap_enter(), watch a program keep working, done. That demo undersells what Capsicum is. It is a lightweight capability framework that blends with Unix APIs by making file descriptors explicit tokens of authority and by restricting access to global namespaces. The result is not a policy file outside the program; useful confinement comes from how the program opens, narrows, and delegates resources.

Capsicum first appeared in FreeBSD 9.0. The current capsicum(4) manual describes two core ideas: capability mode, which is entered with cap_enter(2), and capabilities derived from file descriptors. The mode is inherited by descendants and cannot be cleared. This one-way boundary is deliberate: trusted setup code may delegate authority to a less-trusted worker, but the worker cannot later recover the setup process’s ambient namespace access.

The problem Capsicum actually solves

Traditional Unix access control is largely ambient: a process carries credentials such as UID, GID, and supplementary groups, then supplies pathnames or PIDs to global namespaces. Code reached through a parser bug can attempt any operation available to those credentials, even when the parser needed only one input and one output. Dropping UID helps, but it still leaves the complete authority of the replacement account.

In capability mode, unrestricted access to global filesystem, IPC, and PID namespaces is denied or constrained. A call such as open() with a global path fails with ECAPMODE; operations on a descriptor can fail with ENOTCAPABLE when the descriptor lacks a required right. Some system calls remain available because they operate only on process-local state or already-held descriptors. It is therefore more precise to say that global authority is removed than that an arbitrary list of system calls is blocked.

Capsicum also does not revoke memory already mapped into the process, erase secrets already read, or make a dangerous descriptor harmless. Passing a writable directory, a connected administrative socket, or an unrestricted process descriptor into the sandbox delegates exactly that authority. The capability inventory is the security policy.

cap_rights_t: rights live on the descriptor, not the process

The core data type is cap_rights_t, which represents rights such as CAP_READ, CAP_WRITE, CAP_SEEK, CAP_FSTAT, and CAP_LOOKUP. The rights(4) manual explicitly warns that it is not a simple bitmask and must not be assembled with ordinary bitwise OR. Use the cap_rights_init(3) family, then apply the result with cap_rights_limit(2).

cap_rights_t rights;

cap_rights_init(&rights, CAP_READ, CAP_FSTAT, CAP_SEEK);
if (cap_rights_limit(config_fd, &rights) == -1)
    err(1, "cap_rights_limit");

After this call, config_fd can be read, seeked, and inspected with fstat, but not written. Rights can be reduced but never expanded. The kernel checks them on later operations; narrowing is not merely an assertion made by the application. A robust program treats failure to narrow a descriptor as fatal instead of continuing with broader authority.

Descriptor control has two additional dimensions. cap_fcntls_limit(2) restricts which fcntl(2) commands remain available, while cap_ioctls_limit(2) restricts allowed ioctl(2) requests. A network service that narrows read and write rights but leaves an unnecessary family of device ioctls available has not completed its capability analysis.

A minimal service boundary

A useful design separates privileged setup from request processing. The setup phase parses trusted configuration, opens a listening socket, opens required directories, creates logging or IPC channels, and resolves any identities it needs. It then narrows each descriptor, enters capability mode, and invokes the worker loop.

#include <sys/capsicum.h>
#include <err.h>

static void
enter_worker(int input_fd, int output_fd)
{
    cap_rights_t rights;

    cap_rights_init(&rights, CAP_READ, CAP_FSTAT);
    if (cap_rights_limit(input_fd, &rights) == -1)
        err(1, "limit input");

    cap_rights_init(&rights, CAP_WRITE, CAP_FSTAT);
    if (cap_rights_limit(output_fd, &rights) == -1)
        err(1, "limit output");

    if (cap_enter() == -1)
        err(1, "cap_enter");
}

The example is intentionally incomplete: production code must decide what standard input, output, error, inherited sockets, shared memory, and event descriptors should remain open. Before entering capability mode, enumerate descriptors during testing with procstat -f PID. An accidentally inherited descriptor can defeat an otherwise careful design.

Designing around it, not retrofitting it

The naive approach — write a normal program, then try to cap_enter() right before the “risky” part — usually fails immediately, because normal programs assume they can open() paths on demand. Capsicum-aware services are structured differently from the start:

Complete namespace operations before entry. Open required files, directories, sockets, and IPC objects while setup code still has ambient access.
Narrow immediately. Apply descriptor rights, permitted fcntl commands, and permitted ioctls as soon as the descriptor is created.
Close everything else. Capability mode limits namespace lookup; it does not neutralize inherited descriptors.
Enter early and fail closed. If any narrowing operation or cap_enter() fails, do not start the untrusted worker.
Keep the worker interface small. Pass data or narrowly scoped descriptors rather than configuration paths and instructions to reopen resources.

A directory descriptor with CAP_LOOKUP permits constrained relative lookup using openat(2). This is useful for a file-serving worker, but the exact rights needed by the resulting open, path rules, symlink behavior, and application policy must be tested. The safe abstraction is not “paths work normally below this directory”; it is “lookup is delegated through this specific directory capability under Capsicum’s constrained *at semantics.” Avoid textual prefix checks, which are not a substitute for kernel-enforced lookup.

Casper for services that require names

Some operations are inherently namespace-based: DNS resolution, password and group database lookup, selected sysctls, and access to system randomness are common examples. FreeBSD’s libcasper(3) provides capability-oriented service channels for such tasks. A trusted process opens a Casper channel and limits the service before entering capability mode; the worker later sends requests over that descriptor instead of regaining ambient access.

This broker pattern preserves a narrow interface. A DNS service can perform resolution without giving the parser a general socket-creation capability. A password service can answer constrained identity queries without exposing arbitrary filesystem paths. The service still requires policy review: an unrestricted broker that will perform any request is ambient authority hidden behind IPC.

Process descriptors: capability mode for process management too

A subtler piece is pdfork(2), which returns a process descriptor representing the child as well as creating the child process. The descriptor can be waited on with pdwait4(2), signaled with pdkill(2), and queried with pdgetpid(2) when its rights allow those operations. Relevant rights include CAP_PDWAIT, CAP_PDKILL, and CAP_PDGETPID.

This avoids treating a reusable numeric PID as the sole authority reference. A supervisor can delegate “wait for this child” without also delegating “signal this child,” and descriptor lifecycle rules reduce PID-reuse races. Process descriptors do not replace all process management, but they let a capability-oriented design remain descriptor-oriented at that boundary.

Testing the negative space

A sandbox test must prove denials, not merely prove that the happy path still runs. After entry, deliberately confirm that opening an unrelated absolute path fails with ECAPMODE, writing through a read-only capability fails with ENOTCAPABLE, and an unapproved ioctl is rejected. Then verify every legitimate request still succeeds.

Run those tests under the same startup mechanism as production. Service managers, libraries, locale handling, name resolution, and logging can open files lazily after the first request. A program that enters capability mode successfully and fails only when rotating logs is not production-ready. Move that operation to the trusted supervisor or delegate a narrowly designed service.

Capsicum also complements rather than replaces ordinary controls. Run the service under an unprivileged account, limit jails and filesystem permissions, apply resource controls, validate input, and minimize memory-unsafe code. Capsicum answers what a compromised compartment can reach; it does not prevent the initial memory corruption or guarantee the integrity of data intentionally delegated to it.

Where this actually gets used

The FreeBSD Project lists base utilities and daemons including tcpdump, dhclient, hastd, rwhod, and kdump as Capsicum users. Their source code is more valuable than a synthetic demo because it shows compatibility handling, descriptor cleanup, and the relationship with Casper services. Do not infer that every component of a large daemon is sandboxed merely because one execution path calls cap_enter(); inspect the exact version’s source and process architecture.

The value proposition is a reviewable, kernel-enforced answer to “what remains reachable if this worker is fully compromised?” Reaching that answer requires a descriptor inventory, rights tests, constrained brokers, and failure handling. cap_enter() is the irreversible boundary at the end of that design work, not the design itself.

Related:

Sources: