FreeBSD's VFS Layer: How Multiple Filesystems Share One Interface

FreeBSD applications call the same open(2), read(2), write(2), and stat(2) interfaces whether a path lives on UFS, ZFS, tmpfs, NFS, nullfs, or devfs. The Virtual File System layer makes that possible, but it does not erase the differences between those filesystems. VFS defines common kernel objects and dispatch interfaces; each filesystem supplies implementations and retains its own storage model, locking rules, capabilities, and failure modes.

The useful mental model is a chain. A process owns a file descriptor; the descriptor refers to a kernel file object; for a regular filesystem object, that file object refers to a vnode; the vnode belongs to a mounted filesystem and dispatches operations to that filesystem’s code. Conflating those layers leads to errors such as calling a vnode “one open file.” Many descriptors and file objects can refer to the same vnode, and a vnode can remain active even when no pathname currently reaches it.

Mounts describe filesystem instances; vnodes describe objects

A struct mount represents one mounted filesystem instance and connects it to VFS-wide state and filesystem-specific operations. VFS operations commonly called through VFS_* cover work at filesystem scope: mounting, unmounting, obtaining the root vnode, synchronizing data, and reporting filesystem statistics. Two mounts using the same filesystem implementation still have separate mount structures, options, roots, and backing resources.

A struct vnode is the kernel’s filesystem-independent representation of an active object such as a regular file, directory, symbolic link, device, or socket in a filesystem namespace. It carries a vnode type, mount association, locks and reference state, plus private data interpreted by its filesystem. UFS may associate it with an inode; ZFS uses its own object structures; NFS associates it with remote identity and client state.

Generic kernel code invokes vnode operations through VOP_* dispatch. VOP_READ, VOP_WRITE, VOP_GETATTR, VOP_SETATTR, VOP_LOOKUP, VOP_CREATE, VOP_REMOVE, and VOP_RENAME name common contracts. The selected vnode’s operation vector chooses the actual implementation. That is dynamic polymorphism inside the kernel: the syscall path can request a read without embedding UFS, ZFS, and NFS branches throughout generic code.

The abstraction is not permission to call operations without respecting their contracts. Kernel developers must obey the documented vnode lock state, reference rules, credential handling, and pathname-component semantics. A VOP can sleep, return a filesystem-specific error, or trigger remote I/O. The vnode(9) and individual VOP_*(9) manual pages, not intuition from userspace, define those obligations.

Path lookup walks components and crosses mount points

Opening /usr/local/bin/tool begins with pathname resolution, not a single filesystem lookup. The namei(9) machinery selects a starting directory—root or the process’s current directory—and resolves one component at a time. For each ordinary component it consults the name cache and, when necessary, calls the directory vnode’s VOP_LOOKUP implementation. Permissions, symbolic-link expansion, . and .., mount boundaries, and the final operation’s intent all affect the walk.

VFS mediates mount-point crossings. When lookup reaches a directory covered by another mount, resolution continues from the mounted filesystem’s root vnode. Walking .. from that root requires the inverse transition back across the mount boundary. Applications normally see one namespace even when adjacent components are supplied by unrelated implementations or a remote server.

This common namespace does not make all cross-boundary operations possible. rename(2) is atomic within the applicable filesystem’s rules, but renaming from one mounted filesystem to another fails with EXDEV; userspace must copy and remove instead. Hard links likewise cannot cross filesystems. A nullfs mount can expose an existing subtree elsewhere, while devfs synthesizes device nodes, but each still participates through VFS under its own semantics.

Symbolic links add another security-relevant dimension. Lookup may restart or continue from link contents, and race-resistant applications should prefer descriptor-relative interfaces such as openat(2) where appropriate instead of repeatedly validating text paths. VFS provides the mechanism; correct application use is still necessary to avoid time-of-check/time-of-use bugs.

The name cache accelerates both hits and misses

Path traversal would be expensive if every component forced UFS metadata I/O or an NFS request. FreeBSD’s name cache stores relationships between directory vnodes, component names, and resolved targets. A positive cache entry can avoid a filesystem lookup; a negative entry can remember that a name was absent, which is especially valuable for repeated $PATH searches and probing optional configuration files.

Negative caching makes invalidation just as important as acceleration. After a create, link, unlink, rename, or mount event, stale entries must not keep returning an old vnode or an old “not found.” Filesystem implementations and VFS coordinate cache purges and updates around namespace changes. Cache correctness is part of filesystem correctness, not an optional performance layer.

The name cache is not file data caching. File contents may pass through the virtual-memory system, buffer mechanisms, a filesystem-specific cache such as ZFS ARC, or a remote client’s cache. Seeing a high pathname-cache hit rate says little about whether data reads reach storage. Performance investigations should separate pathname lookup, metadata operations, and data I/O.

Administrators can observe the mounted topology and filesystem types without kernel debugging:

mount -v
df -T
stat -f /path/to/object
procstat -f "$PID"

mount -v reports mounts and options; df -T identifies filesystem types and space accounting; stat -f reports the filesystem containing a path; and procstat -f shows a process’s open descriptors. None alone proves which physical disk served a read—ZFS, GEOM, encryption, network mounts, and caches may add more layers—but together they establish the VFS-visible path.

Vnode lifetime is not pathname lifetime

Unix permits unlinking an open file. The directory entry disappears, yet a process holding the file descriptor can continue reading or writing. This works because the pathname and the active vnode/file-object references are distinct. The filesystem reclaims storage only when namespace links and active references permit it.

That behavior explains the classic “df says full but du cannot find the bytes” incident. A daemon may hold a large deleted log open. du walks reachable names and no longer sees it, while the filesystem still accounts for allocated blocks. On FreeBSD, procstat -f and fstat(1) help identify open files and the responsible process. Restarting the right service or making it reopen logs releases the reference; deleting unrelated files does not address the cause.

Vnodes are cached and recycled under controlled lifecycle rules. Kernel code uses reference and hold mechanisms plus vnode locking so an object is not reclaimed while another thread operates on it. Filesystems implement reclaim/inactive behavior to detach private state at the correct point. A stale raw vnode pointer in a kernel module is therefore a correctness and potentially security-critical bug; ownership must be represented with the documented APIs.

The interface unifies syscalls, not filesystem semantics

VFS supplies broad contracts, but portable code cannot assume every mounted filesystem implements every feature identically. Case sensitivity, timestamp precision, ACL models, extended attributes, sparse-file behavior, quota reporting, stable file identifiers, synchronous-write guarantees, and crash recovery vary. NFS can return errors caused by server state or network partitions that local tmpfs cannot. Read-only mounts reject modifications regardless of a filesystem’s underlying capability.

Even space metrics differ. Copy-on-write snapshots may retain blocks no current pathname references; compression changes physical usage; reservations affect availability; remote servers decide their own accounting. statfs(2) provides a common structure, not an assurance that every field has identical physical meaning across UFS, ZFS, and NFS.

Filesystem-specific features therefore often use ioctls, control utilities, or additional interfaces. ZFS datasets and snapshots cannot be expressed entirely through open and write; NFS mount policy needs network-specific controls. This is not a failure of VFS. A smallest common interface keeps ordinary applications portable while allowing a specialized filesystem to expose capabilities outside that common denominator.

What the abstraction costs—and what it buys

VOP dispatch adds indirect calls and locking/lifetime machinery. Path lookup may traverse cache and mount logic before filesystem code runs. Those costs are real, but claiming that one function-pointer dereference dominates filesystem performance is usually unsupported. Storage latency, network round trips, checksum and compression work, copy-on-write allocation, contention, and cache misses are often far larger. Measure the workload before attributing a bottleneck to “VFS overhead.”

The return is substantial. System calls, descriptor semantics, permission checks, pathname traversal, and process integration can be implemented against stable kernel abstractions. A new filesystem supplies mount and vnode operations instead of rewriting every syscall. Stacking filesystems such as nullfs can reuse another namespace, and pseudo-filesystems can represent kernel-created objects with the same tools users already understand.

For an incident investigator, the layered view prevents premature conclusions: a pathname can cross mounts; a descriptor can outlive its name; a cache can answer without I/O; and a common syscall can end in local, remote, memory-backed, or copy-on-write code. VFS hides those seams from ordinary application logic, but effective diagnosis depends on knowing exactly where they remain.

Related:

Sources: