Fixing bhyve VMs That Refuse to Boot With UEFI Firmware

A bhyve guest that fails under UEFI can be broken at four different layers: the host may not load the firmware, the firmware may have no visible console, the virtual disk may contain no EFI boot path, or the guest kernel may lack the presented storage driver. Diagnose those layers in order. Changing CPU, disk, and firmware options simultaneously destroys the evidence that identifies which boundary failed.

Verify the actual command and firmware package

On amd64, the bhyve(8) manual identifies the edk2-bhyve package as the provider of UEFI firmware. Confirm the installed package and locate its files rather than copying a path from a release with a different package layout:

pkg info edk2-bhyve
pkg info -l edk2-bhyve
bhyve -l help

The launch command must include a valid boot ROM, for example:

-l bootrom,/usr/local/share/uefi-firmware/BHYVE_UEFI.fd

Treat that path as an example. Use the filename reported by pkg info -l on the host. Capture the final command emitted by a VM manager because its configuration file may not match what it actually executes. A nonexistent or unreadable ROM normally makes bhyve exit with an error; it does not prove the guest disk is bad.

Some firmware variants use a writable variable-store file to preserve UEFI NVRAM entries. When the selected bhyve and firmware version supports a ROM plus variable file, give every VM its own writable copy of the template. Sharing one variable store among guests risks cross-VM state and concurrent corruption. A read-only or recreated store explains boot entries that disappear after shutdown, while it does not explain a firmware process that never starts.

After a failed invocation, remove only the stale VM instance before retrying:

bhyvectl --vm=myvm --get-stats
bhyvectl --vm=myvm --destroy

Use the real VM name and first confirm it is not still running. Reusing an existing VM context can produce misleading allocation errors unrelated to UEFI.

Separate “no display” from “no boot”

UEFI commonly writes early output to a graphical framebuffer. Add an fbuf device and connect with VNC:

-s 29,fbuf,tcp=127.0.0.1:5900,w=1024,h=768,wait \
-s 30,xhci,tablet

Binding VNC to 127.0.0.1 avoids exposing an unauthenticated console on every host interface; use an SSH tunnel for remote access. The optional wait setting deliberately pauses guest execution until a viewer connects, so forgetting that option is present looks exactly like a hang.

If the firmware menu appears, bhyve, the ROM, and framebuffer are functioning. The fault has moved into boot-device discovery or the guest. If no VNC listener exists, inspect the bhyve process and launch error. If a listener exists but the screen stays blank, test a known matching firmware file and a minimal device set before changing guest storage.

A serial console is useful only when both firmware or guest loader and operating system are configured to use it. Absence of serial output does not establish that UEFI stopped. Conversely, a guest installed for serial-only output may boot successfully while VNC goes quiet after the loader.

Use installation media as a controlled boot probe

Attach a known-good installer ISO through a controller the firmware supports, while keeping the target disk attached separately:

-s 3,ahci-cd,/vm/iso/installer.iso \
-s 4,virtio-blk,/vm/myvm/disk.img

Verify the ISO checksum against its publisher before the test. If UEFI boots the installer, displays it through VNC, and the installer sees the target disk, the ROM, framebuffer, and disk device model are functional. The remaining problem is likely the installed disk’s EFI loader, boot entry, or guest operating system. If the installer boots but cannot see the disk, investigate controller selection and guest drivers.

Do not leave an installer attached indefinitely or allow it to outrank the system disk silently. Record the firmware boot menu’s device list and remove the ISO after testing. A successful ISO boot is a diagnostic boundary, not a repair by itself.

Confirm that the virtual disk is EFI-bootable

UEFI needs a bootable path. A fresh OS installation launched under UEFI normally creates a GPT disk with an EFI System Partition and installs an architecture-specific loader. A disk installed under legacy BIOS may contain only MBR boot code and cannot become UEFI-bootable merely because bhyve now loads a ROM.

Boot the guest’s installer or recovery environment with the same virtual disk controller used for normal startup. Inside that environment, inspect its partition table and EFI System Partition using that operating system’s native tools. For FreeBSD guests, gpart show and a read-only mount of the efi partition can confirm whether EFI/BOOT/BOOTX64.EFI or a configured FreeBSD loader exists.

Do not run host gpart recover, format commands, or bootcode writes against a VM image until its partition layout and backup are confirmed. Reinstalling a small EFI loader is different from rebuilding a partition table, and guessing at the latter can destroy an otherwise recoverable guest.

If firmware sees the disk but has lost its NVRAM boot entry, the standard removable-media path EFI/BOOT/BOOTX64.EFI can provide a fallback on amd64. Otherwise use the firmware menu or guest installer to recreate the entry. This is where a persistent, per-guest variable store matters.

Match the emulated controller to guest drivers

A loader can run and then fail when the kernel cannot attach its root disk. Compare the normal launch command with the controller used during installation:

-s 4,virtio-blk,/vm/myvm/disk.img
-s 5,virtio-net,tap0

These are examples, not universally correct choices. Older Windows installations may need VirtIO drivers loaded during setup. A guest installed on an emulated AHCI disk may not yet contain a boot-critical VirtIO driver. Temporarily returning to the original controller is a clean test; randomly changing disk bus, slot, and image together is not.

Also check boot order and media. An attached installer ISO can keep winning the firmware selection, while an empty CD device or wrong disk path can make a valid system appear missing. Resolve all image paths to files, verify permissions for the account launching bhyve, and ensure no two guests write the same disk.

Reduce the VM to a controlled reproduction

Preserve the failing command, firmware checksum, package version, FreeBSD version, and guest-disk backup. Then remove nonessential passthrough and USB devices while keeping CPU, memory, boot ROM, framebuffer, and one boot disk. If the minimal VM reaches its loader, add devices back one at a time.

Check the host’s matching release notes and errata before replacing firmware. Updating only edk2-bhyve may change behavior; upgrading the host may also change bhyve device models. Keep a rollback path and never overwrite the sole known-working firmware during diagnosis.

The decisive observations are simple: an immediate host error implicates launch or ROM configuration; a visible UEFI menu implicates boot discovery; a known installer boot proves the firmware path; a loader followed by “root not found” implicates controller or guest drivers; and a boot entry that vanishes implicates NVRAM persistence. That layered evidence is more reliable than treating every blank console as a firmware failure.

Related:

Sources: