Files
Michael Crosby b438e97b93 Add cloud-hypervisor VMM backend for Linux hosts (#782)
apple/containerization currently runs containers in per-container VMs on
macOS hosts via Virtualization.framework. This adds a second VMM backend
so the same Swift orchestration layer (LinuxContainer / LinuxPod /
Vminitd gRPC contract) runs on Linux hosts via cloud-hypervisor + KVM.

**CloudHypervisor Swift package** (`Sources/CloudHypervisor/`) — a thin
client for cloud-hypervisor's REST-over-UDS API, layered on
AsyncHTTPClient. Endpoints cover VMM / VM lifecycle / hotplug (disk, fs,
net, vsock, remove-device). Cross-platform (compiles on macOS for unit
tests; consumed at runtime only by the Linux side of Containerization).

**CH backend in Containerization** — one cloud-hypervisor subprocess per
VM, gated behind `#if os(Linux)`. CHVirtualMachineManager /
CHVirtualMachineInstance mirror the VZ shape behind the existing
VirtualMachineManager / VirtualMachineInstance protocol. CHProcess and
VirtiofsdProcess manage the binaries; CHHotplugProvider handles
virtio-blk and virtio-fs runtime hotplug (with one virtiofsd per unique
source-hash tag, refcounted across containers).

**Linux host networking** — BridgeManager brings up a Linux bridge with
an IPv4 subnet and (opt-in via `--enable-nat`) iptables MASQUERADE +
scoped FORWARD rules. LinuxBridgedNetwork enslaves a fresh TAP per
container to the bridge. State is recorded under `/run/containerization`
so `cctl bridge delete` reverses exactly what create did. Bridge
teardown verifies the link kind via sysfs to refuse deleting non-bridge
interfaces.

**cctl run / bridge** — end-to-end Linux container run path (image pull,
ext4 rootfs assembly, VM boot, container exec) plus `cctl bridge
create|delete` for the host network plumbing.

**Build & dist** — `make linux-build` / `make linux-integration` build
and exercise the host side inside an apple/container `--virtualization`
dev container. `make dist-x86_64` produces a deployment tarball (cctl +
cloud-hypervisor + virtiofsd + initfs + kernel) cross-compiled from the
aarch64 dev container; pipeline documented in `docs/x86_64-build.md`.
Static-musl C deps and the Zig cross compiler are pinned by SHA256.

The host orchestrator runs as root. Per-VM runtime state lives under
`/run/containerization/ch/<UUID>` with mode 0700; UDS sockets inside are
bound with mode 0600. Vminitd's gRPC channel inherits that trust
boundary — socket-file perms are the auth.

Sandbox flags are upstream-secure by default. Two per-component opt-outs
exist for the apple/container dev-container case (where the host seccomp
profile SIGSYS-kills CH and virtiofsd):
- `CONTAINERIZATION_NO_CH_SECCOMP=1` — `cloud-hypervisor --seccomp
false`.
- `CONTAINERIZATION_NO_VIRTIOFSD_SANDBOX=1` — `virtiofsd --sandbox
none`. Each emits a one-shot `logger.warning` at process start. Legacy
alias `CONTAINERIZATION_RELAXED_SANDBOX=1` flips both. cctl spawns both
binaries with `setsid` and a minimal env allowlist (PATH / HOME /
RUST_LOG / RUST_BACKTRACE) so the parent's secrets don't leak to
children.

`make linux-integration` runs the cross-platform integration suite
against a real cloud-hypervisor VM inside the dev container. Linux runs
the cross-platform subset (`process true`/`false`/`echo hi`, virtiofs
round-trip, hotplug); the macOS suite is unchanged.

Signed-off-by: michael_crosby <michael_crosby@apple.com>
2026-07-02 11:20:22 -04:00
..

CloudHypervisor

A standalone Swift library for driving the cloud-hypervisor REST API over a Unix domain socket. The package compiles on both macOS and Linux, though cloud-hypervisor itself only runs on Linux.

Dependencies

There are no transitive dependencies on any other containerization library types.

Usage

import CloudHypervisor

let client = try CloudHypervisor.Client(
    socketPath: URL(filePath: "/tmp/ch-foo/api.sock")
)

try await client.vmmPing()
try await client.vmCreate(VmConfig(/* ... */))
try await client.vmBoot()

Full example with shared event loop group

import CloudHypervisor
import NIOPosix

let group = MultiThreadedEventLoopGroup(numberOfThreads: 2)
defer { try? group.syncShutdownGracefully() }

let client = try CloudHypervisor.Client(
    socketPath: URL(filePath: "/run/ch/vm0.sock"),
    eventLoopGroup: group
)

let info = try await client.vmInfo()
print(info.state)

Supported Endpoints (v1)

VMM

  • vmmPing() -> VmmPingResponse — verify the VMM process is alive
  • vmmShutdown() — shut down the VMM process
  • vmmInfo() -> VmmInfo — query VMM-level metadata

VM Lifecycle

  • vmCreate(_ config: VmConfig) — define a new VM
  • vmBoot() — start the VM
  • vmShutdown() — gracefully shut down the VM
  • vmInfo() -> VmInfo — query VM state and configuration
  • vmPause() — pause a running VM
  • vmResume() — resume a paused VM

Hotplug

  • vmAddDisk(_ config: DiskConfig) -> PciDeviceInfo — hot-add a block device
  • vmAddFs(_ config: FsConfig) -> PciDeviceInfo — hot-add a virtio-fs share
  • vmAddNet(_ config: NetConfig) -> PciDeviceInfo — hot-add a network device
  • vmAddVsock(_ config: VsockConfig) -> PciDeviceInfo — hot-add a vsock device
  • vmRemoveDevice(id: String) — hot-remove a device by ID

Minimum Supported cloud-hypervisor Version

The package targets the /api/v1/ REST namespace. It is tested against cloud-hypervisor v40 and later. Earlier releases may be missing endpoints or use incompatible JSON schemas.

Error Model

All failures are reported through CloudHypervisor.Error:

  • .transport(any Swift.Error) — a network or NIO-level failure before the HTTP response was received
  • .http(status:body:) — the server responded with a non-2xx HTTP status; body contains the raw response bytes
  • .decoding(any Swift.Error, body:) — the response had a 2xx status but JSON decoding failed; body is the raw bytes for diagnostics
  • .invalidSocketPath(String) — the URL passed to Client.init is not a file:// URL

Non-2xx responses always produce .http, never a decode error, so callers can distinguish protocol-level errors from unexpected payloads.

Concurrency

Client is Sendable and all endpoint methods are async throws. Each call opens a fresh TCP-over-UDS connection to cloud-hypervisor and closes it when the response is complete.

By default the client creates and owns a MultiThreadedEventLoopGroup and shuts it down in deinit. If you already have an event loop group (e.g. from NIO or another library), pass it via the eventLoopGroup: parameter — in that case the client does not shut the group down on deinit, leaving lifecycle management to the caller.

Non-Goals (v1)

  • Not a high-level VM orchestration layer — for that, use the Containerization library.
  • Not exhaustive coverage of cloud-hypervisor's full OpenAPI surface — only the 14 endpoints listed above are implemented; additional endpoints can be added incrementally.
  • No connection pooling — a fresh connection is opened per request, which is appropriate for low-volume control-plane use.
  • No streaming response bodies — response payloads are buffered in memory before decoding.