Files
apple--containerization/Sources/Containerization/VirtualMachineInstance.swift
T
Michael Crosby b438e97b93 Add cloud-hypervisor VMM backend for Linux hosts (#782)
apple/containerization currently runs containers in per-container VMs on
macOS hosts via Virtualization.framework. This adds a second VMM backend
so the same Swift orchestration layer (LinuxContainer / LinuxPod /
Vminitd gRPC contract) runs on Linux hosts via cloud-hypervisor + KVM.

**CloudHypervisor Swift package** (`Sources/CloudHypervisor/`) — a thin
client for cloud-hypervisor's REST-over-UDS API, layered on
AsyncHTTPClient. Endpoints cover VMM / VM lifecycle / hotplug (disk, fs,
net, vsock, remove-device). Cross-platform (compiles on macOS for unit
tests; consumed at runtime only by the Linux side of Containerization).

**CH backend in Containerization** — one cloud-hypervisor subprocess per
VM, gated behind `#if os(Linux)`. CHVirtualMachineManager /
CHVirtualMachineInstance mirror the VZ shape behind the existing
VirtualMachineManager / VirtualMachineInstance protocol. CHProcess and
VirtiofsdProcess manage the binaries; CHHotplugProvider handles
virtio-blk and virtio-fs runtime hotplug (with one virtiofsd per unique
source-hash tag, refcounted across containers).

**Linux host networking** — BridgeManager brings up a Linux bridge with
an IPv4 subnet and (opt-in via `--enable-nat`) iptables MASQUERADE +
scoped FORWARD rules. LinuxBridgedNetwork enslaves a fresh TAP per
container to the bridge. State is recorded under `/run/containerization`
so `cctl bridge delete` reverses exactly what create did. Bridge
teardown verifies the link kind via sysfs to refuse deleting non-bridge
interfaces.

**cctl run / bridge** — end-to-end Linux container run path (image pull,
ext4 rootfs assembly, VM boot, container exec) plus `cctl bridge
create|delete` for the host network plumbing.

**Build & dist** — `make linux-build` / `make linux-integration` build
and exercise the host side inside an apple/container `--virtualization`
dev container. `make dist-x86_64` produces a deployment tarball (cctl +
cloud-hypervisor + virtiofsd + initfs + kernel) cross-compiled from the
aarch64 dev container; pipeline documented in `docs/x86_64-build.md`.
Static-musl C deps and the Zig cross compiler are pinned by SHA256.

The host orchestrator runs as root. Per-VM runtime state lives under
`/run/containerization/ch/<UUID>` with mode 0700; UDS sockets inside are
bound with mode 0600. Vminitd's gRPC channel inherits that trust
boundary — socket-file perms are the auth.

Sandbox flags are upstream-secure by default. Two per-component opt-outs
exist for the apple/container dev-container case (where the host seccomp
profile SIGSYS-kills CH and virtiofsd):
- `CONTAINERIZATION_NO_CH_SECCOMP=1` — `cloud-hypervisor --seccomp
false`.
- `CONTAINERIZATION_NO_VIRTIOFSD_SANDBOX=1` — `virtiofsd --sandbox
none`. Each emits a one-shot `logger.warning` at process start. Legacy
alias `CONTAINERIZATION_RELAXED_SANDBOX=1` flips both. cctl spawns both
binaries with `setsid` and a minimal env allowlist (PATH / HOME /
RUST_LOG / RUST_BACKTRACE) so the parent's secrets don't leak to
children.

`make linux-integration` runs the cross-platform integration suite
against a real cloud-hypervisor VM inside the dev container. Linux runs
the cross-platform subset (`process true`/`false`/`echo hi`, virtiofs
round-trip, hotplug); the macOS suite is unchanged.

Signed-off-by: michael_crosby <michael_crosby@apple.com>
2026-07-02 11:20:22 -04:00

122 lines
5.0 KiB
Swift

//===----------------------------------------------------------------------===//
// Copyright © 2025-2026 Apple Inc. and the Containerization project authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//===----------------------------------------------------------------------===//
import ContainerizationError
import Foundation
/// The runtime state of the virtual machine instance.
public enum VirtualMachineInstanceState: Sendable {
case starting
case running
case stopped
case stopping
case unknown
}
/// How the VMM exposes virtiofs devices to the guest.
///
/// - `unified`: a single virtio-fs device (tag `virtiofs`) carries all
/// shares as subdirectories (Apple's `VZMultipleDirectoryShare` model).
/// - `perTag`: one virtio-fs device per source-hash tag, each mounted
/// separately in the guest (cloud-hypervisor / virtiofsd model).
public enum VirtiofsLayout: Sendable {
case unified
case perTag
}
/// A live instance of a virtual machine.
public protocol VirtualMachineInstance: Sendable {
associatedtype Agent: VirtualMachineAgent
// The state of the virtual machine.
var state: VirtualMachineInstanceState { get }
var mounts: [String: [AttachedFilesystem]] { get }
/// How this VMM exposes virtiofs devices to the guest. Defaults to
/// `.unified` (the VZ-shaped behavior); CH overrides to `.perTag`.
var virtiofsLayout: VirtiofsLayout { get }
/// Dial the Agent. It's up the VirtualMachineInstance to determine
/// what port the agent is listening on.
func dialAgent() async throws -> Agent
/// Dial a vsock port in the guest.
func dial(_ port: UInt32) async throws -> FileHandle
/// Listen on a host vsock port.
func listen(_ port: UInt32) throws -> VsockListener
/// Start the virtual machine.
func start() async throws
/// Stop the virtual machine.
func stop() async throws
/// Pause the virtual machine.
func pause() async throws
/// Resume the virtual machine.
func resume() async throws
/// Hotplug a block device, returning the attached filesystem info.
/// Throws if the VMM does not support hotplug or not available
/// - Parameter block: The mount configuration for the block device to hotplug
/// - Parameter id: The metadata ID to associate with this mount (e.g. container ID)
/// - Returns: AttachedFilesystem with the device path in the guest
func hotplug(_ block: Mount, id: String) async throws -> AttachedFilesystem
/// Register mounts for a container after hotplug.
/// This is used to add the rootfs and additional mounts to the VM's mount registry
/// so they can be found when building the container's OCI spec.
/// - Parameter id: The container ID
/// - Parameter rootfs: The rootfs attachment from hotplug
/// - Parameter additionalMounts: Additional mounts (like /proc, /sys) to register
func registerMounts(id: String, rootfs: AttachedFilesystem, additionalMounts: [Mount]) throws
/// Release a hotplug device.
/// This should be called when a hotplugged container is stopped or fails to start.
/// - Parameter id: The container ID whose hotplug should be released
func releaseHotplug(id: String) async throws
/// Hotplug virtiofs directories into the running VM.
/// - Parameter mounts: The virtiofs mounts to add
/// - Parameter id: The container ID that owns these mounts
func hotplugVirtioFS(_ mounts: [Mount], id: String) async throws
/// Release virtiofs shares for a container.
/// - Parameter id: The container ID whose virtiofs shares should be released
func releaseVirtioFS(id: String) async throws
}
extension VirtualMachineInstance {
public var virtiofsLayout: VirtiofsLayout { .unified }
public func pause() async throws {
throw ContainerizationError(.unsupported, message: "pause")
}
public func resume() async throws {
throw ContainerizationError(.unsupported, message: "resume")
}
public func hotplug(_ block: Mount, id: String) async throws -> AttachedFilesystem {
throw ContainerizationError(.unsupported, message: "hotplug not supported")
}
public func registerMounts(id: String, rootfs: AttachedFilesystem, additionalMounts: [Mount]) throws {
// no-op default
}
public func releaseHotplug(id: String) async throws {
// no-op default
}
public func hotplugVirtioFS(_ mounts: [Mount], id: String) async throws {
// no-op default
}
public func releaseVirtioFS(id: String) async throws {
// no-op default
}
}