Files
Michael Crosby b438e97b93 Add cloud-hypervisor VMM backend for Linux hosts (#782)
apple/containerization currently runs containers in per-container VMs on
macOS hosts via Virtualization.framework. This adds a second VMM backend
so the same Swift orchestration layer (LinuxContainer / LinuxPod /
Vminitd gRPC contract) runs on Linux hosts via cloud-hypervisor + KVM.

**CloudHypervisor Swift package** (`Sources/CloudHypervisor/`) — a thin
client for cloud-hypervisor's REST-over-UDS API, layered on
AsyncHTTPClient. Endpoints cover VMM / VM lifecycle / hotplug (disk, fs,
net, vsock, remove-device). Cross-platform (compiles on macOS for unit
tests; consumed at runtime only by the Linux side of Containerization).

**CH backend in Containerization** — one cloud-hypervisor subprocess per
VM, gated behind `#if os(Linux)`. CHVirtualMachineManager /
CHVirtualMachineInstance mirror the VZ shape behind the existing
VirtualMachineManager / VirtualMachineInstance protocol. CHProcess and
VirtiofsdProcess manage the binaries; CHHotplugProvider handles
virtio-blk and virtio-fs runtime hotplug (with one virtiofsd per unique
source-hash tag, refcounted across containers).

**Linux host networking** — BridgeManager brings up a Linux bridge with
an IPv4 subnet and (opt-in via `--enable-nat`) iptables MASQUERADE +
scoped FORWARD rules. LinuxBridgedNetwork enslaves a fresh TAP per
container to the bridge. State is recorded under `/run/containerization`
so `cctl bridge delete` reverses exactly what create did. Bridge
teardown verifies the link kind via sysfs to refuse deleting non-bridge
interfaces.

**cctl run / bridge** — end-to-end Linux container run path (image pull,
ext4 rootfs assembly, VM boot, container exec) plus `cctl bridge
create|delete` for the host network plumbing.

**Build & dist** — `make linux-build` / `make linux-integration` build
and exercise the host side inside an apple/container `--virtualization`
dev container. `make dist-x86_64` produces a deployment tarball (cctl +
cloud-hypervisor + virtiofsd + initfs + kernel) cross-compiled from the
aarch64 dev container; pipeline documented in `docs/x86_64-build.md`.
Static-musl C deps and the Zig cross compiler are pinned by SHA256.

The host orchestrator runs as root. Per-VM runtime state lives under
`/run/containerization/ch/<UUID>` with mode 0700; UDS sockets inside are
bound with mode 0600. Vminitd's gRPC channel inherits that trust
boundary — socket-file perms are the auth.

Sandbox flags are upstream-secure by default. Two per-component opt-outs
exist for the apple/container dev-container case (where the host seccomp
profile SIGSYS-kills CH and virtiofsd):
- `CONTAINERIZATION_NO_CH_SECCOMP=1` — `cloud-hypervisor --seccomp
false`.
- `CONTAINERIZATION_NO_VIRTIOFSD_SANDBOX=1` — `virtiofsd --sandbox
none`. Each emits a one-shot `logger.warning` at process start. Legacy
alias `CONTAINERIZATION_RELAXED_SANDBOX=1` flips both. cctl spawns both
binaries with `setsid` and a minimal env allowlist (PATH / HOME /
RUST_LOG / RUST_BACKTRACE) so the parent's secrets don't leak to
children.

`make linux-integration` runs the cross-platform integration suite
against a real cloud-hypervisor VM inside the dev container. Linux runs
the cross-platform subset (`process true`/`false`/`echo hi`, virtiofs
round-trip, hotplug); the macOS suite is unchanged.

Signed-off-by: michael_crosby <michael_crosby@apple.com>
2026-07-02 11:20:22 -04:00

159 lines
6.6 KiB
Swift

//===----------------------------------------------------------------------===//
// Copyright © 2026 Apple Inc. and the Containerization project authors.
//
// Licensed under the Apache License, Version 2.0 (the "License");
// you may not use this file except in compliance with the License.
// You may obtain a copy of the License at
//
// https://www.apache.org/licenses/LICENSE-2.0
//
// Unless required by applicable law or agreed to in writing, software
// distributed under the License is distributed on an "AS IS" BASIS,
// WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
// See the License for the specific language governing permissions and
// limitations under the License.
//===----------------------------------------------------------------------===//
#if os(Linux)
import ContainerizationError
import ContainerizationOS
import Foundation
#if canImport(Musl)
import Musl
#elseif canImport(Glibc)
import Glibc
#endif
// MARK: - Cloud Hypervisor hybrid vsock host-side helpers
//
// Cloud Hypervisor exposes its vsock device to the host as a Unix-domain
// socket pair, not the kernel AF_VSOCK (this avoids the host needing the
// vhost-vsock kernel module).
//
// - Host → guest dials use the "base" UDS (`VsockConfig.socket`) with a
// one-line `CONNECT <port>\n` request, answered by `OK <port>\n`. After
// that the connection is bridged transparently.
// - Guest → host dials are accepted on per-port UDS files at the
// conventional path `<base>_<port>` that the host pre-creates.
//
// Spec: `docs/vsock.md` in the cloud-hypervisor repository.
/// Returns the conventional per-port UDS path for guest→host vsock connections,
/// derived by suffixing the base socket path with `_<port>`.
func chVsockListenSocketPath(baseSocket: URL, port: UInt32) -> URL {
URL(fileURLWithPath: "\(baseSocket.path)_\(port)")
}
/// Bind + listen a fresh AF_UNIX SOCK_STREAM at `path`, unlinking any stale
/// socket file at that path first. Returns the listening fd; ownership is
/// transferred to the caller.
///
/// The socket file is created with mode `perms` (default `0o600`). The
/// per-VM workDir already restricts access via its own `0o700` mode, but
/// tightening the socket itself is cheap defense-in-depth — vminitd's gRPC
/// surface trusts whoever can `connect(2)` and exposes full container
/// control, so any local-user reach into these sockets is a privilege
/// escalation primitive.
func chVsockBindListener(at path: URL, perms: mode_t = 0o600) throws -> Int32 {
let unix = try UnixType(path: path.path, perms: perms, unlinkExisting: true)
let socket = try Socket(type: unix, closeOnDeinit: false)
do {
try socket.listen()
} catch {
try? socket.close()
throw error
}
return socket.fileDescriptor
}
/// Dial guest port `port` over the cloud-hypervisor hybrid vsock at
/// `baseSocket`. Returns a `FileHandle` wrapping the connected fd; the
/// FileHandle does **not** close the fd on deinit — ownership of the fd
/// transfers to the caller (typically `Vminitd.init`, which hands it to
/// NIO via `withConnectedSocket`; NIO is then responsible for closing it
/// when the channel is torn down). Callers using the FileHandle directly
/// must close the underlying fd themselves.
func chVsockDial(baseSocket: URL, port: UInt32) async throws -> FileHandle {
try await Task.detached {
try chVsockDialSync(baseSocket: baseSocket, port: port)
}.value
}
// MARK: - Internals
private func chVsockDialSync(baseSocket: URL, port: UInt32) throws -> FileHandle {
let unix = try UnixType(path: baseSocket.path)
let socket = try Socket(type: unix, closeOnDeinit: false)
do {
try socket.connect()
// Bound the bootstrap reply read so a hung cloud-hypervisor muxer
// can't pin this thread forever. CH replies within milliseconds in
// healthy operation; 30 s is well outside that and matches the
// CloudHypervisor REST client default. After bootstrap the fd is
// handed to NIO which puts it in non-blocking mode, where
// SO_RCVTIMEO has no effect — so leaving the timeout in place is
// harmless.
try socket.setTimeout(option: .receive, seconds: 30)
let request = "CONNECT \(port)\n"
_ = try socket.write(data: Data(request.utf8))
let response = try readLine(fd: socket.fileDescriptor)
// Cloud Hypervisor responds with "OK <local-port>\n" where
// <local-port> is the local-side port the muxer allocated for this
// forwarded connection — NOT the peer port we asked for. So we just
// require the response to start with "OK " and parse a UInt32 after.
guard response.hasPrefix("OK "),
UInt32(response.dropFirst(3)) != nil
else {
throw ContainerizationError(
.invalidState,
message: "unexpected vsock CONNECT response: \(response.debugDescription)"
)
}
return FileHandle(fileDescriptor: socket.fileDescriptor, closeOnDealloc: false)
} catch {
try? socket.close()
throw error
}
}
/// Read CH's hybrid-vsock `CONNECT` reply line (`OK <local-port>\n`) one
/// byte at a time. Reads from `fd` until a `\n` is seen or `maxLength` is
/// reached; the returned string excludes the terminating newline. We do
/// this by hand because the fd is still in blocking mode (NIO takes over
/// only after the bootstrap completes) and there's no Foundation /
/// NIO line reader that operates on a raw blocking POSIX fd.
private func readLine(fd: Int32, maxLength: Int = 256) throws -> String {
var bytes: [UInt8] = []
bytes.reserveCapacity(maxLength)
while bytes.count < maxLength {
var byte: UInt8 = 0
let n = withUnsafeMutablePointer(to: &byte) { ptr -> ssize_t in
read(fd, ptr, 1)
}
if n == 0 {
break
}
if n < 0 {
let savedErrno = errno
// SO_RCVTIMEO expiry surfaces as EAGAIN / EWOULDBLOCK on a
// blocking socket. Translate to a clear timeout error so callers
// don't have to inspect errno.
if savedErrno == EAGAIN || savedErrno == EWOULDBLOCK {
throw ContainerizationError(
.timeout,
message: "vsock CONNECT response not received within socket receive timeout"
)
}
throw POSIXError(POSIXErrorCode(rawValue: savedErrno) ?? .EIO)
}
if byte == UInt8(ascii: "\n") {
break
}
bytes.append(byte)
}
return String(decoding: bytes, as: UTF8.self)
}
#endif