✨ feat: Linux x86_64 runtime + native AF_VSOCK control plane#52
✨ feat: Linux x86_64 runtime + native AF_VSOCK control plane#52
Conversation
Greptile SummaryThis PR brings Linux x86_64 to full parity with the macOS Apple Virt experience by wiring up native Key changes:
Notable concerns (all non-blocking P2s):
Confidence Score: 4/5Safe to merge; all identified issues are non-blocking P2 quality-of-life concerns with no correctness or data-loss risk The core logic is sound and well-tested: the build-tag split is correct, the manifest selector is clean and has thorough table-driven tests, the vsock fallback to TCP is safe, and the CID allocator is correctly documented as best-effort. The three P2 comments (O_TRUNC log overwrite, stale CID retention, duplicated platform config) are improvement suggestions that don't affect correctness or safety of the primary user path. End-to-end testing on Fedora 43 x86_64 with multi-VM CID allocation is a strong signal. pkg/vm/qemu.go (O_TRUNC on qemu.log), pkg/vm/util.go (CID recycling), cmd/vmhost/platform_linux_amd64.go + pkg/vm/backend_linux_amd64.go (duplicated config) Important Files Changed
Sequence DiagramsequenceDiagram
participant mb as mb (client)
participant daemon as daemon
participant vmhost as vmhost process
participant qemu as QEMU child
participant guest as stereosd (guest)
mb->>daemon: mb up
daemon->>daemon: PrepareQEMUDisk() copy image, EFI vars, state.json
daemon->>vmhost: spawn vmhost process
vmhost->>vmhost: HostHasVsock()? /dev/vhost-vsock + !WSL2
alt vsock available
vmhost->>vmhost: allocateVsockCID() scan state.json → lowest free CID >= 3
vmhost->>qemu: qemu-system-x86_64 -machine q35,accel=kvm -device vhost-vsock-pci,guest-cid=N
else fallback TCP
vmhost->>vmhost: allocatePort() for stereosd
vmhost->>qemu: qemu-system-x86_64 -machine q35,accel=kvm -netdev user,hostfwd=tcp::PORT-:1024
end
qemu->>guest: kernel boots via direct -kernel/-initrd
guest->>guest: stereosd listens on vsock port 1024
vmhost->>guest: VsockTransport.Dial(CID, 1024) mdvsock.Dial in goroutine + timeout race
vmhost->>guest: WaitForReady / SetConfig / InjectSSHKey / Secrets / Mounts
vmhost->>daemon: boot complete
daemon->>mb: sandbox ready
Reviews (1): Last reviewed commit: "✨ feat: Linux x86_64 runtime + native AF..." | Re-trigger Greptile |
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
| func allocateVsockCID(baseDir string) (uint32, error) { | ||
| used := map[uint32]bool{} | ||
|
|
||
| entries, err := os.ReadDir(VMsDir(baseDir)) | ||
| if err != nil && !os.IsNotExist(err) { | ||
| return 0, fmt.Errorf("reading VMs directory: %w", err) | ||
| } | ||
| for _, e := range entries { | ||
| if !e.IsDir() { | ||
| continue | ||
| } | ||
| state, err := loadState(filepath.Join(VMsDir(baseDir), e.Name())) | ||
| if err != nil { | ||
| continue | ||
| } | ||
| if state.VsockCID > 0 { | ||
| used[state.VsockCID] = true | ||
| } | ||
| } | ||
|
|
||
| // Pick the lowest free CID. We start at 3 (first non-reserved) and cap | ||
| // at 2^31 defensively — in practice we run out of RAM long before CIDs. | ||
| for cid := uint32(3); cid < 1<<31; cid++ { | ||
| if !used[cid] { | ||
| return cid, nil | ||
| } | ||
| } | ||
| return 0, fmt.Errorf("no vsock CID available (all in use under %s)", baseDir) | ||
| } |
There was a problem hiding this comment.
allocateVsockCID marks a CID as used if it appears in any VM's state.json, regardless of whether that VM is currently running. When a VM is stopped via mb down, its state.json retains the VsockCID field. Each subsequent mb start on that VM consumes a new CID (the old one is still present in the state file and is thus skipped), and the stale CID is never freed until the VM is destroyed.
While the 2³¹ CID space makes exhaustion practically impossible, this also means mb start on a stopped VM always changes the guest's CID rather than reusing the previously assigned one. A simpler fix is to clear VsockCID in the state file when the VM stops:
// In Down/ForceDown: update state to clear the CID once the process exits.
stateFile.VsockCID = 0
_ = saveState(inst.Dir, stateFile)
* Splits `pkg/vm/backend_linux.go` and `cmd/vmhost/platform_linux.go` by GOARCH so amd64 gets `qemu-system-x86_64`, q35, and OVMF firmware paths covering Fedora, Debian/Ubuntu, and NixOS. Was hardcoded to aarch64 under a bare `linux` build tag. * `pkg/mixtapes/pull.go:resolveFromIndex` now filters by `runtime.GOOS`/`GOARCH` with a 4-tier priority (platform+raw → platform-any → any raw → first). Previously picked the first raw manifest, which on a multi-arch index would boot the wrong kernel. * Adds a native AF_VSOCK transport for Linux via `github.com/mdlayher/vsock` — Go's stdlib `net.FileConn` rejects AF_VSOCK with "protocol not supported". Host capability probe checks `/dev/vhost-vsock` and falls back to TCP on WSL2 or hosts without `vhost_vsock`. Per-VM CID allocator scans existing state files to avoid collisions. * Moves `highmem=on` into per-platform `MachineType` strings; it's aarch64-virt-only and QEMU rejects it on x86 q35. * Captures QEMU stderr to `<vm-dir>/qemu.log` so launch failures leave a forensic trace. Signed-off-by: Matt Yeazel <matt@papercompute.com>
The `bucketuploader`/`checksumer`/`ghrelease`/`golangcilint` daggerverse deps pinned in `dagger.json` now require dagger engine v0.20.6, so CI installing v0.20.1 failed with "module requires dagger v0.20.6, but you have v0.20.1". Bumps `dagger.json` `engineVersion` and the `DAGGER_VERSION` env var in all five workflows (ci, pr, cut-release, nightly, release) so the installed engine matches what the resolved module graph wants. Signed-off-by: Matt Yeazel <matt@papercompute.com>
yeazelm
force-pushed
the
mt/x86_64-support
branch
from
c9a7546 to
dc8daa2
Compare
|
No docs update needed. This PR adds Linux x86_64 runtime support with native AF_VSOCK control plane to masterblaster ( PR #52 was merged: ✨ feat: Linux x86_64 runtime + native AF_VSOCK control plane |
2 participants
Summary
Brings
mbon Linux x86_64 to parity with the Apple Virt experience onmacOS: native
qemu-system-x86_64with KVM, host↔guest control planeover AF_VSOCK rather than SLIRP/TCP. Three latent gaps closed:
pkg/vm/backend_linux.goandcmd/vmhost/platform_linux.gowere hardcoded to
qemu-system-aarch64under a barelinuxbuild tag,so
mb upon an x86_64 Linux host launched the ARM emulator. Split byGOARCH — amd64 gets q35 and OVMF paths covering Fedora, Debian/Ubuntu,
and NixOS.
pkg/mixtapes/pull.go:resolveFromIndexpicked the first raw-format manifest with no arch check, so on a
multi-arch index it would boot the wrong kernel. Extracted a pure
selectIndexManifestwith a 4-tier priority(platform+raw → platform-any → any raw → first) keyed off
runtime.GOOS/GOARCH.pkg/vsock/VsockTransportwiringgithub.com/mdlayher/vsockinto the existingTransportinterface.Go's stdlib
net.FileConnrejects AF_VSOCK with "protocol notsupported", so we defer to mdlayher's cgo-free (on Linux) wrapper that
hooks AF_VSOCK into Go's runtime poller. A
HostHasVsock()probechecks
/dev/vhost-vsock, so requesting vsock in the platform configgracefully falls back to TCP hostfwd when unavailable. Per-VM CID
allocator scans existing VM state to avoid kernel-level collisions.
Also folded in:
highmem=onmoved from the generic machine-arg builder to per-platformMachineTypestrings — it's aarch64-virt-only and QEMU rejects it onx86 q35.
<vm-dir>/qemu.logso launch failuresleave a forensic trace (they previously went to
/dev/null).Testing
go test ./...passes.linux/amd64,linux/arm64, anddarwin/arm64.mb upcompletes in 7.2s,state.json: vsock_cid=3,-device vhost-vsock-pci,guest-cid=3+ SSH hostfwd only,mb sshin → stereosd + agentd active,uname -m = x86_64,/run/stereos-readypopulated,ss --vsock --listeninginside the guest shows stereosd on port 1024.
pre-existing sandbox holding CID 3, two new sandboxes
alphaandbetastarted sequentially got CIDs 4 and 5 respectively (theallocator correctly scanned and skipped the in-use CID). Both
ran concurrently, each was independently reachable over its own
SSH hostfwd port, and both guests reported stereosd + agentd
active.