Skip to content

fix(vm): harden build-libkrun.sh for x86_64 and resolve PATH-shadowing issues #806

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
vince-brisebois merged 1 commit into from
Apr 10, 2026
Merged

fix(vm): harden build-libkrun.sh for x86_64 and resolve PATH-shadowing issues #806

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
17 changes: 6 additions & 11 deletions architecture/custom-vm-runtime.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -122,11 +122,10 @@ graph LR

subgraph Linux["Linux CI (build-libkrun.sh)"]
BUILD_L["Build kernel + libkrunfw.so + libkrun.so"]
KERNELC["kernel.c\nKernel as C byte array"]
end

subgraph macOS["macOS CI (build-libkrun-macos.sh)"]
BUILD_M["Compile kernel.c -> libkrunfw.dylib\nBuild libkrun.dylib"]
BUILD_M["Build libkrunfw.dylib + libkrun.dylib"]
end

subgraph Output["target/libkrun-build/"]
Expand All @@ -136,8 +135,7 @@ graph LR

KCONF --> BUILD_L
BUILD_L --> LIB_SO
BUILD_L --> KERNELC
KERNELC --> BUILD_M
KCONF --> BUILD_M
BUILD_M --> LIB_DY
```

Expand Down Expand Up @@ -228,18 +226,15 @@ supported platforms. Runs on-demand or when the kernel config / pinned versions

| Platform | Runner | Build Method |
|----------|--------|-------------|
| Linux ARM64 | `build-arm64` (self-hosted) | Native `build-libkrun.sh` (also exports kernel.c) |
| Linux ARM64 | `build-arm64` (self-hosted) | Native `build-libkrun.sh` |
| Linux x86_64 | `build-amd64` (self-hosted) | Native `build-libkrun.sh` |
| macOS ARM64 | `macos-latest-xlarge` (GitHub-hosted) | `build-libkrun-macos.sh --kernel-dir` (uses pre-built kernel.c from ARM64) |
| macOS ARM64 | `macos-latest-xlarge` (GitHub-hosted) | `build-libkrun-macos.sh` |

Artifacts: `vm-runtime-{platform}.tar.zst` containing libkrun, libkrunfw, gvproxy, and
provenance metadata.

The aarch64 Linux kernel is compiled once on the Linux ARM64 runner. The resulting
`kernel.c` (a C source file containing the kernel as a byte array) is passed to the
macOS job, which compiles it into `libkrunfw.dylib` with Apple's `cc`. This eliminates
the need for krunvm/Fedora VM and cuts macOS CI from ~45 min to ~5 min. The kernel
inside libkrunfw is always Linux regardless of host platform.
Each platform builds its own libkrunfw and libkrun natively. The kernel inside
libkrunfw is always Linux regardless of host platform.

### VM Binary (`release-vm-dev.yml`)

Expand Down
21 changes: 8 additions & 13 deletions crates/openshell-vm/runtime/README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,21 +27,20 @@ runtime/

## Build Pipeline

The kernel is compiled on Linux CI runners. macOS reuses the pre-built `kernel.c`
artifact from the Linux ARM64 build — no krunvm or Fedora VM needed.
Each platform builds its own kernel and runtime natively.

```
Linux ARM64: builds aarch64 kernel -> .so + exports kernel.c (parallel)
Linux AMD64: builds x86_64 kernel -> .so (parallel)
macOS ARM64: reuses aarch64 kernel.c -> .dylib (depends on ARM64)
Linux ARM64: builds aarch64 kernel -> .so (parallel)
Linux AMD64: builds x86_64 kernel -> .so (parallel)
macOS ARM64: builds aarch64 kernel -> .dylib
```

### Build Scripts

| Script | Platform | What it does |
|--------|----------|-------------|
| `tasks/scripts/vm/build-libkrun.sh` | Linux | Builds libkrunfw + libkrun from source, exports kernel.c |
| `tasks/scripts/vm/build-libkrun-macos.sh` | macOS | Compiles pre-built kernel.c into .dylib, builds libkrun |
| `tasks/scripts/vm/build-libkrun.sh` | Linux | Builds libkrunfw + libkrun from source |
| `tasks/scripts/vm/build-libkrun-macos.sh` | macOS | Builds libkrunfw + libkrun from source |
| `tasks/scripts/vm/package-vm-runtime.sh` | Any | Packages runtime tarball (libs + gvproxy + provenance) |

### Quick Build (Linux)
Expand All @@ -56,14 +55,12 @@ FROM_SOURCE=1 mise run vm:setup

### Quick Build (macOS)

On macOS, you need a pre-built `kernel.c` from a Linux ARM64 build:

```bash
# Download pre-built runtime (recommended, ~30s):
mise run vm:setup

# Or if you have kernel.c from a Linux build:
tasks/scripts/vm/build-libkrun-macos.sh --kernel-dir target/libkrun-build
# Or build from source:
FROM_SOURCE=1 mise run vm:setup
```

### Output
Expand All @@ -74,8 +71,6 @@ Build artifacts are placed in `target/libkrun-build/`:
target/libkrun-build/
libkrun.so / libkrun.dylib # The VMM library
libkrunfw.so* / libkrunfw.dylib # Kernel firmware library
kernel.c # Linux kernel as C byte array (Linux only)
ABI_VERSION # ABI version number (Linux only)
```

## Networking
Expand Down
4 changes: 4 additions & 0 deletions crates/openshell-vm/runtime/kernel/openshell.kconfig
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -119,6 +119,10 @@ CONFIG_MEMCG=y
# ── Disable kernel headers archive (avoids cpio issues in CI) ──────────
# CONFIG_IKHEADERS is not set

# ── POSIX message queues (required by runc to mount /dev/mqueue in containers) ─
CONFIG_POSIX_MQUEUE=y
CONFIG_POSIX_MQUEUE_SYSCTL=y

# ── Security features required by the sandbox runtime ───────────────────
CONFIG_SECURITY_LANDLOCK=y
CONFIG_SECCOMP_FILTER=y
32 changes: 32 additions & 0 deletions crates/openshell-vm/src/lib.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -61,6 +61,15 @@ pub enum VmError {
#[error("host setup failed: {0}")]
HostSetup(String),

/// `/dev/kvm` is not accessible (Linux only).
#[error(
"cannot open /dev/kvm: {reason}\n\
KVM access is required to run microVMs on Linux.\n\
Fix: sudo usermod -aG kvm $USER then log out and back in\n\
(or run: newgrp kvm)"
)]
KvmAccess { reason: String },

/// `fork()` failed.
#[error("fork() failed: {0}")]
Fork(String),
Expand Down Expand Up @@ -1180,6 +1189,22 @@ fn path_to_cstring(path: &Path) -> Result<CString, VmError> {
Ok(CString::new(s)?)
}

/// Check that `/dev/kvm` is readable before attempting to boot.
///
/// libkrun panics with an opaque Rust panic (instead of returning an error
/// code) when `/dev/kvm` is inaccessible. This pre-check turns that into a
/// clear, actionable error message.
#[cfg(target_os = "linux")]
fn check_kvm_access() -> Result<(), VmError> {
use std::fs::OpenOptions;
match OpenOptions::new().read(true).open("/dev/kvm") {
Ok(_) => Ok(()),
Err(e) => Err(VmError::KvmAccess {
reason: e.to_string(),
}),
}
}

// ── Launch ──────────────────────────────────────────────────────────────

/// Configure and launch a libkrun microVM.
Expand All @@ -1204,6 +1229,13 @@ pub fn launch(config: &VmConfig) -> Result<i32, VmError> {
path: config.rootfs.display().to_string(),
});
}

// On Linux, libkrun uses KVM for hardware virtualization. Check access
// before starting so a missing kvm group membership produces a clear
// error instead of a cryptic panic inside krun_start_enter.
#[cfg(target_os = "linux")]
check_kvm_access()?;

if config.exec_path == "/srv/openshell-vm-init.sh" {
ensure_vm_not_running(&config.rootfs)?;
}
Expand Down
14 changes: 9 additions & 5 deletions deploy/docker/Dockerfile.images
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,12 @@ COPY crates/openshell-core/Cargo.toml crates/openshell-core/Cargo.toml
COPY crates/openshell-ocsf/Cargo.toml crates/openshell-ocsf/Cargo.toml
COPY crates/openshell-policy/Cargo.toml crates/openshell-policy/Cargo.toml
COPY crates/openshell-providers/Cargo.toml crates/openshell-providers/Cargo.toml
COPY crates/openshell-prover/Cargo.toml crates/openshell-prover/Cargo.toml
COPY crates/openshell-router/Cargo.toml crates/openshell-router/Cargo.toml
COPY crates/openshell-sandbox/Cargo.toml crates/openshell-sandbox/Cargo.toml
COPY crates/openshell-server/Cargo.toml crates/openshell-server/Cargo.toml
COPY crates/openshell-tui/Cargo.toml crates/openshell-tui/Cargo.toml
COPY crates/openshell-prover/Cargo.toml crates/openshell-prover/Cargo.toml
COPY crates/openshell-vm/Cargo.toml crates/openshell-vm/Cargo.toml
COPY crates/openshell-core/build.rs crates/openshell-core/build.rs
COPY proto/ proto/

Expand All @@ -65,24 +66,27 @@ RUN mkdir -p \
crates/openshell-ocsf/src \
crates/openshell-policy/src \
crates/openshell-providers/src \
crates/openshell-prover/src \
crates/openshell-router/src \
crates/openshell-sandbox/src \
crates/openshell-server/src \
crates/openshell-prover/src \
crates/openshell-tui/src && \
crates/openshell-tui/src \
crates/openshell-vm/src && \
touch crates/openshell-bootstrap/src/lib.rs && \
printf 'fn main() {}\n' > crates/openshell-cli/src/main.rs && \
touch crates/openshell-core/src/lib.rs && \
touch crates/openshell-ocsf/src/lib.rs && \
touch crates/openshell-policy/src/lib.rs && \
touch crates/openshell-providers/src/lib.rs && \
touch crates/openshell-prover/src/lib.rs && \
touch crates/openshell-router/src/lib.rs && \
touch crates/openshell-sandbox/src/lib.rs && \
printf 'fn main() {}\n' > crates/openshell-sandbox/src/main.rs && \
touch crates/openshell-server/src/lib.rs && \
printf 'fn main() {}\n' > crates/openshell-server/src/main.rs && \
touch crates/openshell-prover/src/lib.rs && \
touch crates/openshell-tui/src/lib.rs
touch crates/openshell-tui/src/lib.rs && \
touch crates/openshell-vm/src/lib.rs && \
printf 'fn main() {}\n' > crates/openshell-vm/src/main.rs

FROM rust-builder-skeleton AS rust-deps

Expand Down
Loading
Loading