Draft
Conversation
Adds scripts/explore/ — sibling pipeline to scripts/mythos/ — for exploring design spaces where multiple alternatives exist and the right answer requires "implementation as argumentation." Key adaptations from mythos: - Two-axis rank: viability (1-5) AND locus-of-fix (meld | wit-bindgen | spec | wasm-tools | hybrid). Honest "this isn't ours to fix" is a valid output. - Oracle: a fuse_only_test! flips to a passing runtime_test! on the prototype branch, instead of mythos's failing-Kani-plus-PoC pair. - "NOT VIABLE TODAY: <reason>" replaces "NO FINDING: could not satisfy oracle" as the honest-rejection escape hatch. - Validator can reject for "real but uninteresting" the same way mythos does — high maintenance + low generalization is a three-strikes-out for ADR promotion. Adds safety/adr/ as the emission target: - schema.yaml defines design-question and design-adr shapes - ADR-0 is the parent question for the re-exporter resource chain problem (epic #69 / issue #92), enumerating seven candidate paths (D, E, F, G, H, I, J) for sibling ADRs to explore in parallel - Path I introduces the pulseengine/wit-bindgen fork as a venue for the opaque-rep convention proposal No code changes. Pipeline templates only. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Rekey HandleTableInfo from HashMap<usize, _> to HashMap<(usize, String, String), _>, so a re-exporter component with multiple resources gets one handle table per (component, interface, resource_name) instead of a single shared table that misroutes imports the re-exporter passes through. Background: 3-component re-exporter chains (resource_floats, resource_with_lists, resource-import-and-export) trapped at runtime with 'wasm unreachable' (rustc/LLVM assume(ptr.is_aligned()) debug assertion firing because leaf code dereferenced the rep as a Box<T> pointer when the rep was actually a small handle integer). Convergent diagnosis from prototype paths D + E pinpointed the seam: merger.rs:739 redirected ALL [resource-*] imports of any re-exporter component through one shared ht_* regardless of which resource the import was for. Changes: - resolver.rs: add reexporter_resources field to DependencyGraph, populated from adapter sites where callee_defines_resource=false. - merger.rs: rekey handle_tables by (comp, iface, resource_name); refactor allocate_handle_tables to allocate one table per (component, resource); refactor the redirect loop with [export]-prefix / resource-graph discrimination to determine the owner of each [resource-*] import; add ht_export_suffix helper. - fact.rs: add parse_resource_import helper; update three handle-table lookup sites to use per-resource keys (caller-side ht_rep, callee-side ht_new, callee-side ht_rep for own results). - component_wrap.rs: update LocalResource ht_* lookup to use the new per-resource export naming. Validation: - 73-test wit-bindgen suite passes with 0 regressions. - The originally-failing trio's trap shape changes from 'wasm unreachable' to 'unknown handle index 0x110004' — proving the per-resource routing now works (leaf no longer derefs handle as Box<T>) and narrowing the diagnosis to a remaining wrapper-boundary leak. - Opaque-rep oracle (pulseengine/wit-bindgen feat/opaque-rep-attribute, resource_floats_opaque) continues to pass fuse oracle. Known follow-up: at the outer wasmtime canon-lift boundary, a memory-address handle from ht_new (table_base + 4) escapes into wasmtime's resource validation. Needs unwrap shim in component_wrap.rs to translate memory-address handles back to canonical-ABI indices when the fused module's exports cross the wrapper boundary.
…ports Three follow-ups to the path B refactor (commit e027bb1) that extend per-resource handle table routing to additional cases the original commit didn't cover: 1. Iterate ALL components in merger.rs:739, not just those with their own handle tables. A pure consumer (the runner in a 3-component chain) holds handles allocated by the re-exporter's ht_new and must drop them through that same handle table — its [resource-drop] imports also need redirection. 2. component_wrap.rs:1254 falls back to ANY component's handle table matching (iface, rn) when the importer's own component_idx doesn't have one. Mirrors the merger.rs change at the wrapper boundary so consumer-side ImportResolution::LocalResource entries pick up the re-exporter's ht_* export instead of falling through to canonical resource ops. 3. strip_dollar_suffix helper strips meld's $N dedup suffix from resource names before handle-table lookup. The dedup suffix distinguishes duplicate canonical-ABI imports from different components but the canonical resource name is the same, so it must be stripped before keying handle_tables. Status: 73/73 wit-bindgen suite still passes (0 regressions). The originally-failing trio (resource_floats, resource_with_lists, resource-import-and-export) still traps at runtime with 'unknown handle index 0x110004' — the bug now narrowed to wit-bindgen's intermediate code mixing ht_new-returned memory addresses with leaf's canonical handles in the inner-Box / outer-Box composition. Further work needs deeper instrumentation of the actual handle flow across the cabi-export shim. Tracking as continued follow-up.
…back Two refinements to the consumer-side handle-table routing introduced in c104b1b: 1. ImportResolution::LocalResource gains an is_definer flag set by the resolver: true when the import came from an [export]-prefixed module (the importer is the resource's definer / owner), false when the importer is a pure consumer of someone else's resource. The wrapper-side fallback only routes through another component's handle table when is_definer=false. Previously the fallback would incorrectly route a definer's own [resource-new] through a re-exporter's ht_new — causing the definer's canonical [resource-rep] to later return whatever was stored there (a small handle integer instead of the actual Box pointer) and the user code's deref to panic with 'misaligned pointer dereference: address must be a multiple of 0x8 but is 0x2'. 2. merger.rs:739 redirect adds re-exporter-self-import detection: when a non-[export] [resource-*] import comes from a component that ALSO owns a handle table for the same (interface, resource), that import is the inner-component (definer) view from the re- exporter's perspective and must use canonical resource ops, NOT the re-exporter's own ht. Previously a re-exporter's import-side helpers were accidentally routed through its own ht, writing to memory address small_int and corrupting heap state. Status: - 73-test wit-bindgen suite still passes (0 regressions). - resource_floats trio still traps at runtime with 'unknown handle index 0x110004' — the deeper architectural issue (memory-address handles from ht_new escaping into wasmtime canonical resource table at the wrapper boundary) remains. Tracked in issue #107.
Adds the infrastructure for marking individual (interface, resource) pairs
as opaque-rep — for re-exporter components built with the
pulseengine/wit-bindgen feat/opaque-rep-attribute fork that returns
u32 reps instead of boxed pointers. The flag is qualified
(--opaque-rep iface.resource, e.g. test:foo/bar.baz) and repeatable.
What's plumbed:
- meld-cli: --opaque-rep IFACE.RESOURCE arg, parsed to (String, String)
tuples by splitting on the LAST '.' so qualified WIT interface names
(which never contain '.') are preserved intact.
- FuserConfig.opaque_resources: Vec<(String, String)> field.
- Threaded through to Merger via with_opaque_resources(...) builder, and
to wrap_as_component (currently the wrapper-side parameter is unused
pending the right conditional strategy).
- All existing test FuserConfig initializers updated.
What's NOT yet doing anything (intentionally):
- Both conditional strategies tried in this spike were dead ends:
(a) per-component local_resource_types keying alone produces a NEW
'handle index used with the wrong type' trap because wasmtime's
type system rejects cross-component handle passing for
per-component-typed resources. Reverted.
(b) identity ht_* functions break opaque-rep storage semantics —
the rep stored at ht_new(rep) must be retrievable by a LATER
ht_rep(handle), but identity collapses these. Reverted.
- The flag currently has no behavioral effect; the architectural fix
for opaque-rep + meld fusion needs more design work.
Validation:
- Standard wit-bindgen suite: 73/73 (0 regressions).
- Full meld test suite: all 15 integration test files pass.
- Opaque oracle still traps with same shapes from issue #107.
The infrastructure is a clean foundation for future iteration on
conditional opaque-rep handling without re-doing the CLI/plumbing
work.
When intermediate has 'use test.{float}' (re-exporting leaf's test.float
as exports.float), wasmtime unifies them into one canonical resource
type. Path B previously refused to redirect [export]-prefixed (definer)
imports — leaf's [export]test:resource-floats/test [resource-rep]float
fell through to canon resource.rep, which never saw the memory-pointer
handles minted by intermediate's ht_new. Trap: 'unknown handle index'.
Fix: add a resource-name-only fallback in BOTH the merger redirect
(merger.rs:864) and the wrapper resolver (component_wrap.rs:1306) for
definer imports. When self-lookup misses, search any handle table
matching just the resource_name. In well-formed compositions there's
at most one re-exporter per logical resource, so the match is
unambiguous.
Status: 73/73 standard suite still passes (0 regressions). The trio
trap shape changes from 'unknown handle index 0x110004' (1st ht
entry) to 'unknown handle index 0x110010' (5th ht entry) — proving
more redirects are firing AND more handles are being allocated, but
something else still leaks to canon resource.*. Investigation ongoing.
…ame unification
The consumer-side LocalResource fallback was too aggressive: it blocked
redirect when the importing component owned a handle table for ANY
resource ending in the same name (e.g. intermediate's
'test:resource-floats/test [resource-rep]float' was blocked because
intermediate also has 'exports.float' ht — even though they're SEPARATE
imports unified at canon-type via 'use test.{float}').
Refine: check self-owns by SPECIFIC (component, iface, resource) tuple,
not by any-iface match. The resource-name-only alias fallback then
catches the unified consumer view and routes through the lone re-exporter
ht_drop / ht_rep / ht_new.
Also added an Instance::ResourceDrop alias-fallback at component_wrap.rs
(category C path) — for completeness, though resource_floats happens to
not hit that path. Keeps it symmetric with the LocalResource path.
Result on the trio:
- resource_floats: NOW PASSES (was 'unknown handle index 0x110004')
- resource_with_lists: still traps (wasm unreachable; was Option::unwrap)
- resource-import-and-export: still traps (wasm unreachable; was Option::unwrap)
Standard 73-test suite: 73/73, 0 regressions.
The remaining two trio failures are likely the same architectural issue
in a slightly different shape — investigation continuing in Phase 5.
The previous ht_drop just zeroed mem[handle] — the wit-bindgen-rust
Box that backs the rep was never freed AND the canonical-ABI lifetime
hook (`[dtor]<resource>` exported by the owning component) never
fired. Wit-bindgen's _resource_dtor uses Box::from_raw to drop the
Box, which runs user-defined Drop impls (including any Option::take
on inner-handle fields).
Fix: in allocate_handle_tables, look up the matching dtor export
`<iface>#[dtor]<rn>` whose function origin matches the ht-owning
component, and prepend a call to it in ht_drop before zeroing.
Implementation:
- Search merged.exports for entries whose name contains `#[dtor]<rn>`
- Filter by func origin component matching the ht owner
- If found, ht_drop emits: if (handle != 0) { call dtor(load mem[handle]); store 0 }
- If not found (no Box-backed dtor — e.g. opaque-rep), just zero the slot
- handle == 0 short-circuit prevents double-free if drop is called twice
Status:
- 73/73 standard suite still passes (0 regressions)
- resource_floats: still PASSES
- resource_with_lists, resource-import-and-export: still trap with
Option::unwrap() — but the dtor wiring is correct in principle and
needed for memory-correctness anyway. The remaining trap is the
cross-component handle aliasing problem (handles from different
components share one ht namespace; deeper architectural fix needed).
The previous `name.contains("#[dtor]<rn>")` lookup picked the FIRST
matching export regardless of which interface owned it. For fixtures
with the same resource name across multiple interfaces (e.g.
resource_floats has dtors for `exports#[dtor]float`,
`imports#[dtor]float`, and `test:resource-floats/test#[dtor]float`
under one component), the contains-match could wire the wrong dtor.
The origin-comp filter (`func.origin.0 == comp_idx`) doesn't
disambiguate when one component defines the same resource in
multiple interfaces (which is exactly the re-exporter pattern with
`use foo.{r}`). `find_map` short-circuits on the first match,
making this latent unless the export iteration order happens to
favor the wrong one.
Mythos slop-hunt discover stage flagged this as
'buggy-but-untested' — `resource_floats` happened to pick the right
dtor by iteration order, masking the bug. Tightening to exact match
`<iface>#[dtor]<rn>` (with optional `$N` dedup suffix) makes the
selection unambiguous.
Validation: 73/73 standard suite unchanged. resource_floats runtime
still passes. The fix is safe (more specific match) and removes the
latent bug.
Previously reexporter_resources only listed re-exporter components that need handle tables (where callee_defines_resource=false in adapter sites). Option A's per-component-tables-with-bridging architecture also needs the DEFINER component to have its own ht so cross-component handle hand-offs can translate via Phase 3 bridging trampolines: caller_handle -> caller_ht_rep -> rep -> callee_ht_new -> callee_handle. After this change, resource_with_lists shows TWO ht entries (was 1): comp 2 (intermediate/re-exporter) — its existing exports ht comp 0 (leaf/definer) — new, allocated by Phase 1 Standard suite: 73/73, 0 regressions. resource_floats runtime still passes (no regression). resource_with_lists still traps — that's expected; Phases 2 (definer-fallback restriction) and 3 (bridging trampolines in fact.rs) still pending.
…ndles
When per-component handle tables exist on BOTH sides of a cross-
component call (post-Phase-1), the handle value space is no longer
shared — a memory-pointer handle from one component's ht refers to
that component's memory, not the other's. Without translation, the
receiving component's ht_rep loads from its own memory at the
caller's offset → garbage.
Phase 3 inserts bridging trampolines in the FACT adapter:
own<T> result transfer (callee_defines_resource=true case, lines 715+):
When both caller and callee have their own ht for the resource,
emit ResourceOwnResultTransfer { rep_func: callee.ht_rep,
new_func: caller.ht_new }. The pre-existing emit_resource_new_results
emits (call rep_func)(call new_func) which performs the bridge:
callee_handle → callee.ht_rep → rep → caller.ht_new → caller_handle.
borrow<T> param transfer (callee_defines_resource=true, lines 598+):
When BOTH the caller has its own ht AND callee has one too, emit
ResourceBorrowTransfer with callee.ht_new in the new_func slot
(was None previously). Same emit phase produces the bridge.
Gating: bridge fires ONLY when caller and callee are different
components AND caller has its own ht for the resource. Without the
caller-has-ht check, the bridge over-fires for fixtures where the
caller uses canonical [resource-rep] (no ht), corrupting handles by
double-translating canonical → memory-pointer. The check restored
resource_floats after an initial regression.
Status:
- 73/73 standard suite passes (0 regressions)
- resource_floats runtime: still PASSES
- resource_with_lists, resource-import-and-export: still trap with
Option::unwrap on None — bridges fire correctly per debug logs but
the deeper issue is upstream (likely accidental ht_drop firing on
cross-memory bridged box pointers, OR multi-memory addressing of
reps stored in one component's memory and loaded by another).
The bridging architecture is correct in principle — both fixtures
need additional investigation of where the box pointer gets freed or
mis-addressed. Tracking under issue #107.
Three layered fixes get resource_floats, resource_with_lists, and resource-import-and-export passing as runtime_test! fixtures: 1. fact.rs — discriminate the borrow lower based on whether the callee's exported function is a `[method]/[static]/[constructor]` on a locally defined resource (cabi expects REP) versus a top-level function taking borrow<T> on a `use`d resource (cabi expects HANDLE). Suppress `callee.ht_new` for the former; the rep-only path matches the canonical ABI lower of borrow<T>. Without this, the slot address minted by callee.ht_new gets passed as the rep, the export's `ThingBorrow::lift` derefs it, and Option's discriminant byte is the low byte of the stored rep (0 for typical aligned box pointers) → Option::unwrap on None. Applied symmetrically in both the 2-component (callee_defines=true) and 3-component (callee_defines=false) borrow branches. 2. merger.rs — suppress the ht_drop dtor invocation for re-exporter components. Phase 1's per-component HTs for definers store foreign reps placed there by own bridges (intermediate's HT contains leaf box pointers). The dtor (`<iface>#[dtor]<rn>`) casts every stored value as `*mut _ThingRep<LocalT>` and Box::from_raw drops it; for foreign reps this misinterprets memory and triggers re-entrant drops via the wit-bindgen Resource::drop impl, producing unbounded recursion. The standard wit-bindgen design assumes the rep is owned by the component whose ht stores it, but Phase 1 broke that invariant. 3. resolver.rs — sort reexporter_components and reexporter_resources before storing on the graph. They were collected from HashSet, whose iteration order is non-deterministic. Downstream HT allocation and the wrapper alias-fallback both make first-match decisions on this order, so the same fixture would sometimes wire `[resource-drop]` for the runner to leaf's ht_drop and sometimes intermediate's, producing the "passes manually, fails in cargo test" flakiness. Promotes resource_floats, resource_with_lists, and resource-import-and-export from fuse_only_test! to runtime_test! — closes the trio runtime failures tracked in issue #75. Standard 73-test suite still passes. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Trio runtime fixtures now pass — `resource_floats`, `resource_with_lists`, and `resource-import-and-export` execute correctly under wasmtime as runtime tests (#75 closed). Builds on path B + alias-fallback (#108) and the per-component-handle-table bridging work (#109). New CHANGELOG.md documents the user-visible changes and internal fixes landing in 0.3.0. NOTE: blocked on #108 and #109 merging to main first. Rebase this branch onto post-merge main before tagging.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Bumps the workspace from
0.2.0→0.3.0and adds a CHANGELOG.md entry covering the changes landing in 0.3.0.This PR is draft until both of the following merge to
main:Once those land, rebase this branch onto post-merge main so the diff shows only the version bump and CHANGELOG (currently it shows the trio fix commits because it's branched off
feat/per-component-ht-bridging):Pre-release checklist (per AGENTS.md)
cargo test --release --test wit_bindgen_runtime— 73/73 (will be re-verified post-rebase)confirmedfindings, or everyconfirmedfinding maps to anapproved LS-Ninsafety/stpa/loss-scenarios.yamlwith a shipped fixWhat's in 0.3.0
See
CHANGELOG.mdfor the full entry. Headlines:Added
resource_floats,resource_with_lists,resource-import-and-export(closes Promote resource_floats and resource_with_lists to runtime tests #75)--opaque-repCLI flagFixed
callee.ht_newstep)ht_dropdtor recursion in re-exportersTest plan
gh pr checks <PR#> --watchpassesresource_floats,resource_with_lists,resource-import-and-export) and confirm()outputBranch
release/v0.3.0— currently 1 commit on top offeat/per-component-ht-bridging(9be9288 chore: bump version to 0.3.0). Will be rebased ontomainonce #108 and #109 merge.