Skip to content

Prevent over-deep type hierarchies in wasm-smith#2143

Merged
fitzgen merged 2 commits intobytecodealliance:mainfrom
alexcrichton:not-max-depth-subtype
Apr 15, 2025
Merged

Prevent over-deep type hierarchies in wasm-smith#2143
fitzgen merged 2 commits intobytecodealliance:mainfrom
alexcrichton:not-max-depth-subtype

Conversation

@alexcrichton
Copy link
Member

Currently wasm-smith can produce invalid GC modules because there's no limit placed on subtyping depth. This commit keeps track of subtyping depth and appropriately manages the can_subtype list as a result.

Currently wasm-smith can produce invalid GC modules because there's no
limit placed on subtyping depth. This commit keeps track of subtyping
depth and appropriately manages the `can_subtype` list as a result.
@alexcrichton alexcrichton requested a review from fitzgen April 15, 2025 16:05
@alexcrichton
Copy link
Member Author

I don't know why it took fuzzing in Wasmtime to find this.

@alexcrichton
Copy link
Member Author

Well I'm flummoxed. This is fixing a preexisting bug which by all means should have been discovered by fuzzing of just this repo, fuzzing in Wasmtime shouldn't be necessary. Ever since #1793, landed 6 months ago, this should have been discoverable by the validate-valid-module fuzzer.

I double-checked all configuration options Wasmtime is using/setting are at least possible to configure from this fuzzer. Given that I guess it just requires a relatively specific shape of fuzz input and that didn't crop up until later...

Copy link
Member

@fitzgen fitzgen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Rather than calculating this on-demand, and potentially repeating the traversal up the parent chain which could lead to quadratic behavior or worse, I think we want to have a side table and compute the subtyping depth for each type as we generate each type.

More concretely, I'm thinking we want to have a side table similar to super_to_sub_types that maps from a type to its subtyping depth, and we eagerly insert an entry in this table when generating a type. This way, we only ever have to add one to the super type's depth, rather than calculate the whole depth and traverse the whole hierarchy every time.

@fitzgen
Copy link
Member

fitzgen commented Apr 15, 2025

I double-checked all configuration options Wasmtime is using/setting are at least possible to configure from this fuzzer. Given that I guess it just requires a relatively specific shape of fuzz input and that didn't crop up until later...

Huh. I don't have a good answer for this mystery either, other than we just got "lucky" in Wasmtime, or maybe it was somehow rewarding the fuzzer with additional coverage for deeper subtyping in a way that just the parser and validator was not.

@alexcrichton
Copy link
Member Author

Given the maximum recursion depth here is fixed I figured there was no need for quadratic bits, but sure it's not too bad to do that too.

@fitzgen fitzgen added this pull request to the merge queue Apr 15, 2025
Merged via the queue into bytecodealliance:main with commit 90c156f Apr 15, 2025
32 checks passed
@alexcrichton alexcrichton deleted the not-max-depth-subtype branch April 15, 2025 21:14
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants