feat: add support for wit files

[rvolosatovs]

Motivation

(sorry, clicked the "open PR" button too fast)

This PR adds support for WIT files https://github.com/WebAssembly/component-model/blob/main/design/mvp/WIT.md, used by WebAssembly components to generate bindings via https://github.com/bytecodealliance/wit-bindgen

Today, .wit files are trimmed by crane, which means bindgen macro cannot find them (it searches by paths) in the project being built, but, more importantly, in dependencies (here's an example https://github.com/wasmCloud/wasmCloud/actions/runs/4415710680/jobs/7739037159

wasmcloud-deps> error: failed to read file "/nix/store/4f8fkwmzdh62bjr3wpl2kphq2vixdkj6-vendor-cargo-deps/225fe4a8a7631cbf30190f12e81bfc52813aac3e2e059f80292a8f336881417f/host-0.0.0/../wit"
)

An example of an "unbuildable" WIT-enabled Rust project, which is often included as a depedency is https://github.com/bytecodealliance/preview2-prototyping

Would this PR be something you'd be interested in merging @ipetkov ?

Checklist

• added tests to verify new behavior
• added an example template or updated an existing one
• updated docs/API.md (or general documentation) with changes
• updated CHANGELOG.md

[ipetkov]

Thanks for looking into this @rvolosatovs!

Would this PR be something you'd be interested in merging @ipetkov ?
I definitely would like to support such (namely WASM) builds but I would really like to avoid having to special case *.wit files from a maintenance perspective (e.g. wouldn't want to go down the path of special casing *.proto files, etc.)

It looks like the crux of the issue is that the preview2-prototyping project has all the *.wit files in their own directory which is NOT part of any crate source. When vendoring crates only directories with Cargo.toml files are extracted, so naturally that directory gets ignored. This is how cargo vendor behaves, though most people don't realize it because consuming crates from crates.io or as a git dependency happens to copy all the files over

If by using cargo vendor with preview2-prototyping as a dependency succeeds then I would consider it a crane bug that ought to be fixed. If, by using cargo vendor with with preview2-prototyping as a dependency fails then my initial reaction would be to try to "fix" the issue upstream (e.g. maybe by symlinking the wit directory within every crate that uses it and avoid having the bindgen! macros try to look outside of the current directory?)

[rvolosatovs]

Thanks for looking into this @rvolosatovs!

Would this PR be something you'd be interested in merging @ipetkov ?
I definitely would like to support such (namely WASM) builds but I would really like to avoid having to special case *.wit files from a maintenance perspective (e.g. wouldn't want to go down the path of special casing *.proto files, etc.)

It looks like the crux of the issue is that the preview2-prototyping project has all the *.wit files in their own directory which is NOT part of any crate source. When vendoring crates only directories with Cargo.toml files are extracted, so naturally that directory gets ignored. This is how cargo vendor behaves, though most people don't realize it because consuming crates from crates.io or as a git dependency happens to copy all the files over

If by using cargo vendor with preview2-prototyping as a dependency succeeds then I would consider it a crane bug that ought to be fixed. If, by using cargo vendor with with preview2-prototyping as a dependency fails then my initial reaction would be to try to "fix" the issue upstream (e.g. maybe by symlinking the wit directory within every crate that uses it and avoid having the bindgen! macros try to look outside of the current directory?)

You're spot on, cargo vendor case fails completely with a path leading outside of the crate root.

Thanks! I'll file an issue upstream!

[ipetkov]

You're spot on, cargo vendor case fails completely with a path leading outside of the crate root.

Thanks! I'll file an issue upstream!

Cool, thanks for confirming! I'll close this out since we've confirmed this is an upstream issue but let me know if you disagree