Namespace ancillary codegen output into parallel kind-trees (view::, oneofs::, ext::)#54
Namespace ancillary codegen output into parallel kind-trees (view::, oneofs::, ext::)#54
Conversation
ae6dd3b to
e46e6b9
Compare
e46e6b9 to
8319105
Compare
Moves generated types that previously collided with user-declared proto names into dedicated sub-modules, structurally deconflicting the codegen output from proto-derived names. Layout changes: - Views: `pkg::FooView` → `pkg::view::FooView` (top-level), and `pkg::foo::BarView` → `pkg::view::foo::BarView` (nested). Oneof view enums follow the same tree: `pkg::view::foo::KindView`. The `View` suffix is retained so that owned and view types can be imported into the same scope without renaming. - Extensions: `pkg::FOO` → `pkg::ext::FOO`, and per-file `pkg::register_types` → `pkg::ext::register_types`. - Oneof enums: `foo::KindOneof` → `foo::Kind`. The suffix is no longer needed for disambiguation; the owner message's sub-module provides the namespace. Oneof enums themselves remain in the owner message's sub-module (`pkg::foo::Kind`). The view-tree move makes several previously-illegal proto patterns legal — two regression tests in `naming.rs` capture this. Codegen internals: - `GeneratedFile` gains `kind: GeneratedFileKind` + `package: String` fields. Each generated proto now produces three sibling output files (`.rs`, `.__view.rs`, `.__ext.rs`); `generate_module_tree` coalesces them per package so multiple `.proto` files that share a Rust module (e.g. `google.rpc`) merge their `view::` / `ext::` blocks cleanly. - `generate_module_tree` signature widens from `&[(&str, &str)]` to `&[ModuleTreeEntry<'_>]` to carry the kind. - `MessageScope::deeper()` is used for depth-aware path resolution in the parallel view tree. Checked-in generated code regenerated (WKTs + bootstrap descriptor types). Conformance suite, googleapis stress test, and the existing test suite (1442 tests) all pass.
Post-rebase onto main (which now includes #39): convert the prelude_shadow include to include_generated! and update the oneof enum path to oneofs::picker::option::Value (no Oneof suffix under namespacing).
8319105 to
e62b8d0
Compare
The benchmarks/buffa and benchmarks/gen-datasets crates use hand-written include!() blocks (not the auto-generated _include.rs), so they were silently broken under namespacing — generated owned-struct code references super::oneofs::... which only resolves when the four .__view/.__oneofs/.__view_oneofs/.__ext files are all included. CI did not catch this because these crates are outside the workspace. - Add a local include_generated! macro to both crates (mirrors the buffa-test pattern). - Update view-type and oneof-enum imports in benches/protobuf.rs and gen-datasets/main.rs to the namespaced paths.
9ae726f to
0482699
Compare
|
[claude code] Addressed the kind-tree collision feedback: all three ancillary modules now live under a single Kept the existing tree shape under the wrapper ( Also fixed a latent issue found during this: One known follow-up: |
Addresses the collision regression rpb identified: the unguarded
pub mod view {} / pub mod ext {} / pub mod oneofs {} wrappers at every
package depth could collide with proto-derived module names — e.g. a
user message named View with a nested type, or package foo alongside
package foo.view. Both produced E0428 with no detection.
All ancillary kind trees now live under a single pub mod __buffa { ... }
parent at each package level. The __buffa prefix is the only reserved
name codegen claims, and check_module_name_conflicts now rejects any
proto message whose snake_case starts with __buffa (new ReservedName
error variant), so the wrapper can never collide.
Path layout (minimal-delta — tree shape under __buffa unchanged):
pkg::Foo (owned, unchanged)
pkg::__buffa::view::FooView<'a>
pkg::__buffa::view::oneofs::foo::Kind<'a>
pkg::__buffa::oneofs::foo::Kind
pkg::__buffa::ext::CONST / register_types()
Codegen changes:
- view.rs: initial nesting 1→2; view_oneofs (n-2) supers;
owned_oneofs from view (n-1) supers; rewrite_to_view_path strips two
super:: hops same-package, injects __buffa::view:: cross-package.
- message.rs: oneofs_path_prefix emits __buffa::oneofs::; oneof-enum
nesting bump n+1→n+2.
- lib.rs: file-level extension nesting 1→2; register_types refs climb
super::super::; emit_register_fn=false now also routes file-level
extensions into __ext.rs (fixes a latent wrong-nesting path);
generate_module_tree wraps the three kind mods in pub mod __buffa.
- buffa-build generate_include_file: same wrapping.
Consumer updates: include_generated! macro shape in buffa-test,
conformance, benchmarks; explicit includes in buffa-types; sed-based
::view::/::oneofs::/::ext:: → ::__buffa::… path rewrite across tests,
examples, benches; clippy::module_inception added to allow lists for
the message-View-in-view-tree case.
Regression tests:
- naming.rs: message View { Inner } generates cleanly (rpb repro 1);
message named __buffa rejected with ReservedName.
- buffa-build: package foo + package foo.view emits one
pub mod view at foo:: depth (rpb repro 2).
- prelude_shadow.proto: end-to-end message View { Inner } compile.
Regenerated WKT, bootstrap, and logging-example checked-in code.
ae7e632 to
c571068
Compare
|
The
(I'd thought the file-level enum reservation was a gap too but Also we should just pick |
…age stitcher (#62) ## Summary Ancillary types emitted by `buffa-codegen` — view structs, oneof enums, oneof view enums, extension consts, and `register_types` — currently share the package-level Rust namespace with user proto types. Suffix-based disambiguation (`FooView`, `KindOneof`) handles most cases but still rejects pathological inputs (`message FooView` next to `message Foo`; nested type literally named `{X}Oneof` next to `oneof {x}`) with a codegen error. This PR moves all ancillary kinds under a single reserved sentinel module `__buffa::` per package, making collisions structurally impossible without name mangling, and has codegen emit a per-package `<pkg>.mod.rs` stitcher so consumers integrate via one `buffa::include_proto!("pkg")` call instead of hand-authoring the module wrapper. Supersedes #54 and #37. Closes #32. ## Layout | Item | Before | After | |---|---|---| | Owned message | `pkg::Foo` | `pkg::Foo` (unchanged) | | View struct | `pkg::FooView` | `pkg::__buffa::view::FooView` | | Nested view | `pkg::foo::BarView` | `pkg::__buffa::view::foo::BarView` | | Oneof enum | `pkg::foo::KindOneof` | `pkg::__buffa::oneof::foo::Kind` | | View-of-oneof | `pkg::foo::KindOneofView` | `pkg::__buffa::view::oneof::foo::Kind` | | Extension const | `pkg::FOO` | `pkg::__buffa::ext::FOO` | | Registration fn | `pkg::register_types` (per file) | `pkg::__buffa::register_types` (per package) | The sentinel `__buffa` is the **only** name codegen reserves in user namespace. It aligns with the existing `__buffa_` reserved-field-name prefix (`__buffa_cached_size`, `__buffa_unknown_fields`), so the rule is uniformly "anything starting `__buffa` is reserved." `validate_file` errors with `CodeGenError::ReservedModuleName` if any proto package segment, message, or file-level enum would produce a `__buffa`-named module/type. Adding future ancillary kinds nests them under the existing sentinel; no new reserved words. **No convenience re-exports.** Short-path aliases (`pub use __buffa::view::*`) are intentionally not emitted — they would work in the common case but silently change meaning when a user proto shadows them. The canonical `__buffa::` path is unconditional. ## Per-package `.mod.rs` stitcher Codegen now emits, in addition to the five per-proto content files (`<stem>.rs`, `<stem>.__view.rs`, `<stem>.__oneof.rs`, `<stem>.__view_oneof.rs`, `<stem>.__ext.rs`), one `<dotted.pkg>.mod.rs` per package containing the `pub mod __buffa { … }` wrapper and a single merged `register_types` body. Consumers reference only the stitcher: ```rust pub mod my_pkg { buffa::include_proto!("my.pkg"); // → include!(OUT_DIR/my.pkg.mod.rs) } ``` Per-proto content files remain 1:1 with input protos (split-invocation safe for build systems that compile package subsets independently). The stitcher is the only per-package artifact and has the same caveat as any generated include file. This collapses the wrapper-authoring previously duplicated across `buffa-build`, `buffa-types`, `buffa-test`, `conformance`, examples, benchmarks, `gen_lib_rs.py`, and `protoc-gen-buffa-packaging` to one place in codegen (the canonical `ALLOW_LINTS` list lives there too). `buffa-build::generate_include_file` now delegates to `buffa_codegen::generate_module_tree`. ## `register_types` per-package Because the stitcher is per-package, `register_types` is naturally one fn per package. The `emit_register_fn=false` workaround in `gen_wkt_types.rs` (seven WKT files × one fn each colliding in `google.protobuf`) is no longer needed. ## Deleted - `CodeGenError::OneofNameConflict` and `::ViewNameConflict` — collisions are structurally impossible. - `proto_path_to_rust_module` is `#[deprecated]` (consumers should use the stitcher path); follow-up will remove it. ## Test plan - [x] `task lint` — fmt + clippy clean (verified against rustc 1.95) - [x] `task lint-md` — markdownlint clean - [x] `task test` — 1462 tests pass - [x] `task verify` — 6× `CONFORMANCE SUITE PASSED`, baseline numbers exact - [x] `task stress-googleapis` — 47 protos compile clean - [x] examples (addressbook/envelope/logging), benchmarks — compile clean - [x] regression tests for `message View { message Inner {} }` (codegen unit + `prelude_shadow.proto` end-to-end) and `package foo.view;` - [x] reservation tests for `package foo.__buffa;`, `message __Buffa {}`, and `enum __buffa {}` ## Breaking changes API-breaking for downstream consumers of generated code (import paths for views, oneofs, extensions, `register_types`). Migration is mechanical per the layout table above. Pre-1.0. --------- Co-authored-by: Iain McGinniss <309153+iainmcgin@users.noreply.github.com>
Summary
Ancillary types emitted by
buffa-codegen— borrowed views, oneof enums, oneof view enums, extension consts, and the per-file registration fn — previously lived in the same Rust namespace as user-declared proto types and could collide with them. For example, amessage FooViewin a.protowould clash with the generated view type of siblingmessage Foo;message DataType { oneof data_type {...} }would put a struct and an enum of the same name side by side.This PR moves every ancillary generated type into a dedicated kind-namespaced parallel tree (
view::,oneofs::,ext::,view::oneofs::), structurally deconflicting the codegen output from proto-derived names. The design rule (now documented inDESIGN.md) is:Layout changes
pkg::FooViewpkg::view::FooViewpkg::foo::BarView(nested msg view)pkg::view::foo::BarViewpkg::foo::KindOneof(oneof enum)pkg::oneofs::foo::Kindpkg::foo::KindOneofView(oneof view enum)pkg::view::oneofs::foo::Kindpkg::FOO(extension const)pkg::ext::FOOpkg::register_typespkg::ext::register_typesSuffix policy
Viewsuffix (pkg::view::foo::FooView) — this is the one documented exception to the "tree disambiguates, so drop the suffix" rule. Rationale: users commonly import a message and its view into the same scope (use pkg::{Foo, view::FooView};), and a scope-level rename would be friction for every call site.Kind.Co-import ergonomics
Codegen internals
GeneratedFilegainskind: GeneratedFileKindandpackage: Stringfields.GeneratedFileKindhas five variants:Owned,View,Ext,Oneofs,ViewOneofs. Each proto produces five sibling output files (.rs,.__view.rs,.__ext.rs,.__oneofs.rs,.__view_oneofs.rs); empty bodies when the kind has no content for that proto.generate_module_treeaccepts&[ModuleTreeEntry<'_>]and coalesces per-package streams across multiple.protofiles so packages that span files (e.g.google.rpc) merge theirview::/ext::/oneofs::/view::oneofs::blocks into single wrappers per kind.CodeGenError::OneofNameConflictand the reserved-name machinery around oneof-vs-nested-type collisions are deleted — the collisions they guarded against are now structurally impossible.generate_views=false, so its.__view*.rssiblings are empty stubs; the file count is kept consistent for the module tree stitcher.Deconfliction wins
Patterns that were previously rejected at codegen time (or required
OneofNameConflicterrors with proto renames) now compile as-is:message DataType { oneof data_type {...} }message Foo { nested message Kind {...} oneof kind {...} }message M { nested message FooView {...} oneof foo {...} }my_fieldandmy_field_viewDESIGN.md
Adds a "Generated code layout — parallel trees per kind" section covering the formula, the five-sibling invariant, the path-table worked example for
my.pkg.Foo, theView-suffix exception with rationale, and the three structural benefits (collision-free / predictable / feature-gatable per kind).Migration for downstream consumers
Test plan
cargo check --workspacecleantask test— 1448 tests pass (18 suites), 0 failedtask test-codegen— snapshot tests passtask lint—cargo fmt --checkandcargo clippy --workspace --all-targets -- -D warningsboth cleantask lint-md— markdownlint cleantask conformance— 6CONFORMANCE SUITE PASSEDlines, baseline numbers match exactly (std 5549 / no_std 5529 / view 2797 binary+JSON; 883/883/0 text)task stress-googleapis— 235 files compile cleanly (including multi-filegoogle.rpc)tests/naming.rscollision tests are further updated tocoexistassertions;OneofNameConflicttest references removed with the variantDiff breakdown by category
Total: 113 files, +6339 / -3562.
buffa-codegen/src/*.rs, non-test)coexistregression tests for the deconfliction patterns. Net +162.buffa-buildAPI adaptationModuleTreeEntrythrough the build-script path. Net +61.buffa-testconsumer test suitebuffa-types/buffa-descriptorhand-maintainedlib.rs/mod.rshand-stitch thepub mod view { pub mod oneofs {} },pub mod oneofs,pub mod extwrappers. Net +59.addressbook,envelope,logging)loggingexample. Net +197.protoc-gen-buffa-packagingModuleTreeEntrys per proto. Net +31.gen_lib_rs.py)DESIGN.mdbuffa-types/src/generated/)buffa-descriptor/src/generated/)mod.rs+ sibling stubs. Net +34.examples/loggingoutputtask gen-logging-exampleoutput after the layout move. Net +1050.Human-authored change: ~2048 / 936 (32% / 26%) of the total. The rest is codegen-regenerated output where the file count expands because every proto now emits five siblings, but the per-file content shrinks correspondingly.
Breaking changes
API-breaking for downstream consumers of generated code. Pre-1.0 this is expected; the migration is mechanical (adjust import paths per the table above).