Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
45 changes: 45 additions & 0 deletions DESIGN.md
Original file line number Diff line number Diff line change
Expand Up @@ -225,6 +225,51 @@ let view = OwnedView::<PersonView>::decode(bytes)?;
println!("name: {}", view.name); // Deref, zero-copy, 'static + Send
```

**Generated code layout — the `__buffa::` sentinel tree:**

Ancillary generated items (views, oneof enums, file-level extensions, the per-package `register_types` fn) live under a single reserved module per package — `__buffa::` — instead of being interleaved with owned types. The sentinel is the **only** name buffa reserves in user namespace; codegen errors with `ReservedModuleName` if a proto package segment, message name, or file-level enum name would emit a `__buffa` item at package root.

```text
<pkg>::Foo # owned struct (unchanged)
<pkg>::foo::Bar # nested owned (unchanged)
<pkg>::__buffa::view::FooView<'a> # view struct
<pkg>::__buffa::view::foo::BarView<'a> # nested view (mirrors owned tree)
<pkg>::__buffa::view::oneof::foo::Kind<'a> # view oneof enum (no suffix)
<pkg>::__buffa::oneof::foo::Kind # owned oneof enum (no suffix)
<pkg>::__buffa::ext::MY_EXT # file-level extension const
<pkg>::__buffa::register_types(…) # one fn per package
```

Oneof and view-oneof enums drop the `Oneof`/`View` suffix — the tree position disambiguates. View structs keep the `View` suffix because owned and view types are routinely co-imported (`use pkg::{Foo, __buffa::view::FooView}`).

This makes name collisions **structurally impossible**: a oneof `kind` and a nested message `Kind` can coexist because they land in different trees. There is no suffix-escalation or rename escape hatch; codegen emits proto names verbatim.

**File layout — five content files + one stitcher:**

Each `.proto` emits five sibling content files into `OUT_DIR`:

| File | Contents |
|---------------------------|-------------------------------------------|
| `<stem>.rs` | Owned structs, enums, nested extensions |
| `<stem>.__view.rs` | View structs |
| `<stem>.__oneof.rs` | Owned oneof enums |
| `<stem>.__view_oneof.rs` | View oneof enums |
| `<stem>.__ext.rs` | File-level extension consts |

Each proto **package** additionally emits one `<dotted.pkg>.mod.rs` stitcher that `include!`s the content files and authors the `pub mod __buffa { … }` wrapper. Consumers wire up only the stitcher:

```rust,ignore
pub mod my_pkg {
buffa::include_proto!("my.pkg"); // → include!(OUT_DIR/my.pkg.mod.rs)
}
```

`buffa::include_proto_relative!("dir", "my.pkg")` does the same for checked-in generated code (no `OUT_DIR`). `buffa-build`'s `_include.rs` and `protoc-gen-buffa-packaging` both emit module trees that reference only the stitchers.

The per-proto content files mean editing one `.proto` regenerates only its five siblings (incremental friendly); the per-package stitcher means `register_types` is naturally one fn per package, so multi-file packages (e.g. seven WKT files in `google.protobuf`) no longer collide.

**No convenience re-exports.** Short-path aliases like `pub use __buffa::view::*` at package level are deliberately not emitted. They would make `pkg::FooView` work in the common case but silently change meaning (or disappear) when a user-defined `message FooView` exists — a "clean 95% / surprising 5%" pattern that trades predictability for brevity. The canonical `__buffa::` path is the only path; it is unconditional.

### 3. MessageField\<T\> — Ergonomic Optional Messages

Prost uses `Option<Box<M>>` for optional message fields, which creates unwrapping ceremony everywhere:
Expand Down
4 changes: 3 additions & 1 deletion benchmarks/buffa/benches/protobuf.rs
Original file line number Diff line number Diff line change
Expand Up @@ -2,8 +2,10 @@ use buffa::{Message, MessageView};
use criterion::{criterion_group, criterion_main, Criterion, Throughput};
use serde::{de::DeserializeOwned, Serialize};

use bench_buffa::bench::__buffa::view::*;
use bench_buffa::bench::*;
use bench_buffa::benchmarks::BenchmarkDataset;
use bench_buffa::proto3::__buffa::view::GoogleMessage1View;

fn load_dataset(data: &[u8]) -> BenchmarkDataset {
BenchmarkDataset::decode_from_slice(data).expect("failed to decode dataset")
Expand Down Expand Up @@ -180,7 +182,7 @@ fn bench_google_message1_view(c: &mut Criterion) {
group.bench_function("decode_view", |b| {
b.iter(|| {
for payload in &dataset.payload {
let view = bench_buffa::proto3::GoogleMessage1View::decode_view(payload).unwrap();
let view = GoogleMessage1View::decode_view(payload).unwrap();
criterion::black_box(&view);
}
});
Expand Down
6 changes: 3 additions & 3 deletions benchmarks/buffa/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@
dead_code
)]
pub mod bench {
include!(concat!(env!("OUT_DIR"), "/bench_messages.rs"));
buffa::include_proto!("bench");
}

#[allow(
Expand All @@ -23,7 +23,7 @@ pub mod bench {
dead_code
)]
pub mod benchmarks {
include!(concat!(env!("OUT_DIR"), "/benchmarks.rs"));
buffa::include_proto!("benchmarks");
}

#[allow(
Expand All @@ -36,5 +36,5 @@ pub mod benchmarks {
dead_code
)]
pub mod proto3 {
include!(concat!(env!("OUT_DIR"), "/benchmark_message1_proto3.rs"));
buffa::include_proto!("benchmarks.proto3");
}
6 changes: 3 additions & 3 deletions benchmarks/gen-datasets/src/main.rs
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ use std::path::Path;
dead_code
)]
mod proto {
include!(concat!(env!("OUT_DIR"), "/bench_messages.rs"));
buffa::include_proto!("bench");
}
#[allow(
clippy::derivable_impls,
Expand All @@ -30,11 +30,11 @@ mod proto {
dead_code
)]
mod dataset_proto {
include!(concat!(env!("OUT_DIR"), "/benchmarks.rs"));
buffa::include_proto!("benchmarks");
}

use proto::analytics_event::property::ValueOneof as Value;
use proto::analytics_event::{Nested, Property};
use proto::__buffa::oneof::analytics_event::property::Value;
use proto::log_record::Context;
use proto::*;

Expand Down
94 changes: 16 additions & 78 deletions buffa-build/src/lib.rs
Original file line number Diff line number Diff line change
Expand Up @@ -585,33 +585,24 @@ impl Config {
.collect()
};

// Generate Rust source.
// Generate Rust source. Per-proto content files plus a per-package
// `.mod.rs` stitcher; only the stitchers need wiring into the
// module tree (content files are reached via `include!` from
// there).
let generated =
buffa_codegen::generate(&fds.file, &files_to_generate, &self.codegen_config)?;

// Build a map from generated file name to proto package for the
// module tree generator.
let file_to_package: std::collections::HashMap<String, String> = fds
.file
.iter()
.map(|fd| {
let proto_name = fd.name.as_deref().unwrap_or("");
let rs_name = buffa_codegen::proto_path_to_rust_module(proto_name);
let package = fd.package.as_deref().unwrap_or("").to_string();
(rs_name, package)
})
.collect();

// Write output files and collect (name, package) pairs.
// Write output files; collect (name, package) for PackageMod entries.
let mut output_entries: Vec<(String, String)> = Vec::new();
for file in generated {
let path = out_dir.join(&file.name);
if let Some(parent) = path.parent() {
std::fs::create_dir_all(parent)?;
}
write_if_changed(&path, file.content.as_bytes())?;
let package = file_to_package.get(&file.name).cloned().unwrap_or_default();
output_entries.push((file.name, package));
if file.kind == buffa_codegen::GeneratedFileKind::PackageMod {
output_entries.push((file.name, file.package));
}
}

// Generate the include file if requested.
Expand Down Expand Up @@ -825,67 +816,14 @@ fn proto_relative_name(file: &Path, includes: &[PathBuf]) -> String {
/// instead of the `env!("OUT_DIR")` prefix, so the include file works when
/// checked into the source tree and referenced via `mod`.
fn generate_include_file(entries: &[(String, String)], relative: bool) -> String {
use std::collections::BTreeMap;
use std::fmt::Write;

fn escape_mod_name(name: &str) -> String {
buffa_codegen::idents::escape_mod_ident(name)
}

#[derive(Default)]
struct ModNode {
files: Vec<String>,
children: BTreeMap<String, Self>,
}

let mut root = ModNode::default();
for (file_name, package) in entries {
let pkg_parts: Vec<&str> = if package.is_empty() {
vec![]
} else {
package.split('.').collect()
};
let mut node = &mut root;
for seg in &pkg_parts {
node = node.children.entry(seg.to_string()).or_default();
}
node.files.push(file_name.clone());
}

let mut out = String::new();
writeln!(out, "// @generated by buffa-build. DO NOT EDIT.").unwrap();
writeln!(out).unwrap();

fn emit(out: &mut String, node: &ModNode, depth: usize, relative: bool) {
let indent = " ".repeat(depth);
for file in &node.files {
if relative {
writeln!(out, r#"{indent}include!("{file}");"#).unwrap();
} else {
writeln!(
out,
r#"{indent}include!(concat!(env!("OUT_DIR"), "/{file}"));"#
)
.unwrap();
}
}
for (name, child) in &node.children {
let escaped = escape_mod_name(name);
writeln!(
out,
"{indent}#[allow(non_camel_case_types, dead_code, unused_imports, \
clippy::derivable_impls, clippy::match_single_binding)]"
)
.unwrap();
writeln!(out, "{indent}pub mod {escaped} {{").unwrap();
writeln!(out, "{indent} use super::*;").unwrap();
emit(out, child, depth + 1, relative);
writeln!(out, "{indent}}}").unwrap();
}
}

emit(&mut out, &root, 0, relative);
out
let mode = if relative {
buffa_codegen::IncludeMode::Relative("")
} else {
buffa_codegen::IncludeMode::OutDir
};
// Inner-allow off: this output is consumed via `include!` from
// user-authored `lib.rs`, where `#![allow(...)]` is not valid.
buffa_codegen::generate_module_tree(entries, mode, false)
}

#[cfg(test)]
Expand Down
11 changes: 6 additions & 5 deletions buffa-codegen/src/bin/gen_wkt_types.rs
Original file line number Diff line number Diff line change
Expand Up @@ -80,11 +80,12 @@ fn main() {
// correct. `buffa/text` is zero-dep — enabled unconditionally
// in buffa-types so no feature-gate wrapping is needed.
//
// emit_register_fn = false All seven WKT files are `include!`d into
// one namespace — seven `register_types` fns would collide. WKTs
// register via the hand-written `register_wkt_types` in
// `any_ext.rs` anyway. Per-message `__*_TEXT_ANY` consts are
// still emitted (harmless `#[doc(hidden)] pub`).
// emit_register_fn = false Per-package output means one fn would
// be emitted (all seven WKTs share `google.protobuf`), so the
// old multi-file collision is gone — but WKTs register via the
// hand-written `register_wkt_types` in `any_ext.rs` (which knows
// the JSON-Any `is_wkt` special-casing the generic fn doesn't),
// so the generated fn would be redundant.
let mut config = buffa_codegen::CodeGenConfig::default();
config.generate_views = true;
config.preserve_unknown_fields = true;
Expand Down
Loading
Loading