Skip to content

feat: support Protobuf Editions (2023/2024) and modernize build toolchain#157

Merged
wenchy merged 1 commit intomasterfrom
protobuf-editions
Apr 23, 2026
Merged

feat: support Protobuf Editions (2023/2024) and modernize build toolchain#157
wenchy merged 1 commit intomasterfrom
protobuf-editions

Conversation

@Kybxd
Copy link
Copy Markdown
Collaborator

@Kybxd Kybxd commented Mar 24, 2026

Related to tableauio/tableau#370

Summary

This PR enables Protobuf Editions support across all three code generators
(Go / C++ / C#), upgrades the build toolchain, and makes the protobuf submodule
build pipeline version-aware so that both legacy protobuf v3.x and modern
v4+ / v21+ / v32+ releases can be used interchangeably.

Motivation

  • Protobuf Editions (starting from Edition 2023) is the successor of proto2
    and proto3. Downstream users moving to Editions-based .proto files need
    their protoc plugins to advertise FEATURE_SUPPORTS_EDITIONS, otherwise
    code generation fails.
  • The vendored protobuf submodule was previously pinned to a v3.x layout
    (cmake/ subdirectory, no cmake --install step). Modern protobuf releases
    moved CMakeLists.txt to the repo root and rely on protobuf-config.cmake
    generated via cmake --install, so our init scripts and downstream
    CMakeLists.txt needed to be updated.
  • The Go toolchain / google.golang.org/protobuf runtime had to be bumped so
    that the Editions descriptor APIs (Edition_EDITION_PROTO2,
    Edition_EDITION_2024, SupportedEditionsMinimum/Maximum) are available.

Changes

Code generators: advertise Editions support

  • cmd/protoc-gen-go-tableau-loader/main.go
  • cmd/protoc-gen-cpp-tableau-loader/main.go
  • cmd/protoc-gen-csharp-tableau-loader/main.go

Each plugin now sets:

gen.SupportedFeatures = uint64(
    pluginpb.CodeGeneratorResponse_FEATURE_PROTO3_OPTIONAL |
    pluginpb.CodeGeneratorResponse_FEATURE_SUPPORTS_EDITIONS,
)
gen.SupportedEditionsMinimum = descriptorpb.Edition_EDITION_PROTO2
gen.SupportedEditionsMaximum = descriptorpb.Edition_EDITION_2024

Toolchain upgrade

  • go.mod: go 1.21go 1.24.0;
    google.golang.org/protobuf v1.34.2v1.36.11.
  • Release / testing workflows: Go 1.21.x1.24.x.
  • test/csharp-tableau-loader/Loader.csproj:
    Google.Protobuf 3.19.33.32.0.

Version-aware protobuf submodule build (init.sh / init.bat)

  • Support a new PROTOBUF_REF env var to check out any protobuf tag before
    building (used by CI to matrix-test multiple protobuf versions).
  • Auto-detect protobuf major version via git describe:
    • v3.x (legacy): build from the cmake/ subdirectory with the original
      minimal options.
    • v4+ / v21+ / v32+ (modern): build from the repo root with
      -Dutf8_range_ENABLE_INSTALL=ON.
  • Switch build directory from build/ to .build/ (already in protobuf's
    .gitignore) to avoid dirtying the submodule working tree.
  • Add a cmake --install .build --prefix .build/_install step so that
    protobuf-config.cmake, absl-*.cmake, and utf8_range-config.cmake are
    emitted for downstream find_package(Protobuf CONFIG) consumers.

Downstream CMake / docs alignment

  • test/cpp-tableau-loader/CMakeLists.txt: look up protobuf under
    third_party/_submodules/protobuf/.build/_install (install tree) instead of
    the old in-source cmake/build tree.
  • README.md: update dev instructions on Linux / Windows / C# to reflect the
    new .build/_install/bin layout for protoc / buf generate.

CI: multi-version protobuf matrix

  • .github/workflows/testing-{go,cpp,csharp}.yml:
    • Add a protobuf-version matrix dimension: ["32.0", "3.19.3"], so each
      PR is validated against both the latest Editions-capable protobuf and the
      legacy v3.x line.
    • Wire PROTOBUF_REF=v${{ matrix.protobuf-version }} into the init script
      so the submodule is checked out at the matching tag before building.
    • Split Install Protoc into two steps: arduino/setup-protoc@v3 for
      modern versions, and the legacy @v1 action for the 3.x line.
    • Bump the C++ job timeout from 10 → 20 minutes to accommodate the modern
      protobuf build on Windows.

Compatibility

  • No changes to generated code or runtime behavior for existing proto2 /
    proto3 users.
  • Existing users who continue to use protobuf v3.19.x can keep doing so —
    the init scripts detect the version and fall back to the legacy layout.
  • Users on protobuf v32+ / Editions 2023/2024 can now generate loader code
    without hitting feature not supported: EDITIONS errors.

Testing

  • CI matrix: {ubuntu-latest, windows-latest} × {protobuf 32.0, protobuf 3.19.3}
    across Go / C++ / C# suites.
  • Local smoke test on Linux with protobuf v32.0 submodule + Editions-based
    .proto inputs.

@Kybxd Kybxd force-pushed the protobuf-editions branch 7 times, most recently from 40a3727 to a5e0388 Compare April 1, 2026 03:14
@Kybxd Kybxd force-pushed the protobuf-editions branch from 2a65859 to d89e6f8 Compare April 2, 2026 07:33
Comment thread .github/workflows/release-cpp.yml
// versions:
// - protoc-gen-cpp-tableau-loader v0.11.0
// - protoc v3.19.3
// - protoc v6.32.0
Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For better compatibilty, keep v3.19.3 for commited source code. the new protoc generated code has more dependencies (e.g.: C++ absl).

We can test the new features with a feature gate option to enable/disable in the plugin params.

refer to https://github.com/tableauio/loader/blob/v0.5.0/cmd/protoc-gen-cpp-tableau-loader/main.go#L43

Copy link
Copy Markdown
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

okay, our workflow guarantees the generated code works well with protobuf v32.0.

@Kybxd Kybxd force-pushed the protobuf-editions branch 2 times, most recently from 9eced98 to d57868a Compare April 23, 2026 03:04
chore: Update tableau submodule to fe374083

build: Update Go version to 1.24.x across CI/workflows

feat: Add CMake install step for protobuf-config.cmake

refactor: Replace string reference with std::string construction

chore: Update protoc path to use .build directory

chore: Upgrade Protoc and simplify CI workflows

ci: Optimize CI protobuf setup and caching

feat: Add custom Protobuf log handler when ABSL is absent

refactor: Switch to version-based Protobuf log handler guards

ci: Support legacy protoc versions in CI matrix

chore: Remove protobuf skip-build checks across scripts

chore: Unify protobuf setup and CI across platforms

build: Set minimum CMake policy version to 3.5

chore: unify and simplify CI matrix configs

chore: downgrade protobuf to v3.19.3 and go to 1.23.x
Update CI configs and generated files to use protobuf v3.19.3 and Go 1.23.x

chore: Downgrade Go to 1.23.x in release workflows
@Kybxd Kybxd force-pushed the protobuf-editions branch from d57868a to b56f85b Compare April 23, 2026 03:22
@Kybxd Kybxd changed the title feat: support protobuf editions feat: support Protobuf Editions (2023/2024) and modernize build toolchain Apr 23, 2026
@wenchy wenchy merged commit a94bf95 into master Apr 23, 2026
12 checks passed
@wenchy wenchy deleted the protobuf-editions branch April 23, 2026 12:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants