Skip to content

Remove proto-lens SRP #1185

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
carbolymer merged 1 commit into from
Apr 17, 2026
Merged

Remove proto-lens SRP #1185

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changes/20260417_cardano_api_bump_cardano_ledger_core.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
project: cardano-api
pr: 1185
kind:
- breaking
description: |
Bump `cardano-ledger-core` lower bound to `>=1.20` (required by `ouroboros-consensus-3.0.x`).
7 changes: 7 additions & 0 deletions .changes/20260417_cardano_api_remove_proto_lens_srp.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,7 @@
project: cardano-api
pr: 1185
kind:
- maintenance
description: |
Remove `proto-lens` source-repository-package now that `proto-lens >=0.7.1.7` is on Hackage.
Move the `cardano-rpc` protobuf generation check to a dedicated CI job using the nix dev shell.
6 changes: 6 additions & 0 deletions .changes/20260417_cardano_rpc_proto_lens_bump.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
project: cardano-rpc
pr: 1185
kind:
- compatible
description: |
Bump `proto-lens` lower bound to `>=0.7.1.7`.
59 changes: 31 additions & 28 deletions .github/workflows/haskell.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,6 +13,9 @@ on:
schedule:
- cron: '0 0 */6 * *'

permissions:
contents: read

jobs:
build:
runs-on: ${{ matrix.sys.os }}
Expand Down Expand Up @@ -95,32 +98,6 @@ jobs:
cabal-store: ${{ steps.setup-haskell.outputs.cabal-store }}
cache-version: ${{ env.CABAL_CACHE_VERSION }}

- name: '[Linux] [cardano-rpc] Install buf tool'
if: runner.os == 'Linux'
run: |
curl -sSL "https://github.com/bufbuild/buf/releases/latest/download/buf-Linux-x86_64" -o "/usr/local/bin/buf"
chmod +x /usr/local/bin/buf

- name: '[Linux] [cardano-rpc] Install proto-lens-protoc tool'
if: runner.os == 'Linux'
run: cabal install proto-lens-protoc --installdir=$HOME/.local/bin

- name: '[Linux] [cardano-rpc] Generate protobuf code'
if: runner.os == 'Linux'
working-directory: cardano-rpc
run: buf generate proto

- name: '[Linux] [cardano-rpc] Check that generated files from proto definitions are up to date'
if: runner.os == 'Linux'
run: |
git add cardano-rpc/gen
if ! git diff --staged --quiet -- cardano-rpc/gen; then
echo "Generated files differ from repository files:"
git diff --staged --name-status -- cardano-rpc/gen
exit 1
fi
echo "All generated files match repository files"

# Now we build.
- name: Build all
run: cabal build all --enable-tests
Expand Down Expand Up @@ -149,14 +126,40 @@ jobs:
# with:
# limit-access-to-actor: true

check-proto-gen:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4

- uses: cachix/install-nix-action@v30
with:
extra_nix_config: |
accept-flake-config = true

- uses: rrbutani/use-nix-shell-action@v1

- name: Regenerate protobuf code
working-directory: cardano-rpc
run: buf generate proto

- name: Check that generated files from proto definitions are up to date
run: |
git add cardano-rpc/gen
if ! git diff --staged --quiet -- cardano-rpc/gen; then
echo "Generated files differ from repository files:"
git diff --staged --name-status -- cardano-rpc/gen
exit 1
fi
echo "All generated files match repository files"

build-complete:
needs: [build]
needs: [build, check-proto-gen]
if: ${{ always() }}
runs-on: ubuntu-latest
steps:
- name: Check if any previous job failed
run: |
if [[ "${{ needs.build.result }}" == "failure" ]]; then
if [[ "${{ needs.build.result }}" == "failure" || "${{ needs.check-proto-gen.result }}" == "failure" ]]; then
# this ignores skipped dependencies
echo 'Required jobs failed to build.'
exit 1
Expand Down
25 changes: 2 additions & 23 deletions cabal.project
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,8 +13,8 @@ repository cardano-haskell-packages
-- See CONTRIBUTING for information about these, including some Nix commands
-- you need to run if you change them
index-state:
, hackage.haskell.org 2026-03-22T23:17:32Z
, cardano-haskell-packages 2026-04-09T12:05:18Z
, hackage.haskell.org 2026-04-17T09:20:55Z
, cardano-haskell-packages 2026-04-15T11:20:53Z
Comment thread
carbolymer marked this conversation as resolved.

active-repositories:
, :rest
Expand Down Expand Up @@ -144,27 +144,6 @@ if arch(wasm32)
package digest
flags: -pkg-config

-- GHC 9.12 and later support in master
if impl(ghc >= 9.12)
source-repository-package
type: git
location: https://github.com/google/proto-lens
tag: 9b41fe0e10e8fe12ec508a3b361d0f0c2217c491
--sha256: sha256-ruTbbUKVJBPANnm6puigtp26mmiVDd0jMpLfJLOuUpU=
subdir:
discrimination-ieee754
proto-lens-arbitrary
proto-lens-benchmarks
proto-lens-discrimination
proto-lens-optparse
proto-lens-protobuf-types
proto-lens-protoc
proto-lens-runtime
proto-lens-setup
proto-lens-tests-dep
proto-lens-tests
proto-lens

-- IMPORTANT
-- Do NOT add more source-repository-package stanzas here unless they are strictly
-- temporary! Please read the section in CONTRIBUTING about updating dependencies.
Comment thread
carbolymer marked this conversation as resolved.
Expand Down
10 changes: 5 additions & 5 deletions cardano-api/cardano-api.cabal
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -146,7 +146,7 @@ library
cardano-ledger-binary >=1.6,
cardano-ledger-byron >=1.2,
cardano-ledger-conway >=1.19,
cardano-ledger-core >=1.17 && <1.20,
cardano-ledger-core >=1.20,
cardano-ledger-dijkstra >=0.1,
cardano-ledger-mary >=1.8,
cardano-ledger-shelley >=1.16,
Expand Down Expand Up @@ -333,12 +333,12 @@ library gen
cardano-binary >=1.6 && <1.9,
cardano-crypto-class ^>=2.3,
cardano-crypto-wrapper:testlib ^>=1.7,
cardano-ledger-alonzo:{cardano-ledger-alonzo, testlib} >=1.8.1,
cardano-ledger-alonzo:{cardano-ledger-alonzo, testlib},
cardano-ledger-byron:testlib,
cardano-ledger-conway:testlib,
cardano-ledger-core:{cardano-ledger-core, testlib} >=1.17,
cardano-ledger-dijkstra:testlib >=0.1,
cardano-ledger-shelley >=1.13,
cardano-ledger-core:{cardano-ledger-core, testlib},
cardano-ledger-dijkstra:testlib,
cardano-ledger-shelley,
cardano-strict-containers,
containers,
filepath,
Expand Down
2 changes: 1 addition & 1 deletion cardano-rpc/cardano-rpc.cabal
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -87,7 +87,7 @@ library
generic-data,
grapesy,
grpc-spec,
proto-lens >=0.7.1.6,
proto-lens >=0.7.1.7,
proto-lens-protobuf-types,
random,
rio,
Expand Down
12 changes: 6 additions & 6 deletions flake.lock
View file
Open in desktop

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

55 changes: 5 additions & 50 deletions flake.nix
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -123,26 +123,6 @@

# We use cabalProject' to ensure we don't build the plan for
# all systems.
# Fetch proto-lens with submodules and fix symlinks for plan and build phases
protoLensSrc = nixpkgs.fetchgit {
url = "https://github.com/google/proto-lens";
rev = "9b41fe0e10e8fe12ec508a3b361d0f0c2217c491";
sha256 = "sha256-ruTbbUKVJBPANnm6puigtp26mmiVDd0jMpLfJLOuUpU=";
fetchSubmodules = true;
};
fixProtoLensSrc = nixpkgs.runCommand "proto-lens-fixed" {} ''
mkdir -p $out
cp -a ${protoLensSrc}/. $out/
chmod -R +w $out
# Fix proto-lens-imports symlink in proto-lens
rm -rf $out/proto-lens/proto-lens-imports/google
cp -r ${protoLensSrc}/google/protobuf/src/google $out/proto-lens/proto-lens-imports/
# Fix proto-src symlink in proto-lens-protobuf-types
rm -rf $out/proto-lens-protobuf-types/proto-src
cp -r ${protoLensSrc}/google/protobuf/src $out/proto-lens-protobuf-types/proto-src
chmod -R -w $out
'';

cabalProject = nixpkgs.haskell-nix.cabalProject' ({config, ...}: {
src = ./.;
name = "cardano-api";
Expand All @@ -160,7 +140,6 @@
#
inputMap = {
"https://chap.intersectmbo.org/" = inputs.CHaP;
"https://github.com/google/proto-lens/9b41fe0e10e8fe12ec508a3b361d0f0c2217c491" = protoLensSrc;
};
# Also currently needed to make `nix flake lock --update-input CHaP` work.
cabalProjectLocal = ''
Expand Down Expand Up @@ -189,6 +168,7 @@
ghcid = "0.8.9";
cabal-gild = "1.7.0.1";
fourmolu = "0.18.0.0";
proto-lens-protoc = "latest";
haskell-language-server = {
src = inputs.hls;
configureArgs = "--disable-benchmarks --disable-tests";
Expand All @@ -209,7 +189,10 @@
shellcheck
snappy
protobuf
buf
# buf version must match `.github/workflows/haskell.yml` (buf is
# not backwards-compatible across minor versions for
# `buf generate` output).
unstable.buf
Comment thread
carbolymer marked this conversation as resolved.
Copy link
Copy Markdown
Contributor

@Jimbo4350 Jimbo4350 Apr 17, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

unstable.buf doesn't actually pin a version. inputs.unstable.url = "nixpkgs/nixos-unstable" is locked by flake.lock today, so it's deterministic for this checkout but the next nix flake update can silently move buf off v1.66.1 and reintroduce the exact drift this PR is trying to fix. The comment above asserts alignment with CI but nothing enforces it. Could we fetch the pinned binary the same way CI does (or otherwise derive both versions from a single source of truth)?

Copy link
Copy Markdown
Contributor Author

@carbolymer carbolymer Apr 17, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the next nix flake update can silently move buf off v1.66.1 and reintroduce the exact drift this PR is trying to fix.

This is fine in my opinion, as flake update is a thing we can control. I'll do the opposite and implement the check using buf from nixpkgs.

Copy link
Copy Markdown
Contributor Author

@carbolymer carbolymer Apr 17, 2026 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I've split buf generate into separate job - that should speed up GHA since it'll run in parallel. It'll also use the same buf as nix development shell, so it'll be pinned to a version from nixpkgs.

blst
inputs.cardano-dev.packages.${system}.herald
(writeShellScriptBin "haskell-language-server-wrapper" ''exec haskell-language-server "$@"'')
Expand All @@ -228,34 +211,6 @@
# package customizations as needed. Where cabal.project is not
# specific enough, or doesn't allow setting these.
modules = [
# TODO remove this module when removing proto-lens SRP
# Override proto-lens source to use fixed symlinks (inputMap provides the fixed
# source for plan computation; this module provides it for the build phase)
({
lib,
config,
...
}: let
protoLensPackages = [
"proto-lens"
"proto-lens-arbitrary"
"proto-lens-discrimination"
"proto-lens-optparse"
"proto-lens-protobuf-types"
"proto-lens-protoc"
"proto-lens-runtime"
"proto-lens-setup"
"proto-lens-tests-dep"
"proto-lens-tests"
"discrimination-ieee754"
"proto-lens-benchmarks"
];
in {
packages =
lib.genAttrs
(builtins.filter (p: config.packages ? ${p}) protoLensPackages)
(p: {src = lib.mkForce (fixProtoLensSrc + "/${p}");});
})
({...}: {
packages.cardano-api = {
configureFlags = ["--ghc-option=-Werror"];
Expand Down
Loading