Skip to content

Use the gungraun benchmark framework to get reliable instruction count numbers on CI#3370

Merged
badboy merged 2 commits intomainfrom
push-povlnvknxktp
Jan 14, 2026
Merged

Use the gungraun benchmark framework to get reliable instruction count numbers on CI#3370
badboy merged 2 commits intomainfrom
push-povlnvknxktp

Conversation

@badboy
Copy link
Copy Markdown
Member

@badboy badboy commented Jan 9, 2026

This is a first refactoring of our benchmarks.

  • It moves everything into a benchmark crate. That way it's separate from the Glean crates and doesn't interfere with the Rust version requirements or dependency requirements. We are a bit more free to use crates without propagating any vetting directly (plus those deps never show up in m-c then).
  • It adds the Gungraun framework, which uses valgrind's cachegrind. The idea is that Instruction count (and some of the other numbers) is a stable measure that should give us an idea of how code is performing.
  • Because the benchmarking does not depend as much on the hardware it's run on (other than the architecture) we can run those on CI and make them comparable.

I plan to collect those numbers over time and provide a dashboard so we have an (after-the-fact) overview of the performance.

jobs:
bench:
name: Benchmarks
environment: ${{ github.ref_name == 'main' && 'production' || 'pull_request' }}
Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I plan on collecting the data and pushing that to another repository. That will require a secret (the push token). For security we split that out into the production environment, so the token is available when actually run on a push to main.

@badboy badboy marked this pull request as ready for review January 13, 2026 11:00
@badboy badboy requested a review from a team as a code owner January 13, 2026 11:00
@badboy badboy requested review from chutten and removed request for a team January 13, 2026 11:00
@badboy badboy force-pushed the push-povlnvknxktp branch from 1f2612a to 8ef9607 Compare January 13, 2026 11:03
@badboy badboy force-pushed the push-povlnvknxktp branch from 8ef9607 to ad22d69 Compare January 13, 2026 11:22
Copy link
Copy Markdown
Contributor

@chutten chutten left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

r+wc

Comment thread glean-core/benchmark/benches/instruction-count.rs Outdated
Comment thread glean-core/benchmark/Cargo.toml Outdated
…t numbers on CI

This also moves it out of the main workspace, so it properly splits out
the dependencies.
@badboy badboy force-pushed the push-povlnvknxktp branch from ad22d69 to 368a4aa Compare January 14, 2026 11:50
@badboy badboy merged commit e5769c1 into main Jan 14, 2026
28 of 29 checks passed
@badboy badboy deleted the push-povlnvknxktp branch January 14, 2026 12:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants