Skip to content

docs: update incremental benchmarks (3.0.4)#354

Merged
carlos-alm merged 1 commit intomainfrom
benchmark/incremental-v3.0.4-20260306-231056
Mar 7, 2026
Merged

docs: update incremental benchmarks (3.0.4)#354
carlos-alm merged 1 commit intomainfrom
benchmark/incremental-v3.0.4-20260306-231056

Conversation

@github-actions
Copy link
Contributor

@github-actions github-actions bot commented Mar 6, 2026

Automated incremental benchmark update for 3.0.4 from workflow run #221.

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Mar 6, 2026

Greptile Summary

This is an automated docs-only PR that appends incremental benchmark results for version 3.0.4 to generated/benchmarks/INCREMENTAL-BENCHMARKS.md. The new entries cover both the native (Rust) and WASM engines across full build, no-op rebuild, 1-file rebuild, and import resolution metrics. The embedded JSON data block and percentage deltas are all internally consistent and mathematically correct.

  • Native full build improved 41% (1.8s → 1.0s) — a meaningful gain likely from optimisations in the Rust engine.
  • WASM 1-file rebuild regressed 12% (505ms → 568ms) — minor.
  • ⚠️ Native no-op rebuild regressed +7875% (4ms → 319ms) — this is by far the most striking data point. A no-op rebuild should only re-check file hashes; an ~80× slowdown exclusively on the native engine (WASM holds steady at 5ms) is a strong indicator of a performance bug in 3.0.4's native incremental path and should be investigated before the release is broadly adopted.
  • The resolve.nativeBatchMs raw value is 3.6 (table rounds to 4ms); the ↑6% delta is correctly calculated against the underlying float values.

Confidence Score: 3/5

  • Safe to merge as a docs update, but the 7875% native no-op regression it records should be investigated before 3.0.4 is released.
  • The file itself is generated markdown documentation with no runnable code, so there is no merge risk to the codebase. However, the benchmarks it records expose a severe performance regression in the native engine's no-op rebuild path — an issue that exists in the underlying Rust code and should be resolved before shipping 3.0.4 to users.
  • No files in this PR require special attention. However, the native engine source (crates/codegraph-core/) warrants investigation due to the no-op rebuild regression revealed by these benchmarks.

Flowchart

%%{init: {'theme': 'neutral'}}%%
flowchart TD
    A[Incremental Build Triggered] --> B{Any files changed?}
    B -- No --> C[No-op Rebuild]
    B -- Yes, 1 file --> D[1-File Rebuild]
    B -- Yes, all cold --> E[Full Build]

    C --> F{Engine}
    F -- Native 3.0.3 --> G[4ms ✅]
    F -- Native 3.0.4 --> H[319ms ⚠️ +7875%]
    F -- WASM 3.0.4 --> I[5ms ✅]

    D --> J{Engine}
    J -- Native 3.0.4 --> K[338ms ↓2%]
    J -- WASM 3.0.4 --> L[568ms ↑12%]

    E --> M{Engine}
    M -- Native 3.0.4 --> N[1.0s ↓41% ✅]
    M -- WASM 3.0.4 --> O[2.5s ↑6%]
Loading

Last reviewed commit: 8fdb850


| Version | Engine | Files | Full Build | No-op | 1-File | Resolve (native) | Resolve (JS) |
|---------|--------|------:|-----------:|------:|-------:|------------------:|-------------:|
| 3.0.4 | native | 177 | 1.0s ↓41% | 319ms ↑7875% | 338ms ↓2% | 4ms ↑6% | 4ms ~ |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Extreme no-op rebuild regression in native engine

The native no-op rebuild time jumped from 4ms → 319ms (+7875%) in 3.0.4. A no-op rebuild (nothing changed) should be one of the fastest operations — just re-checking hashes — so a ~80× slowdown is a strong signal of a real performance bug in the native engine for 3.0.4.

The WASM engine's no-op remains healthy at 5ms (unchanged), which rules out a test/environment artifact and points squarely at a regression in the native Rust path. This may indicate that hash-based change detection in the native engine is no longer short-circuiting correctly, or that some initialization cost is now always paid regardless of whether any files changed.

This benchmark is auto-generated and simply records the measured result, but the anomaly it captures warrants investigation before 3.0.4 is widely adopted.

Rule Used: CLAUDE.md (source)

@carlos-alm carlos-alm merged commit 5088cd4 into main Mar 7, 2026
4 checks passed
@carlos-alm carlos-alm deleted the benchmark/incremental-v3.0.4-20260306-231056 branch March 7, 2026 10:50
@github-actions github-actions bot locked and limited conversation to collaborators Mar 7, 2026
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant