Symptom
During a fork-replay demo (charon replay --block 91323624 --borrower-file /tmp/seed.txt --hold-secs 1800), the Grafana dashboard's predicted-profit stat panel displayed $0 even though Prometheus had the correct value:
charon_executor_profit_usd_cents_sum{chain="bnb"} 3952448 # = $39,524.48
Other panels with no chain filter (queue depth, positions-by-bucket) populated correctly.
Root cause
The dashboard's \$chain template variable is sourced from:
label_values(charon_listener_blocks_received_total, chain)
charon_listener_blocks_received_total is incremented only by the listener subcommand on each new_heads WS event. In replay mode, no listener runs, so the metric is empty, and label_values() returns an empty set. With \$chain empty, every panel that filters chain=~"\$chain" matches no series.
The 13 \$chain consumers in the dashboard JSON include the predicted-profit stat panel (line 226), histogram heatmap, and several rate panels. All silently render $0 / empty during replay.
Fix
Change the variable source to a metric that emits in both listen and replay modes. charon_scanner_positions is a registered gauge (crates/charon-metrics/src/lib.rs:482-485) set unconditionally on every scanner tick in crates/charon-cli/src/main.rs:1958, 1998, 2026. Replay invokes the same run_block_pipeline (crates/charon-cli/src/main.rs:1833) that emits these gauges, so chain="bnb" is always present in the registry on first scrape.
Reproduction (before fix)
cd deploy/compose && docker compose -f local-stack.yml up -d
cargo run --release --bin charon -- --config config/fork.toml replay --block 91323624 --borrower-file <seed> --hold-secs 1800
- Open Grafana dashboard. Profit panel: $0.
curl -s http://127.0.0.1:9091/metrics | grep profit_usd_cents_sum — metric is 3952448 (= $39 524.48).
Verification (after fix)
Same reproduction, profit panel shows $39 524.48.
Symptom
During a fork-replay demo (
charon replay --block 91323624 --borrower-file /tmp/seed.txt --hold-secs 1800), the Grafana dashboard's predicted-profit stat panel displayed $0 even though Prometheus had the correct value:Other panels with no
chainfilter (queue depth, positions-by-bucket) populated correctly.Root cause
The dashboard's
\$chaintemplate variable is sourced from:charon_listener_blocks_received_totalis incremented only by the listener subcommand on eachnew_headsWS event. Inreplaymode, no listener runs, so the metric is empty, andlabel_values()returns an empty set. With\$chainempty, every panel that filterschain=~"\$chain"matches no series.The 13
\$chainconsumers in the dashboard JSON include the predicted-profit stat panel (line 226), histogram heatmap, and several rate panels. All silently render $0 / empty during replay.Fix
Change the variable source to a metric that emits in both
listenandreplaymodes.charon_scanner_positionsis a registered gauge (crates/charon-metrics/src/lib.rs:482-485) set unconditionally on every scanner tick incrates/charon-cli/src/main.rs:1958, 1998, 2026. Replay invokes the samerun_block_pipeline(crates/charon-cli/src/main.rs:1833) that emits these gauges, sochain="bnb"is always present in the registry on first scrape.Reproduction (before fix)
cd deploy/compose && docker compose -f local-stack.yml up -dcargo run --release --bin charon -- --config config/fork.toml replay --block 91323624 --borrower-file <seed> --hold-secs 1800curl -s http://127.0.0.1:9091/metrics | grep profit_usd_cents_sum— metric is 3952448 (= $39 524.48).Verification (after fix)
Same reproduction, profit panel shows $39 524.48.