[grafana] Cumulative profit panel queries histogram _sum directly — resets on process restart appear as losses

[obchain]

Refs #54

Location

deploy/grafana/charon.json — Panel 5 (cumulative profit)

Query

sum(charon_executor_profit_usd_cents_sum{instance="$instance",chain="$chain"}) / 100

Problem

The panel divides the raw histogram _sum accumulator by 100 to express profit in USD. This pattern has two correctness issues:

1. Process restart resets the counter to zero. The panel shows a sudden step-down visible as a negative-profit event. The operator has no way to distinguish a process restart from a real loss without checking uptime.

2. The _sum accumulator tracks the sum of observed values passed to histogram.observe(). This is only valid for computing cumulative profit if each observe() call records exactly one liquidation's profit in cents. If the histogram is used for distribution analysis (recording per-tx profit to compute percentiles), then summing _sum is a secondary use that should be explicit and documented.

Impact

Misleading profit accounting on the primary financial dashboard. A process restart mid-day renders the panel untrustworthy for the rest of the time window.

Suggested Fix

Two options:

1. Add a dedicated charon_executor_profit_usd_cents_total counter (not a histogram) for cumulative tracking, separate from the histogram used for distribution. Use increase() over the counter for a cumulative view.
2. If keeping _sum, use increase(charon_executor_profit_usd_cents_sum[24h]) to show profit for a rolling window, and annotate the panel with 'resets on restart' in the description field.

[obchain]

Fixed on feature branch; see feat/22-prometheus-metrics / feat/26-grafana-dashboard commits.