Skip to content

PV: track energy metrics and apply forecast scaling to optimizer#29784

Merged
andig merged 14 commits into
masterfrom
feature/pv-metrics-and-adjusted-forecast
May 15, 2026
Merged

PV: track energy metrics and apply forecast scaling to optimizer#29784
andig merged 14 commits into
masterfrom
feature/pv-metrics-and-adjusted-forecast

Conversation

@andig
Copy link
Copy Markdown
Member

@andig andig commented May 9, 2026

Summary

Three related changes around PV energy and the solar forecast scale factor:

  1. Per-PV metrics.Collector (core/site.go)

    • Adds pvCollectors map[string]*metrics.Collector keyed by meter ref, group pv
    • Created alongside the existing in-memory pvEnergy accumulator in Boot()
    • Fed in updatePvMeters via AddEnergy(&mm[i].Energy if >0, nil, mm[i].Power) mirroring the grid/battery patterns
    • Each PV meter now surfaces in the History UI as its own series, so individual modules can be tracked / replaced without losing prior history.
  2. Reusable solarScale() helper (core/site_tariffs.go)

    • Extracts the existing produced/forecasted ratio calc out of solarDetails so the optimizer can call it without re-running the side effects (AddImportEnergy, settings persist).
    • Identical thresholds (fcst > 0, produced+fcst > 0.5 kWh) — no behavior change in the published solarDetails.Scale.
  3. Optimizer applies the scale (core/site_optimizer.go)

    • Previously fed the raw solar forecast into the optimizer regardless of how reliable it had been.
    • Now multiplies the pruned solar rates by solarScale() when available, so the optimizer reflects consistent under-/over-production rather than the bare forecast.
  4. Re-expose the adjusted-forecast toggle in the UI (assets/js/views/Forecast.vue)

    • The && false guard was added in Forecast UI: real data adjust (-100 %) #29165 (forecast metrics could go inconsistent on restart).
    • The underlying issue was addressed by resetSolarForecastMetrics, so the experimental toggle is unblocked.

Test plan

  • go build ./... clean
  • gofmt -l clean on touched Go files
  • go vet ./core/... clean
  • go test ./core/... -count=1
  • npm run lint (eslint, vue-tsc, i18n) clean
  • Manual: configure multiple PV meters, observe per-meter rows in meters table under group pv
  • Manual: run for >0.5 kWh of accumulated forecast+produced, confirm solar.scale is published and Forecast view exposes the adjust toggle in experimental mode
  • Manual: confirm optimizer logs reflect scaled solar input (DEBUG solar forecast: ... scale ...)

Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've reviewed your changes and they look great!


Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@andig andig changed the title PV: per-meter energy + apply forecast scale to optimizer PV: track energy metrics and apply forecast scaling to optimizer May 9, 2026
@andig andig marked this pull request as draft May 9, 2026 09:56
@andig andig mentioned this pull request May 9, 2026
6 tasks
@andig andig added enhancement New feature or request prio Priority labels May 9, 2026
@andig andig force-pushed the feature/pv-metrics-and-adjusted-forecast branch from bafadd5 to cd87cd4 Compare May 9, 2026 10:07
@andig andig marked this pull request as ready for review May 9, 2026 10:14
Copy link
Copy Markdown
Contributor

@sourcery-ai sourcery-ai Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hey - I've left some high level feedback:

  • In updatePvMeters, the error log persist pv %d energy only includes the index; consider including the PV meter name (name) to make diagnosing persistence issues easier.
  • solarScale is now called both from solarDetails and the optimizer, which will duplicate the solar forecast: ... DEBUG logs each cycle; consider either centralizing the call or making logging conditional to avoid noisy duplicate entries.
Prompt for AI Agents
Please address the comments from this code review:

## Overall Comments
- In `updatePvMeters`, the error log `persist pv %d energy` only includes the index; consider including the PV meter name (`name`) to make diagnosing persistence issues easier.
- `solarScale` is now called both from `solarDetails` and the optimizer, which will duplicate the `solar forecast: ...` DEBUG logs each cycle; consider either centralizing the call or making logging conditional to avoid noisy duplicate entries.

Sourcery is free for open source - if you like our reviews please consider sharing them ✨
Help me be more useful! Please click 👍 or 👎 on each comment and I'll use the feedback to improve your reviews.

@andig andig force-pushed the feature/pv-metrics-and-adjusted-forecast branch 3 times, most recently from b2ff268 to ec633bd Compare May 9, 2026 10:21
- Add per-PV metrics.Collector (group=pv) so each PV meter's production
  surfaces in the History UI like grid/home/battery/loadpoints.
- Extract solarScale() helper from solarDetails so the scale is reusable.
- Optimizer applies the solarScale factor to the solar forecast input
  rather than feeding raw forecast rates, making the optimization react
  to consistent under-/over-production.
- Re-enable the experimental adjusted-solar-forecast toggle in the
  Forecast view (issue #29165 was addressed in resetSolarForecastMetrics).
@andig andig force-pushed the feature/pv-metrics-and-adjusted-forecast branch from ec633bd to efa07aa Compare May 9, 2026 10:29
Comment thread core/site_tariffs.go Outdated
Comment thread assets/js/components/ChargingPlans/PlanStaticSettings.vue Outdated
Comment thread assets/js/components/ChargingPlans/PlanStaticSettings.vue Outdated
Comment thread assets/js/views/Forecast.vue Outdated
andig and others added 2 commits May 9, 2026 16:02
- Add new metrics collector group "forecast" alongside pv/grid/battery/home
- Replace fcstEnergy *Accumulator with a *metrics.Collector that buckets
  forecast energy into 15-min slots and persists to the metrics DB
- Drop the per-PV-meter pvProduction accumulator map (the existing
  pv energy collector already records production)
- Compute solarScale() by querying QueryImportEnergy("day") for both
  the "pv" and "forecast" groups so the ratio comes from durable data,
  not in-memory accumulators
- Remove the daily-reset bookkeeping (resetSolarAccumulatorsIfNewDay,
  SolarAccDay/SolarAccForecast/SolarAccYield settings) — the day window
  now falls out of the metrics query (BeginningOfDay → now)
- solarScale now returns (float64, bool); update site_optimizer caller

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
@andig andig force-pushed the feature/pv-metrics-and-adjusted-forecast branch from 6c34558 to 434a7ef Compare May 9, 2026 14:35
@andig andig added the infrastructure Basic functionality label May 10, 2026
@andig
Copy link
Copy Markdown
Member Author

andig commented May 10, 2026

@copilot resolve the merge conflicts in this pull request

Comment thread core/metrics/collector.go Outdated
Copy link
Copy Markdown
Contributor

Copilot AI commented May 10, 2026

@copilot resolve the merge conflicts in this pull request

Merge conflicts are resolved. I merged the remote PR branch and master, kept the branch pushable, and pushed the result in 2f38765.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • 192.168.1.1
    • Triggering command: /tmp/go-build1799151314/b1500/service.test /tmp/go-build1799151314/b1500/service.test -test.testlogfile=/tmp/go-build1799151314/b1500/testlog.txt -test.paniconexit0 -test.timeout=10m0s -j ACCEPT mpile ux-amd64/pkg/tool/linux_amd64/vet -errorsas shservice mpile ux-amd64/pkg/too-buildtags -o off/v5@v5.0.2/ba-errorsas off/v5@v5.0.2/er-ifaceassert 0.1-go1.26.0.lin-nilfunc -p hub.com/golang/g-w mpile 0.1-go1.26.0.linsecurity (packet block)
  • 224.0.0.251
    • Triggering command: REDACTED, pid is -1 (packet block)
  • 239.12.255.254
    • Triggering command: REDACTED, pid is -1 (packet block)
  • api.awattar.at
    • Triggering command: /tmp/go-build1799151314/b1451/tariff.test /tmp/go-build1799151314/b1451/tariff.test -test.testlogfile=/tmp/go-build1799151314/b1451/testlog.txt -test.paniconexit0 -test.timeout=10m0s /tmp/go-build1799151314/b1389/_pkg_.a ecode_amd64.s ncode_amd64.s ux-amd64/pkg/tool/linux_amd64/vet -errorsas se/go-jose/v4/js-atomic mpile qy2w7f007j8N -o r/shoutrrr@v0.14-errorsas tainer/heap/heap.go ux-amd64/pkg/tool/linux_amd64/compile (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

Copilot finished work on behalf of andig May 10, 2026 10:12
@andig andig force-pushed the feature/pv-metrics-and-adjusted-forecast branch from 2f38765 to 972df73 Compare May 10, 2026 10:20
Comment thread core/site_tariffs.go Outdated
@naltatis
Copy link
Copy Markdown
Member

@premultiply is PV production import (implemented here) or export? Looking at the other PR discussions I think we might need to swap here. Can you formulate a rule/rules that we can use in documentation? or even better a simple markdown table for all device types we have (battery charge = import, ...).

@naltatis naltatis mentioned this pull request May 12, 2026
1 task
@andig
Copy link
Copy Markdown
Member Author

andig commented May 12, 2026

@naltatis see #29788 (comment): production is export.
@premultiply this gets confusing though since all PV inverter power values would be positive in that case which would get mapped to the metrics import column unless we deliberately switch the sign?

@andig

This comment was marked as resolved.

@premultiply
Copy link
Copy Markdown
Member

premultiply commented May 12, 2026

Definition: energy direction is from device to energy grid.

Meter type Import Export Positive Power
grid buy / A+ / 1.8.0 sell / A- / 2.8.0 Import
battery charging(+self-consumption) discharging Export
pv self-consumption production Export
ext consumption (production) Import
aux consumption (production) Import
charge charging(+self-consumption) (V2X) Import

Current "legacy" energy mapping is marked in bold. This is also our internal positive power direction (most important / "main usage").

@andig

This comment was marked as resolved.

@naltatis
Copy link
Copy Markdown
Member

@premultiply just to be clear: self-consumption == self-consumption of the device (like standby power), right?

@premultiply
Copy link
Copy Markdown
Member

Yes. This may or may not included in the measurement of the device. Depends on the device and internal setup itself.
But we do not care about self-consumption in detail. Just the be precise that it may be included here.

@VolkerK62
Copy link
Copy Markdown
Collaborator

So battery and pv will have the opposite sign as "normal" (e.g. in log)?

@andig
Copy link
Copy Markdown
Member Author

andig commented May 12, 2026

I've added designator for positive power in table above- still doesn't feel consistent.

@andig
Copy link
Copy Markdown
Member Author

andig commented May 13, 2026

Table represented as diagram. We're still not clear regarding the metrics table how we want to use import/export and how that aligns with the current device power signs:

---
Energy flow
---
flowchart
    loadpoint(Loadpoint)
    site((Site))
    grid(Grid)
    pv(PV)
    battery(Battery)

    grid -- Consumption
    (+ import/energy) --> site
    site -- Feed-in
    (- export/return energy) --> grid

    pv -- Production
    (+ export/energy) --> site

    site -- Charge
    (- import/return energy) --> battery
    battery -- Discharge
    (+ export/energy) --> site

    site  -- Consumption
    (+ import/energy) --> loadpoint
Loading

@florian240483
Copy link
Copy Markdown

Just a thought:
Energy and current flow basically follows Kirchhoff's current law. One possible convention for energy flow could be: Define Site as a node; all incoming connections are counted as positive, all outgoing connections as negative.

@andig
Copy link
Copy Markdown
Member Author

andig commented May 15, 2026

Long story short: we're dumping the import/export concept and settle on energy/return energy.

@andig andig merged commit 6b053bf into master May 15, 2026
8 checks passed
@andig andig deleted the feature/pv-metrics-and-adjusted-forecast branch May 15, 2026 09:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request infrastructure Basic functionality prio Priority

Projects

None yet

Development

Successfully merging this pull request may close these issues.

6 participants