Rework activity limits code (v2) by tsmbland · Pull Request #1018 · EnergySystemsModellingLab/MUSE2

tsmbland · 2025-11-25T14:21:48Z

Description

This is based on #971, however there were enough problems with this that it was easier to write something from scratch (some snippets/ideas have been directly copied from that, however - I certainly don't want to pass this off as entirely my work!)

The main goal of this PR is to allow availability constraints to be added at different levels of time-granularity. We already allow users to provide availability constraints for e.g. "winter.night" or "winter" as a whole, but our approach for the latter is incorrect - we currently apply the constraint to all individual timeslices in winter, whereas actually we should be constraining availability across winter as a whole (i.e. availability can exceed the limit in any individual time slices, as long as the limit is respected across winter as a whole).

Most of the work is done by the new ActivityLimits struct. This stores the limits, does some validation, flags incompatible limits, removes redundant limits, and allows limits to be retrieved (either for a specific timeslice selection, or an iterator of limits that covers the whole year). Limits are first read in from the input data, then we create an ActivityLimits for every process/region/year combination.

I've then reworked the optimisation constraints to apply to the sum of all the activity variables in the selection. For timeslice-level constraints, this is still a single activity variable, but for seasonal/annual this is potentially many. Unsurprisingly, this changes all of the results, as all models have some processes with annual constraints - previously this was applied individually to every time slice in the year, whereas now it applies to the sum of all time slices.

A second goal is to allow users to provide both lower and upper limits on availability - we currently allow either but not both. Alex had the idea of using range syntax in the input files (similar to what we already do for years), so I had a go at implementing this as well. It was easier to do this in this PR because it makes the validation a bit easier. I've changed all the examples to follow this format, and updated the documentation. We also no longer mandate that all processes have explicitly defined availability limits, so I've deleted a few 0..1 annual limits which aren't needed.

I'm still a bit unsure about some of the validation. The rule I've gone for is that if any timeslices/seasons are provided for a particular process/region/year, then ALL timeslices/seasons must be provided for that particular process/region/year. This ended up being the easiest to implement, and I think makes some logical sense (catches the case where the user accidentally omits a time slice). I don't currently have any constraints on which years/regions are provided (i.e. if a user provides data for a process in one particular region/year, then they don't necessarily need to provide data for all regions/years that the process is active - if not provided it will assume there's no limit beyond the time slice length). I think ideally this would be more strict, so the user has to provide all milestone years and commission years for defined assets, but unfortunately I couldn't find a clean/easy way to do this within the framework I've written here.

Finally, I've also reworked some of the tests a bit by adding more fixtures which makes it a little bit easier to construct processes with unique properties.

Fixes #957
Fixes #958
Fixes #743
Fixes #363

Type of change

Bug fix (non-breaking change to fix an issue)
New feature (non-breaking change to add functionality)
Refactoring (non-breaking, non-functional change to improve maintainability)
Optimization (non-breaking change to speed up the code)
Breaking change (whatever its nature)
Documentation (improve or add documentation)

Key checklist

All tests pass: $ cargo test
The documentation builds and looks OK: $ cargo doc

Further checks

Code is commented, particularly in hard-to-understand areas
Tests added that prove fix is effective or that feature works

codecov · 2025-11-27T16:47:37Z

Codecov Report

❌ Patch coverage is 90.47619% with 34 lines in your changes missing coverage. Please review.
✅ Project coverage is 81.53%. Comparing base (19f72c2) to head (6634df6).
⚠️ Report is 19 commits behind head on main.

Files with missing lines	Patch %	Lines
src/simulation/optimisation/constraints.rs	57.57%	14 Missing ⚠️
src/input/process/availability.rs	86.95%	3 Missing and 6 partials ⚠️
src/process.rs	95.70%	4 Missing and 3 partials ⚠️
src/input.rs	66.66%	3 Missing ⚠️
src/input/process.rs	0.00%	0 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1018      +/-   ##
==========================================
+ Coverage   81.07%   81.53%   +0.46%     
==========================================
  Files          52       52              
  Lines        6331     6484     +153     
  Branches     6331     6484     +153     
==========================================
+ Hits         5133     5287     +154     
- Misses        942      944       +2     
+ Partials      256      253       -3

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

dalonsoa

I think this makes sense and is OK. There are some aspects that are not fully clear, but I think I grasp the gist of it.

dalonsoa · 2025-12-02T06:27:52Z

src/input/process/availability.rs

+        left.parse::<f64>()
+            .ok()
+            .with_context(|| format!("Invalid lower availability limit: {left}"))?
+            .into()
+    };


As far as I can tell, this returns a normal float, not a Dimensionless object, as we are expecting. Is rust happy with this?

It's because the .into() will convert it to the required type (if possible, which in this case it is), but I can rewrite this to make it more explicit

dalonsoa · 2025-12-02T06:58:26Z

src/process.rs

+        // Ensure that the new limit is compatible with the current limit
+        ensure!(
+            *limit.start() <= *current_limit.end() && *limit.end() >= *current_limit.start(),
+            "Availability limit for season {season} clashes with time slice limits",
+        );


It took me a while to get this. In other words, there must be overlap between the old and new ranges for the new one to be valid, right?

Yes exactly, otherwise it's impossible to satisfy the seasonal limit within the timeslice-level constraints

Unfortunately I don't think there's a way to make the code clearer (no inbuilt function to do this comparison), but I'll add a comment to clarify

dalonsoa · 2025-12-02T08:24:13Z

src/simulation/optimisation/constraints.rs

    let mut keys = Vec::new();
    let capacity_vars: IndexMap<&AssetRef, highs::Col> = variables.iter_capacity_vars().collect();
-    for (asset, time_slice, activity_var) in variables.iter_activity_vars() {
+


I'm not entirely sure if the loops in these two cases add the same number of elements to the problem or not. I think the new way is more readable, first going through all the assets and adding capacity or activity constrains, in each case iterating through the timeslices and limits within. But I don't fully understand why the same terms to the problem could not be added following the previous approach. Could you elaborate?

Previously, we were iterating over the activity variables (activity of asset X in timeslice Y) and adding one constraint per activity variable (or two for flexible capacity assets).

Now, we still have one/two constraints for each activity variable representing timeslice-level availability (we need these even if no availability constraints are defined, to make sure activity respects the timeslice length), but we may also have seasonal and annual constraints. These will involve multiple activity variables, so we can't simply iterate over the activity variables one by one like before

tsmbland

Thanks @dalonsoa. I'll fix/clarify a couple of things then merge it

tsmbland · 2025-12-02T09:59:11Z

src/simulation/optimisation/constraints.rs

    let mut keys = Vec::new();
    let capacity_vars: IndexMap<&AssetRef, highs::Col> = variables.iter_capacity_vars().collect();
-    for (asset, time_slice, activity_var) in variables.iter_activity_vars() {
+


Previously, we were iterating over the activity variables (activity of asset X in timeslice Y) and adding one constraint per activity variable (or two for flexible capacity assets).

Now, we still have one/two constraints for each activity variable representing timeslice-level availability (we need these even if no availability constraints are defined, to make sure activity respects the timeslice length), but we may also have seasonal and annual constraints. These will involve multiple activity variables, so we can't simply iterate over the activity variables one by one like before

tsmbland · 2025-12-02T10:11:04Z

src/process.rs

+        // Ensure that the new limit is compatible with the current limit
+        ensure!(
+            *limit.start() <= *current_limit.end() && *limit.end() >= *current_limit.start(),
+            "Availability limit for season {season} clashes with time slice limits",
+        );


Yes exactly, otherwise it's impossible to satisfy the seasonal limit within the timeslice-level constraints

Unfortunately I don't think there's a way to make the code clearer (no inbuilt function to do this comparison), but I'll add a comment to clarify

tsmbland · 2025-12-02T10:13:34Z

src/input/process/availability.rs

+        left.parse::<f64>()
+            .ok()
+            .with_context(|| format!("Invalid lower availability limit: {left}"))?
+            .into()
+    };


It's because the .into() will convert it to the required type (if possible, which in this case it is), but I can rewrite this to make it more explicit

tsmbland added 15 commits November 25, 2025 14:20

Create ProcessAvailabilities struct

76a714e

New limits for investment appraisal

d3e6478

New constraints for dispatch

12ed234

Add get_availability_limit function

99a1f5d

Update inputs module

1419f4a

Use range syntax in input files

58910ea

Fix/update tests

987145a

Tidy-ups to ActivityLimits

00f73bf

Better error handling

72bd0a5

Documentation and coverage checks

8ac33cc

Undo delete rows

59d43e3

Remove empty constraints

8355fb5

Update regenerate data script

b6cbd6e

Allow empty limits file

ae17944

Update results files

f7b2673

tsmbland added 2 commits November 27, 2025 17:24

Add tests

93f7b86

Small imprevements from self-review

a0d19cc

tsmbland changed the title ~~Activity limits v2~~ Rework activity limits code (v2) Nov 28, 2025

tsmbland marked this pull request as ready for review November 28, 2025 14:11

tsmbland requested review from Aurashk and dalonsoa November 28, 2025 14:11

tsmbland mentioned this pull request Dec 1, 2025

Add process_investment_constraints input file #1020

Merged

10 tasks

dalonsoa approved these changes Dec 2, 2025

View reviewed changes

tsmbland commented Dec 2, 2025

View reviewed changes

Address review comments

6634df6

tsmbland merged commit 2b7e641 into main Dec 2, 2025
8 checks passed

tsmbland deleted the fix-availability-constraints-v2 branch December 2, 2025 10:24

tsmbland mentioned this pull request Dec 2, 2025

Rework activity limits code #971

Closed

10 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Rework activity limits code (v2)#1018

Rework activity limits code (v2)#1018
tsmbland merged 18 commits intomainfrom
fix-availability-constraints-v2

tsmbland commented Nov 25, 2025 •

edited

Loading

Uh oh!

codecov bot commented Nov 27, 2025 •

edited

Loading

Uh oh!

dalonsoa left a comment

Uh oh!

dalonsoa Dec 2, 2025

Uh oh!

tsmbland Dec 2, 2025

Uh oh!

dalonsoa Dec 2, 2025

Uh oh!

tsmbland Dec 2, 2025

Uh oh!

dalonsoa Dec 2, 2025

Uh oh!

tsmbland Dec 2, 2025

Uh oh!

tsmbland left a comment

Uh oh!

tsmbland Dec 2, 2025

Uh oh!

tsmbland Dec 2, 2025

Uh oh!

tsmbland Dec 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

tsmbland commented Nov 25, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Key checklist

Further checks

Uh oh!

codecov bot commented Nov 27, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

dalonsoa left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

tsmbland left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

tsmbland commented Nov 25, 2025 •

edited

Loading

codecov bot commented Nov 27, 2025 •

edited

Loading