Implement investment limits by tsmbland · Pull Request #1096 · EnergySystemsModellingLab/MUSE2

tsmbland · 2026-01-23T15:50:27Z

Description

Since #1020 we have a file allowing users to specify process investment constraints, but these were so far not being used.

In this PR, I've added a function calculate_investment_limits_for_candidates which use values from this input file to cap the amount of investment allowed for each process. These limits are calculated per-agent based on the agent's share of the commodity demand and the number of years elapsed since the previous milestone year (see discussion in #124).

The implementation was ultimately quite easy because we already have a remaining_candidate_capacity map to cap investments (currently based on demand limiting capacity), so we just needed to modify this map based on the provided investment limits (if any). An alternative would have been to modify the capacity field of candidate assets, but I decided against this as this would modify the tranching behaviour which I didn't want.

Related things left to do:

implement growth limits and total capacity limits (the overall behaviour should be similar to MUSE1)
express limits as a range rather than just an upper limit as we currently do - in this case we'd also need to implement a lower bound on investments and think about how to do this

Fixes #1069

Type of change

Bug fix (non-breaking change to fix an issue)
New feature (non-breaking change to add functionality)
Refactoring (non-breaking, non-functional change to improve maintainability)
Optimization (non-breaking change to speed up the code)
Breaking change (whatever its nature)
Documentation (improve or add documentation)

Key checklist

All tests pass: $ cargo test
The documentation builds and looks OK: $ cargo doc
Update release notes for the latest release if this PR adds a new feature or fixes a bug
present in the previous release

Further checks

Code is commented, particularly in hard-to-understand areas
Tests added that prove fix is effective or that feature works

codecov · 2026-01-23T15:53:52Z

Codecov Report

❌ Patch coverage is 87.12871% with 13 lines in your changes missing coverage. Please review.
✅ Project coverage is 87.55%. Comparing base (b9b1a22) to head (7ed1ab0).
⚠️ Report is 26 commits behind head on main.

Files with missing lines	Patch %	Lines
src/simulation/optimisation.rs	42.10%	8 Missing and 3 partials ⚠️
src/simulation/investment.rs	95.00%	1 Missing and 1 partial ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1096      +/-   ##
==========================================
- Coverage   87.57%   87.55%   -0.02%     
==========================================
  Files          55       55              
  Lines        7500     7580      +80     
  Branches     7500     7580      +80     
==========================================
+ Hits         6568     6637      +69     
- Misses        633      640       +7     
- Partials      299      303       +4

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

Copilot

Pull request overview

This PR implements investment limits for processes using the process_investment_constraints.csv input file that was introduced in PR #1020. Investment limits cap the amount each agent can invest in a process based on annual addition limits, scaled by the agent's share of commodity demand and the number of years elapsed since the previous milestone year.

Changes:

Added calculate_investment_limits_for_candidates function to compute per-agent investment limits
Changed addition_limit field type from f64 to Capacity for better type safety
Added get_annual_addition_limit method to support future extension with growth and total capacity limits

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated no comments.

Show a summary per file

File	Description
src/simulation/investment.rs	Implements the core logic for calculating and applying investment limits to candidate assets, integrates limits into the asset selection process, and adds comprehensive tests
src/process.rs	Changes `addition_limit` type to `Capacity` and adds `get_annual_addition_limit` method for future extensibility
src/input/process/investment_constraints.rs	Updates type handling throughout to use `Capacity` instead of `f64` and updates all tests accordingly
examples/two_regions/process_investment_constraints.csv	Adds example investment constraint data for the two_regions example
examples/muse1_default/process_investment_constraints.csv	Adds example investment constraint data for the muse1_default example

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

tsmbland · 2026-01-26T13:15:29Z

Also, forgot to mention in the description, but I've added investment limits for the muse1_default and two_regions models in line with MUSE1. These limits aren't actually restrictive enough to do anything, but you can play around with them to see it working in action.

And one other thing I've just realised is that these limits won't necessarily be complied with in the circularities algorithm, since we allow capacities to change after the (now constrained) investment stage within capacity_margin, which may exceed the process investment limit, so I need to think about what to do here...

tsmbland · 2026-01-26T13:43:57Z

Converting back to draft - some things I want to tidy up

tsmbland · 2026-01-26T18:40:58Z

@alexdewar This is getting a bit fiddly. Since investment limits depend on an agent property (namely the agent's commodity_portion), it gets a bit messy calculating the limits as you have to pass around the map of agents then go through matching each asset to the agents map. It's fine in select_assets_for_single_market since we're looking at agents one by one and already retrieve the commodity_portion, but for the cycle balancing in select_assets_for_cycle we're lumping assets from different agents together, and everything I've tried just got really icky.

It would be so much easier if assets stored the Agent (or Rc<Agent> like we do for processes), because then for assets with an assigned agent you could just do asset.max_installable_capacity() without having to pass anything else around. Seems like this would be too big a refactor to do in this PR, but do you think that's a good route to go down?

alexdewar · 2026-01-27T10:14:24Z

It would be so much easier if assets stored the Agent (or Rc<Agent> like we do for processes), because then for assets with an assigned agent you could just do asset.max_installable_capacity() without having to pass anything else around. Seems like this would be too big a refactor to do in this PR, but do you think that's a good route to go down?

Yes I do! There have been a couple of times where I started doing that for other reasons, but ended up not needing it, so never opened a PR.

I haven't looked at this PR yet, but we have a couple of options. We could merge as is (possibly adding a warning) or leave this branch here and I can fix it up once we've done the other refactor. Let's talk about it in the meeting.

tsmbland · 2026-01-27T10:49:03Z

@alexdewar Sounds good.

I've added the constraints for the circularities algorithm in b2eb10d as I wanted this to be complete, albeit a bit messy.

If we go down the route of what we've suggested above, then we could just retrieve the investment limit directly for each asset in add_capacity_variables, rather than having to pre-assemble a map and pass it around.

But, with that in mind, I think this is ready to review

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 6 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/asset.rs

src/simulation/optimisation.rs

src/input/process/investment_constraints.rs

src/process.rs

src/asset.rs

src/simulation/investment.rs

…odellingLab/MUSE_2.0 into investment_limits

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

src/simulation/investment.rs

Copilot · 2026-02-10T16:19:38Z

src/simulation/optimisation.rs

            AssetCapacity::Continuous(cap) => {
                // Continuous capacity: capacity variable represents total capacity
                let lower = ((1.0 - capacity_margin) * cap.value()).max(0.0);
-                let upper = (1.0 + capacity_margin) * cap.value();
+                let mut upper = (1.0 + capacity_margin) * cap.value();
+                if let Some(limit) = capacity_limit {
+                    upper = upper.min(limit.total_capacity().value());
+                }
                problem.add_column(coeff.value(), lower..=upper)
            }
            AssetCapacity::Discrete(units, unit_size) => {
                // Discrete capacity: capacity variable represents number of units
                let lower = ((1.0 - capacity_margin) * units as f64).max(0.0);
-                let upper = (1.0 + capacity_margin) * units as f64;
+                let mut upper = (1.0 + capacity_margin) * units as f64;
+                if let Some(limit) = capacity_limit {
+                    upper = upper.min(limit.n_units().unwrap() as f64);
+                }
                problem.add_integer_column((coeff * unit_size).value(), lower..=upper)


When capacity_limit is smaller than the current capacity (or smaller than the lower bound implied by capacity_margin), upper can become < lower, which can make the optimisation problem invalid/infeasible or trigger a solver error. After applying the limit, clamp the bounds so lower <= upper (e.g. reduce lower to upper, or choose a consistent policy such as fixing the variable at the limited upper bound).

Copilot · 2026-02-10T16:19:39Z

src/simulation/investment.rs

+        // Retrieve installable capacity limits for flexible capacity assets.
+        let key = (commodity_id.clone(), year);
+        let mut agent_share_cache = HashMap::new();
+        let capacity_limits = flexible_capacity_assets
+            .iter()
+            .filter_map(|asset| {
+                let agent_id = asset.agent_id().unwrap();
+                let agent_share = *agent_share_cache
+                    .entry(agent_id.clone())
+                    .or_insert_with(|| model.agents[agent_id].commodity_portions[&key]);
+                asset
+                    .max_installable_capacity(agent_share)
+                    .map(|max_capacity| (asset.clone(), max_capacity))
+            })


In cycle dispatch, capacity_limits uses key = (commodity_id.clone(), year) from the current market when looking up agent_share, but flexible_capacity_assets can include Selected assets from earlier markets with different primary commodities. This mis-scales max_installable_capacity for those assets (potentially clamping their capacity to near-zero or above the real limit) and can make cycle balancing infeasible. Compute the commodity portion per asset using the asset’s own primary output commodity (or equivalent process primary commodity) when indexing commodity_portions, and handle missing portions with a clear error instead of implicitly using the current market’s key.

alexdewar

We should also remove the note that the process_investment_constraints.csv file is unused from the docs.

I'm going to implement some of my own suggestions here then merge this, because it looks fine.

alexdewar · 2026-02-10T15:53:22Z

src/input/process/investment_constraints.rs

    regions: String,
    commission_years: String,
-    addition_limit: f64,
+    addition_limit: Capacity,


I think this should actually be CapacityPerYear

alexdewar · 2026-02-10T16:08:43Z

src/input/process/investment_constraints.rs

+            let scaled_limit = record.addition_limit * Dimensionless(years_since_prev as f64);
+
+            let constraint = Rc::new(ProcessInvestmentConstraint {
+                addition_limit: Some(scaled_limit),


@Aurashk Do you know if there was a particular motivation in making addition_limit optional? As far as I can see, all the limits that are None could just not be put into the map, with the same effect.

src/simulation/investment.rs

src/asset.rs

Copilot

Pull request overview

Copilot reviewed 7 out of 7 changed files in this pull request and generated 3 comments.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2026-02-10T16:35:19Z

src/simulation/investment.rs

+        // Retrieve installable capacity limits for flexible capacity assets.
+        let key = (commodity_id.clone(), year);
+        let mut agent_share_cache = HashMap::new();
+        let capacity_limits = flexible_capacity_assets
+            .iter()
+            .filter_map(|asset| {


flexible_capacity_assets contains Selected assets from all previously visited markets in the cycle, but key = (commodity_id.clone(), year) uses only the current loop’s commodity. That means capacity limits/agent shares may be computed against the wrong commodity for many assets (and can panic if the agent has no commodity_portions entry for this commodity/year). Consider deriving the commodity key per asset (e.g., from the asset’s primary output commodity) instead of using the current market’s commodity_id.

Copilot · 2026-02-10T16:35:19Z

src/simulation/investment.rs

+                let agent_share = *agent_share_cache
+                    .entry(agent_id.clone())
+                    .or_insert_with(|| model.agents[agent_id].commodity_portions[&key]);


agent_share_cache is keyed only by agent_id, but the commodity portion is commodity-specific. In a multi-commodity cycle, this will reuse the wrong share across different commodities (and the commodity_portions[&key] indexing can panic when the key is absent). Cache by (agent_id, commodity_id) and avoid direct indexing by using get() with an error/skip path.

Suggested change

let agent_share = *agent_share_cache

.entry(agent_id.clone())

.or_insert_with(|| model.agents[agent_id].commodity_portions[&key]);

let cache_key = (agent_id.clone(), commodity_id.clone());

// Look up in cache first; if missing, safely query commodity_portions.

let agent_share = if let Some(share) = agent_share_cache.get(&cache_key) {

*share

} else {

let agent = &model.agents[agent_id];

let share = match agent.commodity_portions.get(&key) {

Some(s) => *s,

None => {

// No defined share for this (agent, commodity, year); skip this asset.

return None;

}

};

agent_share_cache.insert(cache_key, share);

share

};

This isn't right. The commodity is constant over the region where the cache is used

Copilot · 2026-02-10T16:35:20Z

src/simulation/optimisation.rs

                let lower = ((1.0 - capacity_margin) * cap.value()).max(0.0);
-                let upper = (1.0 + capacity_margin) * cap.value();
+                let mut upper = (1.0 + capacity_margin) * cap.value();
+                if let Some(limit) = capacity_limit {
+                    upper = upper.min(limit.total_capacity().value());
+                }


Applying capacity_limit can make upper smaller than lower (e.g., if a limit is below the asset’s current capacity), which creates an invalid bounds range for the solver. Add a check to ensure lower <= upper after applying limits (e.g., clamp lower to upper or return a structured error when the limit is incompatible with the current capacity).

Hmm, that's a good point. I've got some additional concerns about this bit of code, but I'd like to merge this PR, so I've opened an issue (#1122) and we can revisit later.

alexdewar · 2026-02-11T13:30:35Z

I'm going to merge this now so it doesn't get out of sync with main

tsmbland added 3 commits January 23, 2026 14:00

Apply investment limits based on agent demand share

b2c3488

Scale limit based on number of years

6dbc3e6

Update example models

e2bfc4b

tsmbland added 4 commits January 23, 2026 16:47

Streamline arguments to calculate_investment_limits_for_candidates

6c28f0f

Small tidy ups

864b199

Add tests

4ac9e84

Pre-compute investment limits, better error message

6c4c6c5

tsmbland marked this pull request as ready for review January 26, 2026 11:39

Copilot AI review requested due to automatic review settings January 26, 2026 11:39

Copilot started reviewing on behalf of tsmbland January 26, 2026 11:39 View session

Copilot AI reviewed Jan 26, 2026

View reviewed changes

tsmbland requested review from Aurashk and alexdewar January 26, 2026 11:43

tsmbland marked this pull request as draft January 26, 2026 13:44

tsmbland added 3 commits January 26, 2026 15:21

max_installable_capacity Asset function

da9fafe

Use u32 for number of years

4fc39e6

Pre-scale limits

4afd749

Simplify calculate_investment_limits_for_candidates

2a36db4

Apply constraints in circularities model

b2eb10d

tsmbland marked this pull request as ready for review January 27, 2026 10:50

Copilot AI review requested due to automatic review settings January 27, 2026 10:50

Merge branch 'main' into investment_limits

9da63c4

Copilot started reviewing on behalf of tsmbland January 27, 2026 10:50 View session

Copilot AI reviewed Jan 27, 2026

View reviewed changes

alexdewar mentioned this pull request Jan 27, 2026

Refactor assets so agent is stored as Rc<Agent> #1108

Open

tsmbland and others added 3 commits January 27, 2026 11:32

Copilot fixes

8f05fb6

Merge branch 'investment_limits' of https://github.com/EnergySystemsM…

25304ed

…odellingLab/MUSE_2.0 into investment_limits

Remove unnecessary clone()

51cf7b3

Copilot AI review requested due to automatic review settings February 10, 2026 16:13

Copilot started reviewing on behalf of alexdewar February 10, 2026 16:14 View session

Copilot AI reviewed Feb 10, 2026

View reviewed changes

alexdewar reviewed Feb 10, 2026

View reviewed changes

Tweak doc comment

518ea91

Copilot AI review requested due to automatic review settings February 10, 2026 16:27

Copilot started reviewing on behalf of alexdewar February 10, 2026 16:27 View session

Copilot AI reviewed Feb 10, 2026

View reviewed changes

alexdewar added 8 commits February 11, 2026 09:43

Add tests for calculate_investment_limits_for_candidates

3da198d

AGENTS.md: Add instructions for tests

8694f06

Asset::max_installable_capacity: Check commodity_portion is in range

e3d890b

Remove now-stale note from schema

52c7729

schemas/input/process_investment_constraints.yaml: Fixes and formatting

4d1132c

Add CapacityPerYear unit type

7652932

Use CapacityPerYear type to represent unscaled addition limits

aa045cd

Merge remote-tracking branch 'origin/main' into investment_limits

7ed1ab0

alexdewar approved these changes Feb 11, 2026

View reviewed changes

alexdewar enabled auto-merge February 11, 2026 13:31

alexdewar merged commit bb99332 into main Feb 11, 2026
8 checks passed

alexdewar deleted the investment_limits branch February 11, 2026 13:31

-                let agent_share = *agent_share_cache
-                    .entry(agent_id.clone())
-                    .or_insert_with(|| model.agents[agent_id].commodity_portions[&key]);
+                let cache_key = (agent_id.clone(), commodity_id.clone());
+                // Look up in cache first; if missing, safely query commodity_portions.
+                let agent_share = if let Some(share) = agent_share_cache.get(&cache_key) {
+                    *share
+                } else {
+                    let agent = &model.agents[agent_id];
+                    let share = match agent.commodity_portions.get(&key) {
+                        Some(s) => *s,
+                        None => {
+                            // No defined share for this (agent, commodity, year); skip this asset.
+                            return None;
+                        }
+                    };
+                    agent_share_cache.insert(cache_key, share);
+                    share
+                };

Conversation

tsmbland commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Description

Type of change

Key checklist

Further checks

Uh oh!

codecov bot commented Jan 23, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Codecov Report

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

tsmbland commented Jan 26, 2026

Uh oh!

tsmbland commented Jan 26, 2026

Uh oh!

tsmbland commented Jan 26, 2026

Uh oh!

alexdewar commented Jan 27, 2026

Uh oh!

tsmbland commented Jan 27, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

alexdewar left a comment

Choose a reason for hiding this comment

Uh oh!

alexdewar Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

alexdewar Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

alexdewar Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Feb 10, 2026

Choose a reason for hiding this comment

Uh oh!

alexdewar Feb 11, 2026

Choose a reason for hiding this comment

Uh oh!

alexdewar commented Feb 11, 2026

tsmbland commented Jan 23, 2026 •

edited

Loading

codecov bot commented Jan 23, 2026 •

edited

Loading

tsmbland commented Jan 27, 2026 •

edited

Loading