Ability to specify markets with unmet demand vars

[tsmbland]

Description

Allows us to run dispatch runs with unmet demand variables turned on for a specific subset of markets being balances. This will be necessary for the implementing investment for commodity loops.

Note: this is different to the dispatch runs that we already do with unmet demand variables, which run with unmet demand variables turned on for all markets in the event of an infeasible model, to find out which markets are to blame. The changes here are so that we can intentionally run dispatch with unmet demand variables turned on for some markets.

Also note: the difference between adding an a market to markets_to_allow_unmet_demand and excluding it from markets_to_balance is that the former will incentivise the model to balance supply/demand as much as it can, with imbalance penalised by the "value of lost load" parameter, whereas the latter gives no incentive

Fixes # (issue)

Type of change

• Bug fix (non-breaking change to fix an issue)
• New feature (non-breaking change to add functionality)
• Refactoring (non-breaking, non-functional change to improve maintainability)
• Optimization (non-breaking change to speed up the code)
• Breaking change (whatever its nature)
• Documentation (improve or add documentation)

Key checklist

• All tests pass: $ cargo test
• The documentation builds and looks OK: $ cargo doc

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works

[codecov]

Codecov Report

❌ Patch coverage is 42.30769% with 30 lines in your changes missing coverage. Please review.
✅ Project coverage is 83.93%. Comparing base (e5d5ba3) to head (0d3672f).

Files with missing lines	Patch %	Lines
src/simulation/optimisation.rs	18.91%	28 Missing and 2 partials ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main     #985      +/-   ##
==========================================
- Coverage   84.16%   83.93%   -0.24%     
==========================================
  Files          52       52              
  Lines        5841     5862      +21     
  Branches     5841     5862      +21     
==========================================
+ Hits         4916     4920       +4     
- Misses        686      702      +16     
- Partials      239      240       +1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[dalonsoa]

This looks good to me! Basically, it makes the re-run with in the case of unmet demand explicit in the markets to include, being these all of them (what we had son far) or a specific list of them (new, but unused, at the moment), correct?

[dalonsoa]

While you have added this functionality here, it is not used anywhere, yet, right?

[tsmbland]

This looks good to me! Basically, it makes the re-run with in the case of unmet demand explicit in the markets to include, being these all of them (what we had son far) or a specific list of them (new, but unused, at the moment), correct?

Not exactly. This is so you can explicitly allow certain markets to have unmet demand upfront. If this fails, the re-run will allow all markets to have unmet demand, as we currently do.

I realise it's a bit confusing to include this functionality and not use it. This will be quite an important feature for systems with loops/cycles (#986, nearly done), but I wanted to keep it as a separate PR to stop #986 from getting too big

[tsmbland]

I've actually come up with a solution for the cycles problem that doesn't involve unmet demand variables, so I'm going to close this for now, but will leave the branch up in case this becomes useful in the future