Update examples and tutorials to not use retrofit agents

[tsmbland]

Description

Sorry for the massive PR - this ended up being a bigger job than I expected. Most of the changes, however, are to results files.

The point of this PR is to change most of the example and tutorial models to use agents with a single 'new' share, rather than a pair of 'new' and 'retrofit' shares. This is a more likely scenario for basic users, and makes the tutorials simpler.

This PR also includes the changes made in #386 to increase the maximum_iterations parameter.

Main changes:

• Change the example models to use standard_demand instead of new_and_retro, remove the new_to_retro interaction, remove the Agent2 retrofit agent share, and allocate all existing capacity to the Agent1 (new) agentshare
• These changes have been propagated to the tutorial models, which are all based on the default and default_timeslice` models
• Add a default_retro model to the examples, which is equivalent to the old default model
• The only example model I haven't changed is the trade model, as I couldn't get it to work without the retrofit agents.
• Change the default value of demand_share to standard_demand
• Modify the add_agent parameter in the wizard module to work without retrofit agents
• Update tests to work with the new models
• Changes to the tutorial notebooks in line with the changes to the models. You can see the new version of the tutorials here.

It would probably be a good idea to have a tutorial using retrofit agents in the advanced guide, but I think that should be a separate issue.

Fixes #340
Fixes #350

Type of change

Please add a line in the relevant section of
CHANGELOG.md to
document the change (include PR #) - note reverse order of PR #s.

• New feature (non-breaking change which adds functionality)
• Optimization (non-breaking, back-end change that speeds up the code)
• Bug fix (non-breaking change which fixes an issue)
• Breaking change (whatever its nature)

Key checklist

• All tests pass: $ python -m pytest
• The documentation builds and looks OK: $ python -m sphinx -b html docs docs/build

Further checks

• Code is commented, particularly in hard-to-understand areas
• Tests added that prove fix is effective or that feature works

[tsmbland]

I've just updated the inputs for some of the models so far (haven't regenerated the results), but does this look reasonable? @dalonsoa

[dalonsoa]

I think these changes look sensible, so hopefully they should result in feasible models. Have you tried to run them? Do they result in any error?

[tsmbland]

Yeah they all run and the results look similar to before, so I think it's all good

[codecov]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 71.29%. Comparing base (9456a3f) to head (9ee059d).

Additional details and impacted files

@@             Coverage Diff             @@
##           develop     #349      +/-   ##
===========================================
- Coverage    71.37%   71.29%   -0.09%     
===========================================
  Files           44       44              
  Lines         5890     5901      +11     
  Branches      1162     1166       +4     
===========================================
+ Hits          4204     4207       +3     
- Misses        1365     1373       +8     
  Partials       321      321              

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[tsmbland]

I have no idea why the regressions tests are failing. @dalonsoa @alexdewar Can you try running tutorial 4.2 locally in this branch and let me know if you get any diffs?

[dalonsoa]

This is failing also locally for me. I think it makes sense. The time framework in the settings is [2020, 2022, 2024, 2026, 2028, 2030, 2032, 2034, 2036, 2038, 2040] but in the Results folder that is use as the "correct" values there are also 2025, 2035, 2045 and 2050, which I guess should not be there. As the tests compares the "correct" output with the new output, it fails. You will need to remove the files that should not be there anymore.

[tsmbland]

Makes sense! Thanks

[alexdewar]

I've only given this a quick once-over, but it seems sensible.

[dalonsoa]

Now that it passes, all looks good!

[tsmbland]

I've just realised that changing the default demand share to "standard_demand" could be a breaking change for models where demand share isn't explicitly stated in the settings file. Should I change it back to "new_and_retro"? @dalonsoa @ahawkes

[alexdewar]

I guess we have to change it back if it's a breaking change 😞.

We could issue a warning if people don't specify the demand share though, saying something like "it's defaulting to new_and_retro for backwards compatibility reasons but you probably don't want to use this"

[tsmbland]

Broader point - but how essential is it that we don't make breaking changes, since we know everyone that's using the software and can warn them of these sorts of things? (ignoring the technicalities of semantic versioning)

[alexdewar]

Good question... I'd personally still be inclined to print a warning because even if we know all the users, we don't have any guarantees that they're running the version of the code we asked them to and they might well update by accident! I guess it's up to @ahawkes though

[ahawkes]

I think we should go ahead with this one and just handle the breaks that happen. Not sure I fully understand the extent of potential problems, but I do really want to get away from new_and_retro

…

________________________________ From: Alex Dewar ***@***.***> Sent: 03 July 2024 14:47 To: EnergySystemsModellingLab/MUSE_OS ***@***.***> Cc: Hawkes, Adam D ***@***.***>; Mention ***@***.***> Subject: Re: [EnergySystemsModellingLab/MUSE_OS] Update examples and tutorials to not use retrofit agents (PR #349) This email from ***@***.*** originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list<https://spam.ic.ac.uk/SpamConsole/Senders.aspx> to disable email stamping for this address. Good question... I'd personally still be inclined to print a warning because even if we know all the users, we don't have any guarantees that they're running the version of the code we asked them to and they might well update by accident! I guess it's up to @ahawkes<https://github.com/ahawkes> though — Reply to this email directly, view it on GitHub<#349 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC37JLMGB4TEVXAHJQZKB3LZKP6IBAVCNFSM6AAAAABJRXZ5C2VHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDEMBWGEZDCMJSGY>. You are receiving this because you were mentioned.Message ID: ***@***.***>