Commodity graph validation code doesn't account for processes operating in years when they can't be commissioned

[alexdewar]

The graph validation code in (src/graph.rs) takes the ProcessMap as an input and iterates over it, looking at the years and regions in which that process can be commissioned. Processes may be operating in years when they cannot be commissioned, however. For example, imagine a process that can only be commissioned in the first few milestone years. It may be selected by an agent, commissioned, but will then continue to operate until it is decommissioned (either by agent choice or because its lifetime has elapsed). This process would not contribute to the commodity graphs for the later years of the simulation as the code stands, which could lead to a correct model failing validation or possibly other ill effects.

If we do #927 and #929, this problem will be much easier to solve, as the set of input and output flows will be the same regardless of the year in which the process was commissioned. The alternative will be pretty painful, I think.

There is a separate problem where assets commissioned in non-milestone years (e.g. before the simulation) escape various validation checks, including those in graph.rs. We should remember when fixing this issue that it is possible for the simulation to start with some assets already commissioned though, so these may contribute to the commodity graph for e.g. the first milestone year, even if their processes don't because they can't be commissioned in this year.

[dalonsoa]

Continuing here, @tsmbland .

Is it the problem that although the loop (outside of the function) goes through all milestone years, this bit here skips adding stuff to the graph because the flows are only defined for years within the process commission range, which might not cover all the milestone years?

MUSE2/src/graph.rs

Lines 88 to 93 in 29a9a30

	let key = (region_id.clone(), year);
	for process in processes.values() {
	let Some(flows) = process.flows.get(&key) else {
	// Process doesn't operate in this region/year
	continue;
	};

[dalonsoa]

If the above is the case, it seems like a very deep problem. Within the years where a process can be commissioned, that process is a candidate one to invest in and, if it exist, to produce/consume commodities, so it should be in the graph. That's what happens now. However, beyond those years, the process should be in the graph only if it was invested in the recent enough past and is within its lifetime - becoming an asset. Since we cannot know upfront if that is the case, I do not think that the graph can actually be built during the initialisation phase.

[tsmbland]

I actually don't think this is such a problem to begin with.

Taking the SED commodity electricity as an example, our graph checks make sure that, if there's a process that can be invested in in 2030 that consumes electricity, there must also be a process that can be invested in in 2030 that produces electricity. Now, perhaps we won't need to invest in any electricity producing processes, because existing assets are sufficient, but there's no way for us to know that upfront. In the current approach we're assuming that there might be a need to match new consumption with new supply, so we're making sure upfront we have processes available to do so. I agree it's not 100% ideal, and could be overly restrictive, but I don't really have a good (/easy) alternative suggestion

[dalonsoa]

if there's a process that can be invested in in 2030 that consumes electricity, there must also be a process that can be invested in in 2030 that produces electricity. Now, perhaps we won't need to invest in any electricity producing processes, because existing assets are sufficient, but there's no way for us to know that upfront.

I think this is the problem (the purpose of this issue). A simulation might be perfectly feasible because existing assets in a given year can cover the extra demand from (potentially) new processes, but we are not allowing that situation because we don't know that upfront and require that if new processes (potentially) add demand there must be new processes producing the stuff, as well.

If we are happy with this limitation, then this is not an issue and can be closed or marked On Hold for the future.

[tsmbland]

We could change that snippet you linked above to take process lifetime into account. We'd then be saying "if it's possible for an electricity consuming process to exist in 2030, then we'll let it pass if it's also possible for an electricity producing process to exist in 2030"

As Alex says, since #927 and #929, the set of input and output flows is the same regardless of the year, so it should be easy enough to add processes to graphs beyond their end year. (I don't think the same can be said about process availabilities, which are also factored into the graph validation, so not totally sure what we'd do about this.)

Maybe this is better actually. Thinking of an SVD commodity, I can think of a case of demand decreasing over time, so existing assets will be good enough, even if no new processes can be invested in, but we still need to include this commodity in the graph so it appears in the investment order to decide which of the existing assets to keep.