Production calculated by the solver is never used

[dalonsoa]

We were working under the assumption that, given that the scipy solver calculates the production per asset in addition to the investment, such production was being used to populate the market. The problem identified in #335 was thought to be just a problem when generating the outputs, but that the actual calculation was correct.

Well, it is not. The production calculated by the scipy solver is thrown away immediately after returning from the investment function, and can only be seen if it is requested as cached_output, but it does not end up in the market.

MUSE_OS/src/muse/investments.py

Lines 99 to 102 in 51f97ad

	if isinstance(result, xr.Dataset):
	investment = result["capacity"].rename("investment")
	if "production" in result:
	cache_quantity(production=result["production"])

The production that is used in the market is calculated within the market_variables function:

MUSE_OS/src/muse/sectors/sector.py

Line 236 in 51f97ad

supply, consume, costs = self.market_variables(market, technologies)

Which in turn calls the supply_prod function:

MUSE_OS/src/muse/sectors/sector.py

Lines 319 to 321 in 51f97ad

	supply = self.supply_prod(
	market=market, capacity=capacity, technologies=technologies
	)

Which in reality, it is the maximum_production in disguise, since I don't think it is ever overridden in any example or tutorial:

MUSE_OS/src/muse/sectors/sector.py

Lines 146 to 148 in 51f97ad

	self.supply_prod = (
	supply_prod if supply_prod is not None else maximum_production
	)

In summary, we go back to issue #335 but much more serious this time since the bottom line is that the production that goes to the market does not include other constrains, like minimum service or, more importantly, matching the demand: it is always the maximum_production. So, what @tsmbland identified in #321 in the plot on the right to be due to the minimum_service equal to 1, actually it is because the production is always maximum_production. At least, the supply calculation that was being discussed in #335 honoured the matching demand constrain, if not the minimum service one, but it turns out that the calculated supply that goes to the market does not honour either of them.

I might be missing something, so it will be good if someone else could cross-check this interpretation.

I'm not entirely sure of the best way forward here, but it seems to me this might require a major refactoring of how things are hooked together, from Agents all the way to the MCA.

What's your take on this, @tsmbland @ahawkes ?

[ahawkes]

Does this also apply to MCASupply output?
Anyway, things seem bad here! Do we have a way to calculate production using the adhoc solver at all? For scipy I guess we can adjust to code to actually use the values returned, but is there a simple solution for adhoc?

[ahawkes]

@tsmbland it would be good to get your opinion on this as I think you've been looking at this part of the code?

[tsmbland]

Only had a quick look so far. Diego is right that the production from the scipy solver isn't used. However, on this line

MUSE_OS/src/muse/sectors/sector.py

Lines 146 to 148 in 51f97ad

	self.supply_prod = (
	supply_prod if supply_prod is not None else maximum_production
	)

supply_prod actualy refers to this function, which calls this function, which is the function used by the adhoc solver, which does try to match production to demand, although it doesn't use the minimum service factor constraint.

But then I'm confused because the model does appear to be obeying the minimum service factor (see here)

So not sure what's going on. Need to look into it a bit more.

[ahawkes]

Great, thanks! As you get your head around this any ideas for code simplification much appreciated! It seems wildly complex at the moment. A

…

________________________________ From: Tom Bland ***@***.***> Sent: 24 July 2024 16:15 To: EnergySystemsModellingLab/MUSE_OS ***@***.***> Cc: Hawkes, Adam D ***@***.***>; Mention ***@***.***> Subject: Re: [EnergySystemsModellingLab/MUSE_OS] Production calculated by the solver is never used (Discussion #423) This email from ***@***.*** originates from outside Imperial. Do not click on links and attachments unless you recognise the sender. If you trust the sender, add them to your safe senders list<https://spam.ic.ac.uk/SpamConsole/Senders.aspx> to disable email stamping for this address. Only had a quick look so far. Diego is right that the production from the scipy solver isn't used. However, on this line https://github.com/EnergySystemsModellingLab/MUSE_OS/blob/51f97ad485d2948b50a15282f9c8073dfa6d4f5d/src/muse/sectors/sector.py#L146-L148 supply_prod actualy refers to this function<https://github.com/EnergySystemsModellingLab/MUSE_OS/blob/d644f9cfe059544918733874db2ae48995620cde/src/muse/production.py#L115>, which calls this function<https://github.com/EnergySystemsModellingLab/MUSE_OS/blob/d644f9cfe059544918733874db2ae48995620cde/src/muse/quantities.py#L15>, which is the function used by the adhoc solver, which does try to match production to demand, although it doesn't use the minimum service factor constraint. But then I'm confused because the model does appear to be obeying the minimum service factor (see here<https://muse-os.readthedocs.io/en/latest/user-guide/min-max-timeslice-constraints.html>) So not sure what's going on. Need to look into it a bit more. — Reply to this email directly, view it on GitHub<#423 (comment)>, or unsubscribe<https://github.com/notifications/unsubscribe-auth/AC37JLIUFCOX2GMG7GQJYZLZN7AIVAVCNFSM6AAAAABLFJX2A2VHI2DSMVQWIX3LMV43URDJONRXK43TNFXW4Q3PNVWWK3TUHMYTAMJTHEZTENY>. You are receiving this because you were mentioned.Message ID: ***@***.***>