Refactor levelised costs by Femkemilene · Pull Request #291 · cpmodel/FTT_StandAlone

Femkemilene · 2025-12-23T11:01:53Z

This makes the levelised cost calculation about 5x faster, hopefully also clearer. It uses two types of levelised cost functions:

A simple one for when the purchase happens at t=0 (freight, heat, transport)
A more complicated one when there are multiple build years (power, steel)

Not implemented for freight yet, as I'm waiting for @AmAkther to do the PR for the major update.

Carlos: your tasks are:

To test if the code runs on your side.
To say if you understand the new logic, or whether it needs a better explanation
To test if the implementation is sufficiently similar to before. There are a few changes, as some of the accounting for standard deviations was wrong. Instead of adding all variance parts first, we did it in batches, overestimating uncertainty. I've corrected the original functions so the two functions correspond better. There is a Taylor approximation, which causes small deviations
To critique the implementation: is it sufficiently DRY (non-repetitive), are the docstrings clear, is the style okay.

Plan is to create a unified lcoe bit of code

Using annuity logic

HMAN is not in the main version yet

Copilot

Pull request overview

This PR refactors levelised cost calculations across the FTT modules (Power, Transport, Heat) to achieve approximately 5x performance improvement. The refactoring introduces two generic functions in a new core module that handle levelised cost calculations with clearer mathematical formulations using capital recovery factors (CRF).

Key changes:

Introduces ftt_get_levelised_costs.py with two new functions: one for simple purchases at t=0 and another for multi-year build scenarios
Updates Power, Transport, and Heat modules to use the new generic functions
Corrects variance accounting issues in the original implementations (batched vs. summed variance calculations)

Reviewed changes

Copilot reviewed 6 out of 7 changed files in this pull request and generated 12 comments.

Show a summary per file

File	Description
`SourceCode/ftt_core/ftt_get_levelised_costs.py`	New core module with two generic levelised cost functions using CRF-based calculations
`SourceCode/Transport/ftt_tr_lcot.py`	Refactored to use `get_levelised_costs()` for transport cost calculations; includes testing code comparing with original
`SourceCode/Power/ftt_p_lcoe.py`	Refactored to use `get_levelised_costs_with_build()` for power cost calculations with lead times; includes testing code
`SourceCode/Power/ftt_p_main.py`	Updated function calls to pass `year` parameter to `get_lcoe()`
`SourceCode/Heat/ftt_h_lcoh.py`	Refactored to use `get_levelised_costs()` for heat cost calculations; includes testing code
`settings.ini`	Module configuration changed from FTT-P to FTT-Tr

Critical Issues Identified:

Hardcoded test values for policies in Power module (MTFT, MEWT set to 0.2)
Testing/profiling code and comparisons with original implementations left throughout
Configuration file change appears unintentional
Multiple TODO comments indicating temporary testing code

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Copilot · 2025-12-23T11:06:38Z

SourceCode/ftt_core/ftt_get_levelised_costs.py

+    # The mid-year correction accounts for the fact that variable cost happen mid-year
+    # mid_year_correction = (1 + r) ** -0.5   # TODO: switch to this one when we're happy with PR. This correctly accounts for timing
+    mid_year_correction = (1 + r) ** -1     # This reproduces what we had before
+    crf = r / (1 - (1 + r) ** -lifetimes) * mid_year_correction
+


The TODO comment indicates this is testing code that should be removed before merging. The commented line shows the intended mid-year correction formula (power of -0.5) while the active line (power of -1) reproduces the old behavior. This temporary code should not be committed to the main branch.

Suggested change

# The mid-year correction accounts for the fact that variable cost happen mid-year

# mid_year_correction = (1 + r) ** -0.5 # TODO: switch to this one when we're happy with PR. This correctly accounts for timing

mid_year_correction = (1 + r) ** -1 # This reproduces what we had before

crf = r / (1 - (1 + r) ** -lifetimes) * mid_year_correction

# The mid-year correction accounts for the fact that variable costs occur mid-year

mid_year_correction = (1 + r) ** -0.5

crf = r / (1 - (1 + r) ** -lifetimes) * mid_year_correction

Copilot · 2025-12-23T11:06:39Z

SourceCode/Power/ftt_p_lcoe.py

+    # Testing with some policies:
+    data['MTFT'][:, :, 0] = 0.2
+    data['MEWT'][:, :, 0] = 0.2
+    #data['MEFI'][:, :, 0] = 20
+
+
+


These hardcoded test values for policies should be removed before merging. Lines 82-84 set MTFT and MEWT to 0.2 (and MEFI is commented out), which appears to be for testing the implementation. Production code should use actual policy values from the data.

Suggested change

# Testing with some policies:

data['MTFT'][:, :, 0] = 0.2

data['MEWT'][:, :, 0] = 0.2

#data['MEFI'][:, :, 0] = 20

Copilot · 2025-12-23T11:06:39Z

SourceCode/Power/ftt_p_lcoe.py


 # Third party imports
 import numpy as np
+import time


This import statement is for testing purposes only and should be removed before merging to production. Time profiling code is useful during development but should not be part of the final production code.

settings.ini

SourceCode/ftt_core/ftt_get_levelised_costs.py

Copilot · 2025-12-23T11:06:40Z

SourceCode/Transport/ftt_tr_lcot.py

 import numpy as np
+from SourceCode.ftt_core.ftt_get_levelised_costs import get_levelised_costs

+import time


This import statement appears to be for testing purposes only and should be removed before merging to production. Time profiling code is useful during development but should not be part of the final production code.

Copilot · 2025-12-23T11:06:41Z

SourceCode/Transport/ftt_tr_lcot.py

+    start = time.perf_counter()
+
+    # Categories for the cost matrix (BTTC)
+    c3ti = {category: index for index, category in enumerate(titles['C3TI'])}
+    bttc = data['BTTC']
+
+    # Taxable categories for fuel tax, CNG, EVs and H2 exempt
+    taxable_fuels = np.ones([len(titles['RTI']), len(titles['VTTI']), 1])
+    taxable_fuels[:, 12:15] = 0   # CNG
+    taxable_fuels[:, 18:21] = 0   # EVs
+    taxable_fuels[:, 24:27] = 0   # Hydrogen
+
+    # Taxable categories for carbon tax: only EVs and H2 exempt
+    tf_carbon = np.ones([len(titles['VTTI']), 1])
+    tf_carbon[18:21] = 0   # EVs
+    tf_carbon[24:27] = 0   # Hydrogen
+
+    # Vehicle and usage parameters
+    lt = bttc[:, :, c3ti['8 lifetime']]
+    cf = bttc[:, :, c3ti['12 Cap_F (Mpkm/kseats-y)']]
+    ff = bttc[:, :, c3ti['11 occupancy rate p/sea']]
+    ns = bttc[:, :, c3ti['15 Seats/Veh']]
+    en = bttc[:, :, c3ti['9 energy use (MJ/km)']]
+
+    conv_full = 1 / ns / ff / cf / 1000
+    conv_pkm = 1 / ns / ff
+
+    # Upfront cost (base, policy, standard deviation)
+    upfront = (bttc[:, :, c3ti['1 Prices cars (USD/veh)']] * conv_full)
+    upfront_pol = (
+               (upfront * (1 + data["Base registration rate"][:, :, 0])      # Tax as share upfront
+               + data['TTVT'][:, :, 0]                                     # Purchase tax
+               + data['RTCO'][:, 0] * bttc[:, :, c3ti['14 CO2Emissions']] ) # CO2-dependent tax
+               * conv_full )
+    upfront_sd = bttc[:, :, c3ti['2 Std of price']] * conv_full
+
+    # Annual variable cost (base, policy, standard deviation)
+    annual = ( bttc[:, :, c3ti['3 fuel cost (USD/km)']]             # Fuel cost
+              + bttc[:, :, c3ti['5 O&M costs (USD/km)']]) * conv_pkm        
+    # RTFT must be converted from $/litre to $/MJ (assuming 35 MJ/l)
+    annual_pol = (data['RTFT'][:, :, 0] / 35 * en / ns / ff * taxable_fuels[:, :, 0]   # Fuel tax
+                + data['TTRT'][:, :, 0] * conv_full)                                   # Yearly road tax
+    annual_sd = np.sqrt((bttc[:, :, c3ti['6 std O&M']])**2
+                 + (bttc[:, :, c3ti['4 std fuel cost']])**2) * conv_pkm
+
+    # For simplicity, we have converted the capital cost and annual cost by the
+    # service provided already
+    lcot, lcot_pol, lcot_sd = get_levelised_costs(
+            upfront=upfront,
+            upfront_policies=upfront_pol,
+            upfront_sd=upfront_sd,
+            annual=annual,
+            annual_policies=annual_pol,
+            annual_sd = annual_sd,
+            service_delivered=1,
+            service_sd=0.0,
+            lifetimes=lt,
+            r = bttc[:, :, c3ti['7 Discount rate']])
+
+    # Generalised cost and lognormal transformation
+    gamma = bttc[:, :, c3ti['13 Gamma']]
+    lcot_pol_gam = lcot_pol * (1 + gamma)
+    log_lcot_pol = np.log(lcot_pol**2 / np.sqrt(lcot_sd**2 + lcot_pol**2)) + gamma
+    log_lcot_pol_sd = np.sqrt(np.log(1.0 + lcot_sd**2 / lcot_pol**2))
+
+    # Store outputs
+    data['TEWC'][:, :, 0] = lcot             # The real bare LCOT without taxes
+    data['TETC'][:, :, 0] = lcot_pol         # The real bare LCOT with taxes
+    data['TEGC'][:, :, 0] = lcot_pol_gam     # As seen by consumer (generalised cost)
+    data['TELC'][:, :, 0] = log_lcot_pol     # In lognormal space
+    data['TECD'][:, :, 0] = lcot_sd          # Variation on the LCOT distribution
+    data['TLCD'][:, :, 0] = log_lcot_pol_sd  # Log variation on the LCOT distribution
+
+
+    # TODO: delete testing
+
+
+    elapsed = time.perf_counter() - start
+    start2 = time.perf_counter()
+
+    lcot_old, tlcot, tlcotg, logtlcot, dlcot, dlogtlcot = get_lcot_original(data, titles, year)
+
+    elapsed2 = time.perf_counter() - start2
+
+    print(f"Runtime: {elapsed2 / elapsed:.2f} as fast")
+
+    print(f'Difference between new and old:')
+    print(f'{np.average(((lcot - lcot_old)/lcot_old)**2)}')
+    print(f'{np.average(((lcot_pol - tlcot)/tlcot)**2)}')
+    print(f'{np.nanmean(((lcot_pol_gam - tlcotg)/tlcotg)**2)}')
+    print(f'{np.average(((log_lcot_pol - logtlcot)/logtlcot)**2)}')
+    print(f'{np.average(((lcot_sd - dlcot)/dlcot)**2)}')
+    print(f'{np.average(((log_lcot_pol_sd - dlogtlcot)/dlogtlcot)**2)}')



This entire testing block (lines 43, 117-136) should be removed before merging. This includes the timing code, calls to the original implementation, performance comparisons, and difference calculations. Testing/validation code should not be in production.

Copilot · 2025-12-23T11:06:41Z

SourceCode/Power/ftt_p_lcoe.py

+    # Elapsed time for new implementation
+    elapsed = time.perf_counter() - start
+
+    # --- Testing block: compare with original implementation ---
+    start2 = time.perf_counter()
+    lcoe_bare_old, lcoe_co2_old, lcoe_av_old, lcoe_mu_gamma_old, lcoe_sd_old = get_lcoe_original(data, titles)
+    elapsed2 = time.perf_counter() - start2
+
+    print(f"Runtime: {elapsed2 / elapsed:.2f}x faster (new is {elapsed:.4f}s, old is {elapsed2:.4f}s)")

+    def msre(a, b, eps=1e-12):
+        denom = np.where(np.abs(b) < eps, eps, b)
+        return np.average(((a - b) / denom) ** 2)
+
+    print('Difference between new and old (mean squared relative error):')
+    print(f'MEWC (bare LCOE): {msre(data["MEWC"][:, :, 0], lcoe_bare_old):.6e}')
+    print(f'MECW (LCOE w/ CO2): {msre(data["MECW"][:, :, 0], lcoe_co2_old):.6e}')
+    print(f'MECC (avg LCOE): {msre(data["MECC"][:, :, 0], lcoe_av_old):.6e}')
+    print(f'METC (w/ gamma): {msre(data["METC"][:, :, 0], lcoe_mu_gamma_old):.6e}')
+    print(f'MTCD (std dev): {msre(data["MTCD"][:, :, 0], lcoe_sd_old):.6e}')
+


This entire testing block should be removed before merging. This includes the timing code (lines 67-68, 215-221), calls to the original implementation (line 220), performance comparisons (line 223), helper function (lines 225-227), and difference calculations (lines 229-234). Testing/validation code should not be in production.

Suggested change

# Elapsed time for new implementation

elapsed = time.perf_counter() - start

# --- Testing block: compare with original implementation ---

start2 = time.perf_counter()

lcoe_bare_old, lcoe_co2_old, lcoe_av_old, lcoe_mu_gamma_old, lcoe_sd_old = get_lcoe_original(data, titles)

elapsed2 = time.perf_counter() - start2

print(f"Runtime: {elapsed2 / elapsed:.2f}x faster (new is {elapsed:.4f}s, old is {elapsed2:.4f}s)")

def msre(a, b, eps=1e-12):

denom = np.where(np.abs(b) < eps, eps, b)

return np.average(((a - b) / denom) ** 2)

print('Difference between new and old (mean squared relative error):')

print(f'MEWC (bare LCOE): {msre(data["MEWC"][:, :, 0], lcoe_bare_old):.6e}')

print(f'MECW (LCOE w/ CO2): {msre(data["MECW"][:, :, 0], lcoe_co2_old):.6e}')

print(f'MECC (avg LCOE): {msre(data["MECC"][:, :, 0], lcoe_av_old):.6e}')

print(f'METC (w/ gamma): {msre(data["METC"][:, :, 0], lcoe_mu_gamma_old):.6e}')

print(f'MTCD (std dev): {msre(data["MTCD"][:, :, 0], lcoe_sd_old):.6e}')

Copilot · 2025-12-23T11:06:41Z

SourceCode/Heat/ftt_h_lcoh.py

 from SourceCode.support.divide import divide
+from SourceCode.ftt_core.ftt_get_levelised_costs import get_levelised_costs
+
+import time # TODO: delete after testing


This import statement is for testing purposes only (see the TODO comment on line 46) and should be removed before merging to production. Time profiling code is useful during development but should not be part of the final production code.

Copilot · 2025-12-23T11:06:41Z

SourceCode/Heat/ftt_h_lcoh.py

+    # New implementation
+    # Capacity factor
+    cf = data['BHTC'][:, :, c4ti['13 Capacity factor mean']]
+    dcf = data['BHTC'][:, :, c4ti['14 Capacity factor SD']]


Variable dcf is not used.

Suggested change

dcf = data['BHTC'][:, :, c4ti['14 Capacity factor SD']]

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

…/FTT_StandAlone into refactor_levelised_costs

cadupsg

Yes, the code runs on my side.
Yes, I understood the new logic. It is well explained in the script. It requires the reader to have a previous understanding of LCOE and finance math, but the implementation itself is understandable by the logic and comments.
Yes, results from new implementation seem in line with the results of the old implementation, especially for mean LCOE. There are slightly larger variations for SD, which makes sense with the new calculation. Still, not a big difference in value.
The implementation looks DRY and readable, it is not hard to go along the code and understand each part, mainly with the separation between the base case and build-time case. Docstrings are very helpful, clear and don't disturb the flow of the code. It might be helpful to also add the units in the docstrings or write the formula with the units. I did not test with the right mid_year_correction (-0.5). If you are going to delete the old line when the code is ready (TODO), it is ok, but if you are keeping the "-1" option as a commented line, maybe it is worth adding a comment better explaining this part and why (-0.5) is being adopted.

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Was double counting the conversion factor (spotted by copilot)

Femkemilene · 2026-01-14T13:36:08Z

@cadupsg: let's put a hold on this one, until the other big PR gets accepted. That way, I can immediately include freight as well.

Femkemilene added 2 commits December 18, 2025 17:14

Try-out speed-up

3c845c3

Plan is to create a unified lcoe bit of code

Add new levelised cost calculations

37de2b6

Using annuity logic

Femkemilene requested review from cadupsg and Copilot December 23, 2025 11:01

Copilot started reviewing on behalf of Femkemilene December 23, 2025 11:02 View session

Rm HMAN

7b2370f

HMAN is not in the main version yet

Copilot AI reviewed Dec 23, 2025

View reviewed changes

Femkemilene and others added 3 commits December 23, 2025 11:07

Update SourceCode/ftt_core/ftt_get_levelised_costs.py

30d3f64

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Fix forgotten dcf per Copilot

737ed7d

Merge branch 'refactor_levelised_costs' of https://github.com/cpmodel…

c958855

…/FTT_StandAlone into refactor_levelised_costs

cadupsg reviewed Dec 28, 2025

View reviewed changes

Femkemilene and others added 3 commits January 5, 2026 08:52

Update settings.ini

2547c5b

Co-authored-by: Copilot <175728472+Copilot@users.noreply.github.com>

Fix upfront cost bug

88469d8

Was double counting the conversion factor (spotted by copilot)

Add example units per PR

6325e2f

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Refactor levelised costs#291

Refactor levelised costs#291
Femkemilene wants to merge 9 commits intomainfrom
refactor_levelised_costs

Femkemilene commented Dec 23, 2025

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

Copilot AI Dec 23, 2025

Uh oh!

cadupsg left a comment

Uh oh!

Femkemilene commented Jan 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants

Conversation

Femkemilene commented Dec 23, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

Copilot AI Dec 23, 2025

Choose a reason for hiding this comment

Uh oh!

cadupsg left a comment

Choose a reason for hiding this comment

Uh oh!

Femkemilene commented Jan 14, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

3 participants