HA template for heat pump load forecasting

[BuhJuhWuh]

Hi all,

It's been a bit of a mission (templating has been possibly the least intuitive programming challenge I've taken on in living memory...), but I'd like to share a wee contribution: here is a home assistant template to predict heat pump demand from forecast temperature. It grabs the hourly forecast temperatures for the next 24 hours from the Met.no integration, and converts that to a predicted energy demand for use in Predbat's Load Forecast input based on a curve fit to data which you'll need to gather for your own house & heating.

Here, I've plotted my Vaillant system's total (heating + DHW) daily energy consumption against the daily average outdoor temperature, then fitted a quadratic curve to it. (I think it's probably important to have a dataset with consistent underlying settings, hence my somewhat reduced dataset here (made some big heat curve/scheduling changes a month or so back) - will probably update the coefficients in due course once this has been filled out a bit more.)

You will then need to put the coefficients into a, b & c (from line equation y = ax^2 + bx + c) in the template code below, and add this to your configuration.yaml file.

template:
  - trigger:
      - platform: time_pattern
        hours: /1
        #minutes: /1
    action:
      - service: weather.get_forecasts
        data:
          type: hourly
        target:
          entity_id: weather.forecast_home
        response_variable: hourly_forecast_data
      - variables:
          forecasts: "{{ hourly_forecast_data['weather.forecast_home'].forecast }}"
    sensor:      
      - name: "HP Energy Prediction"
        unique_id: HP_energy_prediction
        state: "{{ now().isoformat() }}"
        attributes:
          HP_Prediction: |            
            {# Choose coefficients for quadratic curve fit - specific to your house/heating demand #}
            {% set a = 0.1424 %}
            {% set b = -3.9174 %}
            {% set c = 38.081 %}
            {% set ns = namespace(hourly_energy_list=[]) %}
            {% set ns.incr_energy_list = [] %}
            {% for f in forecasts %}  
              {# For each hourly entry, get temperature value from 'forecasts' variable and store it in 't' #}
              {% set t = f.temperature | float %}
              {# Calculate hourly energy HP prediction based on temperature and known fit curve #}
              {% set hourly_energy = ((a*(t**2) + b*t + c)/24)|round(2) %}
              {# Add to list of hourly energy values #}
              {% set ns.hourly_energy_list = ns.hourly_energy_list + [hourly_energy] %}
              {# Calculate sum of hourly entries so far #}
              {% set incr_energy = ns.hourly_energy_list|sum %}
              {# Create dictionary entry #}
              {% set incr_entry = {"last_updated" : f.datetime, "energy" : incr_energy} %}
              {# Add to list of dictionary entries #}
              {% set ns.incr_energy_list = ns.incr_energy_list + [incr_entry] %}
            {% endfor %}
            {{ ns.incr_energy_list }}

This is working for me (can see the effect in Predbat’s load forecast column), by adding the following lines at the relevant place in Predbat’s apps.yaml:

# Load forecast can be used to add to the historical load data (heat-pump)
# To link to Predheat
# Data must be in the format of 'last_updated' timestamp and 'energy' for incrementing kWh
  load_forecast:
    - sensor.hp_energy_prediction$HP_Prediction

(The final part I'd like to add would be to update the state from its placeholder timestamp to show the total forecast energy (i.e. the sum of the hourly_energy values, or the final value in the incr_energy_list). Suggestions on how to achieve this welcome!)

Hopefully this is helpful to someone!

All the best,
Jo

[gcoan]

Ooh, this is very interesting. Thank you for all your hard work on this. I agree Jinja templating code is counter-intuitive, what bit I’ve done is lots of trial and error.

Some questions:

1. How do you get from average daily temperature vs heat pump demand graph to the calibration values a, b & c?
2. Do you need to subtract historical heat pump demand from house load history for then load_forecast to add the predicted heat pump demand back on in the predbat forecast?
3. The calibration is done on daily OAT but the forecast is hourly temp, does this cause any issues the granularity difference ?
4. What about the hot water heating cycle which in my case is a spike in the morning at 10am, this presumably can’t be modelled ?

Super, looking forward to incorporate into my own predbat forecast

[BuhJuhWuh]

Hmm, that does look a bit odd. Could it be as simple as an offset between the two datasets or something, so each consumption figure is matched with the wrong temperature data point? I'd expect a much closer correlation (though in my case I'm using the HP's external temp sensor so of course there's a good correlation - it's the key control signal driving the HP!). Could also be worth checking temp correlation between your sensor and a nearby weather station history (e.g. https://www.wunderground.com/history/weekly/gb/inverness/EGPE/date/2024-3-3 is generally within a degree or three of my HP's sensor) just to make sure it's sensible...?

Following on from your earlier Q about how to handle the DHW runs, I'm now swithering over whether I should be using heating+DHW consumption or heating only. Heating only gives a better correlation, but misses out the DHW chunk of the demand. DHW consumption is a bit more random, but does have a temperature element when it comes to efficiency (and therefore consumption). Going to stick with the combined values for now, but will mull that one over in due course.

[springfall2008]

I did start on Predheat which I suspect does something similar and links to Predbat but I haven't had any real feedback on how well it works

[BuhJuhWuh]

Thanks, yes, I did look at that, but felt that the level/complexity of input information required, coupled with the assumptions you have to make (e.g. counting and modelling the performance of all the emitters) was likely to result in a lot of work for a model of unknown accuracy. In my case I had the data handy, and had already spotted the correlation, so this felt like a simpler and more robust method.

The other aspect that didn't quite make sense to me was the idea of simulation at such high frequency - the system response (and changes to outside temperature) are so slow that simulating every 5 mins seemed like overkill - hourly should be adequate, and ought to help smoothing over HP cycling in mild weather.

In other words: I couldn't be bothered - my way seemed simpler 😜 (at least until I tried to learn Jinja for the first time inside HA's templating structures...)

The question that would be useful to know the answer to, though, is how Predbat handles the Load Forecast input: is this simply added to the historical data? Or does it subtract an overall general adjustment first before adding this more precise specific input back in?

[gcoan]

@BuhJuhWuh yes I was suspicious of the average air temperature and whether it was being averaged correctly.
Tried shifting the temperature results one day up and down and it made no difference.

Unfortunately I have no integration whatsoever with my ASHP. Its a pair of LG Therma V heat pumps which have a control panel that seems to double for air on units as well as heat pumps, and the manual doesn’t explain what things are either. There is a wifi module for the heat pump but from what I have read the main way of integrating has been via physical modbus connection. Not straightforward so is on the future project list!

I had looked at the met office for weather history but only found monthly averages, nothing more granular.

On wunderground there’s lots of local weather sites, two in the next village, but when I look at history it only shows me Cambridge Airport which is 20 miles away, and there’s no data for weekends, they must turn the weather station off or something!

Plotting Cambridge Airport temperature vs ASHP kWH gives a much better correlation line

I’ll see if I can find a more local source for temperature history, but this is an improvement.

[BuhJuhWuh]

Yes, that's looking better. I'd also try a few different trendline types and see what fits best - quadratic worked best for me, and looks like it might get you a closer fit than linear too.

We have a very local weather station on Wunderground but it doesn't seem to keep any history. Inverness airport is closest with a historical record - a good few miles away and lower elevation, but generally reasonably close to my HP sensor's readings.

[BuhJuhWuh]

Just a quick update - between March and now, the warmer section of my data curve has filled out nicely, with only fairly minimal changes to the curve fit:

We'll see what the left-hand end does over this coming winter...

[gcoan]

I need to come back to looking at this.

Interesting your filling in of the right hand side of the chart with summer data. I’m not sure I would get meaningful results from mine because I only heat the hot water once a day, overnight at 3:30am on hopefully a cheap agile slot.
Typically the heat pump uses 1-2kWh to heat the hot water cylinder up, but some days it doesn’t even use anything if the tank is still hot enough, and by heating once a day a comparison to average daily OAT would I think be meaningless.

You don’t mention it in your instructions but is there an issue with double counting heat pump consumption in the predbat prediction?

Unless you create a template house load sensor and remove heat pump consumption from the historic house load data, the load history that predbat uses as its forecast will already include the heat pump projection, and then you add on top the predicted heat pump load?
Looking at the template code that goes in configuration.yaml I didn’t see anywhere that the heat pump projection does a delta (plus or minus) against the historical (days_previous) load.

[BuhJuhWuh]

Yes, fair points/questions.

1. Meaningful results - yes, in a household of 7, we're relatively heavy users, so it may be that our variations tend to even out more than in your situation. (You can see, looking at the data for say >16°C, there's a fair band of variation, from 2-8kWh or so, which is the difference between half a tank reheat and 2 tanks reheat in our house!) That said, using the octopus target sensors to trigger DHW runs does give a fairly consistent time of day for the cheapest slots during the afternoon and overnight. We're also in the Scottish Highlands, so probably don't have a huge number of days in the year where the heating is off entirely.
2. Yes - short version of that story: had the same niggle/question, couldn't figure out how to get an answer/assumed Predbat was designed in such a way as to deal with that correctly (otherwise why include that type of input?), seemed to work, forgot about it and moved onto other things. So yes, it may well be double-counting, but I don't really know - if you have a suggestion as to how to determine this, happy to have a poke around. But as I say, haven't noticed any adverse effects as yet...

Cheers,
Jo

[gcoan]

Thanks.

1. We're a family of 3 and my son has an electric shower, so really its only my wife and I using the hot water, so hence the strategy to only heat once a day (also not worth the overhead consumption of the heat pump starting every day).
   At the moment no integration between my heat pumps and HA so simplest to choose a cheapish import period and just leave it at that. Looking at my consumption history from May through September when the heating was off, it averaged 53kWh/month, 41 in July and there is a temperature related curve, but such small variations its not worth considering (for me).

2. One way to prove the double counting hypothesis would be to sum up the projected house load in the predbat plan and compare that to actual consumption. I would imagine you'd also see in the logfile that predbat is applying a large % uplift factor to today's consumption.
   I have to create my house load as a custom sensor as I have two inverters that share the load between them, so extending this to exclude heat pump usage would be easy:

# Home consumption sensor, updated every 5 minutes instead of the default of every minute
- trigger:
    - platform: time_pattern
      minutes: "/5"
  sensor:
    - name: "House Load Today"
      unique_id: "house_load_today"
      unit_of_measurement: kWh
      state_class: total
      device_class: energy
      state: >
        {% set x=(states('sensor.total_solar_energy_today')|float(0) 
          + states('sensor.total_battery_discharge_today')|float(0)
          - states('sensor.total_battery_charge_today')|float(0) 
          + states('sensor.grid_import_today')|float(0)
          - states('sensor.grid_export_today')|float(0) ) 
        %}
        {{ max(x,0)|round(1) }}