Forecast energy above a threshold (e.g. for export limited systems)

[bajarrr]

I realize this idea might be a bit niche, and I'm not sure how complicated it'd be to implement, but thought it worth asking the lovely people here :).

The idea is to have some of the forecast energy sensors report the amount of energy generated when system is above a certain power threshold, i.e. similar to current energy sensors, but only for when forecast power is above a threshold (say 10kW).

For context, I get paid for energy I export to the grid, but my electricity provider limits the power I'm allowed to export, so I have a system that is capable of generating 15kW, but my export limit is 10kW. In determining when to use certain loads (in particular home battery or EV), targeting the time periods when the system is export limiting is particularly advantageous, as the energy is free (i.e., I'm not forgoing export).

Having the ability to see what energy is contained in this 'export-limited' section of the power curve (i.e. just the area of the forecast curve above 10kW) would allow me much more intelligent decisions for battery charging. For example, if the forecast90 shows there's only 2kWh expected then charge up to 8kWh from the general solar energy, and wait to soak up that bit of free energy. If however there's potentially a large amount of free energy, then don't charge in the morning, and charge mostly from the export limited energy.

Some parts of the year have none of this energy available for me, and I can detect that in the forecast by looking at the Peak Power Today/Tomorrow sensors (and do certain things is peak power <10kW), but knowing the expected magnitude of the maximum energy would help automating these decisions greatly.

Even some dynamic decisions would be available with a 'free energy remaining today' type sensor.

I understand the solcast integration calculates a 5 minute interpolated forecast power so I'm assuming this could be used to calculate the energy values that'd really help me, but I don't know enough python or enough about the code to understand if there are complications to what I'm asking for.

Thanks in advance for any insight/help/work anyone can offer.

[BJReplay]

I know that it isn't free for you, assuming that you are not an early adopter, and bearing in mind their warning about an uncertain future, and also you are asking about forecasting rather than just charging excess generation, but that said, this does seem like a very close fit to what you are after in terms of dumping excess generation into an EV, battery, of both, with a lot of smarts and customisation, but not much forcasting.

Now, back to your question: could you find all half hours where the 90% power was forecast to exceed a threshold, and sum the expected energy?

Yes, the data is all there, as is the same for the standard forecast.

You could potentially then use that to make decisions about bringing forward loads.

But, to the extent that ChargeHQ has a real-time solution, it is perfect for diverting zero-value solar to an EV or household battery.

[bajarrr]

Thanks for the suggestion, ChargeHQ is indeed a good solution - I was using it for a few months when I first got an EV and they do have some decent suggestions for dealing with export limited systems within their controls. I switched over to controlling charging from home assistant once ZJBeny (the wall box manufacturer) fixed a bug in their OCPP implementation that allowed the HA OCPP integration to work. I also since I got a second EV that charges from a smart-plug connected 10A charger, it got a bit complicated for ChargeHQ to handle. I also like the smaller time interval of control and local nature of OCPP for the car and modbus for the battery, but ChargeHQ was great to get things going quickly :)

I'm pretty happy with the real time controls I have going for utilising the zero-value solar (I like that term by the way) in home assistant, it's more the forecasting side I'm looking into now.

Using the half hour forecast90 data as you suggest could work well - I just saw there was a 5 minute cubic spline interpolated power curve inside ha-solcast and thought that was the ideal source. Though it is potentially/likely overkill in this case (and I'm now curious to try and calculate how much difference it would make).

[BJReplay]

Great to hear from a happy ChargeHQ user!

Ok, so I am away from home for a few more days (and thus from a computer) to suggest a template based solution using the attributes of the forecasts, but I suspect that it should be possible in HA and the existing sensors and attributes.

[bajarrr]

ChargeHQ is great, I recommend it to anyone I know with an EV :).

I'll see if I can sprinkle some jinja once the kids are asleep.

[bajarrr]

So I came up with this:

{% set detailedForecast = state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast') %}
{% set powerThresh = 10.0 %}
{% set ns = namespace(y = 0) %}
{% for i in range(detailedForecast | length) %}
  {% set powerVal = detailedForecast[i|int]['pv_estimate90'] %}
  {% if (powerVal > powerThresh) %}
    {% set ns.y = ns.y + (powerVal-powerThresh)/2 %}
  {% endif %}
{% endfor %}
{{ ns.y|round(2) }} 

There's potentially a tidier way to do it with filters, but a for loop can be clearer imo, though the jinja namespace stuff messes it up a bit.

Regardless, this runs through the home assistant template engine and spits out 14.96 kWh, and my sensor tracking how much I generated above the threshold today (which runs based off 10second interval data) shows 14.70kWh - and today the generation matched forecast pretty well. But it does seem to show the 30 minute data is accurate enough without interpolation (or potentially today was a fluke). But I'll set it up as a template sensor and give it go.

[BJReplay]

I think that is near enough to perfect:

For loops do make for readability despite namespace

Interpolation won't actually improve forecast, just smooth - the forecast is the forecast.

It spits out a number that you can use.

[autoSteve]

I looked at doing similar in-integration, but it's actually quite hard to look at other sensors in HA from an integration, so that idea was quietly shelved.

I really like this idea, @bajarrr.

I'm going to take a slightly different approach to calculating what's likely to be wasted if not used, because different circumstances.

Basically, the way things around my house goes during the day is that I am 0% reserve on two PowerWalls, so I use every last Watt if I need to. These can charge at a maximum rate of 10kW, and given a 13.2kW DC solar array there is hardly ever any spare power going begging until the beauties are full, which I want (well, I'd prefer 80% max to prolong battery lifespan, but that is what it is). Max solar and battery max charge are therefore quite balanced by design, so no curtailment should be expected until they're full.

There's 27.9kWh of battery, but it takes more energy to charge a battery than you can get out of it. My rough math says 31kWh to fill it, which factors charging losses, plus the overhead of the gubbins to charge it, like watercooling, etc. So I keep an estimated running tally of the expected state of charge in the jinja below.

Next I consider that this joint constantly chews power, even when I am not here. That's the 0.8kW typical load. (Central heating, fridges, network, server, etc.) This also slows the charge rate.

After the battery is full, my feed-in max is 5kW here, and the forecasted left-overs are the expected curtailed generation for the day, again reduced by whatever is expected base load. In non-Summer months, my plan is to check in an automation that the battery is full, and there's curtailment expected, so run the pool filter or other non-essential stuff for a while until "while" runs out, or until actual generation vs. consumption says I'm leaving feed-in tarrif on the table.

Well that's the proposal, given this has not seen real life yet.

        {% set ns = namespace(feedInLimit = 5.0, batteryImport = 31, typicalLoad = 0.8, _total=0, _battery=0) %}
        {% set detailedForecast = state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast') %}
        {% for i in range(detailedForecast | length) %}
          {% set powerVal = max(0, detailedForecast[i|int]['pv_estimate90'] - ns.typicalLoad/2) %}
          {% if ns._battery < ns.batteryImport %}
              {% set ns._battery = ns._battery + (powerVal/2) %}
          {% else %}
            {% if (powerVal > ns.feedInLimit) %}
              {% set ns._total = ns._total + (powerVal-ns.feedInLimit)/2 %}
            {% endif %}
          {% endif %}
        {% endfor %}
        {{ ns._total | round(2) }}

[autoSteve]

Much not real life so far.

I need a battery starting state of charge for the day or it all breaks down, like it will tomorrow. Right now the batteries should drop to ~45% before generation comes on, so ugh. This needs more work to either predict a _battery at sun-up and readjust at sun-up, or probably more reasonably just adjust the _battery at sun-up based on actual SoC... Which means I need to factor a loss for charging for delta, which will be rough math given lithium ion, but should be good enough.

I'll have another go at this tomorrow.

[autoSteve]

Here's what I've come up with to factor state of battery charge.

The weather has turned to crap here, so not much beyond-feed-in to be had. More testing awaits, and I'll edit this should adjustment be required.

To summarise what's going on here.

• I have two PowerWalls, so usable remaining is calculated as the combined total down to five percent (which is when they quit generation to save the cells).
• The state of charge gets tucked away just past midnight in another template sensor.
• The battery remaining (and juice rate to fill it) is adjusted for charging losses of 10%.
• Typical house load is flat 24x7, which is unrealistic for most, but not for me. It could be made more elegant by factoring average daytime use plus average dead-of-night consumption. I have network and servers to feed.
• It's based on the 50% forecast.
• Maximum grid feed-in is capped at 5kW here.

Will post results once the weather clears.

  - sensor:
      - name: "Powerwall Usable Battery Remaining"
        unique_id: "powerwall_usable_battery_remaining"
        state_class: "total"
        unit_of_measurement: "kWh"
        device_class: "energy"
        state: >-
          {%
            set value = (states('sensor.my_home_tg123126002he1_battery_remaining') | float(0) - (5 / 100 * states('sensor.my_home_tg123126002he1_battery_capacity') | float(0))) | round(1) +
            (states('sensor.my_home_tg12311300328c_battery_remaining') | float(0) - (5 / 100 * states('sensor.my_home_tg12311300328c_battery_capacity') | float(0))) | round(1)
          %}
          {{ value if value > 0 else 0 }}
        availability: >
          {{ states('sensor.my_home_tg123126002he1_battery_remaining') | is_number }}
  - trigger:
      - platform: time_pattern
        hours: "0"
        minutes: "1" # Just after midnight to allow the Solcast detailedForecast to update
    sensor:
      - name: "State of charge at midnight"
        unique_id: "state_of_charge_at_midnight"
        state_class: "total"
        unit_of_measurement: "kWh"
        device_class: "energy"
        state: >
          {{ states('sensor.powerwall_usable_battery_remaining') | float(0) }}
        availability: >
          {{ states('sensor.powerwall_usable_battery_remaining') | is_number }}
  - sensor:
      - name: "Forecast surplus today"
        unique_id: "forecast_surplus_today"
        state_class: "total"
        unit_of_measurement: "kWh"
        device_class: "energy"
        state: >-
          {% set ns = namespace(feedInLimit = 5.0, batteryCapacity = 27.9, chargeLoss = 0.1, typicalLoad = 0.8, _total=0, _battery=0) %}
          {% set ns._battery = states('sensor.state_of_charge_at_midnight') | float(0) / (1-ns.chargeLoss) %}
          {% set ns.batteryCapacity = ns.batteryCapacity / (1-ns.chargeLoss) %}
          {% set detailedForecast = state_attr('sensor.solcast_pv_forecast_forecast_today', 'detailedForecast') %}
          {% for i in range(detailedForecast | length) %}
            {% set powerVal = max(0, detailedForecast[i|int]['pv_estimate']) %}
            {% if ns._battery < ns.batteryCapacity %}
              {% set ns._battery = ns._battery + max(0, powerVal - ns.typicalLoad)/2 %}
            {% else %}
              {% if (powerVal > ns.feedInLimit) %}
                {% set ns._total = ns._total + max(0, powerVal - ns.feedInLimit - ns.typicalLoad)/2 %}
              {% endif %}
            {% endif %}
          {% endfor %}
          {{ ns._total | round(2) }}

[autoSteve]

It's after midnight, and after a full battery that dropped to 69.5% at midnight. Disappointing, but probably on the money because weather... It's on track for ~45% by sun-up. I am personally solar powered, so watch me freeze my moist bits off during the day. And if this is right I am projected to at least have full battery again tomorrow.

Hope so, because next day... I will so not have a full battery.