v4.2.4

[autoSteve]

What's Changed

In an attempt to avoid 429/busy status the user agent included in calls to Solcast has changed. This may only temporarily improve things, and we will work with Solcast to hopefully solve this.

Many thanks to @gcoan for bringing the readme in line with Home Assistant changing the name of a service call to be an action.

• Add user-agent header to API calls by @autoSteve
• Refer to action instead of service call by @gcoan

If you didn't get the v4.2.0 update, or see the v4.2.0 announcement, check out the announcement, but keep the discussion here in this discussion thread.

Full Changelog: v4.2.3...v4.2.4

[Nilogax]

My scheduled 8am(ish) update failed on 4.2.3 after 429s - switched to 4.2.4 and did a manual forecast update which succeeded first time. Possibly just solcast load, possibly a result of the mighty awesomeness of 4.2.4. I'll monitor and update if I see more 429s later.

[isorin]

Upgraded to 4.2.4, scheduled update at 10:00am succeeded (yesterday, on 4.2.3 it failed)

[dtswk]

Installed no issues, manual update no issues. :-) Nice work

[BJReplay]

I switched my automation to 24 hours and bumped up my API limit so that it would run through peak EU and US timezons. I blew through my limit at 6:30am (GMT+11) this morning (as the new limit was higher than the combination of those used prior to increasing the limit, and the number used in a legacy forecasting service I run), but until then, all smooth sailing.

The integration handled the failed requests as expected, with no dreaded 429s.

[gcoan]

4.2.4 installed and working absolutely fine for me.

An improvement suggestion on the logged messages:

2024-10-24 01:05:00.201 INFO (MainThread) [custom_components.solcast_solar] Service call: force_update_forecasts
2024-10-24 01:05:00.202 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site bc31-4d4c-e844-6a4e
2024-10-24 01:05:01.935 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site 23bb-0216-85c6-fd94
2024-10-24 01:05:04.854 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update completed successfully
...
2024-10-24 07:40:47.126 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site bc31-4d4c-xxxx-xxxx
2024-10-24 07:40:48.839 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site 23bb-0216-xxxx-xxxx
2024-10-24 07:40:51.671 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update completed successfully
...
2024-10-24 10:12:33.142 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site bc31-4d4c-xxxx-xxxx
2024-10-24 10:12:34.865 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Getting forecast update for site 23bb-0216-xxxx-xxxx
2024-10-24 10:12:37.465 INFO (MainThread) [custom_components.solcast_solar.solcastapi] Forecast update completed successfully

The first update is a manually scheduled update, the second two are auto-scheduled updates.

The first one does say the service call that invoked it but it would be clearer if it said "manual forecast update" or something similar, the second and third don't have any announcement INFO line saying why the update is occurring, would be useful if it said "Auto-scheduled update" or something similar. And maybe in the 'Forecast update completed successfully' it could report the time that the next scheduled update is to be performed?

Just improvement suggestions

Cheers

[autoSteve]

This is done. The time of the next fetch is included with the result of the current fetch, whether successful or unsuccessful (or date and time if that fetch will be tomorrow).

_LOGGER.info("Forced update: Fetching forecast")
_LOGGER.info("Auto update: Fetching forecast")
next_update = f", next update at {next_update}"

[gcoan]

Thanks

[autoSteve]

We have over 2,500 downloads, so a lot of users now using the new User-Agent identification that is unique to the integration.

I would be interesting in hearing about any 429s happening for anyone.

[Nilogax]

A grand total of 11 429s since I switched to 4.2.4 (just after release) but no failed updates, and if I get a 429 it's usually just the one retry before success.

[gcoan]

I can't see any 429's in my logs since upgrading 👍

[AdamGilchrist317]

I had 2 429s just after updating it (just after you released it). None since. Massive improvement, many thanks!

[autoSteve]

Great to know.

(You weren't a wicket keeper in the Australian intl. cricket team were you, Gilly?)

[AdamGilchrist317]

No, but I used to get his emails, from fans all around the world! I'm older than him, so he's got my name, rather than the other way about!

[BJReplay]

Speaking of Australian Cricket, we had a hen named Garry: No 1 son was a wit - when she layed an egg he would say "Nice, Garry" as he collected the eggs.

[isorin]

A "index out of range" issue:

2024-10-27 23:15:00.065 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] Exception in __get_forecast_pv_moment(): list index out of range: Traceback (most recent call last):
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1935, in __get_forecast_pv_moment
    res = self.__get_moment(site, _data_field, (time_utc - day_start).total_seconds())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1814, in __get_moment
    return self._forecasts_moment['all' if site is None else site][self._use_data_field if _data_field is None else _data_field][int(n_min / 300)]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
IndexError: list index out of range

2024-10-27 23:15:00.067 ERROR (MainThread) [custom_components.solcast_solar.sensor] Unable to get sensor value: list index out of range: Traceback (most recent call last):
  File "/config/custom_components/solcast_solar/sensor.py", line 441, in _handle_coordinator_update
    self._sensor_data = self._coordinator.get_sensor_value(self.entity_description.key)
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/coordinator.py", line 347, in get_sensor_value
    return self.solcast.get_power_n_mins(60)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1508, in get_power_n_mins
    return round(1000 * self.__get_forecast_pv_moment(time_utc, site=site, _use_data_field=_use_data_field))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1935, in __get_forecast_pv_moment
    res = self.__get_moment(site, _data_field, (time_utc - day_start).total_seconds())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1814, in __get_moment
    return self._forecasts_moment['all' if site is None else site][self._use_data_field if _data_field is None else _data_field][int(n_min / 300)]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
IndexError: list index out of range

2024-10-27 23:15:00.067 ERROR (MainThread) [custom_components.solcast_solar.solcastapi] Exception in __get_forecast_pv_moment(): list index out of range: Traceback (most recent call last):
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1935, in __get_forecast_pv_moment
    res = self.__get_moment(site, _data_field, (time_utc - day_start).total_seconds())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1814, in __get_moment
    return self._forecasts_moment['all' if site is None else site][self._use_data_field if _data_field is None else _data_field][int(n_min / 300)]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
IndexError: list index out of range

2024-10-27 23:15:00.068 ERROR (MainThread) [custom_components.solcast_solar.sensor] Unable to get sensor value: list index out of range: Traceback (most recent call last):
  File "/config/custom_components/solcast_solar/sensor.py", line 405, in extra_state_attributes
    return self._coordinator.get_sensor_extra_attributes(self.entity_description.key)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/coordinator.py", line 417, in get_sensor_extra_attributes
    return self.solcast.get_sites_power_n_mins(60)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1522, in get_sites_power_n_mins
    res[site['resource_id']] = self.get_power_n_mins(n_mins, site=site['resource_id'])
                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1508, in get_power_n_mins
    return round(1000 * self.__get_forecast_pv_moment(time_utc, site=site, _use_data_field=_use_data_field))
                        ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1935, in __get_forecast_pv_moment
    res = self.__get_moment(site, _data_field, (time_utc - day_start).total_seconds())
          ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/solcast_solar/solcastapi.py", line 1814, in __get_moment
    return self._forecasts_moment['all' if site is None else site][self._use_data_field if _data_field is None else _data_field][int(n_min / 300)]
           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^
IndexError: list index out of range

[BJReplay]

Weird - after running for a week, and not at the DST transition. Only the one set of occurrences?

[BJReplay]

Created #209 to track.

[isorin]

Yes, that's the only occurrence.
It looks related to the DST changeover. (like missing 1h)
I only included the top part of the log as it repeats for each sensor using the same method.

[autoSteve]

The root cause is DST transition. I hit this on my test rig at some point a while back when it was set to Europe/London.

Using get_day_start_utc() and calculating what consitutes "end of day" was just wrong, and this was informing the build moment spline method incorrectly resulting in missing data points.

Start and end of day index calculation is fixed, along with a belt/braces of not throwing an exception, but rather returning zero with a warning logged. Fixing the start/end means the warning would never be hit.

Fixed in v4.2.5.

[autoSteve]

Closing. v4.2.5 is out. Continue at #210.