some sensors for Zappi showing unavailable after a while

[ragg987]

Version of the custom_component

0.0.27

HA OS for amd64 version 2024.5.4

Configuration

There is no custom YAML to configure the sensors - these are as surfaced by the HAOS integration.

Describe the bug

This is an intermittent error. Some of the standard sensors will show as unavailable, usually about 1-2 days after the integration has been started. Prior to this they work well. I have shown images of one example, the Zappi Charge Mode. If I restart the integration the problem will go away, then about 1-2 days later it will stop working. I have used the integration for a few months now and I believe the issue started relatively recently - maybe 2 months ago.

Debug log

The Charge Mode sensor failed at timestamp 2024-05-20 15:12:42 as seen in the development tab against sensor history. I enclose the logs for a few minutes prior to this timestamp. Some personal data is obfuscated.

2024-05-20 15:07:36.969 DEBUG (MainThread) [custom_components.myenergi] Refresh history local start of day in UTC 2024-05-19 23:00:00+00:00 UTC
2024-05-20 15:07:37.810 DEBUG (MainThread) [custom_components.myenergi] Finished fetching myenergi data in 0.841 seconds (success: True)
2024-05-20 15:07:37.823 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 258, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 414, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 168, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 492, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1009, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1132, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1067, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1015, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/select/__init__.py", line 148, in state
    current_option = self.current_option
                     ^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/myenergi/select.py", line 174, in current_option
    return LIBBI_MODE_NAMES[mode]
           ~~~~~~~~~~~~~~~~^^^^^^
KeyError: 'BALANCE'
2024-05-20 15:08:37.969 DEBUG (MainThread) [custom_components.myenergi] Refresh history local start of day in UTC 2024-05-19 23:00:00+00:00 UTC
2024-05-20 15:08:38.793 DEBUG (MainThread) [custom_components.myenergi] Finished fetching myenergi data in 0.824 seconds (success: True)
2024-05-20 15:08:38.800 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 258, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 414, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 168, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 492, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1009, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1132, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1067, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1015, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/select/__init__.py", line 148, in state
    current_option = self.current_option
                     ^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/myenergi/select.py", line 174, in current_option
    return LIBBI_MODE_NAMES[mode]
           ~~~~~~~~~~~~~~~~^^^^^^
KeyError: 'BALANCE'
2024-05-20 15:09:38.969 DEBUG (MainThread) [custom_components.myenergi] Refresh history local start of day in UTC 2024-05-19 23:00:00+00:00 UTC
2024-05-20 15:09:39.781 DEBUG (MainThread) [custom_components.myenergi] Finished fetching myenergi data in 0.811 seconds (success: True)
2024-05-20 15:09:39.796 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 258, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 414, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 168, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 492, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1009, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1132, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1067, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1015, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/select/__init__.py", line 148, in state
    current_option = self.current_option
                     ^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/myenergi/select.py", line 174, in current_option
    return LIBBI_MODE_NAMES[mode]
           ~~~~~~~~~~~~~~~~^^^^^^
KeyError: 'BALANCE'
2024-05-20 15:10:10.522 INFO (Recorder) [homeassistant.components.sensor.recorder] Detected new cycle for sensor.octopus_energy_electricity_xxx_xxxx_current_consumption, last_reset set to 2024-05-20T14:05:12.810930+00:00 (old last_reset 2024-05-20T14:04:11.905687+00:00)
2024-05-20 15:10:10.522 INFO (Recorder) [homeassistant.components.sensor.recorder] Detected new cycle for sensor.octopus_energy_electricity_xxx_xxxx_current_consumption, last_reset set to 2024-05-20T14:06:13.760605+00:00 (old last_reset 2024-05-20T14:05:12.810930+00:00)
2024-05-20 15:10:10.522 INFO (Recorder) [homeassistant.components.sensor.recorder] Detected new cycle for sensor.octopus_energy_electricity_xxx_xxxx_current_consumption, last_reset set to 2024-05-20T14:07:15.433618+00:00 (old last_reset 2024-05-20T14:06:13.760605+00:00)
2024-05-20 15:10:10.522 INFO (Recorder) [homeassistant.components.sensor.recorder] Detected new cycle for sensor.octopus_energy_electricity_xxx_xxxx_current_consumption, last_reset set to 2024-05-20T14:08:15.857182+00:00 (old last_reset 2024-05-20T14:07:15.433618+00:00)
2024-05-20 15:10:10.522 INFO (Recorder) [homeassistant.components.sensor.recorder] Detected new cycle for sensor.octopus_energy_electricity_xxx_xxxx_current_consumption, last_reset set to 2024-05-20T14:09:16.747284+00:00 (old last_reset 2024-05-20T14:08:15.857182+00:00)
2024-05-20 15:10:39.968 DEBUG (MainThread) [custom_components.myenergi] Refresh history local start of day in UTC 2024-05-19 23:00:00+00:00 UTC
2024-05-20 15:10:40.856 DEBUG (MainThread) [custom_components.myenergi] Finished fetching myenergi data in 0.888 seconds (success: True)
2024-05-20 15:10:40.864 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 258, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 414, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 168, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 492, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1009, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1132, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1067, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1015, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/select/__init__.py", line 148, in state
    current_option = self.current_option
                     ^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/myenergi/select.py", line 174, in current_option
    return LIBBI_MODE_NAMES[mode]
           ~~~~~~~~~~~~~~~~^^^^^^
KeyError: 'BALANCE'
2024-05-20 15:11:40.969 DEBUG (MainThread) [custom_components.myenergi] Refresh history local start of day in UTC 2024-05-19 23:00:00+00:00 UTC
2024-05-20 15:11:41.823 DEBUG (MainThread) [custom_components.myenergi] Finished fetching myenergi data in 0.854 seconds (success: True)
2024-05-20 15:11:41.838 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 258, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 414, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 168, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 492, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1009, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1132, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1067, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1015, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/select/__init__.py", line 148, in state
    current_option = self.current_option
                     ^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/myenergi/select.py", line 174, in current_option
    return LIBBI_MODE_NAMES[mode]
           ~~~~~~~~~~~~~~~~^^^^^^
KeyError: 'BALANCE'
2024-05-20 15:12:41.969 DEBUG (MainThread) [custom_components.myenergi] Refresh history local start of day in UTC 2024-05-19 23:00:00+00:00 UTC
2024-05-20 15:12:42.168 ERROR (MainThread) [custom_components.myenergi] Error fetching myenergi data: 
2024-05-20 15:12:42.169 DEBUG (MainThread) [custom_components.myenergi] Finished fetching myenergi data in 0.200 seconds (success: False)
2024-05-20 15:12:42.170 ERROR (MainThread) [homeassistant.components.template.template_entity] TemplateError('ValueError: Template error: float got invalid input 'unavailable' when rendering template '{{ states('sensor.myenergi_libbi_grid_import_today') | float
  - states('sensor.myenergi_weedon_libbi_grid_export_today') | float
  - states('sensor.myenergi_libbi_battery_charge_today') | float
  + states('sensor.myenergi_libbi_battery_discharge_today') | float
  + states('sensor.myenergi_weedon_libbi_generated_today') | float 
}}' but no default was specified') while processing template 'Template<template=({{ states('sensor.myenergi_libbi_grid_import_today') | float
  - states('sensor.myenergi_weedon_libbi_grid_export_today') | float
  - states('sensor.myenergi_libbi_battery_charge_today') | float
  + states('sensor.myenergi_libbi_battery_discharge_today') | float
  + states('sensor.myenergi_weedon_libbi_generated_today') | float 
}}) renders=6798>' for attribute '_attr_native_value' in entity 'sensor.home_consumption_energy_today'
2024-05-20 15:12:42.171 ERROR (MainThread) [homeassistant.components.template.template_entity] TemplateError('ValueError: Template error: int got invalid input 'unavailable' when rendering template '{{ states('sensor.myenergi_libbi_grid_ct2') | int
  - states('sensor.myenergi_libbi_power_ct_internal_load') | int
  - states('sensor.appliances_total_power') | int
  - states('sensor.myenergi_eddi_xxxx_power_ct_internal_load') | int
  - states('sensor.myenergi_libbi_power_ct_monitor') | int
  - states('sensor.myenergi_zappi_xxxx_power_ct_internal_load') | int
  + states('sensor.weedon_total_solar_power') | int
}}' but no default was specified') while processing template 'Template<template=({{ states('sensor.myenergi_libbi_grid_ct2') | int
  - states('sensor.myenergi_libbi_power_ct_internal_load') | int
  - states('sensor.appliances_total_power') | int
  - states('sensor.myenergi_eddi_xxxx_power_ct_internal_load') | int
  - states('sensor.myenergi_libbi_power_ct_monitor') | int
  - states('sensor.myenergi_zappi_xxxx_power_ct_internal_load') | int
  + states('sensor.weedon_total_solar_power') | int
}}) renders=17334>' for attribute '_attr_native_value' in entity 'sensor.home_essential_power'
2024-05-20 15:13:41.969 DEBUG (MainThread) [custom_components.myenergi] Refresh history local start of day in UTC 2024-05-19 23:00:00+00:00 UTC
2024-05-20 15:13:42.972 INFO (MainThread) [custom_components.myenergi] Fetching myenergi data recovered
2024-05-20 15:13:42.972 DEBUG (MainThread) [custom_components.myenergi] Finished fetching myenergi data in 1.003 seconds (success: True)
2024-05-20 15:13:42.989 ERROR (MainThread) [homeassistant] Error doing job: Task exception was never retrieved
Traceback (most recent call last):
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 258, in _handle_refresh_interval
    await self._async_refresh(log_failures=True, scheduled=True)
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 414, in _async_refresh
    self.async_update_listeners()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 168, in async_update_listeners
    update_callback()
  File "/usr/src/homeassistant/homeassistant/helpers/update_coordinator.py", line 492, in _handle_coordinator_update
    self.async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1009, in async_write_ha_state
    self._async_write_ha_state()
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1132, in _async_write_ha_state
    state, attr, capabilities, shadowed_attr = self.__async_calculate_state()
                                               ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1067, in __async_calculate_state
    state = self._stringify_state(available)
            ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1015, in _stringify_state
    if (state := self.state) is None:
                 ^^^^^^^^^^
  File "/usr/src/homeassistant/homeassistant/components/select/__init__.py", line 148, in state
    current_option = self.current_option
                     ^^^^^^^^^^^^^^^^^^^
  File "/config/custom_components/myenergi/select.py", line 174, in current_option
    return LIBBI_MODE_NAMES[mode]
           ~~~~~~~~~~~~~~~~^^^^^^
KeyError: 'BALANCE'

[G6EJD]

I wouldn’t call this an intermediate fault/error, it’s a missing key error. You need to reverse engineer the Libbi API to solve this issue.

[ragg987]

Are you referring to the API key? If missing would not ALL of the myenergi sensors fail to update? What I see is that most keep working but a few fail - for instance 4 (from memory) on the Zappi fail from a full list of about 20. I have not checked on the Libbi or Eddi side. Apologies if these are dumb questions but I don't know how the API works, I rely on the ME add-on to "shield" me from the inner workings...

[G6EJD]

The error report shows a JSON key is missing, so either ME changed the API or whoever created the Libbi integration needs to cater for the situation when the key isn’t present, as this is what ME do when there is no data to report, plus it exposed poor programming in not covering erroneous responses.

[plord12]

The error report shows a JSON key is missing, so either ME changed the API or whoever created the Libbi integration needs to cater for the situation when the key isn’t present, as this is what ME do when there is no data to report, plus it exposed poor programming in not covering erroneous responses.

For libbi we are waiting for 442 to merge in.

Maybe try the pre-release from here #442 (comment) ?

[ragg987]

Ff you are referring to this error in the above logs, this appears to be a Libbi error. However my issue is about a Zappi issue whereby the Zappi sensor becomes unavailable after a period of days. I cannot see Zappi being directly referenced in the log.

KeyError: 'BALANCE'

[G6EJD]

Whatever the cause the code needs to cater for the event of a missing JSON key BALANCE

[jwillemsen]

Can you retest with the latest version of this integration, BALANCE is now a key, see

ha-myenergi/custom_components/myenergi/select.py

Line 13 in de5a7be

LIBBI_MODE_NAMES = {"STOP": "Stopped", "BALANCE": "Normal", "DRAIN": "Export"}