Issues with adapter 3.1

[benoti-git]

Hi gDoor Community,
I have recently installed a gDoor adapter v3.1 and connected it to HA. Everything works fine, but I have 2 issues and would like to get your advice.

1. The device disconnects from WiFi (FRITZ!Box) after 12-24h. No regular pattern visible. How can I debug this, is there a logfile on the adapter to find out what is the cause.
2. When picking up the audio via Türstation, I have a very loud buzzing sound in the background. When I disconnect the adapter from the bus, all is fine again. I also tried shielded cables for the bus connection, but no change.

thanks for your support, Ben

[DaSchaef]

1. May be related to 2, you can enable debug mode via the web page, this will results in debug messages printed via USB.
2. This may be mains voltage 50 Hz(maybe you can check via smartphone, typically it is 50 Hz or multiple of it), it would indicate that you somehow created a ground-loop.
   This would mean you would need to check:

• Do you have an isolated USB power supply, maybe try different USB supplies
• Is the USB bus not connected while normal operation (to not create a ground loop via the USB connector)
• Is the USB power supply on the same phase as the Gira Controller

I would first concentrate on point 2, because point 2 may also be causing distortions on the GDoor PCB as well.
Not all points above are necessary but they help to reduce potential issues.
You may want to test and power the GDoor temporarily via a USB power bank / Battery, the buzz should be gone if no other USB is connected.

(If nothing above helps there may be further solutions which would required hardware changes on the GDoor adapter to test/improve)

[benoti-git]

@DaSchaef: thanks for the hints. I have used a usb powerbank to connect the wemos and the buzzing is gone. Will try different power supplies next and hopefully find a suitable one. If not, I will see if I can use another phase than the one of the Gira controller.

[DaSchaef]

Ok, so it seems to be in the power supply or earth/neutral connection.
Maybe a common mode choke like balun may help you as a filter as well.
No guarantee that it will help, but there is a good possibility.

1. Use a core and wrap around the bus wires before connecting it to the GDoor adapter.
2. Another (or additional) possibility is to use it around the USB cable

You see, how the cables are wrapped around parallel, not twisted!

Another example with clip on core.

The core can be easily purchased, e.g. first amazon link:
https://www.amazon.de/Innfeeltech-Schwarz-Rauschunterdr%C3%BCcker-Kabelclip-Innendurchmesser/dp/B0D3ZYTMHL

[benoti-git]

@DaSchaef thanks again for the tips. I rewired everything and changed both usb cable and power supply. Using a good old apple power supply did the trick. Now I don’t have any noise on the audio channel anymore. No need to use the core, but this would have been the next step, even had one at home.
really looking forward to the audio development. This will make the use of the adapter even better. Thanks for your hard work on this development!!

[benoti-git]

I still have not found a solution for the regular outage of the gdoor adapter. After changing the power supply I had thought that it would have solved the regular connection losses, but they now reoccured... I still have not figured out how to get debug messages via MQTT. I activated the debug mode in the menu, but where can I retrieve the respective debug messages via MQTT or check the log file on the ESP itself. Could you please help me here. Thanks

[DaSchaef]

Debug messages are only send out over the USB port (baudrate: 115200).

[benoti-git]

OK, I connected it via USB. Where do I find the debug information in the data stream?
Here is what I get via serial: {"action": "BUS_IDLE", "parameters": "0000", "source": "000000", "destination": "000000", "type": "TYPE_GDOOR", "event_id": "1"}.
I will now wait for the adapter to disconnect again and hopefully find a useful debug info then...

[DaSchaef]

During a reconnect there should be more output, as now also die Wifi Library is in debug mode

gdoor/firmware/esp32/gdoor/src/wifi_helper.cpp

Line 345 in feb611d

wifiManager.setDebugOutput(debug());

While the connection is stable it should be rather quiet, only the IDLE / HomeAssistent Keepalive packets will show up.

[devcomputer]

same here. 3.1 usb powered and it discconnects after X hours with no pattern.
idle packages show up. I have to unplug and replug device to get gdoor back.

another problem is the IP. cant change it using DHCP or anyhting. once it picked an IP in my LAN it wont take any other IP no matter what IP the DHCP tries to assign the next time.
any solutions to that?

[DaSchaef]

For Wifi, we use WifiManager, I think we need debug logs of connection (DHCP issue) and disconnect to dig deeper.
(FYI: https://github.com/tzapu/WiFiManager)

[DaSchaef]

More "ideas" I had:

• I had problems in the past with "weak" china ESP boards (meaning unstable power supply leading to unstable WiFi).
• I had problems with multiple APs broadcasting the same SSID. ESP itself was sometimes (not always!) confused and sometimes connected to weak signal APs.
• When we loose WiFi connection, maybe the WifiManager falls back into the "configuration mode", opening a STA-AP (ESP access point), resulting in a downtime longer than needed (I could not observe this on my own, but maybe its happening).

EDIT: Debug log should give us hints for above points - I hope :)

[benoti-git]

OK, the issue with connection loss reoccurred today and I can now compare Mqtt and Serial output.
Here is what I see in the log:

Serial Output:
GDoor-Serial wechselte zu {"debug": "MQTT lost connection"}
10:55:11 - Vor 9 Stunden

MQTT:
GDoor Adapter Bus Data wurde nicht verfügbar
10:55:22 - Vor 9 Stunden

While there is no more output via MQTT after this connection loss, the serial adapter continues to do something. Here's an extract of the log - this sequence was triggered multiple times as the event_id counter goes up from 31 to 131 in only 4h!

Maybe this helps to find a root cause...

GDoor-Serial wechselte zu {"action": "BUS_IDLE", "parameters": "0000", "source": "000000", "destination": "000000", "type": "TYPE_GDOOR", "event_id": "31"}
15:37:01 - Vor 5 Stunden

GDoor-Serial wechselte zu {"debug": "MQTT_PRINTER read()"}
15:37:01 - Vor 5 Stunden

GDoor-Serial wechselte zu {"debug": "MQTT_PRINTER publish()"}
15:37:01 - Vor 5 Stunden

GDoor-Serial wechselte zu {"debug": "Output bus data via Serial and MQTT, done"}
15:37:01 - Vor 5 Stunden

GDoor-Serial wurde unbekannt
15:37:01 - Vor 5 Stunden

GDoor-Serial wechselte zu {"debug": "MQTT_PRINTER read()"}
15:37:01 - Vor 5 Stunden

GDoor-Serial wechselte zu {"debug": "MQTT_PRINTER publish()"}
15:37:01 - Vor 5 Stunden

GDoor-Serial wechselte zu {"debug": "Received data from bus"}
15:37:01 - Vor 5 Stunden

GDoor-Serial wechselte zu {"debug": "Gira RX was successfully parsed"}
15:37:01 - Vor 5 Stunden

GDoor-Serial wechselte zu {"debug": "Gira RX done"}
15:37:01 - Vor 5 Stunden

[DaSchaef]

event_id counter goes up from 31 to 131 in only 4h!
This may be normal because GDoor send keepalive (or tries to send them) to tell HA that it is alive.

Your log is missing any deeper debug messages, no output from WiFiManager.
Please attach full log over the time (best as txt). I have no high hopes, as the deeper log messages are somehow missing.
Do you record them via HA? Maybe HA filters them out because I would assume they are no valid JSON.

[benoti-git]

Yes, I am also a bit disappointed by the debug info. But this is all I get via HA "logbuch". I also don't know how to extract it as txt-file.
However, this is what I get when I disconnect the adapter and reconnect it:

GDoor-Serial wechselte zu {"action": "BUS_IDLE", "parameters": "0000", "source": "000000", "destination": "000000", "type": "TYPE_GDOOR", "event_id": "0"}
21:46:52 - Vor 1 Minute

GDoor-Serial wechselte zu {"debug": "MQTT_PRINTER read()"}
21:46:52 - Vor 1 Minute

GDoor-Serial wechselte zu {"debug": "MQTT_PRINTER publish()"}
21:46:52 - Vor 1 Minute

GDoor-Serial wechselte zu {"debug": "Successfully connected MQTT"}
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu {"debug": "Newly connected WIFI detected in MQTT loop"}
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu {"debug": "1.65"}
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu {"debug": "RX Sensitivity: "}
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu {"debug": "22"}
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu {"debug": "RX Pin: "}
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu {"debug": "GDoor Setup done"}
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu *wm:Starting Web Portal
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu *wm:STA IP Address: ........ (entfernt)
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu *wm:AutoConnect: SUCCESS
21:46:35 - Vor 2 Minuten

GDoor-Serial wechselte zu *wm:connectTimeout not set, ESP waitForConnectResult...
21:46:35 - Vor 2 Minuten

GDoor-Serial wurde unbekannt
21:46:25 - Vor 2 Minuten

[DaSchaef]

*wm: These are the messages we need. Otherwise you need to log the raw serial without HA, if HA filters out these messages :-/

I think HA is "hiding" debug messages from us ;-(

[mikburnz]

try it with USB direct connected to your HA. Configuration.ymal :

• platform: serial
  name: "GDoor"
  serial_port: /dev/ttyUSB1 # change device if needed
  baudrate: 115200

ttyUSB1 can be different in your case. Check: System/Hardware/Gesamte Hardware

I'm running two options in parallel. received LOG only from one installation

[benoti-git]

@mikburnz: I already have the adapter connected via serial. The log from GDoor-Serial is what I get via serial port.
While the MQTT-based entity is called GDoor Adapter Bus Data
However, the output is strange. I never get any other messages via serial other than "bus-idle" even though I get correct messages via mqtt when someone rings the bell.

@DaSchaef: I also think that HA is hiding some of the debug messages. How can I directly log the raw serial data from the adapter in the background via console?

[DaSchaef]

Sorry I'm no HA user, and as an embedded linux developer, my toolchain is rather "complicated" :-/

From my memory typical programs are:

On Windows: Hterm
On Linux:

• GUI: gtkterm
• Console: screen /dev/device baud-rate

[benoti-git]

I also checked the HA log and found another hint. The HA log shows an issue to set the state starting at 16:09 with 105 repeats, until at 18:10 the adapter finally loses connection and the mqtt-based entity shows the following:

GDoor Adapter Bus Data wurde nicht verfügbar
18:10:28 - Vor 2 Stunden

Logger: homeassistant.helpers.entity
Quelle: helpers/entity.py:1211
Erstmals aufgetreten: 26. November 2024 um 16:09:20 (105 Vorkommnisse)
Zuletzt protokolliert: 18:10:24

Failed to set state for sensor.gdoor_serial, fall back to unknown
Traceback (most recent call last):
File "/usr/src/homeassistant/homeassistant/helpers/entity.py", line 1211, in _async_write_ha_state
hass.states.async_set_internal(
File "/usr/src/homeassistant/homeassistant/core.py", line 2373, in async_set_internal
state = State(
^^^^^^
File "/usr/src/homeassistant/homeassistant/core.py", line 1817, in init
validate_state(state)
File "/usr/src/homeassistant/homeassistant/core.py", line 246, in validate_state
raise InvalidStateError(
homeassistant.exceptions.InvalidStateError: Invalid state with length 867. State max length is 255 characters.

[DaSchaef]

This seems to be that HA tries to parse some debug output from GDoor and fails - maybe?

[benoti-git]

I have now disabled the debug mode in the adapter and I now get a lot more messages via serial. Very strange, but I will now wait until the disconnect reoccurs, maybe I will see some useful info then…

[xlemassacre]

I think I have the same problem. Can I somehow support with the analysis

[DaSchaef]

I need the "wm:" debug messages from the serial debug console.
Hopefully we can track it down, at least I hope that we can identify what the ESP chip is "thinking" why it lost connection.

[xlemassacre]

I need to check if i'm able to connect it to usb for that.

Didn't see any pattern yet. Happened once after running for 2-3 weeks. Last time it happened already after a few days.

[benoti-git]

On my end I did not experience an outage of the adapter since turning off the debug mode. In the meantime I believe it’s the debug mode itself in combination with HA that leads to a strange behavior and produces all these strange messages via serial and finally freezes the adapter. With debug disabled, I only see correct messages via serial and mqtt and no freezing anymore. Problem solved for now.

[xlemassacre]

I think I found the reason/pattern now.
I reboot my wifi access points once per month (and sometimes after a firmware upgrade). It seems that after the reboot gdoor is not able to reconnect to the wifi (all other clients are able to renonnect after the reboot).

Is it possible that there is an issue in the software with wifi reconnect handling?

[jschroeter]

I can confirm issues with Wifi reconnect: when I turn on the wifi time schedule of my router, e.g. that it turns Wifi off during night, it happens that GDoor doesn't reconnect when Wifi is on again.
I assume that it's caused by the WiFiManager library we use and it seems that there are ways to fix it: tzapu/WiFiManager#1588
But before investing time we should check proogress of #25, in case we tend to switch to ESPHome we should check how Wifi reconnect is handled there.

[DaSchaef]

I would need serial logs of this, so we can narrow it down. I think it is important that the GDoor is reliable and stable, therefore we should have a stable baseline. But I can not reproduce it here with my FritzBox Wifi on/off (but I know such things from other ESP projects, so yes I understand how frustrating such things can be).

[xlemassacre]

I can try to get some serial logs, but I have a question before doing that.
Currently I use the MeanWell HDR-15-5 as power supply, but documentation says:

The safest method to avoid all of above points, without detailed knowledge, is to:

Use a isolating 5V supply e.g. MeanWell HDR-15-5, or a USB power supply without earth connection (most of them). Do not use a computer or similar to power the GDoor adapter.
Do not connect to the USB with your computer while the GDoor adapter is connected to the Gira bus. Only use Wifi.

How should I get the logs now. Disconnect the adapter from the MeanWell power supply and connect it via USB to the computer, while the adapter is still connected to the Gira Bus?

Regarding reproducing the issue, how long did you turn off the Wifi. My Access Points need almost 5 minutes to reboot, maybe turning it off for the same time makes it reproduceable.

[DaSchaef]

Sorry I somehow missed your post!

I would propose to power it via USB so, only one connection.

Disconnect the adapter from the MeanWell power supply and connect it via USB to the computer, while the adapter is still connected to the Gira Bus?

Yes :)

[xlemassacre]

No worries. Getting the logs that way from gdoor is a bit complicated with my setup/available devices.

As there is a lot of progress currently going on with the esphome custom component I'm looking into that direction and will switch to that as soon as possible.