How to add psk_hint_key for ESP mosquitto broker? (IDFGH-14083)

[MacWyznawca]

How to add psk_hint_key for ESP mosquitto broker for encrypted communication with esp_mqtt client via ssl_psk?

[david-cermak]

This is currently not supported.

ESP mosquitto port uses ESP-TLS which does support PSK, but only on the client side.

Adding this option to server side is very simple, but might take some time and discussion (not the best option in terms of security, but very useful in local setup IMO).

To test this, you can apply the attached patch to IDF:
And update the default example to:

--- a/components/mosquitto/examples/broker/main/example_broker.c
+++ b/components/mosquitto/examples/broker/main/example_broker.c
@@ -69,14 +69,15 @@ static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_
     }
 }
 
+static psk_hint_key_t psk = { .hint = "client1", .key = (const uint8_t*) "key", .key_size = 3 };
+
 static void mqtt_app_start(struct mosq_broker_config *config)
 {
     esp_mqtt_client_config_t mqtt_cfg = {
         .broker.address.hostname = "127.0.0.1",
 #if CONFIG_EXAMPLE_BROKER_WITH_TLS
         .broker.address.transport = MQTT_TRANSPORT_OVER_SSL,
-        .broker.verification.certificate = cacert_start,
-        .broker.verification.certificate_len = cacert_end - cacert_start,
+        .broker.verification.psk_hint_key = &psk,
 #else
         .broker.address.transport = MQTT_TRANSPORT_OVER_TCP,
 #endif
@@ -103,10 +104,7 @@ void app_main(void)
 
 #if CONFIG_EXAMPLE_BROKER_WITH_TLS
     esp_tls_cfg_server_t tls_cfg = {
-        .servercert_buf = servercert_start,
-        .servercert_bytes = servercert_end - servercert_start,
-        .serverkey_buf = serverkey_start,
-        .serverkey_bytes = serverkey_end - serverkey_start,
+        .psk_hint_key = &psk
     };
     config.tls_cfg = &tls_cfg;
 #endif

feat-esp_tls-Add-support-for-PSK-on-server.patch.txt

[MacWyznawca]

Hi, thank you very much @david-cermak
Of course it goes and connections on the local network. Somehow I prefer when they are encrypted. And the authentication with password and key locally should be enough.

After applying patches to the IDF and to the mosquitto broker example, everything works.
The network connection between my project and the server in the example also works correctly. Locally my project does not connect. Nor does the client from the example connect to my project's server.
The example is run from the folder MyProj/managed_components/espressif__mosquitto/examples/broker
Can you suggest where to look for the problem?
On the left my project, on the right the example.

[david-cermak]

Nor does the client from the example connect to my project's server.

Please disable the CONFIG_EXAMPLE_BROKER_RUN_LOCAL_MQTT_CLIENT in the broker example.

The example is run from the folder MyProj/managed_components/espressif__mosquitto/examples/broker

You can use this command:

idf.py create-project-from-example "espressif/mosquitto^2.0.28:broker"

to create a separate project from that example, which you can easily modify.

[MacWyznawca]

My mistake @david-cermak!
I was in a hurry 😉
Unfortunately cloning the example didn't change anything. Still my project connects to the example using PSK, I can see the correctness of the work, because the client from the example connects locally to the broker (127.0.0.1).
However, when I change the settings of the client from example. to connect to my broker, errors appear. Same with the local (127.0.0.1) connection of my client to my broker.

[david-cermak]

This must work!

Let's start with something simple -- one broker on one ESP32, only one client on another ESP32:

Server side (from IDF softAP example):

• cd examples/wifi/getting_started/softAP/
• idf.py add-dependency "espressif/mosquitto^2.0.28"
• edit:

--- a/examples/wifi/getting_started/softAP/main/softap_example_main.c
+++ b/examples/wifi/getting_started/softAP/main/softap_example_main.c
@@ -89,6 +89,8 @@ void wifi_init_softap(void)
              EXAMPLE_ESP_WIFI_SSID, EXAMPLE_ESP_WIFI_PASS, EXAMPLE_ESP_WIFI_CHANNEL);
 }
 
+#include "mosq_broker.h"
+
 void app_main(void)
 {
     //Initialize NVS
@@ -101,4 +103,9 @@ void app_main(void)
 
     ESP_LOGI(TAG, "ESP_WIFI_MODE_AP");
     wifi_init_softap();
+
+    static psk_hint_key_t psk = { .hint = "hint", .key = (const uint8_t*) "key", .key_size = 3 };
+    esp_tls_cfg_server_t tls_cfg = { .psk_hint_key = &psk };
+    struct mosq_broker_config config = { .host = "0.0.0.0", .port = 8883, .tls_cfg = &tls_cfg };
+    mosq_broker_run(&config);
 }

Client side (from IDF mqtt example):

• cd examples/protocols/mqtt/tcp/
• idf.py menuconfig

CONFIG_BROKER_URL="mqtts://192.168.4.1"
CONFIG_ESP_TLS_PSK_VERIFICATION=y

• edit:

--- a/examples/protocols/mqtt/tcp/main/app_main.c
+++ b/examples/protocols/mqtt/tcp/main/app_main.c
@@ -98,10 +98,14 @@ static void mqtt_event_handler(void *handler_args, esp_event_base_t base, int32_
     }
 }
 
+#include "esp_tls.h"
+
 static void mqtt_app_start(void)
 {
+    static psk_hint_key_t psk = { .hint = "hint", .key = (const uint8_t*) "key", .key_size = 3 };
     esp_mqtt_client_config_t mqtt_cfg = {
         .broker.address.uri = CONFIG_BROKER_URL,
+        .broker.verification.psk_hint_key = &psk,
     };
 #if CONFIG_BROKER_URL_FROM_STDIN
     char line[128];

[MacWyznawca]

Thanks @david-cermak! I try this, but Your previous example and patches works in clean example, bud not in my project. On my project works only client, and connection with broker from example (Ethernet and Wi-Fi).
My project works excellent but without TLS. I try 1883 and 8883 ports. CONFIG_ I also check.
In my project now works HomeKit (ESP-HomeKit-SDK), Ethernet W5500, ESP-NOW with AES encryption, USB with AES-XTS file encryption, and lots of other task. Maybe I have short of TSL resources? I must investigate this (broker problem only - client works).

[david-cermak]

About resources, yes good to check for memory and number of sockets available on the system. One TLS connection takes about 40k (counts for each endpoint, server(s) and client(s)). But I'd expect a different errors if you're running out of memory.
TBH, from your screenshots it looks like you didn't configure your boards correctly, for example the error above means MBEDTLS_ERR_SSL_NO_CIPHER_CHOSEN, which looks like one endpoint's using PSK and another PKI?

[MacWyznawca]

Memory is not likely to be an issue, I have 145 KB of free DMA and over 6 MB of SPIRAM.
Could the problem be the https client I used to connect to the external notification server?
I'll keep looking, but thanks so much already for your help. If there are problems, I'll put the Broker on another device with ESP32 (which it was supposed to communicate with anyway) that doesn't have all this baggage. The project's own code already takes up over 40 k lines 😉.
With another Broker according to your instructions, my device connects, and that may already be enough.

Still, as if you know how to PSK run a Home Assistant 😉 alt that's a whole other topic.

[david-cermak]

Could the problem be the https client I used to connect to the external notification server?

You could be hitting the socket limit, as adding a broker and a local client would take up 3 sockets, plus if you're having some other external clients which connect and disconnect (esp. not gracefully), the number of sockets used could rise very quickly.
That could be adjusted in menuconfig.

Still, as if you know how to PSK run a Home Assistant

Are you using Home Assistant with ESPHome? AFAIK it uses Arduino by default, but you can optionally use ESP-IDF framework, where the PSK authentication should be available in menuconfig -- at least for the client side, ATM.

[MacWyznawca]

You could be hitting the socket limit, as adding a broker and a local client would take up 3 sockets, plus if you're having some other external clients which connect and disconnect (esp. not gracefully), the number of sockets used could rise very quickly. That could be adjusted in menuconfig.

This is possible. I have max (16) LWIP socket, and 12 for HAP HTTPD (HomeKit).

Still, as if you know how to PSK run a Home Assistant

OK, this I don't know ;-)

Are you using Home Assistant with ESPHome?

Never. I made this system from scratch https://luon.eu/en/

I have now started to add Home Assistant connectivity to our system because customers have been asking for it, and in the process I have ‘discovered’ the possibility of communication between our system's exchanges using MQTT and the tops that will be prepared for HA. However, I would also like to give the possibility to use such connections without having to install an additional broker. Therefore, I want to run the broker on one of our network devices. And on top of that, I don't like unencrypted connections even on the local network 😉

AFAIK it uses Arduino by default, but you can optionally use ESP-IDF framework, where the PSK authentication should be available in menuconfig -- at least for the client side, ATM.

I will fight with the socket settings. If it doesn't help, the broker will run on a screen urn that doesn't need to connect to HomeKit.

Thanks again for your help. If I manage it I'll let you know what worked in my case.