Simple publish-subscribe protocol

SPSP connects low power IoT clients to MQTT¹ or other protocols.

It's an extensible framework for publish-subscribe pattern.

Main goal is to create highly reliable platform for message delivery from and to IoT devices while taking into account protocol-specific, performance and power restrictions.

Usage

Platform

Implemenation is based on ESP-IDF² framework, so all ESP32³ devices (including subtypes) are supported and they are a key target of this project.

You can also run SPSP bridge node on Linux platform and OpenWrt⁴, so you can have an access point serving both regular WiFi clients and IoT devices.

Dependencies

This project currently requires C++ 17 or newer.

If you want to use SPSP in your ESP-IDF-based firmware, ESP-IDF² v5.1 or newer is required. PlatformIO⁵ development is also supported.

If you want to use SPSP on Linux, you will need CMake and WiFi card with support for monitor mode and packet injection.

Setup for ESP-IDF

1. Create ESP-IDF² project.
2. Create idf_component.yml file inside main project directory:

   dependencies:
     spsp:
       git: https://github.com/DavidB137/spsp.git

3. Create sdkconfig.defaults file incide root project directory:

   CONFIG_COMPILER_OPTIMIZATION_SIZE=y
   CONFIG_COMPILER_CXX_EXCEPTIONS=y
   

   SPSP uses exceptions, so they need to be enabled in the build process. It may be also necessary to enable size optimalization depending on the flash size.
4. Rename main.c to main.cpp file inside main project directory and change it's content to:

   #include "spsp/spsp.hpp"
   
   extern "C" void app_main()
   {
       // ...
   }

Setup for Linux

1. Install build tools. On Debian based systems:

   sudo apt update
   sudo apt-get install make cmake build-essential git cmake ccache libffi-dev libssl-dev

2. Create build directory:

   mkdir linux/build

3. Build the binaries:

   cd linux/build
   cmake ..
   make

4. Optionally install it (systemwide):

   sudo make install

Monitor mode

To run SPSP bridge, you will need a WiFi card with support for monitor mode and packet injection. To put it into monitor mode and enable the interface, use:

# Replace `IFACE` with your WiFi card's interface name.
sudo ip link set IFACE down
sudo iw IFACE set monitor none
sudo ip link set IFACE up

Configuration

You will need to adjust the configuration. Example is provided in linux/bridge_espnow_config.ini.example file.

Run

Finally, to run SPSP ESP-NOW bridge:

sudo spsp_bridge_espnow PATH_TO_YOUR_CONFIG.ini

Setup for OpenWrt

See OpenWrt feed repository for SPSP: https://github.com/DavidB137/spsp-openwrt

Generally, Linux process applies. SPSP binaries, however, come packaged, so you will only need to setup network interface and adjust the configuration.

Examples

You can find usage examples in examples/ directory.

Code documentation

Code documentation is automatically generated by Doxygen⁶ from main branch and is available at: https://spsp.davidbenko.dev.

Network architecture

Nodes

Currently, there are 2 node types: client and bridge.

Client

Client is generic node type for low power sensors.

The client typically makes a measurement as quickly as possible (in order of 100s of milliseconds) and goes to deep sleep afterwards (for a longer time period). Always-on clients and any combinations are supported as well.

Upstream connectivity (to the bridge) is provided by local layer (for example ESP-NOW⁷), which allows for this extremly quick wake-up periods. The client somehow (protocol dependent) establishes connection to near-by bridge (think of it as router) processing all messages from client.

The client can publish data or subscribe to upstream data.

Reporting

For debugging purposes, client also publishes signal strength when it receives probe response to _report/rssi/{BRIDGE_ADDR} topic (+ prefix depending on far layer).

Bridge

Bridge connects clients on local layer and far layer. You can think of bridge as a router or gateway.

All received publish messages are forwarded to upstream far layer (typically MQTT¹). When subscribe request is received from a client, topic of the subscription gets stored in the database and registered on the far layer. Any number of clients can subscribe to the same topic and even the bridge itself can be one of them.

The bridge can also publish data, so it can function as both bridge and client at the same time.

Reporting

Bridge reports (topics don't include far layer prefix):

• Signal strength to client nodes: Reported to _report/rssi/{CLIENT_ADDR} topic.
• Probe request payload: Reported to _report/probe_payload/{CLIENT_ADDR} topic if it's not empty. By default it's empty on ESP-NOW clients, but you can set the value in SPSP::LocalLayers::ESPNOW::Config::probePayload. This feature is targeted for firmware version reporting.

Reporting can be configured or completely disabled in bridge configuration.

Local layer protocols

Local layer protocols are used for local communication between nodes (clients and bridges).

There's just one local layer protocol: ESP-NOW. More will come later.

ESP-NOW local layer

ESP-NOW local layer protocol is a wrapper around Espressif's ESP-NOW⁷. It's basically a WiFi without any state management (connecting to AP,...) whatsoever. Just a quick setup, sending of single packet, receiving of delivery confirmation – all over WiFi MAC.

It's built-in all ESP32 SOCs. You don't need any additional hardware – drastically saving cost, labour and enabling usage on third-party devices.

This implemenation includes built-in encryption using ChaCha20 with 256-bit password and 64-bit nonce. To allow multiple separate networks without much interference, 32-bit SSID must be set.

Each packet is checked for delivery status, but not resent when delivery fails! Internally, Espressif's implemenation does some sort of retransmission, but generally, it's not guaranteed that data will be delivered, so custom implementation of retransmission may be needed if required by your use-case (support is implemented).

Important

ESP-NOW is limited to 250 bytes of payload. SPSP's header is 20 bytes long, so len(topic) + len(payload) must be at most 230 bytes! You have to take that into account when sending data. In other words, make data you publish and/or subscribe to short. Packet fragmentation is not currently implemented in SPSP. Binary payload is supported.

Note

Espressif's ESP-NOW implementation limits number of paired devices to 20. SPSP, however, isn't limited by this count, because we dynamically add and remove paired device just before and after sending a single message (packet) to that device. In SPSP, this only limits number of devices concurently waiting for delivery confirmation. In theory, you could connect infinitely many clients to single bridge. Moreover, encryption is handled internally by SPSP, not by Espressif's ESP-NOW implementation.

Clients using ESP-NOW

Clients search for bridges to connect to using probes. During this process, the client iterates through all WiFi channels allowed by country restrictions (see WiFi::Station::setChannelRestrictions) and broadcasts probe requests to the broadcast address. Bridges on the same SSID with the correct password will decrypt the request and send back probe response. The client then chooses bridge with the strongest signal.

Far layer protocols

Far layer protocols are used by bridges as concentrators. They collect all data published by clients and provide data from other sources.

There are two far layer protocols: MQTT and local broker. More will come later.

MQTT far layer

Wrapper for most well known publish-subscribe IoT protocol – MQTT¹.

All standard authentication methods are supported:

• unauthenticated over TCP
• username and password
• TLS
• unauthenticated over WebSockets
• WebSockets secure

Internally ensures retransmission and reconnection to MQTT broker if needed.

Default topic structure for publishing is {PREFIX}/{ADDR}/{TOPIC} where:

• PREFIX is configured topic prefix (spsp by default)
• ADDR is node's address as reasonably-formatted string (in case of ESP-NOW⁷ it's lowercase MAC address without separators)
• TOPIC is topic received in message (e.g. temperature)

Topics for subscribing are not prepended or modified in any way.

Local broker far layer

Basically local MQTT-like broker.

Functions just like MQTT, supports wildcards, but is hosted on the bridge itself. Handy if you only need communication between nearby IoT devices over single bridge (no IP connectivity required).

Default topic structure for publishing is {PREFIX}/{ADDR}/{TOPIC} (see MQTT topic structure above).

Topics for subscribing are not prepended or modified in any way.

Message types

Message types are generic for current and any future protocols.

Bridge discovery message types

• Probe request (PROBE_REQ): Client attempts bridge discovery by broadcasting PROBE_REQ messages on all available channels.
• Probe response (PROBE_RES): Bridge response to received PROBE_REQ from a client. Client listens for PROBE_RES messages and chooses the one with the strongest signal. Address of that bridge is stored and all communication is routed to that address.

Publish and subscribe message types

• Publish (PUB): Client sends PUB message to discovered bridge to publish payload to topic. Bridge forwards it to far layer (i.e. MQTT).
• Subscribe request (SUB_REQ): Client sends SUB_REQ message to discovered bridge to subscribe to topic. This subscription is requested on far layer and valid for 15 minutes. The client can extend the lifetime by sending another SUB_REQ (it's done automatically).
• Subscribe data (SUB_DATA): When bridge receives data from far layer, it sends SUB_DATA messages to all clients that are subscribed to given topic.
• Unsubscribe request (UNSUB): Client can (and should) notify the discovered bridge when it doesn't need subscription anymore by sending explicit UNSUB message.

Time message types

Many applications require current time information. Bridge nodes automatically sync time using SNTP and then provide it to clients.

• Time request (TIME_REQ): Client sends TIME_REQ when Client::syncTime() method is called.
• Time response (TIME_RES): Bridge responds to TIME_REQ by sending TIME_RES with current timestamp as payload. The timestamp has milliseconds precision, but link latency between bridge and client is not compensated.

Other message types

Currently unused, but may be in the future.

• OK (OK): Generic OK message (typically for success responses).
• Failure (FAIL): Generic failure message (typically for failure responses).

Footnotes

1. MQTT https://mqtt.org ↩ ↩² ↩³

2. ESP-IDF: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/ ↩ ↩² ↩³

3. ESP32: https://en.wikipedia.org/wiki/ESP32 ↩

4. OpenWrt: https://openwrt.org ↩

5. PlatformIO: https://platformio.org ↩

6. Doxygen: https://www.doxygen.nl ↩

7. ESP-NOW: https://docs.espressif.com/projects/esp-idf/en/latest/esp32/api-reference/network/esp_now.html ↩ ↩² ↩³