Unserialisable metadata causes plans to fail with "Type is not JSON serializable: <type>"

[tpoliaw]

The metadata in bluesky documents is sent via stomp to blueapi clients. If these messages contain types that are not json serialisable, sending the message fails and the plan is aborted.

Error on server

2025-01-31 13:42:55,273 - Run aborted
...
File "/dls/athena/blueapi/src/blueapi/worker/task_worker.py", line 450, in _on_document
    self._data_events.publish(
  File "/dls/athena/blueapi/src/blueapi/core/event.py", line 80, in publish
    callback(event, correlation_id)
  File "/dls/athena/blueapi/src/blueapi/service/interface.py", line 110, in forward_message
    stomp_client_ref.send(
  File "/home/qan22331/.cache/uv/archive-v0/Msmh6JWEfwmxlqMrRNTom/lib/python3.12/site-packages/bluesky_stomp/messaging.py", line 197, in send
    serialized_message = self._serializer(obj)
                         ^^^^^^^^^^^^^^^^^^^^^
  File "/home/qan22331/.cache/uv/archive-v0/Msmh6JWEfwmxlqMrRNTom/lib/python3.12/site-packages/bluesky_stomp/serdes.py", line 35, in serialize_message
    return orjson.dumps(json_serializable, option=orjson.OPT_SERIALIZE_NUMPY)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Type is not JSON serializable: set

Steps To Reproduce

Steps to reproduce the behavior:

1. Run blueapi server
2. Run a plan that includes a set in its parameter list, eg

   def demo_plan(det: Readable, mot: Movable) -> MsgGenerator:
       yield from scan([det], mot, 1, 2, num=5, md={"unsupported": {1, 2, 3}})

Acceptance Criteria

• Invalid metadata does not cause plans to fail.
• If metadata cannot be sent via stomp it should either be omitted or converted to a supported type.

[tpoliaw]

See also issue in bluesky-stomp: DiamondLightSource/bluesky-stomp#36

[callumforrester]

Hmm, we say the opposite in #797: plans should fail early and loudly. Needs more discussion, probably at sprint review on Wednesday.

[callumforrester]

From discussion, a message failing to send should make the plan fail, because otherwise we can get hidden data loss. So this is now a duplicate of DiamondLightSource/bluesky-stomp#36