Make it possible to exchange msgs to instances within a FlowFuse installation between Teams.

[SynoUser-NL]

Description

Background: https://discourse.nodered.org/t/way-to-send-full-msg-to-other-instance-in-other-team/89344

For us, it would be a godsend to have the possibility to exchange (send-return) messages over to all instances, including those in different Teams, within the same FlowFuse environment.
All other ways to do this (TCP, MQTT) only send payload data. This means that all information must first be moved under payload before sending, at the receiving end the original message must be restored (to be able to keep following the info), and again the whole thing in reverse when returning the message.
When using TCP you have to encode\decode the payload to\from JSON in addition to moving everything under payload.
Not only does this slightly delay all messages, it also makes flows much less straightforward then they (imho) could be.

A change to the existing Project nodes appears to me to be the way to go here, as they already provide inter-instance communication for instances within FlowFuse Teams. But there may be other options.
Or if I'm doing something wrong or have overlooked something with TCP\MQTT, please enlighten me.

Which customers would this be available to

Enterprise Tier Only (EE)

Have you provided an initial effort estimate for this issue?

I am not a FlowFuse team member

[joepavitt]

As we have things architected now, this would be a difficult thing to implement, as on FF we have thousands of teams, and this would open up a lot of security concerns.

Something that had been discussed in the past though is a single layer above teams called Organisations which we could then scope here as "allow comms between teams in organisation"?

[SynoUser-NL]

Hey @joepavitt ,
That sounds like a perfect solution, but I gather would require a lot of development work.
We're begun switching to MQTT now, which does have some advantages and makes things a bit easier. If this is requested by others as well, we'd really welcome it being on the Flowfuse horizon.
Thanks!

[SynoUser-NL]

Hi @joepavitt ,
Is there any progress or timeline on this?

[joepavitt]

It's not likely to be on the short-term roadmap. We've defined teams with the architecture of constraints to those teams in mind, so this would be a significant shift in that approach.

Remind me, are you FF Cloud or Self Hosted? With FF Cloud, we have thousands of teams, opening this up would be a huge security risk.

[SynoUser-NL]

Yes, I understand that would be problematic. But surely in FF Cloud you're also separating between organisations? Is there no one else who expressed the need to exchange msgs between teams within the same organisation?

We're self-hosted, so security wise this is much less of a problem considering no msg data (except for flowfuse-device-agent) is visible outside of the docker-enclosed system.
As said before, we're building our new flows with MQTT (with a separate MQTT broker) which is a better solution than using TCP. But still requires moving keys around before being able to send msg.payload to and fro.. (which is not ideal).

If we're the only ones requesting this, please close this thread. We'll make it work with MQTT.
No need to burden you guys with a feature only we would like to see.

Thanks!

[joepavitt]

But surely in FF Cloud you're also separating between organisations?

We do not currently have the concept of an "Organization". But it has been mentioned/discussed with a few customers.

Is there no one else who expressed the need to exchange msgs between teams within the same organisation?

We've not had it requested from others so far, but not to say it won't be.

My thinking on a "quick win" here could be to introduce an option at the FlowFuse Platform level that allows for Project Nodes to communicate cross-teams. With Project Nodes you have no need to worry about your own broker, all built in with FF.

[joepavitt]

Looking back at the Organization issue, it was mentioned about pub/sub across teams here - but with Marian having left the company, it's difficult to know if that came from Marian, or a customer.

[SynoUser-NL]

So within Flowfuse Cloud, how do you differentiate between different customers? Is a customer not the same as an organisation? Or are there organisations for which you have multiple customers (Flowfuse Cloud instances that are not connected in any way)?
Trying to understand the structure.. :)

Again, we're following the MQTT route now.
If your suggestion could be easily implemented we would welcome this! But if it's a bit more tricky and we're the only ones asking for this your time might be better spent elsewhere.

[joepavitt]

Just want to be clear, we're not disregarding this as a requirement at all, it's just not an immediate priority for us as we have other pressing tasks/requests

[SynoUser-NL]

Yes, understood.. :)