MQTT for Adapter/Agent communication

[scottmwyant]

Writing in response to your proposal to send data from the adapter to the agent over MQTT (4th bullet point in #29).

IMO, it seems the communication between the agent and the adapters is an implementation detail. All of the big projects I've seen dealing with MT Connect (MTConnect Institute, TrakHound, Mazak) provide implementations of both components, Adapter and Agent.

As an aside, it would be good if the 3rd party developers would be clear about how their Agent differs (if it does) from the reference implementation.

As a general consumer of this technology, I don't care what protocol is used between the components as long as they talk. In this mindset, it may only become relevant if I want to mix and match components from different developers.

What appears would make a larger impact to consumers would be an MQTT agent. I believe this is addressed in v2.0 of the spec; I need to go study the spec again.

In any case, the big differences in my mind are who initializes the connection, and how many hops are needed.

Using TCP, the adapter is listening for connections, the Agent knows about the adapter and makes a connection. With MQTT the adapter and the agent would both have to know about a broker. Great abstraction between the adapter and the agent but I'm with you, I don't see an obvious use case. Seems like it would be more useful to have an option for MQTT between the Agent and the client application.

Best case scenario in my mind would be an agent and adapter running locally on the HMI so that the agent can push JSON data to a cloud service, etc.

[PatrickRitchie]

Yes, the Adapter > Agent communication is implementation specific and not technically part of the MTConnect standard. The SHDR over TCP communication was just something that MTConnect developed as a simple protocol to feed data into the Agent from an external application.

One of the original purposes for developing the Agent portion of MTConnect.NET was to eliminate the SHDR/TCP connection as any Adapter that is written in .NET should be able to use MTConnect.NET to implement an Agent in the "Adapter" application itself. So instead of an Adapter application reading from a PLC and sending that data over TCP to the Agent, using MTConnect.NET the Adapter application can read from the PLC and then run an MTConnect Agent inside the same application.

I'm currently working on a new Docs site that has documentation, tutorials, etc. that will hopefully clarify all of the functionality of MTConnect.NET and how to create Agents, Adapters, and Clients using it. This will document what is part of the MTConnect standard as well as the non-standard functionality that I've added.

I agree that using MQTT for the Agent > Client communication would be much more valuable than the Adapter > Agent communication. I started working on an MQTT Agent implementation HERE but I don't have it finished. If you or anyone else is interested in helping (testing, feedback, etc.) there is a simple demo running at the below MQTT connection:

Host: trakhound.com
Port: 1883

I was still working on the topic format and the source code for it is not organized yet but it could be ready to package and test without too much effort if I have anyone that is interested. I would definitely appreciate any feedback on any of that.

In the MQTT Agent I started setting up, the application would read from the PLC and then also host an MQTT broker within the same application. I can also make one that reads from SHDR over TCP and/or one that just sends the MQTT messages to an external broker.

[scottmwyant]

Interesting. I didn't imagine so many different ways of setting this up but, yes, I'm connecting the dots now.

Perhaps this is already what you had in mind... I'm picturing the "Adapter" as you describe, passing data to an IMTConnectAgent interface, which would then either throw it in the buffer if it's the traditional web-server kind of agent (using HTTP) or forward it to a broker if it's an MQTT agent. Either way it's deployed as a single application. Lots of options.

You mention testing; yeah that's something I should be able to help with.

[PatrickRitchie]

Yes, that is what the original idea was, to pass data directly to the IMTConnectAgent interface. This creates an abstraction between both the source of the data and how the data is requested. The IMTConnectAgent interface is responsible for all of the agent logic, creation of response documents, unit conversion, filters, etc. but is not responsible for any communication or formatting.

With MQTT, the IMTConnectAgent is actually using regular .NET events to pass data to the MQTT broker. If the broker is hosted in the same application (as I did in my example), this would allow the Agent to update the broker without any over the wire communication.

For most scenarios, if you are developing a SHDR adapter in .NET, you should be able to just host the Agent in the same application using MTConnect.NET.

For scenarios that still need a separate adapter, my plan is to have a multitude of different Agent applications all setup with various input types (SHDR, etc.) and output types (Http, MQTT, etc).

Examples of that could be:

• SHDR -> HTTP
• SHDR -> MQTT
• HTTP -> HTTP
• MQTT -> HTTP
• MQTT -> MQTT

I should have a new Release that will include all of the latest commits ready sometime next week. I'll also put together a better example of the MQTT agent for you or anyone else to test. I'm still very open to changes about the MQTT topic structure so any feedback would be appreciated with that.

[PatrickRitchie]

I just published the 0.6.0 release that includes some of the updates mentioned in this thread. It includes a lot of changes among which are the updates to ShdrAdapter which I mentioned. Let me know if anyone finds any issues with it or have any questions on how to implement the new classes.

[PatrickRitchie]

@scottmwyant I just published the new 1.3.0 release that contains MQTT Agents. There are two MQTT Agents, one that publishes to an external MQTT broker and one that has an MQTT broker built in.

The topic structure is listed in the ReadMe and is structured so that you can subscribe similar to below:

All Devices:

MTConnect/Devices/#

A specific Device:

MTConnect/Devices/[DEVICE_UUID]/#

A specific Device's Observations:

MTConnect/Devices/[DEVICE_UUID]/Observations/#

A specific Device's Path Observations:

MTConnect/Devices/[DEVICE_UUID]/Observations/Path/#

Let me know if you get a chance to test any of this and let me know if you find any issues or have any other feedback.

Thanks,
-Patrick