Support ResourceUpdateMessages

[juliapampus]

As discussed with @jpieperbeck, @hqarawlus, and @ronjaquensel.

Current situation: After requesting metadata and data from a provider connector, the consumer connector has to periodically send a new DescriptionRequestMessage to notice resource changes to then trigger a new ArtifactRequestMessage.
Target: The provider connector should be able to send a ResourceUpdateMessage to all data consumers as soon as the resource has changed.

A possible sequence (without contract handling) would look like that:

1. Provider: Creates resource.
2. Consumer:
   a) Sends a DescriptionRequestMessage.
   b) Stores the ids:resource as a requested resource in its database.
   c) Sends an ArtifactRequestMessage with the recipient, artifact id, and uuid (for resource database entry) = validation key.
   d) Stores the data to the previously created resource. Adds the linking to a database table consisting of: uuid + artifact id + recipient (url, not connector id).
3. Provider:
   a) Updates the resource via API.
   b) Sends a ResourceUpdateMessage to interested connectors (later: for automation, the provider has to know the URLs of its data consumers) with the updated ids:resource in the payload.
4. Consumer:
   a) Receives ResourceUpdateMessage.
   b) Extracts the resource's @id from the received resource.
   c) Checks for the matching resource in its database.
   d) Checks the previously created table entry for relevant attributes: uuid + artifact id + recipient. (Why? - The ids:resource may have multiple ids:artifacts, but only the data from one of them was consumed.)
   e) Sends an ArtifactRequestMessage with the attributes of d) as parameters.
   f) Responds to the ResourceUpdateMessage with a MessageProcessedNotificationMessage.

Therefore, some changes have to be made, that are linked to issue #116 and #117.

[ghost]

@hqarawlus and I have gathered additional requirements which need to be discussed.

Currently, the handling of the resources is kept internally in the DSC. The current approach updates resources in the connector, but connected systems are not informed about updated data. This is relevant e.g. if the connector provides some policy which might be violated when using old datasets in backend systems. For this reason, the consumer connector needs to be able to broadcast this event/trigger in some way.

Possible solution:
Integration of a websocket exposed by the Connector. The Connector streams update events on this socket. Third party systems connected to the Connector can listen to the socket and trigger internal events based on websocket streams. The tasks consists of the following

• provide a websocket endpoint for the connector
• Step 4.f) While processing the ResourceUpdateMessage, stream event notification to websocket

Additional handling and security measures for endpoint regulation might be needed (authorization/role based access to endpoint)

[ghost]

Correction, websockets can't be used for broadcasting, so I suggest an MQTT broker instead, with the connector publishing in some topic of the broker.

[ghost]

After a short meeting with @brianjahnke we decided on the following:

1. Any Backend System interested in reacting to updates registers a dedicated endpoint in a new route of the DSC (e.g. POST on /admin/api/register with body my.backend.endpoint of type URL). This registers the URL in a dedicated IDS Data App, which keeps a list of all registered URLs.
2. The Data App processing is triggered on ResourceUpdateMessage handling, sending a HTTP message with static information to all registered endpoints with appropriate handling of response codes.

While App integration is currently unsupported, we will implement this feature by providing a module dedicated with the app's functionality (maybe based on the already existing App Template?), integrate the configuration in the connector's API and trigger it's execution in the ResourceUpdateMessageHandler. As to not interfere with current feaures of the DSC, this integration will be toggleable via application.properties.