Skip to content

chore: add data processing rules, bugfixes #91

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 5 commits into from
Jan 10, 2025
Merged

chore: add data processing rules, bugfixes #91

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
5e0cb51
chore: add catalog & dataset examples (#90)
arnoweiss Dec 17, 2024
9ef5d1f
chore: adjust links in scope section (#86)
arnoweiss Dec 17, 2024
4959e1f
chore: remove outside links to ODRL, DCAT. Clarify use of profile (#73)
arnoweiss Dec 18, 2024
5dc6ec0
chore: clarify jsonld processing (#59)
arnoweiss Dec 18, 2024
f3dbe46
chore: link to ids specifications gitbook deployment for release 2024/1
arnoweiss Dec 18, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 2 additions & 4 deletions index.html
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@
<script class='remove'>
var respecConfig = {
specStatus: "unofficial",
latestVersion: null,
latestVersion: "https://docs.internationaldataspaces.org/ids-knowledgebase/dataspace-protocol",
postProcess: [window.respecMermaid.createFigures],
editors: [{
name: "Sebastian Steinbuss",
Expand Down Expand Up @@ -70,9 +70,6 @@ <h1 id="title">Dataspace Protocol</h1>
<section data-include="specifications/common/common.protocol.md" data-include-format="markdown">
</section>

<section data-include="specifications/common/common.binding.https.md" data-include-format="markdown">
</section>

<section data-include="specifications/catalog/catalog.protocol.md" data-include-format="markdown">
</section>

Expand All @@ -92,6 +89,7 @@ <h1 id="title">Dataspace Protocol</h1>
</section>

<section id='conformance'></section>

<section id="tof" class="appendix">
<h1>Notes</h1>
</section>
Expand Down
18 changes: 12 additions & 6 deletions specifications/catalog/catalog.binding.https.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,9 +50,12 @@ Authorization: ...</pre>
##### Response

If the request is successful, the [=Catalog Service=] must return an HTTP 200 (OK) with a response body containing
a [Catalog](#ack-catalog) (which is a
profiled [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) type described by
the [=Catalog Protocol=]).
a [Catalog](#ack-catalog) (which is a profiled [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) type described by the [=Catalog Protocol=]).

<aside class="example" title="Catalog Response">
<pre class="json" data-include="message/example/catalog.json">
</pre>
</aside>

### The `catalog/datasets/:id` Endpoint (Provider-side)

Expand All @@ -77,9 +80,12 @@ Authorization: ...</pre>
##### Response

If the request is successful, the [=Catalog Service=] must return an HTTP 200 (OK) with a response body containing
a [Dataset](#ack-dataset) (which is
a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) type described by
the [=Catalog Protocol=]).
a [Dataset](#ack-dataset) (which isa [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) type described by the [=Catalog Protocol=]).

<aside class="example" title="Dataset Response">
<pre class="json" data-include="message/example/dataset.json">
</pre>
</aside>

## Technical Considerations

Expand Down
89 changes: 5 additions & 84 deletions specifications/catalog/catalog.protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -7,89 +7,12 @@ This document outlines the [=Catalog Protocol=]. The used terms are described in
The Catalog Protocol defines how a [=Catalog=] is requested from a [=Catalog Service=] by a [=Consumer=] using an
abstract message exchange format. The concrete message exchange wire format is defined in the binding specifications.

### DCAT Vocabulary Mapping

This section describes how the DSP Information Model maps to [DCAT] resources [[vocab-dcat-3]].

#### Dataset

A [=Dataset=] is a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset) with the following attributes:

##### odrl:hasPolicy

A [=Dataset=] must have 1..N `hasPolicy` attributes that contain
an [ODRL Offer](https://www.w3.org/TR/odrl-vocab/#term-Offer) defining the [=Usage Policy=] associated with
the [=Catalog=]. Offers must NOT contain any
explicit `target` attributes. The `target` of an [=Offer=] is the associated [=Dataset=]. This is in line with the
semantics of `hasPolicy` as defined in the [ODRL Information Model](https://www.w3.org/TR/odrl-model/#policy-has),
explaining that the subject (here the Dataset) is automatically the `target` of each Rule. To prevent conflicts,
the `target` attribute must not be set explicitely, for example, in the [=Offer=] or Rules.

#### Distributions

A [=Dataset=] may contain 0..N [DCAT Distributions](https://www.w3.org/TR/vocab-dcat-3/#Class:Distribution). Each
distribution must have at least one `DataService` which specifies where the distribution is obtained. Specifically,
a `DataService` specifies the endpoint for initiating a [=Contract Negotiation=] and [=Transfer Process=].

A Distribution may have 0..N `hasPolicy` attributes that contain
an [ODRL Offer](https://www.w3.org/TR/odrl-vocab/#term-Offer) defining the [=Usage Policy=] associated with
the [=Dataset=] and this explicit `Distribution`. [=Offers=] must NOT contain any target attributes. The target of
an [=Offer=] is the [=Dataset=] that contains the distribution.

Support for `hasPolicy` attributes on a `Distribution` is optional. Implementations may choose not to support this
feature, in which case they should return an appropriate error message to clients.

#### Data Service

A Data Service may specify an endpoint supporting the Dataspace Protocol such as a [=Connector=].

##### dspace:dataServiceType

If the Data Service refers to an endpoint that supports the Dataspace Protocol, it must include the
property `dspace:dataServiceType`:

| Category | Description |
|------------|----------------------------------------------------------------------------|
| Definition | Specifies the service type |
| Domain | [dcat:DataService](https://www.w3.org/TR/vocab-dcat-3/#Class:Data_Service) |
| Type | xsd:string |
| Note | The value of this field is left intentionally open for future extension. |

The following table lists well-know endpoint types:

| Value | Description |
|--------------------|---------------------------|
| `dspace:connector` | A [=Connector=] endpoint. |
| | |

##### dcat:servesDataset

Note that the property `dcat:servesDataset` should be omitted from the `DataService`
since [=Datasets=] are included as top-level entries. Clients are not required to process
the contents of `dcat:servesDataset`.

#### Participant Id

The identifier of the participant providing the [=Catalog=] is specified using the `dspace:participantId` attribute on
that [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog).

### DCAT and ODRL Profiles

The [=Catalog=] is a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) with the following restrictions:

1. Each [ODRL Offer](https://www.w3.org/TR/odrl-vocab/#term-Offer) must be unique to a [=Dataset=] since the target of
the [=Offer=] is derived from its enclosing context.
2. A [=Catalog=] must not have an `odrl:hasPolicy` attribute, since it is not intended to negotiate on the access
to [=Catalog=] objects. An implementation might however regulate the visibility and/or the content of its [=Catalog=]
dependent of the requester.
The [=Catalog Protocol=] reuses properties from the DCAT and ODRL vocabularies with restrictions defined in this
specification. This is done implicitly by the use of the JSON schemas and JSON-LD-contexts that are part of the DSP.
Servers have no obligation to process properties that are not part of the schemas.

## Message Types

All messages must be serialized in JSON-LD compact form as specified in
the [JSON-LD 1.1 Processing Algorithms and API](https://www.w3.org/TR/json-ld11-api/#compaction-algorithms).
Further [=Dataspace=] specifications may define additional optional serialization
formats.

### Catalog Request Message

| | |
Expand All @@ -103,8 +26,7 @@ formats.

The Catalog Request Message is message sent by a [=Consumer=] to
a [=Catalog Service=].
The [=Catalog Service=] must respond with a [Catalog](#ack-catalog), which
is a valid instance of a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog).
The [=Catalog Service=] must respond with a [Catalog](#ack-catalog) that adheres to the schema linked above.

- The message may have a `filter` property which contains an implementation-specific query or filter expression type
supported by the [=Catalog Service=].
Expand All @@ -126,8 +48,7 @@ is a valid instance of a [DCAT Catalog](https://www.w3.org/TR/vocab-dcat-3/#Clas

The Dataset Request Message is message sent by a [=Consumer=] to
a [=Catalog Service=].
The [=Catalog Service=] must respond with a [Dataset](#ack-dataset), which
is a valid instance of a [DCAT Dataset](https://www.w3.org/TR/vocab-dcat-3/#Class:Dataset).
The [=Catalog Service=] must respond with a [Dataset](#ack-dataset) that adheres to the schema linked above.

- The message must have a `dataset` property which contains the id of the [=Dataset=].

Expand Down
17 changes: 0 additions & 17 deletions specifications/common/common.binding.https.md
View file
Open in desktop

This file was deleted.

38 changes: 35 additions & 3 deletions specifications/common/common.protocol.md
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,8 @@

## Exposure of Dataspace Protocol Versions

### Generic Definition

[=Connectors=] implementing the Dataspace Protocol may operate on different versions. Therefore, it is necessary that
they can discover the supported versions of each other reliably and unambiguously. Each [=Connector=] must expose
information of at least one Dataspace Protocol Version it supports. The specifics of how this information is obtained
Expand All @@ -23,7 +25,37 @@ offers version `1.0` endpoints at `<host>/some/path/v1`.
}
```

This data object must comply to the [JSON Schema](schema/version-schema.json).
This data object must comply to the [JSON Schema](schema/version-schema.json). The requesting [=Connector=] may select
from the endpoints in the response. If the [=Connector=] can't identify a matching Dataspace Protocol Version, it must
terminate the communication.

### HTTP Binding

#### The Well-Known Version Metadata Endpoint

Each implementation must provide the version metadata endpoint, which must use the `dspace-version` Well-Known Uniform
Resource Identifier [[rfc8615]] at the top of the path hierarchy:

```
/.well-known/dspace-version
```

The contents of the response is a JSON object defined in section [[[#exposure-of-dataspace-protocol-versions]]].

Note that if multiple [=Connectors=] are hosted under the same base URL, a path segment appended to the base well-known
URL can be used, for example, `https://example.com/.well-known/dspace-version/connector1.`

#### Authorization

All requests to HTTPS endpoints should use the `Authorization` header to include an authorization token. The semantics
of such tokens are not part of these specifications. The `Authorization` HTTP header is optional if the [=Connector=]
does not require authorization.

## Schemas, Contexts, and Message Processing

The requesting [=Connector=] may select from the endpoints in the response. If the [=Connector=] can't identify a
matching Dataspace Protocol Version, it must terminate the communication.
All protocol messages are normatively defined by a [json-schema]. This specification also uses JSON-LD 1.1 and provides
a JSON-LD context to serialize data structures and message types as it facilitates extensibility. The JSON-LD context is
designed to produce message serializations using compaction that validate against the Json Schema for the given message
type. This allows implementations to choose whether to process messages as plain Json or as JSON-LD and maintain
interoperability between those approaches. Extensions that use JSON-LD are encouraged to provide similar contexts that
facilitate this approach to interoperability.
43 changes: 29 additions & 14 deletions specifications/common/scope.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,30 +1,45 @@
# Scope

Sharing data between autonomous entities requires the provision of metadata to facilitate the transfer of [Datasets](../model/terminology.md#dataset) by making use of a data transfer (or application layer) protocol.
Sharing data between autonomous entities requires the provision of metadata to facilitate the transfer of [=Datasets=]
by making use of a data transfer (or application layer) protocol.
The __Dataspace Protocol__ defines how this metadata is provisioned:

1. How [Datasets](../model/terminology.md#dataset) are deployed as [DCAT Catalogs](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) and usage control is expressed as [ODRL Policies](https://www.w3.org/TR/odrl-model/).
2. How [Agreements](../model/terminology.md#agreement) that govern data usage are syntactically expressed and electronically negotiated.
3. How [Datasets](../model/terminology.md#dataset) are accessed using [Transfer Process Protocols](../model/terminology.md#transfer-process-protocol).
1. How [=Datasets=] are advertised via DCAT [=Catalogs=] and usage control is expressed as ODRL [=Policies=].
2. How [=Agreements=] that govern data usage are syntactically expressed and electronically negotiated.
3. How [=Datasets=] are accessed using [=Transfer Process Protocols=].

These specifications build on protocols located in the [ISO OSI model (ISO/IEC 7498-1:1994)](https://www.iso.org/standard/20269.html) layers, like HTTPS.
The purpose of this specification is to define interactions between systems independent of such protocols, but describing how to implement it in an unambiguous and extensible way.
To do so, the messages that are exchanged during the process are described in this specification and the states and their transitions are specified as state machines, based on the key terms and concepts of a [Dataspace](../model/terminology.md#dataspace).
The purpose of this specification is to define interactions between systems independent of such protocols, but
describing how to implement it in an unambiguous and extensible way.
To do so, the messages that are exchanged during the process are described in this specification and the states and
their transitions are specified as state machines, based on the key terms and concepts of
a [Dataspace](../model/terminology.md#dataspace).
On this foundation the bindings to data transfer protocols, like HTTPS, are described.

The specifications are organized into the following documents:

* [__*Dataspace Model*__](../model/model.md) and [__*Dataspace Terminology*__](../model/terminology.md) documents that define key terms.
* [__*Common Functionalities*__](common.protocol.md) and their [__*Binding in HTTPS*__](common.binding.https.md) declares cross-cutting functions as, e.g., the declaration of supported versions of this Dataspace Protocol.
* [__*Catalog Protocol*__](../catalog/catalog.protocol.md) and [__*Catalog HTTPS Binding*__](../catalog/catalog.binding.https.md) documents that define how [DCAT Catalogs](https://www.w3.org/TR/vocab-dcat-3/#Class:Catalog) are published and accessed as HTTPS endpoints respectively.
* [__*Contract Negotiation Protocol*__](../negotiation/contract.negotiation.protocol.md) and [__*Contract Negotiation HTTPS Binding*__](../negotiation/contract.negotiation.binding.https.md) documents that define how [Contract Negotiations](../model/terminology.md#contract-negotiation) are conducted and requested via HTTPS endpoints.
* [__*Transfer Process Protocol*__](../transfer/transfer.process.protocol.md) and [__*Transfer Process HTTPS Binding*__](../transfer/transfer.process.binding.https.md) documents that define how [Transfer Processes](../model/terminology.md#transfer-process) using a given data transfer protocol are governed via HTTPS
* [__*Dataspace Model*__](#dataspace-information-model) and [__*Dataspace Terminology*__](#terminology) documents that
define key terms.
* [__*Common Functionalities*__](#general-common-protocol-requirements) declares cross-cutting functions as, e.g., the declaration of
supported versions of this Dataspace Protocol and common data processing rules.
* [__*Catalog Protocol*__](#catalog-protocol) and [__*Catalog HTTPS Binding*__](#catalog-https-binding) documents that
define how [=Catalogs=] are published and accessed as HTTPS endpoints respectively.
* [__*Contract Negotiation Protocol*__](#contract-negotiation-protocol) and [__*Contract Negotiation HTTPS Binding*
__](#contract-negotiation-https-binding) documents that define how [=Contract Negotiations=] are conducted and
requested via HTTPS endpoints.
* [__*Transfer Process Protocol*__](#transfer-process-protocol) and [__*Transfer Process HTTPS Binding*
__](#transfer-process-https-binding) documents that define how [=Transfer Processes=] using a given data transfer
protocol are governed via HTTPS.
endpoints.

> **This specification does not cover the data transfer process as such.**
>
> While the data transfer is controlled by the __*Transfer Process Protocol*__ mentioned above, e.g. the initation of the transfer channels or their decomissioning, the data transfer itself and especially the handling of technical exceptions is an obligation to the Transport Protocol.
> While the data transfer is controlled by the __*Transfer Process Protocol*__ mentioned above, e.g. the initation of
> the transfer channels or their decomissioning, the data transfer itself and especially the handling of technical
> exceptions is an obligation to the Transport Protocol.
>
> As an implication, the data transfer can be conducted in a separated process if required, as long as this process is to the specified extend controlled by the __*Transfer Process Protocol*__.
> As an implication, the data transfer can be conducted in a separated process if required, as long as this process is
> to the specified extend controlled by the __*Transfer Process Protocol*__.
>
> Nevertheless, illustrative message examples are provided in the [__*Transfer Process Protocol section*__](../transfer/transfer.process.protocol.md#2-message-types). The best practices section may contain further non-normative examples and explanations.
> Nevertheless, illustrative message examples are provided in the [__*Transfer Process Protocol section*
__](#message-types-1). The best practices section may contain further non-normative examples and explanations.
Loading
Loading