Skip to content

Commit

Permalink
add graph prototype
Browse files Browse the repository at this point in the history
we are updating the compatibility schema prototypes to be
entirely graphs, meaning that every node must be represented
in the graph. This also makes our lives simpler because we
do not need a separate json schema for some custom format -
we can just use JGF! While there are tools that are using V1,
I reviewed the differences and think V2 is better in that
we can lookup specific nodes in N1 time (without needing
to iterate over a list) and this seems to be the preference
for the json graph maintainers, so we should go that route
and (eventually) update underlying tools that might use
this to support that. In addition, I am adding an example
for a compatibility spec, or what would be an artifact
that represents an application or container image, and
uses one or more compatibility schemas.

Signed-off-by: vsoch <vsoch@users.noreply.github.com>
  • Loading branch information
vsoch committed Feb 2, 2024
1 parent 52c1a92 commit 69f34bb
Show file tree
Hide file tree
Showing 8 changed files with 453 additions and 104 deletions.
19 changes: 16 additions & 3 deletions .github/validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -9,16 +9,20 @@
here = os.path.abspath(__file__)
root = os.path.dirname(os.path.dirname(here))

schema_file = os.path.join(root, "definition-schema.json")
# Validation for the compspec.json files, a compatibility group schema,
# is done with an existing standard, json graph format
schema_file = os.path.join(root, "json-graph-schema_v2.json")
example_schema_file = os.path.join(root, "schema.json")

def recursive_find(base, pattern="compspec.json"):

def recursive_find(base, basename="compspec.json"):
"""
Find all compspec.json below a root
"""
for root, _, filenames in os.walk(base):
for filename in filenames:
fullpath = os.path.join(root, filename)
if re.search(pattern, fullpath):
if filename == basename:
yield fullpath


Expand All @@ -38,11 +42,20 @@ def main(root):
specs = recursive_find(root)

schema = read_json(schema_file)
example_schema = read_json(example_schema_file)

for spec_file in specs:
print(f"⭐️ Validating spec {spec_file}")
spec = read_json(spec_file)
jsonschema.validate(spec, schema=schema)

# Validate examples
for example in os.listdir(os.path.join(root, "example")):
fullpath = os.path.join(root, "example", example)
print(f"⭐️ Validating example {example}")
spec = read_json(fullpath)
jsonschema.validate(spec, schema=example_schema)


if __name__ == "__main__":
if len(sys.argv) > 1:
Expand Down
13 changes: 8 additions & 5 deletions README.md
Original file line number Diff line number Diff line change
@@ -1,19 +1,22 @@
# Compspec

This is a prototype repository for compatibility specifications that are being worked on by the [OCI compatibility working group](https://github.com/opencontainers/wg-image-compatibility). While that work is underway (and the structure and format of these metadata to be determined. For the time being we have defined:
This is a prototype repository for compatibility specifications that are being worked on by the [OCI compatibility working group](https://github.com/opencontainers/wg-image-compatibility). While that work is underway (and the structure and format of these metadata to be determined. For the time being we have defined two things:

- A **compatibility schema** is a graph (nodes and edges) that defines a space of metadata and relationships for compatibility attributes
- A **compatibility spec** is a specific extraction of these metadata attributes for a given application or container image

## JSON Schema Specifications

- [schema.json](schema.json) is used to validate the format of a json defining compatibiilty.
- [definition-schema.json](definition-schema.json) is to validate the format of the keys for each namespace (in the subdirectories here)
- The [JGF schema](https://github.com/jsongraph/json-graph-specification) version 2.0 is used to validate the compatibility schemas (compspec.json) files, which are essentially graphs with nodes and edges. Every node is required to be present in the graph.
- The [schema.json](schema.json) is used to validate a compatibility spec using one or more compatibility schemas

Both of the above are based on [Proposal C](https://github.com/opencontainers/wg-image-compatibility/pull/8) of the Image Compatibility working group, and everything is subject to change. This repository is for prototyping only.
Both of the above are based on proposals [C](https://github.com/opencontainers/wg-image-compatibility/pull/8) and [D](https://github.com/opencontainers/wg-image-compatibility/pull/9) of the Image Compatibility working group, with a stronger orientation toward D to honor the graph. This, however, means that everything is subject to change. This repository is for prototyping only!

## Organization

The different subdirectories of compatibility families (sets of metadata owned by different groups).

Likely these metadata can be moved to be owned properly by the group. They are all kept here for the time being for ease of access. Also for the time being, we have represented the entire set of labels (and smaller namespaces) for one compatibilty family (e.g., supercontainers) in one JSON file, and of course this is subject to change. I am also keeping things simple - for now each compatibility family has different groups that can define one or more key value pairs, and those pairs must be strings. This means that numbers will need to be parsed from strings by any respective tool.
Likely these metadata can be moved to be owned properly by the group. They are all kept here for the time being for ease of access. Also for the time being, we have represented the entire set of labels (and smaller namespaces) for one compatibilty family (e.g., supercontainers) in one JSON file, and of course this is subject to change.

## History

Expand Down
58 changes: 47 additions & 11 deletions archspec/compspec.json
Original file line number Diff line number Diff line change
@@ -1,14 +1,50 @@
{
"compatibilities": {
"io.archspec.cpu": {
"version": "0.0.0",
"annotations": [
"family",
"model",
"target",
"vendor"
]
}
"graph": {
"id": "io.archspec",
"type": "compspec",
"label": "compatibilities",
"nodes": {
"cpu": {
"label": "central processing unit (cpu)"
},
"cpu.family": {
"label": "cpu family"
},
"cpu.model": {
"label": "cpu model"
},
"cpu.target": {
"label": "cpu target"
},
"cpu.vendor": {
"label": "cpu vendor"
}
},
"edges": [
{
"source": "cpu",
"target": "cpu.family",
"relation": "contains"
},
{
"source": "cpu",
"target": "cpu.model",
"relation": "contains"
},
{
"source": "cpu",
"target": "cpu.target",
"relation": "contains"
},
{
"source": "cpu",
"target": "cpu.vendor",
"relation": "contains"
}
],
"metadata": {
"version": "0.0.0",
"source": "https://github.com/supercontainers/compspec"
}
}
}

28 changes: 0 additions & 28 deletions definition-schema.json

This file was deleted.

33 changes: 33 additions & 0 deletions example/compatibility-spec.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,33 @@
{
"version": "0.0.0",
"kind": "CompatibilitySpec",
"metadata": {
"name": "lammps-prototype"
},
"compatibilities": [
{
"name": "io.archspec",
"version": "0.0.0",
"schema": "https://raw.githubusercontent.com/supercontainers/compspec/main/archspec/compspec.json",
"attributes": {
"cpu.model": "13th Gen Intel(R) Core(TM) i5-1335U",
"cpu.target": "amd64",
"cpu.vendor": "GenuineIntel"
}
},
{
"name": "org.supercontainers",
"version": "0.0.0",
"schema": "https://raw.githubusercontent.com/supercontainers/compspec/main/supercontainers/compspec.json",
"attributes": {
"mpi.implementation": "mpich",
"mpi.version": "4.1.1",
"os.name": "Ubuntu 22.04.3 LTS",
"os.release": "22.04.3",
"os.vendor": "ubuntu",
"os.version": "22.04",
"hardware.gpu.available": "yes"
}
}
]
}
138 changes: 138 additions & 0 deletions json-graph-schema_v2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,138 @@
{
"$schema": "http://json-schema.org/draft-07/schema#",
"$id": "http://jsongraphformat.info/v2.1/json-graph-schema.json",
"title": "JSON Graph Schema",
"oneOf": [
{
"type": "object",
"properties": {
"graph": { "$ref": "#/definitions/graph" }
},
"additionalProperties": false,
"required": [
"graph"
]
},
{
"type": "object",
"properties": {
"graphs": {
"type": "array",
"items": { "$ref": "#/definitions/graph" }
}
},
"additionalProperties": false
}
],
"definitions": {
"graph": {
"oneOf": [
{
"type": "object",
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"label": { "type": "string" },
"directed": { "type": [ "boolean" ], "default": true },
"type": { "type": "string" },
"metadata": { "type": [ "object" ] },
"nodes": {
"type": "object",
"additionalProperties": { "$ref": "#/definitions/node" }
},
"edges": {
"type": [ "array" ],
"items": { "$ref": "#/definitions/edge" }
}
}
},
{
"type": "object",
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"label": { "type": "string" },
"directed": { "type": [ "boolean" ], "default": true },
"type": { "type": "string" },
"metadata": { "type": [ "object" ] },
"nodes": {
"type": "object",
"additionalProperties": { "$ref": "#/definitions/node" }
},
"hyperedges": {
"type": [ "array" ],
"items": { "$ref": "#/definitions/directedhyperedge" }
}
}
},
{
"type": "object",
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"label": { "type": "string" },
"directed": { "type": [ "boolean" ], "enum": [false] },
"type": { "type": "string" },
"metadata": { "type": [ "object" ] },
"nodes": {
"type": "object",
"additionalProperties": { "$ref": "#/definitions/node" }
},
"hyperedges": {
"type": [ "array" ],
"items": { "$ref": "#/definitions/undirectedhyperedge" }
}
},
"required": [ "directed" ]
}
]
},
"node": {
"type": "object",
"properties": {
"label": { "type": "string" },
"metadata": { "type": "object" },
"additionalProperties": false
}
},
"edge": {
"type": "object",
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"source": { "type": "string" },
"target": { "type": "string" },
"relation": { "type": "string" },
"directed": { "type": [ "boolean" ], "default": true },
"label": { "type": "string" },
"metadata": { "type": [ "object" ] }
},
"required": [ "source", "target" ]
},
"directedhyperedge": {
"type": "object",
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"source": { "type": "array", "items": { "type": "string" } },
"target": { "type": "array", "items": { "type": "string" } },
"relation": { "type": "string" },
"label": { "type": "string" },
"metadata": { "type": [ "object" ] }
},
"required": [ "source", "target" ]
},
"undirectedhyperedge": {
"type": "object",
"additionalProperties": false,
"properties": {
"id": { "type": "string" },
"nodes": { "type": "array", "items": { "type": "string" } },
"relation": { "type": "string" },
"label": { "type": "string" },
"metadata": { "type": [ "object" ] }
},
"required": [ "nodes" ]
}
}
}
Loading

0 comments on commit 69f34bb

Please sign in to comment.