Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat!: add support for mulitple version of cdevents' specifications #19

Merged
merged 5 commits into from
Mar 24, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 5 additions & 2 deletions .gitmodules
Original file line number Diff line number Diff line change
@@ -1,4 +1,7 @@
[submodule "cdevents-spec"]
path = cdevents-spec
[submodule "cdevents-specs/spec-v0.3"]
path = cdevents-specs/spec-v0.3
url = https://github.com/cdevents/spec.git
branch = spec-v0.3
[submodule "cdevents-specs/main"]
path = cdevents-specs/main
url = https://github.com/cdevents/spec.git
Comment on lines +1 to +7
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like the idea of having multiple submodules, we might do the same for other SDKs.
I'm not entirely sure about the main one - I think it's fine as long as there is a way to exclude it from a release. Events on main in the spec have a -draft version and are not meant to be used until they are released, so we should not release an SDK that supports producing them.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the generator exclude every json with "draft" or any modifier in the version fragment from context.type.

2 changes: 1 addition & 1 deletion Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -66,7 +66,7 @@ clean:
cargo clean

generate:
cargo run -p generator -- --templates-dir "generator/templates" --jsonschema-dir "cdevents-spec/schemas" --dest "cdevents-sdk/src/generated"
cargo run -p generator -- --templates-dir "generator/templates" --jsonschemas "cdevents-specs/*/schemas/*.json" --dest "cdevents-sdk/src/generated"

test:
cargo nextest run --all-features
Expand Down
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -18,7 +18,7 @@ To send a CDEvent as CloudEvent:
// from examples/pipelinerun_finished.rs
use std::error::Error;

use cdevents_sdk::{CDEvent, Subject, pipelinerun_finished, Content};
use cdevents_sdk::{CDEvent, Subject, spec_0_3_0::pipelinerun_finished, Content};
use cloudevents::{Event, AttributesReader};

fn main() -> Result<(), Box<dyn Error>> {
Expand Down
1 change: 1 addition & 0 deletions cdevents-sdk/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,7 @@ time = { version = "0.3", features = ["serde-human-readable"] }
[dev-dependencies]
assert-json-diff = "2.0"
boon = "0.5"
glob = "0.3"
proptest = "1"
rstest = "0.18"

Expand Down
2 changes: 1 addition & 1 deletion cdevents-sdk/examples/pipelinerun_finished.rs
Original file line number Diff line number Diff line change
@@ -1,6 +1,6 @@
use std::error::Error;

use cdevents_sdk::{CDEvent, Subject, pipelinerun_finished};
use cdevents_sdk::{CDEvent, Subject, latest::pipelinerun_finished};
use cloudevents::{Event, AttributesReader};

fn main() -> Result<(), Box<dyn Error>> {
Expand Down
2 changes: 1 addition & 1 deletion cdevents-sdk/src/cloudevents.rs
Original file line number Diff line number Diff line change
Expand Up @@ -67,7 +67,7 @@ mod tests {
#[test]
fn test_into_cloudevent() -> Result<(), Box<dyn std::error::Error>> {
let cdevent = CDEvent::from(
Subject::from(build_queued::Content{})
Subject::from(latest::build_queued::Content{})
.with_id("subject123".try_into()?)
.with_source("/event/source/123".try_into()?)
)
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ use serde::{Serialize, Deserialize};
#[serde(deny_unknown_fields)]
pub struct Content {
#[serde(rename = "signature",)]
pub signature: String,
pub signature: crate::NonEmptyString,
}

#[cfg(test)]
Expand Down
41 changes: 41 additions & 0 deletions cdevents-sdk/src/generated/change_created_0_2_0.rs
Original file line number Diff line number Diff line change
@@ -0,0 +1,41 @@
// @generated
// by cdevents/sdk-rust/generator (subject.hbs)

#[cfg(feature = "testkit")] use proptest_derive::Arbitrary;
use serde::{Serialize, Deserialize};

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[cfg_attr(feature = "testkit", derive(Arbitrary))]
#[serde(deny_unknown_fields)]
pub struct Content {
#[serde(rename = "description", default, skip_serializing_if = "Option::is_none",)]
pub description: Option<crate::NonEmptyString>,
#[serde(rename = "repository", default, skip_serializing_if = "Option::is_none",)]
pub repository: Option<ContentRepository>,
}

#[derive(Debug, Clone, Serialize, Deserialize, PartialEq, Eq)]
#[cfg_attr(feature = "testkit", derive(Arbitrary))]
#[serde(deny_unknown_fields)]
pub struct ContentRepository {
#[serde(rename = "id",)]
pub id: crate::Id,
#[serde(rename = "source", default, skip_serializing_if = "Option::is_none",)]
pub source: Option<crate::UriReference>,
}

#[cfg(test)]
mod tests {
use proptest::prelude::*;
use super::*;

proptest! {
#[test]
#[cfg(feature = "testkit")]
fn arbitraries_are_json_valid(s in any::<Content>()) {
let json_str = serde_json::to_string(&s).unwrap();
let actual = serde_json::from_str::<Content>(&json_str).unwrap();
assert_eq!(s, actual);
}
}
}
1,235 changes: 687 additions & 548 deletions cdevents-sdk/src/generated/mod.rs

Large diffs are not rendered by default.

Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ pub struct Content {
#[serde(rename = "owner", default, skip_serializing_if = "Option::is_none",)]
pub owner: Option<String>,
#[serde(rename = "url",)]
pub url: String,
pub url: crate::Uri,
#[serde(rename = "viewUrl", default, skip_serializing_if = "Option::is_none",)]
pub view_url: Option<String>,
}
Expand Down
6 changes: 3 additions & 3 deletions cdevents-sdk/src/uri.rs
Original file line number Diff line number Diff line change
Expand Up @@ -60,9 +60,9 @@ impl TryFrom<String> for Uri {
}
}

impl ToString for Uri {
fn to_string(&self) -> String {
self.0.as_str().to_owned()//into_string()
impl std::fmt::Display for Uri {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.0.as_str().fmt(f)
}
}

Expand Down
6 changes: 3 additions & 3 deletions cdevents-sdk/src/uri_reference.rs
Original file line number Diff line number Diff line change
Expand Up @@ -49,9 +49,9 @@ impl TryFrom<String> for UriReference {
}
}

impl ToString for UriReference {
fn to_string(&self) -> String {
self.0.as_str().to_owned()//into_string()
impl std::fmt::Display for UriReference {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
self.0.as_str().fmt(f)
}
}

Expand Down
80 changes: 56 additions & 24 deletions cdevents-sdk/tests/specs.rs
Original file line number Diff line number Diff line change
@@ -1,13 +1,60 @@
use assert_json_diff::assert_json_eq;
use cdevents_sdk::CDEvent;
use rstest::rstest;
use std::fs;
use rstest::*;
use std::{collections::HashMap, fs};
use std::path::PathBuf;
use proptest::prelude::*;
use boon::{Schemas, Compiler};
use boon::{Schemas, Compiler, SchemaIndex};
use glob::glob;
use std::sync::OnceLock;

struct EventsSchemas {
schemas: Schemas,
mapping: HashMap<String, SchemaIndex>,
}

impl EventsSchemas {
fn load() -> Self {
let mut schemas = Schemas::new();
let mut compiler = Compiler::new();
let mut mapping = HashMap::new();
for entry in glob("../cdevents-specs/*/schemas/*.json").expect("Failed to read glob pattern") {
let schemapath = entry.unwrap();
//TODO avoid to read the schema twice (as json, then as jsonschema)
davidB marked this conversation as resolved.
Show resolved Hide resolved
let jsonschema: serde_json::Value = serde_json::from_str(&std::fs::read_to_string(&schemapath).unwrap()).unwrap();
let ty = jsonschema["properties"]["context"]["properties"]["type"]["default"].as_str()
.unwrap_or_default()
.to_string();
mapping.entry(ty).or_insert_with(|| {
let sch_index = compiler.compile(&schemapath.to_string_lossy(), &mut schemas);
if let Err(err) = sch_index {
panic!("{err:#}"); //like a assert(false,...)
}
sch_index.unwrap()
});
}
Self {
schemas, mapping
}
}

fn check_against_schema(&self, json: &serde_json::Value, ty: &str) {
let sch_index = self.mapping.get(ty).unwrap_or_else(|| panic!("to have schema for {ty}"));
let result = self.schemas.validate(json, *sch_index);
if let Err(err) = result {
panic!("{err}");
}
}
}

static EVENTS_SCHEMA_CELL: OnceLock<EventsSchemas> = OnceLock::new();

fn events_schemas() -> &'static EventsSchemas {
EVENTS_SCHEMA_CELL.get_or_init(EventsSchemas::load)
}

#[rstest]
fn for_each_example(#[files("../cdevents-spec/examples/*.json")] path: PathBuf) {
fn for_each_example(#[files("../cdevents-specs/spec-v0.3/examples/*.json")] path: PathBuf) {
let example_txt = fs::read_to_string(path).expect("to read file as string");
//HACK uri are stored ad http::Uri, they are "normalized" when serialized, so prenormalization to avoid failure like
// json atoms at path ".subject.content.repository.source" are not equal:
Expand All @@ -30,37 +77,22 @@ fn for_each_example(#[files("../cdevents-spec/examples/*.json")] path: PathBuf)
assert_json_eq!(example_json, cdevent_json);
}

fn check_against_schema(json: &serde_json::Value, ty: &str) {
let (subject, predicate) = cdevents_sdk::extract_subject_predicate(ty).expect("valid type: {ty}");
let schemapath = format!("../cdevents-spec/schemas/{subject}{predicate}.json").to_lowercase();
//TODO optimize to not recompile a previously read schema
davidB marked this conversation as resolved.
Show resolved Hide resolved
let mut schemas = Schemas::new();
let mut compiler = Compiler::new();
let sch_index = compiler.compile(&schemapath, &mut schemas);
if let Err(err) = sch_index {
panic!("{err:#}"); //like a assert(false,...)
}
let sch_index = sch_index.unwrap();
let result = schemas.validate(json, sch_index);
if let Err(err) = result {
panic!("{err}");
}
}

#[rstest]
fn validate_example_against_schema(#[files("../cdevents-spec/examples/*.json")] path: PathBuf) {
fn validate_example_against_schema(#[files("../cdevents-specs/spec-v0.3/examples/*.json")] path: PathBuf) {
let events_schemas = events_schemas();
let example_txt = fs::read_to_string(path).expect("to read file as string");
let example_json: serde_json::Value =
serde_json::from_str(&example_txt).expect("to parse as json");
let ty = example_json["context"]["type"].as_str().expect("valid context.type in json");
check_against_schema(&example_json, ty);
events_schemas.check_against_schema(&example_json, ty);
}

proptest! {
#[test]
#[cfg(feature = "testkit")]
fn arbitraries_check_jsonschema(s in any::<CDEvent>()) {
let events_schemas = events_schemas();
let json = serde_json::to_value(&s).unwrap();
check_against_schema(&json, s.ty());
events_schemas.check_against_schema(&json, s.ty());
}
}
1 change: 1 addition & 0 deletions cdevents-specs/main
Submodule main added at d6b6f8
2 changes: 2 additions & 0 deletions generator/Cargo.toml
Original file line number Diff line number Diff line change
Expand Up @@ -13,11 +13,13 @@ description = "generate cdevents type from json schema on cdevents-spec"
anyhow = "1.0"
clap = { version = "4", features = ["derive"] }
cruet = "0.14"
glob = "0.3"
handlebars = { version = "5", features = ["dir_source"] }
handlebars_misc_helpers = { version = "0.15", default-features = false, features = [
"string",
"json",
] }
indexmap = "2.2"
serde = { version = "1.0", features = ["derive"] }
serde_json = "1.0"
url = "2.5"
4 changes: 2 additions & 2 deletions generator/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -13,9 +13,9 @@ Goals: generate rust code for cdevents from jsonschema provided as part of cdeve

## Run

To generate the `subjects` into sibling crate `cdevents/src/generated` from content of `cdevents-spec/schemas`, from root workspace
To generate the `subjects` into sibling crate `cdevents/src/generated` from content of `cdevents-specs/spec-v0.3/schemas`, from root workspace

```sh
cargo run -p generator -- --help
cargo run -p generator -- --templates-dir "generator/templates" --jsonschema-dir "cdevents-spec/schemas" --dest "cdevents-sdk/src/generated"
cargo run -p generator -- --templates-dir "generator/templates" --jsonschema-dir "cdevents-specs/spec-v0.3/schemas" --dest "cdevents-sdk/src/generated"
```
Loading