feat: add validate_json_schema function

[jlambatl]

validate_json_schema() function support

This pull request creates a VRL function that validates a payload against a JSON schema definition file. The primary goal of this feature has been to provide a mechanism for users to quickly assert that the schema matches what they expect; this simplifies various error-handling conditions into a single function.

The capability validates that the format (if included in the schema definition) matches and allows the user to determine how to handle non-standard types.

Notes:

• This change has a cache within the function; the goal of this is to improve performance. The validator compiles the schema so that subsequent requests use the schema path as the key. When the next validator is called, it uses an already compiled version if the path matches; otherwise, it creates a new validator. I looked at other implementations across the repository and wasn't sure of the idiomatic way that this should be handled. Other functions (protobuf ones) have this built into the library used, so it was a little guesswork on my part.
• For consistency, the jsonschema crate has been included without the ability to call out and download schemas; a schema must be a complete reference, which in some cases may require concatenation or merging by the user.

Change Type

• Bug fix
• New feature
• Non-functional (chore, refactoring, docs)
• Performance

Is this a breaking change?

• Yes
• No

How did you test this PR?

I have created several test cases in the same file as the function that test various features.

Does this PR include user-facing changes?

• Yes. Please add a changelog fragment based on
  our guidelines.
• No. A maintainer will apply the "no-changelog" label to this PR.

Checklist

• Our CONTRIBUTING.md is a good starting place.
• If this PR introduces changes to LICENSE-3rdparty.csv, please
  run dd-rust-license-tool write and commit the changes. More details here.
• For new VRL functions, please also create a sibling PR in Vector to document the new function.

References

[pront]

Thank you @jlambatl! Left a few comments, but nothing major.

[pront]

Awesome

[pront]

For new VRL functions, please also create a sibling PR in Vector to document the new function.

Can you followup after this PR is merged? 🙏

[pront]

Several CI checks are failing. You can run all these locally. For example, ./scripts/checks.sh tests vrl_test. If you don't pass anything to the script it will run all checks.

[jlambatl]

The script is failing the license check because MIT-0 is not explicitly permitted in deny.toml.

MIT-0 should be compatible with MIT, the primary change is that it doesn't require attribution like MIT does. Here's the license declaration https://github.com/aws/mit-0.

I am unsure how to proceed on this. Given that it's licensing-related, I don't think I should update the files without first raising this. I am keen to hear your suggestion on what the best next step is.

I think the errors, apart from the one above, should be resolved.

I have a branch waiting to create a PR in the main vector repository, I'll finalise that tomorrow

[pront]

The script is failing the license check because MIT-0 is not explicitly permitted in deny.toml.

MIT-0 should be compatible with MIT, the primary change is that it doesn't require attribution like MIT does. Here's the license declaration aws/mit-0.

I am unsure how to proceed on this. Given that it's licensing-related, I don't think I should update the files without first raising this. I am keen to hear your suggestion on what the best next step is.

I think the errors, apart from the one above, should be resolved.

I have a branch waiting to create a PR in the main vector repository, I'll finalise that tomorrow

Hi @jlambatl, we can safely add it to the allow list.

[jlambatl]

The PR for Vector can be found here: vectordotdev/vector#23359

I've updated the license check_deny configuration, and that is now passing. looking at why the wasm test/build is not passing.

[jlambatl]

I've had to exclude wasm support for the JSON schema add-on due to its use of getrandom as a transient dependency. I'll continue to examine how this can be resolved.

I'm not 100% confident that this has been done correctly, it's the first time I've had to have target based configuration in rust.

[pront]

I've had to exclude wasm support for the JSON schema add-on due to its use of getrandom as a transient dependency. I'll continue to examine how this can be resolved.

I have attempted this in the past but we had to revert because we couldn't integrate with Vector. See #1351. I don't remember exactly which crate was blocking this, but I think we are still blocked. So excluding sounds good for now.