CPSs: Validation workflow and documentation

[Ryun1]

Goes part way to address #989, specifically just CPS document validation, CIP validation still to come.

This script and schema were used to create and validate the changes in #1128.

Changes

• Added a JSON schema definition for CPS headers
• Added a python script to validate CPS headers, and structure
• Added a github workflow to run the script when a CPS documented is added or changed
• Added a description document explaining the validations ran by the script

For now I have set the workflow to run ONLY when manually triggered
so we can test it out on a few real examples
before having it run across all existing PRs automatically

Motivation

• Improve compliance to CPS template and CIP-9999 for new CPSs
  • Making downstream tooling more consistent
  • Improve overall consistency between CPSs
    • Codifying some of the MUSTs from CIP-9999
• Automate some of the CIP editors role
  • Giving authors automated feedback if they deviate from template

Examples

Examples ran on my CIPs fork (see PR view):

• When a CPS follows expected template: See runs/21783966089 for commit 62fdcf6
• When header missing a field: See runs/21784045648 for commit a68e96e
• When header fields in wrong order: See runs/21784063562 for commit 3a3f846

Additionally I have generated 44 test CPS files on cps-validation-test within .github/tests/valid and .github/tests/invalid.
These test files were all run locally with validate-cps.py.

Open Questions

• Is where I have put the schema and validation description the best place?
  • maybe they're better suited as a part of CIP-999?
• While we are in the .github directory, maybe we could move the templates to their own dedicated directory?, it may help people find them easier

[rphair]

• Is where I have put the schema and validation description the best place?
  • maybe they're better suited as a part of CIP-999?

I don't have enough experience with CI best practice to know if it's the best place, but I know they're better off in a "system" location than in CIP-9999 which is a document intended for authors... just as the other CIPs are primarly intended for developers implementing those specifications.

• i.e. the "implementation" of CIP-9999 is a written CPS: and the GitHub CI is nothing to do with that.

Only editors (or prospective editors, or maybe developers of downstream sites using this content: like cips.cardano.org) will be interested in the precise document structure and/or validation workflow.

• The schema would be of interest to developers only if we had software writing CPSs, which is likely a long way off.
• Perhaps in the nearer term: if someone wants to use it for AI training then it's only a matter of that user plugging in a pathname or two, which they'd have to do anyway.

If taking an "educational" approach, then maybe some of the high-level descriptions could be put on the repository Wiki: as a sup-topic in the 3. Editors section (more esoteric than anything in this section so far: so definitely below all existing pages).

• Please feel free to put anything on the Wiki that you wish: although I think the documentation location is fine the way you have it.

TL;DR I think everywhere you have placed these things is fine. ✅

• While we are in the .github directory, maybe we could move the templates to their own dedicated directory?, it may help people find them easier

Again, I'm not the one to say whether this is "best practice", but I do know that after 5 years of the templates being in the .github directory it could cause some confusion to move them. If you created another directory called templates then the name would refer to "documents" while all the other names refer to "automation"... again, 2 different kettles of fish & better therefore to keep those target audiences separate.

And likewise I wouldn't put any CI scripts or documentation at the top level: most visitors to the repo will only be interested in "standards" documents and it would be awkward to see, at first glance, files or directories serving a different purpose.

So again TL;DR I think your existing "hidden" file/directory structure is both logical and well-precedented. ✅

[rphair]

OK, this is Confirmed from the CIP meeting today & consensus (cc @perturbing) is that we are mainly waiting for @Ryun1 to hit the ON switch 🤓 although we diplomatically agreed to assign Last Check status as soon as he believes it to be ready.

NOTE it also came up in the meeting that generally CIP authors & habitual reviewers don't post comments or reviews about CIP repo PRs like this... although I also mentioned that feedback from those knowledgeable about the CIP process would be welcome to contribute feedback on this PR or any of the related PRs or issues on repo content validation.