Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validation for schema YAML files #2528

Open
8 tasks
yifanmai opened this issue Mar 29, 2024 · 0 comments
Open
8 tasks

Add validation for schema YAML files #2528

yifanmai opened this issue Mar 29, 2024 · 0 comments

Comments

@yifanmai
Copy link
Collaborator

Tasks:

  • Add validation function for schema YAML files
  • Call validation function in summarize.py i.e. helm-summarize
  • Add test that calls the validation function on all schema files included in the HELM package

Things to validate:

  • For each group name in subgroups of a run_group, there is a run_group defined with that name.
  • For each entry in metric_groups of a run_group, there is a metric_group defined with that name.
  • For each entry in metrics of metric_groups, there is either a metric with that name, or it matches the format ${var_name} (where var_name can be any name).
  • For each split in metrics of metric_groups, split is either test, valid, __all__, or matches the format ${var_name} (where var_name can be any name).
  • (maybe?) Each entry in run_groups is either a parent node (contains category and subgroups) or a child node (contains metric_groups) but not both.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

No branches or pull requests

1 participant