How to name/structure tests of ISO testable statements?

[tgraham-antenna]

If the goal is (as I think it should be) to (also) have minimal Schematron files to exercise the testable statements in the ISO standard, how should the tests be named/test directories be structured?

Naming tests after clause number -- e.g., 5.15.11-1.sch, 5.5.11-2.sch -- is potentially fragile since clause numbers have changed between ISO versions (as @AndrewSales points out). It's also opaque. (Who, other than @AndrewSales, knows that Clause 5.5.11 is the role attribute?)

Should the tests be named after what they test? E.g., role-1.sch, role-2.sch?

A future ISO version could invalidate an existing test (once such tests exist). However, putting 2016 in every test name as it's written seems like just adding noise.

I suggest:

• Put all tests in an iso-schematron directory.
• Name tests after what they test, e.g., role-01.sch
• Use two digits when numbering tests so that they sort well on operating systems that don't pretend to be able to recognise files with a numeric sequence. (Can we realistically ever expect more than 99 tests for a single concept?)
• If/when a new ISO version invalidates an existing test, make iso-schematron-2016 and, say, iso-schematron-2018 directories then move the existing test to iso-schematron-2016 and make a new test in iso-schematron-2018. That way, an implementation can say that it passes all of the common and version-specific tests rather than never being able to pass 100% of tests if they are all lumped together.
• Maybe the structure should go straight to iso-schematron/common, iso-schematron/2011, and iso-schematron/2016 directories instead?
• Test names should be unique across all test directories so there's no problems with test frameworks that flatten directories. E.g., if role-01.sch is moved to iso-schematron-2016 (or similar), the 2018(?)-specific test would be named role-02.sch rather than making another role-01.sch test.

Comments welcome but, realistically, the first person making an organised effort at producing tests is going to have the biggest effect on the test naming convention.