The "sibling font problem"

[simoncozens]

In current fontbakery, we take a list of files from the command line and pass them to checks. In theory, we then route each file to appropriate checks based on the file type (eg a METADATA.pb gets routed to checks on metadata). In practice, we don't; (nearly) all checks operate on TTF files, and a check that wants to deal with a METADATA.pb actually receives a Font, asks the font file for its parent directory, and looks in the directory to see if there's a METADATA.pb in there:

def com_google_fonts_check_metadata_parses(family_directory):
    """Check METADATA.pb parse correctly."""
    # ...

    try:
        pb_file = os.path.join(family_directory, "METADATA.pb")

That's obviously very filesystem-dependent, which is unfortunate when it comes to writing a WASM version. But it has the advantage that you can do fontbakery check-googlefonts foo.ttf and it'll also do the metadata checks and the HTML description checks and so on. There's a high Do What I Mean factor there, which I like. The downside is that fontbakery check-googlefonts ZillaSlab-*.ttf checks that the same METADATA.pb file parses 10 times.

And there are some checks which apply across the TTFs - for example, ensuring all the fonts (let's say, regular and italic VFs) have the same UPM or family name or glyphset. We currently assume all the TTF files passed in on the command line are "siblings", and we pass the whole array of fonts to a check which takes a fonts parameter, meaning the check only gets run once. Which is good, but the downside is that you can only check one family at a time - if you have fonts from different directories/families on the same command line, they will be considered siblings even though they aren't, and all the "have the same family name" checks will fail. We've been kind of saying that this is OK, but I'd really love to be able to run fontbakery check-googlefonts ofl/*/*.ttf and check every family at once.

I'm trying to think of a better way to handle this "sibling font" problem in fontbakery-nextgen.

It feels like the obvious thing to do is organise fonts/metadata.pb/HTML files into a TestableCollection based on their directory, run individual font/metadata/etc. checks on each Testable in the collection, and run checks which apply across the collection on the TestableCollection object. I like this, and I started buildiing the infrastructure (check.run_one(&testable) versus check.run_many(&collection), but (a) it's very filesystem-dependent, (b) you then have to either specify all the files you want testing in a single directory (e.g. ofl/*/{*.ttf,METADATA.pb,*.html}) or you consider the directory to be the basic unit of testing, supply a list of directories and have FB read the directory for all the files to test - but I don't want to do that because people might put development versions / old versions / comparison versions of font binaries in their directory and including them in the list of sibling fonts will mess up all the checks. Also reading the directory just totally won't work at all where there is no filesystem.

Where it starts to become really messy is that in the Google Fonts profile, there's a check which takes a Font and looks for a file called article/ARTICLE.en_us.html - i.e. a file which is explicitly in a different subdirectory to the one being checked. So even ofl/*/{*.ttf,METADATA.pb,*.html} won't work.

I can't think of a good solution here. Suggestions welcome...

[Hoolean]

Sounds great! I like the idea of projects being explicitly grouped :)

I wonder if explicit arguments on the CLI could expose this most easily and with the least magic, in a similar manner to what we do with fontmake now. For example:

• fcheck googlefonts --gfproject path/
  • Gathers path/fonts/*.ttfs, path/METADATA.pb, path/*/*.html, etc
• fcheck universal --designspace MyFont.designspace
  • Gathers designspace, and UFOs referenced as sources
• fcheck universal --ttf-family *.ttf
  • Gathers only TTFs specified (grouped as one family)
• fcheck universal --ttf-bulk ofl/*/*.ttf
  • Gathers only TTFs specified (as individual families)

These could be chained for less common operations, for example:

• fcheck googlefonts --gfproject ofl/*
  • Check many independent projects, individually grouped by directory
• fcheck googlefonts --ttf-family MyFontSans-*.ttf --ttf-family MyFontSerif-*.ttf
  • Check two families independently (e.g. same family name expected within families but not across)

This would abstract the gathering and grouping of files for checks away from the checks themselves, and so individual checks could avoid direct file system operations. If the final checks receive a struct of HashMap<Path, Contents> (or even something more specialised for each project structure) then a WASM frontend could implement the same interface as the CLI, with the added benefit that we avoid duplicate IO calls.

The main downside is that it may be difficult to communicate when * arguments are expected to be received as a single family and when they are not; maybe this is something we would have to solve for any solution we implement here though.

[simoncozens]

Thank you, that was helpful. I think it's clarified for me that a) a TestableCollection is the basic unit of testing, b) how we marshal a command-line or WASMy web page full of filenames / directory names into a TestableCollection is potentially tricky, but can be a problem for another day, or with multiple possible solutions, because that's a different level of abstraction. fontspector architecture is split up into multiple components:

• fontspector-api which defines basic data structures and utility functions
• profile-* which implements the checks
• fontspector-cli which is the, uh, CLI interface
• fontspector-web which is the WASM version

I want the API and the checks to have a nice clean interface. Everything else can be messy as hell. :-)