Proposal: JSON Schema $ref for aliases (breaking change) #259
Conversation
drwpow
changed the title
drwpow
requested review from
kaelig,
c1rrus,
ChucKN0risK,
mikekamminga and
resource11
✅ Deploy Preview for dtcg-tr ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
✅ Deploy Preview for dtcg-tr ready!
To edit notification comments on pull requests, go to your Netlify site configuration. |
|
I've literally just read this, so this is a bit of a knee-jerk reaction (perhaps I'll feel differently after sleeping on it :-P), but... I have concerns about this proposal:
Re-reading #166 and thinking about the resolver spec, I wonder whether "referencing tokens in another file" is the right problem to be solving. Clearly, there's a strong demand for being able to split up your design tokens across multiple files - both to keep things manageable when you have a large number of design tokens, and to do stuff like theming by varying which files get included by some mechanism. But that doesn't necesarily require references that point to specific files. In many cases, the intent is to merge all the source files into a single collection of tokens, and then to do stuff with it. This is how tools like Theo and Style Dictionary have been doing it forever and I'd argue the Resolver Spec is another (more advanced) take on that concept too. Resolving design token references doesn't happen until after that file merging process has been done, so there's no need for them to specify a file that a token is in. We can debate whether or not that kind of merge-and-then-resolve-references approach is the "best" way to go, but my point is enabling tokens spread across multiple files can be achieved without changing the current reference syntax. So, considering how big of a breaking change this would be, I wonder if it's worth it. |
Summary
This makes a breaking change to aliases, changing the syntax:
This provides a solution for #166 as well as a future improvement for the upcoming Resolver Spec that addresses #210 (not part of this proposal; this was just partially preparing for future spec additions).
Reasoning
DTCG users have kept their files separately for a long time, and have asked for the ability to reference tokens in other files (#166). Since we were already borrowing from JSON Schema in places (primarily the
$character to mark reserved keys), this brings JSON Schema’s$refkeyword to the concept of token aliasing.The
$refkeyword comes both from JSON Schema and the OpenAPI specification (previously Swagger), and has been in use for over a decade. It is syntactically already consistent with the DTCG format, and functionally consistent as well.For the DTCG format, it is a way to keep all the functionality of aliases while extending it for future needs, borrowing from prior art where this syntax is known, used, and well-defined. For simplicity, this proposal aims to replace the previous alias syntax, rather than keep 2 conflicting ways to accomplish the same thing (which will be expanded on in points below).
Pros
This greatly expands what’s possible with the DTCG format, now that any part of the document (or remote documents, or partial remote documents) may be reused.
$extensionsif they wanted to, while selectively applying overridesWhile this is flagged as a breaking syntax change, it’s a backwards compatible functionality upgrade to aliases. There is no end-behavior nor features that should be lost with this change.
Figma Styles and Variables use the
/character in names, so now token names will map 1:1This is backwards compatible for token names. Tokens can still use
/and#characters in their names; they just have to be escaped with~0and~1respectively ONLY INSIDE$ref, according to the spec:{ "my/token": { "$type": "number", "$value": 5 }, "other-token": { "$ref": "#/my~1token" } }For DTCG parsers, this simplifies parsing/handling of aliases. Consider the old syntax:
{ "font-1": { "$type": "fontFamily", "$value": "Inter" }, "font-2": { "$type": "fontFamily", "$value": "{font-1}" } }A tool maker would have to do additional work figuring out if
$valuewas an alias or not, because both present as strings. But with the$refsyntax, if it’s an object with a$refkey, it is only an alias, no string parsing necessary.Tool makers also benefit from over a decade of prior art and tooling for handling this syntax (even if they may not have used it directly before)
Cons
"{my.token.alias"}) to an object ({ "$ref": "#/my/token/alias" }). Again, though long-term it will be easier to work with, short-term will impose migration pains.Alternatives
Those familiar with JSON Schema will also be familiar with its counterpart
$defs, which allows you to declare reusable parts of a document that can be$ref’d anywhere (but by default are ignored/not parsed). $defs are NOT being proposed here, because it doesn’t solve an immediate problem, and introduces more complexity than necessary (but could be an additive followup)An alternate idea is to keep aliases the same
{color.blue.05}, and introduce this new concept as a reference (and distinguish between the terms). Aliases would just be the “legacy” way to declare token aliases. I didn’t initially propose this because:There were no advantages I could find. Even the automatic
$typeinheritance is possible with$ref, but$refcarries more benefitsI couldn’t think of a sensible usecase of distinguishing “aliases” from “references” after reading this note in the JSON Schema spec (2019-09:
In other words, even the concept of
$reffeels spiritally identical to the existing DTCG alias syntax—there are times when you do want to preserve and reference those values later, and they are significant in some ways (even including overrides).However, I think we could define some sort of deprecation strategy where the old syntax is supported for some time before switching over.
Notes
color.blue.05orcolor/blue/05?.is allowed in token names again?2020-12DOES allow$refto have sibling keys (“overrides”), which is IMPORTANT! Without this, I believe this would lose some functionality of aliases—specifically their ability to inherit$typeautomatically.Resolver Spec problem
Though this proposal does NOT relate to the upcoming Resolver Spec proposal, I’ll share partial syntax that posed a problem:
{ "sets": [ { "values": ["foundation.json"] }, { "values": ["components/button.json"] } ] }Without going too much into detail, part of the resolver spec relied around declaring token “overrides” in individual files (modes/themes/what-have-you). However, the files would have had to be complete, full, valid DTCG format documents. Further, the references MUST be separate documents, otherwise the overrides wouldn’t work.
I believe this would lead to issues where the Resolver Spec would overreach in requiring what tokens went in what files. The DTCG format would swing drastically from “no opinion on files” to “extremely-detailed opinion about files and relative relationships.” At best, it would be a timesink shuffling tokens around between files and file boundaries. At worst, the Resolver Spec wouldn’t work if you declared your files in some magically-incorrect way (especially if you tried to reuse the same token files and apply them in different orders for different themes). And also, within the files, you have no way to reuse parts of other files.
In short, it was not only a user problem; it was an infrastructure problem having to worry about whether local or remote files would be supported, and for design system teams worrying about how to deploy and/or maintain these files in the perfect way.
Instead,
$refis a happy compromise because it allows complete flexibility in file structure. Because it allows referencing sub-documents anywhere—locally, remotely, even partial documents—you could simply organize tokens in separate files how you wanted to first (which is how all DTCG consumers operate today—being in full control of their file separation), while still being able to opt in to the Resolver Spec. So this provides a clear upgrade path with more flexibility, without asking anything in return.