You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
So, transcripts are supposed to be a way to reproduce behavior, and are very helpful for e.g. writing tests. However, if a transcript runs pull in a ucm block, then it is dependent on externally varying data.
Critically, this means that any tests in unison that use transcripts are broken with respect to git -- checking out a previous commit and running tests won't necessarily give you the same test results as when that git revision was created, because base may have changed.
My proposal is that, in addition to the .md transcript file, we also generate a .lock file, which contains the sha of anything that was pulled. Then, by default, when running a transcript that has a .lock file present, we use that sha, not HEAD, for anything we pull.
This reproducibility problem is compounded for transcripts that are run with .fork, but a .lock file could help us there too! It could record the sha of the forked codebase, and the default would be to error out if run on codebase in a different state.
If you want to ignore the lockfile and "update" things, then you could pass a -U flag or something like that. (we could also decide that the "default" behavior is to update, and you need to pass a flag to use the lockfile 🤷).
Relatedly our jit tests are broken with respect to git because they fetch the scheme generated code from unison share without specifying any sort of hash; so when the scheme generating code changes in a meaningful way, all previous revisions will fail jit-tests, even if they were passing at the time. This means we can't do git bisect to find when a thing broke, which is unfortunate.
reacted with thumbs up emoji reacted with thumbs down emoji reacted with laugh emoji reacted with hooray emoji reacted with confused emoji reacted with heart emoji reacted with rocket emoji reacted with eyes emoji
-
So, transcripts are supposed to be a way to reproduce behavior, and are very helpful for e.g. writing tests. However, if a transcript runs
pull
in aucm
block, then it is dependent on externally varying data.Critically, this means that any tests in
unison
that use transcripts are broken with respect to git -- checking out a previous commit and running tests won't necessarily give you the same test results as when that git revision was created, becausebase
may have changed.My proposal is that, in addition to the
.md
transcript file, we also generate a.lock
file, which contains the sha of anything that waspull
ed. Then, by default, when running a transcript that has a.lock
file present, we use that sha, not HEAD, for anything wepull
.This reproducibility problem is compounded for transcripts that are run with
.fork
, but a.lock
file could help us there too! It could record the sha of the forked codebase, and the default would be to error out if run on codebase in a different state.If you want to ignore the lockfile and "update" things, then you could pass a
-U
flag or something like that. (we could also decide that the "default" behavior is to update, and you need to pass a flag to use the lockfile 🤷).Relatedly our
jit
tests are broken with respect to git because they fetch the scheme generated code fromunison share
without specifying any sort of hash; so when the scheme generating code changes in a meaningful way, all previous revisions will failjit-tests
, even if they were passing at the time. This means we can't dogit bisect
to find when a thing broke, which is unfortunate.Beta Was this translation helpful? Give feedback.
All reactions