Splitstore: write epoch determinism

[raulk]

We need a little bit of guidance with a splitstore constraint.

The splitstore tracks the epoch at which writes occur, and this needs to be 100% deterministic. Right now we are subscribing to head change notifications (and assuming those arrive before the corresponding writes), but this:

a. is a bit hacky (a low-level component depending on a high-level API 🤮 ), – and –
b. can lead to unsafe/inconsistent edge cases.

Also, head change notifications are not emitted throughout fast-forward/catch-up sync, which we wrongly assumed they were, and thus was the root cause of a bug we finally managed to hunt down (cc @vyzo).

Proposal

I think a safe way to propagate that information is to add a context.Context to all blockstore methods. (In fact, this has been requested before in the context of non-local blockstores [Postgres/Instalotus]).

The caller (Syncer, StateTree, etc.) can then pass on/propagate the epoch as a context value, for full determinism.

Just wanted to kick off a discussion to see if folks would be happy with this approach.

[jennijuju]

splitstore: #5788