With having the backwards map already avaiable, we could move the forward map advection after the computation of the stream hermite similar to the particle advection. Then we could use heavily simplified schemes to advect everything.
This would then basically mean, that forward map advection and particle advection are kind of similar, with only the forward map benefitting from remapping and being represented on a structured grid