re-enable rematerialization by default after a memory-aware way is found

[t-vi]

#1560 temporarily disables rematerialization by default to mitigate #1337

What happens is that it tries to fuse all nonlinearities, which works if you compute all pre-nonlinearity-activations at once and the put them into the fusion.
So I think we fundamentally need a memory-aware way of moving operations around for things like rematerialization.
One thing could be to not do the full graph approach but try some local surgery.

cc @riccardofelluga