Grad norm inconsistency when DDP / HSDP is applied before and after #2206

[wwwjn]

Bug description

In short, DDP/HSDP and FSDP grad norms have a scaling difference of a factor of 8 for the current version of torchtitan.

Versions

latest torchtitan

[wwwjn]

#2323 is trying to solve the issue but as we are sunsetting DDP and replacing it with FSDP based replicated(), @anshul-si will handle this by disabling gradient averaging.