Add support for running without zero optimizer sharding

[ahoffman-aws]

Would be nice to compare footprint with and without zero stage 1. It is currently not implemented

dlcalc/dlcalc/training_3d.py

Line 226 in 8cec28f

if model_repr.parallelism_cfg.zero_level != ParallelConfig.ZeroLevel.PARTITION_OPTIMIZER:

[jfc4050]

hey! sorry but i'm unlikely to get around to this because for vast majority of cases, there's no downside to using the ZeRO optimizer (less memory, less optimizer compute, less checkpoint load/store volume, and equal comm volume).

if you want to calculate it, in you should be able to multiply the optim states by DP to get back to the unpartitioned footprint.