Correctly get parameters with unsharded grads and parameters to register sharded and sync'ed grads #5611
| Job | Run time |
|---|---|
| 2s | |
| 13s | |
| 59s | |
| 22s | |
| 2m 45s | |
| 2m 24s | |
| 1m 4s | |
| 49s | |
| 2m 29s | |
| 3m 7s | |
| 1s | |
| 14m 15s |
| Job | Run time |
|---|---|
| 2s | |
| 13s | |
| 59s | |
| 22s | |
| 2m 45s | |
| 2m 24s | |
| 1m 4s | |
| 49s | |
| 2m 29s | |
| 3m 7s | |
| 1s | |
| 14m 15s |