use Hydra and Hydra-zen for configuration management and loading

[davidslater]

https://hydra.cc/

Hydra-zen adds python things to it (pure python)
https://github.com/mit-ll-responsible-ai/hydra-zen

[davidslater]

I think it could really simplify a lot of our command line, config parsing, python object loading, and config verification.

[mwartell]

Hydra-zen does offer some things that we have wanted for Armory.

It allows hierarchical dataclass-like initializing from declared defaults: an Experiment (config.json) can be composed of a Dataset, Attack, etc. You can also "subclass" with configuration overrides, for example, a carla_obj_det_dpatch_defended can be described as a carla_obj_det_dpatch_undefended plus a specific Defense override to what is null in the base case. Similarly, a performer could define a new Experiment as taking a dpatch_undefended base configuration and overriding just the parameters relevant to their defense.

Hydra-zen also creates detailed, datetime separated logs of how it was invoked and where parameter variants came from. This could naturally output to .armory/output_dir enriching the data output.json we currently put there.

The documentation is complete, but the learning curve is longer than I'd like because there seems to be a confusing amount of library specific boilerplate. If we use this though, it will be mostly set-and-forget, it seems.

[mwartell]

Related to #1698