Remove cargo audit CI job again

We realized that libraries should probably not check for CVEs.
It will generate too many false positives and provide very little value.
It's up to downstream *program* developers to select exact versions
of transitive dependencies. If it ends up being that no version of one
of our dependencies is safe/works, then that program developer must
report to this library that we should probably consider
upgrading/replacing that dependency with something better.