Automatically restart apps when configmaps change

[blairdrummond]

Might leverage this

https://github.com/xing/kubernetes-deployment-restart-controller

Some apps (goofys injector, minio credential injector, etc) fetch values from a configmap, and do not restart when those settings change.
This means that the gitops flow is a bit manual, even with ArgoCD. We might want a way to automate some rollouts

[Jose-Matsuda]

Additionally look at https://github.com/stakater/Reloader/

[Jose-Matsuda]

The PR is in dev here https://github.com/StatCan/aaw-argocd-manifests/pull/361

Here is a list of configmaps that may be modified a lot causing us needing to do restarts.

grafana-alerting-cm
oidc-authservice-parameters
centraldashboard-config
jupyter-web-app-config --> maybe anything with jwa
notebook-controller-config

ON PROD (we don't have this cm on dev)
non-employee-exceptions
TBD (dev stuff)

apm-server-cm
otelcollector-collector
trino

I took a look at some secrets but they seem inconsequential for the most part / they would be covered by the deployments / resources that would get annotated by the cms anyways.
So we just annotate the resources that use these cms and then bam we're set (at least for dev). And of course if we see one missing we can just add it later, its a single annotation and boom it's done

[Jose-Matsuda]

Closing as completed for DEV. Synched up changed deployments, made changes to the configmap and observed restarts.

Follow up for PROD here; #1899