Example of defining scheduled scaling for Prometheus

[mikkeloscar]

The Scheduled Scaling VPA defines a Vertical Pod Autoscaler (zVPA) resource which is an enriched version of the upstream VPA that adds the ability to change scaling based on Scaling Schedules.

An example zVPA looks like this:

apiVersion: zalando.org/v1
kind: VerticalPodAutoscaler
metadata:
  name: scheduled-scaling-vpa-test
spec:
  resourcePolicy:
    containerPolicies:
    - containerName: main
      minAllowed:
        cpu: 0
        memory: 500Mi
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: scheduled-scaling-vpa-test
  updatePolicy:
    updateMode: Auto
  # above this line looks like a normal VPA
  # below this line are references to scaling schedules and the resourcePolicy
  # overwrite.
  schedules:
  - name: "scheduling-event"
    preStartMinutes: 10
    resourcePolicy:
      containerPolicies:
      - containerName: main
        minAllowed:
          cpu: 0
          memory: 1Gi

The resource is combined with a simple controller that can read zVPA resources and generate/update VPA resources. If a referenced schedule is active it will use the resourcePolicy of the respective schedule, otherwise it will use the default resourcePolicy.

---

This PR showcases how to use this for Prometheus to scale based on central events.