Merge pull request #155 from qonto/add-grafana-crd

Add Grafana dashboard CRD

README.md

@@ -18,7 +18,7 @@ It collects key metrics about:
 
 ## Key metrics
 
-📊 Advanced Metrics: Gain deep visibility with advanced metrics for AWS RDS. Monitor performance, query efficiency, and resource utilization like never before.
+🥇 Advanced Metrics: Gain deep visibility with advanced metrics for AWS RDS. Monitor performance, query efficiency, and resource utilization like never before.
 
 🧩 AWS Quotas Insights: Stay in control with real-time information about AWS quotas. Ensure you never hit limits unexpectedly.
 
@@ -28,7 +28,7 @@ It collects key metrics about:
 
 🛠️ Simple Setup: Getting started is a breeze! Our clear documentation and examples will have you up and running in no time.
 
-📈 Scalable and Reliable: Prometheus-RDS Exporter scales with your AWS infrastructure, providing reliable monitoring even as you grow.
+📊 Dashboards: Prometheus-RDS Exporter export adopts the [USE methodology](https://www.brendangregg.com/usemethod.html) and provides well-designed, ready-to-use dashboards.
 
 🌐 Community-Driven: Join a vibrant community of users and contributors. Collaborate, share knowledge, and shape the future of AWS RDS monitoring together.
 
@@ -119,7 +119,22 @@ It collects key metrics about:
 
 ## Dashboards
 
-The following Grafana dashboards are available in `configs/grafana/public/` and Grafana labs:
+> [!TIP]
+> Grafana dashboards are deployed by default as [GrafanaDashboard CRD](https://grafana.github.io/grafana-operator/docs/dashboards/) when Prometheus RDS exporter is deployed with Helm. If you deployed [Grafana operator](https://grafana.github.io/grafana-operator/) in your Kubernetes cluster, dashboards will be automatically imported and **maintained up-to-date**.
+
+<details>
+  <summary>Why are we recommending Grafana operator?</summary>
+
+We are committed to providing you with the most efficient and user-friendly experience possible. Therefore, we continuously enhance our dashboards and the metrics produced by our exporters to ensure you have access to the most accurate and relevant data.
+
+To ensure an optimal user experience, it's vital to keep your dashboards up to date. This practice guarantees that you are always working with the latest features and improvements, enabling you to make the most out of the data presented to you. However, maintaining multiple versions of dashboards can be challenging and is not desirable. It introduces complexity and can lead to inconsistencies between what you see and the actual data.
+
+By leveraging the Grafana Operator, you can rest assured that the version of your dashboard will always match the metrics presented by your exporter. This synchronization between your dashboards and the underlying data ensures a seamless and accurate monitoring experience. This move towards operator-based deployment is designed to streamline your monitoring process, ensuring accuracy and efficiency in your data visualization efforts.
+
+Kubernetes operators aim to simplify deployments, and as part of this evolution, we will eventually stop publishing dashboards on Grafana Labs.
+</details>
+
+For convenience, dashboards are also available in `configs/grafana/public/` folder and Grafana labs:
 
 <table>
 <tr>
@@ -260,6 +275,12 @@ Terraform users can take example on Terraform code in `configs/terraform/`.
 
 </details>
 
+### Helm
+
+We recommend deployment using helm [Helm](https://helm.sh/).
+
+See all available configuration parameters in [configs/helm/values.yaml](https://github.com/qonto/prometheus-rds-exporter/blob/main/configs/helm/values.yaml)
+
 ## Installation
 
 See the [Development environment](#development-environment) to start the Prometheus RDS exporter, Prometheus, and Grafana with dashboards in a minute.

configs/helm/grafana_dashboards

@@ -0,0 +1 @@
+../grafana/public

configs/helm/templates/grafanadashboard.yaml

@@ -0,0 +1,20 @@
+{{- if .Values.dashboards.enabled -}}
+{{ $currentScope := .}}
+{{ range $path, $_ :=  .Files.Glob  "grafana_dashboards/**.json" }}
+{{- with $currentScope}}
+---
+apiVersion: grafana.integreatly.org/v1beta1
+kind: GrafanaDashboard
+metadata:
+  name: {{ $path | base | trimSuffix ".json" }}
+spec:
+  resyncPeriod: {{ .Values.dashboards.resyncPeriod }}
+  folder: {{ .Values.dashboards.folderName }}
+  instanceSelector:
+    matchLabels:
+      dashboards: {{ .Values.dashboards.instanceSelector }}
+  json: |
+{{ .Files.Get $path | indent 4 }}
+{{- end }}
+{{- end }}
+{{- end }}

configs/helm/tests/grafanadashboard_test.yaml

@@ -0,0 +1,52 @@
+# yaml-language-server: $schema=https://raw.githubusercontent.com/helm-unittest/helm-unittest/main/schema/helm-testsuite.json
+---
+suite: grafana dashboard tests
+templates:
+  - grafanadashboard.yaml
+tests:
+  - it: render default Grafana dashboards
+    asserts:
+      - isKind:
+          of: GrafanaDashboard
+      - hasDocuments:
+          count: 3
+      - documentIndex: 0
+        equal:
+          path: metadata.name
+          value: prometheus-rds-exporter
+      - documentIndex: 1
+        equal:
+          path: metadata.name
+          value: rds-instance
+      - documentIndex: 2
+        equal:
+          path: metadata.name
+          value: rds-instances
+      - equal:
+          path: spec.resyncPeriod
+          value: 24h
+      - equal:
+          path: spec.folder
+          value:
+      - equal:
+          path: spec.instanceSelector.matchLabels.dashboards
+          value: grafana
+  - it: render custom Grafana dashboards settings
+    values:
+      - ./values/with_grafanadashboards.yaml
+    asserts:
+      - equal:
+          path: spec.resyncPeriod
+          value: 1m
+      - equal:
+          path: spec.folder
+          value: dmf
+      - equal:
+          path: spec.instanceSelector.matchLabels.dashboards
+          value: my-grafana
+  - it: disable Grafana dashboards
+    values:
+      - ./values/without_grafanadashboards.yaml
+    asserts:
+      - hasDocuments:
+          count: 0

configs/helm/tests/servicemonitor_test.yaml

@@ -4,7 +4,7 @@ suite: service monitor tests
 templates:
   - servicemonitor.yaml
 tests:
-  - it: render defaul service monitor
+  - it: render default service monitor
     asserts:
       - isKind:
           of: ServiceMonitor

configs/helm/tests/values/with_grafanadashboards.yaml

@@ -0,0 +1,5 @@
+---
+dashboards:
+  resyncPeriod: 1m
+  instanceSelector: my-grafana
+  folderName: dmf

configs/helm/tests/values/without_grafanadashboards.yaml

@@ -0,0 +1,3 @@
+---
+dashboards:
+  enabled: false

configs/helm/values.yaml

@@ -84,3 +84,9 @@ nodeSelector: {}
 tolerations: []
 
 affinity: {}
+
+dashboards:
+  enabled: true  # enabled GrafanaDashboard CRD import
+  resyncPeriod: 24h  # how often the dashboard is refreshed
+  instanceSelector: grafana  # selects Grafana for import
+  folderName:  # folder assignment for dashboard

scripts/kubeconform-test.sh

@@ -52,7 +52,7 @@ do
 		-kubernetes-version ${KUBERNETES_VERSION} \
 		-cache ${KUBECONFORM_CACHE_DIRECTORY} \
 		-schema-location default \
-		-schema-location 'kubeconform/{{ .ResourceKind }}{{ .KindSuffix }}.json' \
+		-schema-location 'scripts/kubeconform/{{.Group}}/{{ .ResourceKind }}_{{.ResourceAPIVersion}}.json' \
 		-schema-location 'https://raw.githubusercontent.com/datreeio/CRDs-catalog/main/{{.Group}}/{{.ResourceKind}}_{{.ResourceAPIVersion}}.json' \
 		-summary
 

scripts/kubeconform/grafana.integreatly.org/README.md

@@ -0,0 +1,3 @@
+# Note for later
+
+Grafana dashboards CRD is required until <https://github.com/datreeio/CRDs-catalog/pull/285> is merged