The helm chart provided in this repo allows you to deploy tools from the peng-data-minimization org on Kubernetes.
Currently, we provide charts for the following tools:
Add the helm repo:
$ helm repo add dm-helm-charts https://peng-data-minimization.github.io/helm-charts
Install the SPI worker:
$ helm install dm-spi dm-helm-charts/spi -f PATH_TO_YOUR_SPI_CONFIG.yaml
Install the complete data minimization pipeline with the SPI worker, a Confluent Kafka stack, Elasticsearch, Kibana for visualizations and Prometheus and Grafana for monitoring:
$ helm install dm-pipeline dm-helm-charts/data-minimization-pipeline -f PATH_TO_YOUR_SPI_CONFIG.yaml
Note that when installing the complete pipeline the worker SPI config yaml has to be prefixed with a top-level spi:
key.
Run kubectl get pods
to make sure that pods have been started correctly.
To deploy the pipeline without any spi worker, just remove any spi worker tasks in the values file (set spi.tasks: []
).
For more information on the configuration, refer to the SPI chart documentation or the kafka-spi repository.
To install the SPI worker chart from source run:
$ helm install spi subcharts/spi/ -f PATH_TO_YOUR_SPI_CONFIG.yaml
To install the data minimization pipeline chart from source run:
$ helm dependency update .
$ helm install dm-pipeline . -f PATH_TO_YOUR_SPI_CONFIG.yaml
To generate a README.md
documentation for the chart's values.yml
run:
$ helm-docs