Added nifi helm charts #336
Conversation
Hsankesara
force-pushed
the
nifi-integration
branch
from
49b11de to
af02efb
Compare
|
Not sure, if I should I change the base to release-1.2 #333 or not? |
environments.yaml.tmpl
Outdated
| @@ -32,3 +32,5 @@ repositories: | |||
| url: https://radar-base.github.io/radar-helm-charts | |||
| - name: cp-radar | |||
| url: https://radar-base.github.io/cp-helm-charts | |||
| - name: cetic | |||
| url: https://cetic.github.io/helm-charts | |||
There was a problem hiding this comment.
Please mirror this chart in our radar-helm-charts external directory and add it from there
There was a problem hiding this comment.
Sure, I think I have opened a PR there -- RADAR-base/radar-helm-charts#302. I'll wait for it to get merged then.
| @@ -564,3 +564,18 @@ velero: | |||
| bucket: radar-base-backups | |||
| config: | |||
| s3Url: https://s3.amazon.com # protocol should be specified | |||
|
|
|||
| nifi: | |||
| _install: false | |||
There was a problem hiding this comment.
Please mention _chart_version and _extra_timeout values here
etc/base.yaml
Outdated
| auth: | ||
| singleUser: | ||
| username: username | ||
| password: password |
There was a problem hiding this comment.
The secrets should be in base-secrets.yaml and if should be distinguished if it needs to be auto generated or should be manual.
| path: /nifi | ||
| className: nginx | ||
| annotations: | ||
| nginx.ingress.kubernetes.io/backend-protocol: HTTPS |
There was a problem hiding this comment.
Please add the cert-manager annotation as exampled in here
https://github.com/RADAR-base/RADAR-Kubernetes/blob/dev/etc/graylog/values.yaml#L29
|
Please add the new component to this list |
The release-1.2 branch is on a feature freeze mode and trying to only do bug fixes on it. |
etc/cp-nifi/values.yaml
Outdated
| resources: | ||
| limits: | ||
| cpu: 4 | ||
| memory: 16Gi |
There was a problem hiding this comment.
No not necessary, I think I can remove it.
| @@ -175,3 +175,13 @@ velero: | |||
| [default] | |||
| aws_access_key_id=change_me | |||
| aws_secret_access_key=change_me | |||
|
|
|||
| nifi: | |||
There was a problem hiding this comment.
perhaps these secrets can be initialised in the bin/generate-secrets script (just add lines for specific values using insert_secret)
There was a problem hiding this comment.
The script will auto generate any secret that has the value of secret
Description of the change
Added support for NiFi deployment.
Benefits
NiFi can provide the user interface to create different ETL pipelines, which can be used to process and manipulate real-time data. The current version uses the cetic/helm-nifi but we can probably be going to move to radar-helm-charts/nifi in future.
Possible drawbacks
Though NiFi provides a UI-based ETL pipeline, it does consume a substantial amount of resources. Also, the cetic/helm-nifi is not being maintained anymore so we probably would need to maintain the helm charts in some capacity or find another alternative to keep up with the new updates.
Additional information
Checklist