A Kubernetes operator for Apache Flink, implemented in Java. See FLIP-212.
The operator is managed helm chart. To install run:
cd helm/flink-operator
helm install flink-operator .
In order to use the webhook for FlinkDeployment validation, you must install the cert-manager on the Kubernetes cluster:
kubectl apply -f https://github.com/jetstack/cert-manager/releases/download/v1.7.1/cert-manager.yaml
The webhook can be disabled during helm install by passing the --set webhook.create=false parameter or editing the values.yaml directly.
The operator supports watching a specific list of namespaces for FlinkDeployment resources. You can enable it by setting the --set watchNamespaces={flink-test} parameter.
The flink-operator will watch the CRD resources and submit a new Flink deployment once the CR is applied.
kubectl create -f examples/basic.yaml
kubectl delete -f examples/basic.yaml
OR
kubectl delete flinkdep {dep_name}
Get all the Flink deployments running in the K8s cluster
kubectl get flinkdep
Describe a specific Flink deployment to show the status(including job status, savepoint, etc.)
kubectl describe flinkdep {dep_name}
docker build . -t <repo>/flink-java-operator:latest
docker push <repo>flink-java-operator:latest
helm install flink-operator . --set image.repository=<repo> --set image.tag=latest
You can run or debug the FlinkOperator from your preferred IDE. The operator itself is accessing the deployed Flink clusters through the REST interface. When running locally the rest.port and rest.address Flink configuration parameters must be modified to a locally accessible value.
When using minikube tunnel the rest service is exposed on localhost:8081
> minikube tunnel
> kubectl get services
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
basic-session-example ClusterIP None <none> 6123/TCP,6124/TCP 14h
basic-session-example-rest LoadBalancer 10.96.36.250 127.0.0.1 8081:30572/TCP 14h
The operator picks up the default log and flink configurations from /opt/flink/conf. You can put the rest configuration parameters here:
cat /opt/flink/conf/flink-conf.yaml
rest.port: 8081
rest.address: localhost
GitHub Actions help you automate your software development workflows in the same place you store code and collaborate on pull requests and issues. You can write individual tasks, called actions, and combine them to create a custom workflow. Workflows are custom automated processes that you can set up in your repository to build, test, package, release, or deploy any code project on GitHub.
Considering the cost of running the builds, the stability, and the maintainability, flink-kubernetes-operator chose GitHub Actions and build the whole CI/CD solution on it. All the unit tests, integration tests, and the end-to-end tests will be triggered for each PR.
Note: Please make sure the CI passed before merging.