Skip to content

Commit

Permalink
Initial commit (#728)
Browse files Browse the repository at this point in the history
  • Loading branch information
Techassi authored Sep 21, 2023
1 parent 23386e5 commit 4da6a45
Show file tree
Hide file tree
Showing 4 changed files with 52 additions and 28 deletions.
17 changes: 9 additions & 8 deletions docs/modules/zookeeper/pages/getting_started/installation.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,15 @@ On this page you will install the Stackable ZooKeeper Operator.

There are 2 ways to run Stackable Operators

1. Using xref:stackablectl::index.adoc[] (recommended)

2. Using Helm
. Using xref:management:stackablectl:index.adoc[] (recommended)
. Using Helm

=== stackablectl

stackablectl is the command line tool to interact with Stackable operators and our recommended way to install Operators.
Follow the xref:stackablectl::installation.adoc[installation steps] for your platform.
`stackablectl` is the command line tool to interact with Stackable operators and our recommended way to install
Operators. Follow the xref:management:stackablectl:installation.adoc[installation steps] for your platform.

After you have installed stackablectl, use it to install the ZooKeeper Operator and its dependencies:
After you have installed `stackablectl`, use it to install the ZooKeeper Operator and its dependencies:

[source,bash]
----
Expand All @@ -27,7 +26,8 @@ The tool will show
[source]
include::example$getting_started/code/install_output.txt[]

TIP: Consult the xref:stackablectl::quickstart.adoc[] to learn more about how to use stackablectl. For example, you can use the `-k` flag to create a Kubernetes cluster with link:https://kind.sigs.k8s.io/[kind].
TIP: Consult the xref:management:stackablectl:quickstart.adoc[] to learn more about how to use `stackablectl`. For
example, you can use the `--cluster kind` flag to create a Kubernetes cluster with link:https://kind.sigs.k8s.io/[kind].

=== Helm

Expand All @@ -43,7 +43,8 @@ Then install the Stackable Operators:
include::example$getting_started/code/getting_started.sh[tag=helm-install-operators]
----

Helm will deploy the Operators in a Kubernetes Deployment and apply the CRDs for the ZooKeeper cluster. You are now ready to deploy Apache ZooKeeper in Kubernetes.
Helm will deploy the Operators in a Kubernetes Deployment and apply the CRDs for the ZooKeeper cluster. You are now
ready to deploy Apache ZooKeeper in Kubernetes.

== What's next

Expand Down
41 changes: 27 additions & 14 deletions docs/modules/zookeeper/pages/index.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,38 +2,51 @@
:description: The Stackable Operator for Apache ZooKeeper is a Kubernetes operator that can manage Apache ZooKeeper ensembles. Learn about its features, resources, dependencies and demos, and see the list of supported ZooKeeper versions.
:keywords: Stackable Operator, Hadoop, Apache ZooKeeper, Kubernetes, k8s, operator, engineer, big data, metadata, storage, cluster

The Stackable Operator for Apache ZooKeeper is a Kubernetes Operator for deploying and managing https://zookeeper.apache.org/[Apache ZooKeeper] ensembles.
Apache ZooKeeper is an open-source distributed coordination service that facilitates synchronization, configuration management and leader election in distributed systems.
ZooKeeper is often used for these tasks in the Apache Hadoop ecosystem. Within the Stackable Platform, the Stackable Operators for xref:hbase:index.adoc[Apache HBase], xref:hdfs:index.adoc[Apache Hadoop HDFS], xref:kafka:index.adoc[Apache Kafka] and xref:nifi:index.adoc[Apache NiFi] depend on the ZooKeeper Operator.
The Stackable Operator for Apache ZooKeeper is a Kubernetes Operator for deploying and managing
https://zookeeper.apache.org/[Apache ZooKeeper] ensembles. Apache ZooKeeper is an open-source distributed coordination
service that facilitates synchronization, configuration management and leader election in distributed systems. ZooKeeper
is often used for these tasks in the Apache Hadoop ecosystem. Within the Stackable Platform, the Stackable Operators for
xref:hbase:index.adoc[Apache HBase], xref:hdfs:index.adoc[Apache Hadoop HDFS], xref:kafka:index.adoc[Apache Kafka] and
xref:nifi:index.adoc[Apache NiFi] depend on the ZooKeeper Operator.

== Getting started

Get started with Apache ZooKeeper and the Stackable Operator by following the xref:getting_started/index.adoc[Getting started] guide, it will guide you through the xref:getting_started/installation.adoc[installation] process. Afterwards, consult the xref:usage_guide/index.adoc[Usage guide] to learn more about configuring ZooKeeper for your needs. You can also deploy a <<demos, demo>> to see an example deployment of ZooKeeper together with other data products.
Get started with Apache ZooKeeper and the Stackable Operator by following the xref:getting_started/index.adoc[Getting
started] guide, it will guide you through the xref:getting_started/installation.adoc[installation] process. Afterwards,
consult the xref:usage_guide/index.adoc[Usage guide] to learn more about configuring ZooKeeper for your needs. You can
also deploy a <<demos, demo>> to see an example deployment of ZooKeeper together with other data products.

== Operator model

The Operator manages two custom resources: _ZookeeperCluster_ and _ZookeeperZnode_. ZooKeeper only has a single process that it runs, so the cluster resource only has a single corresponding xref:concepts:roles-and-role-groups.adoc[role] called _server_.
The Operator manages two custom resources: _ZookeeperCluster_ and _ZookeeperZnode_. ZooKeeper only has a single process
that it runs, so the cluster resource only has a single corresponding xref:concepts:roles-and-role-groups.adoc[role]
called _server_.

image::zookeeper_overview.drawio.svg[A diagram depicting the Kubernetes resources created by the Stackable Operator for Apache ZooKeeper]

For every role group the Operator creates a ConfigMap and StatefulSet which can have multiple replicas (Pods). Every role group is accessible through its own Service, and there is a Service for the whole Cluster.
For every role group the Operator creates a ConfigMap and StatefulSet which can have multiple replicas (Pods). Every
role group is accessible through its own Service, and there is a Service for the whole Cluster.

The Operator creates a xref:concepts:service_discovery.adoc[service discovery ConfigMap] for the ZooKeeper instance, as well as for each ZNode. The discovery ConfigMaps contain information on how to connect to ZooKeeper. The ZNode discovery ConfigMap give access information for the ZNode. xref:zookeeper:znodes.adoc[learn more about ZNodes].
The Operator creates a xref:concepts:service_discovery.adoc[service discovery ConfigMap] for the ZooKeeper instance, as
well as for each ZNode. The discovery ConfigMaps contain information on how to connect to ZooKeeper. The ZNode discovery
ConfigMap give access information for the ZNode. xref:zookeeper:znodes.adoc[learn more about ZNodes].

== Dependencies

Apache ZooKeeper and the Stackable Operator have no dependencies besides the xref:commons-operator:index.adoc[] and xref:secret-operator:index.adoc[].

== [[demos]]Demos

Apache ZooKeeper is a dependency of xref:hbase:index.adoc[Apache HBase], xref:hdfs:index.adoc[Apache Hadoop HDFS], xref:kafka:index.adoc[Apache Kafka] and xref:nifi:index.adoc[Apache NiFi], thus any demo that uses one or more of these components will also deploy a ZooKeeper ensemble. Here is the list of the demos that include ZooKeeper:
Apache ZooKeeper is a dependency of xref:hbase:index.adoc[Apache HBase], xref:hdfs:index.adoc[Apache Hadoop HDFS],
xref:kafka:index.adoc[Apache Kafka] and xref:nifi:index.adoc[Apache NiFi], thus any demo that uses one or more of these
components will also deploy a ZooKeeper ensemble. Here is the list of the demos that include ZooKeeper:

* xref:stackablectl::demos/data-lakehouse-iceberg-trino-spark.adoc[]
* xref:stackablectl::demos/hbase-hdfs-load-cycling-data.adoc[]
* xref:stackablectl::demos/jupyterhub-pyspark-hdfs-anomaly-detection-taxi-data.adoc[]
* xref:stackablectl::demos/logging.adoc[]
* xref:stackablectl::demos/nifi-kafka-druid-earthquake-data.adoc[]
* xref:stackablectl::demos/nifi-kafka-druid-water-level-data.adoc[]
* xref:demos:data-lakehouse-iceberg-trino-spark.adoc[]
* xref:demos:hbase-hdfs-load-cycling-data.adoc[]
* xref:demos:jupyterhub-pyspark-hdfs-anomaly-detection-taxi-data.adoc[]
* xref:demos:logging.adoc[]
* xref:demos:nifi-kafka-druid-earthquake-data.adoc[]
* xref:demos:nifi-kafka-druid-water-level-data.adoc[]

== Supported Versions

Expand Down
16 changes: 12 additions & 4 deletions docs/modules/zookeeper/pages/usage_guide/authentication.adoc
Original file line number Diff line number Diff line change
@@ -1,8 +1,12 @@
= Authentication

The communication between nodes (server to server) is encrypted TLS by default. In order to enforce TLS authentication for client-to-server communication, you can set an `AuthenticationClass` reference in the custom resource provided by the xref:commons-operator::index.adoc[Commons Operator].
The communication between nodes (server to server) is encrypted TLS by default. In order to enforce TLS authentication
for client-to-server communication, you can set an `AuthenticationClass` reference in the custom resource provided by
the xref:commons-operator:index.adoc[Commons Operator].

Currently it is possible to configure a single form of authentication (of type TLS) by adding one (and only one) entry in the `authentication` sequence as shown in the example below. Additional authentication methods, such as Kerberos, are not yet supported.
Currently it is possible to configure a single form of authentication (of type TLS) by adding one (and only one) entry
in the `authentication` sequence as shown in the example below. Additional authentication methods, such as Kerberos, are
not yet supported.

[source,yaml]
----
Expand All @@ -15,6 +19,10 @@ include::example$usage_guide/example-cluster-tls-authentication-secret.yaml[]
<3> The reference to a `SecretClass`.
<4> The `SecretClass` that is referenced by the `AuthenticationClass` in order to provide certificates.

If both `spec.clusterConfig.tls.server.secretClass` and `spec.clusterConfig.authentication.authenticationClass` are set, the authentication class will take precedence over the secret class. The cluster will be encrypted and authenticate only against the authentication class.
If both `spec.clusterConfig.tls.server.secretClass` and `spec.clusterConfig.authentication.authenticationClass` are set,
the authentication class will take precedence over the secret class. The cluster will be encrypted and authenticate only
against the authentication class.

WARNING: Due to a https://issues.apache.org/jira/browse/ZOOKEEPER-4276[bug] in ZooKeeper, the `clientPort` property in combination with `client.portUnification=true` is used instead of the `secureClientPort`. This means that unencrypted and unauthenticated access to the ZooKeeper cluster is still possible.
WARNING: Due to a https://issues.apache.org/jira/browse/ZOOKEEPER-4276[bug] in ZooKeeper, the `clientPort` property in
combination with `client.portUnification=true` is used instead of the `secureClientPort`. This means that unencrypted
and unauthenticated access to the ZooKeeper cluster is still possible.
6 changes: 4 additions & 2 deletions docs/modules/zookeeper/pages/usage_guide/encryption.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
= Encryption

The quorum and client communication are encrypted by default via TLS. This requires the xref:secret-operator::index.adoc[Secret Operator] to be present in order to provide certificates. The utilized certificates can be changed in a top-level config.
The quorum and client communication are encrypted by default via TLS. This requires the
xref:secret-operator:index.adoc[Secret Operator] to be present in order to provide certificates. The utilized
certificates can be changed in a top-level config.

[source,yaml]
----
Expand All @@ -9,7 +11,7 @@ include::example$usage_guide/example-cluster-tls-encryption.yaml[]
<1> The `tls.server.secretClass` refers to the client-to-server encryption. Defaults to the `tls` secret.
<2> The `tls.quorum.secretClass` refers to the server-to-server quorum encryption. Defaults to the `tls` secret.

The `tls` secret is deployed from the xref:secret-operator::index.adoc[Secret Operator] and looks like this:
The `tls` secret is deployed from the xref:secret-operator:index.adoc[Secret Operator] and looks like this:

[source,yaml]
----
Expand Down

0 comments on commit 4da6a45

Please sign in to comment.