Skip to content

Commit

Permalink
Adding descriptions (#662)
Browse files Browse the repository at this point in the history
* first batch of descriptions

* more descriptions

* pre-commit fixes

* more descriptions

* linter fixes

* linter fixes

* more descriptions

* more descriptions

* linter fixes

* Add description bit to style guide

* Add more descriptions

* Add more descriptions

* Remove trailing whitespace

* Update modules/ROOT/pages/licenses.adoc

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

* Update modules/reference/pages/glossary.adoc

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

* Update modules/reference/pages/duration.adoc

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

* Update modules/guides/pages/viewing-and-verifying-sboms.adoc

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

* Update modules/concepts/pages/s3.adoc

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

* Add info about description usage

* Update modules/concepts/pages/tls-server-verification.adoc

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

* Update modules/concepts/pages/stacklet.adoc

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>

---------

Co-authored-by: Nick <10092581+NickLarsenNZ@users.noreply.github.com>
  • Loading branch information
fhennig and NickLarsenNZ authored Sep 19, 2024
1 parent b876795 commit 413528d
Show file tree
Hide file tree
Showing 42 changed files with 67 additions and 26 deletions.
1 change: 1 addition & 0 deletions modules/ROOT/pages/export.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Export Control
:description: Stackable Data Platform is exempt from US EAR export controls due to its publicly available status and use of standard encryption. Code is open source on GitHub.

== USA

Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/pages/getting-started.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Getting Started
:page-aliases: getting_started.adoc
:description: Learn how to set up and test a Stackable Data Platform with Apache ZooKeeper, Kafka, and NiFi using Kubernetes. Ideal for lab environments.

One of the best ways of getting started with a new platform is to try it out. Any big data platform has a lot of moving parts and getting some hands on keyboard time with it helps reinforce learning.

Expand Down
4 changes: 2 additions & 2 deletions modules/ROOT/pages/index.adoc
Original file line number Diff line number Diff line change
@@ -1,10 +1,10 @@
= Stackable Documentation
:page-layout: landing

:k8s-operators: https://kubernetes.io/docs/concepts/extend-kubernetes/operator/
:docs-discussion: https://github.com/stackabletech/community/discussions
:docs-issues: https://github.com/stackabletech/documentation/issues
:docs-repo: https://github.com/stackabletech/documentation
:description: User Documentation for the Stackable Data Platform.

++++
<style>
Expand Down Expand Up @@ -82,7 +82,7 @@ Run a demo data processing pipeline or follow a tutorial to learn about the plat
<h3>Concept explanation</h3>
++++

Read concept explanation to gain a better understanding of how the Stackable Data platform works,
Read concept explanation to gain a better understanding of how the Stackable Data platform works,
or how a specific feature works.

xref:concepts:index.adoc[]
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/pages/kubernetes/index.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Kubernetes
:description: Learn how to install Kubernetes for the Stackable Data Platform. Supported production distributions and local setup options with Kind and K3s.

The Stackable Data Platform runs on Kubernetes, a Kubernetes cluster is a prerequisite to running the platform.
On this page you will find information on the supported Kubernetes distributions for production as well as how to set up a local test installation to try out parts of the platform right away.
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/pages/licenses.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Licenses for the Stackable Data Platform
:description: Find licenses for all Stackable Data Platform components available on GitHub including operators, stackablectl, and Docker images.

The Stackable Data Platform is open source, and the source code of all the components can be found on GitHub. Licenses are also provided alongside the source code, in a file called `LICENSE`.

Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/pages/policies.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Lifecycle policies
:description: Detailed lifecycle policies for Stackable Data Platform, covering SDP, CRD versioning, product support, and compatibility with Kubernetes & OpenShift.

This page details our lifecycle policies around various parts of our product - the Stackable Data Platform (SDP).

Expand Down
2 changes: 2 additions & 0 deletions modules/ROOT/pages/product-information.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,6 @@
= Product information
:description: Learn about concrete specifications of the Stackable Data Platform (SDP) as a product, which components are
included, how they are supplied and which external dependencies exist that you as a customer need to take care of.

This page contains concrete specifications about the Stackable Data Platform (SDP) as a product, which components are
included, how they are supplied and which external dependencies exist that you as a customer need to take care of.
Expand Down
2 changes: 1 addition & 1 deletion modules/ROOT/pages/quickstart.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Quickstart

:latest-release: https://github.com/stackabletech/stackable-cockpit/releases/tag/stackablectl-1.0.0-rc2
:cockpit-releases: https://github.com/stackabletech/stackable-cockpit/releases
:description: Quickstart guide for Stackable: Install stackablectl, set up a demo, and connect to services like Superset and Trino with easy commands and links.

This is the super-short getting started guide that should enable you to get something up and running in less than three
minutes (excluding download times).
Expand Down
1 change: 1 addition & 0 deletions modules/ROOT/pages/release-notes.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Release notes for the Stackable Data Platform
:page-aliases: release_notes.adoc
:description: Learn about the latest features and changes in the release notes for the Stackable Data Platform.

The Stackable platform consists of multiple operators that work together. Periodically a platform release is made,
including all components of the platform at a specific version.
Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/authentication.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Authentication
:keycloak: https://www.keycloak.org/
:description: Overview of Stackable’s user authentication methods and configuration for LDAP, OIDC, TLS, and Static providers.

The Stackable Platform uses the AuthenticationClass as a central mechanism to handle user authentication across supported products.
The authentication mechanism needs to be configured only in the AuthenticationClass which is then referenced in the xref:stacklet.adoc[Stacklet] definition.
Expand Down
3 changes: 2 additions & 1 deletion modules/concepts/pages/container-images.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
:ubi: https://catalog.redhat.com/software/base-images
:stackable-image-registry: https://repo.stackable.tech/#browse/browse
:stackable-sboms: https://sboms.stackable.tech/
:description: Overview of Stackable’s container images, including structure, multi-platform support, and why upstream images are not used.

The core artifacts of the Stackable Data Platform are container images of Kubernetes operators and the products that these operators deploy.

Expand Down Expand Up @@ -37,7 +38,7 @@ See our documentation xref:concepts:multi-platform-support.adoc[on multi-platfor
[#signatures]
== Image signatures

All Stackable images are signed.
All Stackable images are signed.
Image signatures help to ensure the authenticity and integrity of container images.
You can xref:guides:enabling-verification-of-image-signatures.adoc[verify image signatures automatically] in your cluster to make sure that the images you are running are authentic and intact.

Expand Down
3 changes: 2 additions & 1 deletion modules/concepts/pages/index.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Concepts
:description: Explore Stackable Data Platform concepts like Stacklets, configuration mechanisms, resources, connectivity, security, and observability.

== Overview

Expand All @@ -17,7 +18,7 @@ Learn about how to access xref:experimental-arm64-support[ARM64-support].

== Connectivity

Many Platform components depend on other components or expose functionality that you can connect to.
Many Platform components depend on other components or expose functionality that you can connect to.
This connectivity is achived with xref:service-discovery.adoc[service discovery ConfigMaps].
To access your Stackable operated products from outside the Kuberenetes cluster learn more about xref:service-exposition.adoc[].

Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/labels.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Labels
:common-labels: https://kubernetes.io/docs/concepts/overview/working-with-objects/common-labels/
:description: Understand resource labeling by Stackable operators and tools like stackablectl and Helm for better management and organization.

Labels are key/value pairs in the metadata of Kubernetes objects that add identifying information to the object.
They do not have direct semantic implications but can be used to organize and manage resources.
Expand Down
3 changes: 2 additions & 1 deletion modules/concepts/pages/logging.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
:vector-agg-install: https://vector.dev/docs/setup/installation/package-managers/helm/#aggregator
:vector-source-vector: https://vector.dev/docs/reference/configuration/sources/vector/
:vector-topology-centralized: https://vector.dev/docs/setup/deployment/topologies/#centralized
:description: Learn Stackable's logging setup: human-readable and aggregated logs with Vector, consistent configuration across Stacklets, and custom overrides.

Logging is important for observability of the platform.
Stackable provides human-readable plaintext logs for each running container, as well as aggregated and persisted logs with identical structure across the whole platform.
Expand Down Expand Up @@ -93,7 +94,7 @@ spec:
level: NONE
sidecar-container:
console:
level: ERROR
level: ERROR
----
<1> The discovery ConfigMap of the Vector aggregator to publish the logs to. This is set at cluster level.
<2> The role or role group config containing the logging configuration.
Expand Down
8 changes: 3 additions & 5 deletions modules/concepts/pages/multi-platform-support.adoc
Original file line number Diff line number Diff line change
@@ -1,16 +1,14 @@
= Multi-platform for SDP
:description: This page describes multi-platform support for AMD64 and ARM64
:keywords: Multi-Architecture, infrastructure, docker, image, tags
:multi-platform-images: https://docs.docker.com/build/building/multi-platform/
:description: Starting with Stackable 24.7, all images support AMD64 and ARM64 architectures, enabling mixed clusters with architecture-specific tags and automated selection.

WARNING: This status is still experimental, as we work to fine-tune the necessary workflows.

== Description

Starting with Stackable release 24.7, all images are {multi-platform-images}[multi-platform images], supporting the AMD64 and ARM64 architectures.
Starting with the Stackable Data Platform release 24.7, all images are {multi-platform-images}[multi-platform images], supporting the AMD64 and ARM64 architectures.
Each product image is built for each platform with an architecture-specific tag.
For example, the Airflow images with tags `airflow:2.9.2-stackable24.7.0-amd64` and `airflow:2.9.2-stackable24.7.0-arm64` are bundled in the manifest list `airflow:2.9.2-stackable24.7.0` using an automated workflow.
The appropriate image will then be transparently selected for the active platform/architecture.
This also enables mixed Kubernetes clusters, with AMD64 and ARM64 nodes being used simultaneously.

NOTE: Superset 2.x (deprecated, and to be removed in a future version) is not available as a multi-platform product image.
NOTE: Superset 2.x (deprecated, and to be removed in a future version) is not available as a multi-platform product image.
1 change: 1 addition & 0 deletions modules/concepts/pages/opa.adoc
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,7 @@
:opa: https://www.openpolicyagent.org
:rego: https://www.openpolicyagent.org/docs/latest/policy-language/
:opa-docs: https://www.openpolicyagent.org/docs/latest/#overview
:description: Stackable Data Platform uses OpenPolicyAgent (OPA) for policy-based access control with Rego rules, ensuring efficient, local policy evaluation across nodes.

The Stackable Data Platform offers policy-based access control via the {opa}[OpenPolicyAgent] (OPA) operator.
Authorization policies are defined in the {rego}[Rego] language, divided into packages and supplied via ConfigMaps.
Expand Down
3 changes: 2 additions & 1 deletion modules/concepts/pages/operations/cluster_operations.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Cluster operations
:page-aliases: ../cluster_operations.adoc
:description: Manage Stackable clusters with operations like pausing reconciliation, stopping Pods, and manual/automatic restarts. Learn how to update and restart services effectively.

Stackable operators offer different cluster operations to control the reconciliation process.
This is useful when updating operators, debugging or testing of new settings:
Expand Down Expand Up @@ -97,7 +98,7 @@ To restart the HDFS DataNode Pods, run:

[source,shell]
----
$ kubectl rollout restart statefulset dumbo-datanode-default
$ kubectl rollout restart statefulset dumbo-datanode-default
statefulset.apps/dumbo-datanode-default restarted
----

Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/operations/graceful_shutdown.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Graceful shutdown
:description: Graceful shutdown ensures a controlled, clean shutdown of service instances, allowing time for normal shutdown activities.

Graceful shutdown refers to the managed, controlled shutdown of service instances in the manner intended by the software authors.
Typically, an instance will receive a signal indicating the intent for the server to shut down, and it will initiate a controlled shutdown.
Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/operations/index.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Operations
:description: Guidance for maintaining Stackable Data Platform installations. Covers service availability, maintenance actions, and performance optimization.

This section of the documentation is intended for the operations teams that maintain a Stackable Data Platform installation.
It provides you with the necessary details to operate it in a production environment.
Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/operations/pod_disruptions.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Allowed Pod disruptions
:k8s-pdb: https://kubernetes.io/docs/tasks/run-application/configure-pdb/
:description: Configure PodDisruptionBudgets (PDBs) to minimize planned downtime for Stackable products. Default values are based on fault tolerance and can be customized.

Any downtime of our products is generally considered to be bad.
Although downtime can't be prevented 100% of the time - especially if the product does not support High Availability - we can try to do our best to reduce it to an absolute minimum.
Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/operations/pod_placement.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Pod placement
:page-aliases: ../pod_placement.adoc
:description: Configure pod affinity, anti-affinity, and node affinity for Stackable Data Platform operators using YAML definitions.

Several operators of the Stackable Data Platform permit the configuration of pod affinity as described in the Kubernetes https://kubernetes.io/docs/concepts/scheduling-eviction/assign-pod-node/[documentation].
If no affinity is defined in the product's custom resource, the operators apply reasonable defaults that make use of the `preferred_during_scheduling_ignored_during_execution` property.
Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/overrides.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Overrides
:k8s-openapi-deepmerge: https://arnavion.github.io/k8s-openapi/v0.22.x/k8s_openapi/trait.DeepMerge.html
:description: Stackable operators support configuration overrides for Stacklets, including settings for config files, environment variables, and Pod specifications, with precedence rules.

The Stackable operators configure the xref:stacklet.adoc[Stacklets] they are operating with sensible defaults and required settings to enable connectivity and security.
Other important settings are usually exposed in the Stacklet resource definition for you to configure directly.
Expand Down
3 changes: 2 additions & 1 deletion modules/concepts/pages/overview.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Stackable Data Platform explained
:description: The Stackable Data Platform leverages Kubernetes operators to manage products like ZooKeeper and HDFS, using custom resources for deployment and configuration.

The Stackable Data Platform (SDP) is built on Kubernetes.
Its core is a collection of Kubernetes Operators and CustomResourceDefinitions which are designed to work together.
Expand Down Expand Up @@ -76,4 +77,4 @@ image::common_objects.drawio.svg[]

These objects can be reused by all operators that support this feature. The S3 bucket only needs to be described once, and then it can be referenced in all products that support reading and/or writing from/to S3. Learn more about S3 configuration: xref:s3.adoc[].

Similarly for the OpenPolicyAgent (OPA). Configuring it looks the same across all products. Learn more: xref:opa.adoc[].
Similarly for the OpenPolicyAgent (OPA). Configuring it looks the same across all products. Learn more: xref:opa.adoc[].
2 changes: 1 addition & 1 deletion modules/concepts/pages/product-image-selection.adoc
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
= Product image selection
:page-aliases: product_image_selection.adoc
:description: This page describes the different ways of specifying a product image to use in your product deployment.
:keywords: Kubernetes, operator, docker, registry, custom image, tags
:description: Learn how to specify product images for Stackable Data Platform deployments, including using default, custom, or mirrored registries.

To run any product on the Stackable Data Platform, you need to specify which version you want to run in the resource definition (i.e. a SparkApplication or DruidCluster).
The simplest way to do this is this:
Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/resources.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= Resource management
:description: Learn how to manage CPU, memory, and storage resources for Stackable Data Platform products, including setting requests, limits, and StorageClasses.

The Stackable Data Platform and its xref:operators:index.adoc[operators] deploy their products in https://kubernetes.io/docs/concepts/containers/[containers] within https://kubernetes.io/docs/concepts/workloads/pods/[Pods] using https://kubernetes.io/docs/concepts/workloads/controllers/statefulset/[StatefulSets] or https://kubernetes.io/docs/concepts/workloads/controllers/daemonset/[DaemonSets].
In order for the Kubernetes scheduler to select a suitable https://kubernetes.io/docs/concepts/architecture/nodes/[Node] for a Pod, https://kubernetes.io/docs/concepts/configuration/manage-resources-containers/[resource] requests and limits for CPU and memory can be specified.
Expand Down
3 changes: 2 additions & 1 deletion modules/concepts/pages/s3.adoc
Original file line number Diff line number Diff line change
@@ -1,4 +1,5 @@
= S3 resources
:description: Explore how to configure S3 storage with Stackable using S3Connection and S3Bucket objects, including setup examples with credentials and TLS configuration.

// -------------- Intro ----------------

Expand Down Expand Up @@ -48,7 +49,7 @@ image::s3-overview.drawio.svg[A diagram showing four variations (A, B, C, D) of
The diagram above shows four examples of how the objects can be structured.

// Variant A
Variant A shows all S3 objects inlined in a DruidCluster resource. This is a very convenient way to quickly test something since the entire configuration is encapsulated in a single but potentially large manifest.
Variant A shows all S3 objects inlined in a DruidCluster resource. This is a very convenient way to quickly test something since the entire configuration is encapsulated in a single but potentially large manifest.

// Variant B
In variant B the S3 bucket has been split out into its own resource. It can now be referred to by multiple different tools as well.
Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/service-discovery.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Service discovery ConfigMap
:page-aliases: service_discovery.adoc
:description: Learn how Stackable's service discovery ConfigMap enables product connections, providing essential instance info for integration and external connections.

// Abstract
Stackable operators provide a _service discovery ConfigMap_ for each product instance that is deployed. **This ConfigMap has the same name as the product instance** and contains information about how to connect to the instance. The ConfigMap is used by other Operators to connect products together and can also be used by you, the user, to connect external software to Stackable-operated software.
Expand Down
2 changes: 2 additions & 0 deletions modules/concepts/pages/service-exposition.adoc
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
= Service exposition
:k8s-service: https://kubernetes.io/docs/concepts/services-networking/service/
:k8s-service-types: https://kubernetes.io/docs/concepts/services-networking/service/#publishing-services-service-types
:description: Explore Stackable's service exposition options: ClusterIP for internal access, NodePort for unstable external access, and LoadBalancer for stable external access.


Data products expose interfaces to the outside world.
These interfaces (whether UIs, or APIs) can be accessed by other products or by end users.
Expand Down
3 changes: 2 additions & 1 deletion modules/concepts/pages/stacklet.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= Stacklet
:page-aliases: roles-and-role-groups.adoc
:description: A Stacklet is a managed product in Kubernetes, consisting of roles and role groups. Roles allow general configuration of components, while role groups allow finer-grained control over specific groups of components.

A _Stacklet_ is a deployed product that is managed by a Stackable operator.
The running instance is made up of multiple pieces of software called _roles_ and can be further subdivided into _role groups_.
Expand All @@ -13,7 +14,7 @@ The products are usually deployed with StatefulSets from which Pods are created.
Configuration is done with ConfigMaps, and the the software is exposed using Services.
To allow for easier connectivity between Stacklets, some operators also deploy a xref:service-discovery.adoc[] for Stacklets they manage.

CAUTION: Watch out for name collisions when deploying multiple Stacklets in the same namespace!
CAUTION: Watch out for name collisions when deploying multiple Stacklets in the same namespace!
Even though the resource might be different (TrinoCluster, HbaseCluster), there is a name conflict for the discovery ConfigMap if two Stacklets in the same Kubernetes namespace share a name.
It is best to always use unique names for Stacklets.

Expand Down
1 change: 1 addition & 0 deletions modules/concepts/pages/tls-server-verification.adoc
Original file line number Diff line number Diff line change
@@ -1,5 +1,6 @@
= TLS server verification
:page-aliases: tls_server_verification.adoc
:description: Configure TLS server and mutual authentication with custom or public CA certificates.

A TLS section is part of Stackable CRDs and describes how to connect to a TLS enabled system like LDAP or S3.

Expand Down
Loading

0 comments on commit 413528d

Please sign in to comment.