Added README.md and publish workflow (#2)

Added README.md and publish workflow

.github/workflows/publish.yaml

@@ -0,0 +1,46 @@
+name: Publish
+
+on:
+  push:
+    tags:
+      - "v[0-9]+.[0-9]+.[0-9]+"
+
+env:
+  REGISTRY: ghcr.io
+  IMAGE_NAME: ${{ github.repository }}
+
+jobs:
+  test:
+    uses: ./.github/workflows/test.yaml
+  publish_docker_image:
+    name: Publish Docker Image to GitHub Packages
+    needs: test
+    runs-on: ubuntu-latest
+    permissions:
+      contents: read
+      packages: write
+    steps:
+      - name: Checkout
+        uses: actions/checkout@v4
+        with:
+          fetch-depth: 0
+      - name: Login to GHCR
+        uses: docker/login-action@v3
+        with:
+          registry: ${{ env.REGISTRY }}
+          username: ${{ github.actor }}
+          password: ${{ secrets.GITHUB_TOKEN }}
+      - name: Extract metadata for the Docker image
+        id: meta
+        uses: docker/metadata-action@v5
+        with:
+          tags: |
+            type=semver,pattern={{version}}
+            type=semver,pattern={{major}}.{{minor}}
+      - name: Build and push Docker image to GHCR
+        env:
+          IMG_REPO: ${{ env.IMAGE_NAME }}
+          IMG_TAG: ${{ join(fromJSON(steps.meta.outputs.json).tags, ' ') }}
+          PUSH_REPO: ${{ env.REGISTRY }}/${{ env.IMAGE_NAME }}
+        run: |
+          echo "yes" | make docker-build

.github/workflows/test.yaml

@@ -5,6 +5,7 @@ on:
   push:
     branches:
       - "main"
+  workflow_call:
 
 jobs:
   continuous:

README.md

@@ -1,2 +1,67 @@
-# nuodb-sidecar
-Additional NuoDB images for sidecar containers
+# NuoDB sidecar container image
+
+![Build status](https://github.com/nuodb/nuodb-sidecar/actions/workflows/test.yaml/badge.svg)
+
+The NuoDB sidecar container image bundles additional tools which help running NuoDB database in Kubernetes cluster.
+
+## Requirements
+
+| Software   | Release Requirements                           |
+|------------|------------------------------------------------|
+| Kubernetes |  The latest and previous minor released versions of Kubernetes. |
+| NuoDB      |  Version [6.0.2](https://hub.docker.com/r/nuodb/nuodb/tags) and onwards. |
+
+## Tools
+
+| Command          | Description                                            |
+|------------------|--------------------------------------------------------|
+| [config_watcher](./config_watcher/README.md) | Watches for changes in `ConfigMap` or `Secret` resources and creates files out of resource's `data` in a target directory. |
+| [nuodb-operations](./nuodb-operations/README.md) | Starts a server with NuoDB backup hooks and optionally custom operation handlers. |
+
+## Usage
+
+Follow the instructions on the [NuoDB Helm charts](https://github.com/nuodb/nuodb-helm-charts/blob/master/README.md#nuodb-helm-chart-installation) installation page.
+
+NuoDB sidecar is used in [Admin](https://github.com/nuodb/nuodb-helm-charts/tree/master/stable/admin) and [Database](https://github.com/nuodb/nuodb-helm-charts/tree/master/stable/database) charts when enabling NuoDB collector (see `nuocollector.watcher` Helm variable).
+NuoDB sidecar is used in [Database](https://github.com/nuodb/nuodb-helm-charts/tree/master/stable/database) chart when enabling database backup hooks (see `database.backupHooks.image`).
+
+## Contribute
+
+### Test requirements
+
+| Software   | Release Requirements                           |
+|------------|------------------------------------------------|
+| Python     |  Version 3.12 and onwards. |
+| Docker     |  Version 27.4.0 and onwards. |
+
+### Development environment
+
+To develop and test the tools locally, follow below steps.
+
+- Create Python virtual environment with an IDE or by running:
+
+```sh
+python3 -m venv .venv
+source .venv/bin/activate
+```
+
+- Install Python dependencies for the tool that is under development. E.g.
+
+```sh
+python3 -m pip install -r config_watcher/requirements.txt
+python3 -m pip install -r config_watcher/test-requirements.txt
+```
+
+- Make the necessary changes
+
+- Run static analysis
+
+```sh
+make fmt lint
+```
+
+- Run the tests
+
+```sh
+make test
+```

config_watcher/README.md

@@ -0,0 +1,46 @@
+# Configuration watcher
+
+This is an application that watches for changes on `ConfigMap` or `Secret` resources with a specified label selector and stores their `data` as files in a local directory.
+An optional HTTP/s webhook URL is executed on a resource change.
+The main purpose of the config watcher is to run as a sidecar for applications that need their configuration to be updated dynamically from the Kubernetes cluster state.
+
+## Features
+
+- Create, update, and delete files from `ConfigMap` or `Secret` resource `data` field.
+- Execute optional HTTP/s webhook URL on resource change
+- Multi-namespace support
+
+## Configuration
+
+### Kubernetes API server
+
+The config watcher needs read access to watched resources (either `configmaps` or `secrets`) for all watched namespaces.
+For example, below is a `Role` that allows read access to `configmaps` in a single namespace.
+
+```yaml
+apiVersion: rbac.authorization.k8s.io/v1
+kind: Role
+metadata:
+  name: config-watcher
+rules:
+- apiGroups: [""]
+  resources: ["configmaps"]
+  verbs: ["get", "watch", "list"]
+```
+
+### Environment variables
+
+| Name             | Description                                            | Default   | Type | Required |
+|------------------|--------------------------------------------------------|-----------|------|---------|
+| `LABEL_SELECTOR` | Kubernetes [label selector](https://kubernetes.io/docs/concepts/overview/working-with-objects/labels/#label-selectors) used for filtering the resources.  | `null` | string | `true` |
+| `TARGET_DIRECTORY` | Local directory path where the files will be created.  | `null` | string | `true` |
+| `RESOURCE_TYPE` | Resource type to watch for changes.  | `config_map` | `config_map` or `secret` | `false` |
+| `NAMESPACES` | A comma-separated list of Kubernetes namespaces to watch for resource changes. If the application is running in-cluster the default namespace will be inferred from `/var/run/secrets/kubernetes.io/serviceaccount/namespace` file. | `default` | string | `false` |
+| `WATCH_TIMEOUT` | Server-side timeout in seconds for the watch API call. The config watcher will reinitialize a new watch API call once this timeout expires. | `60` | `seconds` | `false` |
+| `WEBHOOK_URL` | Webhook URL to send a request to after a file content update. | `null` | `string` | `false` |
+| `WEBHOOK_METHOD` | Webhook method. | `GET` | `GET`, `POST`, `PUT` or `PATCH` | `false` |
+| `WEBHOOK_PAYLOAD` | JSON payload for the webhook. | `null` | `JSON` | `false` |
+| `WEBHOOK_TIMEOUT` | Timeout in seconds for the webhook request. | `60` | `seconds` | `false` |
+| `WEBHOOK_VERIFY` | Whether to verify the webhook server's TLS certificate. | `true` | `boolean` | `false` |
+| `RETRY_COUNT` | The number of retries for all requests. | `5` | `int` | `false` |
+| `SHUTDOWN_TIMEOUT` | Timeout in seconds for graceful shutdown of the application. If set to a value greater than `0`, the watcher container may be reported in `Terminating` state. | `0` | `seconds` | `false` |

config_watcher/watcher.py

@@ -45,11 +45,11 @@
 WEBHOOK_VERIFY = os.environ.get("WEBHOOK_VERIFY", "true")
 
 NAMESPACE_FILE = "/var/run/secrets/kubernetes.io/serviceaccount/namespace"
-RETRY_COUNT = 5
+RETRY_COUNT = os.environ.get("RETRY_COUNT", "5")
 DEFAULT_RETRIES = Retry(
-    connect=RETRY_COUNT,
-    read=RETRY_COUNT,
-    total=RETRY_COUNT,
+    connect=int(RETRY_COUNT),
+    read=int(RETRY_COUNT),
+    total=int(RETRY_COUNT),
     backoff_factor=0.5,
 )
 
@@ -320,7 +320,7 @@ def start_watchers(
     resource_type=RESOURCE_TYPE,
     label_selector=LABEL_SELECTOR,
     watch_timeout=int(WATCH_TIMEOUT),
-    retry=RETRY_COUNT,
+    retry=int(RETRY_COUNT),
 ):
     threads = []
     for ns in namespaces:

docker/build.sh

@@ -21,22 +21,33 @@ GIT_STATUS="$(git status --porcelain)"
 : ${IMG_REPO:="nuodb/nuodb-sidecar"}
 : ${IMG_TAG:="latest"}
 
-docker build -f ./docker/Dockerfile . --tag "$IMG_REPO:$IMG_TAG" --label org.opencontainers.image.revision="$COMMIT"
+# Extract the first image tag
+MAIN_TAG="$(echo "$IMG_TAG" | awk '{print $1}')"
+
+docker build -f ./docker/Dockerfile . --tag "$IMG_REPO:$MAIN_TAG" --label org.opencontainers.image.revision="$COMMIT"
+
+# Also tag the image with other tags if specified
+for tag in $IMG_TAG; do
+    docker tag "$IMG_REPO:$MAIN_TAG" "$IMG_REPO:$tag"
+done
 
 # Also tag image with commit SHA if there are no uncommitted changes
 if [ "$GIT_STATUS" = "" ]; then
-    docker tag "$IMG_REPO:$IMG_TAG" "$IMG_REPO:sha-$COMMIT"
+    docker tag "$IMG_REPO:$MAIN_TAG" "$IMG_REPO:sha-$COMMIT"
 fi
 
 # Use PUSH_REPO to specify remote Docker repo to publish to
-if [ "$PUSH_REPO" != "" ] && [ "$(read -p "Push image to \"$PUSH_REPO\" with tag \"$IMG_TAG\" (yes/no)? " && echo "$REPLY")" = "yes" ]; then
+if [ "$PUSH_REPO" != "" ] && [ "$(read -p "Push image to \"$PUSH_REPO\" with tag \"$MAIN_TAG\" (yes/no)? " REPLY && echo "$REPLY")" = "yes" ]; then
     # Make sure there are no uncommitted changes
     [ "$GIT_STATUS" = "" ] || fail "Cannot push image with uncommitted changes:\n$GIT_STATUS"
 
-    docker tag "$IMG_REPO:$IMG_TAG" "$PUSH_REPO:$IMG_TAG"
-    docker push "$PUSH_REPO:$IMG_TAG"
+    # Tag the image with other tags if specified
+    for tag in $IMG_TAG; do
+        docker tag "$IMG_REPO:$MAIN_TAG" "$PUSH_REPO:$tag"
+        docker push "$PUSH_REPO:$tag"
+    done
 
     # Also tag image with commit SHA
-    docker tag "$IMG_REPO:$IMG_TAG" "$PUSH_REPO:sha-$COMMIT"
+    docker tag "$IMG_REPO:$MAIN_TAG" "$PUSH_REPO:sha-$COMMIT"
     docker push "$PUSH_REPO:sha-$COMMIT"
 fi

nuodb-operations/README.md

@@ -0,0 +1,54 @@
+# NuoDB operations
+
+An HTTP server that allows remote operations on NuoDB engine containers.
+
+## Features
+
+- _Pre_ and _post_ backup hooks for snapshot-based NuoDB backup
+- Support for custom scripts to be invoked on HTTP requests
+
+## Configuration
+
+### Sidecar configuration
+
+The application must have write access to NuoDB archive and journal volumes which can be mounted using container `volumeMounts`.
+It is required that the `shareProcessNamespace` Pod spec field is set to `true`.
+
+### Environment variables
+
+| Name             | Description                                            | Default   | Type | Required |
+|------------------|--------------------------------------------------------|-----------|------|---------|
+| `FREEZE_MODE` | The freeze mode to be used when executing backup hooks. | `null` | `hotsnap`, `fsfreeze` or `suspend` | `true` |
+| `NUODB_ARCHIVE_DIR` | NuoDB archive directory path. | `/mnt/archive` | string | `false` |
+| `NUODB_JOURNAL_DIR` | NuoDB archive directory path. Must be set if external journal is enabled. | `null` | string | `false` |
+| `FREEZE_TIMEOUT` | Timeout in seconds after which the archive will be automatically unfrozen. | `null` | int | `false` |
+
+### Commands
+
+#### server
+
+Starts HTTP server for backup operations.
+
+| Argument         | Description                                            | Default   |
+|------------------|--------------------------------------------------------|-----------|
+| `--port` | The local port used by the server to listen on. | `80` |
+| `--handler-config` | Path to the YAML file which contains custom handlers to register on the HTTP server. | `/etc/nuodb/handlers.json` |
+
+#### pre-hook
+
+Executes pre-backup hook.
+
+| Argument         | Description                                            | Default   |
+|------------------|--------------------------------------------------------|-----------|
+| `--backup-id` | The ID associated with this backup. Required. | `null` |
+| `--opaque-file` | Path to a file containing user-defined information to associate with the backup. | `null` |
+| `--timeout` | Timeout in seconds after which the archive will be automatically unfrozen. Zero means no timeout. | `0` |
+
+#### post-hook
+
+Executes post-backup hook.
+
+| Argument         | Description                                            | Default   |
+|------------------|--------------------------------------------------------|-----------|
+| `--backup-id` | The ID associated with this backup. Required. | `null` |
+| `--force` | Unfreeze the archive even if the supplied backup ID doesn't match the one for the current backup. | `false` |