Skip to content

Commit

Permalink
Merge branch 'main' of github.com:canonical/mysql-k8s-operator into f…
Browse files Browse the repository at this point in the history
…eature/dpe-4613-router-metadata-gone
  • Loading branch information
paulomach committed Sep 12, 2024
2 parents e2571e4 + 9a20ea8 commit 37d2a1e
Show file tree
Hide file tree
Showing 13 changed files with 264 additions and 103 deletions.
6 changes: 3 additions & 3 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -16,7 +16,7 @@ on:
jobs:
lint:
name: Lint
uses: canonical/data-platform-workflows/.github/workflows/lint.yaml@v20.0.2
uses: canonical/data-platform-workflows/.github/workflows/lint.yaml@v21.0.1

unit-test:
name: Unit test charm
Expand Down Expand Up @@ -56,7 +56,7 @@ jobs:

build:
name: Build charm
uses: canonical/data-platform-workflows/.github/workflows/build_charm.yaml@v20.0.2
uses: canonical/data-platform-workflows/.github/workflows/build_charm.yaml@v21.0.1
with:
cache: true

Expand Down Expand Up @@ -88,7 +88,7 @@ jobs:
- lint
- unit-test
- build
uses: canonical/data-platform-workflows/.github/workflows/integration_test_charm.yaml@v21.0.0
uses: canonical/data-platform-workflows/.github/workflows/integration_test_charm.yaml@v21.0.1
with:
artifact-prefix: ${{ needs.build.outputs.artifact-prefix }}
architecture: ${{ matrix.architecture }}
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -36,15 +36,15 @@ jobs:

build:
name: Build charm
uses: canonical/data-platform-workflows/.github/workflows/build_charm.yaml@v20.0.2
uses: canonical/data-platform-workflows/.github/workflows/build_charm.yaml@v21.0.1

release:
name: Release charm
needs:
- lib-check
- ci-tests
- build
uses: canonical/data-platform-workflows/.github/workflows/release_charm.yaml@v20.0.2
uses: canonical/data-platform-workflows/.github/workflows/release_charm.yaml@v21.0.1
with:
channel: 8.0/edge
artifact-prefix: ${{ needs.build.outputs.artifact-prefix }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/sync_docs.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -10,7 +10,7 @@ on:
jobs:
sync-docs:
name: Sync docs from Discourse
uses: canonical/data-platform-workflows/.github/workflows/sync_docs.yaml@v20.0.2
uses: canonical/data-platform-workflows/.github/workflows/sync_docs.yaml@v21.0.1
with:
reviewers: a-velasco
permissions:
Expand Down
34 changes: 34 additions & 0 deletions docs/explanation/e-audit-logs.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,34 @@
# Audit Logs

The Audit Log plugin allows all login/logout records to be stored in a log file. It is enabled in Charmed MySQL K8s by default.

## Overview

The following is a sample of the audit logs, with format json with login/logout records:

```json
{"audit_record":{"name":"Quit","record":"6_2024-09-03T01:53:14","timestamp":"2024-09-03T01:53:33Z","connection_id":"992","status":0,"user":"clusteradmin","priv_user":"clusteradmin","os_login":"","proxy_user":"","host":"localhost","ip":"","db":""}}
{"audit_record":{"name":"Connect","record":"7_2024-09-03T01:53:14","timestamp":"2024-09-03T01:53:33Z","connection_id":"993","status":1156,"user":"","priv_user":"","os_login":"","proxy_user":"","host":"juju-da2225-8","ip":"10.207.85.214","db":""}}
{"audit_record":{"name":"Connect","record":"8_2024-09-03T01:53:14","timestamp":"2024-09-03T01:53:33Z","connection_id":"994","status":0,"user":"serverconfig","priv_user":"serverconfig","os_login":"","proxy_user":"","host":"juju-da2225-8","ip":"10.207.85.214","db":""}}
```

The logs are stored in the `/var/log/mysql` directory of the mysql container, and it's rotated
every minute to the `/var/log/mysql/archive_audit` directory.
It's recommended to integrate the charm with [COS](/t/9900), from where the logs can be easily persisted and queried using Loki/Grafana.

## Configurations

1. `plugin-audit-enabled` - The audit plugin is enabled by default in the charm, but it's possible to disable it by setting:

```bash
juju config mysql-k8s plugin-audit-enabled=false
```
Valid value are `false` and `true`. By setting it to false, existing logs are still kept in the `archive_audit` directory.

1. `plugin-audit-strategy` - By default the audit plugin writes logs in asynchronous mode for better performance.
To ensure logs are written to disk on more timely fashion, this configuration can be set to semi-synchronous mode:

```bash
juju config mysql-k8s plugin-audit-strategy=semi-async
```
Valid values are `async` and `semi-async`.
186 changes: 106 additions & 80 deletions docs/explanation/e-logs.md

Large diffs are not rendered by default.

6 changes: 6 additions & 0 deletions docs/how-to/h-create-backup.md
Original file line number Diff line number Diff line change
Expand Up @@ -10,6 +10,12 @@ Once Charmed MySQL K8s is `active` and `idle` (check `juju status`), you can cre
juju run-action mysql-k8s/leader create-backup --wait
```

[note]
If you have a cluster of one unit, you can run the `create-backup` action on `mysql-k8s/leader` (which will also be the primary unit).

Otherwise, you must run the `create-backup` action on a non-primary unit (see `juju status` or run `juju run-action mysql-k8s/leader get-cluster-status` to find the primary unit).
[/note]

You can list your available, failed, and in progress backups by running the `list-backups` command:
```shell
juju run-action mysql-k8s/leader list-backups --wait
Expand Down
77 changes: 77 additions & 0 deletions docs/how-to/h-enable-alert-rules.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
# How to enable COS Alert Rules

This guide will show how to set up [Pushover](https://pushover.net/) to receive alert notifications from the COS Alert Manager with [Awesome Alert Rules](https://samber.github.io/awesome-prometheus-alerts/).

Charmed MySQL K8s ships a pre-configured and pre-enabled [list of Awesome Alert Rules].

<details><summary>Screenshot of alert rules in the Grafana web interface</summary>

![Screenshot from 2024-01-18 20-05-52|690x439](upload://j6WSPQ1BzoFzqIg2jm1mTq79SMo.png)
</details>

For information about accessing and managing COS Alert Rules, refer to the [COS documentation](https://charmhub.io/cos-lite).

## Prerequisites
* A deployed [Charmed MySQL K8s operator]
* A deployed [`cos-lite` bundle in a Kubernetes environment](https://charmhub.io/topics/canonical-observability-stack/tutorials/install-microk8s)
* Fully configured [COS Monitoring]

## Enable COS alerts for Pushover
The following section is an example of the [Pushover](https://pushover.net/) alerts aggregator.

The first step is to create a new account on Pushover (or use an existing one). The goal is to have the 'user key' and 'token' to authorize alerts for the Pushover application. Follow this straightforward [Pushover guide](https://support.pushover.net/i175-how-to-get-a-pushover-api-or-pushover-application-token).

Next, create a new [COS Alert Manager](https://charmhub.io/alertmanager-k8s) config (replace `user_key` and `token` with yours):
```shell
cat > myalert.yaml << EOF
```
```yaml
global:
resolve_timeout: 5m
http_config:
follow_redirects: true
enable_http2: true
route:
receiver: placeholder
group_by:
- juju_model_uuid
- juju_application
- juju_model
continue: false
group_wait: 30s
group_interval: 5m
repeat_interval: 1h
receivers:
- name: placeholder
pushover_configs:
- user_key: <relace_with_your_user_key>
token: <relace_with_your_token>
url: http://<relace_with_grafana_public_ip>/cos-grafana/alerting/list
title: "{{ range .Alerts }}{{ .Labels.severity }} - {{ if .Labels.juju_unit }}{{ .Labels.juju_unit }}{{ else }}{{ .Labels.juju_application }}{{ end }} in model {{ .Labels.juju_model }}: {{ .Labels.alertname }} {{ end }}"
message: "{{ range .Alerts }} Job: {{ .Labels.job }} Instance: {{ .Labels.instance }} {{ end }}"
templates: []
EOF
```
Upload and apply newly the created alert manager config:
```
juju switch <k8s_cos_controller>:<cos_model_name>
juju config alertmanager config_file=@myalert.yaml
```

At this stage, the COS Alert Manager will start sending alert notifications to Pushover. Users can receive them on all supported [Pushover clients/apps](https://pushover.net/clients).

The image below shows an example of the Pushover web client:

![image|690x439](upload://vqUcKpZ5R4wQLmY2HYGV5fz5pNU.jpeg)

## Alert receivers

The similar way as above, COS alerts can be send to the long [list of supported receivers](https://prometheus.io/docs/alerting/latest/configuration/#receiver-integration-settings).

Do you have questions? [Contact us]!

<!-- Links -->
[Contact us]: /t/11868
[Charmed MySQL K8s operator]: /t/11869
[COS Monitoring]: /t/9981
[list of Awesome Alert Rules]: https://github.com/canonical/mysql-k8s-operator/tree/main/src/prometheus_alert_rules
3 changes: 3 additions & 0 deletions docs/overview.md
Original file line number Diff line number Diff line change
Expand Up @@ -68,6 +68,7 @@ Charmed MySQL K8s is an official distribution of MySQL. It’s an open-source pr
| 2 | h-monitoring | [Monitoring (COS)]() |
| 3 | h-enable-monitoring | [Enable monitoring](/t/9981) |
| 3 | h-enable-tracing | [Enable tracing](/t/14448) |
| 3 | h-enable-alert-rules | [Enable Alert Rules](/t/15488) |
| 2 | h-upgrade | [Upgrade]() |
| 3 | h-upgrade-intro | [Overview](/t/11754) |
| 3 | h-upgrade-juju | [Upgrade Juju](/t/14333) |
Expand Down Expand Up @@ -101,13 +102,15 @@ Charmed MySQL K8s is an official distribution of MySQL. It’s an open-source pr
| 2 | r-requirements | [Requirements](/t/11421) |
| 2 | r-testing | [Testing](/t/11772) |
| 2 | r-profiles | [Profiles](/t/11892) |
| 2 | r-plugins-extensions | [Plugins/extensions](/t/15482) |
| 2 | r-contacts | [Contacts](/t/11868) |
| 1 | explanation | [Explanation]() |
| 2 | e-architecture | [Architecture](/t/11757) |
| 2 | e-interfaces-endpoints | [Interfaces/endpoints](/t/10249) |
| 2 | e-statuses | [Statuses](/t/11866) |
| 2 | e-users | [Users](/t/10791) |
| 2 | e-logs | [Logs](/t/12080) |
| 3 | e-audit-logs | [Audit Logs](/t/15423) |
| 2 | e-juju | [Juju](/t/11984) |
| 2 | e-flowcharts | [Flowcharts](/t/10031) |
| 2 | e-legacy-charm | [Legacy charm](/t/11236) |
Expand Down
9 changes: 9 additions & 0 deletions docs/reference/r-plugins-extensions.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,9 @@
# Supported plugins/extensions

The following list contains all plugins/extensions supported by Charmed MySQL in alphabetical order. The **revision** column indicates which charm revision introduced support for the extension.

If you need support for other extensions, feel free to [reach out to us](/t/11868).

| Plugin/extension name | Revision |
|--------------------------------|------------------------------------------------------------------------------|
| [plugin-audit-enabled](/t/15423) | [178+](https://github.com/canonical/mysql-k8s-operator/releases/tag/rev179) |
22 changes: 11 additions & 11 deletions poetry.lock

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

10 changes: 5 additions & 5 deletions pyproject.toml
Original file line number Diff line number Diff line change
Expand Up @@ -50,11 +50,11 @@ parameterized = "^0.9.0"

[tool.poetry.group.integration.dependencies]
pytest = "^7.4.0"
pytest-github-secrets = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.0", subdirectory = "python/pytest_plugins/github_secrets"}
pytest-microceph = {git = "https://github.com/canonical/data-platform-workflows", tag = "v20.0.2", subdirectory = "python/pytest_plugins/microceph"}
pytest-github-secrets = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.1", subdirectory = "python/pytest_plugins/github_secrets"}
pytest-microceph = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.1", subdirectory = "python/pytest_plugins/microceph"}
pytest-operator = "^0.28.0"
pytest-operator-cache = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.0", subdirectory = "python/pytest_plugins/pytest_operator_cache"}
pytest-operator-groups = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.0", subdirectory = "python/pytest_plugins/pytest_operator_groups"}
pytest-operator-cache = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.1", subdirectory = "python/pytest_plugins/pytest_operator_cache"}
pytest-operator-groups = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.1", subdirectory = "python/pytest_plugins/pytest_operator_groups"}
juju = "^3.5.2.0"
ops = "^2.15.0"
mysql-connector-python = "~8.0.33"
Expand All @@ -65,7 +65,7 @@ urllib3 = "^1.26.16"
lightkube = "^0.14.0"
kubernetes = "^27.2.0"
allure-pytest = "^2.13.2"
allure-pytest-collection-report = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.0", subdirectory = "python/pytest_plugins/allure_pytest_collection_report"}
allure-pytest-collection-report = {git = "https://github.com/canonical/data-platform-workflows", tag = "v21.0.1", subdirectory = "python/pytest_plugins/allure_pytest_collection_report"}
pytest-asyncio = "^0.21.1"

[tool.coverage.run]
Expand Down
7 changes: 6 additions & 1 deletion src/k8s_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -54,7 +54,11 @@ def create_endpoint_services(self, roles: List[str]) -> None:
roles: List of roles to append on the service name
"""
for role in roles:
selector = {"cluster-name": self.cluster_name, "role": role}
selector = {
"cluster-name": self.cluster_name,
"application-name": self.app_name,
"role": role,
}
service_name = f"{self.app_name}-{role}"
pod0 = self.client.get(
res=Pod,
Expand Down Expand Up @@ -128,6 +132,7 @@ def label_pod(self, role: str, pod_name: Optional[str] = None) -> None:
logger.debug(f"Patching {pod_name=} with {role=}")

pod.metadata.labels["cluster-name"] = self.cluster_name
pod.metadata.labels["application-name"] = self.app_name
pod.metadata.labels["role"] = role
self.client.patch(Pod, pod_name, pod)
except ApiError as e:
Expand Down
1 change: 1 addition & 0 deletions tests/unit/test_k8s_helpers.py
Original file line number Diff line number Diff line change
Expand Up @@ -49,6 +49,7 @@ def test_create_endpoint_service(self, _create):
spec=ServiceSpec(
selector={
"cluster-name": self.harness.charm.app_peer_data.get("cluster-name"),
"application-name": self.harness.charm.model.app.name,
"role": "role1",
},
ports=[ServicePort(port=3306, targetPort=3306)],
Expand Down

0 comments on commit 37d2a1e

Please sign in to comment.