Skip to content

Commit

Permalink
Rename osm_mysql to mysql_root
Browse files Browse the repository at this point in the history
Port over autogeneration of config values from the vm charm

Autogenerate config values for the mysql legacy relation

Add upterm session to debug failing integration test

Always set up upterm session

Pass essential env with built charm to upterm session

Fix minor bugs + remove upterm github action step

Run format

[upload-charm-docs] Migrate charm docs (#266)

* 'modified: docs/explanation/e-flowcharts.md,docs/explanation/e-interfaces.md,docs/index.md,docs/how-to/h-enable-monitoring.md // new: docs/explanation/e-legacy-charm.md,docs/how-to/h-deploy-gke.md'

* 'modified: docs/explanation/e-legacy-charm.md,docs/index.md'

* 'modified: docs/how-to/h-deploy-gke.md'

---------

Co-authored-by: upload-charms-docs-bot <upload-charms-docs-bot@users.noreply.github.com>

DPE-2068 use kill-delay (#258)

* use kill-delay

* update requirements and delay time

DPE-2225 using allocable memory instead of `free` (#256)

* using allocable memory instead of `free`

* use loose opt modifier

* merged from vm

* remove comment

* fix memory units

* set missing trust and testing profile in test

* adjust for low memory

* test change for ci

* fix function call

* typing fixes

* test wait for maintenance of instance before asserting recovery

* extra wait step

* strip trailing newline

* non-active

Fixes for network partition test (#272)

* fixes for chaos-mesh

* ensure chaos-mesh availability before test
* fixes destroy script
* function scoped

* address pr comment

* fix call

Port peer juju secret implementation from the vm charm (#263)

Upgrade mysql (v0.38) and tls (v0.2) charm libs + minor adjustments

[MISC] Revert change on Discourse Sync action (#265)

Co-authored-by: upload-charms-docs-bot <upload-charms-docs-bot@users.noreply.github.com>

[upload-charm-docs] Migrate charm docs (#266)

* 'modified: docs/explanation/e-flowcharts.md,docs/explanation/e-interfaces.md,docs/index.md,docs/how-to/h-enable-monitoring.md // new: docs/explanation/e-legacy-charm.md,docs/how-to/h-deploy-gke.md'

* 'modified: docs/explanation/e-legacy-charm.md,docs/index.md'

* 'modified: docs/how-to/h-deploy-gke.md'

---------

Co-authored-by: upload-charms-docs-bot <upload-charms-docs-bot@users.noreply.github.com>

DPE-2068 use kill-delay (#258)

* use kill-delay

* update requirements and delay time

DPE-2225 using allocable memory instead of `free` (#256)

* using allocable memory instead of `free`

* use loose opt modifier

* merged from vm

* remove comment

* fix memory units

* set missing trust and testing profile in test

* adjust for low memory

* test change for ci

* fix function call

* typing fixes

* test wait for maintenance of instance before asserting recovery

* extra wait step

* strip trailing newline

* non-active

Pin python dependencies with Poetry (#273)

[MISC] Switch maintainers to the DPE mailing list (#269)

* Switch maintainers to the DPE mailing list

* Update metadata.yaml

Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com>

---------

Co-authored-by: Carl Csaposs <carl.csaposs@canonical.com>

Pin charmcraft to 2.3.0 in release.yaml to fix build (#276)

charmcraft pinned to 2.3.0 in integration test build but not release build

canonical/charmcraft#1179

Add missing group markers (#278)

Leftover from #273

No integration tests were running since no groups were collected

Address PR feedback
  • Loading branch information
shayancanonical committed Aug 16, 2023
1 parent be86a11 commit afcbe15
Show file tree
Hide file tree
Showing 42 changed files with 3,321 additions and 746 deletions.
16 changes: 16 additions & 0 deletions .github/renovate.json5
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"$schema": "https://docs.renovatebot.com/renovate-schema.json",
"extends": ["github>canonical/data-platform//renovate_presets/charm.json5"],
"reviewers": ["team:data-platform-mysql"],
"packageRules": [
// Later rules override earlier rules

// MySQL 8.X does not follow semantic versioning (e.g. 8.0.1 -> 8.0.2 can include a breaking change)
// Therefore, use a separate Renovate group so that it has a separate PR
{
"matchManagers": ["poetry"],
"matchPackageNames": ["mysql-connector-python"],
"groupName": "MySQL Connector/Python"
}
]
}
87 changes: 61 additions & 26 deletions .github/workflows/ci.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -15,58 +15,91 @@ on:

jobs:
lint:
uses: canonical/data-platform-workflows/.github/workflows/lint.yaml@v2
name: Lint
uses: canonical/data-platform-workflows/.github/workflows/lint.yaml@v4.2.0

unit-test:
name: Unit tests
name: Unit test charm
runs-on: ubuntu-latest
timeout-minutes: 5
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install tox
# TODO: Consider replacing with custom image on self-hosted runner OR pinning version
run: python3 -m pip install tox
- name: Install tox & poetry
run: |
pipx install tox
pipx install poetry
- name: Run tests
run: tox run -e unit

build:
name: Build charms
uses: canonical/data-platform-workflows/.github/workflows/build_charms_with_cache.yaml@v2
name: Build charm
uses: canonical/data-platform-workflows/.github/workflows/build_charms_with_cache.yaml@v4.2.0
with:
charmcraft-snap-revision: 1349 # version 2.3.0
permissions:
actions: write # Needed to manage GitHub Actions cache

gh-hosted-collect-integration-tests:
name: (GH hosted) Collect integration test groups
needs:
- lint
- unit-test
runs-on: ubuntu-latest
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install tox & poetry
run: |
pipx install tox
pipx install poetry
- name: Select test stability level
id: select-test-stability
run: |
if [[ "${{ github.event_name }}" == "schedule" ]]
then
echo Running unstable and stable tests
echo "mark_expression=" >> "$GITHUB_OUTPUT"
else
echo Skipping unstable tests
echo "mark_expression=not unstable" >> "$GITHUB_OUTPUT"
fi
- name: Collect test groups
id: collect-groups
run: tox run -e integration -- tests/integration -m '${{ steps.select-test-stability.outputs.mark_expression }}' --collect-groups
outputs:
groups: ${{ steps.collect-groups.outputs.groups }}

integration-test:
gh-hosted-integration-test:
strategy:
fail-fast: false
matrix:
tox-environments:
- integration-charm
- integration-database-relation
- integration-mysql-root
- integration-replication
- integration-self-healing
- integration-tls
- integration-backups
name: ${{ matrix.tox-environments }}
groups: ${{ fromJSON(needs.gh-hosted-collect-integration-tests.outputs.groups) }}
name: (GH hosted) ${{ matrix.groups.job_name }}
needs:
- lint
- unit-test
- build
- gh-hosted-collect-integration-tests
runs-on: ubuntu-latest
timeout-minutes: 120
steps:
- name: Checkout
uses: actions/checkout@v3
- name: Install tox & poetry
run: |
pipx install tox
pipx install poetry
- name: Setup operator environment
# TODO: Replace with custom image on self-hosted runner
uses: charmed-kubernetes/actions-operator@main
with:
provider: microk8s
- name: Download packed charm(s)
uses: actions/download-artifact@v3
with:
name: ${{ needs.build.outputs.artifact-name }}
- name: Select tests
id: select-tests
- name: Select test stability level
id: select-test-stability
run: |
if [[ "${{ github.event_name }}" == "schedule" ]]
then
Expand All @@ -77,10 +110,12 @@ jobs:
echo "mark_expression=not unstable" >> "$GITHUB_OUTPUT"
fi
- name: Run integration tests
run: tox run -e "${{ matrix.tox-environments }}" -- -m '${{ steps.select-tests.outputs.mark_expression }}'
run: tox run -e integration -- "${{ matrix.groups.path_to_test_file }}" --group="${{ matrix.groups.group_number }}" -m '${{ steps.select-test-stability.outputs.mark_expression }}'
env:
AWS_ACCESS_KEY: ${{ matrix.tox-environments != 'integration-backups' || secrets.AWS_ACCESS_KEY }}
AWS_SECRET_KEY: ${{ matrix.tox-environments != 'integration-backups' || secrets.AWS_SECRET_KEY }}
GCP_ACCESS_KEY: ${{ matrix.tox-environments != 'integration-backups' || secrets.GCP_ACCESS_KEY }}
GCP_SECRET_KEY: ${{ matrix.tox-environments != 'integration-backups' || secrets.GCP_SECRET_KEY }}
CI_PACKED_CHARMS: ${{ needs.build.outputs.charms }}
SECRETS_FROM_GITHUB: |
{
"AWS_ACCESS_KEY": "${{ secrets.AWS_ACCESS_KEY }}",
"AWS_SECRET_KEY": "${{ secrets.AWS_SECRET_KEY }}",
"GCP_ACCESS_KEY": "${{ secrets.GCP_ACCESS_KEY }}",
"GCP_SECRET_KEY": "${{ secrets.GCP_SECRET_KEY }}",
}
43 changes: 0 additions & 43 deletions .github/workflows/promote.yaml

This file was deleted.

8 changes: 6 additions & 2 deletions .github/workflows/release.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -28,18 +28,22 @@ jobs:
- lib-check
uses: ./.github/workflows/ci.yaml
secrets: inherit
permissions:
actions: write # Needed to manage GitHub Actions cache

build:
name: Build charm
uses: canonical/data-platform-workflows/.github/workflows/build_charm_without_cache.yaml@v2
uses: canonical/data-platform-workflows/.github/workflows/build_charm_without_cache.yaml@v4.2.0
with:
charmcraft-snap-revision: 1349 # version 2.3.0

release:
name: Release charm
needs:
- lib-check
- ci-tests
- build
uses: canonical/data-platform-workflows/.github/workflows/release_charm.yaml@v2
uses: canonical/data-platform-workflows/.github/workflows/release_charm.yaml@v4.2.0
with:
channel: 8.0/edge
artifact-name: ${{ needs.build.outputs.artifact-name }}
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/sync_issue_to_jira.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ on:
jobs:
sync:
name: Sync GitHub issue to Jira
uses: canonical/data-platform-workflows/.github/workflows/sync_issue_to_jira.yaml@v2
uses: canonical/data-platform-workflows/.github/workflows/sync_issue_to_jira.yaml@v4.2.0
with:
jira-base-url: https://warthogs.atlassian.net
jira-project-key: DPE
Expand Down
3 changes: 3 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -1,3 +1,6 @@
/requirements.txt
/requirements-last-build.txt

# Byte-compiled / optimized / DLL files
__pycache__/
*.py[cod]
Expand Down
24 changes: 15 additions & 9 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -20,30 +20,36 @@ this operator.
the `main` branch. This also avoids merge commits and creates a linear Git commit history.

## Developing
Install `tox` and `poetry`
```shell
python3 -m pip install --user pipx
python3 -m pipx ensurepath
pipx install tox
pipx install poetry
```

You can create an environment for development with `tox`:
You can create an environment for development:

```shell
tox devenv -e integration
source venv/bin/activate
poetry install
```

### Testing

```shell
tox run -e format # update your code according to linting rules
tox run -e lint # code style
tox run -e unit # unit tests
tox run -e integration-* # integration tests
tox # runs 'lint' and 'unit' environments
tox run -e format # update your code according to linting rules
tox run -e lint # code style
tox run -e unit # unit tests
tox run -e integration # integration tests
tox # runs 'lint' and 'unit' environments
```

## Build charm

Build the charm in this git repository using:

```shell
charmcraft pack
tox run -e build
```

### Deploy
Expand Down
8 changes: 8 additions & 0 deletions charmcraft.yaml
Original file line number Diff line number Diff line change
Expand Up @@ -11,9 +11,17 @@ bases:
channel: "22.04"
parts:
charm:
override-pull: |
craftctl default
if [[ ! -f requirements.txt ]]
then
echo 'ERROR: Use "tox run -e build" instead of calling "charmcraft pack" directly' >&2
exit 1
fi
build-packages:
- libffi-dev
- libssl-dev
- pkg-config
- rustc
- cargo

43 changes: 22 additions & 21 deletions docs/explanation/e-flowcharts.md
Original file line number Diff line number Diff line change
@@ -1,35 +1,36 @@
# Charm lifecycle flowcharts

[Click to navigate the mermaid diagram on GitHub](https://github.com/canonical/mysql-k8s-operator/blob/main/docs/explanation/e-flowcharts.md).

```mermaid
flowchart TD
id101([leader_elected]) --> id102[generate\npassword/configs]
id102 --> id103[store peer databag]
id103 --> id104[add `configured`\nflag]
id104 --> id105((return))
id103 --> id104((return))
id201([pebble_ready]) --> id202{if not `configured` \nnor peer relation}
id202 --> id203>defer]
id202 -- else --> id204[add pebble layer]
id204 --> id205[configure users]
id205 --> id206[configure instance]
id204 --> id205[configure mysql\nusers]
id205 --> id206[configure instance\nfor GR]
id206 --> id207{is leader?}
id207 -- no --> id208((return))
id207 -- yes --> id209[create cluster]
id207 -- yes --> id209[create GR cluster]
id209 --> id208
id301([peer_relation_joined]) --> id302{if not `configured`}
id302 --> id303>defer]
id302 -- else --> id304{is leader?}
id304 -- no --> id311
id304 -- yes --> id305{container\ncannot connect\nor pebble layer\nnot running}
id305 --> id303
id305 -- no --> id306[check instance configuration]
id306 --> id307{instance configured?}
id307 -- no --> id303
id307 -- yes --> id308{any units in state transfer?}
id308 -- yes --> id303
id308 -- no --> id309[store instance address to databag]
id309 --> id310[add instance to cluster]
id310 --> id311((return))
```

id301([peer_relation_changed\nor\nupdate_status]) --> id302{is waiting\nto join}
id302 -- yes --> id303{is already\nin cluster?}
id303 --> id304[get primary\nfrom any online peer]
id304 --> id305[get current\ncluster node count]
id305 --> id306{is cluster\nat max size}
id306 -- no --> id307{is cluster\ntopology\nchanging}
id307 --> id309[acquire topology\nchange token]
id309 --> id310[join the cluster\nfrom primary]
id310 --> id311[release token]
id306 -- yes --> id308[Set blocked\nand standby]
id302 -- no --> id399((return))
id303 -- yes --> id399
id307 --> id399
id308 --> id399
id311 --> id399
```
2 changes: 1 addition & 1 deletion docs/explanation/e-interfaces.md
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ Charmed MySQL K8s supports modern `mysql_client` and legacy `mysql` interfaces (

## Modern interfaces

This charm provides modern ['mysql_client' ](https://github.com/canonical/charm-relation-interfaces) interface. Applications can easily connect MySQL using ['data_interfaces'](https://charmhub.io/data-platform-libs/libraries/data_interfaces) library from ['data-platform-libs'](https://github.com/canonical/data-platform-libs/).
This charm provides modern ['mysql_client'](https://github.com/canonical/charm-relation-interfaces) interface. Applications can easily connect MySQL using ['data_interfaces'](https://charmhub.io/data-platform-libs/libraries/data_interfaces) library from ['data-platform-libs'](https://github.com/canonical/data-platform-libs/).

### Modern `mysql_client` interface (`database` endpoint):

Expand Down
Loading

0 comments on commit afcbe15

Please sign in to comment.