Merge pull request #39 from openzim/add_versioning

Add versioning and disable pyright bytes type promotion

CHANGELOG.md

@@ -7,12 +7,109 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0
 
 ## [Unreleased]
 
+### Added
+
+- Add versioning to the convention
+- Mention the adherence to Python convention in README.md
+
+### Changed
+
+- Move wiki doc to repo for proper review / versioning
+- Disable pyright bytes type promotions
+
+## [0.2.0] - 2024-02-05
+
+### Changed
+
+- Adopt hatch-openzim for metadata computation
+- Upgrade dependencies
+- Upgrade to Python 3.11 + 3.12 (instead of 3.10 + 3.11)
+- Fix ruff configuration in pyproject.toml to fix depreciation warning in 0.2
+- Fix Pyright ignore rules to adapt to breaking changes in 1.1.348
+
+## [0.1.10] - 2023-12-12
+
+### Fixed
+
+- Add package location to adapt to hatchling 1.19.0
+
+## [0.1.9] - 2023-09-19
+
+### Changed
+
+- Split Docker build for efficiency
+
+## [0.1.8] - 2023-08-17
+
+### Added
+
+- Add sample Docker test configuration for daemon processes
+- Add support for coverage HTML report
+
 ### Fixed
 
-- Dependency installation issue (#2)
+- Fix few invoke task arguments and help
+
+
+## [0.1.7] - 2023-08-04
+
+### Changed
+
+- Update dependencies
+
+## [0.1.6] - 2023-07-26
+
+### Changed
+
+- Enhance CI to do more tests regarding Docker and Python build and publish dev image
+
+## [0.1.5] - 2023-07-24
+
+### Changed
+
+- Source Python version from pyproject.toml
+- Build Docker image in CI `Publish.yaml`
+
+### Fixed
+
+- Adjust Ruff rules ignored by default
+- Use `no-cache-dir` for package install 
+
+## [0.1.4] - 2023-07-17
+
+### Added
+
+- Added debugpy
+
+## [0.1.3] - 2023-07-14
+
+### Added
+
+- Added a `check` feature in hatch
+- Added pyright wrapper to this feature
+- Installed this feature in `dev` environment
+- Used `check-pyright` task in QA workflow
+
+### Changed
+
+- Use major versions for workflows actions
+- Enable `dev` in default hatch environment
+
+## [0.1.2] - 2023-07-13
+
+### Fixed
+
+- Fix version to comply with SemVer
+
+## [0.1.1] - 2023-07-11
+
+### Added
+
+- Add the scripts to lint's features, otherwise we can use the hatch run lint:*
+
 
-## [0.1] - 2023-01-01
+## [0.1.0] - 2023-06-22
 
 ### Added
 
-- Ability to do stuff
+- Initial version

README.md

@@ -31,4 +31,6 @@ You'd want to install it in a dedicated virtual-environment (`python3 -m venv so
 ❯ pip install -e .
 ```
 
-Great-project adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing)
+Great-project adheres to openZIM's [Contribution Guidelines](https://github.com/openzim/overview/wiki/Contributing).
+
+Great-project has implemented openZIM's [Python bootstrap, conventions and policies](https://github.com/openzim/_python-bootstrap/docs/Policy.md) **v0.2.0**.

docs/.gitignore.md

@@ -0,0 +1,5 @@
+Git ignore is built from [gitignore.io](https://www.toptal.com/developers/gitignore) so it's reproducible. We use the `Python` and `macOS` recipes at least.
+
+Should there be additional changes, those are included at the beginning of the file, before the gitignore.io comment. A comment explains the purpose of the change.
+
+Should a change be necessary at the bottom of the file, it is mentioned at the beginning of the file.

docs/.pre-commit-config.yaml.md

@@ -0,0 +1,6 @@
+[pre-commit](https://pre-commit.com/) shared configuration to help ensure commits are linted properly.
+
+Although ruff handles trailing white spaces and EOF checking and fixing, those are very common mistakes which doesn't require much thinking.
+It's thus more convenient to keep them than inspect ruff's output and select only appropriate rules to launch with `--fix`.
+
+**Note**: `pyright` precommit-hook uses system (in `PATH`) `pyright` binary. `pyright` is a node cli tool that must be installed separately.

docs/CHANGELOG.md

@@ -0,0 +1,3 @@
+Mandatory for a versioned project. Must follow [Keep A Changelog](https://keepachangelog.com/en/1.1.0/)'s format.
+
+Ideally, entries in response to Github issues should mention them.

docs/Developer-Setup.md

@@ -0,0 +1,95 @@
+Many different setups are possible. There is no _right_ way of setting up your environment. The following serves as self-documentation and example.
+
+## [rgaudin](https://github.com/rgaudin)'s with Sublime Text
+
+- macOS
+- Python for each minor stable using [python.org packages](https://www.python.org/downloads/)
+- [hatch](https://pypi.org/project/hatch/) installed globally
+- [Sublime Text 4](https://www.sublimetext.com/) (proprietary)
+  - [LSP](https://lsp.sublimetext.io/)
+  - [LSP-ruff](https://packagecontrol.io/packages/LSP-ruff)
+  - [LSP-pyright](https://github.com/sublimelsp/LSP-pyright)
+  - [Debugger](https://github.com/daveleroy/SublimeDebugger)
+  - [sublack](https://github.com/jgirardet/sublack)
+  - [TOML](https://github.com/jasonwilliams/sublime_toml_highlighting)
+  - [Trailing Spaces](https://github.com/SublimeText/TrailingSpaces)
+  - _non-required but useful for openZIM projects_
+  - [Vue Syntax Highlighting](https://github.com/vuejs/vue-syntax-highlight)
+  - [Pretty YAML](https://github.com/aukaost/SublimePrettyYAML)
+  - [LSP-json](https://github.com/sublimelsp/LSP-json)
+  - [LSP-dockerfile](https://github.com/sublimelsp/LSP-dockerfile)
+  - [JSON Reindent](https://github.com/ThomasKliszowski/json_reindent)
+  - [Dockerfile Syntax Highlighting](https://github.com/asbjornenge/Docker.tmbundle)
+
+```sh
+# create (if missing) default env, installing dependencies
+# also verifies default env's python version (use a matrix for default with a single value to set otherwise)
+hatch run python -V
+
+# make sure it's correct
+hatch env show
+hatch env find
+
+# symlink hatch's default environment into `./.venv` (in project)
+ln -s "$(hatch env find)" .venv
+```
+
+The key part is the symlink to `.venv` so that LSP-pyright and other tool can automatically find the environment and properly detect the installed dependencies.
+
+
+## [benoit74](https://github.com/benoit74)'s with Visual Studio Code
+
+- macOS
+- [brew](https://brew.sh/)
+- [pyenv](https://github.com/pyenv/pyenv) (installed with brew) 
+- Python:
+  - at least each supported and released minor versions (i.e. security + bugfix on https://devguide.python.org/versions/)
+  - installed with `pyenv install x.y.z`
+  - all supported and released minor versions are set in `/Users/<username>/.python-version` (i.e. available when looking for a version with python, python3, python3.x, ...)
+- [hatch](https://pypi.org/project/hatch/) installed globally
+- [Visual Studio Code](https://github.com/microsoft/vscode) with following extensions (more or less related to Python development)
+  - Black Formater (Microsoft)
+  - Dev Containers (Microsoft)
+  - Docker (Microsoft)
+  - Excalidraw (pomdtr)
+  - Even Better TOML (tamasfe)
+  - GitLens (GitKraken)
+  - Jupyter (Microsoft)
+  - Pylance (Microsoft)
+  - Python (Microsoft)
+  - Remote Development (Microsoft)
+  - reStructuredText (LeXtudio Inc.)
+  - Ruff (Astral Software)
+  - Volar (Vue)
+  - YAML (Red Hat)
+
+Hatch is configured with an additional section in `config.toml` (configuration file is located in `~/Library/Application Support/hatch`, see https://hatch.pypa.io/latest/config/hatch/, correct file is shown by `hatch status` in any case).
+```
+[dirs.env]
+virtual = ".hatch"
+```
+
+This additional Hatch config section ensures that all virtual environments (the only built-in environment in hatch is `virtual` - for now at least) are created in a `.hatch` subfolder inside the project folder ; it is mandatory so that Visual Studio Code will be able to find these dependencies.
+
+On every project, create a local `.vscode/settings.json` to automatically format your code (adjust `typeCheckingMode` depending on your project):
+``` json
+{
+
+    "[python]": {
+        "editor.defaultFormatter": "ms-python.black-formatter",
+        "editor.formatOnSave": true,
+        "editor.codeActionsOnSave": {
+            "source.organizeImports": true
+        },
+    },
+    "python.analysis.typeCheckingMode": "basic",
+}
+```
+
+When starting to work on a project, start a shell on default env + installing dependencies + start VSCode ; starting VSCode from inside the hatch shell ensures he will find automatically the proper Python version and dependencies
+
+```sh
+hatch shell
+pip install '.[dev]'
+code .
+```

docs/Dockerfile-and-Publish-workflow.md

@@ -0,0 +1,64 @@
+You will find in this page our guidelines to:
+- create a `Dockerfile` to build a Docker image 
+- create a Github workflow to build and publish the Docker image to a registry
+
+# Dockerfile
+
+## Base image
+
+- Debian-based images using the current [debian stable](https://www.debian.org/releases/) release (e.g. bookworm as of July 2023)
+- `slim` variant whenever it is sufficient
+- `alpine` variant only for highly constrained space environment (Offspot apps for instance). See [drawbacks](https://github.com/tiangolo/uvicorn-gunicorn-docker#-alpine-python-warning) for explanations
+- October 2023, latest Python stable is `3.12` and Debian stable is `bookworm` hence `python:3.12-slim-bookworm` or `python:3.12-bookworm`
+
+## RUN commands
+
+- when using multiple lines, add a space before the `\` sign
+- when linking multiple command, place the `&&` sign at the beginning of the next line
+- on multiple lines, align the start of the commands, not the `&&`
+- when listing packages to be installed by apt, add each of the on a new line, one per line, alphabetically order
+- do not hesitate to add comments between lines to explain what is the reason / purpose of the next line
+- do not include `apt-get clean` command since official Debian and Ubuntu images [automatically run apt-get clean](https://github.com/moby/moby/blob/03e2923e42446dbb830c654d0eec323a0b4ef02a/contrib/mkimage/debootstrap#L82-L105), so explicit invocation is not required
+
+
+Sample:
+
+```dockerfile
+
+RUN apt-get update \
+ && apt-get install -y --no-install-recommends \
+    # locales required if tool has any i18n support
+    locales-all \
+ && rm -rf /var/lib/apt/lists/* \
+ && python -m pip install --no-cache-dir -U \
+    build \
+    pip
+```
+
+## CMD
+
+- add a command which displays the program's help
+
+# Github Workflow
+
+## When 
+
+Publishing to the registry should be done on Github release event.
+
+```
+on:
+  release:
+    types: [published]
+```
+
+## Publish action
+
+Except if not appropriate, we recommend to use our specific Github action : [openzim/docker-publish-action](https://github.com/openzim/docker-publish-action) which handles building the image and publishing it.
+
+Some quirks not to forget:
+
+- Image name convention is to use dashes not underscores
+
+## Registry
+
+We publish our images on ghcr.io

docs/Github-Workflows.md

@@ -0,0 +1,3 @@
+- `QA.yml` runs linting and static type checks (always)
+- `Tests.yml` runs the tests on all supported Python versions. On latest version, uploads coverage to codecov. Also builds the package (to ensure build is not broken)
+- `Publish.yaml` builds the packages and uploads them to PyPI

docs/LICENSE.md

@@ -0,0 +1,5 @@
+Should be the _GNU General Public License v3.0_.
+
+We use the Github's license feature to add it (raw, complete, text).
+
+See: [`GPL-3.0`](https://github.com/github/choosealicense.com/blob/gh-pages/_licenses/gpl-3.0.txt)

docs/Policy.md

@@ -0,0 +1,27 @@
+This openZIM Python Bootstrap setup (`main` branch) represents the **current** recommendations for setting-up a Python project.
+
+- It should be used as base for any new Python project
+- It should serve as a reference to migrate older Python projects
+- Requirements for older projects are relaxed:
+  - Static Type Checking:
+    - `pyright` mode might be set to `basic` (with the `typeCheckingMode` parameter)
+    - few `# pyright: ignore` comments are allowed in the code
+  - Ruff rules:
+    - `# noqa: xxx` comments can be placed when necessary
+    - as a last resort (because it means that the situation might continue to get worse), some rule or groups might be completely commented in `pyproject.toml`
+  - Cython-using projects should continue with setuptools/setup.py
+
+## Contributing
+
+Discussions on significant improvements should happen via [Opening an Issue](https://github.com/openzim/_python-bootstrap/issues/new).
+
+Simpler suggestions are expected as [Pull Request](https://github.com/openzim/_python-bootstrap/compare)
+
+
+## Versioning
+
+The Python policy / bootstrap / convention is versioned in the main CHANGELOG.md.
+
+Tags are used to point to the various versions.
+
+Project must specify in their README.md the version they currently support to track required changes.

docs/README.md

@@ -0,0 +1,13 @@
+README (markdown format) should at least contain (in this order):
+
+- project name
+- single line, general, description
+- collection of badges:
+  - [Codefactor](https://www.codefactor.io)
+  - License
+  - Code coverage via codecov.io
+  - [PyPi](https://pypi.org)
+  - Python Version
+- Installation/usage instructions
+- Contribution guidelines (or at least a link to [Contributing](https://github.com/openzim/overview/wiki/Contributing))
+- A reference to the fact the project adheres to this boostrap conventions and current conventions version in place