We recommend using pyenv to manage multiple versions of Python on your system. Refer to the installation guide for your system.
Install Python 3.9 if haven't done so already
pyenv install 3.9Verify that it's the version selected by pyenv for the project.
python -V
Python 3.9If you prefer to use Python 3.9 only for this project, you can run the following command to generate a .python_version file.
pyenv local 3.9Some Python packages require external dependencies to build on macOS, e.g. pymssql. You can use homebrew to install these:
brew install freetdsInstall poetry package manager
curl -sSL https://raw.githubusercontent.com/python-poetry/poetry/master/get-poetry.py | python -Use poetry to setup virtual env
poetry install -E allActivate virtual env during development
poetry shellTo deactivate the virtual env, simply run
exitTo make life easier, we recommend to always develop in a poetry shell. Many of the following commands will only work inside a poetry shell as well.
Install pre-commit then setup git hooks
pre-commit installEach extractor module can be directly invoked from command line during development. The extractor takes a single config_file argument, e.g.
python -m metaphor.postgresql config/postgres.jsonWe use mypy to type-check Python code.
Pre-commit hook is setup to automatically type-check all Python files before committing. You can manually invoke it inside poetry shell,
mypy .We use black && isort to format Python code in an opinionated way.
Pre-commit hook is setup to automatically format all Python files before committing.
We use flake8 linter to check the style and quality of some python code. You can invoke the linter using the following command
flake8 .The PyPI package and docker image are automatically built and published as part of the CD workflow. Please make sure to bump up the version in pyproject.toml along with the PR to trigger the publishing workflow.