-
Notifications
You must be signed in to change notification settings - Fork 35
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Merge pull request #31 from ukgovdatascience/tidy-repository
Add hooks, add tests, improve functionality, refactor documentation, and add GitHub Actions
- Loading branch information
Showing
91 changed files
with
2,412 additions
and
1,188 deletions.
There are no files selected for viewing
This file was deleted.
Oops, something went wrong.
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
15 changes: 8 additions & 7 deletions
15
...rtment_frameworks/GDS/request_template.md → .github/pull_request_template.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,30 +1,31 @@ | ||
| # Summary | ||
|
|
||
| Add your summary here - keep it brief, to the point, and in plain English. For further information about pull requests, | ||
| check out the [GDS Way][gds-way]. | ||
| Add your summary here - keep it brief, to the point, and in plain English. For further | ||
| information about pull requests, check out the | ||
| [GDS Way](https://gds-way.cloudapps.digital/standards/pull-requests.html). | ||
|
|
||
| # Checklists | ||
|
|
||
| <!-- | ||
| These are DO-CONFIRM checklists; it CONFIRMs that you have DOne each item. | ||
| Outstanding actions should be completed before reviewers are assigned; if actions are irrelevant, please try and add a | ||
| comment stating why. | ||
| Outstanding actions should be completed before reviewers are assigned; if actions are | ||
| irrelevant, please try and add a comment stating why. | ||
| Incomplete pull/merge requests MAY be blocked until actions are resolved, or closed at the reviewers' discretion. | ||
| Incomplete pull/merge requests MAY be blocked until actions are resolved, or closed at | ||
| the reviewers' discretion. | ||
| --> | ||
|
|
||
| This pull/merge request meets the following requirements: | ||
|
|
||
| - [ ] Code runs | ||
| - [ ] Developments are **secure** and [**ethical**][data-ethics-framework] | ||
| - [ ] You have made proportionate checks that the code works correctly | ||
| - [ ] You have tested the build of the template | ||
| - [ ] Test suite passes | ||
| - [ ] Assumptions, and caveats log updated (see `docs/aqa/assumptions_caveats.md`), if necessary | ||
| - [ ] [Minimum usable documentation][agilemodeling] written in the `docs` folder | ||
|
|
||
| Comments have been added below around the incomplete checks. | ||
|
|
||
| [agilemodeling]: http://agilemodeling.com/essays/documentLate.htm | ||
| [data-ethics-framework]: https://www.gov.uk/government/publications/data-ethics-framework | ||
| [gds-way]: https://gds-way.cloudapps.digital/standards/pull-requests.html |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,63 @@ | ||
| name: govcookiecutter build | ||
|
|
||
| on: [ push ] | ||
|
|
||
| jobs: | ||
| build: | ||
|
|
||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| matrix: | ||
| os: [ ubuntu-latest, macos-latest, windows-latest ] | ||
| python: [ 3.6, 3.7, 3.8, 3.9 ] | ||
|
|
||
| steps: | ||
| - uses: actions/checkout@v2 | ||
| - name: Set up Python ${{ matrix.python }} | ||
| uses: actions/setup-python@v2 | ||
| with: | ||
| python-version: ${{ matrix.python }} | ||
| - name: Install dependencies | ||
| run: | | ||
| if [[ "$RUNNER_OS" == "Linux" || "$RUNNER_OS" == "macOS" ]]; then | ||
| make requirements | ||
| elif [ "$RUNNER_OS" == "Windows" ]; then | ||
| pip install -U pip setuptools | ||
| pip install -r requirements.txt | ||
| pre-commit install | ||
| else | ||
| echo "$RUNNER_OS not supported" | ||
| exit 1 | ||
| fi; | ||
| shell: bash | ||
| - name: Run pre-commit hooks | ||
| run: pre-commit run --all-files | ||
| - name: Create documentation | ||
| run: | | ||
| if [[ "$RUNNER_OS" == "Linux" || "$RUNNER_OS" == "macOS" ]]; then | ||
| sphinx-build -W -b html ./docs ./docs/_build | ||
| elif [ "$RUNNER_OS" == "Windows" ]; then | ||
| # TODO: Investigate why Sphinx build raises a warning on Windows but not Unix | ||
| sphinx-build -b html ./docs ./docs/_build | ||
| else | ||
| echo "$RUNNER_OS not supported" | ||
| exit 1 | ||
| fi; | ||
| shell: bash | ||
| - name: Execute tests, and create coverage report | ||
| run: | | ||
| if [[ "$RUNNER_OS" == "Linux" || "$RUNNER_OS" == "macOS" ]]; then | ||
| make coverage_xml | ||
| elif [ "$RUNNER_OS" == "Windows" ]; then | ||
| coverage run -m pytest | ||
| coverage xml | ||
| else | ||
| echo "$RUNNER_OS not supported" | ||
| exit 1 | ||
| fi; | ||
| shell: bash | ||
| - name: Upload coverage report to Codecov | ||
| uses: codecov/codecov-action@v1 | ||
| with: | ||
| files: ./coverage.xml | ||
| env_vars: OS=${{ matrix.os }},PYTHON=${{ matrix.python }} |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,74 @@ | ||
| name: govcookiecutter template build | ||
|
|
||
| on: [ pull_request ] | ||
|
|
||
| jobs: | ||
| template-build: | ||
|
|
||
| runs-on: ${{ matrix.os }} | ||
| strategy: | ||
| matrix: | ||
| # TODO: Fix R pre-commit hook issues with Windows | ||
| os: [ ubuntu-latest, macos-latest ] | ||
| python: [ 3.6, 3.7, 3.8, 3.9 ] | ||
| R: | ||
| - {using_R: No, version: N/A} | ||
| - {using_R: Yes, version: 4.0.4} | ||
| - {using_R: Yes, version: 4.0.5} | ||
| - {using_R: Yes, version: 4.1.0} | ||
|
|
||
| steps: | ||
| - name: Set up Python ${{ matrix.python }} | ||
| uses: actions/setup-python@v2 | ||
| with: | ||
| python-version: ${{ matrix.python }} | ||
| - name: Install R ${{ matrix.R.version }} | ||
| uses: r-lib/actions/setup-r@v1 | ||
| with: | ||
| r-version: ${{ matrix.R.version }} | ||
| if: ${{ matrix.R.using_r == 'Yes' }} | ||
| - name: Install other ${{ matrix.os }} R dependencies | ||
| run: sudo apt-get install libcurl4-openssl-dev | ||
| if: ${{ matrix.os == 'ubuntu-latest' && matrix.R.using_R == 'Yes' }} | ||
| - name: Install cookiecutter | ||
| run: | | ||
| if [ "$RUNNER_OS" == "Linux" ]; then | ||
| # TODO: Add pip alternative, as Ubuntu 20.04LTS not currently supported (June 2021) | ||
| sudo apt-get install cookiecutter || pip install cookiecutter | ||
| elif [ "$RUNNER_OS" == "macOS" ]; then | ||
| brew install cookiecutter | ||
| elif [ "$RUNNER_OS" == "Windows" ]; then | ||
| pip install cookiecutter | ||
| else | ||
| echo "$RUNNER_OS not supported" | ||
| exit 1 | ||
| fi; | ||
| shell: bash | ||
| - name: Create project from govcookiecutter for this commit | ||
| run: | | ||
| # Relies on the repository being public | ||
| cookiecutter ${GITHUB_SERVER_URL}/${GITHUB_REPOSITORY}.git --no-input --checkout ${GITHUB_HEAD_REF} repo_name=example using_R=${{ matrix.R.using_R }} | ||
| shell: bash | ||
| - name: Install requirements, and pre-commit hooks, and check Sphinx build | ||
| run: | | ||
| cd example | ||
| git init | ||
| git add . | ||
| if [[ "$RUNNER_OS" == "Linux" || "$RUNNER_OS" == "macOS" ]]; then | ||
| make docs | ||
| elif [ "$RUNNER_OS" == "Windows" ]; then | ||
| pip install -U pip setuptools | ||
| pip install -r requirements.txt | ||
| pre-commit install | ||
| sphinx-build -b html ./docs ./docs/_build | ||
| else | ||
| echo "$RUNNER_OS not supported" | ||
| exit 1 | ||
| fi; | ||
| shell: bash | ||
| - name: Run pre-commit hooks | ||
| run: | | ||
| cd example | ||
| pre-commit run --all-files | ||
| # TODO: Currently no tests - this gives exit code 5, which fails CI, so commenting out for now | ||
| # pytest |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -1,28 +1,38 @@ | ||
| # See https://pre-commit.com for more information | ||
| # See https://pre-commit.com/hooks.html for more hooks | ||
| repos: | ||
| - repo: https://gitlab.com/pycqa/flake8 | ||
| rev: 3.8.4 | ||
| - repo: https://github.com/pre-commit/pre-commit-hooks | ||
| rev: v4.0.1 | ||
| hooks: | ||
| - id: flake8 | ||
| args: ["hooks", "tests", "{{ cookiecutter.repo_name }}/src"] | ||
| - repo: https://github.com/Yelp/detect-secrets | ||
| - id: check-added-large-files | ||
| name: Check for files larger than 5 MB | ||
| args: [ "--maxkb=5120" ] | ||
| - id: end-of-file-fixer | ||
| name: Check for a blank line at the end of scripts (auto-fixes) | ||
| - id: trailing-whitespace | ||
| name: Check for trailing whitespaces (auto-fixes) | ||
| - repo: https://github.com/pycqa/isort | ||
| rev: 5.8.0 | ||
| hooks: | ||
| - id: isort | ||
| name: isort - Sort Python imports (auto-fixes) | ||
| types: [ cython, pyi, python ] | ||
| args: [ "--profile", "black", "--filter-files" ] | ||
| - repo: https://github.com/psf/black | ||
| rev: 21.5b2 # Replace by any tag/version: https://github.com/psf/black/tags | ||
| hooks: | ||
| - id: black | ||
| name: black - consistent Python code formatting (auto-fixes) | ||
| language_version: python # Should be a command that runs python3.6+ | ||
| - repo: https://gitlab.com/pycqa/flake8 | ||
| rev: 3.9.2 | ||
| hooks: | ||
| - id: flake8 | ||
| name: flake8 - Python linting | ||
| - repo: https://github.com/Yelp/detect-secrets | ||
| rev: v1.0.3 | ||
| hooks: | ||
| - id: detect-secrets | ||
| args: ["--baseline", ".secrets.baseline"] | ||
| - id: detect-secrets | ||
| name: detect-secrets - Detect secrets in staged code | ||
| args: [ "--baseline", ".secrets.baseline" ] | ||
| exclude: .*/tests/.* | ||
| - repo: https://github.com/kynan/nbstripout | ||
| rev: 0.3.9 | ||
| hooks: | ||
| - id: nbstripout | ||
| args: | ||
| - --extra-keys | ||
| - "metadata.colab metadata.kernelspec cell.metadata.colab cell.metadata.executionInfo cell.metadata.id cell.metadata.outputId" | ||
| - repo: https://github.com/pre-commit/pre-commit-hooks | ||
| rev: v3.4.0 | ||
| hooks: | ||
| - id: check-added-large-files | ||
| args: ["--maxkb=5120"] | ||
| - id: end-of-file-fixer | ||
| - id: trailing-whitespace |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
|
|
@@ -90,5 +90,5 @@ | |
| } | ||
| ], | ||
| "results": {}, | ||
| "generated_at": "2021-03-04T17:23:56Z" | ||
| "generated_at": "2021-06-14T10:43:14Z" | ||
| } | ||
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Oops, something went wrong.