Skip to content

Commit

Permalink
Remove html report (#28)
Browse files Browse the repository at this point in the history
  • Loading branch information
@alex-treebeard
alex-treebeard authored Feb 26, 2021
1 parent 9d6aef7 commit 07cd2a2
Show file tree
Hide file tree
Showing 14 changed files with 702 additions and 937 deletions.
5 changes: 2 additions & 3 deletions .github/workflows/pytest.yml
View file
Original file line number Diff line number Diff line change
Expand Up @@ -42,7 +42,7 @@ jobs:
extra_args: --verbose --all-files

- run: poetry run pytest tests/test_nb_run.py # run without extras
- run: poetry install -E html
- run: poetry install
- run: poetry run pytest --cov-report=xml --cov=src

- run: poetry run python -m ipykernel install --user --name nbmake
Expand Down Expand Up @@ -88,12 +88,11 @@ jobs:
pytest --nbmake teachopencadd/talktorials/T000_template/talktorial.ipynb \
- shell: bash -l {0}
run: |
pip install './nbmake[html]'
pip install './nbmake'
- shell: bash -l {0}
run: |
pytest \
--nbmake \
--path-output=. \
-n=auto \
teachopencadd/talktorials/T000_template/talktorial.ipynb \
teachopencadd/talktorials/T002_compound_adme/talktorial.ipynb \
Expand Down
23 changes: 23 additions & 0 deletions CONTRIBUTING.MD
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,23 @@

## Developing

### Install local package
```
poetry install
```

### Activate shell
```
poetry shell
```

### Run static checks
```
pre-commit run --all-files
pre-commit install
```

### Run tests
```
pytest
```
61 changes: 44 additions & 17 deletions README.md
View file
Original file line number Diff line number Diff line change
Expand Up @@ -23,39 +23,66 @@ pip install pytest nbmake
pytest --nbmake **/*ipynb
```

**See [docs](https://treebeardtech.github.io/nbmake) for more.**
<br/>
<br/>
## Allow errors and Configure Cell Timeouts

## See Also
nbmake is built on top of [nbclient](https://github.com/jupyter/nbclient) and designed to compatible with nbsphinx and jupyter-book.

* [nbmake-action](https://github.com/treebeardtech/nbmake-action)
The [jupyter book docs](https://jupyterbook.org/content/execute.html?highlight=allow_error#dealing-with-code-that-raises-errors) have a good description of how to ignore errors -- note that nbmake does not work with the external `_config.yml` configuration file. You must use the notebook JSON metadata field.

### HTML Report Example
## Parallelisation

![HTML Report](docs/screen.png)
Parallelisation with xdist is experimental upon initial release, but you can try it out:
```
pip install pytest-xdist
pytest --nbmake -n=auto
```

It is also possible to parallelise at a CI-level using strategies, see [example](https://github.com/LabForComputationalVision/plenoptic/blob/master/.github/workflows/treebeard.yml)

## Developing
### Build Jupyter Books Faster

Using xdist and the `--overwrite` flag let you build a large jupyter book repo faster:

### Install local package
```
poetry install -E html
pytest --nbmake --overwrite -n=auto examples
jb build examples
```
## Advice on Usage

nbmake is best used in a scenario where you use the ipynb files only for development. Consumption of notebooks is primarily done via a docs site, built through jupyter book, nbsphinx, or some other means. If using one of these tools, you are able to write assertion code in cells which will be [hidden from readers](https://jupyterbook.org/interactive/hiding.html).

### Pre-commit

Treating notebooks like source files lets you keep your repo minimal. Some tools, such as plotly may drop several megabytes of javascript in your output cells, as a result, stripping out notebooks on pre-commit is advisable:

### Activate shell
```
poetry shell
# .pre-commit-config.yaml
repos:
- repo: https://github.com/kynan/nbstripout
rev: master
hooks:
- id: nbstripout
```

### Run static checks
See https://pre-commit.com/ for more...

## Disable Nbmake

Implicitly:
```
pre-commit run --all-files
pre-commit install
pytest
```

### Run tests
Explicitly:
```
pytest
pytest -p no:nbmake
```

## See Also:

* [nbmake action](https://github.com/treebeardtech/treebeard)
* [pytest](https://pytest.org/)
* [jupyter book](https://github.com/executablebooks/jupyter-book)
* [jupyter cache](https://github.com/executablebooks/jupyter-cache)
* [MyST-NB](https://github.com/executablebooks/MyST-NB)
229 changes: 2 additions & 227 deletions docs/landing-page.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -11,234 +11,9 @@
"cell_type": "markdown",
"metadata": {},
"source": [
"nbmake tests notebook documentation and is designed for notebooks which are non-deterministic.\n",
"See the README for details: \n",
"\n",
"Despite being a pytest plugin, nbmake is kernel agnostic. It supports any language."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"## Quickstart\n",
"\n",
"```bash\n",
"pip install pytest nbmake\n",
"pytest --nbmake\n",
"```\n",
"\n",
"### Output\n",
"```\n",
"========================================================================================== test session starts ==========================================================================================\n",
"platform darwin -- Python 3.7.6, pytest-6.1.2, py-1.9.0, pluggy-0.13.1\n",
"rootdir: /Users/asdf/git/treebeardtech/nbmake, configfile: pytest.ini\n",
"plugins: nbmake-0.0.1, xdist-2.1.0, cov-2.10.1, forked-1.3.0\n",
"collected 1 item \n",
"\n",
"landing-page.ipynb . [100%]\n",
"\n",
"=========================================================================================== 1 passed in 1.56s ===========================================================================================\n",
"\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Command Line Options\n",
"\n",
"Using the following options you can control how notebooks are executed, and configure CI pipelines"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {
"tags": [
"hide-input"
]
},
"outputs": [],
"source": [
"!pytest -h | grep -A6 'notebook testing'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run Only Against IPYNB Files\n",
"\n",
"```\n",
"pytest --nbmake **/*ipynb\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Create an HTML Report\n",
"\n",
"To view outputs executed remotely, install jupyter-book:\n",
"```\n",
"pip install pytest 'nbmake[html]'\n",
"```\n",
"\n",
"Then specify a path:\n",
"```\n",
"pytest --nbmake --path-output=.\n",
"```\n",
"\n",
"### Output\n",
"```\n",
"========================================================================================== test session starts ==========================================================================================\n",
"platform darwin -- Python 3.7.6, pytest-6.1.2, py-1.9.0, pluggy-0.13.1\n",
"rootdir: /Users/asdf/git/treebeardtech/nbmake, configfile: pytest.ini\n",
"plugins: nbmake-0.0.1, xdist-2.1.0, cov-2.10.1, forked-1.3.0\n",
"collected 1 item \n",
"\n",
"landing-page.ipynb . [100%]\n",
"\n",
"\n",
"2020-12-11 11:10:48 nbmake building test report at: \n",
"\n",
" file:///Users/asdf/git/treebeardtech/nbmake/docs/_build/html/index.html\n",
"\n",
"2020-12-11 11:10:50 done.\n",
"\n",
"=========================================================================================== 1 passed in 4.26s ===========================================================================================\n",
"```\n",
"\n",
"The report contains only the stripped out and executed notebooks\n",
"![HTML report](screen.png)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Run and upload report on GitHub Actions using Netlify\n",
"\n",
"```yaml\n",
" - run: pip install pytest 'nbmake[html]'\n",
" - run: |\n",
" pytest --nbmake --path-output=.\n",
" - if: failure()\n",
" run: |\n",
" netlify deploy --dir=_build/html --auth=${{ secrets.NETLIFY_TOKEN }} --site=${{ secrets.NETLIFY_SITE_API_ID }}\n",
"```\n",
"```\n",
"...\n",
"- Waiting for deploy to go live...\n",
"✔ Deploy is live!\n",
"\n",
"Logs: https://app.netlify.com/sites/festive-payne-ce084c/deploys/5fcf58ec72dc52a440dffcd7\n",
"Website Draft URL: https://5fcf58ec72dc52a440dffcd7--festive-payne-ce084c.netlify.app\n",
"```\n",
"\n",
"Note you can also run the tests using [nbmake-action](https://github.com/treebeardtech/treebeard) for this."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Disable Nbmake\n",
"\n",
"Implicitly:\n",
"```\n",
"pytest\n",
"```\n",
"\n",
"Explicitly:\n",
"```\n",
"pytest -p no:nbmake\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Allow errors and Configure Cell Timeouts\n",
"\n",
"nbmake is designed to compatible with jupyter book config (placed in notebook `metadata`, not the global config.yml)\n",
"\n",
"See [jupyter book docs](https://jupyterbook.org/content/execute.html?highlight=allow_error#dealing-with-code-that-raises-errors)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Parallelisation\n",
"\n",
"Parallelisation with xdist is experimental upon initial release, but you can try it out:\n",
"```\n",
"pip install pytest-xdist\n",
"\n",
"pytest --nbmake -n=auto\n",
"```\n",
"\n",
"It is also possible to parallelise at a CI-level using strategies, see [example](https://github.com/LabForComputationalVision/plenoptic/blob/master/.github/workflows/treebeard.yml)\n",
"\n",
"### Build Jupyter Books Faster\n",
"\n",
"Using xdist and the `--overwrite` flag let you build a large jupyter book repo faster:\n",
"\n",
"```\n",
"pytest --nbmake --overwrite -n=auto examples\n",
"jb build examples\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Advice on Usage\n",
"\n",
"nbmake is best used in a scenario where you use the ipynb files only for development. Consumption of notebooks is primarily done via a docs site, built through jupyter book, nbsphinx, or some other means. If using one of these tools, you are able to write assertion code in cells which will be [hidden from readers](https://jupyterbook.org/interactive/hiding.html).\n",
"\n",
"### Pre-commit\n",
"\n",
"Treating notebooks like source files lets you keep your repo minimal. Some tools, such as plotly may drop several megabytes of javascript in your output cells, as a result, stripping out notebooks on pre-commit is advisable:\n",
"\n",
"```\n",
"# .pre-commit-config.yaml\n",
"repos:\n",
" - repo: https://github.com/kynan/nbstripout\n",
" rev: master\n",
" hooks:\n",
" - id: nbstripout\n",
"```\n",
"\n",
"See https://pre-commit.com/ for more..."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Contributing\n",
"\n",
"Feedback is the best contribution you can make as a user of this tool. Join the [Slack channel](https://join.slack.com/t/treebeard-entmoot/shared_invite/zt-jyvuqted-xBjnbvlfcu5P2ltBvn1~mg) and bug me (Alex), and raise an [issue](https://github.com/treebeardtech/nbmake/issues), even if you think you are the only person with this problem."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## See Also:\n",
"\n",
"* [nbmake action](https://github.com/treebeardtech/treebeard)\n",
"* [pytest](https://pytest.org/)\n",
"* [jupyter book](https://github.com/executablebooks/jupyter-book)\n",
"* [jupyter cache](https://github.com/executablebooks/jupyter-cache)\n",
"* [MyST-NB](https://github.com/executablebooks/MyST-NB)\n"
"https://github.com/treebeardtech/nbmake"
]
}
],
Expand Down
Binary file removed docs/screen.png
View file
Binary file not shown.
Loading

0 comments on commit 07cd2a2

Please sign in to comment.