Skip to content

Commit 1bf5eb2

Browse files
rjavadifcogidiamrit110
authored
Improve documentation for evaluate package (#543)
* Changed documentation theme to sphinx-book-theme * Added details about Pandoc requirment * Added initial document for accuracy.py * Fix pre-commit hook errors * Added confusion matrix doc * Add auroc doc and fix titles * Add average precision docs * Added more documents for metrics * Add index for user guide in side bar * Minor patches (#542) * Fix capitalization of Python * Change ValueError to TypeError and update docstrings * Add Precision-Recall Curve to experimental metrics (#544) * Add ROC curve (#545) * Add ROC metrics to experimental module * fix typo * Fix docstrings * Add AUROC (#546) * Add AUROC metric to experimental module * Refactor binary and multiclass ROC functions * Refactor tests to use a common thresholds list * Fix mypy error * Add regression metrics (#547) * Add regression support * update docstrings * Ignore `no-any-return` errors * Integrate experimental metrics with other modules (#549) * integrate experimental metrics with other modules * add average precision metric to experimental metrics package * fix tutorials * Add type hints and keyword arguments to metrics classes * Update nbsphinx version to 0.9.3 * Update nbconvert version to 7.14.2 * Fix type annotations and formatting issues * Update kernel display name in mortality_prediction.ipynb * Add guard clause to prevent module execution on import * Update `torch_distributed.py` with type hints * Add multiclass and multilabel average precision metrics * Change jupyter kernel * Fix type annotations for metric values in ClassificationPlotter --------- Co-authored-by: Amrit K <amritk@vectorinstitute.ai> * Changed documentation theme to sphinx-book-theme * Don't update deps * Pre-commit fixes * Add doc for ROC Curve * Fix footer in docs * Bump to 0.2.3 Signed-off-by: Amrit Krishnan <amrit110@gmail.com> * Add logo and css fixes * Fix formatting of conf.py file * Small fix * Move evaluation metrics docs under API reference * Fix poetry build * Improve documentation structure * Formatting fixes --------- Signed-off-by: Amrit Krishnan <amrit110@gmail.com> Co-authored-by: Franklin <41602287+fcogidi@users.noreply.github.com> Co-authored-by: Amrit K <amritk@vectorinstitute.ai> Co-authored-by: Amrit Krishnan <amrit110@gmail.com>
1 parent b9f3876 commit 1bf5eb2

Some content is hidden

Large Commits have some content hidden by default. Use the searchbox below for content that may be hidden.

44 files changed

+810
-841
lines changed

‎README.md

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -89,6 +89,8 @@ guidelines.
8989

9090
## 📚 [Documentation](https://vectorinstitute.github.io/cyclops/)
9191

92+
If you need to build the documentations locally, make sure to install ``Pandoc`` in addition to ``docs`` poetry group.
93+
9294

9395
## 📓 Notebooks
9496

‎docs/source/_static/css/cyclops.css

Lines changed: 50 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,50 @@
1+
footer span.commit code,
2+
.rst-content pre.literal-block,
3+
.rst-content div[class^='highlight'] pre,
4+
.rst-content .linenodiv pre,
5+
.rst-content tt,
6+
.rst-content code,
7+
.rst-content pre,
8+
.rst-content kbd,
9+
.rst-content samp {
10+
font-family: 'IBM Plex Mono', monospace;
11+
font-size: 0.8rem;
12+
}
13+
14+
footer {
15+
color: var(--text-color);
16+
}
17+
footer .footer-small-text {
18+
font-weight: 300;
19+
font-size: 0.9rem;
20+
}
21+
footer .copyright {
22+
font-weight: 300;
23+
font-size: 0.8rem;
24+
}
25+
footer div.logo {
26+
display: flex;
27+
flex-wrap: wrap;
28+
justify-content: space-around;
29+
margin: 0px;
30+
padding: 10px 0px 0px 0px;
31+
}
32+
footer a.logo {
33+
/* Using flex here (to vertically align the child img) causes aspect-ratio issues */
34+
flex-basis: 120px;
35+
margin: 10px auto 10px auto;
36+
text-align: center;
37+
}
38+
footer a.logo:hover {
39+
text-decoration: none;
40+
}
41+
footer span.logo {
42+
display: inline-block;
43+
height: 100%;
44+
vertical-align: middle;
45+
}
46+
footer img.logo {
47+
display: inline-block;
48+
vertical-align: middle;
49+
height: auto;
50+
}

‎docs/source/theme/static/cyclops_logo-dark.png renamed to ‎docs/source/_static/cyclops_logo-dark.png

File renamed without changes.

‎docs/source/theme/static/cyclops_logo.png renamed to ‎docs/source/_static/cyclops_logo.png

File renamed without changes.

‎docs/source/theme/static/favicon.ico renamed to ‎docs/source/_static/favicon.ico

File renamed without changes.

‎docs/source/theme/static/logos/gemini_logo.png renamed to ‎docs/source/_static/logos/gemini_logo.png

File renamed without changes.

‎docs/source/theme/static/logos/vector_logo.png renamed to ‎docs/source/_static/logos/vector_logo.png

File renamed without changes.

‎docs/source/_templates/sections/footer.html

Lines changed: 26 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,26 @@
1+
{% block extrafooter %}
2+
3+
<div class="footer-container" style="text-align: center;">
4+
<div class="footer-small-text">
5+
<br> All <a href="https://github.com/VectorInstitute/cyclops">source code for cyclops</a> is freely available under the terms of an <a href="https://opensource.org/licenses/Apache-2.0">Apache-2.0</a> license.
6+
</div>
7+
8+
<div class="footer-small-text">
9+
This work is made possible due to the data obtained from the <a href="https://www.geminimedicine.ca/">GEMINI study</a>.
10+
</div>
11+
12+
<hr/>
13+
14+
<div class="logo">
15+
<a class="logo" href="https://vectorinstitute.ai/">
16+
<span class="logo"></span>
17+
<img alt="Logo of the Vector Institute" style="width:90px; max-width:90px;" src="{{ pathto('_static/logos/vector_logo.png', 1) }}" />
18+
</a>
19+
<a class="logo" href="https://www.geminimedicine.ca/">
20+
<span class="logo"></span>
21+
<img alt="Logo of GEMINI" style="width:90px; max-width:90px;" src="{{ pathto('_static/logos/gemini_logo.png', 1) }}" />
22+
</a>
23+
</div>
24+
</div>
25+
26+
{% endblock %}

‎docs/source/api.rst

Lines changed: 5 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -2,9 +2,13 @@ API Reference
22
=============
33

44
.. toctree::
5+
:maxdepth: 1
6+
:glob:
57

68
reference/api/cyclops.data.rst
79
reference/api/cyclops.tasks.rst
8-
reference/api/cyclops.evaluate.rst
10+
reference/api/evaluator.rst
11+
reference/api/fairness_evaluator.rst
12+
reference/api/metrics.rst
913
reference/api/cyclops.monitor.rst
1014
reference/api/cyclops.report.rst

‎docs/source/conf.py

Lines changed: 19 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -59,6 +59,14 @@
5959
copybutton_prompt_text = r">>> |\.\.\. "
6060
copybutton_prompt_is_regexp = True
6161

62+
html_theme_options = {
63+
"collapse_navigation": False,
64+
"sticky_navigation": True,
65+
"navigation_depth": 4,
66+
"includehidden": True,
67+
"titles_only": False,
68+
}
69+
6270
intersphinx_mapping = {
6371
"python": ("https://docs.python.org/3.9/", None),
6472
"numpy": ("http://docs.scipy.org/doc/numpy/", None),
@@ -84,10 +92,19 @@
8492
# The theme to use for HTML and HTML Help pages. See the documentation for
8593
# a list of builtin themes.
8694
#
87-
html_theme = "theme"
88-
html_theme_path = ["."]
95+
html_theme = "sphinx_book_theme"
8996

9097
# Add any paths that contain custom static files (such as style sheets) here,
9198
# relative to this directory. They are copied after the builtin static files,
9299
# so a file named "default.css" will overwrite the builtin "default.css".
93100
html_static_path = ["_static"]
101+
html_css_files = ["css/cyclops.css"]
102+
html_logo = "_static/cyclops_logo-dark.png"
103+
html_favicon = "_static/favicon.ico"
104+
html_theme_options = {
105+
"logo": { # type: ignore
106+
"text": "cyclops documentation",
107+
"image_light": "_static/cyclops_logo-dark.png",
108+
"image_dark": "_static/cyclops_logo-dark.png",
109+
},
110+
}

‎docs/source/developing.rst

Lines changed: 34 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,34 @@
1+
🧑🏿‍💻 Developing
2+
=======================
3+
4+
Using poetry
5+
------------
6+
7+
The development environment can be set up using `poetry <https://python-poetry.org/docs/#installation>`__. Hence, make sure it is installed and then run:
8+
9+
.. code:: bash
10+
11+
python3 -m poetry install
12+
source $(poetry env info --path)/bin/activate
13+
14+
In order to install dependencies for testing (codestyle, unit tests, integration tests), run:
15+
16+
.. code:: bash
17+
18+
python3 -m poetry install --with test
19+
20+
API documentation is built using `Sphinx <https://www.sphinx-doc.org/en/master/>`__ and can be locally built by:
21+
22+
.. code:: bash
23+
24+
python3 -m poetry install --with docs
25+
cd docs
26+
make html SPHINXOPTS="-D nbsphinx_allow_errors=True"
27+
28+
29+
If you need to build the documentations locally, make sure to install ``Pandoc`` in addition to ``docs`` poetry group.
30+
31+
Contributing
32+
------------
33+
34+
Contributing to cyclops is welcomed. See `Contributing <https://vectorinstitute.github.io/cyclops/api/contributing.html>`__ for guidelines.

‎docs/source/evaluation.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Evaluation
2+
==========

‎docs/source/index.rst

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -6,7 +6,8 @@ Welcome to cyclops's documentation!
66
:maxdepth: 2
77
:caption: Contents:
88

9-
intro
10-
contributing
9+
user_guide
1110
tutorials
11+
developing
12+
contributing
1213
api

‎docs/source/installation.rst

Lines changed: 29 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,29 @@
1+
Installation
2+
============
3+
4+
Using pip
5+
---------
6+
7+
.. code:: bash
8+
9+
python3 -m pip install pycyclops
10+
11+
``cyclops`` has many optional dependencies that are used for specific functionality. For example, the `monai <https://github.com/Project-MONAI/MONAI>`__ library is used for loading DICOM images to create datasets. All optional dependencies can be installed with ``pycyclops[all]``, and specific sets of dependencies are listed in the sections below.
12+
13+
+-----------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------+
14+
| Dependency | pip extra | Notes |
15+
+=============================+==========================+===============================================================================================================+
16+
| xgboost | xgboost | Allows use of `XGBoost <https://xgboost.readthedocs.io/en/stable/>`__ model |
17+
+-----------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------+
18+
| torch | torch | Allows use of `PyTorch <https://pytorch.org/>`__ models |
19+
+-----------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------+
20+
| torchvision | torchvision | Allows use of `Torchvision <https://pytorch.org/vision/stable/index.html>`__ library |
21+
+-----------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------+
22+
| torchxrayvision | torchxrayvision | Uses `TorchXRayVision <https://mlmed.org/torchxrayvision/>`__ library |
23+
+-----------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------+
24+
| monai | monai | Uses `MONAI <https://github.com/Project-MONAI/MONAI>`__ to load and transform images |
25+
+-----------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------+
26+
| alibi | alibi | Uses `Alibi <https://docs.seldon.io/projects/alibi/en/stable/>`__ for additional explainability functionality |
27+
+-----------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------+
28+
| alibi-detect | alibi-detect | Uses `Alibi Detect <https://docs.seldon.io/projects/alibi-detect/en/stable/>`__ for dataset shift detection |
29+
+-----------------------------+--------------------------+---------------------------------------------------------------------------------------------------------------+

‎docs/source/intro.rst

Lines changed: 0 additions & 130 deletions
This file was deleted.

‎docs/source/model_report.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Model Report
2+
============

‎docs/source/monitoring.rst

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,2 @@
1+
Monitoring
2+
==========

0 commit comments

Comments
 (0)