From 21b9768a1ab6b3dc3aff3e077c9612d6fed21721 Mon Sep 17 00:00:00 2001 From: Drew Oldag Date: Tue, 21 Jan 2025 21:20:55 -0800 Subject: [PATCH] Adding scaffolding for documentation pages. --- docs/architecture_overview.rst | 32 +++++++++++++++ docs/conf.py | 1 + docs/configuration.rst | 12 ++++++ docs/data_set_splits.rst | 13 ++++++ docs/dev_guide.rst | 38 ++++++++++++++++++ docs/external_libraries.rst | 7 ++++ docs/index.rst | 73 ++++++++++++++-------------------- docs/model_evaluation.rst | 28 +++++++++++++ docs/requirements.txt | 3 +- 9 files changed, 163 insertions(+), 44 deletions(-) create mode 100644 docs/architecture_overview.rst create mode 100644 docs/data_set_splits.rst create mode 100644 docs/dev_guide.rst create mode 100644 docs/external_libraries.rst create mode 100644 docs/model_evaluation.rst diff --git a/docs/architecture_overview.rst b/docs/architecture_overview.rst new file mode 100644 index 0000000..e4e2ebd --- /dev/null +++ b/docs/architecture_overview.rst @@ -0,0 +1,32 @@ +Architecture overview +===================== + +Fibad uses verbs +---------------- +Fibad defines a set of commands, called verbs, that are the primary mode of interaction. +Verbs are meant to be intuitive and easy to remember. For instance, to train a model, +you would use the ``train`` verb. +To use a trained model for inference, you would use the ``infer`` verb. + +Notebook, CLI, or Both +-------------------------------- +Fibad is designed to be used in a Jupyter notebook or from the command line without +modification. This supports exploration and development in a familiar notebook environment +and deployment to an HPC or Slurm system for large scale training. + +.. tabs:: + + .. group-tab:: Notebook + + .. code-block:: python + + from fibad import Fibad + + fibad_instance = Fibad(config_file = 'my_config.toml') + fibad_instance.train() + + .. group-tab:: CLI + + .. code-block:: bash + + >> fibad train -c my_config.toml diff --git a/docs/conf.py b/docs/conf.py index b752faf..32e4517 100644 --- a/docs/conf.py +++ b/docs/conf.py @@ -28,6 +28,7 @@ extensions.append("autoapi.extension") extensions.append("nbsphinx") +extensions.append("sphinx_tabs.tabs") # -- sphinx-copybutton configuration ---------------------------------------- extensions.append("sphinx_copybutton") diff --git a/docs/configuration.rst b/docs/configuration.rst index 7c27442..34f97f4 100644 --- a/docs/configuration.rst +++ b/docs/configuration.rst @@ -1,5 +1,17 @@ Configuration ============= + +The configuration system +------------------------ +Fibad uses configuration files in the TOML syntax to manage runtime settings. +A choice was made early on to limit the number of command line arguments to support +reproducibility and clarity in the code. + + + +The default configuration file +------------------------------ + The following is the default FIBAD runtime configuration file. .. literalinclude:: ../src/fibad/fibad_default_config.toml diff --git a/docs/data_set_splits.rst b/docs/data_set_splits.rst new file mode 100644 index 0000000..a754365 --- /dev/null +++ b/docs/data_set_splits.rst @@ -0,0 +1,13 @@ +How to define subsets of data +============================= + +Talk about the approach we've taken with train, validate, and test. + +Perhaps a flow chart would be good here. + +Describe how this can be used as percentages or absolute numbers. + +Describe how the splits can be used to accomplish different modes of training. +i.e. train/validate then infer on test splits, +train for epochs, validate after each, then run test set at the very end. +Only train for epochs (no validation) then infer on test split... etc. \ No newline at end of file diff --git a/docs/dev_guide.rst b/docs/dev_guide.rst new file mode 100644 index 0000000..cc97a0d --- /dev/null +++ b/docs/dev_guide.rst @@ -0,0 +1,38 @@ +Developer guide +=============== + +Getting Started +--------------- + +Before installing any dependencies or writing code, it's a great idea to create a +virtual environment. LINCC-Frameworks engineers primarily use `conda` to manage virtual +environments. If you have conda installed locally, you can run the following to +create and activate a new environment. + +.. code-block:: console + + >> conda create env -n python=3.10 + >> conda activate + + +Once you have created a new environment, you can install this project for local +development using the following commands: + +.. code-block:: console + + >> pip install -e .'[dev]' + >> pre-commit install + >> conda install pandoc + + +Notes: + +1) The single quotes around ``'[dev]'`` may not be required for your operating system. +2) ``pre-commit install`` will initialize pre-commit for this local repository, so + that a set of tests will be run prior to completing a local commit. For more + information, see the Python Project Template documentation on + `pre-commit `_. +3) Installing ``pandoc`` allows you to verify that automatic rendering of Jupyter notebooks + into documentation for ReadTheDocs works as expected. For more information, see + the Python Project Template documentation on + `Sphinx and Python Notebooks `_. diff --git a/docs/external_libraries.rst b/docs/external_libraries.rst new file mode 100644 index 0000000..495e393 --- /dev/null +++ b/docs/external_libraries.rst @@ -0,0 +1,7 @@ +External libraries +================== + +Fibad supports external libraries for models and data sets. + +How to specify that Fibad should use an external library using the config file. + diff --git a/docs/index.rst b/docs/index.rst index 24bf563..c63c04b 100644 --- a/docs/index.rst +++ b/docs/index.rst @@ -1,52 +1,39 @@ - -.. fibad documentation main file. - You can adapt this file completely to your liking, but it should at least - contain the root `toctree` directive. - Welcome to fibad's documentation! -======================================================================================== - -Dev Guide - Getting Started ---------------------------- - -Before installing any dependencies or writing code, it's a great idea to create a -virtual environment. LINCC-Frameworks engineers primarily use `conda` to manage virtual -environments. If you have conda installed locally, you can run the following to -create and activate a new environment. - -.. code-block:: console - - >> conda create env -n python=3.10 - >> conda activate - - -Once you have created a new environment, you can install this project for local -development using the following commands: - -.. code-block:: console - - >> pip install -e .'[dev]' - >> pre-commit install - >> conda install pandoc - - -Notes: - -1) The single quotes around ``'[dev]'`` may not be required for your operating system. -2) ``pre-commit install`` will initialize pre-commit for this local repository, so - that a set of tests will be run prior to completing a local commit. For more - information, see the Python Project Template documentation on - `pre-commit `_. -3) Installing ``pandoc`` allows you to verify that automatic rendering of Jupyter notebooks - into documentation for ReadTheDocs works as expected. For more information, see - the Python Project Template documentation on - `Sphinx and Python Notebooks `_. - +================================= + +What is fibad? +-------------- +Fibad is the Framework for Image-Base Anomaly Detection. + +Why did we build fibad? +----------------------- +Image-based ML in astronomy is challenging work. +We've found many bottlenecks in the process that require signifincant effort to overcome. +Most of the time that effort doesn't accrue to science, it's just a means to an end. +But it's repeated over and over again by many different people. +Fibad is our effort to make the process easier and more efficient taking care of +the common tasks so that scientists can focus on the science. + +Fibad's guiding principles +-------------------------- +* **Principle 1** Empower the scientists to do science - not software engineering. +Fibad automatically discovers and uses the most performant hardware available +for training without any changes to the users code. +* **Principle 2** Make the software easy to use. +Fibad is designed to be used in a Jupyter notebook for exploration or from the +command line within HPC or Slurm environments without modification. +* **Principle 3** Make the software extensible to support many different use cases. +Fibad is designed to be easily extended to support new models and data sources. .. toctree:: :hidden: Home page + Architecture overview Configuration + External libraries + Data set splits + Model evaluation + Developer guide API Reference Notebooks diff --git a/docs/model_evaluation.rst b/docs/model_evaluation.rst new file mode 100644 index 0000000..895e2e6 --- /dev/null +++ b/docs/model_evaluation.rst @@ -0,0 +1,28 @@ +Model evaluation +================ + +One goal of fibad is to make model evaluation easier. Many tools exist for visualization +and evaluation of models. Fibad integrates with tensorboard and MLFlow to provide +easy access to these tools. + +Tensorboard +----------- + +Fibad automatically logs training, validation and gpu metrics to tensorboard while +training a model. This allows for easy visualization of the training process. + +For more info about tensorboard see + + +MLFlow +------ + +Fibad supports MLFlow for model tracking and experiment management. +Talk about the defaults that were selected (experiment_name = notebook) and how +to get MLFlow started at the command line. +``mlflow ui --port 5000 --backend-store-uri `` + +For more info about MLFlow see + + + diff --git a/docs/requirements.txt b/docs/requirements.txt index ee05654..8b33569 100644 --- a/docs/requirements.txt +++ b/docs/requirements.txt @@ -7,4 +7,5 @@ nbsphinx sphinx sphinx-autoapi sphinx-copybutton -sphinx-rtd-theme \ No newline at end of file +sphinx-rtd-theme +sphinx-tabs \ No newline at end of file