Define Review Process

[mfixstsci]

There is currently no defined process for reviewing pull requests. This process is very involved for a reviewer and the contributor. When a user submits a notebook for review, it will most likely contain paths pointing to data that are stored on disk or somewhere on central store. However, for the notebooks to run on the Jenkins server, the data will need to be stored on artifactory under

https://bytesalad.stsci.edu/artifactory/webapp/#/artifacts/browse/tree/General/jwst_validation_notebooks

and will need to be retrieved using ci_watson.get_bigdata for single data files or leverage jwst.regtest.regtestdata for jwst asn files. The data can be manually added by someone who has permissions to write to the artifactory repository, whether it is the contributor, reviewer, or someone in DMD.

There are examples of how to use ci_watson and regtestdata in the resample notebook.

Once the data is stored in it's correct location on artifactory and the changes are made in the notebooks pointing to the data on artifactory using ci_watson or regtestdata, the user will need to test that the notebook successfully pulls the data from artifactory and the notebook completes successfully.

Once this is done, the contributor will need to clear the cells of the notebook and the restart the kernel before pushing more changes back to the pull request.

Reviewers should make sure notebooks are being submitted to master with no executed cells, executed notebooks take up a lot of space on your github repository!

The reviewer can verify and merge the notebook into master and the next time the build is run, the html will be generated and will be served via https://jwst-validation-notebooks.stsci.edu/

NOTE: A future goal of this repository is to fold it into the notebooks repository to show users some more in depth analysis of how the pipeline is performing. Once JWST is taking data and we can make calls to astroquery for data, rather than relying on artifactory and adding the data manually.