Skip to content

Latest commit

 

History

History
54 lines (45 loc) · 4.88 KB

MAINTENANCE.md

File metadata and controls

54 lines (45 loc) · 4.88 KB
Raw

turbo_seti Maintenance & Regression Testing

Introduction

The purpose of the regression testing suite is to exercise and validate results from turbo_seti functional modules. This is important in order to minimize potential inadvertent breakage when new development has occured. It is always best to catch bugs as soon as possible after they are introduced.
The primary method of launching regression testing is through the use of the pytest executable. This is invoked in the following ways:

  • Manually by a developer, on the command line in a terminal window. This would follow downloading turbo_seti and setting up the development/testing environment (discussed later).
  • Automatically as part of a Github Pull Request (PR) after finalizing a fork of turbo_seti.
  • Automatically as part of a Github Merge after a PR is approved.

Development Process Overview

  • The development of an amendment to turbo_seti begins with taking a fork from a github site, normally from https://github.com/UCBerkeleySETI/turbo_seti.
  • Also, from the same site, turbo_seti is downloaded to a local computer. The download operations can be performed in a few different ways but the simplest might be to download the zip file by clicking on the Code button and selecting Download ZIP. Once the the zip file is in a local directory, unzip it and move the turbo_seti directory tree to wherever is appropriate for testing. The zip file can now be discarded.
  • Change directory into the test directory and execute python3 download_test_data.py which will perform all required regression testing initialization.
  • When the previous step has completed, change directory up one level to the top of the turbo_seti directory tree.
  • Execute: python3 -m pip install -U git+https://github.com/UCBerkeleySETI/blimpy
  • Execute: python3 setup.py install
  • Execute: python3 -m pip install -r requirements_test.txt
  • Regression testing can now begin.
  • Running the full suite of regression tests is invoked by executing pytest with no parameters specified. It is possible to run a single regression test file by specifying it as an argument to pytest. For example, if one wishes to only run the find event tests, the following is the command line to use: pytest test/test_find_event.py.
  • It is highly encouraged for developers to perform regression testing frequently in order to avoid surprises later on.
  • Once, development activity on the local machine is complete and the last regression test has run verifying the absence of negative side effects, then the new and/or modified turbo_seti files can be uploaded to the developer's fork github site.
  • Be sure to bump the version in setup.py and add an entry to VERSION-HISTORY.md.
  • At the fork github site, the developer can request a pull clicking on the Pull request button. This automatically starts the PR process mentioned in the introduction section.

Testing Artifacts

File Description
download_test_data.py Initialization of regression testing by obtaining and producing files used in test suite execution.
fb_cases_def.py Data definitions for test_fb_cases.py.
fb_cases_util.py Utility functions for test_fb_cases.py and test_pipelines_{1,2,3}.
fb_dat_reference.txt Reference data generated by fb_genref.py, hopefully a one-time event.
fb_genref.py Hopefully, a one-time generator of fb_dat_reference.txt which is used by test_fb_cases.py to validate test results. The reference data values came from an execution of turbo_seti version 1.3.0.
pipelines_util.py Utility functions for test_pipelines_{1,2,3}.
README.md This file.
run_benchmark.sh A bash script used for measuring performance in the Seti BL data centre.
run_tests.sh A bash script used in Github Actions.
test_drift_rates.py Test source for generating multiple min_drift and max_drift tests, employing the setigen tool.
test_fb_cases.py Test source for generating multiple Filterbank tests, employing the setigen tool.
test_find_event.py Test source for generating multiple tests of the find_event.py module.
test_pipelines_1.py Test source for exercising/validation of find_event_pipeline.py and plot_event_pipeline.py.
test_pipelines_2.py Same as test_pipelines_1.py except it tests observation file cadences that start with "OFF" target instead of starting with "ON" target.
test_pipelines_3.py Test source for exercising/validation of find_event_pipeline.py when the collection of input files are a complex cadence.
test_pipelines_4.py Test source for exercising/validation of run_pipeline.py.
test_plot_dat.py Test source for exercising/validation of plot_dat.py.
test_turbo_seti.py Test source for exercising/validation of a variety of turbo_seti functions.