parameterize-whether-a-demo-script-is-running-for-cicd-test

[david-thrower]

Background of the issue:

• We have showcase demo scripts for how to use Cerebros and demonstration of SOTA results.
• We want to use the same scripts for benchmarks in our CICD testing, but have a limit on resources for GitHub Actions.
• We don't want to maintain 2 separate scripts, one for showcase use and one for CICD tests.

Proposed solution:

• Parameterize whether the script it is running for CICD only.
• Option 1:
  • Create an environment variable in the Github runner like CICD_TEST, then make all the Python scripts that demonstrate how to use Cerebros, read in this variable, but default it to False if the variable does not exist.
  • If the execution environment the script runs in has the environment variable CICD_TEST set to true, then a small subset of the data is run in the training jobs. If the variable is set absent or set to false, then the full data set runs.
• Option 2:
  • Use fire.Fire or argparse to run the job, adding a shell arg: --CICD_TEST, which if set will trigger the small subset run.

[sashakolpakov]

We have one successful run, though: https://github.com/david-thrower/cerebros-core-algorithm-alpha/actions/runs/6721781884

…

On Wed, 1 Nov 2023 at 17:46, David Thrower ***@***.***> wrote: Background of the issue: - We have showcase demo scripts for how to use Cerebros and demonstration of SOTA results. - We want to use the same scripts for benchmarks in our CICD testing, but have a limit on resources for GitHub Actions. - We don't want to maintain 2 separate scripts, one for showcase use and one for CICD tests. Proposed solution: - Parameterize whether the script it is running for CICD only. - Option 1: - Create an *environment variable* in the Github runner like CICD_TEST, then make all the Python scripts that demonstrate how to use Cerebros, read in this variable, but default it to False if the variable does not exist. - If the execution environment the script runs in has the environment variable CICD_TEST set to true, then a small subset of the data is run in the training jobs. If the variable is set absent or set to false, then the full data set runs. - Option 2: - Use fire.Fire or argparse to run the job, adding a *shell arg*: --CICD_TEST, which if set will trigger the small subset run. — Reply to this email directly, view it on GitHub <#124>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/ABIOQZLEVH7CNFLZ73QJZKLYCJ4FFAVCNFSM6AAAAAA6ZSFWBGVHI2DSMVQWIX3LMV43ASLTON2WKOZRHE3TENZSHA3TOOI> . You are receiving this because you are subscribed to this thread.Message ID: ***@***.***>