Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

better error messages for failed tests #1424

Conversation

0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q
Copy link
Member

close #1119

  • New feature

Checklist:

  • My code follows the coding etiquette
  • I performed a self-review of my own code
  • I explained my changes within the PR, particularly in hard-to-understand areas
  • I checked that the in-code documentation is up-to-date
  • I adjusted the reporting in remind2 where it was needed
  • I adjusted forbiddenColumnNames in readCheckScenarioConfig.R in case the PR leads to deprecated switches
  • All automated model tests pass (FAIL 0 in the output of make test)
  • The changelog CHANGELOG.md has been updated correctly

Further information (optional):

Tail of the output (on a manually introduced compilation error) looks like

…
Creating full.gms
Error in prepare() : Compiling main_testOneRegi.gms failed, stopping.
Use `less -j 4 --pattern='^\*\*\*\*' output/testOneRegi/main.lst` to investigate compilation errors.
Calls: prepareAndRun -> prepare
2023-09-28 11:56:56: unlocked model.
Execution halted
Error in submit(cfg) : Executing prepareAndRun failed, stopping.
Execution halted
✖ | 1       0 | 02-compile [60.2s]                                                                                                      
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Failure (test_02-compile.R:11:3): remind compiles
`Rscript start.R config/tests/scenario_config_compile.csv` returned exit status 1, not 0
────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
✔ |     1   0 | 03-quick                                                                                                                
✔ |     2   0 | 04-gamscompile                                                                                                          
✔ |     1   0 | 05-default                                                                                                              
✔ |     1   0 | 06-output                                                                                                               
✔ |     1   0 | 20-coupled                                                                                                              
✔ |     1   0 | 99-codeCheck                                                                                                            

══ Results ═════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════════
Duration: 89.5 s

── Skipped tests (9) ───────────────────────────────────────────────────────────────────────────────────────────────────────────────────
• A previous test failed. (3): test_03-quick.R:8:3, test_04-gamscompile.R:11:3, test_99-codeCheck.R:10:3
• Not run in default tests, use `make test-full` to run (takes significantly longer than 10 minutes). (5): test_01-start.R:34:3,
  test_04-gamscompile.R:25:3, test_05-default.R:8:3, test_06-output.R:7:1, test_20-coupled.R:8:1
• standalone models aren't consistent at the moment (1): test_01-gms_readSettings.R:12:3

── Failed tests ────────────────────────────────────────────────────────────────────────────────────────────────────────────────────────
Failure (test_02-compile.R:11:3): remind compiles
`Rscript start.R config/tests/scenario_config_compile.csv` returned exit status 1, not 0

[ FAIL 1 | WARN 0 | SKIP 9 | PASS 79 ]
Error: Test failures
Execution halted
make: *** [Makefile:54: test] Error 1

Copy link
Member

@LaviniaBaumstark LaviniaBaumstark left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

nice improvement

@0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q 0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q merged commit 87f5d85 into remindmodel:develop Oct 5, 2023
1 check passed
@0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q 0UmfHxcvx5J7JoaOhFSs5mncnisTJJ6q deleted the dev/better_test_errors branch October 5, 2023 11:41
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Improve testing suit
2 participants