Skip to content

Commit

Permalink
Merge #329: cmake, doc: Document log output for failed tests
Browse files Browse the repository at this point in the history
784a87e fixup! doc: Update for CMake-based build system (Hennadii Stepanov)

Pull request description:

  On the master branch, `make check` generates a `foo_tests.log` file for each `foo_tests.cpp`.  These log files are only printed when a test fails. Since no one has raised concerns about the log file [renaming](bitcoin#19385 (comment)) in bitcoin#19385, these files appear to serve no other purpose.

  CTest provides its own mechanisms for logging failed tests, making the creation of `foo_tests.log` files unnecessary.

  This PR documents these CTest logging mechanisms.

  An example of output with a mocked test error:
  ```
  $ ctest --test-dir build -j 16 --output-on-failure
  Internal ctest changing into directory: /home/hebasto/git/bitcoin/build
  Test project /home/hebasto/git/bitcoin/build
          Start  10: amount_tests
          Start   6: tests
          Start   5: noverify_tests
          Start 120: coinselector_tests
          Start   7: exhaustive_tests
          Start  29: coins_tests
          Start 130: wallet_tests
          Start  75: random_tests
          Start 102: transaction_tests
          Start   1: util_test_runner
          Start 133: walletload_tests
          Start  31: coinstatsindex_tests
          Start  63: net_tests
          Start  35: crypto_tests
          Start  27: checkqueue_tests
          Start  56: miner_tests
    1/133 Test  #10: amount_tests .........................***Failed    0.02 sec
  Running 4 test cases...
  ./src/test/amount_tests.cpp(19): error: in "amount_tests/MoneyRangeTest": check MoneyRange(MAX_MONEY) == false has failed [true != false]

  *** 1 failure is detected in the test module "Bitcoin Core Test Suite"

          Start 113: validation_block_tests
    2/133 Test #113: validation_block_tests ...............   Passed    1.02 sec
  < snip >
  133/133 Test   #6: tests ................................   Passed  100.94 sec

  99% tests passed, 1 tests failed out of 133

  Total Test time (real) = 100.95 sec

  The following tests FAILED:
   10 - amount_tests (Failed)
  Errors while running CTest
  ```

  ---

  A question for reviewers: Do we really need the `--log_level=test_suite` option for these logs?

ACKs for top commit:
  vasild:
    ACK 784a87e

Tree-SHA512: bfe4ab2c7f9002584a41efa3607c911b6c635720ae5a6d080294b3240c4df546d25e5ac0d667881b836f6c47f2e55915e093ca14765951c40d4856482b52a9fd
  • Loading branch information
hebasto committed Aug 15, 2024
2 parents fb18471 + 784a87e commit 3c467d8
Showing 1 changed file with 2 additions and 1 deletion.
3 changes: 2 additions & 1 deletion src/test/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -112,7 +112,8 @@ see `uint256_tests.cpp`.
### Logging and debugging in unit tests
`make check` will write to a log file `foo_tests.cpp.log` and display this file
`ctest --test-dir build` will write to a log file `build/Testing/Temporary/LastTest.log`. You can
additionaly use the `--output-on-failure` option to display logs of the failed tests automatically
on failure. For running individual tests verbosely, refer to the section
[above](#running-individual-tests).
Expand Down

0 comments on commit 3c467d8

Please sign in to comment.