Add new ParallelTests::RSpec::VerboseFormatter for enhanced debugging #926
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
The new formatter allows users of parallel_rspec to more easily obtain detailed information about the examples running as they occur in real time.
Background:
In my project, it was observed that an RSpec example would hang intermittently on CI. At first, no team member was able to make this reliably fail enough on CI to facilitate debugging and no one was able to reproduce locally at all. The hang only occurred with parallel tests running and never with just the "rspec" command.
In an attempt to gather more information, we began using the RSpec "documentation" formatter which outputs the "description", "context", and "it" strings. Unfortunately, when running in parallel, the output from different processes will interleave with one another making it extremely difficult to understand which example is responsible for the observed hang.
Solution:
Build a "documentation-like" formatter that plays nicely with the parallel_tests framework. The new formatter always ouputs all information on a single line to avoid important details from different processes interleaving with one another. This means the "describe" and "context" strings will be repeated across multiple "it" blocks.
To further assist, the formatter include the PID and parallel process number to help identify which subprocess hangs and which one continues. The output will look something like:
In the output above, it is clear the "Foo foo" example is still running.
Using this formatter, our project's team was able to successfully identify the example causing the intermittent hang and then address it. As such, I'm hoping that this may be useful for a larger audience.
Thank you for your contribution!
Checklist
master
(if not - rebase it).code introduces user-observable changes.