Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 3 additions & 1 deletion CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -1,9 +1,11 @@
## [Unreleased]

- Drop `:process` and `:thread` workers since there are no concrete use cases.

## [0.4.2] - 2024-05-23

- Fix Prism `LoadError` message [#27](https://github.com/ohbarye/pbt/pull/27) by @sambostock

## [0.4.1] - 2024-05-10

- Fix a bug for experimental_ractor_rspec_integration mode. When a test file name starts with a number, it can't be a constant name.
Expand Down
1 change: 0 additions & 1 deletion Gemfile
Original file line number Diff line number Diff line change
Expand Up @@ -7,6 +7,5 @@ gemspec
gem "rake"
gem "rspec"
gem "standard"
gem "parallel"
gem "prism"
gem "benchmark-ips"
42 changes: 6 additions & 36 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -218,7 +218,7 @@ Pbt.configure do |config|
# Whether to print verbose output. Default is `false`.
config.verbose = false

# The concurrency method to use. `:ractor`, `:thread`, `:process` and `:none` are supported. Default is `:none`.
# The concurrency method to use. `:ractor` and `:none` are supported. Default is `:none`.
config.worker = :none

# The number of runs to perform. Default is `100`.
Expand Down Expand Up @@ -249,15 +249,13 @@ end

One of the key features of `Pbt` is its ability to rapidly execute test cases in parallel or concurrently, using a large number of values (by default, `100`) generated by `Arbitrary`.

For concurrent processing, you can specify any of the three workers—`:ractor`, `:process`, or `:thread`—using the `worker` option. Alternatively, choose `:none` for serial execution.

`Pbt` supports 3 concurrency methods and 1 sequential one. You can choose one of them by setting the `worker` option.
For concurrent processing, you can specify `:ractor` using the `worker` option. Alternatively, choose `:none` for serial execution.

Be aware that the performance of each method depends on the test subject. For example, if the test subject is CPU-bound, `:ractor` may be the best choice. Otherwise, `:none` shall be the best choice for most cases. See [benchmarks](benchmark/README.md).

### Ractor

`:ractor` worker is useful for test cases that are CPU-bound. But it's experimental and has some limitations as described below. If you encounter any issues due to those limitations, consider using `:process` as workers whose benchmark is the most similar to `:ractor`.
`:ractor` worker is useful for test cases that are CPU-bound. But it's experimental and has some limitations as described below. If you encounter any issues due to those limitations, consider falling back to `:none`.

```ruby
Pbt.assert(worker: :ractor) do
Expand Down Expand Up @@ -313,37 +311,9 @@ it do
end
```

### Process

If you'd like to run test cases that are CPU-bound and `:ractor` is not available, `:process` becomes a good choice.

```ruby
Pbt.assert(worker: :process) do
Pbt.property(Pbt.integer) do |n|
# ...
end
end
```

If you want to use `:process`, you need to install the [parallel](https://github.com/grosser/parallel) gem.

### Thread

You may not need to run test cases with multi-threads.

```ruby
Pbt.assert(worker: :thread) do
Pbt.property(Pbt.integer) do |n|
# ...
end
end
```

If you want to use `:thread`, you need to install the [parallel](https://github.com/grosser/parallel) gem.

### None

For most cases, `:none` is the best choice. It runs tests sequentially (without parallelism) but most test cases finishes within a reasonable time.
For most cases, `:none` is the best choice. It runs tests sequentially but most test cases finishes within a reasonable time.

```ruby
Pbt.assert(worker: :none) do
Expand All @@ -362,8 +332,8 @@ Once this project finishes the following, we will release v1.0.0.
- [x] Support shrinking
- [x] Support multiple concurrency methods
- [x] Ractor
- [x] Process
- [x] Thread
- [x] Process (dropped)
- [x] Thread (dropped)
- [x] None (Run tests sequentially)
- [x] Documentation
- [x] Add better examples
Expand Down
2 changes: 1 addition & 1 deletion lib/pbt/check/configuration.rb
Original file line number Diff line number Diff line change
Expand Up @@ -13,7 +13,7 @@ module Check
keyword_init: true
) do
# @param verbose [Boolean] Whether to print verbose output. Default is `false`.
# @param worker [Symbol] The concurrency method to use. :ractor`, `:thread`, `:process` and `:none` are supported. Default is `:none`.
# @param worker [Symbol] The concurrency method to use. :ractor` and `:none` are supported. Default is `:none`.
# @param num_runs [Integer] The number of runs to perform. Default is `100`.
# @param seed [Integer] The seed to use for random number generation. It's useful to reproduce failed test with the seed you'd pick up from failure messages. Default is a random seed.
# @param thread_report_on_exception [Boolean] Whether to report exceptions in threads. It's useful to suppress error logs on Ractor that reports many errors. Default is `false`.
Expand Down
58 changes: 0 additions & 58 deletions lib/pbt/check/runner_methods.rb
Original file line number Diff line number Diff line change
Expand Up @@ -96,10 +96,6 @@ def run_it(property, source_values, config)
run_it_in_sequential(property, runner)
in :ractor
run_it_in_ractors(property, runner)
in :process
run_it_in_processes(property, runner)
in :thread
run_it_in_threads(property, runner)
end
end
runner.run_execution
Expand Down Expand Up @@ -153,60 +149,6 @@ def handle_ractor_error(cause, c)
c.exception = e
end
end

# @param property [Property] Property to test.
# @param runner [RunnerIterator]
# @return [void]
def run_it_in_threads(property, runner)
require_parallel

Parallel.map_with_index(runner, in_threads: Parallel.processor_count) do |val, index|
Case.new(val:, index:).tap do |c|
property.run(val)
# Catch all exceptions including RSpec's ExpectationNotMet (It inherits Exception).
rescue Exception => e # standard:disable Lint/RescueException:
c.exception = e
# It's possible to break this loop here by raising `Parallel::Break`.
# But if it raises, we cannot fetch all cases' result. So this loop continues until the end.
end
end.each do |c|
runner.handle_result(c)
break if c.exception
# Ignore the rest of the cases. Just pick up the first failure.
end
end

# @param property [Property] Property to test.
# @param runner [RunnerIterator]
# @return [void]
def run_it_in_processes(property, runner)
require_parallel

Parallel.map_with_index(runner, in_processes: Parallel.processor_count) do |val, index|
Case.new(val:, index:).tap do |c|
property.run(val)
# Catch all exceptions including RSpec's ExpectationNotMet (It inherits Exception).
rescue Exception => e # standard:disable Lint/RescueException:
c.exception = e
# It's possible to break this loop here by raising `Parallel::Break`.
# But if it raises, we cannot fetch all cases' result. So this loop continues until the end.
end
end.each do |c|
runner.handle_result(c)
break if c.exception
# Ignore the rest of the cases. Just pick up the first failure.
end
end

# Load Parallel gem. If it's not installed, raise an error.
# @see https://github.com/grosser/parallel
# @raise [InvalidConfiguration]
def require_parallel
require "parallel"
rescue LoadError
raise InvalidConfiguration,
"Parallel gem (https://github.com/grosser/parallel) is required to use worker `:process` or `:thread`. Please add `gem 'parallel'` to your Gemfile."
end
end
end
end
Loading
Loading