Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add blocked list of tests for batching strategy #954

Open
testing0mon21 opened this issue Jul 3, 2024 · 0 comments
Open

Add blocked list of tests for batching strategy #954

testing0mon21 opened this issue Jul 3, 2024 · 0 comments
Labels
enhancement New feature or request

Comments

@testing0mon21
Copy link

Is your feature request related to a problem? Please describe.

When using the batching strategy in our Marathontool configuration, I often encounter situations where some tests should not be batched together. For instance, certain tests may require exclusive execution or have specific dependencies that cannot be satisfied when executed in parallel. Currently, the only way to achieve this is by creating separate configurations for each test suite, which can lead to a high number of redundant configuration files.

Describe the solution you'd like

I would like to propose an option to specify a "batching strategy" or "test exclusion" per test or test suite in the Marathontool configuration. This could be achieved by introducing a new property or annotation that allows us to define which tests should not be batched together. For example, we could use a @NoBatch annotation on specific test methods or classes.

Describe alternatives you've considered

One possible alternative is to create separate configurations for each test suite that requires exclusive execution. However, this would lead to a proliferation of configuration files and make it more difficult to manage and maintain our testing setup.

Another option could be to implement a custom batching strategy using Marathontool's extensibility features (e.g., custom executors or reporters). While this would provide a high degree of flexibility, it would also require significant development effort and might not be feasible for non-technical users.

Additional context

Here is an example configuration file that demonstrates the issue:

`
batchingStrategy:
type: "fixed-size"
size: ${BATCH_COUNT}
blocklist:
- type: "annotation"
values:
- "com.example.SmokeTest"

`
In this example, we would like to exclude tests with annotation SmokeTest from being batched together.

@testing0mon21 testing0mon21 added the enhancement New feature or request label Jul 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

1 participant