Multi-threaded Step job hangs if @StepScope bean cannot be created

[farnetto]

If in a multi-threaded step a @StepScope bean cannot be created then the batch just hangs waiting in TaskExecutorRepeatTemplate#waitForResults L172.

In my batch I want to pass in the pool size as a job parameter to the TaskExecutor used in the step. If this parameter is missing or cannot be converted into an int, then an exception is added to the deferred list but the TaskExecutorRepeatTemplate thinks a result is still coming from the ResultQueueResultHolder.

Here is my task executor:

@Bean
@StepScope
public TaskExecutor taskExecutor(@Value("#{jobParameters['poolSize']}") int poolSize) {
	ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
	executor.setCorePoolSize(poolSize);
	executor.setMaxPoolSize(poolSize);
	return executor;
}

Here's a pull request demonstrating the problem:

Apress/def-guide-spring-batch#13

[fmbenhassine]

Thank you for reporting this issue. I do confirm that the TaskExecutorRepeatTemplate hangs in this case (I was able to reproduce the issue in this repo). This kind of exceptions should fail fast and not added to the deferred exceptions list.

That said, I see no added value in making the TaskExecutor step-scoped and I believe the poolSize should not be passed as a job parameter in the first place. Job parameters are designed to be used for "business" runtime parameters (like an input file name, database table, partition meta-data, etc) and not technical configuration parameters like thread poolSize, database connection poolSize, database Url, etc.

[farnetto]

We do see added value in making the TaskExecutor step-scoped and passing in the poolSize as a job parameter. We want to be able to change the poolSize easily in production without redeploying the code or the overall job configuration.

[fmbenhassine]

We want to be able to change the poolSize easily in production without redeploying the code or the overall job configuration.

You can do that with a system property (or a similar mechanism provided by Spring Framework or Spring Boot), something like:

-- $>java -jar myjob.jar poolSize=3
++ $>java -jar myjob.jar -DpoolSize=3

@Bean
-- @StepScope
-- public TaskExecutor taskExecutor(@Value("#{jobParameters['poolSize']}") int poolSize) {
++ public TaskExecutor taskExecutor(@Value("#{systemProperties['poolSize']}") int poolSize) {
    ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
    executor.setCorePoolSize(poolSize);
    executor.setMaxPoolSize(poolSize);
    return executor;
}