batch-skip-step

Spring Batch: Skip Specific Data

Table of Contents

• Background
• Implement Logics to Skip Data
  • Returning null
  • Throwing RuntimeException
• Verification

Skip processing specific data through business logic in Spring Batch.

Background

Spring Batch is a framework for batch processing – execution of a series of jobs. A job is composed of a series of steps. Each step consists of a reader, a processor, and a writer. The reader reads data from a data source, the processor processes the data, and the writer writes the processed data to a data source.

There are scenarios where we want to skip processing specific data through business logic. For example, in this guide, we want to skip data where username are either Elwyn.Skiles or Maxime_Nienow. We will look into two approaches; one by returning null and another by throwing RuntimeException. Both approaches will be implemented in the same ItemProcessor.

Our application will process data from users.json file and write the processed data into MySQL database. Content of users.json is taken from JSONPlaceholder.

Implement Logics to Skip Data

Returning null

First approach is to return null from ItemProcessor implementation. This approach is straight forward and does not require additional configuration.

link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]

With that, when the Job detected that the ItemProcessor returns null, it will skip the data and continue to the next.

Throwing RuntimeException

Second approach is to throw RuntimeException from ItemProcessor implementation. This approach requires additional configuration to be done when defining Step.

Implementation in ItemProcessor is as follows:

link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]

Next is to inform Step to skip UsernameNotAllowedException:

link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]
link:src/main/java/zin/rashidi/boot/batch/user/UserJobConfiguration.java[role=include]

With that, when the Job detected that the ItemProcessor throws UsernameNotAllowedException, it will skip the data. Full definition of the Job can be found in UserJobConfiguration.java.

Verification

We will implement integration tests to verify that our implementation is working as intended whereby Elwyn.Skiles and Maxime_Nienow are skipped thus will not be available in the database.

link:src/test/java/zin/rashidi/boot/batch/user/UserBatchJobTests.java[role=include]
link:src/test/java/zin/rashidi/boot/batch/user/UserBatchJobTests.java[role=include]
link:src/test/java/zin/rashidi/boot/batch/user/UserBatchJobTests.java[role=include]

By executing our tests in UserBatchJobTests.java, we will see that all users are processed except Elwyn.Skiles and Maxime_Nienow.