README.adoc

spring-batch-bigquery

Spring Batch extension which contains an ItemWriter and ItemReader implementations for BigQuery.

ItemWriter support:

	Load job	Write API (Commited)	Write API (Pending)
JSON	Supported	Supported	Supported
CSV	Supported

ItemReader support:

JSON	Supported
CSV	Supported

Based on Java BigQuery.

Example of BigQueryLoadJobCsvItemWriter

@Bean
BigQueryLoadJobCsvItemWriter<MyDto> bigQueryCsvWriter() {
    WriteChannelConfiguration writeConfiguration = WriteChannelConfiguration
        .newBuilder(TableId.of("csv_dataset", "csv_table"))
        .setAutodetect(true)
        .setFormatOptions(FormatOptions.csv())
        .build();

    return new BigQueryLoadJobCsvItemWriterBuilder<MyDto>()
        .bigQuery(bigQueryService)
        .writeChannelConfig(writeConfiguration)
        .build();
}

Example of BigQueryItemReader

@Bean
BigQueryItemReader<PersonDto> bigQueryReader() {
    return new BigQueryQueryItemReaderBuilder<PersonDto>()
        .bigQuery(bigQueryService)
        .rowMapper(res -> new PersonDto(res.get("name").getStringValue()))
        .query("SELECT p.name FROM persons p")
        .build();
}

Additional examples could be found in the test folder.