0.21.1

New Features

• PR #413: Pushing all filters to BigQuery Storage API

Bug Fixes

• Issue #412: Supporting WITH queries
• Issue #409: Allowing all whitespaces after the select
• PR #419: Fix a bug where background threads > 2 cases would miss pages (DSv2)
• PR #416: Moved zstd-jni library to be provided in order to solve Spark 2.4 compatibility (DSv2)
• PR #417: Added back column projection to DSv2

0.21.0

Bug Fixes

• Issue #354: users can query a view with different columns in select() and filter()
• Issue #367: Struct column order is fixed
• Issue #383: Fixed table metadata update when writing to a partitioned table
• Issue #390: Allowing additional white-space types in the query
• Issue #393: replacing avro.shaded dependency with guava
• PR #360: Removed redundant UNNEST when compiling IN condition

Dependency Updates

• BigQuery API has been upgraded to version 1.131.1
• BigQuery Storage API has been upgraded to version 1.22.0
• Guava has been upgraded to version 30.1.1-jre
• gRPC has been upgraded to version 1.37.1
• Netty has been upgraded to version 4.1.65.Final

0.20.0

New Features

• PR #375: Added support for pseudo column support - time partitioned table now support the _PARTITIONTIME and _PARTITIONDATE fields

Bug Fixes

• Issue #190: Writing data to BigQuery properly populate the field description
• Issue #265: Fixed nested conjunctions/disjunctions when using the AVRO read format
• Issue #326: Fixing netty_tcnative_windows.dll shading

Dependency Updates

• Arrow has een upgraded to version 4.0.0

0.19.1

Bug Fixes

• PR #324 - Restoring version 0.18.1 dependencies due to networking issues

Dependency Updates

• BigQuery API has been upgraded to version 1.123.2
• BigQuery Storage API has been upgraded to version 1.6.0
• Guava has been upgraded to version 30.0-jre
• Netty has been upgraded to version 4.1.51.Final

0.19.0

New Features

• Issue #247: Allowing to load results of any arbitrary SELECT query from BigQuery.
• Issue #310: Allowing to configure the expiration time of materialized data.
• PR #283: Implemented Datasource v2 write support.
• Improved Spark 3 compatibility.

Dependency Updates

• BigQuery API has been upgraded to version 1.127.4
• BigQuery Storage API has been upgraded to version 1.10.0
• Guava has been upgraded to version 30.1-jre
• Netty has been upgraded to version 4.1.52.Final

0.18.1

New Features

• PR #276: Added the option to enable useAvroLogicalTypes option When writing data to BigQuery.

Bug Fixes

• Issue #248: Reducing the size of the URI list when writing to BigQuery. This allows larger DataFrames (>10,000 partitions) to be safely written.
• Issue #296: Removed redundant packaged slf4j-api.

0.18.0

New Features

• Issue #226: Adding support for HOUR, MONTH, DAY Time Partitions
• Issue #260: Increasing connection timeout to the BigQuery service, and configuring the request retry settings.

Bug Fixes

• Issue #263: Fixed select * error when ColumnarBatch is used (DataSource v2)
• Issue #266: Fixed the external configuration not working regression bug (Introduced in version 0.17.2)
• PR #262: Filters on BigQuery DATE and TIMESTAMP now use the right type.

Dependency Updates

• BigQuery API has been upgraded to version 1.123.2
• BigQuery Storage API has been upgraded to version 1.6.0
• Guava has been upgraded to version 30.0-jre
• Netty has been upgraded to version 4.1.51.Final
• netty-tcnative has been upgraded to version 4.1.34.Final

0.17.3

Bug Fixes

• PR #242, #243: Fixed Spark 3 compatibility, added Spark 3 acceptance test
• Issue #249: Fixing credentials creation from key

0.17.2

Bug Fixes

• PR #239: Ensuring that the BigQuery client will have the proper project id

0.17.1

New Features

• PR #229: Adding support for Spark ML Vector and Matrix data types

Bug Fixes

• Issue #216: removed redundant ALPN dependency
• Issue #219: Fixed the LessThanOrEqual filter SQL compilation in the DataSource v2 implementation
• Issue #221: Fixed ProtobufUtilsTest.java with newer BigQuery dependencies

Dependency Updates

• BigQuery API has been upgraded to version 1.116.8
• BigQuery Storage API has been upgraded to version 1.3.1