0.21.0

Bug Fixes

• Issue #354: users can query a view with different columns in select() and filter()
• Issue #367: Struct column order is fixed
• Issue #383: Fixed table metadata update when writing to a partitioned table
• Issue #390: Allowing additional white-space types in the query
• Issue #393: replacing avro.shaded dependency with guava
• PR #360: Removed redundant UNNEST when compiling IN condition

Dependency Updates

• BigQuery API has been upgraded to version 1.131.1
• BigQuery Storage API has been upgraded to version 1.22.0
• Guava has been upgraded to version 30.1.1-jre
• gRPC has been upgraded to version 1.37.1
• Netty has been upgraded to version 4.1.65.Final

0.20.0

New Features

• PR #375: Added support for pseudo column support - time partitioned table now support the _PARTITIONTIME and _PARTITIONDATE fields

Bug Fixes

• Issue #190: Writing data to BigQuery properly populate the field description
• Issue #265: Fixed nested conjunctions/disjunctions when using the AVRO read format
• Issue #326: Fixing netty_tcnative_windows.dll shading

Dependency Updates

• Arrow has een upgraded to version 4.0.0

0.19.1

Bug Fixes

• PR #324 - Restoring version 0.18.1 dependencies due to networking issues

Dependency Updates

• BigQuery API has been upgraded to version 1.123.2
• BigQuery Storage API has been upgraded to version 1.6.0
• Guava has been upgraded to version 30.0-jre
• Netty has been upgraded to version 4.1.51.Final

0.19.0

New Features

• Issue #247: Allowing to load results of any arbitrary SELECT query from BigQuery.
• Issue #310: Allowing to configure the expiration time of materialized data.
• PR #283: Implemented Datasource v2 write support.
• Improved Spark 3 compatibility.

Dependency Updates

• BigQuery API has been upgraded to version 1.127.4
• BigQuery Storage API has been upgraded to version 1.10.0
• Guava has been upgraded to version 30.1-jre
• Netty has been upgraded to version 4.1.52.Final

0.18.1

New Features

• PR #276: Added the option to enable useAvroLogicalTypes option When writing data to BigQuery.

Bug Fixes

• Issue #248: Reducing the size of the URI list when writing to BigQuery. This allows larger DataFrames (>10,000 partitions) to be safely written.
• Issue #296: Removed redundant packaged slf4j-api.

0.18.0

New Features

• Issue #226: Adding support for HOUR, MONTH, DAY Time Partitions
• Issue #260: Increasing connection timeout to the BigQuery service, and configuring the request retry settings.

Bug Fixes

• Issue #263: Fixed select * error when ColumnarBatch is used (DataSource v2)
• Issue #266: Fixed the external configuration not working regression bug (Introduced in version 0.17.2)
• PR #262: Filters on BigQuery DATE and TIMESTAMP now use the right type.

Dependency Updates

• BigQuery API has been upgraded to version 1.123.2
• BigQuery Storage API has been upgraded to version 1.6.0
• Guava has been upgraded to version 30.0-jre
• Netty has been upgraded to version 4.1.51.Final
• netty-tcnative has been upgraded to version 4.1.34.Final

0.17.3

Bug Fixes

• PR #242, #243: Fixed Spark 3 compatibility, added Spark 3 acceptance test
• Issue #249: Fixing credentials creation from key

0.17.2

Bug Fixes

• PR #239: Ensuring that the BigQuery client will have the proper project id

0.17.1

New Features

• PR #229: Adding support for Spark ML Vector and Matrix data types

Bug Fixes

• Issue #216: removed redundant ALPN dependency
• Issue #219: Fixed the LessThanOrEqual filter SQL compilation in the DataSource v2 implementation
• Issue #221: Fixed ProtobufUtilsTest.java with newer BigQuery dependencies

Dependency Updates

• BigQuery API has been upgraded to version 1.116.8
• BigQuery Storage API has been upgraded to version 1.3.1

0.17.0

New Features

• Structured streaming write is now supported (PR #201, thanks @varundhussa)
• Users now has the option to keep the data on GCS after writing to BigQuery (PR #202, thanks @leoneuwald)
• Enabling to overwrite data of a single date partition (PR #211)
• Supporting MATERIALIZED_VIEW as table type (PR #192)
• Supporting columnar batch reads from Spark in the DataSource V2 implementation. (PR #198) It is not ready for production use.

Bug Fixes

• Conditions on StructType fields are now handled by Spark and not the connector, Fixing Issue #197

Dependency Updates

• BigQuery API has been upgraded to version 1.116.3
• BigQuery Storage API has been upgraded to version 1.0.0
• Netty has been upgraded to version 4.1.48.Final (Fixing issue #200)