v1.4.0

Important Changes

What's new?

All adapters

• Global variables for rsrc_default_dtype, timestamp_default_dtype, stg_default_dtype, and derived_column_default_dtype are now adapter specific
• Adapter specific column escaping
• Enforcing default datatype of ldts: The load date timestamp column is explicitly cast to the specified timestamp_default_dtype

PostgreSQL

• Usage of adapter specific end_of_all_times

What's changed?

Snowflake

• Huge improvements in column name escaping, to support all kinds of column names

PostgreSQL

• Default Hash Datatype set to VARCHAR(32) instead of STRING
• Broader support of Datatypes in Ghost Record generation
• Improvement of incremental loading for NH_Links
• Fix hash macro parameters #207
• Fix ref_sat_v0 and sat_v0 macro parameters #206
• Fix control_snap_v0 macro parameters #205

Redshift

• Default Hash Datatype set to VARCHAR(32) instead of STRING
• Broader support of Datatypes in Ghost Record generation #204
• Improvement of incremental loading for NH_Links
• Usage of DATEADD() instead of TIMESTAMPSUB() in PIT #155
• CROSS JOIN instead of FULL OUTER JOIN in ref_table
• Fix hash macro parameters #207
• Fix ref_sat_v0 and sat_v0 macro parameters #206
• Fix control_snap_v0 macro parameters #205

Synapse

• Align the snapshot control tables v0 and v1 with the other adapters

Bugfixes

All adapters

• Usage of dbt.type_timestamp() instead of just type_timestamp()

Postgres

• Timestamp format for Postgres changed to YYYY-MM-DD HH24:MI:SS
• Fixed Interval issues in Control_Snap_v1
• Added option for disable_hwm and source_is_single_batch to sat_v0 to align with other adapters

Redshift:

• string_to_timestamp now outputs the datatype that is set as value timestamp_default_dtype

Snowflake

• Fixed issue with multi-source hubs which had different business key names

Google BigQuery

• Fixed issue with duplicate column names when using an input column as rsrc or ldts
• Fixed uniqueness issue for Multi Source reference hubs

Synapse

• Fixed issue with multi-source hubs and links
• Fixed unnecessary quoting in PIT for the pit_type
• Remove unnecessary subquery inside the ref_table

Exasol

• Fixed Issues with Multi Active Hashing
• Fixed several issues with column escaping
• Added disable_hwm parameter to various RDV models, to align with other adapters

v1.3.3

What's Changed

• deleted comments and changed schemas back to generic placeholders by @polat-deniz in #179
• Fixed combination of derived cols and prejoins in Snowflake by @tkirschke in #183
• Removed double comma for Multi Active Hashkey calculation by @tkirschke in #184
• Add parameters to postgres macros by @tkiehn in #186

Full Changelog: v1.3.2...v1.3.3

v1.3.2

What's Changed

• Delete logs directory by @polat-deniz in #169
• Added CAST() to fix hash_datatype length issue with Snowflake by @migma-ciklum in #174
• Change global var hash datatype to string by @tkiehn in #175
• change copy_rsrc_ldts_input_columns to false by @tkiehn in #176
• Various fixes for v1 Satellites by @tkirschke in #178

New Contributors

• @migma-ciklum made their first contribution in #174

Full Changelog: v1.3.1...v1.3.2

v1.3.1

What's Changed

• add big query macro in ghost_record_per_datatype by @tkiehn in #162

New Contributors

• @tkiehn made their first contribution in #162

Full Changelog: v1.3.0...v1.3.1

v1.3.0

What's Changed

• Added fix for ghost record's default values for date data type by @tta-scalefree in #157
• Azure Synapse release ready by @DPolat-Scalefree in #159
• Added Synapse Support to Readme by @tkirschke in #160

New Contributors

• @DPolat-Scalefree made their first contribution in #159

Full Changelog: v1.2.2...v1.3.0

Hotfix v1.2.2

What's Changed

• Redshift patch by @thoffmann-sf in #152

Full Changelog: v1.2.1...v1.2.2

v1.2.1

What's Changed

• Added fix for hashing bug in multi-active stage. by @tta-scalefree in #148
• Fully replicate source column name in derived column name for Snowflake by @bschlottfeldt in #150

New Contributors

• @tta-scalefree made their first contribution in #148

Full Changelog: v1.2.0...v1.2.1

v1.2.0

What's Changed

• PostgreSQL and Redshift Adapter implementation by @thoffmann-sf in #142

Full Changelog: v1.1.5...v1.2

Hotfix v1.1.5

What's Changed

• Passed "snapshot_trigger_column" to macro by @tkirschke in #133
• [BUGFIX][BigQuery] Fix Control Snapshot v0 by @bschlottfeldt in #135

Full Changelog: v1.1.4...v1.1.5

v1.1.4

ATTENTION! IMPORTANT DISCLAIMER FOR v1.1.4

There is a fix for the hashing algorithm in the Snowflake adapter that will generate different hash for columns with strings containing the small character 'v' . The reason behind this is that Snowflake doesnt interpret the \v as a special character alias for the vertical tab, so instead it interprets that you are escaping the letter v. The issue was first presented at #109

This fix means if you have a BK with value 'virtual', for example, in version 1.1.3 or earlier versions the hash algorithm was actually removing the small character 'v', so the value to be hashed was 'irtual' (which is wrong), the value to be hashed should be 'virtual', in PR #118 we fix the hash so that it will now hash the correct value.
Therefore, if this key comes into the stage it will be hashed and generate a different hash value than the one encountered previously loaded in your Raw Vault. This hashkey will then be inserted into you Raw Vault, even though it represents the same BK, because it was not inserted before.

We strongly recommend that the user manually modifies the hashes which include this small character in the Raw Vault, updating the value to the correct one generated by the hash algorithm in this presented version, so that your stage wont falsely detect "new" keys.
We deeply apologize for the inconvenience that this generates.

Kind regards,
Your Scalefree datavault4dbt team.

What's Changed

All adapters - new feature and fix:

• Extended logarithmic logic of snapshot control tables by @bschlottfeldt in #116
• Fixed sat_v0 : Same hashdiff being inserted again into satellite without in between deltas by @bschlottfeldt in #123

Snowflake adapter - fix and new feature:

• Fixed replacement of vertical tab in hash_standardization - Snowflake adapter by @thoffmann-sf in #118
• Added RTRIM Hashdiff Option for Snowflake by @bschlottfeldt in #119 - More information on how to use this new feature can be found in https://github.com/ScalefreeCOM/datavault4dbt/wiki/Adding-new-columns-to-Hashdiff-and-Satellite

New Contributors

• @thoffmann-sf made their first contribution in #118

Full Changelog: v1.1.3...v1.1.4