2.8.4

What's Changed

• Fix ActionIndex.find() when callers pass an unknown database/schema value. by @BenBirt in #1643

Full Changelog: 2.8.3...2.8.4

2.8.3

What's Changed

• Make Vscode Buildable Again by @GJMcGowan in #1632
• VSCode: avoid errors upon completion requests by @barna-emarsys in #1627
• VSCode: Add ability to configure compile command, add option to skip compilation on save by @bmagyarkuti in #1639
• Escape backslashes and backticks in multiline string literals by @diasdauletov in #1641

New Contributors

• @barna-emarsys made their first contribution in #1627
• @bmagyarkuti made their first contribution in #1639

Full Changelog: 2.8.2...2.8.3

2.8.2

What's Changed

• Fix the @dataform/cli --table_prefix flag. by @BenBirt in #1623
• Append a compile error when resolve() is unable to resolve the given reference to an action. by @BenBirt in #1624

Full Changelog: 2.8.1...2.8.2

2.8.1

What's Changed

• Make a packages readme by @Ekrekr in #1612
• Fix doc link, and tidying by @Ekrekr in #1614
• Updates mssql dependencies by @bit-shifter in #1618
• Remove unused babel-preset-env dependency, which transitively removes a dependency on babel-traverse (CVE-2023-45133). by @BenBirt in #1619
• Bump version number. by @BenBirt in #1620

New Contributors

• @bit-shifter made their first contribution in #1618

Full Changelog: 2.8.0...2.8.1

2.8.0: Packaging and compilation performance improvements

The highlights of this release are two significant performance improvements for GCP Dataform projects:

• The @dataform/core NPM package now no longer has any dependencies, with all dependency packages bundled in to existing minified bundle. This reduces the size of installation significantly and should improve package installation performance and reliability on Dataform on GCP. See #1552 for more info.
• We've rolled out a change that reduces the amount of work required to decode/encode compiled graphs that we expect to improve compilation performance on Dataform on GCP by ~2x. See #1570 for more info.

What's Changed

• Remove formatter test files from examples by @Ekrekr in #1558
• Improve example projects by @Ekrekr in #1559
• Bump postcss from 8.3.9 to 8.4.31 by @dependabot in #1548
• Allow ArrayBuffer args and return from the main function, more than doubling compilation speed for large projects by @Ekrekr in #1570
• Remove snowflake test dependency on S3 by @lewish in #1583
• Minify @dataform/core and bundle all dependencies by @lewish in #1552
• Bump browserify-sign from 4.2.1 to 4.2.2 by @dependabot in #1586
• Bump axios from 0.21.2 to 1.6.0 by @dependabot in #1574
• Add support for publishing pre-release/next packages (and stage 2.8.0-beta.1) by @lewish in #1587
• Update analytics-node to unabandoned version by @claydiffrient in #1588
• Update version of snowflake-sdk to 1.9.1 by @claydiffrient in #1592
• Add project config override cli options by @Ekrekr in #1454

New Contributors

• @claydiffrient made their first contribution in #1588

Full Changelog: 2.7.0...2.8.0

2.7.0: Updates for Dataform GCP incremental SQL

From version 2.7.0 onwards, Dataform projects running on Google Cloud Platform will use updated SQL generation logic for incremental insert tables (tables of type incremental without a uniqueKey specified).

Explicit column names

Column names will be explicitly listed in the insert call, which is inline with OSS Dataform behaviour and prevents schema mismatch during insert, for example:

• if source_table has columns in a different order that the target - it can lead to data corruption (column values can be swapped during insert)
• if source_table has different number of columns - the insert into fails since target table columns count does not match with the source

For the new script the incremental query has to list all target table columns (can list other extra columns, but at least must contain target table columns) in any order.

Example of new generated SQL:

INSERT INTO $target_table 
    ($target_columns_list) -- listing target columns
    SELECT target_columns_list -- reordering columns so that subquery column order matches the target column order
    FROM (
        $incrementalQuery
    );

Execution within a procedure

In order to facilitate explicit columns, the new code is executed within a procedure, which will be created on the fly. For example:

EXECUTE IMMEDIATE
"""
CREATE OR REPLACE PROCEDURE $procedure_name()
BEGIN
    $preOperations
    $incrementalInsertStatement
    $postOperations
END;
"""
CALL $procedure_name();
DROP PROCEDURE IF EXISTS $procedure_name;

2.6.8

Fixing bug for escaping target name and canonical target name. PR: #1551

2.6.7: Support `NO_COLOR` environment variable

@dataform/cli now supports turning off coloured output via setting the NO_COLOR environment variable, per the standard detailed at https://no-color.org/.

2.6.6: Enforce tighter requirements around "automatic" importing of `includes` files

We have fixed two bugs which inadvertently loosened our (expected) requirements around usage of "automatic" includes imports.

For context: in general, to reference the contents of a file in includes (specifically a file's module.exports object), the callsite should call require() on that file, e.g. const foo = require("includes/subdirectory/foo.js");.

Dataform simplifies this for "top-level" includes files, i.e. direct children of the includes directory, by automatically making these files available globally. For example, in order to use includes/foo.js, a callsite does not need to require("includes/foo.js"); instead, a foo object is made available to all Dataform code in the project.

Two bugs have been found and fixed:

1. includes files can no longer implicitly depend on other includes files
2. only top-level includes files are now automatically available globally

This unfortunately results in a potentially breaking change to some Dataform projects - but this will only happen upon upgrading @dataform/core to >= 2.6.6.

In order to fix any breakages, the calling file must be changed to explicitly require() the relevant includes file.

For example, in SQLX:

js {
  const whatever = require("includes/subdirectory/whatever.js");
}

Or in JavaScript:

const whatever = require("includes/whatever.js");

For more context, see https://issuetracker.google.com/issues/296162656#comment3.

2.6.5: Fix `@dataform/cli` for `@dataform/core` pre-`2.6.5`.

This version fixes behaviour in rare circumstances which was broken by @dataform/cli version 2.6.3. See https://issuetracker.google.com/issues/296162656#comment3 for details.