Skip to content

Commit

Permalink
For nested fields we were using lateral flatten. The issue with it is…
Browse files Browse the repository at this point in the history
… that if the

nested field is used in a join, snowflake complains that lateral flatten can't be
on left side of join (https://stackoverflow.com/questions/63397022/snowflake-lateral-cannot-be-on-the-left-side-of-join)

fix is to always convert lateral flatten to a left join

Also simplify the array[scalar] vs array[record] treatment. earlier, at the
unnesting step we were creating a object for array[scalar] case to make it
consistent with the array[record] case. Now at the reference time, based on
the case, use the right expression
- array[record] : parent.value:sql_field
- array[scalar] : parent.value

Signed-off-by: Amit Aggarwal <amit@datairis.io>
  • Loading branch information
amitaggarwal1 committed Jan 3, 2025
1 parent 9cbdd4d commit 261dbf8
Showing 1 changed file with 11 additions and 2 deletions.
13 changes: 11 additions & 2 deletions packages/malloy/src/dialect/snowflake/snowflake.ts
Original file line number Diff line number Diff line change
Expand Up @@ -204,7 +204,7 @@ export class SnowflakeDialect extends Dialect {
): string {
const as = this.sqlMaybeQuoteIdentifier(alias);
if (isArray) {
return `,LATERAL FLATTEN(INPUT => ${source}) AS ${alias}_1, LATERAL (SELECT ${alias}_1.INDEX, object_construct('value', ${alias}_1.value) as value ) as ${as}`;
return `LEFT JOIN lateral flatten(input => ${source}) as ${as}`;
} else {
// have to have a non empty row or it treats it like an inner join :barf-emoji:
return `LEFT JOIN LATERAL FLATTEN(INPUT => ifnull(${source},[1])) AS ${as}`;
Expand Down Expand Up @@ -266,7 +266,16 @@ export class SnowflakeDialect extends Dialect {
parentType === 'array[scalar]' ||
parentType === 'array[record]'
) {
const arrayRef = `"${parentAlias}".value:${sqlName}`;
// Case 1: this is an array of scalar. We can reference the field
// simply as parent.value. in fact the sqlName will be value always
// because there is nothing else to reference.
// Case 2: this is an array of records. In this case, parent.value
// gives the record and then parent.value:sql_name gives the
// right field in the record.
let arrayRef = `"${parentAlias}".value:${sqlName}`;
if (parentType === 'array[scalar]') {
arrayRef = `"${parentAlias}".value`;
}
switch (childType) {
case 'record':
case 'array':
Expand Down

0 comments on commit 261dbf8

Please sign in to comment.