Lots of UNION ALL causes "memory access out of bounds"

[jasonmcole]

What happens?

When running a query with enough unions, the error "memory access out of bounds" is thrown. Once this error is thrown, all further queries fail to execute with the same error.

duckdb-browser-eh.worker.js.gz:1
RuntimeError: memory access out of bounds
    at 08812f2a:0x1eeaa86
    at 08812f2a:0x1f33b65
    at 08812f2a:0xbf0524
    at 08812f2a:0xb0b644
    at 08812f2a:0xafa20e
    at 08812f2a:0xaf6d4d
    at 08812f2a:0xaf68b5
    at 08812f2a:0xaf9c9a
    at 08812f2a:0xaf6d4d
    at 08812f2a:0xaf68b5

To Reproduce

Run this query, with 86 unions (85 or fewer seem to work fine). This is reproducible on shell.duckdb.org, but not in native DuckDB.

select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num
union all
select 1 as num

Browser/Environment:

Chrome 132.0.6834.160 (arm64)

Device:

Mac Mini, M4 Pro, Sequoia 15.2

DuckDB-Wasm Version:

1.29.1-dev3.0

DuckDB-Wasm Deployment:

https://count.co or https://shell.duckdb.org/

Full Name:

Jason Cole

Affiliation:

Count

[carlopi]

If this were to be a machine generated query, making the tree more unbalanced might help, example:

SELECT 1 AS num UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num)UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num) UNION ALL (SELECT 1 AS num UNION ALL SELECT 1 AS num);

This might work around the problem, and we will eventually do the transformation duckdb/duckdb side automatically (native side has similar problems, but given stack limits are different it's harder to bump against those.

I need to look how to address to avoid the crash in any case.

[jasonmcole]

Thank you for the suggestion!