Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug] Incremental failed for on_schema_change "sync_all_columns" when there's only a type change. #319

Open
2 tasks done
nguyensinhtu opened this issue Sep 27, 2024 · 1 comment
Labels
awaiting_response bug Something isn't working

Comments

@nguyensinhtu
Copy link

nguyensinhtu commented Sep 27, 2024

Is this a new bug?

  • I believe this is a new bug
  • I have searched the existing issues, and I could not find an existing issue for this bug

Current Behavior

For incremental model config:

    config(
        materialized='incremental',
       on_schema_change='sync_all_columns'
    )
}}

select ...

This fails when a column type changes. Following the logic here:

{% macro default__alter_column_type(relation, column_name, new_column_type) -%}
,
in step 2, DBT is trying to copy data with an old type to a temporary column with a new type.

Expected Behavior

It's hard to decide what to do in this case:
Here a few options I can think of:

  • Add casting type when copying data over.
  • Better error msg.
  • Or you should discard support for data type changes.

Steps To Reproduce

  1. with config
config(
        materialized='incremental',
       on_schema_change='sync_all_columns'
    )
  1. Failed on incremental run not full-refresh.

Relevant log output

log error: column "id__dbt_alter" is of type bigint but expression is of type character

Environment

- OS: Ubuntu 20.04
- Python:3.11.6 
- dbt-adapters: 1.5.0

Additional Context

No response

@nguyensinhtu nguyensinhtu added bug Something isn't working triage labels Sep 27, 2024
@amychen1776
Copy link

Hi @nguyensinhtu - I'm curious what adapter are you using this on and what version?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting_response bug Something isn't working
Projects
None yet
Development

No branches or pull requests

2 participants