Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: added snowflake warehouse vars config #1281

Conversation

christopher-turnbull
Copy link

resolves #1280
docs dbt-labs/docs.getdbt.com/#

Problem

Currently, users can only specify Snowflake warehouses through model-level config blocks or project-level configurations in dbt_project.yml. There's no way to override the warehouse at runtime without modifying these files, which makes it difficult to dynamically change warehouses for different scenarios.

Solution

This PR adds support for overriding the Snowflake warehouse using dbt vars, which works for both dbt run and dbt test commands. Users can now specify a warehouse at runtime like:

dbt run --vars '{snowflake_warehouse: LARGE_WAREHOUSE}'
dbt test --vars '{snowflake_warehouse: LARGE_WAREHOUSE}'

This provides more flexibility for users who need to:

  • Dynamically change warehouses without modifying model or project files
  • Use different warehouses for different environments or contexts
  • Override warehouse settings in CI/CD pipelines

All tests have been added and are passing, including specific tests for both run and test commands.

Checklist

  • I have read the contributing guide and understand what's expected of me
  • I have run this code in development and it appears to resolve the stated issue
  • This PR includes tests, or tests are not required/relevant for this PR
  • This PR has no interface changes (e.g. macros, cli, logs, json artifacts, config files, adapter interface, etc) or this PR has already received feedback and approval from Product or DX

Checklist

@christopher-turnbull christopher-turnbull requested a review from a team as a code owner January 3, 2025 20:34
Copy link

cla-bot bot commented Jan 3, 2025

Thanks for your pull request, and welcome to our community! We require contributors to sign our Contributor License Agreement and we don't seem to have your signature on file. Check out this article for more information on why we have a CLA.

In order for us to review and merge your code, please submit the Individual Contributor License Agreement form attached above above. If you have questions about the CLA, or if you believe you've received this message in error, please reach out through a comment on this PR.

CLA has not been signed by users: @christopher-turnbull

@amychen1776
Copy link

amychen1776 commented Jan 9, 2025

Thank you for creating this PR and doing this work. Unfortunately, this is not how we would want to support this functionality. By treating the snowflake_warehouse config as special, this deviated from how we have designed everything and made scalability/maintenance harder (because it's an edge case that would have to be considered). If we were to support passing in vars at a project level, it can't just be for this input but something we would likely support at the framework level on dbt-adapters or dbt core. Due to this, I will close this PR for now to keep things clean to remove it from the project board. We are open to new PRs of course with something more scalable.

As you know, closing the PR does not mean we can't continue the discussion here either.

@amychen1776 amychen1776 closed this Jan 9, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Feature] Support Snowflake warehouse override without model configs
2 participants