Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

What's the best practice for managing Dynamic tables through schemachange, versioned or repeatable? #262

Open
mayur0207 opened this issue Jul 16, 2024 · 2 comments
Labels
question Further information is requested Workaround Issues that can be addressed via a workaround

Comments

@mayur0207
Copy link

mayur0207 commented Jul 16, 2024

** What are you trying to accomplish? **
We are using dynamic tables heavily. Decided to use schemachange for Gitlab CI pipeline.
I've structured the folders, but there's not much documentation about how to handle dynamic tables, should they be treated as versioned objects similar to simple SQL tables or repeatable objects like views?

** What options have you tried so far ? **
For now, I've structured them as versioned objects, but obvious concern is if there's any change to implementation logic of DT, need to create new version script for DT
But if I put them as repeatable, then problem is data may get deleted on next CI pipeline run.
Please provide guidance and recommended approach, it'll be helpful for others if this is included in the documentation of schemachange as well

@jamesweakley, @jeremiahhansen

@mayur0207 mayur0207 added question Further information is requested Workaround Issues that can be addressed via a workaround labels Jul 16, 2024
@jamesweakley
Copy link
Collaborator

Hi @mayur0207 ,
Based on my limited use of dynamic tables so far, changes to their implementation logic (the body of the query), are designed to be a drop+create action, regardless of which change management tool you're using. So for that reason, I think they're a good fit for versioned objects and I don't really see any other way.

@ryanlewist
Copy link

Dynamic tables belong in repeatable scripts.
I don't believe you will experience any data loss on subsequent CI pipeline runs unless you make a change to the dynamic table script, and if you do make a change to the script, then you would want to recreate the dynamic table anyway.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested Workaround Issues that can be addressed via a workaround
Projects
None yet
Development

No branches or pull requests

3 participants