Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat: Add environments command to list environments #3770

Open
wants to merge 7 commits into
base: main
Choose a base branch
from

Conversation

lafirm
Copy link
Contributor

@lafirm lafirm commented Feb 1, 2025

Fixes #3732

I came across this feature request and found it interesting. I made some assumptions and created this PR. Please review and share your feedback. thanks :)

sqlmesh/cli/main.py Outdated Show resolved Hide resolved
sqlmesh/cli/main.py Outdated Show resolved Hide resolved
@izeigerman
Copy link
Member

Thanks a lot for your contribution! Left some comments

@lafirm lafirm force-pushed the list_environments branch 10 times, most recently from 76e3f18 to eda4fa7 Compare February 9, 2025 14:11
@lafirm lafirm force-pushed the list_environments branch from 639bdd6 to 91d134b Compare February 9, 2025 14:38
@lafirm lafirm requested a review from izeigerman February 9, 2025 14:39
@lafirm
Copy link
Contributor Author

lafirm commented Feb 10, 2025

@izeigerman, have reworked the PR based on your comments, really appreciate if you re-review. Thanks :)

sqlmesh/cli/main.py Show resolved Hide resolved
sqlmesh/core/context.py Outdated Show resolved Hide resolved
Comment on lines +138 to +146
@abc.abstractmethod
def get_environment_names(
self, get_expiry_ts: bool = True
) -> t.Optional[t.List[t.Tuple[str, ...]]]:
"""Fetches all environment names along with expiry datetime if get_expiry_ts is True.
Returns:
A list of all environment names along with expiry datetime if get_expiry_ts is True.
"""
Copy link
Contributor

@georgesittas georgesittas Feb 12, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

  • This will make the PR breaking, FYI.
  • I wonder if we want to make this more generic, i.e., being able to query any state table, possibly selecting only a certain subset of fields. I don't feel strongly about this, but the current API feels somewhat weird, because the method is "get names" but we're including a flag to fetch another field. I'll defer to Iaroslav.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

yeah, I understand your point but,

I wonder if we want to make this more generic, i.e., being able to query any state table, possibly selecting only a certain subset of fields

  1. can be more generic, but I haven't used any new method in this PR but tweaked _environments_query method to select only the fields we want. And I feel that making a generic method should be a separate PR, however I really appreciate your view on this. @izeigerman, @georgesittas

I don't feel strongly about this, but the current API feels somewhat weird, because the method is "get names" but we're including a flag to fetch another field.

  1. I've seen this kind of method in many places, where we add an arg and that makes the method name partially true. One for ex. is plan method in our project which not only plans but apply the changes when we add arg auto_apply. I felt that its okay, but happy to change if you guys disagree.

sqlmesh/core/state_sync/engine_adapter.py Outdated Show resolved Hide resolved
def environments(self, context: Context, line: str) -> None:
"""Prints the list of SQLMesh environments with its expiry datetime."""
args = parse_argstring(self.environments, line)
context.print_environment_names(show_expiry=args.show_expiry)
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[Nit] this is an opinionated styling suggestion, don't necessarily have to do it in this PR if others agree with it:

Active SQLMesh environments (<number>):
├── prod
├── dev_1
    ...
└── dev_n

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

"Active" in this messaging implies that we'd only be showing unexpired environments. Haven't cross-checked if this is true in this PR. Perhaps something worth considering?

Copy link
Contributor Author

@lafirm lafirm Feb 13, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think that the actual use case is to find the list of unexpired (active) envs and invalidate it and run the janitor to cleanup. What you're saying makes sense but its good to see the expired envs (to run janitor) as well I believe, can't think of any use case around showing all the environments including the expired envs except when the janitor has been skipped for a long time.

While thinking about this, I got a random question - what'd happen in the backend when I've an expired environment called env1 and I created a plan for the same env1. will it rejuvenate the env1 or invalidate, janitor process and create the env1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Add command to list environments
3 participants