Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: "Dependencies" View in Compiled Query Preview Tab #49

Open
HampB opened this issue Nov 11, 2024 · 3 comments
Open

feat: "Dependencies" View in Compiled Query Preview Tab #49

HampB opened this issue Nov 11, 2024 · 3 comments
Assignees
Labels
enhancement New feature or request

Comments

@HampB
Copy link
Collaborator

HampB commented Nov 11, 2024

Summary
It would be useful to add a dedicated "Dependencies" view within the Compiled Query Preview tab, displaying only the upstream and downstream dependencies of the currently active model. This would offer a quick overview of dependencies without needing to load the full dependency tree (even with the fuzzysearch this is not nice when running hundreds of graph nodes) .

Proposed Implementation
Add a "Dependencies" button in the Compiled Query Preview tab, as shown here:
image

Future Enhancements
In later iterations, this view (and the existing dep graph) could support:

  • Selecting one or more nodes to initiate a run directly from the dependencies view.
  • Displaying additional details for each node, such as type, columns, and other metadata (similar to dbt Power User’s model lineage example: https://docs.myaltimate.com/test/lineage/#model-lineage).

--

Any thoughts on that?

@ashish10alex
Copy link
Owner

ashish10alex commented Nov 11, 2024

Hi @HampB ,
Nice idea, I have been thinking about showing the dependencies in the webview but had not yet added something as dependencies could be seen from the sideview panel. However, most users would not know about this as it is not explicitly visible.

--

  • I have now added dependencies as a collapsable item it the compiled query web view instead of a separable tab for now. Please see the MR for a video demo

  • Showing the dependent will need some additional development work as the output of dataform --json only has dependencies by default. So, I assume we need to manually compute dependents ?

CleanShot 2024-11-11 at 17 12 48@2x

--

Future enhancements

  • Selecting one or more nodes to initiate a run directly from the dependencies view - Yes, we can explore this as a future feature to add. For now, I personally, use command pallet to run most operations. However, most users might not

  • Displaying additional details for each node - Yes, at some point. The reason I have not pushed to make the dependency graph pretty is (1) the currently one is functional and lets me do most things I want. (2) Time I spend with dependancy graph is much smaller than the time spend on looking at compiled query, running the code etc. (3) Currently the web view for dependancy graph is written in raw javascript. To use something for sophisticated like react flow we might have to use react to make the development smoother.

@HampB
Copy link
Collaborator Author

HampB commented Nov 12, 2024

Nice, that’s a great start! I believe that dependents are generally less crucial than dependencies, but it would still be nice to calculate the dependents.

I’m with you on not needing a fully refined dependency graph for now—the current version serves its purpose, especially since most of the work is focused on compiled queries and running code.

However, I do think there’s still a need for easy navigation through dependencies at multiple levels. I that my initial idea on just showing the dependency graph may not be ideal for larger dependency trees. For instance, here’s a large model with the names removed:
image

I’m not entirely sure of the best approach to achieve this—perhaps a nested, collapsible structure could work?

@ashish10alex
Copy link
Owner

Hi @HampB / @benjaminwestern ,
I have added the feature to show the dependent in the compiled query web view in this PR. @benjaminwestern , happy to migrate to the larger refactor you were working on once that is ready. But I wanted something users could use in the meantime.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants