Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[components] Add workspace.yaml scaffolding (BUILD-652) #27477

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

smackesey
Copy link
Collaborator

@smackesey smackesey commented Feb 1, 2025

Summary & Motivation

This adds workspace.yaml scaffolding to dg deployment scaffold and dg code-location scaffold. We will want to replace this in the future with whatever pyproject.toml-based replacement API we have in mind for workspace.yaml. But in the meantime, it makes sense to scaffold workspace.yaml.

  • dg deployment scaffold will give you an empty workspace.yaml (except for some explanatory comments)
  • dg code-location scaffold will modify the workspace.yaml, injecting the necessary entry for the code location to be loaded in its own environment by dagster dev

Caveats:

  • The method of YAML modification wipes out comments-- workspace.yaml becomes a dg-managed file.

How I Tested These Changes

New unit tests.

Copy link
Collaborator Author

smackesey commented Feb 1, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

@smackesey smackesey changed the title [components] Add workspace.yaml scaffolding [components] Add workspace.yaml scaffolding (BUILD-652) Feb 1, 2025
@smackesey smackesey force-pushed the sean/components/code-location-scaffold-workspace branch from 5a68c83 to c147077 Compare February 1, 2025 00:35
@smackesey smackesey marked this pull request as ready for review February 1, 2025 00:36
@smackesey smackesey requested a review from neverett as a code owner February 1, 2025 00:36
Copy link

github-actions bot commented Feb 1, 2025

Deploy preview for dagster-docs ready!

Preview available at https://dagster-docs-nmgd6t8i9-elementl.vercel.app

Direct link to changed pages:

@smackesey smackesey force-pushed the sean/components/code-location-scaffold-workspace branch from c147077 to e5052e0 Compare February 1, 2025 00:59
# for each code location.
#
# ##### EXAMPLE
#
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove trailing whitespace after the # character to fix Vale linting error

Spotted by Graphite Reviewer (based on CI logs)

Is this helpful? React 👍 or 👎 to let us know.

@smackesey smackesey force-pushed the sean/components/code-location-scaffold-workspace branch from e5052e0 to 0599367 Compare February 1, 2025 01:56
@schrockn
Copy link
Member

schrockn commented Feb 1, 2025

We should consider directly jumping to a pyproject.yaml-based approach IMO.

Copy link
Member

@schrockn schrockn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

dg code-location scaffold will modify the workspace.yaml, injecting the necessary entry for the code location to be loaded in its own environment by dagster dev

Requesting changes based on this, which I do not think we should support. Let's just add the capability to pyproject.toml

@smackesey
Copy link
Collaborator Author

Requesting changes based on this, which I do not think we should support. Let's just add the capability to pyproject.toml

Hmm ok. Thought I would need this for dg dev but am just generating a temporary workspace.yaml under the hood for now: #27481

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants