Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Ensure py.typed is present in all packages (BUILD-660) #27491

Merged
merged 2 commits into from
Feb 2, 2025

Conversation

smackesey
Copy link
Collaborator

@smackesey smackesey commented Feb 2, 2025

Summary & Motivation

  • Add py.typed to all Python packages (anything with a setup.py that can be installed into an environment)
  • Add a test ensuring all packages have py.typed, and all published packages include it in their MANIFEST.in

There are a few reasons for this:

  • In published packages, type annotations from a package will not always be used by a static analyzer if py.typed is not present. In particular, pyright/pylance will not use annotations from a non-py.typed package unless useLibraryCodeForTypes is enabled (and we don't control whether our users enable this). Not all of our packages are 100% typed, but they at least do not provide wrong type information (which some packages, especially those using fancy metaprogramming, do), which means it is safe to expose the type information they do have.
  • Even our non-published packages need py.typed to avoid some strange behavior around make quick_pyright, which runs in our configured pre-push hooks. If only some files in a non-py.typed package are touched in a diff, it can lead to odd import resolution errors. This occurs because the files not touched by the diff are both not passed directly to pyright for inspection, and also unavailable from the venv environment pyright uses due to lacking py.typed. The result is the occasional seemingly random error when make quick_pyright is run-- this PR fixes that.

How I Tested These Changes

See notes on above test.

Changelog

All published dagster libraries now include a py.typed file, which means their type annotations will be used by static analyzers. Previously a few libraries were missing this file.

Copy link
Collaborator Author

smackesey commented Feb 2, 2025

This stack of pull requests is managed by Graphite. Learn more about stacking.

@smackesey smackesey marked this pull request as ready for review February 2, 2025 03:51
@smackesey smackesey requested a review from neverett as a code owner February 2, 2025 03:51
@smackesey smackesey requested review from alangenfeld and schrockn and removed request for neverett February 2, 2025 03:52
Comment on lines +48 to +53
def git_ls_files(pattern: str) -> list[str]:
return (
subprocess.run(["git", "ls-files", pattern], check=True, text=True, capture_output=True)
.stdout.strip()
.split("\n")
)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

When git ls-files finds no matches, stdout.strip() returns an empty string, which split('\n') converts to ['']. To handle this case correctly, the code should return an empty list instead:

.split('\n') if stdout.strip() else []

Spotted by Graphite Reviewer

Is this helpful? React 👍 or 👎 to let us know.

@smackesey smackesey force-pushed the sean/py-typed-packages branch from 8e0d228 to 65ff6fc Compare February 2, 2025 03:56
Copy link

github-actions bot commented Feb 2, 2025

Deploy preview for dagster-docs ready!

Preview available at https://dagster-docs-crvhf1x5x-elementl.vercel.app

Direct link to changed pages:

@smackesey smackesey changed the base branch from master to graphite-base/27491 February 2, 2025 04:15
@smackesey smackesey force-pushed the sean/py-typed-packages branch from 65ff6fc to 6ae2e37 Compare February 2, 2025 04:15
@smackesey smackesey force-pushed the graphite-base/27491 branch from 357f918 to 79a0cdb Compare February 2, 2025 04:15
@smackesey smackesey changed the base branch from graphite-base/27491 to sean/components/components-py-typed February 2, 2025 04:15
Copy link
Member

@schrockn schrockn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

👍🏻

Base automatically changed from sean/components/components-py-typed to master February 2, 2025 21:00
Copy link
Collaborator Author

smackesey commented Feb 2, 2025

Merge activity

  • Feb 2, 4:02 PM EST: A user started a stack merge that includes this pull request via Graphite.
  • Feb 2, 4:02 PM EST: A user merged this pull request with Graphite.

@smackesey smackesey merged commit 525fcbb into master Feb 2, 2025
2 of 3 checks passed
@smackesey smackesey deleted the sean/py-typed-packages branch February 2, 2025 21:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants