Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Generic @with_columns dataframe support #1158

Open
skrawcz opened this issue Sep 25, 2024 · 1 comment
Open

Generic @with_columns dataframe support #1158

skrawcz opened this issue Sep 25, 2024 · 1 comment
Labels
decorators enhancement New feature or request

Comments

@skrawcz
Copy link
Collaborator

skrawcz commented Sep 25, 2024

Is your feature request related to a problem? Please describe.
We have @with_columns for pyspark, but not for other dataframes.

Describe the solution you'd like
@with_columns works for any dataframe type.

Describe alternatives you've considered
N/A

Additional context
This would help with reusing transforms across dataframes without getting into naming issues.

@skrawcz skrawcz added enhancement New feature or request decorators core-work Work that is "core". Likely overseen by core team in most cases. and removed core-work Work that is "core". Likely overseen by core team in most cases. labels Sep 25, 2024
@zilto
Copy link
Collaborator

zilto commented Nov 12, 2024

If we implement @with_columns for Polars, we can leverage Narwhals to support cuDF, Modin, pandas, Polars, Pyarrow, Dask, Ibis, Duckdb, Vaex, and more. We have to distinguish eager and lazy execution though.

Narwhals is an ok dependency to add because @with_columns is already an extension feature and Narwhals itself only depends on the standard library.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
decorators enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants