Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hamilton tracker gives an error with datetime columns in polars dataframes #1127

Open
elyase opened this issue Sep 8, 2024 · 3 comments
Open
Labels
bug Something isn't working SDK Related to hamilton SDK for th UI triage label for issues that need to be triaged.

Comments

@elyase
Copy link
Contributor

elyase commented Sep 8, 2024

Reproduction

from datetime import datetime

import polars as pl
from hamilton import driver
from hamilton_sdk import adapters

import __main__ as dag


def df() -> pl.Series:
    return pl.Series(
        "timestamp",
        [
            datetime(2021, 1, 1),
            datetime(2021, 1, 2),
            datetime(2021, 1, 3),
        ],
    )

tracker = adapters.HamiltonTracker(
    project_id=1,
    username="elyase",
    dag_name="polars",
)

dr = driver.Builder().with_modules(dag).with_adapters(tracker).build()
result = dr.execute(["df"])

Stack Traces

  File "/Users/yaser/Documents/shitcoins/.venv/lib/python3.12/site-packages/hamilton/node.py", line 249, in __call__
    return self.callable(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/elyase/test/.venv/lib/python3.12/site-packages/hamilton_sdk/tracking/polars_col_stats.py", line 52, in std
    return col.std()
           ^^^^^^^^^
  File "/Users/elyase/test/.venv/lib/python3.12/site-packages/polars/series/series.py", line 2049, in std
    return self._s.std(ddof)
           ^^^^^^^^^^^^^^^^^
polars.exceptions.InvalidOperationError: `std` operation not supported for dtype `datetime[μs]`
@elyase elyase added the triage label for issues that need to be triaged. label Sep 8, 2024
@skrawcz skrawcz linked a pull request Sep 9, 2024 that will close this issue
7 tasks
@skrawcz
Copy link
Collaborator

skrawcz commented Sep 9, 2024

Thanks @elyase ! We might not have kept up with all the polars changes.

@skrawcz
Copy link
Collaborator

skrawcz commented Sep 9, 2024

Do you have an example dataframe this breaks on?

@skrawcz skrawcz added bug Something isn't working SDK Related to hamilton SDK for th UI labels Sep 9, 2024
@elyase
Copy link
Contributor Author

elyase commented Sep 10, 2024

hi @skrawcz, thanks for your reply, I edited the ticket with a reproduction

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working SDK Related to hamilton SDK for th UI triage label for issues that need to be triaged.
Projects
None yet
Development

No branches or pull requests

2 participants