Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Update pyarrow version, use pyarrow backend for pandas #513

Merged
merged 5 commits into from
Nov 27, 2023
+164 −184
Merged

Update pyarrow version, use pyarrow backend for pandas #513

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
5 commits
Select commit Hold shift + click to select a range
1a6a75a
Update pyarrow version, los prediction notebook
amrit110 Nov 24, 2023
a7c4dae
Small fix to unique (cast as np array)
amrit110 Nov 24, 2023
fc7bdc3
Merge branch 'main' into update_pyarrow
amrit110 Nov 24, 2023
ec1031d
Update dependencies in pyproject.toml
fcogidi Nov 27, 2023
3399546
Fix data type check in `filter_range` function
fcogidi Nov 27, 2023
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
7 changes: 4 additions & 3 deletions cyclops/data/slicer.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@
import numpy as np
import pandas as pd
from dateutil.parser import parse
from pandas.api.types import is_datetime64_any_dtype, is_numeric_dtype


@dataclass
Expand Down Expand Up @@ -552,11 +553,11 @@ def filter_range(
example_values = pd.Series(
examples[column_name],
dtype="datetime64[ns]" if value_is_datetime else None,
).to_numpy()
)

if not ( # column does not contain number or datetime values
np.issubdtype(example_values.dtype, np.number)
or np.issubdtype(example_values.dtype, np.datetime64)
is_numeric_dtype(example_values.dtype)
or is_datetime64_any_dtype(example_values.dtype)
):
raise TypeError(
"Expected feature to be numeric or datetime, but got "
Expand Down
2 changes: 1 addition & 1 deletion cyclops/process/feature/handle_types.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -50,7 +50,7 @@ def get_unique(

"""
if unique is None:
return values.unique() # type: ignore
return np.array(values.unique()) # type: ignore

return unique

Expand Down
Loading