Skip to content

Commit

Permalink
SNOW-1657459 Improved dtype results for TIMESTAMP_LTZ type to show co…
Browse files Browse the repository at this point in the history
…rrect timezone (#2328)

<!---
Please answer these questions before creating your pull request. Thanks!
--->

1. Which Jira issue is this PR addressing? Make sure that there is an
accompanying issue to your PR.

   <!---
   In this section, please add a Snowflake Jira issue number.
   
Note that if a corresponding GitHub issue exists, you should still
include
   the Snowflake Jira issue number. For example, for GitHub issue
#1400, you should
   add "SNOW-1335071" here.
    --->

   Fixes SNOW-1657459

2. Fill out the following pre-review checklist:

- [ ] I am adding a new automated test(s) to verify correctness of my
new code
- [ ] If this test skips Local Testing mode, I'm requesting review from
@snowflakedb/local-testing
   - [ ] I am adding new logging messages
   - [ ] I am adding a new telemetry message
   - [ ] I am adding new credentials
   - [ ] I am adding a new dependency
- [ ] If this is a new feature/behavior, I'm adding the Local Testing
parity changes.

3. Please describe how your code solves the related issue.

Please write a short description of how your code change solves the
related issue.

make sure dtype for timestamp_ltz is correct.
  • Loading branch information
sfc-gh-azhan authored Sep 19, 2024
1 parent f07d7d1 commit cd1133b
Show file tree
Hide file tree
Showing 4 changed files with 13 additions and 1 deletion.
1 change: 1 addition & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@

- Improved `to_pandas` to persist the original timezone offset for TIMESTAMP_TZ type.
- Improved `dtype` results for TIMESTAMP_TZ type to show correct timezone offset.
- Improved `dtype` results for TIMESTAMP_LTZ type to show correct timezone.
- Improved error message when passing non-bool value to `numeric_only` for groupby aggregations.
- Removed unnecessary warning about sort algorithm in `sort_values`.

Expand Down
9 changes: 9 additions & 0 deletions src/snowflake/snowpark/modin/plugin/_internal/frame.py
Original file line number Diff line number Diff line change
Expand Up @@ -1489,5 +1489,14 @@ def get_datetime64tz_from_timestamp_tz(
timestamp_tz_snowfalke_quoted_identifier,
)

def get_datetime64tz_from_timestamp_ltz(self) -> DatetimeTZDtype:
"""
Get DatetimeTZDtype for TIMESTAMP_LTZ by reading the session local timezone.
"""
tz = self.ordered_dataframe.session._conn._get_client_side_session_parameter(
"TIMEZONE", default_value="UTC"
)
return DatetimeTZDtype(tz=tz)

# END: Internal Frame mutation APIs.
###########################################################################
Original file line number Diff line number Diff line change
Expand Up @@ -496,6 +496,8 @@ def _get_dtypes(
return [
self._modin_frame.get_datetime64tz_from_timestamp_tz(i)
if t == TimestampType(TimestampTimeZone.TZ)
else self._modin_frame.get_datetime64tz_from_timestamp_ltz()
if t == TimestampType(TimestampTimeZone.LTZ)
else TypeMapper.to_pandas(t)
for i, t in type_map.items()
]
Expand Down
2 changes: 1 addition & 1 deletion tests/integ/modin/test_dtype_mapping.py
Original file line number Diff line number Diff line change
Expand Up @@ -265,7 +265,7 @@
(
"timestamp_ltz timestamp_ltz",
"values ('2023-01-01 00:00:01.001'), ('2023-12-31 23:59:59.999')",
dtype("<M8[ns]"),
"datetime64[ns, America/Los_Angeles]",
"datetime64[ns, America/Los_Angeles]",
native_pd.DataFrame(
[
Expand Down

0 comments on commit cd1133b

Please sign in to comment.