Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Map INTERVAL types to Python types #475

Merged
merged 2 commits into from
Oct 24, 2024

Conversation

hovaesco
Copy link
Member

@hovaesco hovaesco commented Jul 19, 2024

Description

Resolves #356

  • Map INTERVAL YEAR TO MONTH to relativedelta
  • Map INTERVAL DAY TO SECOND to timedelta

Non-technical explanation

Release notes

( ) This is not user-visible or docs only and no release notes are required.
( ) Release notes are required, please propose a release note for me.
( ) Release notes are required, with the following suggested text:

* Fix some things. ({issue}`issuenumber`)

@cla-bot cla-bot bot added the cla-signed label Jul 19, 2024
@hovaesco hovaesco requested a review from hashhar July 19, 2024 12:11
SqlTest(trino_connection) \
.add_field(sql="CAST(null AS INTERVAL YEAR TO MONTH)", python=None) \
.add_field(sql="INTERVAL '10' YEAR", python=relativedelta(years=10)) \
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Please test for min/max valid trino value and min/max valid python values too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does this test verify both directions? i.e. converting from Trino string representation to Python object AND converting from Python object to Trino SQL (e.g. queries with parameters?)?

Copy link
Member

@hashhar hashhar left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM % questions

.add_field(sql="INTERVAL '-5 23:59:57.000' DAY TO SECOND", python=timedelta(days=-6, seconds=3)) \
.add_field(sql="INTERVAL '12 10:45' DAY TO MINUTE", python=timedelta(days=12, seconds=38700)) \
.add_field(sql="INTERVAL '45:32.123' MINUTE TO SECOND", python=timedelta(seconds=2732, microseconds=123000)) \
.add_field(sql="INTERVAL '32.123' SECOND", python=timedelta(seconds=32, microseconds=123000)) \
Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For INTERVAL '2147483647 23:59:59.999999' DAY TO SECOND I recieved below error. So it looks like we cannot really use timedelta for INTERVAL DAY TO SECOND type.

>       return timedelta(days=days, hours=hours, minutes=minutes, seconds=seconds, milliseconds=milliseconds)
E       OverflowError: days=2147483647; must have magnitude <= 999999999

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As discussed offline let's check what the JDBC driver does when a Trino value is too large/small to fit into corresponding Java type. If it throws an error then the existing implementation we have here seems fine to me.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We do use TrinoDataError in some places where values from Trino cannot be mapped to Python.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added test cases for edge scenarios. Raised TrinoDataError when Trino value is too large/small. PTAL @hashhar

@damian3031 damian3031 force-pushed the hovaesco/interval-mappings branch 2 times, most recently from 585acf1 to d5f5e54 Compare October 10, 2024 06:59
@hashhar hashhar merged commit 4c57774 into trinodb:master Oct 24, 2024
12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Development

Successfully merging this pull request may close these issues.

Map INTERVAL types to Python types
3 participants