-
Notifications
You must be signed in to change notification settings - Fork 110
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
SNOW-888898: Session.create_dataframe fails to properly set nullable columns #994
Comments
In this case when you create a dataframe with What you want to do here is explicitely define schema using |
I understand but the current implementation isn't correct. |
I don't understand why line 391 should always be
Does that explain why the current code in line 391 shouldn't always be true? |
Your first point, you mention that if a field is in Just as an example, given the code below:
Currently, writing to a table will fail as mentioned above and the output is:
If I make the suggested change, writing to a table is successful and the output is:
From what I can tell, this second schema represents the provided data more accurately and still considers the nullability of field |
Thanks for the explanation @eugene-roach. You are right. I'll update the code |
Thanks! |
Pretty sure the fix is just to change change https://github.com/snowflakedb/snowpark-python/blob/main/src/snowflake/snowpark/_internal/type_utils.py#L391
to
StructField(n, name_to_datatype_b[n], True)
since the columns is nullable if DataTypea
is missing the fieldThe text was updated successfully, but these errors were encountered: