Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

SNOW-878135 is_permanent behavior fix #989

Merged
merged 8 commits into from
Aug 8, 2023

Conversation

sfc-gh-aalam
Copy link
Contributor

Please answer these questions before submitting your pull requests. Thanks!

  1. What GitHub issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

    Fixes #SNOW-878135

  2. Fill out the following pre-review checklist:

    • I am adding a new automated test(s) to verify correctness of my new code
    • I am adding new logging messages
    • I am adding a new telemetry message
    • I am adding new credentials
    • I am adding a new dependency
  3. Please describe how your code solves the related issue.

    Please write a short description of how your code change solves the related issue.

@sfc-gh-aalam sfc-gh-aalam changed the title SNOW-878135 is permanent behavior fix SNOW-878135 is_permanent behavior fix Aug 4, 2023
@sfc-gh-aalam sfc-gh-aalam marked this pull request as ready for review August 4, 2023 23:31
@sfc-gh-aalam sfc-gh-aalam requested a review from a team as a code owner August 4, 2023 23:31
@codecov
Copy link

codecov bot commented Aug 5, 2023

Codecov Report

Merging #989 (5082600) into main (2830b9a) will increase coverage by 0.01%.
The diff coverage is 100.00%.

❗ Current head 5082600 differs from pull request most recent head d979db2. Consider uploading reports for the commit d979db2 to get more accurate results

@@            Coverage Diff             @@
##             main     #989      +/-   ##
==========================================
+ Coverage   98.45%   98.46%   +0.01%     
==========================================
  Files          51       51              
  Lines        9318     9317       -1     
  Branches     1696     1695       -1     
==========================================
  Hits         9174     9174              
  Misses         59       59              
+ Partials       85       84       -1     
Files Changed Coverage Δ
src/snowflake/snowpark/_internal/udf_utils.py 95.49% <100.00%> (+0.25%) ⬆️
src/snowflake/snowpark/stored_procedure.py 97.64% <100.00%> (ø)
src/snowflake/snowpark/udaf.py 91.54% <100.00%> (ø)
src/snowflake/snowpark/udf.py 97.40% <100.00%> (ø)
src/snowflake/snowpark/udtf.py 97.29% <100.00%> (ø)

... and 1 file with indirect coverage changes

📣 We’re building smart automated test selection to slash your CI/CD build times. Learn more


if parallel < 1 or parallel > 99:
raise ValueError(
"Supported values of parallel are from 1 to 99, " f"but got {parallel}"
)

return stage_location if is_permanent else None
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think checking and warning about stage_location here is a good idea, but for the actual fix, should we consider passing down is_permanent to _do_register_udf, instead of using stage_location as a proxy to deduce its value? Will that be less error prone?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah. I think that is the right way to do things here. I'll reformat the code.

@sfc-gh-stan
Copy link
Collaborator

Does it make sense to get rid of is_permanent?

@sfc-gh-aalam
Copy link
Contributor Author

Does it make sense to get rid of is_permanent?

Not to me. If we get rid of it, do you suggest we use stage_location as a proxy? I don't think that's intuitive.

Copy link
Collaborator

@sfc-gh-sfan sfc-gh-sfan left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We probably also want to add similar test to udtf and udaf?

CHANGELOG.md Outdated
@@ -9,6 +9,10 @@
- `array_flatten`
- Added support for replicating your local Python environment on Snowflake via `Session.replicate_local_environment`.

### Behavior Changes

- When creating stored procedures, UDFs, UDTFs, UDAFs with parameter `is_permanent=False` will now create temporary objects even when `stage_name` is provided. Earlier `is_permanent=False` with non-None `stage_name` try to create permanent objects at the given stage location.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we also need to call out that the default behavior is changing because the default value of is_permanent is None. So the behavior of a UDF that only has stage_name specified will see their UDF becomes temporary.

@@ -790,7 +793,7 @@ def _do_register_sp(
object_name=udf_name,
all_imports=all_imports,
all_packages=all_packages,
is_temporary=stage_location is None,
is_temporary=not is_permanent,
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if it makes sense to rename is_temporary to is_permanent? This way we only need to do negation once instead of in all four classes.

We probably should have made the public param is_temporary at the beginning so we don't need this awkward flip lol. But now all is done 🤷

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we could rename is_temporary -> is_permanent. public API already has is_permanent so we shouldn't change that.

@sfc-gh-stan
Copy link
Collaborator

e stage_location as a

What about something like permanent_stage_location ? I prefer only having one parameter here but I understand that will be a BCR.

@sfc-gh-aalam sfc-gh-aalam merged commit 10af52d into main Aug 8, 2023
40 checks passed
@sfc-gh-aalam sfc-gh-aalam deleted the aalam-SNOW-878135-is-permanent-behavior-fix branch August 8, 2023 23:44
@github-actions github-actions bot locked and limited conversation to collaborators Aug 8, 2023
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants