Update modin dependency to 0.30.1

[sfc-gh-joshi]

1. Which Jira issue is this PR addressing? Make sure that there is an accompanying issue to your PR.

   Fixes SNOW-1552497

2. Fill out the following pre-review checklist:

   • I am adding a new automated test(s) to verify correctness of my new code
     • If this test skips Local Testing mode, I'm requesting review from @snowflakedb/local-testing
   • I am adding new logging messages
   • I am adding a new telemetry message
   • I am adding new credentials
   • I am adding a new dependency
   • If this is a new feature/behavior, I'm adding the Local Testing parity changes.
   • I acknowledge that I have ensured my changes to be thread-safe. Follow the link for more information: Thread-safe Developer Guidelines

3. Please describe how your code solves the related issue.

This PR updates modin to 0.30.1, which unpins the pandas patch version and fixes bugs related to the extensions/docstring modules.

Dependency-related changes:

• Moves the pinned modin version from 0.28.1 to 0.30.1--it is available in Snowflake anaconda but currently blocked on prod/preprod, and must be manually enabled for testing accounts in those environments. We will unblock modin 0.30.1 when the release containing this change is completed.
• Adds a new job to the daily test (test-pandas-patch-versions), which runs the Snowpark pandas daily test suite on AWS with Python 3.9 against all supported pandas versions currently available in anaconda (2.1 and 2.2; we should add 2.3 once it's available in Snowflake anaconda)
• Some native pandas error messages checked in tests have changed; relevant tests now check the error message based on pandas version.
• IMPORTANT The most recent available versions of snowflake-snowpark-python available in Anaconda pin the modin dependency to 0.28.1, so attempting to specify modin==0.30.1 when creating a stored procedure makes the solver pick an older version of snowflake-snowpark-python from before Snowpark pandas was added (1.16.0). As a workaround, in test_modin_stored_procedures.py we need to upload snowflake-snowpark-python as a staged package. Calling apply and applymap (which both create UDFs) within a stored procedure created in this way fails; I have XFAILed those tests for now and need help debugging them (either in this PR or as a separate one). Within test_modin_stored_procedures.py we pin pandas==2.2.1 and modin==0.28.1 to test the version available in Anaconda; other UDF/stored proc-creating tests are XFAILed when the pandas version is 2.2.3 or newer.

Frontend changes:

• modin 0.30.1 introduces the pd.DataFrame.modin accessor object for non-pandas methods, such as pd.DataFrame.modin.to_pandas and pd.DataFrame.modin.to_ray. We will automatically raise NotImplementedError for all methods on this accessor object except to_pandas.
• DataFrame/Series.drop_duplicates was changed to use a different query compiler method to better support distributed backends; for now I've added back an override with the old frontend implementation.
• Series.unique was changed to have an additional to_pandas() call, which does not affect the number of queries since the old implementation called to_numpy(), but does change the return type. I've added back the old frontend implementation as an override.

[sfc-gh-azhan]

When modin 0.30.1 will be available in Snowflake anaconda?

[sfc-gh-dpetersohn]

@sfc-gh-azhan should be very soon. Do you know why all the tests would pass even though it is in Anaconda yet?

[sfc-gh-azhan]

@sfc-gh-azhan should be very soon. Do you know why all the tests would pass even though it is in Anaconda yet?

Those tests here are using pypi. Only the Sproc tests or Snowflake streamlit or notebook tests relies on Anaconda.

[sfc-gh-azhan]

Once I done with https://snowflakecomputing.atlassian.net/browse/SNOW-1563225, I could run your PR for those sproc tests and I believe they will fail before modin is upgraded in Anaconda.

[sfc-gh-jjiao]

@sfc-gh-joshi Thanks for the PR, I like the very clear description with enough details.

[sfc-gh-azhan]

I started https://ci-dev-142.int.snowflakecomputing.com/job/SnowparkPandasStoredProcPrecommitTest/158/. Let's see how it goes. @sfc-gh-joshi

[sfc-gh-helmeleegy]

LGTM. Thanks for keeping our dependencies up-to-date. (But please resolve all open conversations before merging.)

[sfc-gh-joshi]

(from discussion on slack) Holding this PR until after 1.24 is cut.