docs: Document that the write_parquet methods use mkdir in polars

[MoritzPotthoffQC]

Motivation

We would like to clarify in the docs that the write_parquet methods use the mkdir kwarg to control whether directories should be created.

Changes

• Bumped to polars 1.33 (needed for mkdir)
• Updated docs
• Added tests

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 100.00%. Comparing base (22f4b7d) to head (ff60351).
⚠️ Report is 3 commits behind head on main.

Additional details and impacted files

@@            Coverage Diff            @@
##              main      #142   +/-   ##
=========================================
  Coverage   100.00%   100.00%           
=========================================
  Files           49        50    +1     
  Lines         2818      2851   +33     
=========================================
+ Hits          2818      2851   +33     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[MoritzPotthoffQC]

@AndreasAlbertQC I did not consider delta storage in this PR. If there is a more elegant way to fix this, feel free to amend/close this PR.

[borchero]

Mh, I thought that write_parquet takes care of this, unfortunately not... 😅 I think we should be consistent with write_parquet from polars though and not create parents (and document it accordingly).

[borchero]

I think that we shouldn't do this here but rather in the specialized functions for collections. Also, I wouldn't set parents=True (see main comment)

[MoritzPotthoffQC]

Hm fair point. I looked into this more. pl.write_parquet has a parameter mkdir (default False). So to get the exact same behavior, I guess we would have to add this. But I see that the additional complexity is not worth it and we can just always not do it.

We currently rely on the fact that write_parquet sets mkdir to True for partitioned writes and creates all parent directories here because we add an extra layer of directories that is then going to be created by polars.

I think if we do not want an mkdir argument, we should not create anything anywhere, except for partitioned writes. That would not require any code changes, just updates to the docs. Is this what you have in mind?

[MoritzPotthoffQC]

We want to just delegate this to the mkdir parameter in polars.

@borchero I just realized that the mkdir parameter is only exposed from polars 1.33 onwards. So I think we should wait with this PR until we bump there anyway.

[borchero]

We will bump the version requirement as part of #139, do you want to wait until then?

[MoritzPotthoffQC]

Yes, that works for me. I will redraft this for now then.