Commits on Sep 20, 2024

1. DeltaCatalog#createTable should respect write options (delta-io#3674) …

   <!--
   Thanks for sending a pull request!  Here are some tips for you:
   1. If this is your first time, please read our contributor guidelines:
   https://github.com/delta-io/delta/blob/master/CONTRIBUTING.md
   2. If the PR is unfinished, add '[WIP]' in your PR title, e.g., '[WIP]
   Your PR title ...'.
     3. Be sure to keep the PR description updated to reflect all changes.
     4. Please write your PR title to summarize what this PR proposes.
   5. If possible, provide a concise example to reproduce the issue for a
   faster review.
   6. If applicable, include the corresponding issue number in the PR title
   and link it in the body.
   -->
   
   <!--
   Please add the component selected below to the beginning of the pull
   request title
   For example: [Spark] Title of my pull request
   -->
   
   - [x] Spark
   - [ ] Standalone
   - [ ] Flink
   - [ ] Kernel
   - [ ] Other (fill in here)
   
   <!--
   - Describe what this PR changes.
   - Describe why we need the change.
   
   If this PR resolves an issue be sure to include "Resolves #XXX" to
   correctly link and close the issue upon merge.
   -->
   
   As of today, Delta extracts write options from table properties only for
   CTAS (in `StagedDeltaTableV2#commitStagedChanges`), but not for CREATE
   TABLE. In general, this makes sense because CREATE TABLE has no
   data-writing but CTAS has. However, the write options can be file system
   configs that we should respect because CREATE TABLE needs to access
   Delta logs.
   
   This PR makes Delta CREATE TABLE to follow CTAS and also extract write
   options from table properties.
   
   <!--
   If tests were added, say they were added here. Please make sure to test
   the changes thoroughly including negative and positive cases if
   possible.
   If the changes were tested in any way other than unit tests, please
   clarify how you tested step by step (ideally copy and paste-able, so
   that other reviewers can test and check, and descendants can verify in
   the future).
   If the changes were not tested, please explain why.
   -->
   Locally tested with Unity Catalog. It's hard to write a test in Delta
   because the write options are not persisted but only used during table
   creation.
   
   <!--
   If yes, please clarify the previous behavior and the change this PR
   proposes - provide the console output, description and/or an example to
   show the behavior difference if possible.
   If possible, please also clarify if this is a user-facing change
   compared to the released Delta Lake versions or within the unreleased
   branches such as master.
   If no, write 'No'.
   -->
   No

   

   cloud-fan committed Sep 20, 2024
   Configuration menu

   • View commit details
   • Copy full SHA for 3b31c9e
   • Browse repository at this point

   Copy the full SHA

   3b31c9e View commit details

   Browse the repository at this point in the history