Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[AMORO-3176] Support multiple format in internal catalog #3177

Merged
merged 1 commit into from
Sep 6, 2024

Conversation

zhoujinsong
Copy link
Contributor

Why are the changes needed?

Close #3176.

Brief change log

  • Make InternalCatalogImpl support both Iceberg and Mixed-Iceberg tables.
  • Fix some bugs in spark about using unified catalog.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not documented)

@github-actions github-actions bot added module:mixed-spark Spark module for Mixed Format module:ams-server Ams server module module:mixed-hive Hive moduel for Mixed Format labels Sep 5, 2024
@zhoujinsong
Copy link
Contributor Author

Tested with the following SQL with the internal catalog:

create database mixed;
create database iceberg;

create table mixed.tb1 (
    id int,
    name string, 
    ts timestamp,
    primary key (id)
) using mixed_iceberg 
partitioned by (days(ts));

create table iceberg.tb1 (
    id int,
    name string, 
    ts timestamp
) using iceberg 
partitioned by (days(ts));

Copy link
Contributor

@baiyangtx baiyangtx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@zhoujinsong zhoujinsong merged commit a1a1219 into apache:master Sep 6, 2024
4 checks passed
@zhoujinsong zhoujinsong deleted the support-rest-unified branch September 6, 2024 02:24
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
module:ams-server Ams server module module:mixed-hive Hive moduel for Mixed Format module:mixed-spark Spark module for Mixed Format
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Bug]: Cannot use both Iceberg and Mixed-Iceberg tables in the internal catalog at the same time
2 participants