[AMORO-3301] Support OSS for iceberg in InternalCatalog

[shouwangyw]

Why are the changes needed?

Close #3301.

Brief change log

• Add OSS storage type, support OSS for iceberg in InternalCatalog

How was this patch tested?

• Add some test cases that check the changes thoroughly including negative and positive cases if possible

• Add screenshots for manual tests if appropriate

• Run test locally before making a pull request

Documentation

• Does this pull request introduce a new feature? (yes / no)
• If yes, how is the feature documented? (not applicable / docs / JavaDocs / not documented)

[czy006]

Thank you for your contribution. As far as I know, the current S3 protocol can also use Ali Cloud OSS object storage. Is iceberg storage support added here? By the way, we should make OSS optional at compile time

[czy006]

It is best to have screenshots or unit tests to verify that the functionality is correct ～

[shouwangyw]

Thank you for your contribution. As far as I know, the current S3 protocol can also use Ali Cloud OSS object storage. Is iceberg storage support added here? By the way, we should make OSS optional at compile time

I don't understand how to use Ali Cloud OSS object storage with the S3 protocol？
This PR has added Ali Cloud OSS related dependencies

[shouwangyw]

OSS support can be correctly added through the front end

I have completed the test in my local environment, able to obtain relevant data correctly

[czy006]

you can use s3 protocol to read it

cc @shouwangyw

[czy006]

CI is Failing

[shouwangyw]

you can use s3 protocol to read it

cc @czy006

But this might not be intuitive

[shouwangyw]

CI is Failing

I have fixed the ci

[czy006]

you can use s3 protocol to read it
cc @czy006

But this might not be intuitive

Of course this is a compromise. I would like to ask if we can have a compilation switch to compile the OSS part when we needed? As object storage support increases, compiling all by default will increase Amoro's binary installation package

[shouwangyw]

you can use s3 protocol to read it
cc @czy006

But this might not be intuitive

Of course this is a compromise. I would like to ask if we can have a compilation switch to compile the OSS part when we needed? As object storage support increases, compiling all by default will increase Amoro's binary installation package

yes, I can choose whether to package aliyun sdk by scope

[czy006]

Thanks for your contribution, I left a few comments. and suggest include documentation about this feature

[czy006]

I recommend not compiling Aliyun OSS by default, if the user wants to use it can use -Paliyun-oss-sdk

[codecov-commenter]

Codecov Report

All modified and coverable lines are covered by tests ✅

Project coverage is 22.54%. Comparing base (f13f3f3) to head (99e0eba).
Report is 7 commits behind head on master.

❗ There is a different number of reports uploaded between BASE (f13f3f3) and HEAD (99e0eba). Click for more details.

HEAD has 1 upload less than BASE

Flag	BASE (f13f3f3)	HEAD (99e0eba)
core	1	0

Additional details and impacted files

@@             Coverage Diff              @@
##             master    #3306      +/-   ##
============================================
- Coverage     30.17%   22.54%   -7.63%     
+ Complexity     3840     2312    -1528     
============================================
  Files           580      397     -183     
  Lines         48020    38201    -9819     
  Branches       6207     5437     -770     
============================================
- Hits          14488     8611    -5877     
+ Misses        32542    28863    -3679     
+ Partials        990      727     -263     

Flag	Coverage Δ
core	?
trino	22.54% <ø> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[zhoujinsong]

@shouwangyw Thanks for the contribution!
The PR looks good to me in general, I left a question, PTAL when you are free.

[zhoujinsong]

I am not sure what is the usage of this new file?

[shouwangyw]

This is for easy testing of HTTP interfaces in a local environment, and the 'HTTP Client' plugin can be installed in IDEA.