-
Notifications
You must be signed in to change notification settings - Fork 317
[add][plugin][icebergwriter] add icebergwriter #1208
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
wgzhao
merged 50 commits into
wgzhao:master
from
awol2005ex:awol2005ex-br-icebergwriter-20250220
May 16, 2025
Merged
Changes from all commits
Commits
Show all changes
50 commits
Select commit
Hold shift + click to select a range
a1b83cd
set hive jdbc connection loginTimeout ,fixed the hive connection time…
awol2005ex 69f806e
rdbmsreader jdbc_url is string not list
awol2005ex 16dcda5
S3writer: add pathStyleAccessEnabled config for fixed error 400 when …
awol2005ex fe0b7c8
Merge remote-tracking branch 'github/master' into awol2005x-my
awol2005ex 796703c
[bugfix][plugin][s3writer]S3writer:don't submit upload when no record…
awol2005ex 6d329da
Merge remote-tracking branch 'github/master' into awol2005x-my
awol2005ex 663d9ae
S3 textwriter
awol2005ex a12a4f1
S3 orcwriter
awol2005ex 357ab19
S3 parquetwriter
awol2005ex 8667e8f
DataBaseType should not be call set method,so that the driver is erro…
awol2005ex d150110
add other dbs drivers
awol2005ex 218a738
Merge branch 'master' into awol2005x-my
awol2005ex b697d54
[feature][plugin][paimonwriter] add paimon format file writer
awol2005ex d5d53f3
fixed timestamp,decimal,null type value bug
awol2005ex c9c14b9
Merge branch 'master' into awol2005x-my
awol2005ex e74ef7b
1.change author
awol2005ex 10a7ac8
Merge branch 'master' into awol2005x-my
awol2005ex 992d7a3
[add] add paimonwriter.json to docs
awol2005ex 9fe9180
Merge branch 'master' into awol2005x-my
awol2005ex c0dedee
fixed: timestamp time zone error
awol2005ex 4089d91
Merge branch 'master' into awol2005x-my
awol2005ex 561c5a5
Merge branch 'master' into awol2005x-my
awol2005ex 1665649
Merge branch 'master' into awol2005x-my
awol2005ex c017309
merge with master
awol2005ex 9f738d1
Merge remote-tracking branch 'github/master' into awol2005x-my
awol2005ex ecc045b
[add][plugin][icebergwriter] Add support for writing files in Iceberg…
awol2005ex 8ac19e4
[add][plugin][icebergwriter] fixed date type insert error
awol2005ex 1444931
Merge remote-tracking branch 'github/master' into awol2005x-my
awol2005ex b8d8aa0
[add][plugin][icebergwriter] package.xml add fileSet
awol2005ex 0accdef
[add][plugin][icebergwriter] add docs
awol2005ex b21da1b
[add][plugin][icebergwriter] add build scripts to docs
awol2005ex c1b876b
[add][plugin][icebergwriter] docs fixed
awol2005ex dd744a7
[add][plugin][icebergwriter] docs
awol2005ex ab8bd11
[add][plugin][icebergwriter] docs name error
awol2005ex a9b2fe2
[add][plugin][icebergwriter] add icebergwriter code
awol2005ex bab8a0c
[add][plugin][icebergwriter] change to streamreader at example
awol2005ex 6583174
[add][plugin][icebergwriter] fixed bug when partition iceberg table
awol2005ex 34f7e6d
[add][plugin][icebergwriter] fixed bug when partition iceberg table
awol2005ex b1c96a8
Merge remote-tracking branch 'origin/awol2005ex-br-icebergwriter-2025…
awol2005ex d598f14
Merge remote-tracking branch 'github/master' into awol2005x-my
awol2005ex 323cf31
merge master
awol2005ex 4c8cd9c
merge master
awol2005ex 5ae32be
Merge remote-tracking branch 'github/master' into awol2005x-my
awol2005ex 5a6625d
Merge remote-tracking branch 'github/master' into awol2005x-my
awol2005ex 24ea3ce
Merge remote-tracking branch 'github/master' into awol2005x-my
awol2005ex 0193c92
Merge branch 'master' into awol2005x-my
awol2005ex a83e9ec
merge master
awol2005ex 408fa57
[bugfix][writer][icebergwriter] fix dependency errors
awol2005ex 104dfae
[bugfix][writer][icebergwriter] remove profile for the jdk is upgrade…
awol2005ex 15b1b18
[bugfix][writer][icebergwriter] merge master
awol2005ex File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,54 @@ | ||
{ | ||
"job": { | ||
"setting": { | ||
"speed": { | ||
"channel": 3 | ||
}, | ||
"errorLimit": { | ||
"record": 0, | ||
"percentage": 0 | ||
} | ||
}, | ||
"content": [ | ||
{ | ||
"reader": { | ||
"name": "streamreader", | ||
"parameter": { | ||
"column": [ | ||
{ | ||
"value": "1", | ||
"type": "long" | ||
}, | ||
{ | ||
"value": "1989-06-04 00:00:00", | ||
"type": "timestamp" | ||
}, | ||
{ | ||
"value": "test1", | ||
"type": "string" | ||
} | ||
], | ||
"sliceRecordCount": 1000 | ||
} | ||
}, | ||
"writer": { | ||
"name": "icebergwriter", | ||
"parameter": { | ||
"tableName": "test.test1", | ||
"writeMode": "truncate", | ||
"catalogType": "hadoop", | ||
"warehouse": "s3a://pvc-91d1e2cd-4d25-45c9-8613-6c4f7bf0a4cc/iceberg", | ||
"hadoopConfig": { | ||
"fs.s3a.endpoint": "http://localhost:9000", | ||
"fs.s3a.access.key": "gy0dX5lALP176g6c9fYf", | ||
"fs.s3a.secret.key": "ReuUrCzzu5wKWAegtswoHIWV389BYl9AB1ZQbiKr", | ||
"fs.s3a.connection.ssl.enabled": "false", | ||
"fs.s3a.path.style.access": "true", | ||
"fs.s3a.impl": "org.apache.hadoop.fs.s3a.S3AFileSystem" | ||
} | ||
} | ||
} | ||
} | ||
] | ||
} | ||
} |
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.