HIVE-29238:upgrade kafka version to fix CVE-2024-31141 and CVE-2021-3… #6110
Conversation
| <junit.jupiter.version>5.13.3</junit.jupiter.version> | ||
| <junit.vintage.version>5.13.3</junit.vintage.version> | ||
| <kafka.version>2.5.0</kafka.version> | ||
| <kafka.version>3.9.1</kafka.version> |
There was a problem hiding this comment.
If I recall well there were issues with previous upgrade attempts. Please check the (git) history and related PRs for more information to ensure that code remains functional.
There was a problem hiding this comment.
yes I am aware of that, its wip right now, I am planning to address those issues if I get a grren label in current state
asf-ci-hive
added
tests pending
tests unstable
and removed
tests failed
tests pending
tests unstable
labels
asf-ci-hive
added
tests pending
tests unstable
and removed
tests unstable
tests pending
labels
asf-ci-hive
added
tests failed
tests pending
tests passed
and removed
tests pending
tests failed
labels
ramitg254
changed the title
ramitg254
changed the title
|
earlier kafka upgrade was made for the sake of compatibility needed as mentioned in #4082 and this was reverted because some test cases were disabled to check upgrade properly and this was not an issue anymore due to refractor as mentioned in this revert ticket https://issues.apache.org/jira/browse/HIVE-27475 and after that a flaky test related to it was enabled int this ticket https://issues.apache.org/jira/browse/HIVE-27502 and it is also passing regularly locally after the upgrade. |
asf-ci-hive
added
tests pending
tests passed
and removed
tests passed
tests pending
labels
|
most changes proposed are in accordance to following changes in kafka dependency: https://github.com/apache/kafka/pull/12590/files |
zabetak
left a comment
zabetak
left a comment
There was a problem hiding this comment.
Before merging this PR we should ensure that kafka_storage_handler.q runs fine so addressing https://issues.apache.org/jira/browse/HIVE-23985 seems like a prerequisite for merging this PR. I will start a flaky check for kafka_storage_handler.q in master and if it passes I will reenable the test.
In the meantime, you can run the test locally and check if it passes with the changes in this PR.
with the current changes I am seeing errors while running this test so trying to solve those errors in the meantime |
|
@ramitg254 Please take a look at the PR for https://issues.apache.org/jira/browse/HIVE-27407. In the past I fixes some of these errors there. |
|
@zabetak Thanks for the info, some props changes were left , I have added those changes and enabled the kafka_storage_handler.q and results are getting generated so its fine now. |
asf-ci-hive
added
tests pending
tests failed
and removed
tests failed
tests pending
labels
|
3 participants
…8153
What changes were proposed in this pull request?
Why are the changes needed?
Does this PR introduce any user-facing change?
How was this patch tested?