-
-
Notifications
You must be signed in to change notification settings - Fork 12
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Introduce Kafka topic init task #943
base: main
Are you sure you want to change the base?
Conversation
Extends the Compose setup, which previously solely relied on Redpanda, with Apache Kafka and Tansu. Defaults to Apache Kafka (native) in accordance with #854. While a Tansu setup is added, Tansu can not yet be used with Hyades. It runs into errors when the API server tries to produce to it. Extends e2e tests to support testing again different Kafka providers. Currently supported are Apache Kafka, Apache Kafka native, and Redpanda. Tansu will be added once it's functional. Removes the `create-topics.sh` script and the `redpanda-init` container in favour of the new kafka topics init task (introduced in DependencyTrack/hyades-apiserver#943). Signed-off-by: nscuro <nscuro@protonmail.com>
Coverage summary from CodacySee diff coverage on Codacy
Coverage variation details
Coverage variation is the difference between the coverage for the head and common ancestor commits of the pull request branch: Diff coverage details
Diff coverage is the percentage of lines that are covered by tests out of the coverable lines that the pull request added or modified: See your quality gate settings Change summary preferencesCodacy stopped sending the deprecated coverage status on June 5th, 2024. Learn more |
Extends the Compose setup, which previously solely relied on Redpanda, with Apache Kafka and Tansu. Defaults to Apache Kafka (native) in accordance with #854. While a Tansu setup is added, Tansu can not yet be used with Hyades. It runs into errors when the API server tries to produce to it. Extends e2e tests to support testing again different Kafka providers. Currently supported are Apache Kafka, Apache Kafka native, and Redpanda. Tansu will be added once it's functional. Removes the `create-topics.sh` script and the `redpanda-init` container in favour of the new kafka topics init task (introduced in DependencyTrack/hyades-apiserver#943). Signed-off-by: nscuro <nscuro@protonmail.com>
Adds a new init task that creates all required topics using their respective default configuration during startup. This replaces the need for a separate init container (i.e. `redpanda-init` in our `docker-compose.yml`: https://github.com/DependencyTrack/hyades/blob/008c3a1b024969a159de085d5ca6b464147faa9a/docker-compose.yml#L251-L273). The creation of topics is also faster because it can be done in a single batched request, vs. 10s of repetitive CLI invocations. This init task is disabled by default for now, since it's not meant to be used for production deployments. The idea is to build on this work, and allow users to customize the default topic configurations (e.g. via env vars or config files). Relates to DependencyTrack/hyades#1195 Signed-off-by: nscuro <nscuro@protonmail.com>
d65dde7
to
a8556fb
Compare
Extends the Compose setup, which previously solely relied on Redpanda, with Apache Kafka and Tansu. Defaults to Apache Kafka (native) in accordance with #854. While a Tansu setup is added, Tansu can not yet be used with Hyades. It runs into errors when the API server tries to produce to it. Extends e2e tests to support testing again different Kafka providers. Currently supported are Apache Kafka, Apache Kafka native, and Redpanda. Tansu will be added once it's functional. Removes the `create-topics.sh` script and the `redpanda-init` container in favour of the new kafka topics init task (introduced in DependencyTrack/hyades-apiserver#943). Signed-off-by: nscuro <nscuro@protonmail.com>
Extends the Compose setup, which previously solely relied on Redpanda, with Apache Kafka and Tansu. Defaults to Apache Kafka (native) in accordance with #854. While a Tansu setup is added, Tansu can not yet be used with Hyades. It runs into errors when the API server tries to produce to it. Extends e2e tests to support testing again different Kafka providers. Currently supported are Apache Kafka, Apache Kafka native, and Redpanda. Tansu will be added once it's functional. Removes the `create-topics.sh` script and the `redpanda-init` container in favour of the new kafka topics init task (introduced in DependencyTrack/hyades-apiserver#943). Signed-off-by: nscuro <nscuro@protonmail.com>
Description
Adds a new init task that creates all required topics using their respective default configuration during startup.
This replaces the need for a separate init container (i.e.
redpanda-init
in ourdocker-compose.yml
: https://github.com/DependencyTrack/hyades/blob/008c3a1b024969a159de085d5ca6b464147faa9a/docker-compose.yml#L251-L273).The creation of topics is also faster because it can be done in a single batched request, vs. 10s of repetitive CLI invocations.
This init task is disabled by default for now, since it's not meant to be used for production deployments. The idea is to build on this work, and allow users to customize the default topic configurations (e.g. via env vars or config files).
Addressed Issue
Relates to DependencyTrack/hyades#1195
Additional Details
N/A
Checklist
This PR fixes a defect, and I have provided tests to verify that the fix is effectiveThis PR introduces changes to the database model, and I have updated the migration changelog accordinglyThis PR introduces new or alters existing behavior, and I have updated the documentation accordingly