support regex replacement for sink table naming #1241
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Problem
Currently this jdbc connect can only sink data to a table with fixed name or use the whole topic name as part of the table name, but in many cases, people may need a more flexible way to named their target tables.
For example, while using debezium source connect, the kafka topic would be named in a format of
<server-id>.<source-database>.<source-table>
, which is not suitable to be use as part of the sink table name.Solution
One of the most flexible way for user to specify a table naming rule is regular expression replacement, which means take the kafka topic name and modify it via a regular expression before put it into the final sink table name.
In this PR, an expression of
${topic/Pattern/Replacement/}
is supported for advanced sink table naming. This expression format is inspired bysed
andbash
.Does this solution apply anywhere else?
If yes, where?
bash
support a very similar variable value replacement format as${var/Pattern/Replacement}
, andsed
support a more regular expression friendly replacement format as/Pattern/Replacement/
, while the char/
can be substitute by any other symbol such as#
.Test Strategy
Unit test case is included in this PR.
Testing done:
Release Plan