Feature request: Reuse connector WITH parameters

[xxchan]

(requested by a user)

e.g., for PostgreSQL CDC, users need to repeatedly enter hostname, port, username, password, ... for all tables. It might be better if there are some ways to reuse the common parameters.

At the same time, are there any existing workarounds?

[fuyufjh]

For CDC sources, @StrikeW is working on RFC: Share a change log stream for multiple CDC tables (@StrikeW Please help migrate this doc to RFC repo), which introduces a new syntax to define multiple tables from one upstream DB.

For other things, we don't have a solution yet. Welcome for any discussions & proposals.

[tabVersion]

e.g., for PostgreSQL CDC, users need to repeatedly enter hostname, port, username, password, ... for all tables. It might be better if there are some ways to reuse the common parameters.

actually, I have proposed the connection syntax for this case, risingwavelabs/rfcs#35 but we only adopt the private link part. Maybe it is time to review the design.

[xxchan]

BTW, I also feel that this request is somewhat similar to #10151, where code reusability is wanted, instead of a new "capability".

One typical argument is that we can simply use global string substitution, or dbt macros.

[ly9chee]

I'm also interested in this feature, repeatedly enter hostname, port, username, password is not just verbose and it may cause sensitive information leaks. crdb provides the create external connection syntax that allows you to encapsulate network and authentication settings into a connection object. a user with higher privileges (such as root) can manage these connections by defining the necessary connection parameters. Once the connection object is created, normal users can reference it to create sources and sinks without needing to specify the network and authentication settings each time, also normal users cannot extract any sensitive information from the connection object without grants.

[github-actions]

This issue has been open for 60 days with no activity. Could you please update the status? Feel free to continue discussion or close as not planned.

[StrikeW]

We will introduce Connection to support this feature. close as stale.