About config the default parameters of the table

[zhongqishang]

I want to start a discussion about the config the default parameters of the table, as discussed earlier #1608 .

Now if we want to change some default values we have to hard code them.

I think of two ways :

1. Via configuration files like config.yaml/flink-conf.yaml
2. As JinMat said, We could do it in Settings module with visible editing

The difference between the two is that 1) can modify the configuration before the first startup and it is valid, and 2) needs to start AMS before it can be modified.

About this what do you have any ideas.

[shidayang]

I think the Setting page configuration would be more convenient.

[zhoujinsong]

AFAIK， We can configure the default property values for tables in a catalog with catalog properties, and similarly, the engine can override these defaults by the engine catalog properties. reference: https://arctic.netease.com/ch/configurations/#_1

Is this discussion about configuring the default values for all tables under all catalogs at the system level? I am wondering if configuring default values at the catalog level is sufficient to meet the requirements, or whether system-level default values are always necessary?

[zhongqishang]

AFAIK， We can configure the default property values for tables in a catalog with catalog properties, and similarly, the engine can override these defaults by the engine catalog properties. reference: https://arctic.netease.com/ch/configurations/#_1

Is this discussion about configuring the default values for all tables under all catalogs at the system level? I am wondering if configuring default values at the catalog level is sufficient to meet the requirements

Yes, We can configure the default property values for tables in a catalog with catalog properties. As i know, Catalog only provides default values when the table is created for the first time. The scenario I am targeting is mainly the modification of global attributes. For example, if I want to modify the configuration self-optimizing.minor.trigger.file-count, do I need to modify the property of all tables or only modify catalog properties ?

or whether system-level default values are always necessary?

I think this is very necessary. For me, I don't need to open up too many things to the my user to reduce their burden.

[zhoujinsong]

As i know, Catalog only provides default values when the table is created for the first time

In the latest implementation, catalog properties are not used to generate properties for new tables when creating tables. Instead, catalog properties and table properties are dynamically merged each time a table is loaded. This means that even for tables created in the past, catalog properties can override the default values on them.

reference:https://github.com/NetEase/arctic/blob/master/core/src/main/java/com/netease/arctic/table/BasicUnkeyedTable.java#L155

The scenario I am targeting is mainly the modification of global attributes. For example, if I want to modify the configuration self-optimizing.minor.trigger.file-count, do I need to modify the property of all tables or only modify catalog properties ?

As I mentioned above, in this case, only updating the catalog properties is necessary.

[zhongqishang]

Thanks for the detailed instructions, I think this works.