[CELEBORN-1054] Support db based dynamic config service

[RexXiong]

What changes were proposed in this pull request?

Support database based store backend implementation for dynamic configuration management

Why are the changes needed?

Currently celeborn provides FsConfigServiceImpl implementation for dynamic config service which is based on file system, We cloud Support database based store backend implementation.

Does this PR introduce any user-facing change?

No

How was this patch tested?

• ConfigServiceSuiteJ#testDbConfig

[pan3793]

My suggestion may be a bit paranoid: can we avoid using the ORM framework and involve additional configuration files other than celeborn-defaults.conf? specific to this PR, I mean iBatis and Hikari

With a thin wrapper, we can write the vanilla JDBC API like

   implicit val hikariDataSource = new HikariDataSource(...)

   // query case
   JdbcUtils.executeQueryWithRowMapper(
      s"""SELECT item1, item2
         |FROM table
         |WHERE state=?
         |ORDER BY create_time ASC LIMIT 1
         |""".stripMargin) { stmt =>
      // set parameter
      stmt.setString(1, "balabala")
    } { resultSet =>
      // map the ResultSet to data struct, usually a POJO
      resultSet.getString(1)
    }

[codecov]

Codecov Report

Attention: 74 lines in your changes are missing coverage. Please review.

Comparison is base (a6682d5) 47.82% compared to head (6519de1) 48.55%.
Report is 5 commits behind head on main.

Files	Patch %	Lines
...rver/common/service/model/ClusterTenantConfig.java	48.89%	23 Missing ⚠️
.../common/service/store/db/DbServiceManagerImpl.java	77.20%	12 Missing and 1 partial ⚠️
...r/common/service/config/BaseConfigServiceImpl.java	59.26%	10 Missing and 1 partial ⚠️
...eleborn/server/common/service/utils/JsonUtils.java	0.00%	11 Missing ⚠️
...rver/common/service/model/ClusterSystemConfig.java	77.28%	5 Missing ⚠️
...eborn/server/common/service/model/ClusterInfo.java	78.95%	4 Missing ⚠️
...on/service/config/DynamicConfigServiceFactory.java	0.00%	3 Missing ⚠️
...rver/common/service/store/db/DBSessionFactory.java	94.74%	1 Missing and 1 partial ⚠️
...ver/common/service/config/DbConfigServiceImpl.java	92.86%	0 Missing and 1 partial ⚠️
...orn/server/common/service/config/TenantConfig.java	90.00%	1 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2273      +/-   ##
==========================================
+ Coverage   47.82%   48.55%   +0.73%     
==========================================
  Files         200      208       +8     
  Lines       12449    12762     +313     
  Branches     1088     1096       +8     
==========================================
+ Hits         5953     6195     +242     
- Misses       6103     6171      +68     
- Partials      393      396       +3     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[RexXiong]

My suggestion may be a bit paranoid: can we avoid using the ORM framework and involve additional configuration files other than celeborn-defaults.conf? specific to this PR, I mean iBatis and Hikari

With a thin wrapper, we can write the vanilla JDBC API like

   implicit val hikariDataSource = new HikariDataSource(...)

   // query case
   JdbcUtils.executeQueryWithRowMapper(
      s"""SELECT item1, item2
         |FROM table
         |WHERE state=?
         |ORDER BY create_time ASC LIMIT 1
         |""".stripMargin) { stmt =>
      // set parameter
      stmt.setString(1, "balabala")
    } { resultSet =>
      // map the ResultSet to data struct, usually a POJO
      resultSet.getString(1)
    }

After offline discussion, we currently tend to keep using the orm (ibatis) framework, and at the same time unify Database-related configurations to CelebornConf.

[pan3793]

Some minor issues, almost good

[SteNicholas]

LGTM.

[RexXiong]

Thanks all, merge to main(v0.5.0)