HIVE-29258: Remove Class.forName() for driver loading #6207

Aggarwal-Raghav · 2025-11-21T15:10:24Z

What changes were proposed in this pull request?

Why are the changes needed?

This PR removes explicit Class.forName calls, leveraging the JDBC 4.0+ Service Provider Interface (SPI) for automatic driver discovery. The DriverManager now utilizes ServiceLoader to locate drivers defined in META-INF/services and registers them dynamically.

Does this PR introduce any user-facing change?

NO

How was this patch tested?

Ran the affected files UT locally and will see CI outcome.

okumin

We can amend some points, following SonarQube.

I wonder if we can remove some more. Probably, the answer is no.

% git grep 'Class.forName' | grep Driver
beeline/src/java/org/apache/hive/beeline/BeeLine.java:        Driver driver = (Driver) Class.forName(clazzName, true,
beeline/src/java/org/apache/hive/beeline/Commands.java:        Driver driver = (Driver) Class.forName(name).newInstance();
beeline/src/java/org/apache/hive/beeline/DatabaseConnection.java:            (Driver) Class.forName(clazzName, true, Thread.currentThread().getContextClassLoader())
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("com.mysql.jdbc.Driver"));
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("org.postgresql.Driver"));
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("oracle.jdbc.OracleDriver"));
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver"));
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("com.ibm.db2.jcc.DB2Driver"));
ql/src/java/org/apache/hadoop/hive/ql/DriverFactory.java:          Class<?> cls = Class.forName(name);
ql/src/test/org/apache/hadoop/hive/ql/TestDriverFactory.java:          Class<?> cls = Class.forName(name);

If we can reduce the usage, is it possible to remove CONNECTION_DRIVER in HiveConf and MetastoreConf?

okumin · 2025-11-22T01:20:56Z

itests/hive-minikdc/src/test/java/org/apache/hive/minikdc/TestHS2AuthMechsWithMiniKdc.java

  public void setUpBefore() throws Exception {
    if (miniHS2 == null) {
-      Class.forName(MiniHS2.getJdbcDriverName());
+


Suggested change

An extra line has been added to some files and not been added to some files. I prefer to have no extra lines consistently

okumin · 2025-11-22T01:26:00Z

...ore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/jdbc/NoPoolConnectionPool.java

      }
-      try {
-        LOG.info("Going to load JDBC driver {}", driverName);
-        driver = (Driver) Class.forName(driverName).newInstance();


This might change the behavior of L67 because this.driver is never assigned now

Removed driver , connString as class attributes. Made connString local to method getConnection()

…ommands.java to leverage ServiceLoader

Aggarwal-Raghav · 2025-11-22T09:03:34Z

We can amend some points, following SonarQube.

I wonder if we can remove some more. Probably, the answer is no.

% git grep 'Class.forName' | grep Driver
beeline/src/java/org/apache/hive/beeline/BeeLine.java:        Driver driver = (Driver) Class.forName(clazzName, true,
beeline/src/java/org/apache/hive/beeline/Commands.java:        Driver driver = (Driver) Class.forName(name).newInstance();
beeline/src/java/org/apache/hive/beeline/DatabaseConnection.java:            (Driver) Class.forName(clazzName, true, Thread.currentThread().getContextClassLoader())
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("com.mysql.jdbc.Driver"));
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("org.postgresql.Driver"));
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("oracle.jdbc.OracleDriver"));
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("com.microsoft.sqlserver.jdbc.SQLServerDriver"));
jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/JdbcStorageHandler.java:      classesToLoad.add(Class.forName("com.ibm.db2.jcc.DB2Driver"));
ql/src/java/org/apache/hadoop/hive/ql/DriverFactory.java:          Class<?> cls = Class.forName(name);
ql/src/test/org/apache/hadoop/hive/ql/TestDriverFactory.java:          Class<?> cls = Class.forName(name);

If we can reduce the usage, is it possible to remove CONNECTION_DRIVER in HiveConf and MetastoreConf?

Have removed reflection code changes from BeeLine.java, Commands.java, DatabaseConnection.java. Please review it. Changes in JdbcStorageHandler.java, DriverFactory.java are not specific to DriverManager and are outside the scope of PR IMO.

okumin

I'm feeling now MetastoreConf.ConfVars.CONNECTION_DRIVER is used only for logging purposes, and keeping it might cause a confusion; what if CONNECTION_DRIVER is for Postgres but the connection URL is for MySQL?
I wonder if we can remote it and HiveConf.ConfVars.METASTORE_CONNECTION_DRIVER in this pull request, or if it's impossible, or if we should separatre a pull request

okumin · 2025-11-24T09:07:50Z

beeline/src/java/org/apache/hive/beeline/DatabaseConnection.java


-  public DatabaseConnection(BeeLine beeLine, String driver, String url,
-       Properties info) throws SQLException {
+  public DatabaseConnection(BeeLine beeLine, String url, Properties info) throws SQLException {


Please follow these ones
https://sonarcloud.io/project/issues?id=apache_hive&pullRequest=6207&issueStatuses=OPEN,CONFIRMED&sinceLeakPeriod=true

Aggarwal-Raghav · 2025-11-25T18:00:57Z

I'm feeling now MetastoreConf.ConfVars.CONNECTION_DRIVER is used only for logging purposes, and keeping it might cause a confusion; what if CONNECTION_DRIVER is for Postgres but the connection URL is for MySQL? I wonder if we can remote it and HiveConf.ConfVars.METASTORE_CONNECTION_DRIVER in this pull request, or if it's impossible, or if we should separatre a pull request

@okumin , thanks for the thorough review. What you mentioned is correct but I'm not sure, it will be that easy to remove CONNECTION_DRIVER config from HiveConf

HiveConf => Used, despite being Deprecated

hive/jdbc-handler/src/main/java/org/apache/hive/storage/jdbc/dao/DatabaseAccessorFactory.java

Line 73 in 526bd87

if (driverClass.contains("MYSQL")) {

MetaStoreConf=> Used in QTests (possible + achievable but requires cleanup from lot of places which is outside scope of this PR)

hive/standalone-metastore/metastore-server/src/test/java/org/apache/hadoop/hive/metastore/dbinstall/rules/PostgresTPCDS.java

Line 66 in 594e570

"-driver", getJdbcDriver()),

I'm hoping we can target this in separate JIRA. I have filed JIRA HIVE-29289 sometime back . Maybe as part of that cleanup as a follow-up I can try this.
Also, the recent sonarcloud was there before my changes as well and I was hoping it can be ignored if no changes in this PR are required. Running CI again doesn't seems ok to me. Let me know your thoughts on this part.

mdayakar · 2025-11-25T14:22:49Z

beeline/src/java/org/apache/hive/beeline/Commands.java

        } else {
          beeLine.output(beeLine.getColorBuffer().red(msg.getMono()));
        }
      } catch (Throwable t) {


is the catch (Throwable t) still required?

I have mixed opinion on this.
As jdbcCompliant(), getMajorVersion(), getMinorVersion() don't have checked exception but there is a possibility that a faulty/3rd party driver in classpath can through Exception at runtime. Hence, I was inclined to keep it. Let me know if you think otherwise.

I don't think jdbcCompliant(), getMajorVersion(), getMinorVersion() these APIs throw any RuntimeExceptions from driver. Anyway its not a major issue, you can take a call.

mdayakar · 2025-11-25T14:26:52Z

beeline/src/java/org/apache/hive/beeline/DatabaseConnection.java


-  public DatabaseConnection(BeeLine beeLine, String driver, String url,
-       Properties info) throws SQLException {
+  public DatabaseConnection(BeeLine beeLine, String url, Properties info) throws SQLException {


From the DatabaseConnection constructor code it won't throw any exception so throws SQLException can be removed.

This is what sonaType also flagged. will fix this.

mdayakar · 2025-11-26T08:12:35Z

...ore-server/src/main/java/org/apache/hadoop/hive/metastore/txn/jdbc/NoPoolConnectionPool.java

-            e);
-      }
-      connString = MetastoreConf.getVar(conf, MetastoreConf.ConfVars.CONNECT_URL_KEY);
+    String driverName = MetastoreConf.getVar(conf, MetastoreConf.ConfVars.CONNECTION_DRIVER);


Here we are reading the drivername from config and not using it. If not configured throwing exception for the config which we are not using, I feel this is not correct.

Sure. will remove driverName from here.

mdayakar · 2025-11-26T08:29:34Z

hplsql/src/main/java/org/apache/hive/hplsql/Conn.java

    if (connStr != null) {
      String[] c = connStr.split(";");
-      if (c.length >= 1) {
-        driver = c[0];


if driver name is not expected in the connStr then connStr.split(";"); need to be adjusted accordingly.

connStr is coming a method argument and I'm not changing the logic of how the connStr is passed by caller i.e.
connStr = driver_class_name;jdbc_url;user;password will remain same. So, there is no need to update connStr.split(";")

sonarqubecloud · 2025-11-26T18:34:17Z

Quality Gate passed

Issues
1 New issue
0 Accepted issues

Measures
0 Security Hotspots
0.0% Coverage on New Code
0.0% Duplication on New Code

See analysis details on SonarQube Cloud

okumin

+1

HIVE-29258: Remove Class.forName() for driver loading

fa36e5e

asf-ci-hive added the tests pending label Nov 21, 2025

Addressing SonaType

58edbdf

asf-ci-hive added tests passed and removed tests pending labels Nov 21, 2025

Aggarwal-Raghav marked this pull request as ready for review November 21, 2025 17:48

asf-ci-hive added tests pending tests passed and removed tests passed tests pending labels Nov 21, 2025

okumin reviewed Nov 22, 2025

View reviewed changes

Aggarwal-Raghav added 2 commits November 22, 2025 12:50

Addressing review comments

a3c2fcf

Remove Reflection code in BeeLine.java, DatabaseConnection.java and C…

dc72e29

…ommands.java to leverage ServiceLoader

asf-ci-hive added tests pending and removed tests passed labels Nov 22, 2025

asf-ci-hive added tests passed and removed tests pending labels Nov 22, 2025

okumin reviewed Nov 24, 2025

View reviewed changes

mdayakar reviewed Nov 26, 2025

View reviewed changes

Review comments

40356d2

asf-ci-hive added tests pending and removed tests passed labels Nov 26, 2025

asf-ci-hive added tests passed and removed tests pending labels Nov 26, 2025

okumin approved these changes Nov 29, 2025

View reviewed changes

HIVE-29258: Remove Class.forName() for driver loading #6207

Are you sure you want to change the base?

HIVE-29258: Remove Class.forName() for driver loading #6207

Conversation

Aggarwal-Raghav commented Nov 21, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

What changes were proposed in this pull request?

Why are the changes needed?

Does this PR introduce any user-facing change?

How was this patch tested?

Uh oh!

okumin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Aggarwal-Raghav commented Nov 22, 2025

Uh oh!

okumin left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Aggarwal-Raghav commented Nov 25, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

sonarqubecloud bot commented Nov 26, 2025

Quality Gate passed

Uh oh!

okumin left a comment

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

Aggarwal-Raghav commented Nov 21, 2025 •

edited

Loading