Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Improvement] Support env.sh and add HADOOP_CONF_DIR and HIVE_CONF_DIR to CLASSPATH #3222

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

baiyangtx
Copy link
Contributor

Why are the changes needed?

HiveConf can load the existing hive-site.xml from the classpath during initialization.

So Amoro should support adding some directories to the classpath at startup.

In addition, user-defined environment variable configuration files need to be supported.

Brief change log

  • Add a env.sh in ${AMORO_CONF_DIR}
  • If exists env variables HADOOP_CONF_DIR and HIVE_CONF_DIR, add them to CLASSPATH
  • Support add additional directories by env variable AMORO_ADDITION_CLASSPATH.
  • Print all env variables and their value at startup.

How was this patch tested?

  • Add some test cases that check the changes thoroughly including negative and positive cases if possible

  • Add screenshots for manual tests if appropriate

  • Run test locally before making a pull request

Documentation

  • Does this pull request introduce a new feature? (no)
  • If yes, how is the feature documented? (not documented)

@baiyangtx baiyangtx requested review from zhoujinsong and czy006 and removed request for zhoujinsong September 24, 2024 06:31
@huyuanfeng2018
Copy link
Contributor

If we can export hadoop's classpath, do we still need to package the hadoop client into amoro?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants