HDFS-17570 Respect Non-Default HADOOP_ROOT_LOGGER when HADOOP_DAEMON_ROOT_LOGGER is not specified in Daemon mode #6922

VicoWu · 2024-07-04T03:25:13Z

Description of PR

the -Dhadoop.root.logger will use the value of HADOOP_ROOT_LOGGER.
In most cases, hadoop administrator tends to setup HADOOP_ROOT_LOGGER directly, it is effective in non-daemon mode and most hadoop administrator are not even aware of the existence of HADOOP_DAEMON_ROOT_LOGGER.
So when he started up HADOOP as daemon mode, no matter how he set up HADOOP_ROOT_LOGGER, the -Dhadoop.root.logger is not changed because in Daemon mode, the HADOOP_ROOT_LOGGER is set up by HADOOP_DAEMON_ROOT_LOGGER which is in fact a default value. The code is as below:

  if [[ "${HADOOP_DAEMON_MODE}" != "default" ]]; then
    HADOOP_ROOT_LOGGER="${HADOOP_DAEMON_ROOT_LOGGER}"
    if [[ "${HADOOP_SUBCMD_SECURESERVICE}" = true ]]; then
      HADOOP_LOGFILE="hadoop-${HADOOP_SECURE_USER}-${HADOOP_IDENT_STRING}-${HADOOP_SUBCMD}-${HOSTNAME}.log"
    else
      HADOOP_LOGFILE="hadoop-${HADOOP_IDENT_STRING}-${HADOOP_SUBCMD}-${HOSTNAME}.log"
    fi
  fi

This makes log setup and troubleshooting becomes very time consuming before we finally figure out the reason and the best practice without reading the code.

My change is:
In daemon mode

If user has customize(changed) HADOOP_ROOT_LOGGER but no HADOOP_DAEMON_ROOT_LOGGER, then we will respect HADOOP_ROOT_LOGGER as the final -Dhadoop.root.logger value, instead of setting up HADOOP_ROOT_LOGGER with the default HADOOP_DAEMON_ROOT_LOGGER and print a warning
In other cases, we will use HADOOP_DAEMON_ROOT_LOGGER(no matter it is default or customized), we will setup the HADOOP_ROOT_LOGGER with this customized HADOOP_DAEMON_ROOT_LOGGER

How was this patch tested?

Testing Case	Expected Result
Customized `HADOOP_LOGLEVEL`	Respect `HADOOP_LOGLEVEL`
Without customizing anything	Respect default `HADOOP_DAEMON_ROOT_LOGGER` in daemon mode
Customize the `HADOOP_DAEMON_ROOT_LOGGER` without customizing `HADOOP_ROOT_LOGGER`	Respect customized `HADOOP_DAEMON_ROOT_LOGGER`
Customize the `HADOOP_ROOT_LOGGER` but not customize `HADOOP_DAEMON_ROOT_LOGGER`	Respect customized `HADOOP_ROOT_LOGGER` with a warning
Customize both `HADOOP_ROOT_LOGGER` and `HADOOP_DAEMON_ROOT_LOGGER`	Respect customized `HADOOP_DAEMON_ROOT_LOGGER`

Customize the HADOOP_LOGLEVEL, should respect the HADOOP_LOGLEVEL and HADOOP_DAEMON_ROOT_LOGGER

export HADOOP_LOGLEVEL=DEBUG

root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
hadoop   2852464       1 45 04:04 pts/1    00:00:09 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=DEBUG,RFA -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode

Without customizing anything, should respect default HADOOP_DAEMON_ROOT_LOGGER

root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
hadoop   2855220       1 43 04:05 pts/1    00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=INFO,RFA -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode

Customize the HADOOP_DAEMON_ROOT_LOGGER without customizing HADOOP_ROOT_LOGGER, should respect customized HADOOP_DAEMON_ROOT_LOGGER

export HADOOP_DAEMON_ROOT_LOGGER=TRACE,RFA

 root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
 hadoop   2857093       1 99 04:06 pts/1    00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=TRACE,RFA -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode
 root     2857504 2745542  0 04:07 pts/1    00:00:00 grep --color=auto NameNode

Customize the HADOOP_ROOT_LOGGER but not customize HADOOP_DAEMON_ROOT_LOGGER, should respect customized HADOOP_ROOT_LOGGER with a warning

export HADOOP_ROOT_LOGGER=TRACE,console

root@rccd101-6a:/home/hadoop/current-hadoop# sudo -u hadoop bin/hdfs --daemon stop namenode;
Using customized HADOOP_ROOT_LOGGER as final hadoop.root.logger. Recommending using HADOOP_LOGLEVEL or customize HADOOP_DAEMON_ROOT_LOGGER for logging in daemon mode.

root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
hadoop   2874729       1 99 04:16 pts/1    00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=TRACE,console -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode

Customize both HADOOP_ROOT_LOGGER and HADOOP_DAEMON_ROOT_LOGGER, should respect customized HADOOP_DAEMON_ROOT_LOGGER

export HADOOP_DAEMON_ROOT_LOGGER=TRACE,RFA
export HADOOP_ROOT_LOGGER=TRACE,console

root@rccd101-6a:/home/hadoop/current-hadoop# ps -ef|grep NameNode
hadoop   2877521       1 61 04:18 pts/1    00:00:07 /usr/lib/jvm/java-11-openjdk-amd64/bin/java -Dproc_namenode -Djava.net.preferIPv4Stack=true -Dhdfs.audit.logger=INFO,NullAppender -Dlog4j.configuration=file:/home/hadoop/current-hadoop/etc/hadoop/log4j-namenode.properties -XX:+UseG1GC -Xmx4096m -Dcom.sun.management.jmxremote=true -Dcom.sun.management.jmxremote.authenticate=false -Dcom.sun.management.jmxremote.ssl=false -Dcom.sun.management.jmxremote.port=1029 -Dhadoop.security.logger=ERROR,RFAS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=*:5005 -Dyarn.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dyarn.log.file=test-host-name.dev.corp.com.log -Dyarn.home.dir=/home/hadoop/hadoop-3.2.4 -Dyarn.root.logger=INFO,console -Djava.library.path=/home/hadoop/hadoop-3.2.4/lib/native -Dhadoop.log.dir=/home/hadoop/hadoop-3.2.4/../logs -Dhadoop.log.file=test-host-name.dev.corp.com.log -Dhadoop.home.dir=/home/hadoop/hadoop-3.2.4 -Dhadoop.id.str=hadoop -Dhadoop.root.logger=TRACE,RFA -Dhadoop.policy.file=hadoop-policy.xml org.apache.hadoop.hdfs.server.namenode.NameNode

For code changes:

Does the title or this PR starts with the corresponding JIRA issue id (e.g. 'HADOOP-17799. Your PR title ...')?
Object storage: have the integration tests been executed and the endpoint declared according to the connector-specific documentation?
If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under ASF 2.0?
If applicable, have you updated the LICENSE, LICENSE-binary, NOTICE-binary files?

hadoop-yetus · 2024-07-04T04:35:24Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	10m 8s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 43s		trunk passed
+1 💚	mvnsite	0m 48s		trunk passed
+1 💚	shadedclient	19m 42s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 41s		the patch passed
+1 💚	shellcheck	0m 2s		No new issues.
-1 ❌	shadedclient	2m 13s		patch has errors when building and testing our client artifacts.
			_ Other Tests _
-1 ❌	unit	0m 55s	/patch-unit-hadoop-common-project_hadoop-common.txt	hadoop-common in the patch failed.
+1 💚	asflicense	0m 23s		The patch does not generate ASF License warnings.
		69m 16s

Reason	Tests
Failed TAP tests	hadoop_basic_init.bats.tap

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/1/artifact/out/Dockerfile
GITHUB PR	#6922
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname	Linux 72b4515ca60b 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `e65cb38`
TAP logs	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/1/artifact/out/patch-hadoop-common-project_hadoop-common.tap
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/1/testReport/
Max. process+thread count	552 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/1/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-07-04T14:04:53Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	35m 33s		trunk passed
+1 💚	mvnsite	0m 53s		trunk passed
+1 💚	shadedclient	23m 12s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 34s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 46s		the patch passed
+1 💚	shellcheck	0m 2s		No new issues.
+1 💚	shadedclient	23m 19s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	1m 2s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 24s		The patch does not generate ASF License warnings.
		87m 23s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/2/artifact/out/Dockerfile
GITHUB PR	#6922
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname	Linux fd88dc2efd10 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `13dcc40`
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/2/testReport/
Max. process+thread count	557 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/2/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-07-06T05:45:26Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 18s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	32m 34s		trunk passed
+1 💚	mvnsite	0m 47s		trunk passed
+1 💚	shadedclient	19m 44s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 44s		the patch passed
+1 💚	shellcheck	0m 2s		No new issues.
+1 💚	shadedclient	19m 29s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 59s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 26s		The patch does not generate ASF License warnings.
		76m 39s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/3/artifact/out/Dockerfile
GITHUB PR	#6922
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname	Linux fbf8e0079eba 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `6a82029`
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/3/testReport/
Max. process+thread count	552 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/3/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

VicoWu · 2024-07-22T02:42:10Z

@aw-was-here @aajisaka Would you please take a look for this?

hadoop-yetus · 2024-07-22T03:54:52Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	4m 1s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 47s		trunk passed
+1 💚	mvnsite	0m 47s		trunk passed
+1 💚	shadedclient	21m 43s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 30s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 41s		the patch passed
+1 💚	shellcheck	0m 2s		No new issues.
+1 💚	shadedclient	22m 14s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 55s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 22s		The patch does not generate ASF License warnings.
		89m 27s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/4/artifact/out/Dockerfile
GITHUB PR	#6922
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname	Linux 5da9cb7bc23f 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `83659c7`
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/4/testReport/
Max. process+thread count	552 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/4/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-07-25T14:07:30Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 21s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+0 🆗	mvndep	13m 43s		Maven dependency ordering for branch
+1 💚	mvninstall	23m 9s		trunk passed
+1 💚	compile	10m 46s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	compile	10m 9s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	checkstyle	2m 13s		trunk passed
+1 💚	mvnsite	1m 56s		trunk passed
+1 💚	javadoc	1m 31s		trunk passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 56s		trunk passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	3m 40s		trunk passed
+1 💚	shadedclient	28m 17s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+0 🆗	mvndep	0m 21s		Maven dependency ordering for patch
+1 💚	mvninstall	1m 19s		the patch passed
+1 💚	compile	10m 50s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javac	10m 50s		the patch passed
+1 💚	compile	9m 55s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	javac	9m 55s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	checkstyle	2m 19s		the patch passed
+1 💚	mvnsite	1m 45s		the patch passed
+1 💚	shellcheck	0m 2s		No new issues.
+1 💚	javadoc	1m 19s		the patch passed with JDK Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2
+1 💚	javadoc	1m 40s		the patch passed with JDK Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
+1 💚	spotbugs	3m 47s		the patch passed
+1 💚	shadedclient	28m 51s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	16m 20s		hadoop-common in the patch passed.
-1 ❌	unit	207m 16s	/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt	hadoop-hdfs in the patch passed.
+1 💚	asflicense	0m 47s		The patch does not generate ASF License warnings.
		383m 43s

Reason	Tests
Failed junit tests	hadoop.hdfs.server.namenode.TestAddOverReplicatedStripedBlocks
	hadoop.hdfs.server.namenode.TestDefaultBlockPlacementPolicy
	hadoop.hdfs.server.namenode.TestReencryption
	hadoop.hdfs.server.blockmanagement.TestBlocksWithNotEnoughRacks
	hadoop.hdfs.server.balancer.TestBalancer
	hadoop.hdfs.server.balancer.TestBalancerWithMultipleNameNodes
	hadoop.hdfs.TestReconstructStripedFile
	hadoop.hdfs.TestErasureCodingMultipleRacks

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/5/artifact/out/Dockerfile
GITHUB PR	#6922
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs compile javac javadoc mvninstall shadedclient spotbugs checkstyle
uname	Linux 4dbaabf14a14 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `cf8ead0`
Default Java	Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Multi-JDK versions	/usr/lib/jvm/java-11-openjdk-amd64:Ubuntu-11.0.23+9-post-Ubuntu-1ubuntu120.04.2 /usr/lib/jvm/java-8-openjdk-amd64:Private Build-1.8.0_412-8u412-ga-1~20.04.1-b08
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/5/testReport/
Max. process+thread count	4188 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common hadoop-hdfs-project/hadoop-hdfs U: .
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/5/console
versions	git=2.25.1 maven=3.6.3 spotbugs=4.2.2 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

hadoop-yetus · 2024-07-26T05:41:38Z

💔 -1 overall

Vote	Subsystem	Runtime	Logfile	Comment
+0 🆗	reexec	0m 19s		Docker mode activated.
			_ Prechecks _
+1 💚	dupname	0m 0s		No case conflicting files found.
+0 🆗	codespell	0m 0s		codespell was not available.
+0 🆗	detsecrets	0m 0s		detect-secrets was not available.
+0 🆗	shelldocs	0m 0s		Shelldocs was not available.
+1 💚	@author	0m 0s		The patch does not contain any @author tags.
-1 ❌	test4tests	0m 0s		The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
			_ trunk Compile Tests _
+1 💚	mvninstall	36m 30s		trunk passed
+1 💚	mvnsite	0m 51s		trunk passed
+1 💚	shadedclient	24m 51s		branch has no errors when building and testing our client artifacts.
			_ Patch Compile Tests _
+1 💚	mvninstall	0m 29s		the patch passed
+1 💚	blanks	0m 0s		The patch has no blanks issues.
+1 💚	mvnsite	0m 40s		the patch passed
+1 💚	shellcheck	0m 2s		No new issues.
+1 💚	shadedclient	25m 0s		patch has no errors when building and testing our client artifacts.
			_ Other Tests _
+1 💚	unit	0m 56s		hadoop-common in the patch passed.
+1 💚	asflicense	0m 20s		The patch does not generate ASF License warnings.
		91m 26s

Subsystem	Report/Notes
Docker	ClientAPI=1.46 ServerAPI=1.46 base: https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/6/artifact/out/Dockerfile
GITHUB PR	#6922
Optional Tests	dupname asflicense mvnsite unit codespell detsecrets shellcheck shelldocs
uname	Linux 5068e7a51e84 5.15.0-106-generic #116-Ubuntu SMP Wed Apr 17 09:17:56 UTC 2024 x86_64 x86_64 x86_64 GNU/Linux
Build tool	maven
Personality	dev-support/bin/hadoop.sh
git revision	trunk / `0535f2d`
Test Results	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/6/testReport/
Max. process+thread count	551 (vs. ulimit of 5500)
modules	C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
Console output	https://ci-hadoop.apache.org/job/hadoop-multibranch/job/PR-6922/6/console
versions	git=2.25.1 maven=3.6.3 shellcheck=0.7.0
Powered by	Apache Yetus 0.14.0 https://yetus.apache.org

This message was automatically generated.

github-actions bot added trunk Common labels Jul 4, 2024

VicoWu marked this pull request as draft July 4, 2024 03:25

VicoWu marked this pull request as ready for review July 4, 2024 07:51

VicoWu marked this pull request as draft July 4, 2024 07:51

VicoWu changed the title ~~refine HADOOP_ROOT_LOGGER and HADOOP_DAEMON_ROOT_LOGGER logic~~ HDFS-17570 Respect Non-Default HADOOP_ROOT_LOGGER when HADOOP_DAEMON_ROOT_LOGGER is not specified in Daemon mode Jul 4, 2024

VicoWu marked this pull request as ready for review July 22, 2024 02:23

VicoWu force-pushed the refine-daemon-root-log-level branch from 6a82029 to 83659c7 Compare July 22, 2024 02:24

github-actions bot added the HDFS label Jul 25, 2024

VicoWu added 3 commits July 26, 2024 12:08

refine HADOOP_ROOT_LOGGER and HADOOP_DAEMON_ROOT_LOGGER logic

2702d3a

fix shell bugs

96f02de

refine code

0535f2d

VicoWu force-pushed the refine-daemon-root-log-level branch from cf8ead0 to 0535f2d Compare July 26, 2024 04:09

github-actions bot removed the HDFS label Jul 26, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

HDFS-17570 Respect Non-Default HADOOP_ROOT_LOGGER when HADOOP_DAEMON_ROOT_LOGGER is not specified in Daemon mode #6922

HDFS-17570 Respect Non-Default HADOOP_ROOT_LOGGER when HADOOP_DAEMON_ROOT_LOGGER is not specified in Daemon mode #6922

VicoWu commented Jul 4, 2024 •

edited

Loading

hadoop-yetus commented Jul 4, 2024

hadoop-yetus commented Jul 4, 2024

hadoop-yetus commented Jul 6, 2024

VicoWu commented Jul 22, 2024

hadoop-yetus commented Jul 22, 2024

hadoop-yetus commented Jul 25, 2024

hadoop-yetus commented Jul 26, 2024

HDFS-17570 Respect Non-Default HADOOP_ROOT_LOGGER when HADOOP_DAEMON_ROOT_LOGGER is not specified in Daemon mode #6922

Are you sure you want to change the base?

HDFS-17570 Respect Non-Default HADOOP_ROOT_LOGGER when HADOOP_DAEMON_ROOT_LOGGER is not specified in Daemon mode #6922

Conversation

VicoWu commented Jul 4, 2024 • edited Loading

Description of PR

How was this patch tested?

For code changes:

hadoop-yetus commented Jul 4, 2024

hadoop-yetus commented Jul 4, 2024

hadoop-yetus commented Jul 6, 2024

VicoWu commented Jul 22, 2024

hadoop-yetus commented Jul 22, 2024

hadoop-yetus commented Jul 25, 2024

hadoop-yetus commented Jul 26, 2024

VicoWu commented Jul 4, 2024 •

edited

Loading