Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BatchCompilerTest.test248_jar_ref_in_jar fails on Windows #2766

Closed
Tracked by #2761
iloveeclipse opened this issue Jul 26, 2024 · 6 comments
Closed
Tracked by #2761

BatchCompilerTest.test248_jar_ref_in_jar fails on Windows #2766

iloveeclipse opened this issue Jul 26, 2024 · 6 comments
Labels
bug Something isn't working regression Something was broken by a previous change test Work on unit tests, no change of productive code Windows Happens on Windows OS

Comments

@iloveeclipse
Copy link
Member

After merge of #2551 we see that BatchCompilerTest.test248_jar_ref_in_jar fails on Windows.

https://download.eclipse.org/eclipse/downloads/drops4/I20240725-1800/testresults/html/org.eclipse.jdt.core.tests.compiler_ep433I-unit-win32-java17_win32.win32.x86_64_17.html

Unexpected problems [out: ][err: ---------- 1. ERROR in C:\Users\genie.releng\AppData\Local\Temp\comptest\run.1721963082240\regression\src\p\X.java (at line 4) A a; ^ A cannot be resolved to a type ---------- 2. ERROR in C:\Users\genie.releng\AppData\Local\Temp\comptest\run.1721963082240\regression\src\p\X.java (at line 5) G g; ^ G cannot be resolved to a type ---------- 2 problems (2 errors) ]

junit.framework.AssertionFailedError: Unexpected problems [out: ][err: ----------
1. ERROR in C:\Users\genie.releng\AppData\Local\Temp\comptest\run.1721963082240\regression\src\p\X.java (at line 4)
A a;
^
A cannot be resolved to a type
----------
2. ERROR in C:\Users\genie.releng\AppData\Local\Temp\comptest\run.1721963082240\regression\src\p\X.java (at line 5)
G g;
^
G cannot be resolved to a type
----------
2 problems (2 errors)
]
at junit.framework.Assert.fail(Assert.java:57)
at junit.framework.Assert.assertTrue(Assert.java:22)
at junit.framework.TestCase.assertTrue(TestCase.java:192)
at org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest.runTest(AbstractBatchCompilerTest.java:651)
at org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest.runConformTest(AbstractBatchCompilerTest.java:478)
at org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest.test248_jar_ref_in_jar(BatchCompilerTest.java:8979)
@iloveeclipse iloveeclipse added bug Something isn't working regression Something was broken by a previous change Windows Happens on Windows OS test Work on unit tests, no change of productive code labels Jul 26, 2024
@iloveeclipse iloveeclipse mentioned this issue Jul 26, 2024
Open
12 tasks
@iloveeclipse
Copy link
Member Author

iloveeclipse commented Aug 2, 2024 •
edited
Loading

On Windows I see that the tests start to misbehave after test test245_jar_ref_in_jar - the test succeeds, but report this warning at the end and following 6 tests are failing.

org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest.test245_jar_ref_in_jar - 1.8

WARNING in test: org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest - tearDown
	- problems occured while deleting C:\tmp\comptest\run.1722611893110\lib\lib8.jar
	- stack trace:
		-> org.eclipse.jdt.core.tests.util.Util.waitUntilFileDeleted(Util.java:1357)
		-> org.eclipse.jdt.core.tests.util.Util.delete(Util.java:536)
		-> org.eclipse.jdt.core.tests.util.Util.flushDirectoryContent(Util.java:800)
		-> org.eclipse.jdt.core.tests.compiler.regression.AbstractRegressionTest.tearDown(AbstractRegressionTest.java:4143)
	- lib file info: read, write, exist, dir
		+ children: 
			- lib8.jar file info: read, write, exist, file
			- lib9.jar file info: read, write, exist, file
			- s file info: read, write, exist, dir
				+ children: 
					- lib6.jar file info: read, write, exist, file

	!!! ERROR: C:\tmp\comptest\run.1722611893110\lib\lib8.jar was never deleted even after having waited 10000ms!!!

org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest.test246_jar_ref_in_jar - 1.8

I assume org.eclipse.jdt.internal.compiler.batch.Main.performCompilation() doesn't properly clean up opened nested jars and that seem to happen now after update to 1.8 because we probably run into some code path that was never entered with Java 1.4 compilation mode.

The code is called from org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest.invokeCompiler(PrintWriter, PrintWriter, Object, TestCompilationProgress) and the failing tests in question have rather complex project libraries setup involving nested jar libraries.

The test failures are actually follow up problems of the test setup/teardown being unable to delete / create required jars.

iloveeclipse added a commit to iloveeclipse/eclipse.jdt.core that referenced this issue Aug 2, 2024
@iloveeclipse
Smaller test code cleanup to fail early
1e21c61 
Don't hide exceptions that cause test to fail with seemingly unrelated
errors later.

See eclipse-jdt#2766
@iloveeclipse iloveeclipse mentioned this issue Aug 2, 2024
Merged
@iloveeclipse
Copy link
Member Author

On Linux, thanks to lsof, I see that the code in question works as expected, and the opened file descriptor is closed in org.eclipse.jdt.internal.compiler.batch.ClasspathJar.reset() for the lib/lib8.jar with this stack:

ClasspathJar.reset() line: 281	
FileSystem.cleanup() line: 430	
Main.performCompilation() line: 4726	
Main.compile(String[]) line: 1711	
BatchCompilerTest(AbstractBatchCompilerTest).invokeCompiler(PrintWriter, PrintWriter, Object, AbstractBatchCompilerTest$TestCompilationProgress) line: 672	
BatchCompilerTest(AbstractBatchCompilerTest).runTest(boolean, String[], Object, String, String, boolean, AbstractBatchCompilerTest$TestCompilationProgress) line: 589	
BatchCompilerTest(AbstractBatchCompilerTest).runConformTest(String[], String, String, String, boolean) line: 475	
BatchCompilerTest.test245_jar_ref_in_jar() line: 8902

So at least from Linux point of view, JDT works as expected...

iloveeclipse added a commit that referenced this issue Aug 3, 2024
@iloveeclipse
Smaller test code cleanup to fail early
d714ced 
Don't hide exceptions that cause test to fail with seemingly unrelated
errors later.

See #2766
@iloveeclipse iloveeclipse mentioned this issue Aug 12, 2024
Closed
@jukzi
Copy link
Contributor

jukzi commented Aug 14, 2024

It is an error in JDK that JarFile and its underlying ZIpFile ist never closed when System.getSecurityManager()==null in
jdk.internal.loader.URLClassPath.JarLoader.checkJar(JarFile)

stacktrace when it happens (with JDK 21.0.2+13):

Thread [main] (Suspended)	
	jdk.internal.loader.URLClassPath$JarLoader.checkJar(java.util.jar.JarFile) line: 798	
	jdk.internal.loader.URLClassPath$JarLoader.getJarFile(java.net.URL) line: 808	
	jdk.internal.loader.URLClassPath$JarLoader$1.run() line: 774	
	jdk.internal.loader.URLClassPath$JarLoader$1.run() line: 768	
	java.security.AccessController.executePrivileged(java.security.PrivilegedExceptionAction<T>, java.security.AccessControlContext, java.lang.Class<?>) line: 809	
	java.security.AccessController.doPrivileged(java.security.PrivilegedExceptionAction<T>, java.security.AccessControlContext) line: 714	
	jdk.internal.loader.URLClassPath$JarLoader.ensureOpen() line: 767	
	jdk.internal.loader.URLClassPath$JarLoader.<init>(java.net.URL, java.net.URLStreamHandler, java.util.HashMap<java.lang.String,jdk.internal.loader.URLClassPath.Loader>, java.security.AccessControlContext) line: 734	
	jdk.internal.loader.URLClassPath$3.run() line: 497	
	jdk.internal.loader.URLClassPath$3.run() line: 479	
	java.security.AccessController.executePrivileged(java.security.PrivilegedExceptionAction<T>, java.security.AccessControlContext, java.lang.Class<?>) line: 809	
	java.security.AccessController.doPrivileged(java.security.PrivilegedExceptionAction<T>, java.security.AccessControlContext) line: 714	
	jdk.internal.loader.URLClassPath.getLoader(java.net.URL) line: 478	
	jdk.internal.loader.URLClassPath.getLoader(int) line: 446	
	jdk.internal.loader.URLClassPath$1.next() line: 342	
	jdk.internal.loader.URLClassPath$1.hasMoreElements() line: 353	
	java.net.URLClassLoader$3$1.run() line: 662	
	java.net.URLClassLoader$3$1.run() line: 660	
	java.security.AccessController.executePrivileged(java.security.PrivilegedAction<T>, java.security.AccessControlContext, java.lang.Class<?>) line: 778	
	java.security.AccessController.doPrivileged(java.security.PrivilegedAction<T>, java.security.AccessControlContext) line: 400	
	java.net.URLClassLoader$3.next() line: 659	
	java.net.URLClassLoader$3.hasMoreElements() line: 684	
	java.lang.CompoundEnumeration<E>.next() line: 2756	
	java.lang.CompoundEnumeration<E>.hasMoreElements() line: 2765	
	java.util.ServiceLoader$LazyClassPathLookupIterator<T>.nextProviderClass() line: 1210	
	java.util.ServiceLoader$LazyClassPathLookupIterator<T>.hasNextService() line: 1228	
	java.util.ServiceLoader$LazyClassPathLookupIterator<T>.hasNext() line: 1273	
	java.util.ServiceLoader$2.hasNext() line: 1309	
	java.util.ServiceLoader$3.hasNext() line: 1393	
	org.eclipse.jdt.internal.compiler.apt.dispatch.BatchAnnotationProcessorManager.discoverNextProcessor() line: 177	
	org.eclipse.jdt.internal.compiler.apt.dispatch.RoundDispatcher.round() line: 118	
	org.eclipse.jdt.internal.compiler.apt.dispatch.BatchAnnotationProcessorManager(org.eclipse.jdt.internal.compiler.apt.dispatch.BaseAnnotationProcessorManager).processAnnotations(org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration[], org.eclipse.jdt.internal.compiler.lookup.ReferenceBinding[], boolean) line: 172	
	org.eclipse.jdt.internal.compiler.Compiler.processAnnotations() line: 972	
	org.eclipse.jdt.internal.compiler.Compiler.compile(org.eclipse.jdt.internal.compiler.env.ICompilationUnit[], boolean) line: 449	
	org.eclipse.jdt.internal.compiler.Compiler.compile(org.eclipse.jdt.internal.compiler.env.ICompilationUnit[]) line: 425	
	org.eclipse.jdt.internal.compiler.batch.Main.performCompilation() line: 4710	
	org.eclipse.jdt.internal.compiler.batch.Main.compile(java.lang.String[]) line: 1711	
	org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest(org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest).invokeCompiler(java.io.PrintWriter, java.io.PrintWriter, java.lang.Object, org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest$TestCompilationProgress) line: 672	
	org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest(org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest).runTest(boolean, java.lang.String[], java.lang.Object, java.lang.String, java.lang.String, boolean, org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest$TestCompilationProgress) line: 589	
	org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest(org.eclipse.jdt.core.tests.compiler.regression.AbstractBatchCompilerTest).runConformTest(java.lang.String[], java.lang.String, java.lang.String, java.lang.String, boolean) line: 475	
	org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest.test245_jar_ref_in_jar() line: 608	
	java.lang.invoke.LambdaForm$DMH.0x000001d130234000.invokeVirtual(java.lang.Object, java.lang.Object) line: not available	
	java.lang.invoke.LambdaForm$MH.0x000001d130098800.invoke(java.lang.Object, java.lang.Object) line: not available	
	java.lang.invoke.Invokers$Holder.invokeExact_MT(java.lang.Object, java.lang.Object, java.lang.Object) line: not available	
	jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(java.lang.Object, java.lang.Object[]) line: 153	
	jdk.internal.reflect.DirectMethodHandleAccessor.invoke(java.lang.Object, java.lang.Object[]) line: 103	
	java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object...) line: 580	
	org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest(junit.framework.TestCase).runTest() line: 177	
	org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest(org.eclipse.jdt.core.tests.junit.extension.TestCase).runTest() line: 972	
	org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest(junit.framework.TestCase).runBare() line: 142	
	junit.framework.TestResult$1.protect() line: 122	
	junit.framework.TestResult.runProtected(junit.framework.Test, junit.framework.Protectable) line: 142	
	junit.framework.TestResult.run(junit.framework.TestCase) line: 125	
	org.eclipse.jdt.core.tests.compiler.regression.BatchCompilerTest(junit.framework.TestCase).run(junit.framework.TestResult) line: 130	
	org.eclipse.jdt.core.tests.compiler.regression.RegressionTestSetup(junit.framework.TestSuite).runTest(junit.framework.Test, junit.framework.TestResult) line: 241	
	org.eclipse.jdt.core.tests.compiler.regression.RegressionTestSetup(junit.framework.TestSuite).run(junit.framework.TestResult) line: 236	
	org.eclipse.jdt.core.tests.compiler.regression.RegressionTestSetup(org.eclipse.jdt.core.tests.util.CompilerTestSetup).run(junit.framework.TestResult) line: 59	
	junit.framework.TestSuite.runTest(junit.framework.Test, junit.framework.TestResult) line: 241	
	junit.framework.TestSuite.run(junit.framework.TestResult) line: 236	
	org.junit.internal.runners.SuiteMethod(org.junit.internal.runners.JUnit38ClassRunner).run(org.junit.runner.notification.RunNotifier) line: 90	
	org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(org.eclipse.jdt.internal.junit.runner.TestExecution) line: 93	
	org.eclipse.jdt.internal.junit.runner.TestExecution.run(org.eclipse.jdt.internal.junit.runner.ITestReference[]) line: 40	
	org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner(org.eclipse.jdt.internal.junit.runner.RemoteTestRunner).runTests(java.lang.String[], java.lang.String, org.eclipse.jdt.internal.junit.runner.TestExecution) line: 530	
	org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner(org.eclipse.jdt.internal.junit.runner.RemoteTestRunner).runTests(org.eclipse.jdt.internal.junit.runner.TestExecution) line: 758	
	org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner(org.eclipse.jdt.internal.junit.runner.RemoteTestRunner).run() line: 453	
	org.eclipse.pde.internal.junit.runtime.RemotePluginTestRunner.main(java.lang.String[]) line: 83	
	org.eclipse.pde.internal.junit.runtime.CoreTestApplication.start(org.eclipse.equinox.app.IApplicationContext) line: 28	
	org.eclipse.equinox.internal.app.EclipseAppHandle.run(java.lang.Object) line: 208	
	org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.runApplication(java.lang.Object) line: 143	
	org.eclipse.core.runtime.internal.adaptor.EclipseAppLauncher.start(java.lang.Object) line: 109	
	org.eclipse.core.runtime.adaptor.EclipseStarter.run(java.lang.Object) line: 439	
	org.eclipse.core.runtime.adaptor.EclipseStarter.run(java.lang.String[], java.lang.Runnable) line: 271	
	java.lang.invoke.DirectMethodHandle$Holder.invokeStatic(java.lang.Object, java.lang.Object, java.lang.Object) line: not available	
	java.lang.invoke.LambdaForm$MH.0x000001d130005c00.invoke(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object) line: not available	
	java.lang.invoke.LambdaForm$MH.0x000001d130006000.invokeExact_MT(java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object, java.lang.Object) line: not available	
	jdk.internal.reflect.DirectMethodHandleAccessor.invokeImpl(java.lang.Object, java.lang.Object[]) line: 155	
	jdk.internal.reflect.DirectMethodHandleAccessor.invoke(java.lang.Object, java.lang.Object[]) line: 103	
	java.lang.reflect.Method.invoke(java.lang.Object, java.lang.Object...) line: 580	
	org.eclipse.equinox.launcher.Main.invokeFramework(java.lang.String[], java.net.URL[]) line: 668	
	org.eclipse.equinox.launcher.Main.basicRun(java.lang.String[]) line: 605	
	org.eclipse.equinox.launcher.Main.run(java.lang.String[]) line: 1481	
	org.eclipse.equinox.launcher.Main.main(java.lang.String[]) line: 1454

And even the Cleaner does not automatically close the ZipFile as soon as it become only Phantomrechable (is only referenced by PhantomCleanable) but only if - by chance - a GC happens:
image
Manually triggering a GC in that state using VisualVM invokes the cleaner. That can be also used as workaround by calling System.gc() when waiting for the File to be deleted.

jukzi pushed a commit to jukzi/eclipse.jdt.core that referenced this issue Aug 14, 2024
@EcljpseB0T
Fix BatchCompilerTest - ZipFile "used by another process" on win ecli…
8ee066e 
…pse-jdt#2766

eclipse-jdt#2766

Works around error in JDK which forgets to close Jar when
System.getSecurityManager()==null

As workaround trigger a GC to make a associated Cleaner Run for the
ZipFile no longer referenced.
@jukzi jukzi mentioned this issue Aug 14, 2024
Merged
1 task
@jukzi
Copy link
Contributor

jukzi commented Aug 14, 2024 •
edited
Loading

see https://github.com/openjdk/jdk/blob/38bd8a36704a962f0ad1052fd2ec150a61663256/src/java.base/share/classes/jdk/internal/loader/URLClassPath.java#L780

jukzi pushed a commit to jukzi/eclipse.jdt.core that referenced this issue Aug 14, 2024
@EcljpseB0T
Fix BatchCompilerTest - ZipFile "used by another process" on win ecli…
89ceaa9 
…pse-jdt#2766

eclipse-jdt#2766

Works around error in JDK which forgets to close Jar when
System.getSecurityManager()==null

As workaround trigger a GC to make a associated Cleaner Run for the
ZipFile no longer referenced.
jukzi pushed a commit that referenced this issue Aug 14, 2024
@EcljpseB0T @jukzi
Fix BatchCompilerTest - ZipFile "used by another process" on win #2766
d116b06 
#2766

Works around error in JDK which forgets to close Jar when
System.getSecurityManager()==null

As workaround trigger a GC to make a associated Cleaner Run for the
ZipFile no longer referenced.
@jukzi
Copy link
Contributor

jukzi commented Aug 15, 2024

errors gone in https://download.eclipse.org/eclipse/downloads/drops4/I20240814-1800/testResults.php

@jukzi jukzi closed this as completed Aug 15, 2024
@jukzi
Copy link
Contributor

jukzi commented Aug 15, 2024

reported to JDK https://bugs.openjdk.org/browse/JDK-8338445

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
bug Something isn't working regression Something was broken by a previous change test Work on unit tests, no change of productive code Windows Happens on Windows OS
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@iloveeclipse @jukzi