NPE in org.eclipse.jdt.internal.compiler.ast.MessageSend.analyseCode

[martinlippert]

Follow up from #452 (comment)

JDT core compiler throws a

java.lang.NullPointerException: Cannot invoke "org.eclipse.jdt.internal.compiler.lookup.MethodBinding.isStatic()" because "this.binding" is null
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.analyseCode(MessageSend.java:149)
	at org.eclipse.jdt.internal.compiler.ast.LambdaExpression.analyseCode(LambdaExpression.java:599)
	at org.eclipse.jdt.internal.compiler.ast.ReferenceExpression.generateImplicitLambda(ReferenceExpression.java:263)
	at org.eclipse.jdt.internal.compiler.ast.ReferenceExpression.generateCode(ReferenceExpression.java:334)
	at org.eclipse.jdt.internal.compiler.ast.Statement.generateArguments(Statement.java:443)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.generateCode(MessageSend.java:549)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.generateCode(MessageSend.java:542)
	at org.eclipse.jdt.internal.compiler.ast.Expression.generateCode(Expression.java:764)
	at org.eclipse.jdt.internal.compiler.ast.Block.generateCode(Block.java:88)
	at org.eclipse.jdt.internal.compiler.ast.IfStatement.generateCode(IfStatement.java:205)
	at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.generateCode(AbstractMethodDeclaration.java:358)
	at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.generateCode(AbstractMethodDeclaration.java:292)
	at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.generateCode(TypeDeclaration.java:759)
	at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.generateCode(TypeDeclaration.java:829)
	at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.generateCode(CompilationUnitDeclaration.java:414)
	at org.eclipse.jdt.internal.compiler.Compiler.process(Compiler.java:915)
	at org.eclipse.jdt.internal.compiler.ProcessTaskManager.run(ProcessTaskManager.java:145)
	at java.base/java.lang.Thread.run(Thread.java:840)

Steps to reproduce:

• vanilla Eclipse 2023-12 M3 Java Package install (I use this to have Buildship included to easily import project)
• clone https://github.com/spring-projects/spring-integration.git
• Import project as Gradle project, select root of cloned project, walk through the import wizard without selecting anything specifically
• the workspace is populated with 46 projects (sorry for the somewhat large project/workspace)

Initial build runs and error popup appears. Same happens when doing a Project -> Clean... on all projects or on the spring-integration-core project individually.

(Unfortunately, I haven't tracked this down to the exact code snippet that causes this, but I hope this helps to reproduce this nevertheless)

[stephan-herrmann]

Unfortunately, this very code locations throws billions of expected NPEs, which are then caught in LambdaExpression.analyzeExceptions()

The following patch makes the specific situation directly observable using a simple breakpoint on the added sysout statement:

diff --git a/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/LambdaExpression.java b/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/LambdaExpression.java
index 3afbbf6..edbcdd2 100644
--- a/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/LambdaExpression.java
+++ b/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/LambdaExpression.java
@@ -560,6 +560,7 @@
 		boolean oldAnalyseResources = compilerOptions.analyseResourceLeaks;
 		compilerOptions.analyseResourceLeaks = false;
 		try {
+			this.scope.inAnalyseExceptions = true;
 			this.body.analyseCode(this.scope,
 									 ehfc = new ExceptionInferenceFlowContext(null, this, Binding.NO_EXCEPTIONS, null, this.scope, FlowInfo.DEAD_END),
 									 UnconditionalFlowInfo.fakeInitializedFlowInfo(this.firstLocalLocal, this.scope.referenceType().maxFieldCount));
@@ -567,6 +568,7 @@
 		} catch (Exception e) {
 			// drop silently.
 		} finally {
+			this.scope.inAnalyseExceptions = false;
 			compilerOptions.analyseResourceLeaks = oldAnalyseResources;
 		}
 	}
diff --git a/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/MessageSend.java b/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/MessageSend.java
index eed0f2d..51bef94 100644
--- a/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/MessageSend.java
+++ b/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/ast/MessageSend.java
@@ -146,6 +146,8 @@
 
 @Override
 public FlowInfo analyseCode(BlockScope currentScope, FlowContext flowContext, FlowInfo flowInfo) {
+	if (this.binding == null && !currentScope.isInAnalyseException())
+		System.out.println("HERE!");
 	boolean nonStatic = !this.binding.isStatic();
 	boolean wasInsideAssert = ((flowContext.tagBits & FlowContext.HIDE_NULL_COMPARISON_WARNING) != 0);
 	flowInfo = this.receiver.analyseCode(currentScope, flowContext, flowInfo, nonStatic).unconditionalInits();
diff --git a/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/lookup/Scope.java b/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/lookup/Scope.java
index 85a2913..f60e8ed 100644
--- a/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/lookup/Scope.java
+++ b/org.eclipse.jdt.core.compiler.batch/src/org/eclipse/jdt/internal/compiler/lookup/Scope.java
@@ -118,6 +118,14 @@
 	public final Scope parent;
 	public final CompilationUnitScope compilationUnitScope;
 	private Map<String, Supplier<ReferenceBinding>> commonTypeBindings = null;
+	public boolean inAnalyseExceptions;
+	public boolean isInAnalyseException() {
+		if (this.inAnalyseExceptions)
+			return true;
+		if (this.parent != null)
+			return this.parent.isInAnalyseException();
+		return false;
+	}
 
 	private static class NullDefaultRange {
 		final int start, end;

I would attach the file, but for something.patch as well as something.PATCH github simple tells me:

We don’t support that file type.

Try again with GIF, JPEG, JPG, MOV, MP4, PNG, SVG, WEBM, CPUPROFILE, CSV, DMP, DOCX, FODG, FODP, FODS, FODT, GZ, JSON, JSONC, LOG, MD, ODF, ODG, ODP, ODS, ODT, PATCH, PDF, PPTX, TGZ, TXT, XLS, XLSX or ZIP.

If a patch is not a patch then github doesn't look like a tool for software development.

[martinlippert]

We (actually my colleague @jvalkeal did) found the exact piece of code that causes this issue:

It is:
https://github.com/spring-projects/spring-integration/blob/fd8e0368fbd3314b0cc4e0340f0dc442bcc7c5fc/spring-integration-core/src/main/java/org/springframework/integration/dsl/ConsumerEndpointSpec.java#L372

.acceptIfNotEmpty(this.notPropagatedHeaders, producingMessageHandler::setNotPropagatedHeaders)

The problem seems to be the method reference producingMessageHandler::setNotPropagatedHeaders. If you replace this with:

.acceptIfNotEmpty(this.notPropagatedHeaders, (x) -> {producingMessageHandler.setNotPropagatedHeaders(x);})

the compiler doesn't throw a NPE.

The referenced method looks like this:

public void setNotPropagatedHeaders(String... headers) {
	updateNotPropagatedHeaders(headers, false);
}

You can reproduce the error also by opening the source file for the type ConsumerEndpointSpec in the editor. As soon as you have the Java source editor open, it will throw a bunch of those exceptions into the error log:

java.lang.NullPointerException: Cannot invoke "org.eclipse.jdt.internal.compiler.lookup.MethodBinding.isStatic()" because "this.binding" is null
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.analyseCode(MessageSend.java:149)
	at org.eclipse.jdt.internal.compiler.ast.LambdaExpression.analyseCode(LambdaExpression.java:599)
	at org.eclipse.jdt.internal.compiler.ast.ReferenceExpression.generateImplicitLambda(ReferenceExpression.java:263)
	at org.eclipse.jdt.internal.compiler.ast.ReferenceExpression.generateCode(ReferenceExpression.java:334)
	at org.eclipse.jdt.internal.compiler.ast.Statement.generateArguments(Statement.java:443)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.generateCode(MessageSend.java:549)
	at org.eclipse.jdt.internal.compiler.ast.MessageSend.generateCode(MessageSend.java:542)
	at org.eclipse.jdt.internal.compiler.ast.Expression.generateCode(Expression.java:764)
	at org.eclipse.jdt.internal.compiler.ast.Block.generateCode(Block.java:88)
	at org.eclipse.jdt.internal.compiler.ast.IfStatement.generateCode(IfStatement.java:205)
	at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.generateCode(AbstractMethodDeclaration.java:358)
	at org.eclipse.jdt.internal.compiler.ast.AbstractMethodDeclaration.generateCode(AbstractMethodDeclaration.java:292)
	at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.generateCode(TypeDeclaration.java:759)
	at org.eclipse.jdt.internal.compiler.ast.TypeDeclaration.generateCode(TypeDeclaration.java:829)
	at org.eclipse.jdt.internal.compiler.ast.CompilationUnitDeclaration.generateCode(CompilationUnitDeclaration.java:414)
	at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:1324)
	at org.eclipse.jdt.core.dom.CompilationUnitResolver.resolve(CompilationUnitResolver.java:790)
	at org.eclipse.jdt.core.dom.ASTParser.internalCreateAST(ASTParser.java:1245)
	at org.eclipse.jdt.core.dom.ASTParser.createAST(ASTParser.java:868)
	at org.eclipse.jdt.core.manipulation.CoreASTProvider$1.run(CoreASTProvider.java:294)
	at org.eclipse.core.runtime.SafeRunner.run(SafeRunner.java:47)
	at org.eclipse.jdt.core.manipulation.CoreASTProvider.createAST(CoreASTProvider.java:286)
	at org.eclipse.jdt.core.manipulation.CoreASTProvider.getAST(CoreASTProvider.java:199)
	at org.eclipse.jdt.core.manipulation.SharedASTProviderCore.getAST(SharedASTProviderCore.java:138)
	at org.eclipse.jdt.internal.ui.viewsupport.SelectionListenerWithASTManager$PartListenerGroup.calculateASTandInform(SelectionListenerWithASTManager.java:166)
	at org.eclipse.jdt.internal.ui.viewsupport.SelectionListenerWithASTManager$PartListenerGroup$1.run(SelectionListenerWithASTManager.java:151)
	at org.eclipse.core.internal.jobs.Worker.run(Worker.java:63)

[stephan-herrmann]

Minimal reproducer:

package b;

import java.util.function.Consumer;

public class ConsumerEndpointSpec {
	public void setNotPropagatedHeaders(String... headers) { }
	void foo(Object h) {
		if (h instanceof ConsumerEndpointSpec producingMessageHandler) {
			acceptIfNotEmpty(new String[1], producingMessageHandler::setNotPropagatedHeaders);
		}
	}
	public <T> void acceptIfNotEmpty(T[] value, Consumer<T[]> consumer) { }
}

[stephan-herrmann]

Actually, all these need to meet to trigger the bug:

• we have a varargs methods
• we refer to that method in a reference expression, which forces generation of a synthetic lambda
• the reference expression's receiver is a pattern variable

In that case the synthetic lambda was unable to resolve that flow-scoped variable, causing resolveType() to leave MessageSend.binding unassigned.

Indeed different from any previous reports about NPE at this line of code.

[jvalkeal]

Thanks a lot finding the low level root cause! I suspected it has to do something with varargs but totally failed to create a minimal sample out from spring-integration code.

[stephan-herrmann]

Thanks a lot finding the low level root cause! I suspected it has to do something with varargs but totally failed to create a minimal sample out from spring-integration code.

Thanks for pointing in the right direction in the first place! :)

Indeed creating a minimal sample can be quite tricky. In this case I started with just the offending method and then stubbed lots of types from spring (marveling at all those generic hierarchies :) ), just to make it compilable. How little of that was used for reproducing came as a surprise for myself.

I couldn't understand the low level connections by looking at the example, though, I needed to debug it, to discover the chain of causes and effects, but that's our business as JDT committers :)

[martinlippert]

Any chance to get this into the 2023-12 release? I saw RC2 is already out the door. Maybe a candidate for a RC2a or RC3?

[martinlippert]

@stephan-herrmann Thanks so much for looking into this, very very much appreciated, and awesome to see that you already got this fixed!!!

[foal]

Hi, which release will be including this fix?

[stephan-herrmann]

Hi, which release will be including this fix?

2024-03 (it was the first commit after 2023-12).