Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Inconsistent handling of files in an output folder - full vs. incremental build #2921

Merged
merged 4 commits into from
Feb 4, 2024

Conversation

stephan-herrmann
Copy link
Contributor

fixes #2920

incremental build

fixes eclipse-xtext#2920

Signed-off-by: Stephan Herrmann <stephan.herrmann@berlin.de>
@stephan-herrmann
Copy link
Contributor Author

I did poke around existing tests, to see if anything could be used as a template for creating a test challenging this change, but I failed to find any test that would even invoke XtextBuilder.incrementalBuild(). If such tests exist, I'd be happy to try to work from there. Any hints?

@cdietrich
Copy link
Contributor

i assume some of the xtend.ide tests test some of the usecases
/org.eclipse.xtend.ide.tests/src/org/eclipse/xtend/ide/tests/builder
as these do creation/changing files

=> you might check the history of code in xtext-eclipse
and then search corrsponding tests
in xtext-xtend
e.g. https://github.com/eclipse/xtext-xtend/pull/1309/files

there is also a builder test language in builder.tests.
but i am not sure what it is used for

@stephan-herrmann
Copy link
Contributor Author

By copying from org.eclipse.xtext.builder.impl.Bug386476Test I managed to write a test that get's close to the problem. Just when I thought I have a witness, I learned that the problem is normally prevented deep down in the following call chain:

  • XtextBuilder.createDeltaVisitor().new IResourceDeltaVisitor().visit()
    • toBeBuiltComputer.updateStorage(..) <-- here we indeed pass a resource from "bin"
      • ToBeBuildComputer.getURI(IStorage)
        • mapper.getUri(file)
          • isValidUri(..)
            • uriValidator.isValid(..)
              • resourceServiceProvider.canHandle(..)

In the last call, if the provider is a DefaultResourceUIServiceProvider, then this snippet blocks the URI:

	if (isJavaCoreAvailable()) {
		return !isJavaTargetFolder(storage);
	}

Going back to our original problematic case, I realize that we have a test case with a combination that's perhaps unsupported: we invoke the builder on a proper workspace, but we don't have UI, so we only get a DefaultResourceServiceProvider. This situation arose when I tried to create minimal tests with capability to challenge the builder.

Am I right that Xtext doesn't have a "mode" for a headless eclipse? Is that why only the "UI" variant asks isJavaCoreAvailable()? IOW, no distinction headless vs. ui but only standalone vs. IDE?

If the answer is indeed "unsupported", then my change may be "correct but irrelevant". So would you still want it?

@szarnekow wdyt?

Signed-off-by: Stephan Herrmann <stephan.herrmann@berlin.de>
@cdietrich
Copy link
Contributor

My assumption would be that even in headless eclipse with workspace
DefaultResourceUIServiceProvider Is bound.
As Xtext does not support this ootb
At some customers I saw special activators / ui modules for that use case

how do you implement that

@LorenzoBettini
Copy link
Contributor

@stephan-herrmann in such tests, do you use the UI injector provider?

@stephan-herrmann
Copy link
Contributor Author

Our headless tests use a slim handcrafted HeadlessSharedStateModule including a HeadlessJdtHelper.

All this is inspired by https://github.com/eclipse-jdt/eclipse.jdt.core/tree/master/org.eclipse.jdt.core.tests.builder which has no dependencies on anything ui :)

DefaultResourceUIServiceProvider, OTOH, lives in org.eclipse.xtext.ui and hence will never work in a truly headless eclipse, where none of org.eclipse.ui* are available.

Initially I had assumed the separation between o.e.xtext and o.e.xtext.ui can be compared to the separation between o.e.jdt.core and o.e.jdt.ui, but apparently xtext is not prepared to leverage jdt.core without any ui.

As all this is only about minimizing tests (we don't ship headless applications), it is not a problem for production.

@cdietrich
Copy link
Contributor

cdietrich commented Jan 31, 2024

if all the deps are optional xtext.ui still should live in an equinox env without workbench.
of course we dont have separate bundles though

@stephan-herrmann
Copy link
Contributor Author

if all the deps are optional

but that's not the case 😄 - I see lots of non-optional ui dependencies in xtext.ui, so this plugin will simply not resolve in a truly headless environment.

xtext.ui still should live in an equinox env without workbench. of course we dont have separate bundles though

fair enough.

@cdietrich
Copy link
Contributor

the funny thing is in the project i worked with andrey loskutov on it does. (of course with a bunch of extra bundles in the product)

@stephan-herrmann
Copy link
Contributor Author

With all that I learned in this PR, I'd be fine with closing it without merge.

Is anyone still planing to review? I mean it still seems to be the "right" thing to do, but the effective benefit might be low (a tiny performance improvement, more symmetric code).

}

@Override
public void build(IBuildContext context, IProgressMonitor monitor) throws CoreException {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@stephan-herrmann This test is not sufficient to validate that the fix actually fixes something. It's green when I run this on main. Do you plan to augment it somehow or would you like to merge this branch as is?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Have you seen this comment?

In essence we'd need an environment with XtextBuilder but without DefaultResourceUIServiceProvider.

I can confirm that one of our tests, which uses DefaultResourceServiceProvider instead, does trigger the problem. Do you want me to construct such a test for xtext? It looked like that would need jumping through some hoops.

That's why I was uncertain whether we should actually pursue this further...

@LorenzoBettini
Copy link
Contributor

@stephan-herrmann maybe I didn't get the overall picture but if the problem is

In the last call, if the provider is a DefaultResourceUIServiceProvider, then this snippet blocks the URI:

	if (isJavaCoreAvailable()) {
		return !isJavaTargetFolder(storage);
	}

To recreate your scenario, where you wouldn't want the isJavaCoreAvailable to return true even in a UI test, shouldn't it be enough to create a custom implementation of JdtHelper that implements https://github.com/eclipse/xtext/blob/88acf1555988cd81e1756abc0d6d2c3f1f1efa11/org.eclipse.xtext.ui.shared/src/org/eclipse/xtext/ui/shared/JdtHelper.java#L52 always returning false and make sure your test is run with your custom implementation injected (this requires to create a custom injector provider)?

could that recreate the problem and verify the fix correctly, @szarnekow ?

@stephan-herrmann
Copy link
Contributor Author

@stephan-herrmann maybe I didn't get the overall picture but if the problem is

In the last call, if the provider is a DefaultResourceUIServiceProvider, then this snippet blocks the URI:

	if (isJavaCoreAvailable()) {
		return !isJavaTargetFolder(storage);
	}

To recreate your scenario, where you wouldn't want the isJavaCoreAvailable to return true even in a UI test, shouldn't it be enough to create a custom implementation of JdtHelper that implements ...

This is correct, but I don't know if it's significantly easier to re-bind JdtHelper then IResourceServiceProvider. Frankly, I just didn't feel like creating a whole now language with its own guice module, assuming that would be overkill.

@LorenzoBettini
Copy link
Contributor

I didn't mean create a new language: just use a custom ui injector provider, inheriting the one currently used. If you want I can have a try.

@stephan-herrmann
Copy link
Contributor Author

I didn't mean create a new language: just use a custom ui injector provider, inheriting the one currently used. If you want I can have a try.

that'd be cool, thanks.

@LorenzoBettini
Copy link
Contributor

Apparently, I was too optimistic because I've never had problems in customizing test injectors for headless tests, but I never did that for UI tests.

This does not work as expected:

@RunWith(XtextRunner.class)
@InjectWith(GH2920Test.BuilderTestLanguageInjectorProviderCustom.class)
public class GH2920Test extends AbstractParticipatingBuilderTest {
	private IJavaProject javaProject;

	@Inject private IResourceDescriptionsProvider descriptionsProvider;

	@Singleton
	public static class JdtHelperCustom extends JdtHelper {
		@Override
		protected boolean computeJavaCoreAvailable() {
			return false;
		}
	}

	public static class BuilderTestLanguageInjectorProviderCustom extends BuilderTestLanguageInjectorProvider {
		@Override
		public Injector getInjector() {
			Module mixin = Modules2.mixin(new BuilderTestLanguageRuntimeModule(),
				new SharedStateModule() {
					@Override
					public Provider<IJdtHelper> provideJdtHelper() {
						return new Provider<>() {
							@Override
							public IJdtHelper get() {
								return new JdtHelperCustom();
							}
						};
					}
				},
				new BuilderTestLanguageUiModule(TestsActivator.getInstance())
			);
			return Guice.createInjector(mixin);
		}
	}
...

In fact, the custom injector is used only to inject things in the test case, but the workspace and other services, including the build participants, are injected with the executable extension factory, i.e., with the injector created in the test language activator, which cannot be customized.

Maybe @szarnekow can shed some light on whether what I wanted to do is feasible without creating a new language.

@LorenzoBettini
Copy link
Contributor

I think I found a solution to recreate the problem. I have to do a few clean up and then I'll tell you about that

@LorenzoBettini
Copy link
Contributor

@stephan-herrmann I created a PR against your branch with the test reproducing the problem. Of course, in the PR the test succeeds, but if you try uncommenting your modifications to XtextBuilder it properly fails.
My PR is based on top your branch https://github.com/stephan-herrmann/xtext/tree/issue2920 so it also contains your fix for the issue.
If you merge it in your branch, this very PR will be updated.

LorenzoBettini and others added 2 commits February 3, 2024 21:44
Test case that effectively demonstrates the problem

the customization happens at the level of the executable extension
factory
Signed-off-by: Stephan Herrmann <stephan.herrmann@berlin.de>
@stephan-herrmann
Copy link
Contributor Author

@stephan-herrmann I created a PR against your branch with the test reproducing the problem. Of course, in the PR the test succeeds, but if you try uncommenting your modifications to XtextBuilder it properly fails. My PR is based on top your branch https://github.com/stephan-herrmann/xtext/tree/issue2920 so it also contains your fix for the issue. If you merge it in your branch, this very PR will be updated.

Thanks, @LorenzoBettini for going the extra mile!

Now on top of your changes I proposed a variant that comes even closer to our original problem: in our case the IResourceServiceProvider was the point where behavior deviated, so I bound something that looks like an IResourceUIServiceProvider (because that's what is requested in the test setup) and behaves like a DefaultResourceServiceProvider. This, too, succeeds to demonstrate the problem when the actual fix is not in.

If that's too much magic in the test, I'm fine with reverting this last commit.

Copy link
Contributor

@szarnekow szarnekow left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very nice!

@szarnekow
Copy link
Contributor

Thank you for the improved test. I like the approach! Let’s wait for the build to succeed and merge afterwards.

@LorenzoBettini LorenzoBettini added this to the Release_2.34 milestone Feb 4, 2024
@LorenzoBettini LorenzoBettini merged commit a144255 into eclipse-xtext:main Feb 4, 2024
5 checks passed
@stephan-herrmann stephan-herrmann deleted the issue2920 branch February 4, 2024 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Inconsistent handling of files in an output folder - full vs. incremental build
4 participants