Moving expensive code from runtime to compilation using ahead-of-time compilation?

[tobiasschaefer]

Hi,

I want to execute "expensive" code at compile time and save the result so that it is available at runtime. It could be a simple string but maybe also an object. What is the best approach?

For example I want to determine all xyz-files in the resources folder and have that constant list of file names available at runtime without scanning the resources.

I got some hints here to use ahead of time compilation so I defined an annotation (ResourceScan) and implemented a visitor. This visitor does the calculation and calls annotate(...)

However, this has some limitations

• The self-defined annotation ResourceScan can be placed on the application so that the visitor is triggered and the visitor then calls annotate(...). However, it doesn't seem possible to get the value if the annotation is placed on the application because the application is not a bean. Where could it be saved alternatively? I tried writing a file to "build/resources/main/myfile.json". That works but is quite hacky.
• Alternatively, the self-defined annotation ResourceScan could be placed on a random bean so that the visitor is triggered and the visitor then calls annotate(...). However, I don't like this approach because of the randomness. And it doesn't prevent the annotation being set multiple times - which I don't want either.
• It seems that the visitor is not triggered every time there is a change, e.g. a new file in the project will not always trigger it.

Any hints how to accomplish this in general? Or a specific solution to scan for all xyz files in the resources?

[graemerocher]

So in general Micronaut's compile time facilities are for processing source code and generating files based on the definition of annotations in the source code via annotation processing.

There are two modes for this, one is an isolating generator that generates a single source file for each annotated class. Say for example you want to generate some schema JSON file for each class annotated with @Entity you could do that by defining an isolating TypeElementVisitor that visits the Entity annotation and generates a file for each class, here is some example pseudo code:

public final class SchemaGeneratingTypeElementVisitor implements TypeElementVisitor<Entity, Object> {

    private Map<ClassElement, Map> jsonPerClass = new LinkedHashMap();

    @Override
    public VisitorKind getVisitorKind() {
        return VisitorKind.ISOLATING;
    }

    @Override
    public void visitClass(ClassElement element, VisitorContext context) {
        jsonPerClass.put(element, createSchemaJSON(element));
    }

    @Override
     public void finish(VisitorContext visitorContext) {
        jsonPerClass.forEach((classElement, json) -> {
             visitorContext.visitMetaInfFile(classElement.getSimpleName() + ".json", classElement)
                  .ifPresent((generatedFile) -> {
                       try (Writer writer = generatedFile.openWriter()) {
                               objectMapper.writeJson(json, writer);
                        }
                  });
       });
    }
}

This works well incrementally since each time you change an entity and can re-generate for just a single entity.

The other supported approach is aggregating where a single registry type file is produced from lots of entities. Here is another example:

public final class SchemaGeneratingTypeElementVisitor implements TypeElementVisitor<Entity, Object> {

    private Map<String, Map> schema = new LinkedHashMap();
    private Set<ClassElement> classElements = new HashSet();

    @Override
    public VisitorKind getVisitorKind() {
        return VisitorKind.AGGREGATING;
    }

    @Override
    public void visitClass(ClassElement element, VisitorContext context) {
        schema.put(element.getName(), createSchemaJSON(element));
        classElements.add(element);
    }

    @Override
     public void finish(VisitorContext visitorContext) {
            visitorContext.visitMetaInfFile("schema.json", classElements.toArray(new ClassElement[0]))
                  .ifPresent((generatedFile) -> {
                       try (Writer writer = generatedFile.openWriter()) {
                               objectMapper.writeJson(schema, writer);
                        }
                  });
    }
}

In this case every time any entity is changed it needs to re-run the whole visitor for all class elements.

So those are the two approaches supported.

What you are asking doesn't really fit either of those because there is no way for the compiler to say or know that for example the Application class should ALWAYS be re-compiled in order to regenerate your resource definitions.

You can probably do it by generating a file with no originating elements, but it would break incremental compilation and mean EVERY change the user makes would need to re-compile everything which is maybe undesirable but could meet your requirements.

Looking at the requirements it sounds like you would be better off using an external build tool level task to achieve what you want to do.

[tobiasschaefer]

Thanks @graemerocher for that detailed response!

I'm implementing a library so I'd prefer to keep it simple for the application developer. I'd prefer not to support both a Maven and Gradle plugin but that might be the best way to go.

However, I'd like to try it further with annotation processing and maybe accept that the build will not be incrememental.

So just some more questions:

1. why must the visitor know if it is ISOLATING or AGGREGATING? Is this "only" relevant for the incremental build feature so that it know when it must be triggered?
2. Can you please detail "You can probably do it by generating a file with no originating elements". I don't understand this part. Do you have a code snippet?
3. Will anything break if the main Application class is annotated with @Singleton. This would be the case if @ResourceScan is a meta annotation that includes @Singleton so that I can read the value from there.
4. Alternatively, to (3): is there an API to dynamically create a bean? I know that Micronaut generates classes so there must be a way but I haven't found it.
5. So if the requirement is to find all files with extension "xyz" in the resources then is there no approach supported by Micronaut? i.e. I'd need to have the user configure the paths or have the build determine these?

I know these are quite a few more questions. I'd be happy to hear from you.

[graemerocher]

1. why must the visitor know if it is ISOLATING or AGGREGATING? Is this "only" relevant for the incremental build feature so that it know when it must be triggered?

Yes. If there is a 1 to 1 relationship between a class and a file being generated then it should be isolating since it means for incremental build Gradle knows it only needs to regenerate the one file if the class changes and not the whole project.

2. Can you please detail "You can probably do it by generating a file with no originating elements". I don't understand this part. Do you have a code snippet?

For example

micronaut-core/graal/src/main/java/io/micronaut/graal/reflect/GraalTypeElementVisitor.java

Line 316 in 8d931d4

final Optional<GeneratedFile> generatedFile = visitorContext.visitMetaInfFile(resourcesFile);

The graal visitor does this and hence is not incremental. We have plans to move the behaviour of the graal visitor to the gradle/maven plugins for Micronaut

3. Will anything break if the main Application class is annotated with @Singleton. This would be the case if @ResourceScan is a meta annotation that includes @Singleton so that I can read the value from there.

Shouldn't do

4. Alternatively, to (3): is there an API to dynamically create a bean? I know that Micronaut generates classes so there must be a way but I haven't found it.

Not currently, but we have plans to add one. You could try use the BeanDefinitionWriter API to generate a bean if you want.

5. So if the requirement is to find all files with extension "xyz" in the resources then is there no approach supported by Micronaut? i.e. I'd need to have the user configure the paths or have the build determine these?

Not via annotation processing no. The issue is the annotation processor cannot know when those resources change without doing a clean recompile. It is better to do this kind of work as part of the build generally.

I know these are quite a few more questions. I'd be happy to hear from you.