ILC should run after ComputeResolvedFilesToPublishList

[sbomer]

Native AOT hooks into the publish logic using BeforeTargets="ComputeResolvedFilesToPublishList":

runtime/src/coreclr/nativeaot/BuildIntegration/Microsoft.NETCore.Native.Publish.targets

Lines 20 to 22 in 44b7ba3

	<Target Name="ComputeLinkedFilesToPublish"
	BeforeTargets="ComputeResolvedFilesToPublishList"
	DependsOnTargets="_ComputeAssembliesToCompileToNative;LinkNative">

However, this target has some conflict resolution that should be done before ILC runs:
https://github.com/dotnet/sdk/blob/456aa42f693f9ff022c7e5f9f7473e0dc5d3b2f4/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Publish.targets#L542-L559

I noticed this in a local build where ILLink was picking a different version of an assembly than ILC (because ILLink runs after ComputeResolvedFilesToPublishList:
https://github.com/dotnet/sdk/blob/456aa42f693f9ff022c7e5f9f7473e0dc5d3b2f4/src/Tasks/Microsoft.NET.Build.Tasks/targets/Microsoft.NET.Publish.targets#L453-L456

The ilc.rsp file in my scenario has both:

-r:C:\Users\svbomer\.nuget\packages\microsoft.azure.amqp\2.6.7\lib\netstandard2.0\Microsoft.Azure.Amqp.dll
-r:D:\azure-amqp\bin\Release\Microsoft.Azure.Amqp\net9.0\Microsoft.Azure.Amqp.dll

ILC seems to use the first dll passed on the command-line.

Compare this with the PublishSelfContained output. During ComputeResolvedFilesToPublishList, the conflict is resolved and only the dll from the nuget package is removed, leaving the one from a local build. The local build is passed to ILLink.

Unfortunately I wasn't able to come up with a simple repro, but the project file looked like this:

<Project Sdk="Microsoft.NET.Sdk">
  <PropertyGroup>
    <OutputType>Exe</OutputType>
    <TargetFramework>net9.0</TargetFramework>
    <!-- <PublishTrimmed>true</PublishTrimmed>
    <PublishAot>true</PublishAot> -->
    <PublishSelfContained>true</PublishSelfContained>
    <TrimmerSingleWarn>false</TrimmerSingleWarn>
    <IsTestSupportProject>true</IsTestSupportProject>
    <_TrimmerDumpDependencies>true</_TrimmerDumpDependencies>
  </PropertyGroup>
  <ItemGroup>
    <ProjectReference Include="..\..\..\..\sdk\eventhub\Azure.Messaging.EventHubs.Processor\src\Azure.Messaging.EventHubs.Processor.csproj" />
    <ProjectReference Include="..\..\..\..\sdk\eventhub\Azure.Messaging.EventHubs\src\Azure.Messaging.EventHubs.csproj" />
    <ProjectReference Include="D:\azure-amqp\src\Microsoft.Azure.Amqp.csproj" />
    <TrimmerRootAssembly Include="Azure.Messaging.EventHubs.Processor" />
  </ItemGroup>
  <ItemGroup>
    <!-- Update this dependency to its latest, which has all the annotations -->
    <PackageReference Include="Microsoft.Extensions.Logging.Configuration" Version="8.0.0" />
  </ItemGroup>
</Project>

Binlogs available upon request.

[MichalStrehovsky]

We should ideally redo how the compiler is hooked up into publish in general. The way it's hooked up right now was inherited from CoreRT and never touched since. SDK is now allowed to know about PublishAot and we should use same mechanisms as PublishTrimmed and PublishSingleFile use (that the SDK knows about and special cases). It would fix other problems, like dotnet/sdk#40488.

[agocke]

@sbomer Could you put this on your backlog? I think it's time to cleanup publishing for AOT and get this into a good state.

[jkotas]

I think it's time to cleanup publishing for AOT and get this into a good state.

For a servicing fix that addresses #109872, we may want to do a targeted change. E.g. take file versions into account in https://github.com/dotnet/runtime/blob/main/src/coreclr/tools/aot/ILCompiler.Build.Tasks/ComputeManagedAssembliesToCompileToNative.cs#L154 and let the file with higher version win.

[sbomer]

There are at least five places that can potentially do conflict resolution in a native AOT publish:

• GenerateBuildDependencyFile
• _HandlePackageFileConflicts
• _HandlePackageFileConflictsForPublish
• ILC's conflict resolution (ComputeManagedAssembliesToCompileToNative)
• ResolveOverlappingItemGroupConflicts (during ComputeResolvedFilesToPublishList)

The goal is for ILC's special handling to go away and to use the built-in conflict resolution. There are multiple issues here:

1. ILC hooks into publish in a way that bypasses ComputeResolvedFilesToPublishList (this issue). For this we need to change some of the build integration logic.
2. ILC doesn't get the built-in conflict resolution for runtime packs (IIRC this is done in GenerateBuildDependencyFile). For this we need to switch to using runtime packs. This is already supported, but we might want to make it the default.

[jamiewinder]

Sorry to ask, but where are things with this issue? All the issues raised around this are closed, and while there has since been an SDK update I don't think is yet resolved?

At the moment using AOT at all feels very precarious since these v9 dependencies can creep in as transitive dependencies very easily (as occurred with OpenTelemetry with 1.10.0 which first gave me the issue).

Would be great to get an update.

[jkotas]

This issue tracks cleaning up how native AOT publish is hooked up.

The NAOT dependency conflict resolution bug was tracked by #109872, and the fix was back ported to both .NET 8 and .NET 9 SDKs: #110059 and #110058 . If you still see a issues with NAOT dependency conflict resolution in the latest .NET SDK updates, please open a new issue on it.