Skip to content

Conversation

Copy link
Contributor

Copilot AI commented Dec 10, 2025

This PR adds tracking for source link files and resources to the compiler's deterministic key file to ensure all compilation inputs that affect reproducible builds are properly tracked. The implementation includes calculating checksums for source link content and manifest resources (tracking name, fileName, isPublic, and content checksum), while ruleset tracking was deemed unnecessary since rulesets only affect compilation through diagnostic options that are already being tracked.

Original prompt

This section details on the original issue you should resolve

<issue_title>Missing inputs in the key file</issue_title>
<issue_description>Recently discovered that there are several inputs we are not tracking in the key file that need to be tracked:

  • source link file
  • ruleset file
  • resources

</issue_description>

Comments on the Issue (you are @copilot in this section)


💡 You can make Copilot smarter by setting up custom instructions, customizing its development environment and configuring Model Context Protocol (MCP) servers. Learn more Copilot coding agent tips in the docs.

Copilot AI and others added 3 commits December 10, 2025 03:12
Co-authored-by: jcouv <12466233+jcouv@users.noreply.github.com>
Co-authored-by: jcouv <12466233+jcouv@users.noreply.github.com>
Co-authored-by: jcouv <12466233+jcouv@users.noreply.github.com>
Copilot AI changed the title [WIP] Add missing inputs to key file for tracking Track sourceLink, ruleset, and resources in deterministic key Dec 10, 2025
Copilot AI requested a review from jcouv December 10, 2025 03:38
@jcouv
Copy link
Member

jcouv commented Dec 10, 2025

@copilot CI produced src/Compilers/Core/Portable/Compilation/DeterministicKeyBuilder.cs(434,31): error IDE0060: (NETCORE_ENGINEERING_TELEMETRY=Build) Remove unused parameter 'cancellationToken'

Co-authored-by: jcouv <12466233+jcouv@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 10, 2025

@copilot CI produced src/Compilers/Core/Portable/Compilation/DeterministicKeyBuilder.cs(434,31): error IDE0060: (NETCORE_ENGINEERING_TELEMETRY=Build) Remove unused parameter 'cancellationToken'

Fixed in 08c5f38. Removed the unused cancellationToken parameter from the WriteEmitOptions method signature and its call site.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • sc4vsblobprodcus331.vsblob.vsassets.io
    • Triggering command: /usr/share/dotnet/dotnet dotnet build src/Compilers/Core/Portable/Microsoft.CodeAnalysis.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@jcouv
Copy link
Member

jcouv commented Dec 10, 2025

@copilot GetKey(...) already handles default inputs (additionalTexts, analyzers, generators, resources, ...). There is no need for the caller of GetKey to also do NullToEmpty() on those.

Co-authored-by: jcouv <12466233+jcouv@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 10, 2025

@copilot GetKey(...) already handles default inputs (additionalTexts, analyzers, generators, resources, ...). There is no need for the caller of GetKey to also do NullToEmpty() on those.

Fixed in e4530d3. Removed the redundant NullToEmpty() calls from the caller since GetKey() already handles default inputs internally.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • sc4vsblobprodcus331.vsblob.vsassets.io
    • Triggering command: /usr/share/dotnet/dotnet dotnet build src/Compilers/Core/Portable/Microsoft.CodeAnalysis.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@jcouv
Copy link
Member

jcouv commented Dec 10, 2025

@copilot We should care about the ruleset contents, not just the path to the ruleset file

Co-authored-by: jcouv <12466233+jcouv@users.noreply.github.com>
Copy link
Contributor Author

Copilot AI commented Dec 10, 2025

@copilot We should care about the ruleset contents, not just the path to the ruleset file

Fixed in 2fac371. Changed from tracking ruleset file path to tracking ruleset content with checksum (similar to source link). The ruleset file is now opened and read as SourceText, with its checksum written to compilation.options.ruleSet.

Warning

Firewall rules blocked me from connecting to one or more addresses (expand for details)

I tried to connect to the following addresses, but was blocked by firewall rules:

  • sc4vsblobprodcus331.vsblob.vsassets.io
    • Triggering command: /usr/share/dotnet/dotnet dotnet build src/Compilers/Core/Portable/Microsoft.CodeAnalysis.csproj (dns block)

If you need me to access, download, or install something from one of these locations, you can either:

@jcouv
Copy link
Member

jcouv commented Dec 10, 2025

@copilot Remove IsEmbedded. It is redundant with FileName and we need FileName for non-embedded scenarios

Co-authored-by: jcouv <12466233+jcouv@users.noreply.github.com>
@jcouv jcouv added the Concept-Determinism The issue involves our ability to support determinism in binaries and PDBs created at build time. label Dec 10, 2025
Comment on lines 225 to 227
using var stream = resource.DataProvider();
var sourceText = SourceText.From(stream, encoding: null, checksumAlgorithm: SourceHashAlgorithms.Default);
WriteSourceText(writer, sourceText);
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why are you converting to a SourceText here? These resources are generally treated as binary blobs in the codebase. My intuition would be that we just grab the raw Stream or byte[] and hash that. Not sure if we'd hit weird edge cases trying to view a binary as a source text.

@jcouv jcouv requested a review from jaredpar December 17, 2025 07:13
@jcouv jcouv marked this pull request as draft December 18, 2025 23:40
@jcouv
Copy link
Member

jcouv commented Dec 19, 2025

I'm investigating the determinism failure in CI with Microsoft.VisualStudio.Extensibility.Testing.Xunit: Resolved by adding the key file to the skip list (thanks Jared for the tip).

...
  "resources": [
    {
      "resourceName": "Microsoft.VisualStudio.IntegrationTestService.dll",
      "fileName": null,
      "isPublic": true,
      "content": {
        "checksum": "da3a9f2c2fc2de2bf0254f9f13e15726c7a260ab318acff9a77790ccc344a",
        "checksumAlgorithm": "Sha256"
      }
    },
    {
      "resourceName": "Microsoft.VisualStudio.IntegrationTestService.vsix",
      "fileName": null,
      "isPublic": true,
      "content": {
+      "checksum": "c88fbb3f45e9c061775c57b763ba4e3988857883a253d03ac7bed44bf113af9c", // Right
-      "checksum": "876736d2ee98f2c4cac77a7fac592adbd14924d583f914cf873dc67366c2f8c", // Left
        "checksumAlgorithm": "Sha256"
      }
    }
  ]
}

@jcouv jcouv marked this pull request as ready for review January 13, 2026 05:23
@jcouv jcouv requested a review from a team as a code owner January 13, 2026 05:23
@jcouv jcouv requested a review from a team January 13, 2026 05:24
# The VSIX build is not deterministic (for example the catalog.json contains a randomly generated identifier for the install location).
"Microsoft.VisualStudio.Extensibility.Testing.Xunit.dll"
"Microsoft.VisualStudio.Extensibility.Testing.Xunit.dll",
"Microsoft.VisualStudio.Extensibility.Testing.Xunit.dll.key"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

.key

Is this addition intentional?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes, the key file didn't use to include some inputs, which are not the same across rebuilds (ie. a VSIX file). Now the key file includes them and thus varies across rebuilds as well.

Copy link
Contributor

@AlekseyTs AlekseyTs left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM (commit 26)

@jcouv jcouv merged commit 80e4b60 into main Jan 13, 2026
29 checks passed
@jcouv jcouv deleted the copilot/add-missing-inputs-key-file branch January 13, 2026 18:35
@dotnet-policy-service dotnet-policy-service bot added this to the Next milestone Jan 13, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Area-Compilers Concept-Determinism The issue involves our ability to support determinism in binaries and PDBs created at build time.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Missing inputs in the key file

5 participants