diff --git a/.vscode/settings.json b/.vscode/settings.json new file mode 100644 index 000000000..7f06c1459 --- /dev/null +++ b/.vscode/settings.json @@ -0,0 +1,7 @@ +{ + "[markdown]": { + "editor.rulers": [80], + "editor.wordWrap": "bounded", + "editor.wordWrapColumn": 80 + }, +} diff --git a/INDEX.md b/INDEX.md index ff720741c..8747cf955 100644 --- a/INDEX.md +++ b/INDEX.md @@ -96,6 +96,7 @@ Use update-index to regenerate it: |Year|Title|Owners| |----|-----|------| +| | [Provide SDK hint paths in global.json](proposed/local-sdk-global-json.md) | | | | [Rate limits](proposed/rate-limit.md) | [John Luo](https://github.com/juntaoluo), [Sourabh Shirhatti](https://github.com/shirhatti) | | | [Readonly references in C# and IL verification.](proposed/verifiable-ref-readonly.md) | | | | [Ref returns in C# and IL verification.](proposed/verifiable-ref-returns.md) | | diff --git a/proposed/local-sdk-global-json.md b/proposed/local-sdk-global-json.md new file mode 100644 index 000000000..2523a2c4f --- /dev/null +++ b/proposed/local-sdk-global-json.md @@ -0,0 +1,235 @@ +# Provide SDK hint paths in global.json + +## Summary + +This proposal adds two new properties to the `sdk` object in +[global.json][global-json-schema] + +```json +{ + "sdk": { + "paths": [ ".dotnet", "$host$" ], + "errorMessage": "The .NET SDK could not be found, please run ./install.sh." + } +} +``` + +These properties will be considered by the resolver host during .NET SDK +resolution. The `paths` property lists the locations that the resolver should +consider when attempting to locate a compatible .NET SDK. The `errorMessage` +property controls what the resolver displays when it cannot find a compatible +.NET SDK. + +This particular configuration would cause the local directory `.dotnet` to be +considered _in addition_ to the current set of locations. Further if resolution +failed the resolver would display the contents of `errorMessage` instead of +the default error message. + +## Motivation + +There is currently a disconnect between the ways the .NET SDK is deployed in +practice and what the host resolver can discover when searching for compatible +SDKs. By default the host resolver is only going to search for SDKs next to +the running `dotnet`. This often means machine-wide locations, since users +and tools typically rely on `dotnet` already being on the user's path when +launching, instead of specifying a full path to the executable. The .NET SDK +though is commonly deployed to local locations: `%LocalAppData%\Microsoft\dotnet`, +`$HOME/.dotnet`. Many repos embrace this and restore the correct .NET for their +builds into a local `.dotnet` directory. + +The behavior of the host resolver is incompatible with local based deployments. +It will not find these deployments without additional environment variable +configuration and only search next to the running `dotnet`. That means tools +like Visual Studio and VS Code simply do not work with local deployment by +default. Developers must take additional steps like manipulating `%PATH%` before +launching these editors. That reduces the usefulness of items like the quick +launch bar, short cuts, etc. + +This is further complicated when developers mix local and machine wide +installations. The host resolver will find the first `dotnet` according to its +lookup rules and search only there for a compatible SDK. Once developers +manipulate `%PATH%` to prefer local SDKS the resolver will stop considering +machine wide SDKS. That can lead to situations where there is machine wide SDK +that works for a given global.json but the host resolver will not consider it +because the developer setup `%PATH%` to consider a locally installed SDK. That +can be very frustrating for end users. + +This disconnect between the resolver and deployment has lead to customers +introducing a number of creative work arounds: + +- [scripts][example-scripts-razor] to launch VS Code while considering locally +deployed .NET SDKs +- [docs and scripts][example-scripts-build] to setup the environment and launch +VS so it can find the deployed .NET SDKs. +- [scripts][example-scripts-dotnet] that wrap `dotnet` to find the _correct_ +`dotnet` to use during build. + +These scripts are not one offs, they are increasingly common items in repos in +`github.com/dotnet` to attempt to fix the disconnect. Even so many of these +solutions are incomplete because they themselves only consider local deployment. +They don't fully support the full set of ways the SDK can be deployed. + +This problem also manifests in how customers naturally want to use our +development tools like Visual Studio or VS Code. It's felt sharply on the .NET +team, or any external customer who wants to contribute to .NET, due to how +.NET Arcade infrastructure uses xcopy deployment into `.dotnet`. External teams +like Unity also feel this pain in their development: + +- This [issue][cases-sdk-issue] from 2017 attempting +to solve this problem. It gets several hits a year from customers who are +similarly struggling with our toolings inability to handle local deployment. +- This [internal discussion][cases-internal-discussion] from a C# team member. +They wanted to use VS as the product is shipped to customers and got blocked +when we shipped an SDK that didn't have a corresponding MSI and hence VS +couldn't load Roslyn anymore. +- [VS Code][cases-vscode] having to adjust to consider local directories for SDK +because our resolver can't find them. + +## Detailed Design + +The global.json file will support two new properties under the `sdk` object: + +- `"paths"`: this is a list of paths that the host resolver should +consider when looking for compatible SDKs. In the case this property is `null` +or not specified, the host resolver will behave as it does today. +- `"errorMessage"`: when the host resolver cannot find a compatible .NET SDK it +will display the contents of this property instead of the default error message. +In the case this property is `null` or not specified, the default error message +will be displayed. + +The values in the `paths` property can be a relative path, absolute path or +`$host$`. When a relative path is used it will be resolved relative to the +location of the containing global.json. The value `$host$` is a special value +that represents the machine wide installation path of .NET SDK for the +[current host][installation-doc]. + +The values in `paths` are considered in the order they are defined. The host +resolver will stop when it finds the first path with a compatible .NET SDK. +For example: + +```json +{ + "sdk": { + "paths": [ ".dotnet", "$host$" ], + } +} +``` + +In this configuration the host resolver would find a compatible .NET SDK, if it +exists in `.dotnet` or a machine wide location. + +This lookup will stop on the first match which means it won't necessarily find +the best match. Consider a scenario with a global.json that has: + +```json +{ + "sdk": { + "paths": [ ".dotnet", "$host$" ], + "version": "7.0.200", + "rollForward": "latestFeature" + } +} +``` + +In a scenario where the `.dotnet` directory had 7.0.200 SDK but there was a +machine wide install of 7.0.300 SDK, the host resolver would pick 7.0.200 out +of `.dotnet`. That location is considered first, it has a matching .NET SDK and +hence discovery stops there. + +This design requires us to only change the host resolver. That means other +tooling like Visual Studio, VS Code, MSBuild, etc ... would transparently +benefit from this change. Repositories could update global.json to have +`paths` support `.dotnet` and Visual Studio would automatically find it without +any design changes. + +## Considerations + +### Installation Points + +One item to keep in mind when considering this area is the .NET SDK can be +installed in many locations. The most common are: + +- Machine wide +- User wide: `%LocalAppData%\Microsoft\dotnet` on Windows and `$HOME/.dotnet` +on Linux/macOS. +- Repo: `.dotnet` + +Our installation tooling tends to avoid redundant installations. For example, if +restoring a repository that requires 7.0.400, the tooling will not install it +locally if 7.0.400 is installed machine wide. It also will not necessarily +delete the local `.dotnet` folder or the user wide folder. That means developers +end up with .NET SDK installs in all three locations but only the machine wide +install has the correct ones. + +As a result solutions like "just use .dotnet, if it exists" fall short. It will +work in a lot of cases but will fail in more complex scenarios. To completely +close the disconnect here we need to consider all the possible locations. + +### Best match or first match? + +This proposal is designed at giving global.json more control over how SDKs are +found. If the global.json asked for a specific path to be considered and it has +a matching SDK but a different SDK was chosen, that seems counter intuitive. +Even in the case where the chosen SDK was _better_. This is a motivating +scenario for CI where certainty around SDK is often more desirable than +_better_. This is why the host discovery stops at first match vs. looking at +all location and choosing the best match. + +Best match is a valid approach though. Can certainly see the argument for some +customers wanting that. Feel like it cuts against the proposal a bit because it +devalues `paths` a bit. If the resolver is switched to best match then, the need +for configuration around best versus first match is much stronger. There would +certainly be a customer segment that wanted to isolate from machine state in +that case. + +### dotnet exec + +This proposal only impacts how .NET SDK commands do runtime discovery. The +command `dotnet exec` is not an .NET SDK command but instead a way to invoke +the app directly using the runtime installed with `dotnet`. + +It is reasonable for complex builds to build and use small tools. For example +building a tool for linting the build, running complex validation, etc ... To +work with local SDK discovery these builds need to leverage `dotnet run` to +execute such tools instead of `dotnet exec`. + +```cmd +# Avoid +> dotnet exec artifacts/bin/MyTool/Release/net8.0/MyTool.dll +# Prefer +> dotnet run --no-build --framework net7.0 src/Tools/MyTool/MyTool.csproj +``` + +### Environment variables + +Previous versions of this proposal included support for using environment +variables inside `paths`. This was removed due to lack of motivating +scenarios and potential for creating user confusion as different machines can +reasonably have different environment variables. + +This could be reconsidered if motivating scenarios are found. + +### Other Designs + +[This is a proposal][designs-other] similar in nature to this one. There are a +few differences: + +1. This proposal is more configurable and supports all standard local +installation points, not just the `.dotnet` variant. +2. This proposal doesn't change what SDK is chosen: the rules for global.json +on what SDKs are allowed still apply. It simply changes the locations where the +SDK is looked for. +3. No consideration for changing the command line. This is completely driven +through global.json changes. + +Otherwise the proposals are very similar in nature. + +[global-json-schema]: https://learn.microsoft.com/en-us/dotnet/core/tools/global-json#globaljson-schema +[example-scripts-razor]: https://github.com/dotnet/razor/pull/9550 +[example-scripts-build]: https://github.com/dotnet/sdk/blob/518c60dbe98b51193b3a9ad9fc44e055e6e10fa0/documentation/project-docs/developer-guide.md?plain=1#L38 +[example-scripts-dotnet]: https://github.com/dotnet/runtime/blob/main/dotnet.cmd +[cases-sdk-issue]: https://github.com/dotnet/sdk/issues/8254 +[cases-internal-discussion]: https://teams.microsoft.com/l/message/19:ed7a508bf00c4b088a7760359f0d0308@thread.skype/1698341652961?tenantId=72f988bf-86f1-41af-91ab-2d7cd011db47&groupId=4ba7372f-2799-4677-89f0-7a1aaea3706c&parentMessageId=1698341652961&teamName=.NET%20Developer%20Experience&channelName=InfraSwat&createdTime=1698341652961 +[cases-vscode]: https://github.com/dotnet/vscode-csharp/issues/6471 +[designs-other]: https://github.com/dotnet/designs/blob/main/accepted/2022/version-selection.md#local-dotnet +[installation-doc]: https://github.com/dotnet/designs/blob/main/accepted/2021/install-location-per-architecture.md