[.NET 8 Preview 7] dotnet msbuild fails when using local copied dotnet not on the PATH

[mrward]

Version Used:

Microsoft (R) Visual C# Compiler version 4.8.0-1.23361.12 (86156fa)

.NET SDK:
Version: 8.0.100-preview.7.23362.1
Commit: cb8652231d

Steps to Reproduce:

Following steps are for macOS but I believe this would also repro on Windows.

1. Install .NET 7.0 SDK using pkg - https://dotnet.microsoft.com/en-us/download/dotnet/7.0
2. Download .tar.gz of .NET 8 preview 7 and extract that to a separate folder
   a. Download https://dotnetbuilds.azureedge.net/public/Sdk/8.0.100-preview.7.23362.1/dotnet-sdk-8.0.100-preview.7.23362.1-osx-x86.tar.gz
   b. mkdir .dotnet
   c. /usr/bin/tar -C .dotnet -xf dotnet-sdk-8.0.100-preview.7.23362.1-osx-x64.tar.gz
3. Use .NET 8 CLI to create a new console project
   a. /full/path/to/.dotnet/dotnet new console -o test
4. Build the new console project.
   a. cd test
   b. /full/path/to/.dotnet/.dotnet/dotnet msbuild

Diagnostic Id:

If this is a report about a bug in an analyzer, please include the diagnostic if possible (e.g. "IDE0030").

Expected Behavior:

Console project builds successfully.
dotnet from the local copied directory is used by msbuild.

Actual Behavior:

Build fails:

.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : You must install or update .NET to run this application. [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :  [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : App: .dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/bincore/csc.dll [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : Architecture: x64 [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : Framework: 'Microsoft.NETCore.App', version '8.0.0-preview.7.23361.9' (x64) [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : .NET location: /usr/local/share/dotnet/ [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :  [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : The following frameworks were found: [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :   7.0.9 at [/usr/local/share/dotnet/shared/Microsoft.NETCore.App] [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :  [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : Learn about framework resolution: [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : https://aka.ms/dotnet/app-launch-failed [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error :  [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : To install missing framework, download: [~/Projects/Tests/dotnet8/test/test.csproj]
.dotnet/sdk/8.0.100-preview.7.23362.1/Roslyn/Microsoft.CSharp.Core.targets(80,5): error : https://aka.ms/dotnet-core-applaunch?framework=Microsoft.NETCore.App&framework_version=8.0.0-preview.7.23361.9&arch=x64&rid=osx.12-x64 [~/Projects/Tests/dotnet8/test/test.csproj]

Build works with .NET 8 preview 6 SDK.

Other workarounds:

1. Set the DOTNET_ROOT environment variable to point to dotnet CLI from .NET 8 install
2. Add .NET 8 install path as first part of the PATH environment variable
3. Remove /usr/local/share/dotnet from the PATH environment variable

My guess is that this may be due to this change:

854fa07

Various parts of dotnet/sdk seem to set the DOTNET_HOST_PATH environment variable, and this is no longer being used.

Also some discussion here about the DOTNET_HOST_PATH environment variable - dotnet/runtime#88754 (comment)

[jaredpar]

At the moment this behavior is "By Design". The compiler will use the dotnet which is available on $PATH.

Various parts of dotnet/sdk seem to set the DOTNET_HOST_PATH environment variable, and this is no longer being used.

That is an implementation detail of our internal architecture that leaked into the shipping product. Very confident our conclusion is going to be that this needs to be removed from the product.

If you feel a better appproach is in order here please put the discussion on dotnet/runtime#88754. That is the design we are following here.