Unable to connect to testing platform runner process when using testing platform server mode #4729

xperiandri · 2025-01-20T01:25:30Z

Describe the bug

When this flag is enable

I see

Log level is set to Informational (Default).
Multiple source code repositories are available, using C:\Users\Andrii\Dev\eCierge\PropertyManagement as the repository root.
Connected to test environment '< Local Windows Environment >'
Test data store opened in 0,060 sec.
========== Starting test discovery ==========
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.Identity\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.Identity.dll'.
Aborting test discovery because a build was performed.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Administration\Tests.Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Administration.GraphQL.Tests.Integration.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Construction\Tests.Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Construction.GraphQL.Tests.Integration.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.AzureCosmos\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.AzureCosmos.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Tests.Integration.GraphQL.dll'.
========== Test discovery aborted: 0 Tests found in 3,4 sec ==========
========== Starting test discovery ==========
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Administration\Tests.Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Administration.GraphQL.Tests.Integration.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Construction\Tests.Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Construction.GraphQL.Tests.Integration.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.Identity\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.Identity.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Tests.Integration.GraphQL.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.AzureCosmos\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.AzureCosmos.dll'.
No test is available in C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.Models.Hedgehog\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.Models.Hedgehog.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
No test is available in C:\Users\Andrii\Dev\eCierge\PropertyManagement\Shared.Tests\Models.Hedgehog\bin\Debug\net8.0\Ecierge.Tests.Models.Hedgehog.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
========== Test discovery aborted: 43 Tests found in 9,2 sec ==========
Resuming previously aborted test discovery
========== Starting test discovery ==========
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.Identity\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.Identity.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.AzureCosmos\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.AzureCosmos.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Tests.Integration.GraphQL.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Administration\Tests.Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Administration.GraphQL.Tests.Integration.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Construction\Tests.Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Construction.GraphQL.Tests.Integration.dll'.
========== Test discovery aborted: 0 Tests found in 3,9 sec ==========
========== Starting test discovery ==========
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Construction\Tests.Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Construction.GraphQL.Tests.Integration.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Administration\Tests.Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Administration.GraphQL.Tests.Integration.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.AzureCosmos\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.AzureCosmos.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.Identity\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.Identity.dll'.
Unable to connect to testing platform runner process 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Integration.GraphQL\bin\Debug\net8.0\Ecierge.Server.Tests.Integration.GraphQL.dll'.
========== Test discovery aborted: 0 Tests found in 4,1 sec ==========

Steps To Reproduce

I don't know. How can I reproduce that?

Expected behavior

Tests are discovered

Actual behavior

Tests are not discovered

Additional context

Both
Microsoft Visual Studio Enterprise 2022 (64-bit) Version 17.12.4
Microsoft Visual Studio Enterprise 2022 (64-bit) Version 17.13.0 Preview 2.1
affected

<PackageVersion Include="Microsoft.TestPlatform.TestHost" Version="17.10.0" />
<PackageVersion Include="Microsoft.NET.Test.Sdk" Version="17.12.0" />
<PackageVersion Include="MSTest.TestAdapter" Version="3.7.1" />
<PackageVersion Include="MSTest.TestFramework" Version="3.7.1" />

also I see

[MSTest][Discovery][C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Integration\bin\Debug\net8.0\Ecierge.Server.Tests.Integration.dll] TestClass attribute defined on generic non-abstract class Ecierge.Server.Tests.Integration.IntegrationTestBase`1
No test is available in C:\Users\Andrii\Dev\eCierge\PropertyManagement\Server.Tests\Persistence.Models.Hedgehog\bin\Debug\net8.0\Ecierge.Server.Tests.Persistence.Models.Hedgehog.dll. Make sure that test discoverer & executors are registered and platform & framework version settings are appropriate and try again.
Error:
  An assembly specified in the application dependencies manifest (Ecierge.Testing.deps.json) was not found:
    package: 'BlackFox.MasterOfFoo', version: '2.1.0'
    path: 'lib/net5.0/BlackFox.MasterOfFoo.dll'
Microsoft.VisualStudio.TestPlatform.ObjectModel.TestPlatformException: Testhost process for source(s) 'C:\Users\Andrii\Dev\eCierge\PropertyManagement\Testing\bin\Debug\net8.0\Ecierge.Testing.dll' exited with error: Error:
  An assembly specified in the application dependencies manifest (Ecierge.Testing.deps.json) was not found:
    package: 'BlackFox.MasterOfFoo', version: '2.1.0'
    path: 'lib/net5.0/BlackFox.MasterOfFoo.dll'
. Please check the diagnostic logs for more information.
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.ThrowOnTestHostExited(IEnumerable`1 sources, Boolean testHostExited) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Client/ProxyOperationManager.cs:line 524
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyOperationManager.SetupChannel(IEnumerable`1 sources, String runSettings) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Client/ProxyOperationManager.cs:line 293
   at Microsoft.VisualStudio.TestPlatform.CrossPlatEngine.Client.ProxyDiscoveryManager.InitializeDiscovery(DiscoveryCriteria discoveryCriteria, ITestDiscoveryEventsHandler2 eventHandler, Boolean skipDefaultAdapters) in /_/src/Microsoft.TestPlatform.CrossPlatEngine/Client/ProxyDiscoveryManager.cs:line 147

AB#2353765

The text was updated successfully, but these errors were encountered:

Evangelink · 2025-01-20T05:39:56Z

Are you using VSTest or Microsoft.Testing.Platform? Have you enabled EnableMSTestRunner property? If so, can you try to DotNet run or F5 the test project to see if it works?

If you have then you don't need the Test host package.

Joy-less · 2025-01-21T18:51:31Z

Can confirm that this issue prevents testing with MSTest (freezes indefinitely) and can be fixed by disabling the Use testing platform server mode flag.

EvgenyMuryshkin · 2025-01-21T22:17:30Z

Without server mode, MSTest discovery fails with

Could not load file or assembly 'Microsoft.VisualStudio.TestPlatform.TestFramework.Extensions, Version=14.0.0.0, Culture=neutral, PublicKeyToken=b03f5f7f11d50a3a'. The system cannot find the file specified.

xperiandri · 2025-01-22T01:14:50Z

If so, can you try to DotNet run or F5 the test project to see if it works?

It works

xperiandri · 2025-01-22T01:15:02Z

Have you enabled EnableMSTestRunner property?

Yes

xperiandri · 2025-01-22T01:16:38Z

Without server mode, MSTest discovery fails

@EvgenyMuryshkin Probably SDK package is too new for your Visual Studio

Evangelink · 2025-01-22T08:18:43Z

@xperiandri is dropping <PackageVersion Include="Microsoft.TestPlatform.TestHost" Version="17.10.0" /> helping? This package is VSTest oriented and should not be mixed with MSTest runner.

@EvgenyMuryshkin Can you provide more info and maybe open a different issue as it seems unrelated to this problem.

EvgenyMuryshkin · 2025-01-22T22:28:49Z

@Evangelink

I am not sure how to describe the issue. Most of the time it is boiled down to

pending Windows updates
something is screwed in PC, need to restart
various versions of test packages across different projects (even those that are not related to the current test run)
mixture of MSTest and NUnit - sometimes need to add refs to MSTest.TestFramework, sometimes remove them (depending on the version of VS)
need to delete bin, obj, and do full rebuild manually.

In general, it is 2-3 hours of random walking and googling until it runs.

I need to pin it to something more specific before creating an issue, otherwise, it will need more info and then be closed eventually.

Thanks,
Regards,
Evgeny

drognanar · 2025-01-23T13:22:49Z

@xperiandri can you enable diagnostic logging (tools -> options -> test -> logging level)? the output window should include the full command that's executed by the test explorer (Starting test runner 'C:\Program Files\dotnet\dotnet.exe [test_project_binary_path.dll] --server --diagnostic --diagnostic-verbosity Trace --diagnostic-output-directory "[path to diagnostic directory]" --results-directory "[path to results directory]" --client-port [temporary client port]').

could you run that command from command line? it should fail with a SocketException when trying to connect to the client port (because client port is only created for the duration of the test explorer run). the error message suggests that the application crashes sooner than that and it would be good to see the failing stack trace

thomhurst · 2025-01-23T15:03:20Z

@xperiandri is dropping <PackageVersion Include="Microsoft.TestPlatform.TestHost" Version="17.10.0" /> helping? This package is VSTest oriented and should not be mixed with MSTest runner.

@EvgenyMuryshkin Can you provide more info and maybe open a different issue as it seems unrelated to this problem.

The same with Microsoft.NET.Test.Sdk too - That sets some VSTest stuff doesn't it.

EvgenyMuryshkin · 2025-01-23T21:10:48Z

Test output window

Log level is set to Informational (Default).
Connected to test environment '< Local Windows Environment >'
Test data store opened in 0.127 sec.
Could not determine target device configuration. Exception: System.Runtime.InteropServices.COMException (0x80020009): The method or operation is not implemented.
   at EnvDTE.Property.get_Value()
   at Microsoft.VisualStudio.TestWindow.ShellServices.ProjectDataImpl.<GetRemoteMachineAddressAsync>d__36.MoveNext()

Evangelink · 2025-01-24T17:23:21Z

Test output window

Log level is set to Informational (Default).
Connected to test environment '< Local Windows Environment >'
Test data store opened in 0.127 sec.
Could not determine target device configuration. Exception: System.Runtime.InteropServices.COMException (0x80020009): The method or operation is not implemented.
   at EnvDTE.Property.get_Value()
   at Microsoft.VisualStudio.TestWindow.ShellServices.ProjectDataImpl.<GetRemoteMachineAddressAsync>d__36.MoveNext()

@EvgenyMuryshkin This is sadly a known Test Explorer behavior but the VS team is not fixing it https://developercommunity.visualstudio.com/t/Warning-Could-not-determine-target-dev/10411558. There is nothing I can do and it's not linked to MSTest so please feel free to comment on the community issue linked above.

Evangelink · 2025-01-24T17:24:01Z

The same with Microsoft.NET.Test.Sdk too - That sets some VSTest stuff doesn't it.

As per my tests this package is not having impact. @thomhurst do you have a scenario where it's causing an issue?

thomhurst · 2025-01-24T17:34:44Z

The same with Microsoft.NET.Test.Sdk too - That sets some VSTest stuff doesn't it.

As per my tests this package is not having impact. @thomhurst do you have a scenario where it's causing an issue?

It definitely breaks TUnit.

Code:

public class Tests
{
    [Test]
    public void Test1()
    {
    }
}

Working csproj

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>net8.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
    </PropertyGroup>

    <ItemGroup>
      <PackageReference Include="TUnit" Version="0.7.3" />
    </ItemGroup>

</Project>

Broken csproj

<Project Sdk="Microsoft.NET.Sdk">

    <PropertyGroup>
        <OutputType>Exe</OutputType>
        <TargetFramework>net8.0</TargetFramework>
        <ImplicitUsings>enable</ImplicitUsings>
        <Nullable>enable</Nullable>
    </PropertyGroup>

    <ItemGroup>
      <PackageReference Include="Microsoft.NET.Test.Sdk" Version="17.12.0" />
      <PackageReference Include="TUnit" Version="0.7.3" />
    </ItemGroup>

</Project>

Try a simple dotnet run before and after.

drognanar · 2025-01-30T12:18:06Z

@Evangelink it seems that Microsoft.NET.Test.Sdk and TUnit are fighting over the generated entry point Main method. setting the property <GenerateProgramFile>false</GenerateProgramFile> fixes the issue

drognanar · 2025-01-30T12:20:31Z

the other issues might be different, which is why knowing the tfm of the failing projects and trying to run the full command on command line would help in understanding what is the exception that causes the process to be unable to start up

Evangelink · 2025-01-30T12:44:35Z

@Evangelink it seems that Microsoft.NET.Test.Sdk and TUnit are fighting over the generated entry point Main method. setting the property <GenerateProgramFile>false</GenerateProgramFile> fixes the issue

@thomhurst We do disable VSTest main when we enable support for MTP. We will let you decide whether you want to disable it preventively if people are using Microsoft.NET.Test.Sdk or if you want to keep the build failure. This is definitely not a wrong behavior of MTP.

thomhurst · 2025-01-30T12:55:19Z

How would I make TUnit force disabling VSTest?

Evangelink · 2025-02-06T19:22:39Z

You want to add the following to your prop file <GenerateProgramFile>false</GenerateProgramFile>

thomhurst · 2025-02-06T19:24:55Z

You want to add the following to your prop file <GenerateProgramFile>false</GenerateProgramFile>

Why doesn't the use test platform set that out of curiosity? They'd never be compatible?

ruzena-kavanova · 2025-02-10T11:34:52Z

I have XUnit and MSTest in my test project and I failed to run the tests with "Unable to connect to testing platform.." error. Unsetting the flag "Use testing platform server mode" in Preview Features helped to solve the problem. Thank you guys for usefull discussion

Evangelink · 2025-02-10T12:25:51Z

Why doesn't the use test platform set that out of curiosity?

Why should the platform be aware of the other platform? ;)

I have XUnit and MSTest in my test project

You cannot mix test frameworks for a given project

EvgenyMuryshkin · 2025-02-10T20:42:26Z

@Evangelink

You cannot mix test frameworks for a given project

Occasionally, if there is no reference to MSTest.TestAdapter and MSTest.TestFramework, then (XUnit or NUnit) tests are not discovered.
So have to have both libs added.

This situation changes from VS build to build.
Update VS and then tests are not discovered or not running.
Then have to either add or remove ref to MSTest.TestAdapter.

I will create a new issue once this happens again with VS versions.

Evangelink · 2025-02-11T08:38:15Z

Occasionally, if there is no reference to MSTest.TestAdapter and MSTest.TestFramework, then (XUnit or NUnit) tests are not discovered.
So have to have both libs added.

If you can create and share a reproducer that would be great because there should be no reason to have a framework impacting another one. Especially for MTP.

jimiscott · 2025-02-11T10:48:09Z

You cannot mix test frameworks for a given project

Ha? You can mix test frameworks - we have numerous projects mixing MSTest and Xunit which up until this issue were working harmoniously without issue - Like @ruzena-kavanova unchecking Use testing platform server mode resolved our issue.

Evangelink · 2025-02-11T12:19:43Z

Ha? You can mix test frameworks

You can with VSTest, you cannot with MTP. If you do enable MTP support (property depends upon your test framework) you should see an error message explaining this is not supported.

microsoft-github-policy-service bot added the Needs: Triage 🔍 label Jan 20, 2025

Youssef1313 added Area: MTP Belongs to the Microsoft.Testing.Platform core library Area: Server Mode labels Jan 20, 2025

Evangelink added the External: Test Explorer label Jan 23, 2025

pavelhorak assigned drognanar Jan 27, 2025

pavelhorak added the sprint label Jan 27, 2025

Evangelink removed the Needs: Triage 🔍 label Jan 31, 2025

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Unable to connect to testing platform runner process when using testing platform server mode #4729

Unable to connect to testing platform runner process when using testing platform server mode #4729

xperiandri commented Jan 20, 2025 •

edited by pavelhorak

Loading

Evangelink commented Jan 20, 2025

Joy-less commented Jan 21, 2025 •

edited

Loading

EvgenyMuryshkin commented Jan 21, 2025

xperiandri commented Jan 22, 2025

xperiandri commented Jan 22, 2025

xperiandri commented Jan 22, 2025

Evangelink commented Jan 22, 2025

EvgenyMuryshkin commented Jan 22, 2025

drognanar commented Jan 23, 2025

thomhurst commented Jan 23, 2025

EvgenyMuryshkin commented Jan 23, 2025

Evangelink commented Jan 24, 2025

Evangelink commented Jan 24, 2025

thomhurst commented Jan 24, 2025

drognanar commented Jan 30, 2025

drognanar commented Jan 30, 2025

Evangelink commented Jan 30, 2025

thomhurst commented Jan 30, 2025

Evangelink commented Feb 6, 2025

thomhurst commented Feb 6, 2025

ruzena-kavanova commented Feb 10, 2025

Evangelink commented Feb 10, 2025

EvgenyMuryshkin commented Feb 10, 2025 •

edited

Loading

Evangelink commented Feb 11, 2025

jimiscott commented Feb 11, 2025 •

edited

Loading

Evangelink commented Feb 11, 2025

Unable to connect to testing platform runner process when using testing platform server mode #4729

Unable to connect to testing platform runner process when using testing platform server mode #4729

Comments

xperiandri commented Jan 20, 2025 • edited by pavelhorak Loading

Describe the bug

Steps To Reproduce

Expected behavior

Actual behavior

Additional context

Evangelink commented Jan 20, 2025

Joy-less commented Jan 21, 2025 • edited Loading

EvgenyMuryshkin commented Jan 21, 2025

xperiandri commented Jan 22, 2025

xperiandri commented Jan 22, 2025

xperiandri commented Jan 22, 2025

Evangelink commented Jan 22, 2025

EvgenyMuryshkin commented Jan 22, 2025

drognanar commented Jan 23, 2025

thomhurst commented Jan 23, 2025

EvgenyMuryshkin commented Jan 23, 2025

Evangelink commented Jan 24, 2025

Evangelink commented Jan 24, 2025

thomhurst commented Jan 24, 2025

drognanar commented Jan 30, 2025

drognanar commented Jan 30, 2025

Evangelink commented Jan 30, 2025

thomhurst commented Jan 30, 2025

Evangelink commented Feb 6, 2025

thomhurst commented Feb 6, 2025

ruzena-kavanova commented Feb 10, 2025

Evangelink commented Feb 10, 2025

EvgenyMuryshkin commented Feb 10, 2025 • edited Loading

Evangelink commented Feb 11, 2025

jimiscott commented Feb 11, 2025 • edited Loading

Evangelink commented Feb 11, 2025

xperiandri commented Jan 20, 2025 •

edited by pavelhorak

Loading

Joy-less commented Jan 21, 2025 •

edited

Loading

EvgenyMuryshkin commented Feb 10, 2025 •

edited

Loading

jimiscott commented Feb 11, 2025 •

edited

Loading