Skip to content
This repository has been archived by the owner on Dec 5, 2022. It is now read-only.

vstest.console.exe fails with 'Unhandled Exception: System.ArgumentException: Illegal characters in path.' on rerun #268

Open
ZeddzZ opened this issue Aug 30, 2022 · 0 comments

Comments

@ZeddzZ
Copy link

ZeddzZ commented Aug 30, 2022

After running tests using vstest.console.exe in ADO pipeline with some failures rerun throws an exception. After short investigation I found that the reason can be with failed tests, tagged with NUnit Test case attribute, because their final name is in format MyTestName("value1", "value2"). After such test failed it is added to rerun string and on next vstest.console.exe call fails, because braces are counted as special characters and can't be used as a part of filter, even if escaped.

Here is the command created on rerun:

vstest.console.exe /TestCaseFilter:"FullyQualifiedName=TestScenarios.TestClass1.FileReturnWholePageTest|FullyQualifiedName=TestScenarios.TestClass3.ReviewSideMenuTest(\"option1\"\,\"option2\, option3\")|FullyQualifiedName=TestScenarios.TestClass2.DataEntryOrderingTest|FullyQualifiedName=TestScenarios.TestClass1.FIleReturnEsignatureTest|FullyQualifiedName=TestScenarios.TestClass3.ReviewErrorSectionTest(\"option1\"\,\"option2\, option3\")|FullyQualifiedName=TestScenarios.TestClass3.ReviewWholePageTest(\"option1\"\,\"option2\, option3\")|FullyQualifiedName=TestScenarios.TestClass4.ProjectGlobalHeaderTest|FullyQualifiedName=TestScenarios.TestClass1.FileReturnSideMenuTest|FullyQualifiedName=TestScenarios.TestClass3.ReviewWholePageTestt(\"option1\"\,\"option2\, option3\")|FullyQualifiedName=TestScenarios.TestClass3.ReviewSideMenuTest(\"option4\"\,\"option5\, option6\")|FullyQualifiedName=TestScenarios.TestClass1.FileReturnErrorSectionTest|FullyQualifiedName=TestScenarios.TestClass2.DatEntryExpandedSideSectionTest|FullyQualifiedName=TestScenarios.TestClass2.DatEntryInterviewSubSectionTest|FullyQualifiedName=TestScenarios.TestClass1.FileReturnReportLinksTest|FullyQualifiedName=TestScenarios.TestClass2.DataEntryDownloadImportRlSlipsTest|FullyQualifiedName=TestScenarios.TestClass3.ReviewReportSectionTestt(\"option1\"\,\"option2\, option3\")|FullyQualifiedName=TestScenarios.TestClass5.ProjectPropertiesWholePageTest|FullyQualifiedName=TestScenarios.TestClass2.DatEntryCollapsedSideSectionTest|FullyQualifiedName=TestScenarios.TestClass2.DataEntrySearchFieldTest|FullyQualifiedName=TestScenarios.TestClass1.FileReturnReportLinksIncorrectProjectTest|FullyQualifiedName=TestScenarios.TestClass3.ReviewErrorSectionTest\"option4\"\,\"option5\, option6\")|FullyQualifiedName=TestScenarios.TestClass1.FileReturnReportSectionTest|FullyQualifiedName=TestScenarios.TestClass2.DatEntryNotInterviewSubSectionTest|FullyQualifiedName=TestScenarios.TestClass2.DatEntryWholePageTest|FullyQualifiedName=TestScenarios.TestClass6.T1SettingsHeaderHierarchyTest"
"E:\AP06\DAB06_01_work\64\s\xxx\TestScenarios.dll"
/Settings:"E:\AP06\DAB06_01_work_temp\ct3fvllxq0q.tmp.runsettings"
/Logger:"trx"
/TestAdapterPath:"E:\AP06\DAB06_01_work\64\s"

Whole error thrown:

##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.ArgumentProcessorFactory.<>c__DisplayClass20_0.b__0()
##[error] at System.Lazy`1.CreateValue()
##[error] at System.Lazy`1.LazyInitValue()
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.ArgumentProcessorFactory.<>c__DisplayClass20_0.b__0()
##[error] at System.Lazy`1.CreateValue()
##[error] at System.Lazy`1.LazyInitValue()
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.ArgumentProcessorFactory.<>c__DisplayClass20_0.b__0()
##[error] at System.Lazy`1.CreateValue()
##[error] at System.Lazy`1.LazyInitValue()
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.ArgumentProcessorFactory.<>c__DisplayClass20_0.b__0()
##[error] at System.Lazy`1.CreateValue()
##[error] at System.Lazy`1.LazyInitValue()
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.ArgumentProcessorFactory.<>c__DisplayClass20_0.b__0()
##[error] at System.Lazy`1.CreateValue()
##[error] at System.Lazy`1.LazyInitValue()
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.ArgumentProcessorFactory.<>c__DisplayClass20_0.b__0()
##[error] at System.Lazy`1.CreateValue()
##[error] at System.Lazy`1.LazyInitValue()
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Processors.ArgumentProcessorFactory.<>c__DisplayClass20_0.b__0()
##[error] at System.Lazy`1.CreateValue()
##[error] at System.Lazy`1.LazyInitValue()
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Executor.GetArgumentProcessors(String[] args, List`1& processors)
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Executor.Execute(String[] args)
##[error] at Microsoft.VisualStudio.TestPlatform.CommandLine.Program.Main(String[] args)

Here is ADO YAML code

steps:
- task: VSTest@2
displayName: 'VsTest - testAssemblies'
inputs:
testAssemblyVer2: |
$(BuildConfiguration)$(TestDll)
!
\obj**
platform: '$(BuildPlatform)'
configuration: '$(BuildConfiguration)'
rerunFailedTests: true
rerunFailedThreshold: 50
rerunMaxAttempts: 2
continueOnError: true

Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant