Initial implementation of the test command and API

[JanKrivanek]

Problem

initial implementation of #3868
looking for general comments and suggestions

Solution

• VerificationCommand
• VerificationEngine (+ options)
• Sample unit test

TBD

• code cleaning
• localization
• tests
• Integration of new Verify features (Directory diffing, etc.)

Checks - TBD

• Added unit tests
• Added #nullable enable to all the modified files ?

[vlada-shubina]

First pass - looks good, mostly minor comments and missing docs.
Haven't got to tests yet.

Need to digest and do another pass.

[vlada-shubina]

just curious: why this was not needed before?

[SimonCropp]

@vlada-shubina https://github.com/VerifyTests/Verify/blob/main/docs/upgrade17-18.md#move-to-argon

[vlada-shubina]

@SimonCropp is there any way we can avoid this workaround?

[SimonCropp]

@vlada-shubina i can give u static SetMethodInfo api on UsesVerifyAttribute. it would need to be called on the same thread before a verify occurs. or i can give u an optional MethodInfo parameter on all the Verify* methods.

[JanKrivanek]

@SimonCropp I'd say it's still very similar situation - a dummy MethodInfo needs to be passed even though it doesn't end up being used (we need to name directories/files by scenarios run, not by the code that was invoking them - so the type and method name are overriden by PathInfo and parameters are not used as method is intentionally parameterless).
In ideal case the InnerVerifier would pass Lazy that would throw only on access (plus there would need to be a way to indicate that parameters should not be used).
Though our scenario is pretty specific (and easily workaroundable) - so I understand this might not meet the bar.

[vlada-shubina]

Thanks for this work - looks pretty good.

Some of suggestions can be done later in a separate tasks.
It would be good to have one XUnit test showcasing the usage before we move forward.

[SimonCropp]

a more correct CreateVerificationTask

    private static Task CreateVerificationTask(string contentDir, TemplateVerifierOptions options)
    {
        List<string> exclusionsList = options.DisableDefaultVerificationExcludePatterns ?? false
            ? new()
            : new(_defaultVerificationExcludePatterns);

        if (options.VerificationExcludePatterns != null)
        {
            exclusionsList.AddRange(options.VerificationExcludePatterns);
        }

        List<Glob> globs = exclusionsList.Select(pattern => Glob.Parse(pattern)).ToList();

        var verifySettings = new VerifySettings();

        if (options.CustomScrubbers != null)
        {
            if (options.CustomScrubbers.GeneralScrubber != null)
            {
                verifySettings.AddScrubber(options.CustomScrubbers.GeneralScrubber);
            }

            foreach (var pair in options.CustomScrubbers.ScrubersByExtension)
            {
                verifySettings.AddScrubber(pair.Key, pair.Value);
            }
        }

        if (options.CustomDirectoryVerifier != null)
        {
            return options.CustomDirectoryVerifier(
                contentDir,
                new Lazy<IAsyncEnumerable<(string FilePath, string ScrubbedContent)>>(
                    GetVerificationContent(contentDir, globs, options.CustomScrubbers)));
        }

        if (options.TemplateName != null)
        {
            verifySettings.UseTypeName(options.TemplateName);
        }

        if (options.TemplateName != null)
        {
            verifySettings.UseDirectory(options.ExpectationsDirectory ?? "VerifyExpectations");
        }

        verifySettings.UseMethodName(EncodeArgsAsPath(options.TemplateSpecificArgs));

        if ((options.UniqueFor ?? UniqueForOption.None) != UniqueForOption.None)
        {
            foreach (UniqueForOption value in Enum.GetValues(typeof(UniqueForOption)))
            {
                if ((options.UniqueFor & value) == value)
                {
                    switch (value)
                    {
                        case UniqueForOption.None:
                            break;
                        case UniqueForOption.Architecture:
                            verifySettings.UniqueForArchitecture();
                            break;
                        case UniqueForOption.OsPlatform:
                            verifySettings.UniqueForOSPlatform();
                            break;
                        case UniqueForOption.Runtime:
                            verifySettings.UniqueForRuntime();
                            break;
                        case UniqueForOption.RuntimeAndVersion:
                            verifySettings.UniqueForRuntimeAndVersion();
                            break;
                        case UniqueForOption.TargetFramework:
                            verifySettings.UniqueForTargetFramework();
                            break;
                        case UniqueForOption.TargetFrameworkAndVersion:
                            verifySettings.UniqueForTargetFrameworkAndVersion();
                            break;
                        default:
                            throw new ArgumentOutOfRangeException();
                    }
                }
            }
        }

        if (options.DisableDiffTool ?? false)
        {
            verifySettings.DisableDiff();
        }

        return Verifier.VerifyDirectory(
            contentDir,
            filePath => !globs.Any(g => g.IsMatch(filePath)),
            settings: verifySettings);
    }

[JanKrivanek]

It would be good to have one XUnit test showcasing the usage before we move forward.

The VerificationEngineSampleDogfoodTest shows an example usage

https://github.com/dotnet/templating/pull/5263/files#diff-3b14e1a4453010f532943aebb758e25255f7bde1f4ebe1a47dc6cfca363692c5R151-R176

[vlada-shubina]

great improvements, thanks

[vlada-shubina]

we have an IPhysicalFileSystem interface in Abstractions. Can it be useful here?

[JanKrivanek]

It doesn't expose the async methods (due to support for netcore 2.0). But I have extended it and reused what I could