ProGuide Regression Testing

Language:              cs
Subject:               Framework
Contributor:           ArcGIS Pro SDK Team <arcgisprosdk@esri.com>
Organization:          Esri, http://www.esri.com
Date:                  12/21/2022
ArcGIS Pro:            3.0
Visual Studio:         2022
.NET Target Framework: .Net 6

This ProGuide explains the step by step process as was described in the DevSummit 2021 ArcGIS Pro SDK .NET: Regression Testing your ArcGIS Pro Add-in technical session. Use it in conjunction with the technical session to learn how to create a unit test project in Visual Studio and author tests to test ArcGIS Pro Add-Ins.

In this topic

• Prerequisites
• Part 1: Create unit test project
  • Step 1. Add new project
  • Step 2. Create a unit test project
  • Step 3. Create a runsettings file
  • Step 4. Configure runsettings file
  • Step 5. Add references
  • Step 6. Add Helper classes
    • Code change for Configurations
  • Step 7. Add test methods
• Part 2. Update Add-In project
  • Step 1. Allow access to internal members
  • Step 2. Update view-model constructor access modifier
  • Step 3. Refactor CmdRefreshSwatches property
  • Step 4. Add UpdateCollection method
  • Step 5. Run the tests

Prerequisites

1. If ArcGIS Pro is licensed using Named User license then it must be authorized to work offline. This is not required if ArcGIS Pro is licensed using either Single or Concurrent Use license.
2. Create a blank Pro project and import the Map.mapx file that is available in the ArcGIS Pro SDK Community Samples Data. Sample data for ArcGIS Pro SDK Community Samples can be downloaded from the repo releases page. If your test project already has a map named "Map" then delete the existing map before importing the attached .mapx file. For the purpose of running tests in this ProGuide it is okay to have broken layers in the map, it is important that the layer names and symbology remain unchanged.
3. Open the Map-Authoring/GetSymbolSwatch Add-In solution in Visual Studio. Clone the repo or download zip from Esri/arcgis-pro-sdk-community-samples.
4. Developers are encouraged to use this ProGuide in conjunction with the DevSummit 2021 (ArcGIS Pro SDK .NET: Regression Testing your ArcGIS Pro Add-in](https://www.youtube.com/watch?v=apxqfcFXYUk&list=PLaPDDLTCmy4btgVu7omfiaw6PhKUzyk0M&index=20) technical session
5. Download the example code used in the session and referred to in this guide from our ArcGIS Pro SDK for .NET Technical Sessions page.
6. There are various paths hardcoded into the example code in this ProGuide. Please change the paths, as needed, to reflect the location of the referenced resources/dlls, etc. on your individual machine.

Part 1. Create unit test project

The first step is to add a unit test project in the Add-In solution. You could also create a project in a new solution but it is preferable to create the test project in the same solution as the Add-In so that it is easier to run the tests as soon as the Add-In code is modified.

Step 1. Add new project

In Visual Studio right click on GetSymbolSwatch solution, click on "Add" and then "New Project".

Step 2. Create a unit test project

In "Create a New Project" dialog search for "test". From the list of projects that show up select "MSTest Project". Click "Next".

In "Configure your new project" dialog edit project name to "GetSymbolSwatchTests". Click "Next".

In "Additional information" dialog select ".Net 6.0 (Long-term support)". Click "Create".

Step 3. Create a runsettings file

In "Solution Explorer", right click on the "GetSymbolSwatchTests" project, click on "Add", and then "New Item".

Scroll through the list and select "XML File". Name the file "protests.runsettings" and click "Add".

Edit "protests.runsettings" file and replace the existing contents of the file with the settings shown below

<?xml version="1.0" encoding="utf-8"?>
<RunSettings>
  <RunConfiguration>
    <MaxCpuCount>0</MaxCpuCount>
    <ResultsDirectory>.\TestResults</ResultsDirectory>
    <TestSessionTimeout>9000000</TestSessionTimeout>
  </RunConfiguration>

  <LoggerRunSettings>
    <Loggers>
      <Logger friendlyName="trx" enabled="true" />
    </Loggers>
  </LoggerRunSettings>

  <DataCollectionRunSettings>
    <DataCollectors>
      <DataCollector friendlyName="blame" enabled="True" />
    </DataCollectors>
  </DataCollectionRunSettings>

  <MSTest>
    <TestTimeout>90000</TestTimeout>
    <MapInconclusiveToFailed>false</MapInconclusiveToFailed>
    <CaptureTraceOutput>true</CaptureTraceOutput>
    <DeleteDeploymentDirectoryAfterTestRunIsComplete>true</DeleteDeploymentDirectoryAfterTestRunIsComplete>
    <DeploymentEnabled>false</DeploymentEnabled>
    <AssemblyResolution>
      <Directory path="C:\Program Files\ArcGIS\Pro\bin\" includeSubDirectories="true"/>
    </AssemblyResolution>
  </MSTest>
</RunSettings>

Note the Pro installation path referenced in <Directory path="....." />. Change this to point to the Pro installation path on your machine.

Step 4. Configure runsettings file

In Visual Studio click on "Test" menu, click on "Configure Run Settings", and then "Select Solution Wide runsettings File".

In the "Open Settings File" browse dialog that opens navigate to and select the runsettings file from the previous step. Click Open.

Step 5. Add references

The test project should include all the references that are included in the Add-In project, including the Add-In project itself.

---

💡 An easy option to add reference assemblies that are in the Add-In project would be to open the "GetSymbolSwatch" Add-In project file (csproj) in a text editor like Notepad, copy the required references, and paste them in the "GetSymbolSwatchTests" project file.

---

In Solution Explorer, right click on the "GetSymbolSwatchTests" project and select the option to "Edit Project File". Replace the <PropertyGroup> with the following snippet:

  <PropertyGroup>
    <TargetFramework>net6.0-windows</TargetFramework>
    <ImplicitUsings>enable</ImplicitUsings>
    <Nullable>enable</Nullable>
    <Platforms>x64</Platforms>
    <PlatformTarget>x64</PlatformTarget>
    <ResolveAssemblyWarnOrErrorOnTargetArchitectureMismatch>None
        </ResolveAssemblyWarnOrErrorOnTargetArchitectureMismatch>
    <GenerateAssemblyInfo>false</GenerateAssemblyInfo>
    <EnableDefaultEmbeddedResourceItems>false</EnableDefaultEmbeddedResourceItems>
    <UseWindowsForms>true</UseWindowsForms>
    <UseWPF>true</UseWPF>
    <ImportWindowsDesktopTargets>true</ImportWindowsDesktopTargets>
    <IsTestProject>true</IsTestProject>
    <EnableNETAnalyzers>false</EnableNETAnalyzers>
    <OutputPath>bin\$(Configuration)</OutputPath>
  </PropertyGroup>

In "Solution Explorer" expand "GetSymbolSwatchTests" project and right click on the "Dependencies" node. Click "Add Project Reference".

In the "Reference Manager" dialog expand "Projects" node and select "Solution". Select the option to add "GetSymbolSwatch" project.

In the "Reference Manager" dialog select "Browse" node. Click on the "Browse.." button to navigate and add all the ArcGIS Pro assemblies referenced in the Add-In project.

Make sure "ArcGISPro.dll" from the "bin" folder is also selected, this isn't referenced in the Add-In project.

Click OK.

In "Solution Explorer" expand "Dependencies" and then expand "Assemblies" nodes in the "GetSymbolSwatchTests" project. Select all Pro assemblies, right click and select Properties.

Modify "Copy Local" property and set it to "No".

In Solution Explorer, right click on the "GetSymbolSwatchTests" project and select the option to "Edit Project File". For each of the Pro assemblies that is referenced add an entry <Private>False</Private>. For example,

    <Reference Include="ArcGISPro">
      <HintPath>C:\Program Files\ArcGIS\Pro\bin\ArcGISPro.dll</HintPath>
      <CopyLocal>False</CopyLocal>
      <Private>False</Private>
    </Reference>

Step 6. Add Helper classes

Right click on "GetSymbolSwatchTests" project, click on "Add", and then "Class". Name the new class "TestEnvironment.cs".

Replace the contents of the file with the following code snippet.

using ArcGIS.Core;
using ArcGIS.Desktop.Core;
using ArcGIS.Desktop.Internal.Core;
using ArcGISTest;
using System.Diagnostics;
using System.Windows.Markup;

namespace GetSymbolSwatchTests
{
  [ArcGISTestClass]
  public static class TestEnvironment
  {
    [AssemblyInitialize()]
    public static void AssemblyInitialize(TestContext testContext)
    {
      StartApplication();
    }

    [AssemblyCleanup]
    public static void AssemblyCleanup()
    {
      StopApplication();
    }

    /// <summary>
    /// Starts an instance of ArcGIS Pro Application
    /// </summary>
    public static async void StartApplication()
    {
      var evt = new System.Threading.ManualResetEvent(false);
      System.Threading.Tasks.Task ready = null;

      var uiThread = new System.Threading.Thread(() =>
      {
        try
        {
          Application = new ProApp();
          ready = Application.TestModeInitializeAsync();
          evt.Set();
        }
        catch (XamlParseException)
        {
          throw new FatalArcGISException("Pro is not licensed");
        }
        catch (Exception ex)
        {
          throw ex;
        }
        finally
        {
          evt.Set();
        }

        System.Windows.Threading.Dispatcher.Run();
      });

      uiThread.TrySetApartmentState(System.Threading.ApartmentState.STA);
      uiThread.Name = "Test UI Thread";
      uiThread.IsBackground = true;
      uiThread.Start();

      evt.WaitOne(); // Task ready to wait on.

      if (ready != null)
      {
        try
        {
          await ready;
        }
        catch (Exception ex)
        {
          throw ex;
        }
      }
    }

    /// <summary>
    /// Shuts down the ArcGIS Pro Application instance
    /// </summary>
    [System.Diagnostics.CodeAnalysis.SuppressMessage(
         "Microsoft.Design", "CA1031:DoNotCatchGeneralExceptionTypes")]
    public static void StopApplication()
    {
      try
      {
        if (Application != null)
          Application.Shutdown();
      }
      catch (Exception e)
      {
        //do not re-throw the exception here.
        Debug.Print("Application.Shutdown threw an exception that was ignored. message: {0}", e.Message);
      }
    }

    /// <summary>
    /// Get an instance of ArcGIS Pro Application
    /// </summary>
    public static ProApp Application { get; private set; }
  }
}

Right click on "GetSymbolSwatchTests" project, click on "Add", and then "Class". Name the new class "ArcGISTestClassAttribute.cs".

Replace the contents of the file with the following code snippet.

namespace ArcGISTest
{
  public class ArcGISTestClass: Microsoft.VisualStudio.TestTools.UnitTesting.TestClassAttribute
  {
    public ArcGISTestClass(string productId = "ArcGISPro") : base()
    {
      // Install domain wide assembly resolver
      TestResolver.Install(productId);
    }
  }
}

Right click on "GetSymbolSwatchTests" project, click on "Add", and then "Class". Name the new class "TestResolver.cs".

Replace the contents of the file with the following code snippet.

// Copyright (c) Microsoft Corporation. All rights reserved.
// Licensed under the MIT license. See LICENSE file in the project root for full 
// license information.
using System;
using System.Collections.Generic;
using System.IO;
using System.Reflection;
using System.Text.Json;
using Microsoft.Win32;

namespace ArcGISTest
{
  internal static class TestResolver
  {
    private static Dictionary<string, string> assemblyMap = new Dictionary<string, string>();

    private static string _productId = "ArcGISPro";
    private static string _installDir = String.Empty;

    static TestResolver()
    {
    }

    static public void Install(string productId = "ArcGISPro")
    {
      _productId = productId;
      ConfigureResolver();
      AppDomain.CurrentDomain.AssemblyResolve += new ResolveEventHandler(CustomResolverHandler);
    }

    private static Assembly CustomResolverHandler(object sender, ResolveEventArgs args)
    {
      System.Reflection.Assembly retAsm = null;

      try
      {
        string filename = args.Name.Split(',')[0];

        if (filename.Contains(".resources.dll"))
          return null;

        string folder = string.Empty;
        var match = assemblyMap.TryGetValue(filename, out folder);

        filename = String.Concat(filename, ".dll");

        if (match)
        {
          retAsm ??= Assembly.LoadFrom(Path.Combine(folder, filename));
        } else
        {
          retAsm ??= Assembly.LoadFrom(Path.Combine(GetProInstallLocation(), 
                                                                     filename));
        }
      }
      catch
      {
        try
        {
          retAsm ??= Assembly.LoadFrom(Path.Combine(
                       GetProInstallLocation(), "Configurations", "Intelligence", 
                           "ArcGIS.Desktop.Intelligence.Configuration.dll"));
        }
        catch
        {
        }
      }

      return retAsm;
    }

    private static string GetProInstallLocation()
    {
      if (!String.IsNullOrEmpty(_installDir))
        return _installDir;

      try
      {
        var sk = Registry.LocalMachine.OpenSubKey(@$"Software\ESRI\{_productId}");
        _installDir = sk.GetValue("InstallDir") as string;
      }
      catch
      {
        try
        {
          var sku = Registry.CurrentUser.OpenSubKey(@$"Software\ESRI\{_productId}");
          _installDir = sku.GetValue("InstallDir") as string;
        }
        catch
        {
        }
      }

      _installDir = Path.Combine(_installDir, "bin");
      return _installDir;
    }

    private static void ConfigureResolver()
    {
      // Already configured
      if (assemblyMap.Count != 0)
      {
        return;
      }

      string installPath = GetProInstallLocation();
      string jsonPath = Path.Combine(installPath, "InstallDependencies.json");
      if (!File.Exists(jsonPath))
      {
        return;
      }

      try
      {
        string fileContent = File.ReadAllText(jsonPath);
        var jd = JsonDocument.Parse(fileContent);
        var root = jd.RootElement;
        var installationNode = root.GetProperty("Installation");

        var folders = installationNode.GetProperty("Folders");
        int count = folders.GetArrayLength();

        for (int i = 0; i < count; ++i)
        {
          var folder = folders[i];
          var folderPath = folder.GetProperty("Path");
          var fullPath = Path.Combine(installPath, folderPath.ToString());

          var assemblies = folder.GetProperty("Assemblies");
          int asmCount = assemblies.GetArrayLength();

          for (int j = 0; j < asmCount; ++j)
          {
            var assm = assemblies[j];
            var asmName = assm.GetProperty("Name");

            assemblyMap.Add(asmName.ToString(), fullPath);
          }
        }
      }
      catch (Exception e)
      {
        System.Diagnostics.Trace.WriteLine(
                  $"Error processing dependency file: {e.Message}");
      }
    }
  }
}

Rebuild the "GetSymbolSwatch" solution to make sure there are no compile errors.

Code change for Configurations

Make the following modification to TestEnvironment when testing a Configuration. Use the ProApp constructor overload that takes the name of the configuration as the parameter:

public static async void StartApplication()
    {
      ...

      var uiThread = new System.Threading.Thread(() =>
      {
        try
        {
          //pass in the name of the configuration under test
          Application = new ProApp("MyConfiguration");
          ready = Application.TestModeInitializeAsync();
          evt.Set();
        }
        catch (XamlParseException) ...
        catch (Exception ex) ...
        finally ...
        ...
      });
      ...
    }

Step 7. Add test methods

In "Solution Explorer" rename existing UnitTest1.cs to ShowSymbolSwatchDockpaneViewModelTests.cs.

Replace the existing code with the following code snippet. In the ClassInitialize method update the path to ArcGIS Pro project that was created as part of the Prerequisites step.

using ArcGIS.Desktop.Core;
using ArcGIS.Desktop.Framework.Threading.Tasks;
using ArcGIS.Desktop.Mapping;
using ArcGISTest;
using GetSymbolSwatch;
using Microsoft.VisualStudio.TestTools.UnitTesting;
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;

namespace GetSymbolSwatchTests
{
  [ArcGISTestClass]
  public class ShowSymbolSwatchDockpaneViewModelTests
  {
    static Map _map = null;
    static IEnumerable<Layer> _layers = null;

    [ClassInitialize]
    public static void ClassInitialize(TestContext testContext)
    {
      //=========================================
      //TODO: Change this path to point to the location of "DevSummit21.aprx" on your machine or to
      //whichever project you are using
      //
      Project project = Project.OpenAsync(@"..\DevSummit21\DevSummit21.aprx").Result;
      QueuedTask.Run(() =>
      {
        _map = project.GetItems<MapProjectItem>().Where(
                             x => x.Name.Equals("Map")).FirstOrDefault().GetMap();
        _layers = _map.GetLayersAsFlattenedList();
      }).Wait();
    }

    [ClassCleanup]
    public static void ClassCleanup()
    {
    }

    [TestInitialize]
    public void TestInitialize()
    {
    }

    [TestCleanup]
    public void TestCleanup()
    {
    }

    [TestMethod]
    public async Task SwatchCountTest()
    {
      ShowSymbolSwatchDockpaneViewModel showSymbolSwatchDockpaneViewModel = 
                                                new ShowSymbolSwatchDockpaneViewModel();

      //var cmdRefreshSwatches = showSymbolSwatchDockpaneViewModel.CmdRefreshSwatches;
      //await _map.OpenViewAsync();
      //cmdRefreshSwatches.Execute(null);

      await showSymbolSwatchDockpaneViewModel.UpdateCollection(_layers.OfType<FeatureLayer>());

      Assert.AreEqual(3, showSymbolSwatchDockpaneViewModel.SymbolSwatchInfoList.Count, 
                         "SymbolSwatchList does not contain expected number of swatches");

    }

    [DataTestMethod]
    [DataRow(0, "Cities", "CIMSimpleRenderer", null, DisplayName = "SimpleRenderer")]
    [DataRow(0, "Volcanos", "CIMClassBreaksRenderer", "127.000000 - 141.000000", 
                                                             DisplayName = "ClassBreaksRenderer")]
    [DataRow(1, "Volcanos", "CIMClassBreaksRenderer", "141.000001 - 165.000000", 
                                                             DisplayName = "ClassBreaksRenderer")]
    public async Task SwatchInfoTest(int swatchIndex, string featureLayerName, 
                                                                   string rendererType, string note)
    {
      ShowSymbolSwatchDockpaneViewModel showSymbolSwatchDockpaneViewModel = 
                                                          new ShowSymbolSwatchDockpaneViewModel();
      await showSymbolSwatchDockpaneViewModel.UpdateCollection(
                 _layers.Where((layer) => { 
                       if (layer.Name == featureLayerName) return true; else return false; 
                 }).OfType<FeatureLayer>());

      SymbolSwatchInfo swatchInfo = 
                        showSymbolSwatchDockpaneViewModel.SymbolSwatchInfoList[swatchIndex];
      Assert.AreEqual(featureLayerName, swatchInfo.FeatureClassName, 
                                       "SwatchInfo feature layer name does not match expected value");
      Assert.AreEqual(rendererType, swatchInfo.RendererType, 
                                       "SwatchInfo feature renderer type does not match expected value");

      if (note is null)
        Assert.IsNull(note);
      else
        Assert.AreEqual(note, swatchInfo.Note);
    }
  }
}

Part 2. Update Add-In project

In Part 2 we will modify the Add-In project in order to resolve compile errors in "GetSymbolSwatchTests" project.

Step 1. Allow access to internal members

In "Solution Explorer", expand "GetSymbolSwatch" project, and then expand "Properties" node. Double click on "AssemblyInfo.cs" to edit the file.

Scroll to the bottom of the file and add a new line with following statement

[assembly: InternalsVisibleTo("GetSymbolSwatchTests")]

Step 2. Update view-model constructor access modifier

In "Solution Explorer" double click on "ShowSymbolSwatchDockpaneViewModel.cs" file to edit the file.

Add internal access modifier to ShowSymbolSwatchDockpaneViewModel constructor.

    internal protected ShowSymbolSwatchDockpaneViewModel() {
      BindingOperations.EnableCollectionSynchronization(SymbolSwatchInfoList, _lockCollection);
    }

Step 3. Refactor CmdRefreshSwatches property

In this step and the following step we will refactor CmdRefreshSwatches property so that we can test the business logic in this property's getter.

First step is to get rid of QueuedTask.Run method in CmdRefreshSwatches property getter and replace it with a call to a new UpdateCollection that we are going to add in the following step.

public ICommand CmdRefreshSwatches
{
  get
  {
    return _cmdRefreshSwatches ?? (_cmdRefreshSwatches = new RelayCommand(() =>
    {
      SymbolSwatchInfoList.Clear();
      if (MapView.Active?.Map == null) return;
      var layers = MapView.Active.Map.GetLayersAsFlattenedList().OfType<FeatureLayer>();

      UpdateCollection(layers);
    }));
  }
}

Step 4. Add UpdateCollection method

Insert the following code to add a new method called UpdateCollection

public async Task UpdateCollection(IEnumerable<FeatureLayer> layers)
{
  await QueuedTask.Run(() =>
  {
    foreach (FeatureLayer layer in layers)
    {
      CIMRenderer renderer = layer.GetRenderer();
      var types = renderer.GetType().ToString().Split(new char[] { '.' });
      var type = types[types.Length - 1];
      switch (type)
      {
        case "CIMSimpleRenderer":
          var simpleSi = GetSymbolStyleItem((renderer as CIMSimpleRenderer).Symbol.Symbol);
          var newSimpleSymbolSwatchInfo = 
                          new SymbolSwatchInfo(layer.Name, type, simpleSi.PreviewImage);
          lock (_lockCollection)
          {
            SymbolSwatchInfoList.Add(newSimpleSymbolSwatchInfo);
          }
          break;
        case "CIMUniqueValueRenderer":
          var uniqueValueRenderer = renderer as CIMUniqueValueRenderer;

          if (uniqueValueRenderer.Groups is null)
            continue;

          foreach (CIMUniqueValueGroup nextGroup in uniqueValueRenderer.Groups)
          {
            foreach (CIMUniqueValueClass nextClass in nextGroup.Classes)
            {
              CIMMultiLayerSymbol multiLayerSymbol = nextClass.Symbol.Symbol as CIMMultiLayerSymbol;
              var mlSi = GetSymbolStyleItem(multiLayerSymbol);
              var newMlSymbolSwatchInfo = new SymbolSwatchInfo(layer.Name, type, mlSi.PreviewImage)
              {
                Note = nextClass.Label
              };
              lock (_lockCollection)
              {
                SymbolSwatchInfoList.Add(newMlSymbolSwatchInfo);
              }
            }
          }
          break;
        case "CIMClassBreaksRenderer":
          var classBreaksRenderer = renderer as CIMClassBreaksRenderer;
          foreach (CIMClassBreak nextClass in classBreaksRenderer.Breaks)
          {
            var classBreakSymbol = nextClass.Symbol.Symbol as CIMSymbol;
            var mlSi = GetSymbolStyleItem(classBreakSymbol);
            var newMlSymbolSwatchInfo = new SymbolSwatchInfo(layer.Name, type, mlSi.PreviewImage)
            {
              Note = nextClass.Label
            };
            lock (_lockCollection)
            {
              SymbolSwatchInfoList.Add(newMlSymbolSwatchInfo);
            }
          }
          if (classBreaksRenderer.UseExclusionSymbol)
          {
            var classBreakSymbol = classBreaksRenderer.ExclusionSymbol.Symbol as CIMSymbol;
            var mlSi = GetSymbolStyleItem(classBreakSymbol);
            var newMlSymbolSwatchInfo = new SymbolSwatchInfo(layer.Name, type, mlSi.PreviewImage)
            {
              Note = classBreaksRenderer.ExclusionLabel
            };
            lock (_lockCollection)
            {
              SymbolSwatchInfoList.Add(newMlSymbolSwatchInfo);
            }
          }
          break;
        default:
          var defaultSymbolSwatchInfo = new SymbolSwatchInfo(layer.Name, type, null)
          {
            Note = "Sample code not implemented"
          };
          lock (_lockCollection)
          {
            SymbolSwatchInfoList.Add(defaultSymbolSwatchInfo);
          }
          break;
      }
    }
  });
}

Rebuild the solution and make sure all compilation errors are resolved.

Step 5. Run the tests

If Test Explorer window isn't already open you can open this window from Visual Studio's "Test" menu, and then "Test Explorer".

Right click on a test and select the option to "Run".