Shared Steps are not being updated in the Test Case

[MikeWarner0118]

I am trying to migrate my test cases. I broke my .json file apart and migrated over just my shared steps. I verified that those migrations moved over all of the shared steps from my source project (TFS2012). I then tried to migrate my test cases (also using WorkItemMigrationConfig). It does not seem to be updating the link to the shared steps in the Test Case steps.

Is this done in another processor? If so, which one? Below is the processor section of my .json file. For debugging puproses, I am only running one process at a time and with the WorkItemMigrationConfig process, I am executing that one work item at a time as well:

"Processors":
[
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.NodeStructuresMigrationConfig",
"Enabled": false,
"PrefixProjectToNodes": false
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.WorkItemMigrationConfig",
"Enabled": true,
"PrefixProjectToNodes": false,
"UpdateCreatedDate": true,
"UpdateCreatedBy": true,
"UpdateSoureReflectedId": false,
"QueryBit": "AND [System.WorkItemType] IN ('Test Case')"
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.LinkMigrationConfig",
"Enabled": false,
"QueryBit": "AND ([System.ExternalLinkCount] > 0 OR [System.RelatedLinkCount] > 0)"
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.AttachementExportMigrationConfig",
"Enabled": false,
"QueryBit": "AND [System.AttachedFileCount] > 0"
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.AttachementImportMigrationConfig",
"Enabled": false
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.TestVariablesMigrationConfig",
"Enabled": false
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.TestConfigurationsMigrationConfig",
"Enabled": false
},
{
"ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.TestPlansAndSuitsMigrationConfig",
"Enabled": false,
"PrefixProjectToNodes": false
}
]

[MrHinsh]

Good catch. Yes, there needs to be another processor, and no it has not been built yet.

Although its an awesome feature most teams don't use Shared Steps, and we have not implemented anything around there.

There used to be some code on "[Fix broken Shared steps link after Migration from TFS Integration platform](Fix broken Shared steps link after Migration from TFS Integration platform)" but it looks like that blog is down or moving...

The new processor needs to loop though all of the Shared Steps and update the XML in the steps field to point to the new SharedStep ID... Ill look at it...

[MikeWarner0118]

I am looking at the code and there is code in the project for migrating shared steps, it just looks like when the shared steps work items got migrated, the ReflectedWorkItemId field did not get populated with the source information (original ID). The test cases that were migrated have the updated ReflectedWorkItemID field. The MigrateShraredSteps() seems to rely on finding the corresponding migrated shared steps using that field.

I am going to reset my project and step through the migration of the shared steps to see if I can find why that field is not populating.

[MikeWarner0118]

It may not be the cleanest method, but i found that when the Shared Step is being migrated, it does not have access to the ReflectedWorkItemID field. (at least in my organizations configuration of VSTS.) So, when the Shared Step gets migrated, I add a line to the description with the source ID and then query the descrpition field during LinkMigration.

Like I said, it may not be the cleanest and there may be a better location for the information, but in my situation it works.

[Abinash-Deepak]

So, when the Shared Step gets migrated, I add a line to the description with the source ID and then query the descrpition field during LinkMigration.

Can you please elaborate on this ? The "Shared Steps" is not shown as a separate work item in the Account Settings and hence it is not possible to add the 'ReflectedWorkItemID' field to it. When I did the migration, the shared steps did not get migrated. So, I would like to try what you describe above.

Does it mean that before running the 'WorkItemRevisionReplayMigrationConfig' processor, I have to fill the Description field with something ?

[MikeWarner0118]

Correct the "Shared Steps" is not a work item that is setup in the account settings. But, it is still a Work Item Type. In VSTS, you can run a query where Work Item Type = "Shared Steps"

So, in the .json file I have an entry for WorkItemTypeDefinition of "Shared Steps" : "Shared Steps". This is also in the WorkItemMigrationConfig processor.

In the WorkItemMigrationContext.cs I have a simple If statement where the description string is being built:
if (destType == "Shared Steps") { description.Append("Migrated From TFS ID = " + oldWi.Id); }

The rest of the work I have set up during the LinkMigration. In the MigrateSharedSteps() I changed the call to FindReflectedWorkItemByReflectedWorkItemId:
WorkItem matchingTargetSharedStep = targetStore.FindReflectedWorkItemByReflectedWorkItemId(sourceSharedStep, "SharedSteps");

So, in the WirkItemStoreContext.cs FindReflectedWorkItemByReflectedWorkItemId function, I added another if to build a different query string if the reflectedWorkItemIdField == "SharedSteps"

if (reflectedWorkItemIdField == "SharedSteps")
            {
                query.Query = string.Format(@"SELECT [System.Id] FROM WorkItems  WHERE [System.TeamProject]=@TeamProject AND [{0}] Contains @idToFind", "System.Description");
                query.AddParameter("idToFind", "'Migrated From TFS ID = " + refId.ToString() + "'");
                query.AddParameter("TeamProject", this.targetTfs.Name);
            }
            else
            {
                query.Query = string.Format(@"SELECT [System.Id] FROM WorkItems  WHERE [System.TeamProject]=@TeamProject AND [{0}] = @idToFind", reflectedWorkItemIdField);
                query.AddParameter("idToFind", refId.ToString());
                query.AddParameter("TeamProject", this.targetTfs.Name);
           }

This seems to do the trick of updating the steps within the test case with the new reference to the migrated Shared Steps.

[Abinash-Deepak]

I made the above changes suggested by you and then deleted the existing shared steps in the "To" Project. Then I run the exe and I got the following error:

vstssyncmigrator.exe Information: 0 :  Access granted
vstssyncmigrator.exe Information: 0 :  Migration Context Start WorkItemMigrationContext
TfsQueryContext: TfsQueryContext: TeamProjectCollection: https://XXX.visualstudio.com/
TfsQueryContext: TfsQueryContext: TeamProject: XXXX
vstssyncmigrator.exe Information: 0 :  Query Complete: found 16 work items in 446ms
WorkItemMigrationContext: Migrate 16 work items?
WorkItemMigrationContext: Found target project as XX
vstssyncmigrator.exe Warning: 0 :   [EXCEPTION] An item with the same key has already been added.

Unhandled Exception: System.ArgumentException: An item with the same key has already been added.
   at VstsSyncMigrator.Engine.MigrationContextBase.Execute() in C:\tools\VSTSSyncMigration\Code\tags\7.5.15\src\VstsSyncMigrator.Core\Execution\MigrationContext\MigrationContextBase.cs:line 80
   at VstsSyncMigrator.Engine.MigrationEngine.Run() in C:\tools\VSTSSyncMigration\Code\tags\7.5.15\src\VstsSyncMigrator.Core\MigrationEngine.cs:line 127
   at VstsSyncMigrator.ConsoleApp.Program.RunExecuteAndReturnExitCode(RunOptions opts) in C:\tools\VSTSSyncMigration\Code\tags\7.5.15\src\VstsSyncMigrator.Console\Program.cs:line 156
   at VstsSyncMigrator.ConsoleApp.Program.<>c.<Main>b__4_1(RunOptions opts) in C:\tools\VSTSSyncMigration\Code\tags\7.5.15\src\VstsSyncMigrator.Console\Program.cs:line 94
   at CommandLine.ParserResultExtensions.MapResult[T1,T2,T3,TResult](ParserResult`1 result, Func`2 parsedFunc1, Func`2 parsedFunc2, Func`2 parsedFunc3, Func`2 notParsedFunc)
   at VstsSyncMigrator.ConsoleApp.Program.Main(String[] args) in C:\tools\VSTSSyncMigration\Code\tags\7.5.15\src\VstsSyncMigrator.Console\Program.cs:line 92

It is apparently something to do with the dictionary in the MigrationContextBase.cs file but I'm not able to find the solution. Did you face any such error?

[MrHinsh]

Yes.. and could not isolate it... I found that closing CMD and opening a new one resolved it for me.

[Abinash-Deepak]

I tried that. No use. The exception is still there.
Which version did you try the above changes ? I tried it on 7.5.15. There were other errors when I tried on new versions like 7.5.25.

[MrHinsh]

Can you share your .json?

[Abinash-Deepak]

{
  "TelemetryEnableTrace": true,
  "Source": {
    "Collection": "https://XXXX.visualstudio.com/",
    "Name": "YYYY"
  },
  "Target": {
    "Collection": "https://XX.visualstudio.com/",
    "Name": "YYY"
  },
  "ReflectedWorkItemIDFieldName": "YYYY.ReflectedWorkItemId",
  "FieldMaps": [
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.MultiValueConditionalMapConfig",
      "WorkItemTypeName": "*",
      "sourceFieldsAndValues": {
        "Field1": "Value1",
        "Field2": "Value2"
      },
      "targetFieldsAndValues": {
        "Field1": "Value1",
        "Field2": "Value2"
      }
    },
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldBlankMapConfig",
      "WorkItemTypeName": "*",
      "targetField": "YYYY.ReflectedWorkItemId"
    },
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValueMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "System.State",
      "targetField": "System.State",
      "valueMapping": {
        "Approved": "New",
        "New": "New",
        "Committed": "Active",
        "In Progress": "Active",
        "To Do": "New",
        "Done": "Closed"
      }
    },
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldtoFieldMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "Microsoft.VSTS.Common.BacklogPriority",
      "targetField": "Microsoft.VSTS.Common.StackRank"
    },
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldtoTagMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "System.State",
      "formatExpression": "ScrumState:{0}"
    },
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldMergeMapConfig",
      "WorkItemTypeName": "*",
      "sourceField1": "System.Description",
      "sourceField2": "Microsoft.VSTS.Common.AcceptanceCriteria",
      "targetField": "System.Description",
      "formatExpression": "{0} <br/><br/><h3>Acceptance Criteria</h3>{1}",
      "doneMatch": "##DONE##"
    },
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.RegexFieldMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "COMPANY.PRODUCT.Release",
      "targetField": "COMPANY.DEVISION.MinorReleaseVersion",
      "pattern": "PRODUCT \\d{4}.(\\d{1})",
      "replacement": "$1"
    },
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.FieldValuetoTagMapConfig",
      "WorkItemTypeName": "*",
      "sourceField": "Microsoft.VSTS.CMMI.Blocked",
      "pattern": "Yes",
      "formatExpression": "{0}"
    },
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.FieldMap.TreeToTagMapConfig",
      "WorkItemTypeName": "*",
      "toSkip": 3,
      "timeTravel": 1
    }
  ],
  "WorkItemTypeDefinition": {
    "Bug": "Bug",
    "Product Backlog Item": "Product Backlog Item",
	"Requirement": "Requirement",
	"Test Case": "Test Case",
	"Test Plan": "Test Plan",
	"Test Suite": "Test Suite",
	"Shared Steps" : "Shared Steps"
  },
  "Processors": [
    {
      "ObjectType": "VstsSyncMigrator.Engine.Configuration.Processing.WorkItemMigrationConfig",
      "PrefixProjectToNodes": true,
      "UpdateCreatedDate": true,
      "UpdateCreatedBy": true,
      "UpdateSoureReflectedId": false,
      "QueryBit": "AND [YYYY.ReflectedWorkItemId] = '' AND  [Microsoft.VSTS.Common.ClosedDate] = '' AND [System.WorkItemType] IN ('Shared Steps')",
      "Enabled": true
    },

[Abinash-Deepak]

@MikeWarner0118

I think there are more modifications in the code required than the ones you have mentioned here. Or may be I misunderstood.

For me (in the version 7.5.25) the following code in the LinkMigrationContext.cs file

try
                {
                    wiTargetL = targetWitsc.FindReflectedWorkItem(wiSourceL, me.ReflectedWorkItemIdFieldName, true);
                }

always returns NULL because the me.ReflectedWorkItemIdFieldName gives the "Processname.ReflectedWorkItemId" value at this point. Because of this, the function call

                WorkItem matchingTargetSharedStep =
                    targetStore.FindReflectedWorkItemByReflectedWorkItemId(sourceSharedStep,
                        "SharedSteps");

does not get executed at all.

Also in the above comment you mentioned

So, in the .json file I have an entry for WorkItemTypeDefinition of "Shared Steps" : "Shared Steps". This is also in the WorkItemMigrationConfig processor.

What do you mean ? Can you please look at my JSON file above and let me know if it is correct ?

[MikeWarner0118]

Sorry it has taken me so long to respond.

To me, it looks like in your JSON file you have:
"ReflectedWorkItemIDFieldName": "YYYY.ReflectedWorkItemId",

Try removing the YYYY from this value and all of the other places in the JSON when you are trying to reference this field. I don't think you need to tell the code the Project name of the ReflectedWorkItemId field.

[DanielBMann9000]

Does this tool support migration of Shared Steps, or does it not?

I see references to shared step migration in the code, but this issue is nearly two years old and still open with no resolution.

If shared steps are not supported, that should be explicitly called out in the documentation.

[MrHinsh]

Yes it does.

You need to migrate Test Cases and Shared Steps with the Work Item Migrator. The link between the Test Case and the shared step should be fixed as part of the migration of the Test Case.

Note: This tool is supported by the community and covers only the edge cases that have been encountered by this community. The least executed part of the tool is the Test Suits & Plans migrator and thus it has the most missing edge cases.

[deeepakmhaskar]

@MrHinsh, The Links are working fine however the shared steps are not displayed in the test case. The steps area showing empty steps.
Can this be fixed?

[MrHinsh]

After some investigation, the tool will only support SharedSteps on-premises.

Azure DevOps Services does not allow the customisation of the Shared Step work item to add the required ReflectedWorkItemID field.

[WorkChris]

@deeepakmhaskar @MikeWarner0118 have a look at my workaround for test cases with shared steps.
#518
Pretty sure it is not a supported method, but it is something that I use in special edge cases where there are shared steps in use.