Orchestration Versioning

Orchestration Versioning

This section will outline the process for upgrading the long running orchestrations and provide sample code.

When you are using an orchestration for scheduling and workflow management, you will inevitably come across the following challenges:

1. Orchestrations and activities need to be updated, versioned, or removed.
2. New versions of the orchestrations might not be backward compatible. In fact, we strictly assume that they are never backward compatible.
3. When you are doing upgrade, an orchestration may be in the middle of doing some work. We must allow it to run to completion in order to avoid leaving the system in a potentially inconsistent state. The alternative is to design orchestrations in such a way that deleting the hub (and, therefore, terminating an orchestration in the middle) will not leave the system in an inconsistent state.

There is a process for handling these challenges, and here is the outline.

Process outline

1. When we schedule orchestrations, we refer to them not by class name, but by string name and version. This will allow us to treat two different orchestration classes as two different versions of the same orchestration.

2. When we update orchestrations and/or activities, we deploy both the old and the new code to our workers. This is required to let the previously-started orchestrations run to completion and then gracefully switch over to the new version at run time.

3. We register both the old and the new orchestration classes in the Worker under the same name, but different versions. I will cover this in the examples, but the idea is that we use an ObjectCreator factory-like class.

4. The old orchestration code is modified slightly: the ContinueAsNew method (which starts a new generation) is now provided with the new version. For example, when you first had the orchestration of version "1", you could have something like this in your code:

   Context.ContinueAsNew("1", input);

   Now, we will change that to

   Context.ContinueAsNew("2", input);
   So, when the old orchestration finishes, it will re-schedule itself with a new version. In practice, I do not recommend changing the code directly; instead, we should have a file with constants that we can then use in our code. Of course we must re-build the orchestration code so that it gets the new value of the constant, or use a property instead.

5. Clients shall be updated to schedule the new version of an orchestration (for consistency purposes)

6. To stop and delete an orchestration, the same process is used as for updating, except that the new version is a no-op orchestration which exits immediately. During the next deployment, both versions can be safely deleted.

Code

In this section, I’ll show you which method overloads we’ll need to use to implement the process outlined in the previous subsection.
Assume we have two classes, both deployed:
InfiniteOrchestrationV1 and InfiniteOrchestrationV2
Also, let’s give them a string name “Infinite” and versions (respectively) “1” and “2”. Versions shall be strings, but they shall be parseable to int.

First, we schedule both of the versions in the worker.
We first create two ObjectCreator subtypes “TaskHubObjectCreator<TaskOrchestration>” that accept three parameters:

• Orchestration logical name (in our case, both are named “Infinite”)
• Versions (“1” and “2” respectively)
• Lambda that actually creates the instance of the correct orchestration for the (name, version) pair:

var InfiniteV1 = new TaskHubObjectCreator<TaskOrchestration>("Infinite", "1", () => { return new InfiniteDemoOrchestrationV1(); });

var InfiniteV2 = new TaskHubObjectCreator<TaskOrchestration>("Infinite", "2", () => { return new InfiniteDemoOrchestrationV2(); });

I will provide the definition for TaskHubObjectCreator class below. Then, we put the instances into an array:

var InfiniteOrchestrations = new[] {InfiniteV1, InfiniteV2};

And finally, we use the TaskHubWorker AddTaskOrchestrations overload that accepts a list of ObjectCreator instances:

var taskWorkerHub2 = new TaskHubWorker(Constants.HubName, Constants.ServiceBusConnString)
    .AddTaskOrchestrations(InfiniteOrchestrations)
    .AddTaskActivities(typeof(DemoActivityV1), typeof(DemoActivityV2));

In this way, the worker is now aware of both versions of the orchestration and can correlate them as having the same name, but different versions.

Here is the definition of the TaskHubObjectCreator class: using DurableTask;

using System;
using DurableTask;

/// <summary>
/// A factory class which allows creation an orchestration instance based on the string ID and version
/// </summary>
public class TaskHubObjectCreator : ObjectCreator<TaskOrchestration>
{
    private readonly Func<TaskOrchestration> objectCreatorFunc;

    /// <summary>
    /// Constructs an instance of TaskHubObjectCreator
    /// </summary>
    /// <param name="name">The string name of the orchestration which this factory creates. Several different classes which are conceptually related can have the same string ID but differnet version.</param>
    /// <param name="version">The version of the orchestration that this object creates</param>
    /// <param name="objectCreatorFunc">Creator function. This function must create the correct object for the (name, version) pair provided</param>
    public TaskHubObjectCreator(string name, string version, Func<TaskOrchestration> objectCreatorFunc)
    {
        if (string.IsNullOrEmpty(name))
        {
            throw new ArgumentNullException("name");
        }

        if (string.IsNullOrEmpty(version))
        {
            throw new ArgumentNullException("version");
        }

        if (objectCreatorFunc == null)
        {
            throw new ArgumentNullException("objectCreatorFunc");
        }

        this.Name = name;
        this.Version = version;
        this.objectCreatorFunc = objectCreatorFunc;
    }

    /// <summary>
    /// Invokes the creator function, thus creating the correct instance for (name, version) provided.
    /// </summary>
    /// <returns>An instance of an orchestration</returns>
    public override TaskOrchestration Create()
    {
        return this.objectCreatorFunc();
    }
}

}

Once we are done configuring the worker, we need to write client differently from default to allow this process to work:
orchestrationInstance = client.CreateOrchestrationInstance("Infinite", "1", "1", input);

The first parameter is the name of the orchestration. Normally, that would be the Type or a Type’s .ToString value. But because we registered our orchestration by the logical name “Infinite”, we shall use that instead.

The second parameter is the version, equal to “1”. This shall be one of the versions that the worker can understand.
The third parameter (also “1”) is the ID. This should be consistent across calls for idempotence considerations.
The last parameter is the input to your orchestration, whose type is dictated by generics.

Finally, don’t forget to update your constants with versions number during deployment: your old orchestration must continue with a new version once it completes.