[Question] How to interrupt the streaming output?

[MayDay-wpf]

Here is a part of my code

Currently, I am using Redis to save a variable in order to interrupt the output operation. However, in reality, the request is not interrupted. Is there any way to stop the streaming output? If so, could you please guide me on how to use it? Thank you very much.

[kayhantolga]

Hi, you can pass a cancellation token and cancel the token if you want to stop streaming. Here is a sample from the playground (the sample is for legacy completion, but chat completion should work the same way).

public static async Task RunSimpleCompletionStreamTestWithCancellationToken(IOpenAIService sdk)
{
    ConsoleExtensions.WriteLine("Completion Stream Testing is starting:", ConsoleColor.Cyan);
    var cancellationToken = new CancellationTokenSource(TimeSpan.FromSeconds(2)).Token;
    cancellationToken.Register(() => Console.WriteLine("Cancellation Token has been cancelled."));

    try
    {
        ConsoleExtensions.WriteLine("Completion Stream Test:", ConsoleColor.DarkCyan);
        var completionResult = sdk.Completions.CreateCompletionAsStream(new CompletionCreateRequest
        {
            Prompt = "Once upon a time",
            MaxTokens = 100
        }, Models.Davinci, cancellationToken);

        try
        {
            await foreach (var completion in completionResult.WithCancellation(cancellationToken))
            {
                if (completion.Successful)
                {
                    Console.Write(completion.Choices.FirstOrDefault()?.Text);
                }
                else
                {
                    if (completion.Error == null)
                    {
                        throw new Exception("Unknown Error");
                    }

                    Console.WriteLine($"{completion.Error.Code}: {completion.Error.Message}");
                }
            }
        }
        catch (OperationCanceledException)
        {
            ConsoleExtensions.WriteLine("Operation Cancelled", ConsoleColor.Green);
        }

        Console.WriteLine("");
        Console.WriteLine("Complete");
    }
    catch (Exception e)
    {
        Console.WriteLine(e);
        throw;
    }
}