[Bug Report]: JsonSerializerOptions is not forwarded to the JsonWriter

[fabionaspolini]

Prerequisites

• I have searched issues to ensure it has not already been reported

Description

Part of the serializer options are being lost when creating Utf8JsonWriter at this point:

https://github.com/Farfetch/kafkaflow/blob/3561fe6334e8d090963fbd1491dee1e09e743440/src/KafkaFlow.Serializer.JsonCore/JsonCoreSerializer.cs#L35C42-L35C42

Utf8JsonWriter class receive a JsonWriterOptions argument, and some properties need by pass here.

Example: Encoder specified in JsonSerializerOptions is ignore because use Utf8JsonWriter to writer

Steps to reproduce

var serializerOptions = new JsonSerializerOptions
{
    Encoder = System.Text.Encodings.Web.JavaScriptEncoder.UnsafeRelaxedJsonEscaping // IS IGNORED
};

var services = new ServiceCollection();
services.AddKafka(kafka => kafka
    .UseConsoleLog()
    .AddCluster(cluster => cluster
        .WithBrokers(new[] { "localhost:9092" })
        .CreateTopicIfNotExists(TopicName, 10, 1)
        .AddProducer(
            name: "my-producer",
            producer => producer
                .DefaultTopic(TopicName)
                .AddMiddlewares(m => m.AddSerializer(x => new JsonCoreSerializer(serializerOptions)))))); // Custom json serializer ioptions
var serviceProvider = services.BuildServiceProvider();
var producer = serviceProvider.GetRequiredService<IProducerAccessor>().GetProducer("my-producer");
var message = new HelloMessage("Olá!!!");
await producer.ProduceAsync(TopicName, "1", message);

Expected behavior

Use encoder configured in JsonSerializerOptions (UnsafeRelaxedJsonEscaping) and generate special characters in serialization.

 {"Text":"Olá!!!"}

Actual behavior

Output is generated by default encoder and ignore custom configuration.

 {"Text":"Ol\u00E1!!!"}

KafkaFlow version

2.4.0

[erik-catalao]

Hello @fabionaspolini, I've been looking into this issue, but I'm unable to reproduce it locally.

Could you show me the way you are configuring your consumer? I've set it up similarly as you, using a producer and a consumer with the same JsonCoreSerializer and the same options, and it seems to function without issues.

[joelfoliveira]

Hi @fabionaspolini, the serializer options were being used by the JsonCoreSerializer, the issue was that the escaping of those characters was being done when writing the serialization result to the stream.

We created a Pull Request that allows to pass a JsonWriteOptions instance to the JsonCoreSerializer. You can check the PR #425.

To use this you just need to change your snippet to the following. Please confirm that this fixes your issue.

var writerOptions = new JsonWriterOptions
{
    Encoder = JavaScriptEncoder.UnsafeRelaxedJsonEscaping
};

var services = new ServiceCollection();
services.AddKafka(kafka => kafka
    .UseConsoleLog()
    .AddCluster(cluster => cluster
        .WithBrokers(new[] { "localhost:9092" })
        .CreateTopicIfNotExists(TopicName, 10, 1)
        .AddProducer(
            name: "my-producer",
            producer => producer
                .DefaultTopic(TopicName)
                .AddMiddlewares(m => m.AddSerializer(x => new JsonCoreSerializer(writerOptions)))))); // Custom json serializer ioptions
var serviceProvider = services.BuildServiceProvider();
var producer = serviceProvider.GetRequiredService<IProducerAccessor>().GetProducer("my-producer");
var message = new HelloMessage("Olá!!!");
await producer.ProduceAsync(TopicName, "1", message);