AvroConvert

Rapid Avro serializer for C# .NET

Docs

Apache Avro format documentation

First steps with Avro in the .NET

AvroConvert documentation

Benefits

Avro format combines readability of JSON and data compression of binary serialization.

	AvroConvert	Apache.Avro	Newtonsoft.Json
Rapid serialization	✔️	✔️	✔️
Low memory allocation	✔️	✔️	✔️
Readable schema of data structure	✔️	✔️	✔️
Support for C# native objects (Dictionary, List, DateTime...)	✔️	❌	✔️
Built-in data encryption	✔️	✔️	❌
Support for compression codecs	Deflate Snappy GZip Brotli	Deflate	❌

Introducing Avro to the projects brings three main benefits:

• Reduction of data size and storage cost
• Decrease of the communication time and the network traffic between microservices
• Increased security - the data is not visible in plain text format

Article describing Avro format specification and Avro API idea: https://www.c-sharpcorner.com/blogs/avro-rest-api-as-the-evolution-of-json-based-communication-between-mic

Conclusion:
Using Avro for communication between your services significantly reduces data size and network traffic. Additionally choosing encoding (compression algorithm) can improve the results even further.

Full Changelog

Features

• Serialization

 byte[] avroObject = AvroConvert.Serialize(object yourObject);

• Deserialization

CustomClass deserializedObject = AvroConvert.Deserialize<CustomClass>(byte[] avroObject);

• Read schema from Avro object

string schemaInJsonFormat = AvroConvert.GetSchema(byte[] avroObject)

• Deserialization of large collection of Avro objects one by one

using (var reader = AvroConvert.OpenDeserializer<CustomClass>(new MemoryStream(avroObject)))
{
    while (reader.HasNext())
    {
        var item = reader.ReadNext();
        // process item
    }
}

• Generation of C# models from Avro file or schema

  string resultModel = AvroConvert.GenerateModel(avroObject);

• Conversion of Avro to JSON directly

  var resultJson = AvroConvert.Avro2Json(avroObject);

Extended list:

1. Serialization of .NET object to Avro format
   • Standard
   • Excluding header
2. Deserialization of Avro data to .NET object
   • Standard
   • Excluding header
   • Processing collection items one by one
3. Other Avro data related operations
   • Merge Avro objects
   • JSON to Avro conversion
   • Avro to JSON conversion
4. Schema related
   • Get schema from Avro data
   • Generate schema from .NET object
   • Generate .NET model from Avro data or schema
   • Generate Avro schema for JSON data

Full documentation

License

AvroConvert is licensed under the CC BY-NC-SA 3.0 license.

For commercial use, please purchase a license.

Contribution

We want to improve AvroConvert as much as possible. If you have any idea, found next possible feature, optimization opportunity or better way for integration, leave a comment or pull request.

Thank you a million to all the contributors to the library, including those that raise issues, started conversations, and those who send pull requests. Thank you!

These amazing people have contributed to AvroConvert:

Related Work

• - online Avro Schema to C# model converter

• - Library containing functionalities, which enable communication between microservices via Http using Avro data format

• - Library containing components needed for Confluent Kafka integration

• Avro API article