simple-protobuf

simple data struct serialization library for C++. With this library you can serialize and deserialize POD C++ structs directly to JSON or protobuf. When used together with etl library it doesn't need to allocate any memory, so its suitable for embedded environments (see extensions).

namespace PhoneBook
{
struct Person {
    enum class PhoneType : int32_t {
        MOBILE = 0,
        HOME   = 1,
        WORK   = 2,
    };
    struct PhoneNumber {
        // phone number is always required
        etl::string<16> number;
        std::optional< PhoneType > type;
    };
    std::optional< std::string > name;
    // Unique ID number for this person.
    std::optional< int32_t > id;
    std::optional< std::string > email;
    // all registered phones
    std::vector< PhoneNumber > phones;
};
}// namespace PhoneBook

auto john = PhoneBook::Person{
        .name  = "John Doe",
        .id    = 1234,
        .email = "jdoe@example.com",
    };
//- serialize john to json-string
auto json    = spb::json::serialize< std::string >( john );
//- deserialize john from json-string
auto person  = spb::json::deserialize< PhoneBook::Person >( json );
//- serialize john to protobuf-vector
auto pb      = spb::pb::serialize< std::vector< std::byte > >( john );
//- deserialize john from protobuf-vector
auto person2 = spb::pb::deserialize< PhoneBook::Person >( pb );
//- john == person == person2

goal

goal of this library is to make JSON and protobuf part of the C++ language itself.

reason

There are literally a tons of JSON C++ libraries but most of them are designed in a way that the user needs to construct the json Object via some API and for serialization and deserialization there is a lot of boilerplate code like type/schema checking, to_json, from_json, macros... All this is needed to be done by the user, and it usually ends up with a conversion to some C++ struct.

spb works the other way around, from C++ struct to JSON or protobuf. With this approach user can focus only on the data, C++ struct, which is much more natural and spb will handle all the boring stuff like serialization/deserialization and type/schema checking.

about

spb is an alternative implementation of protobuf for C++. This is not an plugin for protoc but an replacement for protoc, so you don't need protobuf or protoc installed to use it. Serialization and deserialization to JSON or protobuf is compatible with protoc, in other words, data serialized with code generated by spb-protoc can be deserialized by code generated by protoc and vice versa.

usage

dependencies

• C++ compiler (at least C++20)
• cmake
• std library
• (optional) clang-format for code formatting

cheat sheet

# add this repo to your project
add_subdirectory(external/spb-proto)
# compile proto files to C++
spb_protobuf_generate(PROTO_SRCS PROTO_HDRS ${CMAKE_SOURCE_DIR}/proto/person.proto)
# add generated files to your project
add_executable(myapp ${PROTO_SRCS} ${PROTO_HDRS})
# `spb-proto` is an interface library 
# the main purpose is to update include path of `myapp`
target_link_libraries(myapp PUBLIC spb-proto)

how to use

1. define a schema for you data in a person.proto file

package PhoneBook;

message Person {
  optional string name = 1;
  optional int32 id = 2;  // Unique ID number for this person.
  optional string email = 3;

  enum PhoneType {
    MOBILE = 0;
    HOME = 1;
    WORK = 2;
  }

  message PhoneNumber {
    required string number = 1; // phone number is always required
    optional PhoneType type = 2;
  }
  // all registered phones
  repeated PhoneNumber phones = 4;
}

2. compile person.proto with spb-protoc into person.pb.h and person.pb.cc

spb_protobuf_generate(PROTO_SRCS PROTO_HDRS ${CMAKE_SOURCE_DIR}/proto/person.proto)

observe the beautifully generated person.pb.h and person.pb.cc

namespace PhoneBook
{
struct Person {
    enum class PhoneType : int32_t {
        MOBILE = 0,
        HOME   = 1,
        WORK   = 2,
    };
    struct PhoneNumber {
        // phone number is always required
        std::string number;
        std::optional< PhoneType > type;
    };
    std::optional< std::string > name;
    // Unique ID number for this person.
    std::optional< int32_t > id;
    std::optional< std::string > email;
    // all registered phones
    std::vector< PhoneNumber > phones;
};
}// namespace PhoneBook

3. use Person struct natively and de/serialize to/from json/pb

#include <person.pb.h>

auto john = PhoneBook::Person{
        .name  = "John Doe",
        .id    = 1234,
        .email = "jdoe@example.com",
    };

auto json    = spb::json::serialize( john );
auto person  = spb::json::deserialize< PhoneBook::Person >( json );
auto pb      = spb::pb::serialize( john );
auto person2 = spb::pb::deserialize< PhoneBook::Person >( pb );
//- john == person == person2

API

All generated messages (and enums) are using the following API include/spb/json.hpp and include/spb/pb.hpp

//- serialize message via writer (all other `serialize` are just wrappers around this one)
//- example: `auto serialized_size = spb::pb::serialize( message, my_writer );`
auto serialize( const auto & message, spb::io::writer on_write ) -> size_t;

//- return size in bytes of serialized message
//- example: `auto serialized_size = spb::pb::serialize_size( message );`
auto serialize_size( const auto & message ) -> size_t;

//- serialize message into container like std::string, std::vector, ...
//- example: `auto serialized_size = spb::pb::serialize( message, my_string );`
template < typename Message, spb::resizable_container Container >
auto serialize( const Message & message, Container & result ) -> size_t;

//- serialize message and return container like std::string, std::vector, ...
//- example: `auto my_string = spb::pb::serialize< std::string >( message );`
template < spb::resizable_container Container = std::string, typename Message >
auto serialize( const Message & message ) -> Container;

//- deserialize message from reader (all other `deserialize` are just wrappers around this one)
//- example: `spb::pb::deserialize( message, my_reader );`
void deserialize( auto & message, spb::io::reader on_read );

//- deserialize message from container like std::string, std::vector, ...
//- example: `spb::pb::deserialize( message, my_string );`
template < typename Message, spb::size_container Container >
void deserialize( Message & message, const Container & protobuf );


//- return deserialized message from container like std::string, std::vector, ...
//- example: `auto message = spb::pb::deserialize< Message >( my_string );`
template < typename Message, spb::size_container Container >
auto deserialize( const Container & protobuf ) -> Message;

//- return deserialized message from reader
//- example: `auto message = spb::pb::deserialize< Message >( my_reader );`
template < typename Message >
auto deserialize( spb::io::reader reader ) -> Message;

API is prefixed with spb::json:: for json and spb::pb:: for protobuf, template concepts spb::size_container and spb::resizable_container are defined in include/spb/concepts.h, spb::io::reader and spb::io::writer are user specified functions for IO, more info at include/io/io.hpp

type mapping

proto type	CPP type	GPB encoding
bool	bool	varint
float	float	4 bytes
double	double	8 bytes
int32	int32_t	varint
sint32	int32_t	zig-zag varint
uint32	uint32_t	varint
int64	int64_t	varint
sint64	int64_t	zig-zag varint
uint64	uint64_t	varint
fixed32	uint32_t	4 bytes
sfixed32	int32_t	4 bytes
fixed64	uint64_t	8 bytes
sfixed64	int64_t	8 bytes
string	std::string	utf8 string
bytes	std::vector< std::byte >	base64 encoded in json

proto type modifier	CPP type modifier	Notes
optional	std::optional<Message>
optional	std::unique_ptr<Message>	if there is cyclic dependency between messages ( A -> B, B -> A )
repeated	std::vector<Message>

See also extensions for user specific types and advanced usage.

example

navigate to the example directory.

status

• Make it work
• Make it right
• Make it fast

roadmap

• parser for proto files (supported syntax: proto2 and proto3)
• compile proto message to C++ data struct
• generate json de/serializer for generated C++ data struct (serialized json has to be compatible with GPB)
• generate protobuf de/serializer for generated C++ data struct (serialized pb has to be compatible with GPB)

missing features

• RPC is not implemented
• extend is not implemented