Requirements for BinaryType template parameter?

[epvbergen]

Hi, thank you for an excellent piece of infrastructure.

I'm trying to use a custom BinaryType with basic_json. However I've been unsucessful in passing anything other than std::vector<uint8_t> there. Even a plain std::vector<intptr_t> fails to compile with nlohmann::json 3.8.0 and GCC 9.3.0.

Can you say anything more regarding the requirements for BinaryType? I understand it needs to be a sequence container as you want to provide the option of dumping as an array instead of specialized serialization. But what about the container's value type? What restrictions are there by design and what restrictions are there by accident?

Second question: if a conversion from binary to JSON array wouldn't have been required, which I understand is only a debugging feature, couldn't the type requirements be relaxed so that the BinaryType doesn't need to implement a full STL container interface?

Background:

One of my biggest uses for nlohmann::json is its capability to easily store and pass arbitrarily nested trees of named variant values, to slice them and access their values in a type safe manner. An universal, structured, dynamically typed value, like a Python dict or a Javascript object. Serialization and parsing are only secondary and only used whenever the data leaves the process.

Even if supplying a recursive variant datastructure with a very user friendly and STL-compatible API is not the intended use for this library, it does this very well. Duplicating all that in something home grown because "nlohmann::json is intended for parsing and serialization of JSON and JSON-like data structures" seems like a waste of effort, especially considering how much of nlohmann::json in terms of LOC isn't concerned with serialization or parsing but with representing the parsed data and providing a nice interface for that.

From this perspective it's becoming a burden that basic_json insists it owns its data and cannot own references that point inside buffers owned elsewhere, neither the buffers they were parsed from, nor user supplied data. I've been looking at ways to parameterize one of the value types, especially the recently introduced BinaryType, to see if I can make that happen.

But as said, I can't seem to put anything other than std::vector<uint8_t> there. Even other integral types in the vector don't work.

As far as implementation is concerned, could it be an idea to move responsibility for (de)serialization and dumping to the binary subtype wrapper (binary_t = nlohmann::byte_container_with_subtype) that was recently added, and using that wrapper as the BinaryType parameter?

A default BinaryWrapper which users can use as a base class could provide the serializations for the existing serializations and subtypes.

This change would allow zero-copy parsing without SAX and could provide a way for the user to implement even more data types of the various JSON supersets to be implemented, without putting that burden on the library itself.