Serialization System

[randomPoison]

Having a proper serialization system would help a bunch in stabilizing hotloading, which in turn would make the game development process much nicer. Something like MessagePack would be good for making sure we can reload the game state even if the declaration of serialized structs have changed (which currently leads to segfaults when hotloading).

Requirements are as follows:

• Support fast in-memory serialization and deserialization of large chunks of binary data. The largest chunks of data are in the component managers, which are almost always one or more large arrays of data. Being able to quickly blast these out to memory and bring them back in is important.
• At least recognize when a types contents/layout has changed. This is necessary to gracefully handle changes in data layout without segfaulting.

Super star goals:

• Automatically handle rearranging/changing struct data if type changed between serialization and deserialization. For example, if a user adds a new member to one of their components with hotloading enabled it would be great to be able to automatically add that new member in while deserialization so long as there is a reasonable default to be used.

[randomPoison]

I think this will have to get promoted to a milestone. It's too large a goal be to summed up in a single task, and there are still design questions that haven't been answered. I'm going to leave this here as a placeholder, but once work on this starts it should be moved to a milestone and broken into more manageable tasks.

[randomPoison]

It's worth looking at the way Serde implements it since it's pretty much the Rust standard for serialization (and for good reason, it looks really good! If I weren't committed from doing everything from scratch I would totally just drop Serde in and use that). The clever method that Serde uses is to implement a visitor pattern (which is basically the only way to serialize static types as far as I can tell) but the serialize() and deserialize() accept Serializer and Deserializer trait objects, meaning that you can swap your serialization target at runtime. I'd like to try something similar to this, but with static dispatch of the visitor pattern through generics. I'm guessing that the better method will be the one Serde takes as it is more flexible for only a minor performance cost, but it's still worth exploring a different route since we can always add the more dynamic method in through another set of traits.