I've noticed a couple of threads on save system but I wanted to start a more theoretical debate, not the usual how do you serialize a class in C++.

To give a bit of background on the problem - let's have a game with many types of objects and each type of object saving a different set of information about itself, even maybe different between objects of the same type. (One example can be a container which has different sizes). I've used a binary, linear dump of variables until now: clean read and write of variable data with symmetrical Read/Write object methods. A versioning value is used to handle backwards compatibility while reading.

However this approach quickly spirals into utter madness. Every time I need to add a byte with some extra info for an object I have to add a special case for the save versions before having that byte. The nightmare only goes worse if I am to remove a byte from the opaque save structure.

Hence I've started thinking if there might be a “cleaner” solution. I know I could go full text and pair values with labels and that would fix the structure change problem (json/xml/etc) but honestly I would rather stay in the binary data area with no text labels and no text conversions.

One approach that is crossing my mind is to organize the whole save into clearly delimited “blobs” of information. Each object might have it's blob, marked with it's size in the file. Right now as I read binary from the file missing a byte or reading an extra byte miss-aligns everything after. With the blobs I could evade this issue and just handle in each object's save/load logic the structure discrepancies between save generations. I could maybe even make highly volatile objects text structures and keep binary data for mature “frozen” data structure.

Another approach that crossed my mind is to prefix every binary variable saved in the file with an index value. Create something to a symbol table of sorts. While an object reads its information it will read first that prefix index value. If the index has become obsolete it can just skip it. In the end any indexes missing from the file read can be reset to some default value. That should allow me to change the save structure with no care in the world from one version to another as long as I just increment the prefix indexes as I add more info or I just stop writing an obsolete index in the file. The system can work with a stack of sorts to allow objects inside objects. The logic might be on the line:

Read Index;

Find Index in saved information table;

Read the specific information of the index or skip it if it's obsolete.

Inside the specific information I would use the objects own Index table and rinse and repeat.

I am curious if anyone else had tried these paths or has any other suggestions to creating a cool flexible and robust binary save system.

It doesn't matter if you try to serialize something or write a savegame manager, you always have to target one problem: How to identify your data. This basic kind of a problem is usually solved with loader classes. A typical solution of mine is to give some ID to every object class and put that as the first 4 bytes into the stream. If you want your serialized data to be optimized in size, you can use variable int types like in a database but it isn't necessary for now.

Now you have the choice between trust in the loader class or add some security bytes into the data. From my experience in active development, such a trust is making more trouble than it'll solve in the end so either do some security checks or go for a chunk based solution. I used such a chunk based solution when working with database so content could grow and shrink without issues but for the savegame, I'd suggest security bytes instead. Keep in mind that you never serialize a container “as is” but usually add the amount of items to the head and then item bytes.

It should always be a goal for a game to be backwards compatible. It is the most annoying thing in a players world when you path the game frequently and a player looses hours of time invested into it so keep that in mind. Google Protobuf is a good source for reference also. What I use is something similar; a serialization system which provides IDs to a classes properties. Such IDs are read from the stream of data and it is performed a lookup if the property still exists or needs to be skipped. I also add the length of serialized data after each property ID to be able to skip the data as I otherwise don't know which kind of data it is and how long the binary serialized data needs to be. Let's imagine you renamed/removed an object class so that the binary form doesn't match the current structure anymore. Putting the length of the blog to the stream really helps to not get into mismatching indices.

Finally I add a type ID to everything I serialize as the first few bytes of the data. This is an additional hint to check if the type still matches the type once serialized