[Boost.Serialization] (De-)serialization with class versioning
Hi, i use the class versioning feature of Boost.Serialization, with an external (non-intrusive) serialization function like this: BOOST_CLASS_VERSION(GAMEINFO, 1) template <typename Archive> void serialize(Archive &ar, GAMEINFO& gameSettings, const unsigned int version) { ar & gameSettings.NPODS; // ... if (version > 0) ar & gameSettings.ARENA_GOAL_EXCLUSION_RADIUS; } Now as soon, as i set BOOST_CLASS_VERSION(GAMEINFO, 1), i get version==1 in this serialize function, even when i *deserialize* (load from archive). I would have expected to get the version of the data in the saved archive in this case. Say i saved with an old version=0. Then my class evolved and i set BOOST_CLASS_VERSION(GAMEINFO, 1). But when reading the old archive, i expect version==0 in the function so i can react on it. Instead, i get always version==1, as soon as i compile with BOOST_CLASS_VERSION(GAMEINFO, 1), even when i'm reading old files! What am i missing? regards, Lars R.
On Sunday, March 11, 2018, Lars Ruoff via Boost-users < boost-users@lists.boost.org> wrote:
Hi, i use the class versioning feature of Boost.Serialization, with an external (non-intrusive) serialization function like this:
BOOST_CLASS_VERSION(GAMEINFO, 1)
template <typename Archive> void serialize(Archive &ar, GAMEINFO& gameSettings, const unsigned int version) { ar & gameSettings.NPODS; // ... if (version > 0) ar & gameSettings.ARENA_GOAL_EXCLUSION_RADIUS; }
Now as soon, as i set BOOST_CLASS_VERSION(GAMEINFO, 1), i get version==1 in this serialize function, even when i *deserialize* (load from archive). I would have expected to get the version of the data in the saved archive in this case.
Say i saved with an old version=0. Then my class evolved and i set BOOST_CLASS_VERSION(GAMEINFO, 1). But when reading the old archive, i expect version==0 in the function so i can react on it. Instead, i get always version==1, as soon as i compile with BOOST_CLASS_VERSION(GAMEINFO, 1), even when i'm reading old files! What am i missing?
regards, Lars R.
You're not missing anything based on my similar experience. We ended up not using the version functionality because it turned out to be useless. It made rolling out code changes a tedious affair, which if not choreographed carefully would lead to deserialisation changes in the best case, and crashes in the worst. We plan to drop boost serialisation entirely. Maybe someone will point out what we're both doing wrong.
On Sunday, March 11, 2018, james <dirtydroog@gmail.com> wrote:
On Sunday, March 11, 2018, Lars Ruoff via Boost-users < boost-users@lists.boost.org> wrote:
Hi, i use the class versioning feature of Boost.Serialization, with an external (non-intrusive) serialization function like this:
BOOST_CLASS_VERSION(GAMEINFO, 1)
template <typename Archive> void serialize(Archive &ar, GAMEINFO& gameSettings, const unsigned int version) { ar & gameSettings.NPODS; // ... if (version > 0) ar & gameSettings.ARENA_GOAL_EXCLUSION_RADIUS; }
Now as soon, as i set BOOST_CLASS_VERSION(GAMEINFO, 1), i get version==1 in this serialize function, even when i *deserialize* (load from archive). I would have expected to get the version of the data in the saved archive in this case.
Say i saved with an old version=0. Then my class evolved and i set BOOST_CLASS_VERSION(GAMEINFO, 1). But when reading the old archive, i expect version==0 in the function so i can react on it. Instead, i get always version==1, as soon as i compile with BOOST_CLASS_VERSION(GAMEINFO, 1), even when i'm reading old files! What am i missing?
regards, Lars R.
You're not missing anything based on my similar experience. We ended up not using the version functionality because it turned out to be useless. It made rolling out code changes a tedious affair, which if not choreographed carefully would lead to deserialisation changes in the best case, and crashes in the worst. We plan to drop boost serialisation entirely.
Maybe someone will point out what we're both doing wrong.
I meant to type 'deserialisation exceptions' instead of 'deserialisation changes' above
Ok, i realized my mistake: I explicitly called the top level serialization functions giving it the hard-coded version numbers, like: replayArchive = new boost::archive::binary_iarchive(replayFile); CPodballHeader header; serialize<boost::archive::binary_iarchive>(*replayArchive, header, boost::serialization::version<CPodballHeader>::value); serialize<boost::archive::binary_iarchive>(*replayArchive, matchSettings, boost::serialization::version<CMatchSettings>::value); serialize<boost::archive::binary_iarchive>(*replayArchive, gameSettings, boost::serialization::version<GAMEINFO>::value); i have changed the latter calls to *replayArchive & header; *replayArchive & matchSettings; *replayArchive & gameSettings; and now i get the correct old version when reading from old files. This problem solved. However i got some weird exceptions now. What i realized is that the above change seemed to have changed the binary layout of my files. Older version had this dump from head: 16 00 00 00 73 65 72 69 61 6c 69 7a 61 74 69 6f ....serializatio 6e 3a 3a 61 72 63 68 69 76 65 10 00 04 04 04 08 n::archive...... 01 00 00 00 0e 00 00 00 70 6f 64 62 61 6c 6c 2d ........podball- 72 65 70 6c 61 79 00 00 04 02 02 00 00 00 40 1f replay........@. Newer version has this head: 16 00 00 00 73 65 72 69 61 6c 69 7a 61 74 69 6f ....serializatio 6e 3a 3a 61 72 63 68 69 76 65 10 00 04 04 04 08 n::archive...... 01 00 00 00 00 00 00 00 00 0e 00 00 00 70 6f 64 .............pod 62 61 6c 6c 2d 72 65 70 6c 61 79 00 00 04 02 00 ball-replay..... "podball-replay" is the content of a std::string and the very first item to be serialized from CPodballHeader. As you can see there are 5 additional bytes before this in the newer version. This is probably what is causing problems? I guess the bytes are for some bookkeeping, but why has it changed with the way i call things? (PS: I'm using MS Visual Studio Community 2017) On Sun, Mar 11, 2018 at 5:35 PM, james <dirtydroog@gmail.com> wrote:
On Sunday, March 11, 2018, james <dirtydroog@gmail.com> wrote:
On Sunday, March 11, 2018, Lars Ruoff via Boost-users < boost-users@lists.boost.org> wrote:
Hi, i use the class versioning feature of Boost.Serialization, with an external (non-intrusive) serialization function like this:
BOOST_CLASS_VERSION(GAMEINFO, 1)
template <typename Archive> void serialize(Archive &ar, GAMEINFO& gameSettings, const unsigned int version) { ar & gameSettings.NPODS; // ... if (version > 0) ar & gameSettings.ARENA_GOAL_EXCLUSION_RADIUS; }
Now as soon, as i set BOOST_CLASS_VERSION(GAMEINFO, 1), i get version==1 in this serialize function, even when i *deserialize* (load from archive). I would have expected to get the version of the data in the saved archive in this case.
Say i saved with an old version=0. Then my class evolved and i set BOOST_CLASS_VERSION(GAMEINFO, 1). But when reading the old archive, i expect version==0 in the function so i can react on it. Instead, i get always version==1, as soon as i compile with BOOST_CLASS_VERSION(GAMEINFO, 1), even when i'm reading old files! What am i missing?
regards, Lars R.
You're not missing anything based on my similar experience. We ended up not using the version functionality because it turned out to be useless. It made rolling out code changes a tedious affair, which if not choreographed carefully would lead to deserialisation changes in the best case, and crashes in the worst. We plan to drop boost serialisation entirely.
Maybe someone will point out what we're both doing wrong.
I meant to type 'deserialisation exceptions' instead of 'deserialisation changes' above
On 3/11/18 9:44 AM, Lars Ruoff via Boost-users wrote:
Ok, i realized my mistake: I explicitly called the top level serialization functions giving it the hard-coded version numbers, like:
You should never, ever call the serialize function from a user program. This should be called by the serialization libary and only the serialization library. If you find yourself doing this, you are making an error and misunderstanding in some way how the library is meant to be used. Robert Ramey
Thanks Robert, got it. Any ideas about the binary file issue? On Sun, Mar 11, 2018 at 7:41 PM, Robert Ramey via Boost-users < boost-users@lists.boost.org> wrote:
On 3/11/18 9:44 AM, Lars Ruoff via Boost-users wrote:
Ok, i realized my mistake: I explicitly called the top level serialization functions giving it the hard-coded version numbers, like:
You should never, ever call the serialize function from a user program. This should be called by the serialization libary and only the serialization library. If you find yourself doing this, you are making an error and misunderstanding in some way how the library is meant to be used.
Robert Ramey
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users
On 3/11/18 11:56 AM, Lars Ruoff via Boost-users wrote:
Thanks Robert, got it. Any ideas about the binary file issue?
Hmmm - changing the version can be expected to change the data in the archives. Rerun your test from scratch - everything should function smooth as silk. If it doesn't try using a text or even better an xml archive. This will make it much easier to see what is going on. Robert Ramey
On 3/11/18 11:41 AM, Robert Ramey via Boost-users wrote:
On 3/11/18 9:44 AM, Lars Ruoff via Boost-users wrote:
Ok, i realized my mistake: I explicitly called the top level serialization functions giving it the hard-coded version numbers, like:
You should never, ever call the serialize function from a user program. This should be called by the serialization libary and only the serialization library. If you find yourself doing this, you are making an error and misunderstanding in some way how the library is meant to be used.
FYI - I recommend making the intrusive serialize functions private. Then using friend to give the serialization library - and only the serialization library - access to them. This makes it much harder to accidentally misuse the library. Of course this gives no help for non-intrusive serialize functions. Robert Ramey
participants (1)
-
james -
Lars Ruoff -
Robert Ramey