Is there a binary serialization class ?
Hi, Is there a binary serialization class in any of the boost file lib ? Also, how do I re-target and compile the boost libraries with the newly released Visual Studio Express edition ( VC 8.0 ) ? Thank in advance. Raja
what would a binary serialization class do? At Sunday 2004-08-29 13:08, you wrote:
Hi, Is there a binary serialization class in any of the boost file lib ?
Also, how do I re-target and compile the boost libraries with the newly released Visual Studio Express edition ( VC 8.0 ) ?
Thank in advance.
Raja
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"
Victor A. Wagner Jr. <vawjr <at> rudbek.com> writes:
what would a binary serialization class do?
To archive the class object data to be used at a later time in a file, and to not let the users simply read the contents. Also to easily apply binary encode/decode schemes that I have, to the data just before I serialize/de-serialize.
On Tue, Aug 31, 2004 at 12:42:17AM +0000, Raj wrote:
what would a binary serialization class do?
To archive the class object data to be used at a later time in a file, and to not let the users simply read the contents.
A general "binary serialization class" would never know where builtin-variables begin and end (or what type they are) and therefore cannot swap bytes on Big-Endian machines. Opens his eggs on the little size, -- Carlo Wood <carlo@alinoe.com>
Carlo Wood wrote:
On Tue, Aug 31, 2004 at 12:42:17AM +0000, Raj wrote:
what would a binary serialization class do?
To archive the class object data to be used at a later time in a file, and to not let the users simply read the contents.
A general "binary serialization class" would never know where builtin-variables begin and end (or what type they are) and therefore cannot swap bytes on Big-Endian machines.
You may be right, depending on the meaning of "general", but useful binary serializers do exist. You just need to decide how to represent the built-in types. For example, I've chosen that the external representation of a char is 8 bits, a short 16, int and long 32. This is perfectly portable, as long as the values of my variables do not exceed these limits.
On Tue, Aug 31, 2004 at 02:27:02PM +0300, Peter Dimov wrote:
A general "binary serialization class" would never know where builtin-variables begin and end (or what type they are) and therefore cannot swap bytes on Big-Endian machines.
You may be right, depending on the meaning of "general", but useful binary serializers do exist. You just need to decide how to represent the built-in types. For example, I've chosen that the external representation of a char is 8 bits, a short 16, int and long 32. This is perfectly portable, as long as the values of my variables do not exceed these limits.
What does size have to do with endianness? if you have the following struct: struct Data { char c1; char lt[3]; int s; unsigned short p1; unsigned short p2; bool init; bool flags[5]; unsigned char t[8]; }; Then how would your general (== does not know anything about the internals of 'Data') serializer write that to a TCP/IP socket when running on a Big-Endian machine, such that both Big-Endian and Little-Endian machines can read the result from the network? -- Carlo Wood <carlo@alinoe.com>
Carlo Wood wrote:
On Tue, Aug 31, 2004 at 02:27:02PM +0300, Peter Dimov wrote:
A general "binary serialization class" would never know where builtin-variables begin and end (or what type they are) and therefore cannot swap bytes on Big-Endian machines.
You may be right, depending on the meaning of "general", but useful binary serializers do exist. You just need to decide how to represent the built-in types. For example, I've chosen that the external representation of a char is 8 bits, a short 16, int and long 32. This is perfectly portable, as long as the values of my variables do not exceed these limits.
What does size have to do with endianness? if you have the following struct:
struct Data { char c1; char lt[3]; int s; unsigned short p1; unsigned short p2; bool init; bool flags[5]; unsigned char t[8]; };
Then how would your general (== does not know anything about the internals of 'Data') serializer write that to a TCP/IP socket when running on a Big-Endian machine, such that both Big-Endian and Little-Endian machines can read the result from the network?
No way. But my serializer isn't "general" by your definition (endianness is only one of the problems, there's also sizeof() and padding). It is an ordinary boost::serialization-style serializer that happens to use a binary external format.
On Tue, Aug 31, 2004 at 05:04:48PM +0300, Peter Dimov wrote:
Carlo Wood wrote:
On Tue, Aug 31, 2004 at 02:27:02PM +0300, Peter Dimov wrote:
A general "binary serialization class" would never know where builtin-variables begin and end (or what type they are) and therefore cannot swap bytes on Big-Endian machines.
You may be right, depending on the meaning of "general", but useful binary serializers do exist. You just need to decide how to represent the built-in types. For example, I've chosen that the external representation of a char is 8 bits, a short 16, int and long 32. This is perfectly portable, as long as the values of my variables do not exceed these limits.
What does size have to do with endianness? if you have the following struct:
struct Data { char c1; char lt[3]; int s; unsigned short p1; unsigned short p2; bool init; bool flags[5]; unsigned char t[8]; };
Then how would your general (== does not know anything about the internals of 'Data') serializer write that to a TCP/IP socket when running on a Big-Endian machine, such that both Big-Endian and Little-Endian machines can read the result from the network?
No way. But my serializer isn't "general" by your definition (endianness is only one of the problems, there's also sizeof() and padding). It is an ordinary boost::serialization-style serializer that happens to use a binary external format.
Hmm, since this subject is coming up here and has also just popped up on comp.lang.c++.moderated, I'll venture an offering. A little over a year ago, I wrote a general-purpose binary packing/unpacking class (inspired by Perl and Python's pack() and unpack()) that has held up pretty well in production for over a year now. I've always wanted to ask my company for permission to release it into the public domain (maybe even into Boost?!?), but haven't pushed it as I'd not heard anyone looking for anything quite like it--until now. If anyone's interested, I'll pursue getting permission from my company to open it up ASAP. To solve the endianness, size, and padding problems, the approach I'd take with my class would be to make sure the protocol expected a single character indicating the byte order of the data stream first; then you'd grab the appropriate unpacker from the factory and start unpacking away. It's up to the protocol writer to know how to unpack the bytes in the proper order into the proper structures from there. Of course, packing/unpacking isn't quite the same as serialization, I'm sure, but maybe serialization isn't exactly the term that applies here? Mike
Hi Mike, I think we could all use that class, especially if it comes with boost. Let us know what happens and thanks :) George
Hmm, since this subject is coming up here and has also just popped up on comp.lang.c++.moderated, I'll venture an offering.
A little over a year ago, I wrote a general-purpose binary packing/unpacking class (inspired by Perl and Python's pack() and unpack()) that has held up pretty well in production for over a year now. I've always wanted to ask my company for permission to release it into the public domain (maybe even into Boost?!?), but haven't pushed it as I'd not heard anyone looking for anything quite like it--until now.
If anyone's interested, I'll pursue getting permission from my company to open it up ASAP.
To solve the endianness, size, and padding problems, the approach I'd take with my class would be to make sure the protocol expected a single character indicating the byte order of the data stream first; then you'd grab the appropriate unpacker from the factory and start unpacking away. It's up to the protocol writer to know how to unpack the bytes in the proper order into the proper structures from there.
Of course, packing/unpacking isn't quite the same as serialization, I'm sure, but maybe serialization isn't exactly the term that applies here?
Mike
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Wed, 1 Sep 2004, Georgios Diamantopoulos wrote:
Hi Mike,
I think we could all use that class, especially if it comes with boost.
Let us know what happens and thanks :)
George
Hmm, since this subject is coming up here and has also just popped up on comp.lang.c++.moderated, I'll venture an offering.
A little over a year ago, I wrote a general-purpose binary packing/unpacking class (inspired by Perl and Python's pack() and unpack()) that has held up pretty well in production for over a year now. I've always wanted to ask my company for permission to release it into the public domain (maybe even into Boost?!?), but haven't pushed it as I'd not heard anyone looking for anything quite like it--until now.
If anyone's interested, I'll pursue getting permission from my company to open it up ASAP.
To solve the endianness, size, and padding problems, the approach I'd take with my class would be to make sure the protocol expected a single character indicating the byte order of the data stream first; then you'd grab the appropriate unpacker from the factory and start unpacking away. It's up to the protocol writer to know how to unpack the bytes in the proper order into the proper structures from there.
Of course, packing/unpacking isn't quite the same as serialization, I'm sure, but maybe serialization isn't exactly the term that applies here?
CORBA (omg.org) uses CDR (Common Data Representation) in its transport protocol. In the (free) ACE framework (http://www.cs.wustl.edu/~schmidt/ACE.html) it is generally used for sending binary data over the wire. Since CDR is a standard and there are many implementations of it it might be an alternative. Volker -- Volker Börchers TECON Systems AG, http://www.tecon.de
On Fri, 3 Sep 2004 16:11:32 +0200 (CEST), Volker Boerchers wrote
On Wed, 1 Sep 2004, Georgios Diamantopoulos wrote: CORBA (omg.org) uses CDR (Common Data Representation) in its transport protocol. In the (free) ACE framework (http://www.cs.wustl.edu/~schmidt/ACE.html) it is generally used for sending binary data over the wire.
Since CDR is a standard and there are many implementations of it it might be an alternative.
During the second serialization review there were proposals, discussions, and some code of a CDR type archive that would be portable. This code was more of a demonstration, and hasn't been refined to the point where it can be included in the library, but it is something someone that really has a need should do... Jeff
On Sep 3, 2004, at 1:06 PM, Jeff Garland wrote:
On Fri, 3 Sep 2004 16:11:32 +0200 (CEST), Volker Boerchers wrote
On Wed, 1 Sep 2004, Georgios Diamantopoulos wrote: CORBA (omg.org) uses CDR (Common Data Representation) in its transport protocol. In the (free) ACE framework (http://www.cs.wustl.edu/~schmidt/ACE.html) it is generally used for sending binary data over the wire.
Since CDR is a standard and there are many implementations of it it might be an alternative.
During the second serialization review there were proposals, discussions, and some code of a CDR type archive that would be portable. This code was more of a demonstration, and hasn't been refined to the point where it can be included in the library, but it is something someone that really has a need should do...
Hi, But if CDR is not used then do you guys know what is ? Because, please correct me if I am wrong (I am just reading and learning with this thread), isn't this functionality (machine agnostic serialization) crucial to frameworks like ACE and to specifications like CORBA and to any cross-platform network communication library ? Thank you for your attention, Mauricio Gomes Pensar Digital phone: 55-11-3803-9707 mobile: 55-11-8319-9610 http://pensardigital.com
On Fri, 3 Sep 2004 15:02:43 -0300, Mauricio Gomes wrote
On Sep 3, 2004, at 1:06 PM, Jeff Garland wrote: Hi,
But if CDR is not used then do you guys know what is ?
Because, please correct me if I am wrong (I am just reading and learning with this thread), isn't this functionality (machine agnostic serialization) crucial to frameworks like ACE and to specifications like CORBA and to any cross-platform network communication library ?
Yes, CDR is Common Data Representation and it is part of the CORBA standard. It deals with things like byte swapping etc. ACE has a CDR Stream implementation that is used to stream data for the TAO CORBA implementation. The serialization library uses a 'local' binary format so the binary archive cannot necessarily be ported to another machine or even possibly another compiler on the same machine. This is likely a much more performant option than a CDR binary archive for those applications that use a single machine/compiler. So, the bottom line is, the serialization library can be extended to support CDR, but as far as I know no one is working on actually doing it... Jeff
On Sep 3, 2004, at 3:25 PM, Jeff Garland wrote:
On Fri, 3 Sep 2004 15:02:43 -0300, Mauricio Gomes wrote
On Sep 3, 2004, at 1:06 PM, Jeff Garland wrote: Hi,
But if CDR is not used then do you guys know what is ?
Because, please correct me if I am wrong (I am just reading and learning with this thread), isn't this functionality (machine agnostic serialization) crucial to frameworks like ACE and to specifications like CORBA and to any cross-platform network communication library ?
Yes, CDR is Common Data Representation and it is part of the CORBA standard. It deals with things like byte swapping etc. ACE has a CDR Stream implementation that is used to stream data for the TAO CORBA implementation.
The serialization library uses a 'local' binary format so the binary archive cannot necessarily be ported to another machine or even possibly another compiler on the same machine. This is likely a much more performant option than a CDR binary archive for those applications that use a single machine/compiler. So, the bottom line is, the serialization library can be extended to support CDR, but as far as I know no one is working on actually doing it...
Jeff
Sorry I think I am transforming this in a thread more suitable for an ACE list but I can't resist this last comment/question. So ... does this mean that I canNOT use ACE/TAO to make my PowerPC G4 Mac talk to my Intel Pentium PC ? Disappointing. Thank you for the information Jeff. Mauricio Gomes Pensar Digital http://pensardigital.com On Sep 3, 2004, at 3:25 PM, Jeff Garland wrote:
On Fri, 3 Sep 2004 15:02:43 -0300, Mauricio Gomes wrote
On Sep 3, 2004, at 1:06 PM, Jeff Garland wrote: Hi,
But if CDR is not used then do you guys know what is ?
Because, please correct me if I am wrong (I am just reading and learning with this thread), isn't this functionality (machine agnostic serialization) crucial to frameworks like ACE and to specifications like CORBA and to any cross-platform network communication library ?
Yes, CDR is Common Data Representation and it is part of the CORBA standard. It deals with things like byte swapping etc. ACE has a CDR Stream implementation that is used to stream data for the TAO CORBA implementation.
The serialization library uses a 'local' binary format so the binary archive cannot necessarily be ported to another machine or even possibly another compiler on the same machine. This is likely a much more performant option than a CDR binary archive for those applications that use a single machine/compiler. So, the bottom line is, the serialization library can be extended to support CDR, but as far as I know no one is working on actually doing it...
Jeff
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Fri, 3 Sep 2004 16:08:41 -0300, Mauricio Gomes wrote
Sorry I think I am transforming this in a thread more suitable for an ACE list but I can't resist this last comment/question. So ... does this mean that I canNOT use ACE/TAO to make my PowerPC G4 Mac talk to my Intel Pentium PC ? Disappointing.
Actually it sounds like I've confused you. Yes, you absolutely CAN use ACE/TAO to talk cross-platform/cross-language seemlessly using CDR. What you CAN'T do is use boost.serialization to create a binary file on one platform that you can read on the other because it doesn't have a CDR archiver. And there is obviously no tie between ACE/TAO and boost.serialization. And, yes, this is getting off-topic.
Thank you for the information Jeff.
Sure, HTH Jeff
participants (1)
-
Carlo Wood -
Georgios Diamantopoulos -
Jeff Garland -
Mauricio Gomes -
Mike Bland -
Peter Dimov -
Raj -
Victor A. Wagner Jr. -
Volker Boerchers