Convert big endian to little endian (floating point values)
Hello, I am trying to convert a floating point value from big endian to little endian, but the code below does not compile: double x = 10.0; const double x_big_endian = boost::endian::native_to_big(x); boost::endian::native_to_big_inplace(x); and the compiler (gcc 5.2) only offers overloads for integer types. This is with boost 1.59. Are floating point values currently supported (the website of boost::endian says so)? Am I doing something wrong? Thank you. Martin
From: Martin Vymazal <martin.vymazal@vki.ac.be> Hello,
I am trying to convert a floating point value from big endian to little
endian, but the code below does not compile:
double x = 10.0;
const double x_big_endian = boost::endian::native_to_big(x); boost::endian::native_to_big_inplace(x);
Be very careful how you swap floats. They aren't like integer data types where the swapped representation can fit nicely into the same type. Not all swapped floats are valid floats, and you'll find that the bits will be messaged into a valid float, thus changing your value without telling you. You can only swap a float into an integer. You can't swap a float into a float.
On 2015-12-08 19:40, Aaron_Wright@selinc.com wrote:
From: Martin Vymazal <martin.vymazal@vki.ac.be> Hello,
I am trying to convert a floating point value from big endian to little endian, but the code below does not compile:
double x = 10.0;
const double x_big_endian = boost::endian::native_to_big(x); boost::endian::native_to_big_inplace(x);
Be very careful how you swap floats. They aren't like integer data types where the swapped representation can fit nicely into the same type. Not all swapped floats are valid floats, and you'll find that the bits will be messaged into a valid float, thus changing your value without telling you. You can only swap a float into an integer. You can't swap a float into a float.
Hi, ok, here's the whole story. We have a code that computes some data, compresses them through zlib and saves as binary. We also use our own utility to postprocess the compressed file and convert it to human-readable format. What happened recently is that the code ran on a big-endian machine and since zlib preserves the endian-ess of the data, it generated a binary with big-endian doubles. We would like to have the possibility of bringing that file on a little-endian machine (say user's laptop) and postprocess there, instead of doing everything on the big-endian machine. Do you have an alternative suggestion how to approach this problem? Let's assume that writing the output data in any other format than the current one is not possible. I am also still interested in knowing whether boost (1.59) supports conversions between endian representations of floating-point values. Martin
On 8 December 2015 at 22:14, Martin Vymazal <martin.vymazal@vki.ac.be> wrote:
Hi,
ok, here's the whole story. We have a code that computes some data, compresses them through zlib and saves as binary. We also use our own utility to postprocess the compressed file and convert it to human-readable format.
What happened recently is that the code ran on a big-endian machine and since zlib preserves the endian-ess of the data, it generated a binary with big-endian doubles. We would like to have the possibility of bringing that file on a little-endian machine (say user's laptop) and postprocess there, instead of doing everything on the big-endian machine. Do you have an alternative suggestion how to approach this problem? Let's assume that writing the output data in any other format than the current one is not possible.
I am also still interested in knowing whether boost (1.59) supports conversions between endian representations of floating-point values.
If you absolutely must, the easiest way to do this is to read the data as if it's an unsigned integer type (of the correct size), convert endianness, then reinterpret_cast (or something similarly horrible) to the floating point type. Of course, not relying on the platform specific bit representation of primitive data types when serializing data is a much better idea. -- Maarten
On 2015-12-08 22:48, Maarten de Vries wrote:
On 8 December 2015 at 22:14, Martin Vymazal <martin.vymazal@vki.ac.be> wrote:
Hi,
ok, here's the whole story. We have a code that computes some data, compresses them through zlib and saves as binary. We also use our own utility to postprocess the compressed file and convert it to human-readable format.
What happened recently is that the code ran on a big-endian machine and since zlib preserves the endian-ess of the data, it generated a binary with big-endian doubles. We would like to have the possibility of bringing that file on a little-endian machine (say user's laptop) and postprocess there, instead of doing everything on the big-endian machine. Do you have an alternative suggestion how to approach this problem? Let's assume that writing the output data in any other format than the current one is not possible.
I am also still interested in knowing whether boost (1.59) supports conversions between endian representations of floating-point values.
If you absolutely must, the easiest way to do this is to read the data as if it's an unsigned integer type (of the correct size), convert endianness, then reinterpret_cast (or something similarly horrible) to the floating point type.
Of course, not relying on the platform specific bit representation of primitive data types when serializing data is a much better idea.
-- Maarten
What would be a possible approach to save the data without 'relying on the platform specific bit representation of primitive data types' as you put it? I can not save the data as plain ascii text, because the files would simply be huge (the're already big as they are). Is there some more intelligent solution? I'm not a software engineer, so I'd be happy to hear what is the common practice here. I will also try one last time: does boost support conversion of floating point types or not? The website says so, but it doesn't work for me. I am not asking because I plan to use this functionality, I'm simply curious since I already started playing with it. Martin
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Not currently -- right from the docs under floating point support it sez... http://www.boost.org/doc/libs/1_59_0/libs/endian/doc/index.html Support for floating point types was removed from Boost 1.58.0 because there was not enough time to resolve reliability concerns. It is expected that floating point support will be available in Boost 1.59.0. Since those are the 1.59 docs I'd infer that it wasn't fixed in 1.59. Jeff On Tue, Dec 8, 2015 at 3:18 PM, Martin Vymazal <martin.vymazal@vki.ac.be> wrote:
On 2015-12-08 22:48, Maarten de Vries wrote:
On 8 December 2015 at 22:14, Martin Vymazal <martin.vymazal@vki.ac.be> wrote:
Hi,
ok, here's the whole story. We have a code that computes some data, compresses them through zlib and saves as binary. We also use our own utility to postprocess the compressed file and convert it to human-readable format.
What happened recently is that the code ran on a big-endian machine and since zlib preserves the endian-ess of the data, it generated a binary with big-endian doubles. We would like to have the possibility of bringing that file on a little-endian machine (say user's laptop) and postprocess there, instead of doing everything on the big-endian machine. Do you have an alternative suggestion how to approach this problem? Let's assume that writing the output data in any other format than the current one is not possible.
I am also still interested in knowing whether boost (1.59) supports conversions between endian representations of floating-point values.
If you absolutely must, the easiest way to do this is to read the data as if it's an unsigned integer type (of the correct size), convert endianness, then reinterpret_cast (or something similarly horrible) to the floating point type.
Of course, not relying on the platform specific bit representation of primitive data types when serializing data is a much better idea.
-- Maarten
What would be a possible approach to save the data without 'relying on the platform specific bit representation of primitive data types' as you put it? I can not save the data as plain ascii text, because the files would simply be huge (the're already big as they are). Is there some more intelligent solution? I'm not a software engineer, so I'd be happy to hear what is the common practice here.
I will also try one last time: does boost support conversion of floating point types or not? The website says so, but it doesn't work for me. I am not asking because I plan to use this functionality, I'm simply curious since I already started playing with it.
Martin
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
On Wed, Dec 9, 2015 at 7:18 AM, Martin Vymazal <martin.vymazal@vki.ac.be> wrote:
What would be a possible approach to save the data without 'relying on the platform specific bit representation of primitive data types' as you put it?
What about using boost integer to find an unsigned integer (let's call its type Lint) longer or equal than the float (type Float) you have and make an union? union Lint_Float_t { Lint store; Float read; }; static_assert(sizeof(Lint) >= sizeof(Float), ""); To store you copy the float in the union, and save the integer. To load you load the integer and copy from the union the float. It is definitely not portable (reading and writing different types in a union is at most implementation dependant) and you keep the original bits, but it should work at least on machine of the same type. Cheers, Paolo
On 12/8/2015 11:32 PM, Paolo Bolzoni wrote:
On Wed, Dec 9, 2015 at 7:18 AM, Martin Vymazal <martin.vymazal@vki.ac.be> wrote:
What would be a possible approach to save the data without 'relying on the platform specific bit representation of primitive data types' as you put it?
What about using boost integer to find an unsigned integer (let's call its type Lint) longer or equal than the float (type Float) you have and make an union?
Ugh, this again? Third time this week... just no.
union Lint_Float_t { Lint store; Float read; }; static_assert(sizeof(Lint) >= sizeof(Float), "");
To store you copy the float in the union, and save the integer. To load you load the integer and copy from the union the float.
It is definitely not portable (reading and writing different types in a union is at most implementation dependant) and you keep the original bits, but it should work at least on machine of the same type.
Reading from an inactive member of a union is plain and simply undefined behavior. To read and write the bits of the `float` in a well defined way use `std::memcpy`. Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com
Reading from an inactive member of a union is plain and simply undefined behavior. To read and write the bits of the `float` in a well defined way use `std::memcpy`.
Curiously, it is undefined in C++, whereas well-defined in C: "6.5.2.3 Structure and union members If the member used to read the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called ‘‘type punning’’). This might be a trap representation." Since C++ includes C as a normative reference it would be a good idea to allow it also in C++. memcpy is fine if the compiler optimizes it as an intrinsic, but inefficient in some implementations to copy only a couple of bytes. Best, Ion
On 12/11/2015 4:54 PM, Ion Gaztañaga wrote:
Reading from an inactive member of a union is plain and simply undefined behavior. To read and write the bits of the `float` in a well defined way use `std::memcpy`.
Curiously, it is undefined in C++, whereas well-defined in C:
"6.5.2.3 Structure and union members
If the member used to read the contents of a union object is not the same as the member last used to store a value in the object, the appropriate part of the object representation of the value is reinterpreted as an object representation in the new type as described in 6.2.6 (a process sometimes called ‘‘type punning’’). This might be a trap representation."
Since C++ includes C as a normative reference it would be a good idea to allow it also in C++. memcpy is fine if the compiler optimizes it as an intrinsic, but inefficient in some implementations to copy only a couple of bytes.
C has *vastly* weaker aliasing rules than C++ (to the point that several C implementations choose to follow the C++ rules instead). For more details see the thread at http://lists.boost.org/Archives/boost/2015/11/226663.php If your implementation of choice gives you poor codegen for `memcpy` then I'll suggest you file a bug against it. We would all benefit from that. Regards, -- Agustín K-ballo Bergé.- http://talesofcpp.fusionfenix.com
On 8 December 2015 at 23:18, Martin Vymazal <martin.vymazal@vki.ac.be> wrote:
On 2015-12-08 22:48, Maarten de Vries wrote:
Of course, not relying on the platform specific bit representation of primitive data types when serializing data is a much better idea.
-- Maarten
What would be a possible approach to save the data without 'relying on the platform specific bit representation of primitive data types' as you put it? I can not save the data as plain ascii text, because the files would simply be huge (the're already big as they are). Is there some more intelligent solution? I'm not a software engineer, so I'd be happy to hear what is the common practice here.
Martin
If you're interested: You can still use a binary format, though it depends on your needs. You could look at boost serialization [1]. I've never worked with it, so I can't comment on performance (either in terms of speed or serialized size). But in theory it solves the problem of saving and loading data to/from files (and more). Another option might be Binn [2]. It's C but it looks simple and clean enough (although I also never worked with it). [1] http://www.boost.org/doc/libs/1_59_0/libs/serialization/doc/index.html [2] https://github.com/liteserver/binn
participants (2)
-
Aaron_Wright＠selinc.com -
Agustín K-ballo Bergé -
Ion Gaztañaga -
Jeff Garland -
Maarten de Vries -
Martin Vymazal -
Paolo Bolzoni