[mpi] many undefined references to serialize<boost::mpi::detail::mpi_datatype_oarchive>

newer
[Boost.Test] What's the best way...

Brian Budge

10 Sep 2012 10 Sep '12

10:51 p.m.

Hi all - I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save. I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast. I've added explicit instantiations for the boost::mpi::packed_iarchive and packed_oarchive. My code compiles, but at the end I get a bunch of undefined references for serializeboost::mpi::detail::mpi_datatype_oarchive. I can add the explicit instantiations for this oarchive, and this seems to fix my linking issue, but this seems broken, i.e. I should not have to deal with anything in a boost::detail namespace? Is there another suggestion for how to get this to work? Thanks, Brian

Show replies by date

Robert Ramey

11 Sep 11 Sep

5:18 a.m.

New subject: [mpi] many undefined references toserialize<boost::mpi::detail::mpi_datatype_oarchive>

Brian Budge wrote:

...

Hi all -

I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save.

I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast. I've added explicit instantiations for the boost::mpi::packed_iarchive and packed_oarchive. My code compiles, but at the end I get a bunch of undefined references for serializeboost::mpi::detail::mpi_datatype_oarchive.

I can add the explicit instantiations for this oarchive, and this seems to fix my linking issue, but this seems broken, i.e. I should not have to deal with anything in a boost::detail namespace?

I agree. Note that the main boost archive classses - text_oarchive etc. do have *.cpp files with expicit explicit instantiations. Maybe these are missing from the mpi archives. Feel free to investigate and suggest a patch. Robert Ramey

...

Is there another suggestion for how to get this to work?

Thanks, Brian

Riccardo Murri

3:50 p.m.

Hello, On Tue, Sep 11, 2012 at 12:51 AM, Brian Budge wrote:

...

I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save.

I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast.

If your class has serialization support, that's enough. You should not provide specialization for mpi::detail classes. (Of course, YMMV - since you didn't provide any code, my comment above could be completely off ...) Cheers, Riccardo

Brian Budge

4:22 p.m.

On Tue, Sep 11, 2012 at 8:50 AM, Riccardo Murri wrote:

...

Hello,

On Tue, Sep 11, 2012 at 12:51 AM, Brian Budge wrote:

...
I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save.

I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast.

If your class has serialization support, that's enough. You should not provide specialization for mpi::detail classes.

(Of course, YMMV - since you didn't provide any code, my comment above could be completely off ...)

Cheers, Riccardo

Thanks for the replies. Here are the macros I made to perform the explicit instantiation: #define INSTANTIATE_FUNCTION_FOR_INPUT_ARCHIVES(FUNC) \ template void FUNC(boost::archive::binary_iarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::packed_iarchive &archive, \ const unsigned int version); \ template void FUNC(boost::archive::text_iarchive &archive, \ const unsigned int version) #define INSTANTIATE_FUNCTION_FOR_OUTPUT_ARCHIVES(FUNC) \ template void FUNC(boost::archive::binary_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::packed_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::detail::mpi_datatype_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::archive::text_oarchive &archive, \ const unsigned int version) #define MAKE_SERIALIZE_FUNCTIONS(CLASS) \ INSTANTIATE_FUNCTION_FOR_INPUT_ARCHIVES(CLASS::serialize); \ INSTANTIATE_FUNCTION_FOR_OUTPUT_ARCHIVES(CLASS::serialize) Then, along with implementing the templatized versions of serialize for each class, I will call the MAKE_SERIALIZE_FUNCTIONS(myclass) in each myclass.cpp. So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. I was just surprised to find that I needed to explicitly instantiate with something from detail rather than just the packed archives. Thanks, Brian

...

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Oswin Krause

5:11 p.m.

New subject: [mpi] many undefined references to serialize<boost::mpi::detail::mpi_datatype_oarchive>

Hi, Is there a reason you ar doing it this way? Because your question now boils down to: "Why does it hurt, when I turn automatic template generation off?". As an user of an serializable class, I would never expect that my choice of archive would impact whether my code compiles or not. Especially since the interface states that i can (through template<class Archive>). Instead I would assume that i could safely add my own archives without checking weird macros in theird party libraries. Greetings, Oswin because it cna assume it to be there. On 2012-09-11 18:22, Brian Budge wrote:

...

On Tue, Sep 11, 2012 at 8:50 AM, Riccardo Murri wrote:

...
Hello,

On Tue, Sep 11, 2012 at 12:51 AM, Brian Budge wrote:

...
I've got a class hierarchy that can already be serialized into text or binary archives. I'm explicitly instantiating the template functions for serialize and/or load/save.

I'm trying to add the capability to have boost mpi perform the serialization through send/recv/broadcast.

If your class has serialization support, that's enough. You should not provide specialization for mpi::detail classes.

(Of course, YMMV - since you didn't provide any code, my comment above could be completely off ...)

Cheers, Riccardo

Thanks for the replies. Here are the macros I made to perform the explicit instantiation:

#define INSTANTIATE_FUNCTION_FOR_INPUT_ARCHIVES(FUNC) \ template void FUNC(boost::archive::binary_iarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::packed_iarchive &archive, \ const unsigned int version); \ template void FUNC(boost::archive::text_iarchive &archive, \ const unsigned int version)

#define INSTANTIATE_FUNCTION_FOR_OUTPUT_ARCHIVES(FUNC) \ template void FUNC(boost::archive::binary_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::packed_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::mpi::detail::mpi_datatype_oarchive &archive, \ const unsigned int version); \ template void FUNC(boost::archive::text_oarchive &archive, \ const unsigned int version)

#define MAKE_SERIALIZE_FUNCTIONS(CLASS) \ INSTANTIATE_FUNCTION_FOR_INPUT_ARCHIVES(CLASS::serialize); \ INSTANTIATE_FUNCTION_FOR_OUTPUT_ARCHIVES(CLASS::serialize)

Then, along with implementing the templatized versions of serialize for each class, I will call the MAKE_SERIALIZE_FUNCTIONS(myclass) in each myclass.cpp.

So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. I was just surprised to find that I needed to explicitly instantiate with something from detail rather than just the packed archives.

Thanks, Brian

...
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Brian Budge

5:27 p.m.

On Tue, Sep 11, 2012 at 10:11 AM, Oswin Krause wrote:

...

Hi,

Is there a reason you ar doing it this way? Because your question now boils down to: "Why does it hurt, when I turn automatic template generation off?".

As an user of an serializable class, I would never expect that my choice of archive would impact whether my code compiles or not. Especially since the interface states that i can (through template<class Archive>). Instead I would assume that i could safely add my own archives without checking weird macros in theird party libraries.

Greetings, Oswin

Hi Oswin - We are serializing an abstract hierarchy. Because we pass only the base pointer type(s) into the serialization code, we need to explicitly instantiate the functions so that the functions are available at runtime. I could be missing something, but that's the reasoning. If there's a better way, I'm definitely up for that. Thanks, Brian

Oswin Krause

6:03 p.m.

New subject: [mpi] many undefined references to serialize<boost::mpi::detail::mpi_datatype_oarchive>

Hi Brian, Does this help? http://www.boost.org/doc/libs/1_51_0/libs/serialization/doc/serialization.ht... Greetings, Oswin On 2012-09-11 19:27, Brian Budge wrote:

...

On Tue, Sep 11, 2012 at 10:11 AM, Oswin Krause wrote:

...
Hi,

Is there a reason you ar doing it this way? Because your question now boils down to: "Why does it hurt, when I turn automatic template generation off?".

As an user of an serializable class, I would never expect that my choice of archive would impact whether my code compiles or not. Especially since the interface states that i can (through template<class Archive>). Instead I would assume that i could safely add my own archives without checking weird macros in theird party libraries.

Greetings, Oswin

Hi Oswin -

We are serializing an abstract hierarchy. Because we pass only the base pointer type(s) into the serialization code, we need to explicitly instantiate the functions so that the functions are available at runtime. I could be missing something, but that's the reasoning. If there's a better way, I'm definitely up for that.

Thanks, Brian _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Brian Budge

6:36 p.m.

On Tue, Sep 11, 2012 at 11:03 AM, Oswin Krause wrote:

...

Hi Brian,

Does this help?

http://www.boost.org/doc/libs/1_51_0/libs/serialization/doc/serialization.ht...

Greetings, Oswin

I think I see. I believe my co-worker did this so that only the header for the base object was required for serialization into the archive. Changing this requires all relevant headers to be included everywhere an archive is used where that type could be encountered during serialization. I will have to see if we can make this change so we can remove our dependence on the detail archive in boost::mpi. Thanks, Brian

Robert Ramey

6:29 p.m.

New subject: [mpi] many undefined references toserialize<boost::mpi::detail::mpi_datatype_oarchive>

Brian Budge wrote:

...

So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. ...

look at .../libs/serialization/src/binary_oarchive.cpp This file contains the explict instantiations that are added to the library. I would expect that something like this could be made for mpi archives. Robert Ramey

...

Thanks, Brian

...
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Brian Budge

10:30 p.m.

New subject: [mpi] many undefined references toserialize<boost::mpi::detail::mpi_datatype_oarchive>

On Tue, Sep 11, 2012 at 11:29 AM, Robert Ramey wrote:

...

Brian Budge wrote:

...
So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. ...

look at .../libs/serialization/src/binary_oarchive.cpp

This file contains the explict instantiations that are added to the library. I would expect that something like this could be made for mpi archives.

Robert Ramey

I think you're right Robert. I can get my types to serialize fine with the boost::binary_*archive archives, but not with the mpi ones. It's strange because I can get it to work for send, but not recv. I get an exception related to unregistered type. Indeed if I go and manually serialize every message pointer to a char buffer vis binary_oarchive, and send that via boost mpi, and manually deserialize on the other side from char buffer via binary_iarchive, my program works. I'm unsure what exactly might need to be instantiated in boost::mpi though. Thanks, Brian

Robert Ramey

12 Sep 12 Sep

4:32 a.m.

New subject: [mpi] many undefined referencestoserialize<boost::mpi::detail::mpi_datatype_oarchive>

Brian Budge wrote:

...

On Tue, Sep 11, 2012 at 11:29 AM, Robert Ramey wrote:

...
Brian Budge wrote:

...
So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. ...

look at .../libs/serialization/src/binary_oarchive.cpp

This file contains the explict instantiations that are added to the library. I would expect that something like this could be made for mpi archives.

Robert Ramey

I think you're right Robert. I can get my types to serialize fine with the boost::binary_*archive archives, but not with the mpi ones. It's strange because I can get it to work for send, but not recv. I get an exception related to unregistered type.

Indeed if I go and manually serialize every message pointer to a char buffer vis binary_oarchive, and send that via boost mpi, and manually deserialize on the other side from char buffer via binary_iarchive, my program works.

I'm unsure what exactly might need to be instantiated in boost::mpi though.

just use binary_iarchive as a guide for mpi_archive.

...

Thanks, Brian

Matthias Troyer

30 Oct 30 Oct

9:44 p.m.

New subject: [mpi] many undefined references toserialize<boost::mpi::detail::mpi_datatype_oarchive>

PLease send me some test code that shows the problem. Are you linking against the Boost.MPI libtrary and to you correctly register your derived types? Matthias On Sep 12, 2012, at 12:30 AM, Brian Budge wrote:

...

On Tue, Sep 11, 2012 at 11:29 AM, Robert Ramey wrote:

...
Brian Budge wrote:

...
So to Robert's question, I don't think this is something that could be baked into the boost_mpi.so library, since they rely on my own classes. ...

look at .../libs/serialization/src/binary_oarchive.cpp

This file contains the explict instantiations that are added to the library. I would expect that something like this could be made for mpi archives.

Robert Ramey

I think you're right Robert. I can get my types to serialize fine with the boost::binary_*archive archives, but not with the mpi ones. It's strange because I can get it to work for send, but not recv. I get an exception related to unregistered type.

Indeed if I go and manually serialize every message pointer to a char buffer vis binary_oarchive, and send that via boost mpi, and manually deserialize on the other side from char buffer via binary_iarchive, my program works.

I'm unsure what exactly might need to be instantiated in boost::mpi though.

Thanks, Brian _______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

4412

Age (days ago)

4462

Last active (days ago)

List overview

Download

11 comments

5 participants

participants (5)

Brian Budge
Matthias Troyer
Oswin Krause
Riccardo Murri
Robert Ramey