[MPI] missing symbols using serialization's static library design
Hello all - On Mac OS X Lion, we're seeing errors when linking our application using Boost.MPI. The application is rather large and uses Boost.Serialization for both checkpoint/restart and MPI communication. The checkpoint/restart bits made serialization a bit of a disaster in terms of compile times (due to headers including basically every other header in the project). To solve that problem, we followed the Boost.Serialization advice on static libraries, and all is good. Except that on Lion (and possibly elsewhere, but definitely on Lion) using the gcc front-end (i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)), we're seeing link failures due to serialization calls for MPI archivers not being found when compiling with -O2 or higher on Boost 1.50.0 and 1.51.0: CXXLD sst.x Undefined symbols for architecture x86_64: "void SST::Introspector::serialize<boost::mpi::packed_iarchive>(boost::mpi::packe d_iarchive&, unsigned int)", referenced from: boost::archive::detail::iserializer<boost::mpi::packed_iarchive, SST::Introspector>::load_object_data(boost::archive::detail::basic_iarchive &, void*, unsigned int) const in introspectedComponent.o boost::archive::detail::iserializer<boost::mpi::packed_iarchive, SST::Introspector>::load_object_data(boost::archive::detail::basic_iarchive &, void*, unsigned int) consti n simulation.o While I could simply explicitly insatiate the missing archivers, that seems like a bad idea for something in the detail namespace and it appears that the list of archivers I'd need to instantiate has changed over time. Is there a better solution to the problem? There's a test case the exhibits the problem available here on our Google code project bug tracker. It's essentially the Boost.Serialization demo_pimpl example, but using MPI instead of a text archiver. The code is at: http://sst-simulator.googlecode.com/issues/attachment?aid=280000000&name=mp i_serialization_test.tar.gz&token=1MTkBuFrhrfUUAZRZYH3kZrElIg%3A13500682075 60 Thanks, Brian -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories
Barrett, Brian W wrote:
Hello all -
On Mac OS X Lion, we're seeing errors when linking our application using Boost.MPI. The application is rather large and uses Boost.Serialization for both checkpoint/restart and MPI communication. The checkpoint/restart bits made serialization a bit of a disaster in terms of compile times (due to headers including basically every other header in the project). To solve that problem, we followed the Boost.Serialization advice on static libraries, and all is good.
Except that on Lion (and possibly elsewhere, but definitely on Lion) using the gcc front-end (i686-apple-darwin11-llvm-gcc-4.2 (GCC) 4.2.1 (Based on Apple Inc. build 5658) (LLVM build 2336.11.00)), we're seeing link failures due to serialization calls for MPI archivers not being found when compiling with -O2 or higher on Boost 1.50.0 and 1.51.0:
CXXLD sst.x Undefined symbols for architecture x86_64: "void SST::Introspector::serialize<boost::mpi::packed_iarchive>(boost::mpi::packe d_iarchive&, unsigned int)", referenced from: boost::archive::detail::iserializer<boost::mpi::packed_iarchive, SST::Introspector>::load_object_data(boost::archive::detail::basic_iarchive &, void*, unsigned int) const in introspectedComponent.o boost::archive::detail::iserializer<boost::mpi::packed_iarchive, SST::Introspector>::load_object_data(boost::archive::detail::basic_iarchive &, void*, unsigned int) consti n simulation.o
While I could simply explicitly insatiate the missing archivers, that seems like a bad idea for something in the detail namespace and it appears that the list of archivers I'd need to instantiate has changed over time. Is there a better solution to the problem?
There's a test case the exhibits the problem available here on our Google code project bug tracker. It's essentially the Boost.Serialization demo_pimpl example, but using MPI instead of a text archiver. The code is at:
http://sst-simulator.googlecode.com/issues/attachment?aid=280000000&name=mp i_serialization_test.tar.gz&token=1MTkBuFrhrfUUAZRZYH3kZrElIg%3A13500682075 60
I'm guessing that one will need to explicitly initialilze some stuff in the mpi archives static library. This is done for the other serialization static libraries. I would compare the mpi serialization build against the other archive build and see which, if any explicit instantiations are in one but not the other.
Thanks,
Brian
On 10/13/12 1:09 AM, "Robert Ramey" <ramey@rrsd.com> wrote:
Barrett, Brian W wrote:
While I could simply explicitly insatiate the missing archivers, that seems like a bad idea for something in the detail namespace and it appears that the list of archivers I'd need to instantiate has changed over time. Is there a better solution to the problem?
There's a test case the exhibits the problem available here on our Google code project bug tracker. It's essentially the Boost.Serialization demo_pimpl example, but using MPI instead of a text archiver. The code is at:
I'm guessing that one will need to explicitly initialilze some stuff in the mpi archives static library. This is done for the other serialization static libraries. I would compare the mpi serialization build against the other archive build and see which, if any explicit instantiations are in one but not the other.
I know how to determine which MPI archivers need to be explicitly instantiated, but as I said, that's a rather unsatisfying answer to the problem. It appears that the list of archivers I have to explicitly instantiate has changed along the way from Boost 1.43.0 (the version that was current when we started the project) and Boost 1.51.0. At least one of the archivers is in the detail namespace, so the authors clearly believe such details are internal to the implementation. This puts us in a rather large support bind. I was hoping that there would be a better way (like a macro to help with the explicit instantiation or something similar. Brian -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories
Barrett, Brian W wrote:
On 10/13/12 1:09 AM, "Robert Ramey" <ramey@rrsd.com> wrote:
Barrett, Brian W wrote:
While I could simply explicitly insatiate the missing archivers, that seems like a bad idea for something in the detail namespace and it appears that the list of archivers I'd need to instantiate has changed over time. Is there a better solution to the problem?
There's a test case the exhibits the problem available here on our Google code project bug tracker. It's essentially the Boost.Serialization demo_pimpl example, but using MPI instead of a text archiver. The code is at:
hmmm: couldn't find it. It was always a disappointment to me that we couldn test the MPI serialization in the same way all the other archive types are tested. That is, if understand correctly, we can't just dropin the mpi_?archive into the tests and run tem all.
I'm guessing that one will need to explicitly initialilze some stuff in the mpi archives static library. This is done for the other serialization static libraries. I would compare the mpi serialization build against the other archive build and see which, if any explicit instantiations are in one but not the other.
I know how to determine which MPI archivers need to be explicitly instantiated, but as I said, that's a rather unsatisfying answer to the problem. It appears that the list of archivers I have to explicitly instantiate has changed along the way from Boost 1.43.0 (the version that was current when we started the project) and Boost 1.51.0. At least one of the archivers is in the detail namespace, so the authors clearly believe such details are internal to the implementation. This puts us in a rather large support bind. I was hoping that there would be a better way (like a macro to help with the explicit instantiation or something similar.
How about suggesting a patch to the MPI serialization inplementation which would sync this implementation up with the others. BTW -
Brian
On 10/17/12 1:00 PM, "Robert Ramey" <ramey@rrsd.com> wrote:
Barrett, Brian W wrote:
I know how to determine which MPI archivers need to be explicitly instantiated, but as I said, that's a rather unsatisfying answer to the problem. It appears that the list of archivers I have to explicitly instantiate has changed along the way from Boost 1.43.0 (the version that was current when we started the project) and Boost 1.51.0. At least one of the archivers is in the detail namespace, so the authors clearly believe such details are internal to the implementation. This puts us in a rather large support bind. I was hoping that there would be a better way (like a macro to help with the explicit instantiation or something similar.
How about suggesting a patch to the MPI serialization inplementation which would sync this implementation up with the others.
Unfortunately, I'm barely competent in using Boost.MPI and Boost.Serialization. I'm definitely not in a position to be able to apply a patch. What I'm getting from your responses and the general silence elsewhere is that the right answer is to deal with the incompatibilities between versions of Boost and explicitly instantiate all the internal MPI archivers and hope for the best. It's a little painful, but I think we can do that. Thanks, Brian -- Brian W. Barrett Scalable System Software Group Sandia National Laboratories
On Oct 18, 2012, at 5:26 PM, "Barrett, Brian W" <bwbarre@sandia.gov> wrote:
On 10/17/12 1:00 PM, "Robert Ramey" <ramey@rrsd.com> wrote:
Barrett, Brian W wrote:
I know how to determine which MPI archivers need to be explicitly instantiated, but as I said, that's a rather unsatisfying answer to the problem. It appears that the list of archivers I have to explicitly instantiate has changed along the way from Boost 1.43.0 (the version that was current when we started the project) and Boost 1.51.0. At least one of the archivers is in the detail namespace, so the authors clearly believe such details are internal to the implementation. This puts us in a rather large support bind. I was hoping that there would be a better way (like a macro to help with the explicit instantiation or something similar.
How about suggesting a patch to the MPI serialization inplementation which would sync this implementation up with the others.
Unfortunately, I'm barely competent in using Boost.MPI and Boost.Serialization. I'm definitely not in a position to be able to apply a patch.
What I'm getting from your responses and the general silence elsewhere is that the right answer is to deal with the incompatibilities between versions of Boost and explicitly instantiate all the internal MPI archivers and hope for the best. It's a little painful, but I think we can do that.
can you send me a test code that shows the problem and I will try to resolve your issue? ALso, which hardware platform are you trying to compile it on? Matthias
participants (1)
-
Barrett, Brian W -
Matthias Troyer -
Robert Ramey