flyweights, nested flyweights and shared libraries

Andreas Buykx

19 Feb 2021 19 Feb '21

3:54 p.m.

I want to use boost::flyweight to capture some structures containing const data. The documentation states that if flyweights are exported across shared library boundaries that the intermodule holder should be used. Since my flyweights are all created in the same shared library I would prefer to use the static holder but I keep getting crashes. I tried the intermodule holder but I'm having trouble compiling my sources with that. So my thinking was to wrap all flyweight types in accessor classes which are exposed instead, and the accessor classes create the flyweights within the same shared library to store the data. Would that be a valid approach? Also, is it possible to have nested flyweights (when they are wrapped by accessor classes)? And would that cause any additional problems in the above scenario? Below is an example of what I'm trying to achieve... Thanks for your advice! Andreas === Foo.h === struct FooDetails { FooDetails( std::string text_, std::vector<double> values ); std::string text; std::vector<double> values; }; class STUFF_EXPORT Foo { public: Foo( std::string text_, std::vector<double> values ); // rule-of-5 constructors declared, and default-implemented in cpp file const std::string& text() const; const std::vector<double>& values() const; private: boost::flyweight<FooDetails> m_data; }; === Foo.cpp === FooDetails( ( std::string text_, std::vector<double> values_ ) : text(text_), values( values_ ) { } Foo::Foo( std::string text_, std::vector<double> values_ ) m_data( FooDetails( text_, values_ ) ) { } std::string Foo::text() const { return m_data.get().text; } std::vector<double> Foo::values() const { return m_data.get().values; } === Bar.h === struct BarDetails { BarDetails( Foo foo_, std::vector<long> ids_ ); Foo foo; //< Foo holds a flyweight std::vector<long> ids; }; class STUFF_EXPORT Bar { public: Bar( std::string text_, std::vector<double> values_, std::vector<long> ids_ ); // rule-of-5 constructors declared, and default-implemented in cpp file const std::string& text() const; const std::vector<double>& values() const; const std::vector<long>& ids() const; private: boost::flyweight<BarDetails> m_bar; }; === Bar.cpp === BarDetails::BarDetails( Foo foo_, std::vector<long> ids_ ) : foo( foo_ ), ids( ids_ ) { } Bar::Bar( std::string text_, std::vector<double> values_, std::vector<long> ids_ ) : m_bar( Foo( text_, values), ids_ ) { } const std::string& Bar::text() const { return m_bar.get().m_foo.text(); } const std::vector<double>& Bar::values() const { return m_bar.get().foo.values(); } const std::vector<long>& Bar::ids() const { return m_bar.get().ids; }

Attachments:

attachment.html (text/html — 10.7 KB)

Show replies by date

Joaquin M López Muñoz

20 Feb 20 Feb

3:21 p.m.

El 19/02/2021 a las 16:54, Andreas Buykx via Boost-users escribió:

...

I want to use boost::flyweight to capture some structures containing const data.

The documentation states that if flyweights are exported across shared library boundaries that the intermodule holder should be used.

Since my flyweights are all created in the same shared library I would prefer to use the static holder but I keep getting crashes.

It is not only necessary that all flyweights are created in the same DLL: they must also be destroyed within that DLL. I guess that's the problem you're experiencing.

...

I tried the intermodule holder but I’m having trouble compiling my sources with that.

Can you provide more info on the kind of problems you're running into? intermodule_holder tests successfully in Linux, Mac and Windows platforms. I presume it is Windows you use.

...

So my thinking was to wrap all flyweight types in accessor classes which are exposed instead, and the accessor classes create the flyweights within the same shared library to store the data. Would that be a valid approach?

I think your first option should be to make intermodule_holder work. That said, the approach you outline below in your mail looks OK, as the lifetime of flyweight objects is handled entirely within DLL-specific exported functions. A simpler approach would be to define a wrapper around boost::flyweight: // template class to be instantiated *only* // within your exporting DLL template<typename T> struct DLLEXPORT dll_flyweight:boost::flyweight<T> { using boost::flyweight<T>::flyweight; };

...

Also, is it possible to have nested flyweights (when they are wrapped by accessor classes)? And would that cause any additional problems in the above scenario?

I don't see any problem with that. Best, Joaquín M López Muñoz

Andreas Buykx

23 Feb 23 Feb

9:59 a.m.

Hello Joaquín, Thanks for your help. This morning it dawned on me what you meant with the remark about the destruction: I implemented all constructors as explicit default in the cpp-file but I had forgotten to do that for the destructor. Now everything runs like a breeze, except for one situation where I get an assertion: boost::flyweights::detail::recursive_lightweight_mutex::scoped_lock::scoped_lock(boost::flyweights::detail::recursive_lightweight_mutex &): Assertion `pthread_mutex_lock(&m_)==0' failed. This exception is raised while destroying the nested flyweight wrapper from the nesting flyweight wrapper, but only in one of my many test cases where these flyweight wrappers are used. Do you have any suggestions on why this might happen? Regarding the use of intermodule_holder I mistook a link error for a compile error: /opt/rh/devtoolset-6/root/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/ld: Test/appz/ppf/unittests/model/results/CMakeFiles/unittest_PpfModelResultUnitTest.dir/PpfResultCaseUnitTest.cpp.o: undefined reference to symbol 'shm_unlink@@GLIBC_2.2.5' //lib64/librt.so.1: error adding symbols: DSO missing from command line All that aside: you mention that getting intermodule_holder is the preferred option, but given that it's "... intermodule_holder is considerably more onerous than static_holder in terms of compilation times and introduces a non-negligible overhead at program start-up ...", isn't wrapping the flyweight in an accessor class (especially your templated construct) at least as good an option that maybe deserves a mention in the documentation? Thanks again for your help, Andreas From: Boost-users On Behalf Of Joaquin M López Muñoz via Boost-users Sent: zaterdag 20 februari 2021 16:22 To: boost-users@lists.boost.org Cc: Joaquin M López Muñoz Subject: Re: [Boost-users] flyweights, nested flyweights and shared libraries El 19/02/2021 a las 16:54, Andreas Buykx via Boost-users escribió: I want to use boost::flyweight to capture some structures containing const data. The documentation states that if flyweights are exported across shared library boundaries that the intermodule holder should be used. Since my flyweights are all created in the same shared library I would prefer to use the static holder but I keep getting crashes. It is not only necessary that all flyweights are created in the same DLL: they must also be destroyed within that DLL. I guess that's the problem you're experiencing. I tried the intermodule holder but I'm having trouble compiling my sources with that. Can you provide more info on the kind of problems you're running into? intermodule_holder tests successfully in Linux, Mac and Windows platforms. I presume it is Windows you use. So my thinking was to wrap all flyweight types in accessor classes which are exposed instead, and the accessor classes create the flyweights within the same shared library to store the data. Would that be a valid approach? I think your first option should be to make intermodule_holder work. That said, the approach you outline below in your mail looks OK, as the lifetime of flyweight objects is handled entirely within DLL-specific exported functions. A simpler approach would be to define a wrapper around boost::flyweight: // template class to be instantiated *only* // within your exporting DLL template<typename T> struct DLLEXPORT dll_flyweight:boost::flyweight<T> { using boost::flyweight<T>::flyweight; }; Also, is it possible to have nested flyweights (when they are wrapped by accessor classes)? And would that cause any additional problems in the above scenario? I don't see any problem with that. Best, Joaquín M López Muñoz

Joaquin M López Muñoz

6:12 p.m.

El 23/02/2021 a las 10:59, Andreas Buykx via Boost-users escribió:

...

Hello Joaquín,

Hi, please don't top-post, see: https://www.boost.org/community/policy.html#quoting

...

Thanks for your help. This morning it dawned on me what you meant with the remark about the destruction: I implemented all constructors as explicit default in the cpp-file but I had forgotten to do that for the destructor. Now everything runs like a breeze, except for one situation where I get an assertion:

boost::flyweights::detail::recursive_lightweight_mutex::scoped_lock::scoped_lock(boost::flyweights::detail::recursive_lightweight_mutex &): Assertion `pthread_mutex_lock(&m_)==0' failed.

This exception is raised while destroying the nested flyweight wrapper from the nesting flyweight wrapper, but only in one of my many test cases where these flyweight wrappers are used. Do you have any suggestions on why this might happen?

Umm.. Do you have any *global* static variable of type nesting_flyweight? If so, you may be running into the sort of static data initialization issues described at: https://www.boost.org/ibs/flyweight/doc/tutorial/technical.html#static_init To see if this is the problem, insert static your_nested_flyweight_type::initializer fwinit; before the potentially offending global varable.

...

Regarding the use of intermodule_holder I mistook a link error for a compile error:

/opt/rh/devtoolset-6/root/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/ld: Test/appz/ppf/unittests/model/results/CMakeFiles/unittest_PpfModelResultUnitTest.dir/PpfResultCaseUnitTest.cpp.o: undefined reference to symbol 'shm_unlink@@GLIBC_2.2.5'

//lib64/librt.so.1: error adding symbols: DSO missing from command line

I'm no Linux expert, does this article help? https://stackoverflow.com/questions/9923495/undefined-reference-shm-open-alr...

...

All that aside: you mention that getting intermodule_holder is the preferred option, but given that it’s “… |intermodule_holder| is considerably more onerous than |static_holder| in terms of compilation times and introduces a non-negligible overhead at program start-up …”, isn’t wrapping the flyweight in an accessor class (especially your templated construct) at least as good an option that maybe deserves a mention in the documentation?

I'm not sure this is robust enough for general use. What happens if two different modules libA and libB both export the same, say, dll_flyweightstd::string type?

...

Thanks again for your help,

Andreas

Joaquín M López Muñoz

Deniz Bahadir

24 Feb 24 Feb

9:39 a.m.

Am 23.02.21 um 19:12 schrieb Joaquin M López Muñoz via Boost-users:

...

El 23/02/2021 a las 10:59, Andreas Buykx via Boost-users escribió:

... Umm.. Do you have any *global* static variable of type nesting_flyweight? If so, you may be running into the sort of static data initialization issues described at:

https://www.boost.org/ibs/flyweight/doc/tutorial/technical.html#static_init

That link is not working. But I guess you meant: https://www.boost.org/doc/libs/1_75_0/libs/flyweight/doc/tutorial/technical....

...

...

...
/opt/rh/devtoolset-6/root/usr/libexec/gcc/x86_64-redhat-linux/6.3.1/ld: Test/appz/ppf/unittests/model/results/CMakeFiles/unittest_PpfModelResultUnitTest.dir/PpfResultCaseUnitTest.cpp.o: undefined reference to symbol 'shm_unlink@@GLIBC_2.2.5'

//lib64/librt.so.1: error adding symbols: DSO missing from command line

I'm no Linux expert, does this article help?

https://stackoverflow.com/questions/9923495/undefined-reference-shm-open-alr...

It did help me when I had that same problem two weeks ago. Deniz

Joaquin M López Muñoz

6:28 p.m.

El 24/02/2021 a las 10:39, Deniz Bahadir via Boost-users escribió:

...

Am 23.02.21 um 19:12 schrieb Joaquin M López Muñoz via Boost-users:

...
El 23/02/2021 a las 10:59, Andreas Buykx via Boost-users escribió:

... Umm.. Do you have any *global* static variable of type nesting_flyweight? If so, you may be running into the sort of static data initialization issues described at:

https://www.boost.org/ibs/flyweight/doc/tutorial/technical.html#static_init

That link is not working. But I guess you meant: https://www.boost.org/doc/libs/1_75_0/libs/flyweight/doc/tutorial/technical....

Ouch, thanks for spotting this! The correct version-indepedent link is: https://www.boost.org/libs/flyweight/doc/tutorial/technical.html#static_init Best Joaquín M López Muñoz

Andreas Buykx

25 Feb 25 Feb

11:44 a.m.

From: Boost-users On Behalf Of Joaquin M López Muñoz via Boost-users Sent: zaterdag 20 februari 2021 16:22 ... El 19/02/2021 a las 16:54, Andreas Buykx via Boost-users escribió: ... Also, is it possible to have nested flyweights (when they are wrapped by accessor classes)? And would that cause any additional problems in the above scenario? I don't see any problem with that. [Andreas Buykx] One thing to be aware of though is that boost::flyweight<nested>::init() must be called before creating the first nesting flyweight. This way you make sure that at program exit the nesting flyweight's static data structures are destroyed before the nested flyweight's static data structures.

Joaquin M López Muñoz

12:52 p.m.

El 25/02/2021 a las 12:44, Andreas Buykx via Boost-users escribió:

...

*From:*Boost-users *On Behalf Of *Joaquin M López Muñoz via Boost-users *Sent:* zaterdag 20 februari 2021 16:22

…

El 19/02/2021 a las 16:54, Andreas Buykx via Boost-users escribió:

…

Also, is it possible to have nested flyweights (when they are wrapped by accessor classes)? And would that cause any additional problems in the above scenario?

I don't see any problem with that.

*/[Andreas Buykx] /*

*/One thing to be aware of though is that boost::flyweight<nested>::init() must be called before creating the first nesting flyweight. This way you make sure that at program exit the nesting flyweight’s static data structures are destroyed before the nested flyweight’s static data structures. /*

Yes, exactly. Did this solve your problem? Joaquín M López Muñoz

Andreas Buykx

1:20 p.m.

From: Boost-users On Behalf Of Joaquin M López Muñoz via Boost-users Sent: donderdag 25 februari 2021 13:52 ... El 25/02/2021 a las 12:44, Andreas Buykx via Boost-users escribió: From: Boost-users mailto:boost-users-bounces@lists.boost.org On Behalf Of Joaquin M López Muñoz via Boost-users Sent: zaterdag 20 februari 2021 16:22 ... El 19/02/2021 a las 16:54, Andreas Buykx via Boost-users escribió: ... Also, is it possible to have nested flyweights (when they are wrapped by accessor classes)? And would that cause any additional problems in the above scenario? I don't see any problem with that. [Andreas Buykx] One thing to be aware of though is that boost::flyweight<nested>::init() must be called before creating the first nesting flyweight. This way you make sure that at program exit the nesting flyweight's static data structures are destroyed before the nested flyweight's static data structures. Yes, exactly. Did this solve your problem? [Andreas Buykx] Yes it did. Thanks a lot for your help.

1370

Age (days ago)

1376

Last active (days ago)

List overview

Download

8 comments

3 participants

participants (3)

Andreas Buykx
Deniz Bahadir
Joaquin M López Muñoz