2014-07-29 2:47 GMT+04:00 Renato Forti : <...>

Please send your ideas and comments, so I can improve the lib! Thanks!

I'll take a closer look to the library probably in next months and will make a few additions. PS. You wrote: Boost C++ Application Development Cookbook? I have your

book, and I always use it in my work, very good source! Thanks for that!

Nice to hear it! Thanks. I've moved the source codes from that book into the git recently: https://github.com/apolukhin/Boost-Cookbook-4880OS . Hope that helps someone. -- Best regards, Antony Polukhin

On July 28, 2014 5:13:00 PM EDT, Antony Polukhin wrote:

2014-07-28 21:46 GMT+04:00 Renato Forti :

* how about adding an additional template parameter to the get_symbol method:

template ReturnType* get_symbol(const symbol_type<T> &sb);

Now ((pluginapi_create)sl("create_my_plugin")) will look slightly better:

sl.get_symbol("create_my_plugin");

+1

* what will happen with the application, if shred library goes out of scope but we continue to use symbols from it? Must it be prevented somehow by the Application library? Is it described in docs?

It would be nice if get_symbol returned a smart pointer that contained a shared_ptr to the library. When all such references are released, the library can be unloaded. ___ Rob (Sent from my portable computation engine)

On July 29, 2014 3:39:16 AM EDT, "Klaim - Joël Lamotte" wrote:

Sent from mobile. On 29 Jul 2014 02:33, "Rob Stewart" wrote:

...

On July 28, 2014 5:13:00 PM EDT, Antony Polukhin

wrote:

...

...

* what will happen with the application, if shred library goes out of scope but we continue to use symbols from it? Must it be prevented somehow by the Application library? Is it described in docs?

It would be nice if get_symbol returned a smart pointer that

contained a shared_ptr to the library. When all such references are released, the library can be unloaded.

I disagree. In my experience it would be more useful if it contained a weak_ptr to the library, with a centralised system keeping all loaded libraries alive and allowing the used to unload the library explicitely. The get_symbol return type would then be testable. That way it is easier to find issues related to plugins and its still possible for the user to prevent these issues at runtime.

Please explain. Using a weak_ptr means each access must be tested, with synchronized access, to boot, versus assurance of validity with the shared_ptr. I can see some value in being able to unload a library even with outstanding references, but I'm not certain the overhead is worthwhile. I'll grant that calling such functions is likely not in a critical path, but I'd like to understand your use case. I'm also not enamored implications of the "centralized system" you mentioned to track the open libraries. [snip my earlier signature block and the list's footer] Please trim all unneeded text from your replies. ___ Rob (Sent from my portable computation engine)

On Tue, Jul 29, 2014 at 5:43 AM, Rob Stewart wrote:

On July 29, 2014 3:39:16 AM EDT, "Klaim - Joël Lamotte" wrote:

...

I disagree. In my experience it would be more useful if it contained a weak_ptr to the library, with a centralised system keeping all loaded libraries alive and allowing the used to unload the library explicitely. The get_symbol return type would then be testable. That way it is easier to find issues related to plugins and its still possible for the user to prevent these issues at runtime.

Please explain. Using a weak_ptr means each access must be tested, with synchronized access, to boot, versus assurance of validity with the shared_ptr.

I think there are two scenarios. In one, you're dynamically loading the library, and you want it to remain in memory for the duration of the application's run (you never intend to deallocate it until the application is finished). You might consider this scenario for situations where you're licensing different facets of your application, and licensing determines available features. However, there's another scenario where you dynamically load the library, then deallocate it as soon as you're done with it (and where it is important to do so). For those scenarios, this centralized system of keeping all loaded libraries alive is an unnecessary and unwanted burden. You might consider this scenario in cases where you have a long-running service that you cannot lightly take down, but you want to be capable of making updates to features the service use. At least on Windows, you cannot replace a DLL if it is loaded, so if it's only loaded when in use, you'd be able to swap out the library without stopping the service. I think both scenarios can be supported by having an object that acts as this centralized system which one may use if desired, while also having the current system. - Trey

On July 29, 2014 7:27:09 AM EDT, "Klaim - Joël Lamotte" wrote:

On Tue, Jul 29, 2014 at 12:19 PM, Joseph Van Riper < fleeb.fantastique@gmail.com> wrote:

...

On Tue, Jul 29, 2014 at 5:43 AM, Rob Stewart wrote:

...

On July 29, 2014 3:39:16 AM EDT, "Klaim - Joël Lamotte" < mjklaim@gmail.com> wrote:

...

I disagree. In my experience it would be more useful if it contained a weak_ptr to the library, with a centralised system keeping all loaded libraries alive and allowing the used to unload the library explicitely. The get_symbol return type would then be testable. That way it is easier to find issues related to plugins and its still possible for the user to prevent these issues at runtime.

Please explain. Using a weak_ptr means each access must be tested, with synchronized access, to boot, versus assurance of validity with the shared_ptr.

I think there are two scenarios. In one, you're dynamically loading the library, and you want it to remain in memory for the duration of the application's run (you never intend to deallocate it until the application is finished).

[snip] That's nothing more than a collection of shared_ptr's to the dynamic library objects. It had no bearing on the objects returned by get_symbol().

...

However, there's another scenario where you dynamically load the library, then deallocate it as soon as you're done with it (and where it is important to do so). For those scenarios, this centralized system of keeping all loaded libraries alive is an unnecessary and unwanted burden.

[snip] Indeed.

...

I think both scenarios can be supported by having an object that acts as this centralized system which one may use if desired, while also having the current system.

I think you've argued against such a "centralized system".

Thanks this is exactly what I meant. I understand the first case but when using dynamically loaded libraries/modules, I more often endup in the second case (in particular when it's "plugins" of a tool or game or simulation engine, that need to be activated and deactivated when necessary).

In the first case, you indeed want to make sure that the necessary libraries are always loaded. However in the second, you really need to have a way to be notified when calling something from an unloaded library. In this case, you might just trigger an error, or just do some logging or something. In my own cases I would not test the access returned object, because I assume that that object must not be accessible anymore once the related library is unloaded. Therefore, I consider this case a bug and want the application to crash early if that happen.

It sounds to me as if you're suggesting that you wouldn't actually use the weak_ptr, but instead simply call an unloaded function and live with the consequences.

I was suggesting weak_ptr (wrapped inside something that looks like a std::function<>) so that the user decides if or not he wants to check if the library is still loaded before using the function. But personally I wouldn't.

I'm still at a loss to understand what you'd actually want get_symbol() to return to solve your use cases. If the returned object holds a shared_ptr to the library, the library can't be unloaded until all such objects have been destroyed or until they are told to release their ref count. That is safer, however. Holding a weak_ptr instead implies a runtime check on each invocation, but doesn't prevent unloading the library. However, you've said you wouldn't use the weak_ptr, so I'm missing something. ___ Rob (Sent from my portable computation engine)

On Wed, Jul 30, 2014 at 2:02 PM, Klaim - Joël Lamotte wrote:

On Wed, Jul 30, 2014 at 11:27 AM, Rob Stewart wrote:

...

...

I was suggesting weak_ptr (wrapped inside something that looks like a std::function<>) so that the user decides if or not he wants to check if the library is still loaded before using the function. But personally I wouldn't.

I'm still at a loss to understand what you'd actually want get_symbol() to return to solve your use cases.

If the returned object holds a shared_ptr to the library, the library can't be unloaded until all such objects have been destroyed or until they are told to release their ref count. That is safer, however. Holding a weak_ptr instead implies a runtime check on each invocation, but doesn't prevent unloading the library. However, you've said you wouldn't use the weak_ptr, so I'm missing something.

Very sorry, I was not clear at all and I also mixed up details about the get_symbol() return type (it's not a callable but the call result, I mixed up this part).

Let me try again:

In most use cases I had, the lifetime of a plugin was necessarily separate from it's usage: I would have a way to say that the plugin must be unloaded and that would invalidate all code trying to use it. This is because unloading-the-plugin-event is then propagated through the users systems and they have to do the cleaning work (that is, destroy all accesses to the plugin). Once the plugin is unloaded, in my use cases, we want it to not be usable at all anymore, explicitly. And we want the whole application to fail early if access to the unloaded plugin is tried.

Therefore, I agree that get_symbol() should return a smart pointer, but containing a weak_ptr instead of a shared_ptr, so that the symbol lifetimes don't force the plugin to still be loaded when we wanted it not to. Once the plugin is unloaded (the shared_ptr to it is destroyed), when the returned smart pointer is accessed, the weak_ptr would test locking and would, by default, throw an exception. Personally I would prefer the default behaviour in this case to be overwriteable, so that I can for example just log and call terminate(), then change this behaviour in the shiped product.

In some specific cases I would want to test if the symbol is valid before using it (but in most cases I had, I don't want to because it should fail at this point - it's what I meant by "but personally I wouldn't").

Centralization or not of the plugins lifetime is let to the user, this part was more noise than argument. The user can or not share the plugin lifetime between different systems, but I disagree that returns of the symbols should force the lifetime of the plugin to be longer than necessary.

Am I more clear?

Maybe two different get_symbol() functions could return either a "safe" (managing a shared_ptr to the plugin) or "unsafe" (managing a weak_ptr to the plugin and throwing an exception by default when it's not available and we try to use the data) smart pointer? Maybe that way our different use cases would be all solved.

I think you can achieve any behavior you described with the interface Peter suggested, can't you? // p1 locks lib from being unloaded shared_ptr< foo > p1 = get_symbol< foo >(lib, "foo"); // p2 does not lock lib from being unloaded and is testable weak_ptr< foo > p2 = get_symbol< foo >(lib, "foo"); // p3 does not lock lib from being unloaded and crashes if used after lib is unloaded foo* p3 = get_symbol< foo >(lib, "foo").get(); Personally, I'd prefer get_symbol to return something immediately usable, i.e. shared_ptr. I'd also like to emphasize that it should return a pointer and not a function since the symbol may be data. Just my 2 cents.

2014-07-30 14:57 GMT+04:00 Klaim - Joël Lamotte :

On Wed, Jul 30, 2014 at 12:20 PM, Andrey Semashev < andrey.semashev@gmail.com

...

wrote:

...

I think you can achieve any behavior you described with the interface Peter suggested, can't you?

// p1 locks lib from being unloaded shared_ptr< foo > p1 = get_symbol< foo >(lib, "foo"); // p2 does not lock lib from being unloaded and is testable weak_ptr< foo > p2 = get_symbol< foo >(lib, "foo"); // p3 does not lock lib from being unloaded and crashes if used after lib is unloaded foo* p3 = get_symbol< foo >(lib, "foo").get();

Personally, I'd prefer get_symbol to return something immediately usable, i.e. shared_ptr. I'd also like to emphasize that it should return a pointer and not a function since the symbol may be data. Just my 2 cents.

I might have misunderstood what Peter suggested: I thought he suggested that get_symbol<> would return a smart pointer (not shared_ptr) that would _internally_ contain a shared_ptr to the plugin and the access to the data.

Returning a raw pointer is not user friendly. User will need to look through the docs to determinate how to deal with pointer (must it be freed, can we use that pointer after library unload). How about returning a reference? This is intuitive interface that rises no questions: int *p; { shred_library lib("/usr/lib/myapp/plugins/plugin.so"); auto& f = lib.get_symbol("factory_func"); f(); auto& i = lib.get_symbol<int>("global_int"); ++i; p = &i; } // This looks incorrect even without referencing the docs *p = 100; If there is a requirement to make a refcounting version of shared_library, then it could be easily achieved using previous impl: struct shred_library_refcountable { shred_library_refcountable(name) : impl_(make_shared(name)) {} template <class T> shared_ptr<T> get_symbol(name) { auto ptr = impl_; return shared_ptr( &impl_->get_symbol(name), [ptr](void *) { /* do nothing*/ } // holds a shared ptr ); } }; shared_ptr<int> i; { shred_library_refcountable lib("/usr/lib/myapp/plugins/plugin.so"); i = lib.get_symbol<int>("global_int"); } *i = 100; // OK to use -- Best regards, Antony Polukhin

Klaim - Joël Lamotte wrote:

I might have misunderstood what Peter suggested: I thought he suggested that get_symbol<> would return a smart pointer (not shared_ptr) that would _internally_ contain a shared_ptr to the plugin and the access to the data.

No, get_symbol returns a 'real' shared_ptr in my suggestion. shared_ptr<library> p = load_library( "mylib.so" ); shared_ptr p2 = get_symbol( p, "myfunction" ); (*p2)(); // calls myfunction() in mylib.so By creating p2 using the aliasing constructor of shared_ptr, you can have p2 and p share ownership of the loaded library. template<class Y> shared_ptr(const shared_ptr<Y>& r, T *p) noexcept; Effects: Constructs a shared_ptr instance that stores p and shares ownership with r. Postconditions: get() == p && use_count() == r.use_count() [ Note: To avoid the possibility of a dangling pointer, the user of this constructor must ensure that p remains valid at least until the ownership group of r is destroyed. —end note ]

Klaim - Joël Lamotte wrote:

Could you clarify what would happen here:

shared_ptr p2; { shared_ptr<library> p = load_library( "mylib.so" ); p2 = get_symbol( p, "myfunction" ); } // p is destroyed

(*p2)(); // ?

The library is kept alive by p2. But if you make p2 a weak_ptr, it isn't.