​I read the documentation and it looks good starting point, it's complete enough to start working with. Some typo: - Getting Started page: "just *cunstruct* that class with a path to the library as a parameter" should be "construct" - There is a missing '*' in this line in the tutorial page: "Linking plugin into the executable has the advantages of * reducing common size of distibution * simplification of installation of distribution * faster load times of plugin" Some suggestions for the doc: - provide some pages to help with declaring correctly plugin targets with some commonly used (meta-)build system, using this library macros. For example a CMake and a VS projects pages would be useful additions (I think) - clarify early what should happen when you use the library incorrectly (link errors most of the time): this in particular can help a lot understanding issues when you first try to setup a plugin system and don't have yet a good understanding of how it works; - some typical examples on when using "modes" on constructing shared_library can be useful (at least it's not clear to me when it is useful); - there is no reference page for the modes and no link generated when it appears in shared_library reference page. - a note clarifying that BOOST_SYMBOL_EXPORT is not defined by this library but by boost.config would be nice, also if you clarify that you could use your own symbol export/import macro if you already have one, it wouldn't change a thing for boost.plugin; - a page gathering the platform-specific warning per platform would be a useful reference, in particular if it describe the important/noticeable difference in behaviours of the different platforms; I took a very quick look at the implementation but it seems that some parts are temporary (FIXME comments) so I'll ignore that for now. Anyway it looks as simple as it should so far. I will play with it soon, see if I see any problem. Question: - would have it been possible/better to not expose the shared_ptr<> s in the interfaces? Maybe using concept-based type-erasing type instead?

El 15/08/2014 16:28, Antony Polukhin wrote:

In theory a better solution would be to return unique_ptr from shared_variable() functions, however there is no unique_ptr at the moment in Boost and forcing user to use C++11 is not preferable.

We need to fix this ASAP. Not having boost::unique_ptr is an embarrassment.

On Fri, Aug 15, 2014 at 6:44 PM, Niall Douglas wrote:

On 15 Aug 2014 at 15:36, Antony Polukhin wrote:

...

There is a basic implementation of a header only Plugin library: https://github.com/apolukhin/Boost.Plugin

Documentation with examples could be found here: http://apolukhin.github.io/Boost.Plugin/index.html

Is there an interest in such library in Boost community? What features are missing in the library?

Looking forward to hearing from you. Any feedback is more than welcomed.

It definitely has the wrong name because it doesn't implement plugins, it basically is a wrapper for the POSIX and Windows dynamic shared library functions. Boost.SharedLibrary is a far better name.

In my experience the "plugin" word is used as soon as a shared library could be loaded at an indeterminable time during execution (and be unloaded too). This is different from just "shared library" which is both a plugin implementation and a library linkage, including having the library automatically loaded at startup. To me plugin is far more precise than "shared library". You don't need such a library to make a shared library. You would need such a library to make a plugin (or extension, or module) library. Even CMake call them "module", not "shared" libraries. Even if implemented in unix-like systems as just "shared" libraries, it's still a special case of shared library, not the general case.

Now, if you did want to implement a Boost Plugins library, combining John Bandela's CppComponents (https://github.com/jbandela/cppcomponents) header only library which turns C++ objects into portable Microsoft COM compatible objects and Boost.ASIO as the event dispatch loop would probably be an optimal solution. Not only does that seamlessly plug into all Microsoft technologies, it also is an easy way of SWIG exposing your object into all the languages SWIG supports. If anyone is interested in implementing this solution to Boost quality, I believe I might be able to get you hourly funding to do it - contact me off list.

Could you clarify what you mean by plugin here? I don't see why ASIO would be a nice dependency for such a tool, but I don't understand what you mean either.

That said, I have no objection to your library or its design which is solid, so long as it is named more appropriately to what it provides. I suppose my only real qualm is that I wish your library did more.

Niall

-- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Fri, Aug 15, 2014 at 8:42 PM, Niall Douglas wrote:

On 15 Aug 2014 at 19:22, Klaim - Joël Lamotte wrote:

...

...

It definitely has the wrong name because it doesn't implement plugins, it basically is a wrapper for the POSIX and Windows dynamic shared library functions. Boost.SharedLibrary is a far better name.

In my experience the "plugin" word is used as soon as a shared library could be loaded at an indeterminable time during execution (and be unloaded too). This is different from just "shared library" which is both a plugin implementation and a library linkage, including having the library automatically loaded at startup. To me plugin is far more precise than "shared library". You don't need such a library to make a shared library. You would need such a library to make a plugin (or extension, or module) library.

A plugin/extension implies a well defined exported symbols and event processing ABI. Something adhering to Microsoft COM is a good example.

Antony's library is far more low level, it really does just thinly wrap the two popular shared library APIs into a portable API. It says nothing about what the ABI should be other than C compatible, any form of inspection other than C format symbols, or anything about any event model. I wouldn't mind calling his library a plugin solution for C, but it isn't up to task for C++.

I see. To me it's a good starting point and I fear that a more C++ than C library ends-up like boost.extension. But if it's not too hard to match your definition of plugin, it would be a win for me too as a user.

...

Even CMake call them "module", not "shared" libraries. Even if implemented in unix-like systems as just "shared" libraries, it's still a special case of shared library, not the general case.

Unfortunately Boost.Module might get confused with forthcoming C++ Modules. Otherwise I wouldn't object to that name.

Indeed. (I wasn't proposing such name, for the same reason).

...

...

Now, if you did want to implement a Boost Plugins library, combining John Bandela's CppComponents (https://github.com/jbandela/cppcomponents) header only library which turns C++ objects into portable Microsoft COM compatible objects and Boost.ASIO as the event dispatch loop would probably be an optimal solution. Not only does that seamlessly plug into all Microsoft technologies, it also is an easy way of SWIG exposing your object into all the languages SWIG supports. If anyone is interested in implementing this solution to Boost quality, I believe I might be able to get you hourly funding to do it - contact me off list.

Could you clarify what you mean by plugin here? I don't see why ASIO would be a nice dependency for such a tool, but I don't understand what you mean either.

ASIO is shortly to become the officially endorsed C++ event loop and dispatcher. As event processing is by definition part of any plugin or extension ABI - even if defined to be "no event processing", that means ASIO.

I would prefer to be able to use any event loop implementation instead of forcing asio, personally, because it's not a depencency of my projects right now and I already have another event loop which is specifically designed for my case (and is not general at all and require a very specific impl, for a very specific feature). Basically I prefer composing library instead of using a library that assume that everybody uses the same event loop. I might be wrong though, I don't see the whole picture with asio yet. But otherwise I see your general point now.

So, just to be clear, if you load a Qt Plugin, the Qt event pump gets routed through ASIO's event pump. That way we can extend Vicente's monadic continuations framework easily to interoperate with all Qt threading and event objects. That also goes for all third party C++ libraries, and now you can mix and match loading Qt plugins with WinRT plugins with wxWidget plugins.

You may find my C++ Now 2014 position paper useful. http://arxiv.org/pdf/1405.3323.pdf, starting from page 15 onwards.

Just to be clear though, my earlier offer of potential paid work for a CppComponents based plugin system integrated with ASIO is a small subset of the C++ Now 2014 paper proposal, and is much, much easier. My current employer may decide on a Microsoft COM driven SWIG language bindings, and CppComponents is a quick way of metaprogramming the C++ compiler to spit out Microsoft COM compliant binaries. As always, there is a lack of available sufficiently experienced engineers on this.

Niall

-- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On August 15, 2014 2:42:38 PM EDT, Niall Douglas wrote:

On 15 Aug 2014 at 19:22, Klaim - Joël Lamotte wrote:

...

...

It definitely has the wrong name because it doesn't implement plugins, it basically is a wrapper for the POSIX and Windows dynamic shared library functions. Boost.SharedLibrary is a far better name.

In my experience the "plugin" word is used as soon as a shared library could be loaded at an indeterminable time during execution (and be unloaded too). This is different from just "shared library" which is both a plugin implementation and a library linkage, including having the library automatically loaded at startup. To me plugin is far more precise than "shared library". You don't need such a library to make a shared library. You would need such a library to make a plugin (or extension, or module) library.

A plugin/extension implies a well defined exported symbols and event processing ABI. Something adhering to Microsoft COM is a good example.

What that term implies is based upon one's experience. "Plugin", to me is any dynamically loaded component. The ABI you mention depends entirely upon the ubiquity of the plugin. If it must be usable by many applications and customers, it must have a documented, even standardized, API. Just as easily, however, one can create plugins based solely upon dlopen() (or LoadLibrary()) and friends of the plugins are used only for one application.

Antony's library is far more low level, it really does just thinly wrap the two popular shared library APIs into a portable API. It says nothing about what the ABI should be other than C compatible, any form of inspection other than C format symbols, or anything about any event model. I wouldn't mind calling his library a plugin solution for C, but it isn't up to task for C++.

...

Even CMake call them "module", not "shared" libraries. Even if implemented in unix-like systems as just "shared" libraries, it's still a special case of shared library, not the general case.

Unfortunately Boost.Module might get confused with forthcoming C++ Modules. Otherwise I wouldn't object to that name.

I use "dynamic library" as the generic name in my library. ___ Rob (Sent from my portable computation engine)

On August 16, 2014 12:27:20 PM EDT, Niall Douglas wrote:

On 15 Aug 2014 at 17:34, Rob Stewart wrote:

...

...

A plugin/extension implies a well defined exported symbols and event processing ABI. Something adhering to Microsoft COM is a good example.

What that term implies is based upon one's experience. "Plugin", to me is any dynamically loaded component.

I agree. But this library doesn't load components, it loads shared object binaries.

Your reading too much into "plugin". I don't consider that term to require what you think it implies.

...

The ABI you mention depends entirely upon the ubiquity of the plugin. If it must be usable by many applications and customers, it must have a documented, even standardized, API. Just as easily, however, one can create plugins based solely upon dlopen() (or LoadLibrary()) and friends of the plugins are used only for one application.

API != ABI.

ABI is standardised way of describing an API to external code. dlsym/GetProcAddress adhere loosely to the C ABI, and aren't particularly useful for C++. Even if you added a mangled symbol to partial AST decoder you still only have a toolkit with which one /could/ build a Plugin library, because one has yet to implement a dynamic invocation and lifetime management layer, plus an event processing standard. One also would find that some additional metadata is needed above what can be deduced from what the C ABI and RTTI makes available.

Did you notice that my paragraph began with ABI? The switch to API was accidental. I fully comprehend the two. As for all that you described, it is not the minimum required to make such a library useful.

John Bandela's CppComponents library has done a huge amount of all of this grunt work already, plus it's all header only and is a very easy way of making your C++ 14 code available to anything which can speak Microsoft COM. He's presented at both the 2013 and 2014 C++ Now's, his talks are well worth viewing. I agree that Microsoft COM reduces your C++ API to an early 1990s level of C++, but that unfortunately is still the current state of the art because of the wider C++ community's continuing failure to resolve Exported Templates - though, with C++ Modules we are most of the way there now, indeed Chandler was joking last C++ Now that clang might just implement exported templates for fun as they are trivial once Modules are fully implemented.

What you describe is fine, when you need all of it, but overkill when you don't. The OP has agreed to change the name. It doesn't need the features you describe to be useful and those features can be built atop this library. ___ Rob (Sent from my portable computation engine)

On August 17, 2014 8:05:32 AM EDT, Bjorn Reese wrote:

On 08/15/2014 11:34 PM, Rob Stewart wrote:

...

What that term implies is based upon one's experience. "Plugin", to me is any dynamically loaded component. The ABI you mention depends entirely upon the ubiquity of the plugin. If it must be usable by many applications and customers, it must have a documented, even standardized, API. Just as easily, however, one can create plugins based solely upon dlopen() (or LoadLibrary()) and friends of the plugins are used only for one application.

My interpretation of a "plugin" is a bit different. Normally a component decides its own interface and the application has to adjust to it. In the opposite case, when an application dictates the interface and the component has to adjust, then it is called a plugin. Dynamically loading is orthogonal to this.

I was a little loose in my description. I didn't mean to suggest that just calling dlopen() means you have a plugin. Your plugin description is reasonable. ___ Rob (Sent from my portable computation engine)

On Sat, Aug 16, 2014 at 10:12 AM, Antony Polukhin wrote:

Answering to all the naming proposals: SharedLibray/DLL/DynamicLinkLibrary sound too platform specific. Plugin sounds more generic, thou it confuses users. How about Dynamic Library Load and namespace boost::dll:: ?

Dynamic Library seems to be a nice compromise. By the way, would it be problematic to reuse the "extension" name?

2014-08-16 16:29 GMT+04:00 Hartmut Kaiser : <...>

...

...

- the ability to extract a list of plugin object names from a given shared library ( I assume that your library allows to have several plugin types in the same shared library)

Could be queried using the bool res = lib.search_symbol("some_name_of_the_plugin")

That's not what I meant. You're suggesting to find a known symbol. I was suggesting to find the list of names for the existing (unknown) plugins.

This can be currently achieved using the following: // in plugin/dll namespace foo { std::vectorstd::string list_plugins(); } BOOST_PLUGIN_ALIAS(foo::list_plugins, list_plugins) // in code that uses plugin shared_library lib(path_to_lib); auto plugins = lib.getstd::string()>("list_plugins"); for (auto& s: plugins) { ... lib.get(s) ... } Hmmm... This does not look user-friendly. I could do some experiments with putting the aliased names into a separate section of the binary and iterating over values in that section. Will iteration over BOOST_PLUGIN_ALIASed symbols suit you? Iterating over all the exported symbols does not look useful: there;ll be a lot of mangled names and sompiler-specific trash. -- Best regards, Antony Polukhin

2014-08-16 20:35 GMT+04:00 Niall Douglas :

On 16 Aug 2014 at 12:12, Antony Polukhin wrote:

...

Answering to all the naming proposals: SharedLibray/DLL/DynamicLinkLibrary sound too platform specific. Plugin sounds more generic, thou it confuses users. How about Dynamic Library Load and namespace boost::dll:: ?

I wonder if Microsoft have trademarked "DLL"? :)

Possible trademark threat is exactly why name "Plugin" was chosen over "DLL" :) -- Best regards, Antony Polukhin

On 15 Aug 2014 at 18:39, Hartmut Kaiser wrote:

- the ability to extract the full file system path of a loaded shared library (this is not always the same as the file name you specified to load the library)

Yes, a very good point. I can supply some reference code once working on all major platforms which could be derived from to implement this. The code can take any arbitrary pointer to memory and tell you which shared library "owns" it. The code is not fast, but it can be cached and it's very, very useful for implementing everything from per-plugin human language string translations to debugging tracebacks.

- the ability to extract a list of plugin object names from a given shared library ( I assume that your library allows to have several plugin types in the same shared library)

A mangled symbol to partial AST generator is the ultimate solution here and would be an enormously valuable addition to Boost. It's probably its own standalone library though. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

2014-08-31 19:18 GMT+04:00 :

On 2014-08-15 07:36, Antony Polukhin wrote:

There is a basic implementation of a header only Plugin library:

...

https://github.com/apolukhin/Boost.Plugin

Documentation with examples could be found here: http://apolukhin.github.io/Boost.Plugin/index.html

Is there an interest in such library in Boost community? What features are missing in the library?

Looking forward to hearing from you. Any feedback is more than welcomed.

Yes, tons of interest.

Looking at the documentation, 2 features seem lacking; correct me if I am wrong.

1) Notification of shared library unloading and loading. If users and library/framework stick to the reference counting, which is recommended, how can a framework get an opportunity to do things when the end users/references finally disappear. While loading I may want to read a manifest, add services to a services registry. While unloading I may delete a dated version of the shared library as a more recent dated version of the same library has already been loaded. ... For web development or any development I would like to hot swap to speed development by shortening the redeploy test cycle like found in Java JSPs, Java JRebel or any dynamic language. To build limited interface based variations of this, notification seems essential.

Well, in DLL/DSO you can track loads unloads using some global variables: struct do_at_load { do_at_load() { std::cout << "Loading\n"; } ~do_at_load() { std::cout << "Unloading\n"; } }; do_at_load instance; I'll update the examples to reflect this.

2) How do we make our plugins object methods return references to other shared pointers which is also tied to the shared library counter? A better nested object example is in order.

This is up to the plugin developer. However a few functions could make a developer's life easier. I was thinking of adding a function `shared_ptr this_library();` which would return an instance of a shared library in which `this_library()` function was called. Thanks to Niall, now I have some code that does things close to `this_library()`. -- Best regards, Antony Polukhin

2014-09-02 13:49 GMT+04:00 Niall Douglas :

On 2 Sep 2014 at 11:14, Antony Polukhin wrote:

...

...

1) Notification of shared library unloading and loading.

Well, in DLL/DSO you can track loads unloads using some global variables:

I think he meant that he wants the ability to hook into the load/unload of some arbitrary DLL/DSO, so like a DLL/DSO local atexit() as it were.

This won't be supported out of the box because there's no nice way to do it without overhead for users that do not require such functionality.

This is trivial to implement, simply a static object defined per DLL/DSO which accepts a list of bound functions to call when it gets destructed.

Something kinda cool would be combining that hook with the pointer-to-DLL/DSO code I sent you, so you could have code like:

// Tell me when the DLL/DSO owning this string gets unloaded so I can // delete my use of it dll_from_ptr(str)->on_unload(std::bind(this, &remove_string, str));

You are right. Added an example how to do it using the existing functionality of the library. Example took about 30 lines of code (including #includes, namespaces, comments, class decorations). -- Best regards, Antony Polukhin