[Boost.DLL] Formal Review request

older
[review] Formal review period for...

Antony Polukhin

2 Oct 2014 2 Oct '14

11:18 a.m.

Dear Boost, Some time passed since checking interest in plugin development library and now all the requested features are implemented (self loading, getting symbol location, better docs, automatic platform-specific library decorations append, much more examples, listing library exports and so on...). While there's still some room for minor improvements, library API is stable and the library will benefit greatly from a larger user base and more feedback. So unless someone thinks the library isn't ready or would get rejected right away in its current state for reason X, I am requesting a formal review for Boost.DLL. Library location: https://github.com/apolukhin/Boost.DLL Library Docs: http://apolukhin.github.io/Boost.DLL/index.html -- Best regards, Antony Polukhin

Show replies by date

Mathias Gaunard

2 Oct 2 Oct

12:18 p.m.

On 02/10/14 13:18, Antony Polukhin wrote:

...

Dear Boost,

Some time passed since checking interest in plugin development library and now all the requested features are implemented (self loading, getting symbol location, better docs, automatic platform-specific library decorations append, much more examples, listing library exports and so on...).

While there's still some room for minor improvements, library API is stable and the library will benefit greatly from a larger user base and more feedback.

So unless someone thinks the library isn't ready or would get rejected right away in its current state for reason X, I am requesting a formal review for Boost.DLL.

Library location: https://github.com/apolukhin/Boost.DLL Library Docs: http://apolukhin.github.io/Boost.DLL/index.html

Just a couple of pre-review comments: - I did not see any mention of previous work. I remember there used to be Boost.Plugin and Boost.Extension libraries written to address that same problem. - I did not find any rationale for the alias mechanism. Surely you can work with C++ symbols without needing to alias them with a C name?

Antony Polukhin

12:45 p.m.

2014-10-02 16:18 GMT+04:00 Mathias Gaunard :

...

On 02/10/14 13:18, Antony Polukhin wrote:

...
<...> So unless someone thinks the library isn't ready or would get rejected right away in its current state for reason X, I am requesting a formal review for Boost.DLL.

Library location: https://github.com/apolukhin/Boost.DLL Library Docs: http://apolukhin.github.io/Boost.DLL/index.html

Just a couple of pre-review comments:

- I did not see any mention of previous work. I remember there used to be Boost.Plugin and Boost.Extension libraries written to address that same problem.

Boost.Plugin is the previous name of the library (this can be seen in Revision History section). Some very intial version of shared_library class was taken from Boost.Application library (there's a note in README.md, thou it must be duplicated in Revision History section). I've discovered Boost.Extension project somewhere in the middle of Boost.DLL development, but no ideas were taken from it. - I did not find any rationale for the alias mechanism. Surely you can work

...

with C++ symbols without needing to alias them with a C name?

Yes, you can. Though C++ name will be mangled and some `void boost::foo(std::sting)` will change to something like `N5boostN3foosE`. Importing function by `N5boostN3foosE` name does not looks user friendly, especially assuming the fact that different compilers have different mangling scheme. The alias name - is a not mangled name for C++ symbol. -- Best regards, Antony Polukhin

Mathias Gaunard

3 Oct 3 Oct

9:44 a.m.

On 02/10/14 14:45, Antony Polukhin wrote:

...

Yes, you can. Though C++ name will be mangled and some `void boost::foo(std::sting)` will change to something like `N5boostN3foosE`. Importing function by `N5boostN3foosE` name does not looks user friendly, especially assuming the fact that different compilers have different mangling scheme.

The alias name - is a not mangled name for C++ symbol.

Your library could simply be extended to support automatic mangling/demangling for whatever platform you're running it on. All compilers provide functions for demangling (__cxa_demangle, UnDecorateSymbolName/__unDName), however I'm not aware of functions being provided for mangling.

Antony Polukhin

10:28 a.m.

2014-10-03 13:44 GMT+04:00 Mathias Gaunard :

...

On 02/10/14 14:45, Antony Polukhin wrote:

Yes, you can. Though C++ name will be mangled and some `void

...
boost::foo(std::sting)` will change to something like `N5boostN3foosE`. Importing function by `N5boostN3foosE` name does not looks user friendly, especially assuming the fact that different compilers have different mangling scheme.

The alias name - is a not mangled name for C++ symbol.

Your library could simply be extended to support automatic mangling/demangling for whatever platform you're running it on. All compilers provide functions for demangling (__cxa_demangle, UnDecorateSymbolName/__unDName), however I'm not aware of functions being provided for mangling.

I've tried to do so, but mangling is a very big unsolvable (on a library level) problem. Mangling depends on source code, for example "boost::foo" could be foo function or foo variable. Depending on that knowledge it must be mangled in different ways. More problems arise if foo is an overloaded function that accepts parameters: "boost::foo(variant)". In that case full name of parameter must be specified, which could be boost::variant or variant ... There was an idea to allow user to forward declare function and generate mangled name from it: namespace boost { void foo(variant); } boost::dll::magic_mangle(boost::foo); But that idea has epic failed... Because of linker problems and no reliable way to get mangled symbol name from compiler internals at compile time. At least I have not found one (any ideas are welcomed). That's why aliases were considered a lesser evil: BOOST_DLL_ALIAS(boost::foo, foo_variant) // in plugin "foo_variant" // in plugin importer P.S.: I'll add this question to design rationale -- Best regards, Antony Polukhin

Klaim - Joël Lamotte

1:24 p.m.

Would it be possible to provide a macro that generate a symbol name automatically following an arbitrary (but relatively safe) set of rules instead of asking the user to do it? Instead of : BOOST_DLL_ALIAS( my_namespace::cpp_function_name, cpp_function_alias_name ) Something like: BOOST_DLL_AUTO_ALIAS( my_namespace::cpp_function_name ) Not sure if it's doable but I think it would ease usability in most cases.

Klaim - Joël Lamotte

1:34 p.m.

On Fri, Oct 3, 2014 at 3:24 PM, Klaim - Joël Lamotte wrote:

...

BOOST_DLL_AUTO_ALIAS( my_namespace::cpp_function_name )

Not sure if it's doable but I think it would ease usability in most cases.

I meant that I suppose that it's doable because of type_index generating names which could be transformed to generate these alias, but I might be wrong.

Klaim - Joël Lamotte

20 Nov 20 Nov

1:09 p.m.

Antony, On Fri, Oct 3, 2014 at 3:24 PM, Klaim - Joël Lamotte wrote:

...

Instead of :

BOOST_DLL_ALIAS( my_namespace::cpp_function_name, cpp_function_alias_name )

Something like:

BOOST_DLL_AUTO_ALIAS( my_namespace::cpp_function_name )

did you have the time to consider this suggestion? Is this doable? I don't see it in the develop branch so if you found an issue with this I would like to learn from it. Thanks for your time.

Antony Polukhin

2:29 p.m.

2014-11-20 17:09 GMT+04:00 Klaim - Joël Lamotte :

...

Antony,

On Fri, Oct 3, 2014 at 3:24 PM, Klaim - Joël Lamotte wrote:

...
Instead of :

BOOST_DLL_ALIAS( my_namespace::cpp_function_name, cpp_function_alias_name )

Something like:

BOOST_DLL_AUTO_ALIAS( my_namespace::cpp_function_name )

did you have the time to consider this suggestion? Is this doable? I don't see it in the develop branch so if you found an issue with this I would like to learn from it.

The idea is great, but I do not know how to implement it. The following macro: BOOST_DLL_ALIAS( my_namespace::cpp_function_name, cpp_function_alias_name ) is expanded into the following: extern "C" void* cpp_function_alias_name = &my_namespace::cpp_function_name; It means that BOOST_DLL_AUTO_ALIAS(my_namespace::cpp_function_name) must be able somehow to cut off the namespaces leaving only the variable/function name. -- Best regards, Antony Polukhin

Klaim - Joël Lamotte

2:54 p.m.

On Thu, Nov 20, 2014 at 3:29 PM, Antony Polukhin wrote:

...

It means that BOOST_DLL_AUTO_ALIAS(my_namespace::cpp_function_name) must be able somehow to cut off the namespaces leaving only the variable/function name.

I was imagining that there would be a way to just replace "::"with _ (and not lose the namespace) to get: extern "C" void* my_namespace_cpp_function_alias_name = &my_namespace::cpp_function_name; I'm not experienced with macros so I'm not sure if it's doable. Maybe separating the differents name parts would work: BOOST_DLL_AUTO_ALIAS( namespaceA, namespaceB, function_name) // implemented with recursion for the namespaces? extern "C" void* namespaceA_namespaceB_function_name = &namespaceA:: namespaceB::cpp_function_name; But it seems less "elegant".

Antony Polukhin

21 Nov 21 Nov

2:06 p.m.

2014-11-20 18:54 GMT+04:00 Klaim - Joël Lamotte :

...

On Thu, Nov 20, 2014 at 3:29 PM, Antony Polukhin wrote:

...
It means that BOOST_DLL_AUTO_ALIAS(my_namespace::cpp_function_name) must be able somehow to cut off the namespaces leaving only the variable/function name.

I was imagining that there would be a way to just replace "::"with _ (and not lose the namespace) to get:

extern "C" void* my_namespace_cpp_function_alias_name = &my_namespace::cpp_function_name;

I'm not experienced with macros so I'm not sure if it's doable. Maybe separating the differents name parts would work:

BOOST_DLL_AUTO_ALIAS( namespaceA, namespaceB, function_name) // implemented with recursion for the namespaces?

extern "C" void* namespaceA_namespaceB_function_name = &namespaceA:: namespaceB::cpp_function_name;

But it seems less "elegant".

I've committed BOOST_DLL_AUTO_ALIAS macro. It's not exactly what you wanted, take a look and tell me what you think http://apolukhin.github.io/Boost.DLL/boost_dll/introduction0.html I've also added new refcounting import functions that automatically decide to return shared_ptr<T> or function<T>, according to the type of T. -- Best regards, Antony Polukhin

Klaim - Joël Lamotte

4:11 p.m.

On Fri, Nov 21, 2014 at 3:06 PM, Antony Polukhin wrote:

...

I've committed BOOST_DLL_AUTO_ALIAS macro. It's not exactly what you wanted, take a look and tell me what you think http://apolukhin.github.io/Boost.DLL/boost_dll/introduction0.html

If I understand correctly, in exchange of requiring to be used in the argument name's scope, it does exactly the same thing as BOOST_DLL_ALIAS() except it will generate a mangled name without the namespace scope? If yes, I think it's a good shortcut for most cases.

...

I've also added new refcounting import functions that automatically decide to return shared_ptr<T> or function<T>, according to the type of T.

-- Best regards, Antony Polukhin

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Niall Douglas

3 Oct 3 Oct

2:54 p.m.

On 3 Oct 2014 at 14:28, Antony Polukhin wrote:

...

...
Your library could simply be extended to support automatic mangling/demangling for whatever platform you're running it on. All compilers provide functions for demangling (__cxa_demangle, UnDecorateSymbolName/__unDName), however I'm not aware of functions being provided for mangling.

I've tried to do so, but mangling is a very big unsolvable (on a library level) problem. Mangling depends on source code, for example "boost::foo" could be foo function or foo variable. Depending on that knowledge it must be mangled in different ways. More problems arise if foo is an overloaded function that accepts parameters: "boost::foo(variant)". In that case full name of parameter must be specified, which could be boost::variant or variant ...

There was an idea to allow user to forward declare function and generate mangled name from it:

namespace boost { void foo(variant); }

boost::dll::magic_mangle(boost::foo);

But that idea has epic failed... Because of linker problems and no reliable way to get mangled symbol name from compiler internals at compile time. At least I have not found one (any ideas are welcomed).

Both Itanium and MSVC mangling schemes happily have consistent template parameter mangling, so if you wrap the thing you want to mangle like the above and have template<class C> magic_mangle(C &&f) call __func__ or its equivalent to fetch the mangled name of the templated function, extracting the mangled template parameter is a simple string chop. You chop off a fixed length beginning and end, and prepend a little bit of mangling depending on its type of function if it's a function and you should be done. If you remember Antony I sent you a little demo of this back during the review of TypeIndex.

...

That's why aliases were considered a lesser evil:

BOOST_DLL_ALIAS(boost::foo, foo_variant) // in plugin "foo_variant" // in plugin importer

The neato thing about such mangling generation support is that one can examine, in a natural way, a series of DLLs for the presence of some C++ API. You simply supply the prototype and name of what you want, and it finds it. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Andrey Semashev

3:09 p.m.

On Friday 03 October 2014 15:54:24 Niall Douglas wrote:

...

On 3 Oct 2014 at 14:28, Antony Polukhin wrote:

Both Itanium and MSVC mangling schemes happily have consistent template parameter mangling, so if you wrap the thing you want to mangle like the above and have template<class C> magic_mangle(C &&f) call __func__ or its equivalent to fetch the mangled name of the templated function, extracting the mangled template parameter is a simple string chop. You chop off a fixed length beginning and end, and prepend a little bit of mangling depending on its type of function if it's a function and you should be done.

If you remember Antony I sent you a little demo of this back during the review of TypeIndex.

I think any such technique is non-portable since there is no portable way to obtain mangled name of a function in compile time. It works in TypeIndex because of its specifics, which I believe don't apply to Boost.DLL. It is unspecified what string identifies a type and you generally don't care about its format. This allows to use demangled function names or whatever __func__ returns in TypeIndex. This is not the case in a library ABI, which you define by Boost.DLL macros. You should be able to precisely tell the symbol names you export, and these names should be compatible with whatever restrictions the target system requires. I.e. the standard __func__ will likely not work.

...

...
That's why aliases were considered a lesser evil:

BOOST_DLL_ALIAS(boost::foo, foo_variant) // in plugin "foo_variant" // in plugin importer

The neato thing about such mangling generation support is that one can examine, in a natural way, a series of DLLs for the presence of some C++ API. You simply supply the prototype and name of what you want, and it finds it.

We should not forget about binding symbols in other languages.

Niall Douglas

10:59 p.m.

On 3 Oct 2014 at 19:09, Andrey Semashev wrote:

...

...
Both Itanium and MSVC mangling schemes happily have consistent template parameter mangling, so if you wrap the thing you want to mangle like the above and have template<class C> magic_mangle(C &&f) call __func__ or its equivalent to fetch the mangled name of the templated function, extracting the mangled template parameter is a simple string chop. You chop off a fixed length beginning and end, and prepend a little bit of mangling depending on its type of function if it's a function and you should be done.

If you remember Antony I sent you a little demo of this back during the review of TypeIndex.

I think any such technique is non-portable since there is no portable way to obtain mangled name of a function in compile time.

Who cares. All the major compilers do provide this, and a large number of the minor ones. When myself and Antony surveyed the situation during TypeIndex we were pleasantly surprised at just how widespread support is for some magic macro which expands to the mangled form of the enclosing function. Mangling schemes have also basically reduced to just two kinds in recent compilers, with the Microsoft mangling scheme considerably the harder to parse. Mangling schemes are also very stable over time. That makes this proposal tractable.

...

It works in TypeIndex because of its specifics, which I believe don't apply to Boost.DLL. It is unspecified what string identifies a type and you generally don't care about its format. This allows to use demangled function names or whatever __func__ returns in TypeIndex.

This is not the case in a library ABI, which you define by Boost.DLL macros. You should be able to precisely tell the symbol names you export, and these names should be compatible with whatever restrictions the target system requires. I.e. the standard __func__ will likely not work.

Depends on the use case. Any Boost.DLL should be capable on inspecting arbitrary shared libraries, not necessarily ones exporting a Boost.DLL built interface. You're right if you want MSVC to inspect a Mingw DLL things get tricky.

...

...
...
That's why aliases were considered a lesser evil:

BOOST_DLL_ALIAS(boost::foo, foo_variant) // in plugin "foo_variant" // in plugin importer

The neato thing about such mangling generation support is that one can examine, in a natural way, a series of DLLs for the presence of some C++ API. You simply supply the prototype and name of what you want, and it finds it.

We should not forget about binding symbols in other languages.

On this part I do agree - a generalised partial AST library for constructing and destructing arbitrary mangled symbols would be super. However, it is not a trivial amount of work, some months of someone's time would probably be needed (I suggest replicating clang's internal C++ AST API, might as well not reinvent the wheel). Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Andrey Semashev

4 Oct 4 Oct

8:41 a.m.

On Friday 03 October 2014 23:59:03 Niall Douglas wrote:

...

On 3 Oct 2014 at 19:09, Andrey Semashev wrote:

...
I think any such technique is non-portable since there is no portable way to obtain mangled name of a function in compile time.

Who cares.

I do, obviously.

...

All the major compilers do provide this, and a large number of the minor ones. When myself and Antony surveyed the situation during TypeIndex we were pleasantly surprised at just how widespread support is for some magic macro which expands to the mangled form of the enclosing function. Mangling schemes have also basically reduced to just two kinds in recent compilers, with the Microsoft mangling scheme considerably the harder to parse. Mangling schemes are also very stable over time. That makes this proposal tractable.

We had a similar discussion already. IMO, the library in its core should not rely on compiler specific features and should be implementable in pure C++ and standard APIs. This ensures its portability. It doesn't matter how many compilers implement the extension because there will always be the one that doesn't. In any case, I don't think that mangled names are useful for other reasons I mentioned.

Mathias Gaunard

10:10 a.m.

On 04/10/2014 10:41, Andrey Semashev wrote:

...

We had a similar discussion already. IMO, the library in its core should not rely on compiler specific features

You do realize DLLs and DSOs are already compiler-specific features?

Andrey Semashev

10:19 a.m.

On Saturday 04 October 2014 12:10:48 Mathias Gaunard wrote:

...

On 04/10/2014 10:41, Andrey Semashev wrote:

...
We had a similar discussion already. IMO, the library in its core should not rely on compiler specific features

You do realize DLLs and DSOs are already compiler-specific features?

I'd say it's a feature that depends on the system API but not necessarily on the compiler. It does require mangling code and as such the support from the compiler on Windows but not other systems.

Niall Douglas

11:24 a.m.

On 4 Oct 2014 at 12:41, Andrey Semashev wrote:

...

...
...
I think any such technique is non-portable since there is no portable way to obtain mangled name of a function in compile time.

Who cares.

I do, obviously.

The library abstracts a whole ton of non-portable stuff around DLLs. Mangling is just another one of those.

...

...
All the major compilers do provide this, and a large number of the minor ones. When myself and Antony surveyed the situation during TypeIndex we were pleasantly surprised at just how widespread support is for some magic macro which expands to the mangled form of the enclosing function. Mangling schemes have also basically reduced to just two kinds in recent compilers, with the Microsoft mangling scheme considerably the harder to parse. Mangling schemes are also very stable over time. That makes this proposal tractable.

We had a similar discussion already. IMO, the library in its core should not rely on compiler specific features and should be implementable in pure C++ and standard APIs. This ensures its portability. It doesn't matter how many compilers implement the extension because there will always be the one that doesn't.

Andrey, please list here three C++ compilers which had at least one version release in the past five years which doesn't provide a magic macro expanding out to the mangling of the enclosing function.

...

In any case, I don't think that mangled names are useful for other reasons I mentioned.

With such support, the following: plugin = dll::import_variable_alias( lib_path / "my_plugin_sum", "plugin", dll::load_mode::append_decorations ); can become: BOOST_DLL_IMPORT( lib_path / "my_plugin_sum", BOOST_DLL_AUTO_ALIAS(my_namespace::plugin), // import a variable BOOST_DLL_AUTO_ALIAS(my_namespace::function), // import a function BOOST_DLL_AUTO_ALIAS(my_namespace::object), // import a class implementation ... ) For those very few compilers without sufficient support, I am sure a BOOST_DLL_EXPLICIT_ALIAS macro could let you manually specify the mangling for some imported symbol. This would be useful in any case for MSVC to inspect Mingw binaries or vice versa. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Andrey Semashev

11:56 a.m.

On Saturday 04 October 2014 12:24:23 Niall Douglas wrote:

...

On 4 Oct 2014 at 12:41, Andrey Semashev wrote:

...
We had a similar discussion already. IMO, the library in its core should not rely on compiler specific features and should be implementable in pure C++ and standard APIs. This ensures its portability. It doesn't matter how many compilers implement the extension because there will always be the one that doesn't.

Andrey, please list here three C++ compilers which had at least one version release in the past five years which doesn't provide a magic macro expanding out to the mangling of the enclosing function.

I don't have such a survey, but I suspect you can't guarantee that every compiler out there supports it either.

...

...
In any case, I don't think that mangled names are useful for other reasons I mentioned.

With such support, the following:

plugin = dll::import_variable_alias( lib_path / "my_plugin_sum", "plugin", dll::load_mode::append_decorations );

can become:

BOOST_DLL_IMPORT( lib_path / "my_plugin_sum", BOOST_DLL_AUTO_ALIAS(my_namespace::plugin), // import a variable BOOST_DLL_AUTO_ALIAS(my_namespace::function), // import a function BOOST_DLL_AUTO_ALIAS(my_namespace::object), // import a class implementation ... )

I'm not sure how is that an improvement. Maybe I just missed the point of your example.

...

For those very few compilers without sufficient support, I am sure a BOOST_DLL_EXPLICIT_ALIAS macro could let you manually specify the mangling for some imported symbol. This would be useful in any case for MSVC to inspect Mingw binaries or vice versa.

I don't mind if the library offers an optional functionality to obtain a mangled name of a symbol. But it should be exactly that - an optional feature, which is only supported on select compilers and works on certain platforms. In its core the library should still provide a platform-neutral user interface.

Antony Polukhin

12:07 p.m.

2014-10-04 15:24 GMT+04:00 Niall Douglas :

...

On 4 Oct 2014 at 12:41, Andrey Semashev wrote: <...>

...
All the major compilers do provide this, and a large

...
number of the minor ones. When myself and Antony surveyed the situation during TypeIndex we were pleasantly surprised at just how widespread support is for some magic macro which expands to the mangled form of the enclosing function. Mangling schemes have also basically reduced to just two kinds in recent compilers, with the Microsoft mangling scheme considerably the harder to parse. Mangling schemes are also very stable over time. That makes this proposal tractable.

We had a similar discussion already. IMO, the library in its core should not rely on compiler specific features and should be implementable in pure C++ and standard APIs. This ensures its portability. It doesn't matter how many compilers implement the extension because there will always be the one that doesn't.

Andrey, please list here three C++ compilers which had at least one version release in the past five years which doesn't provide a magic macro expanding out to the mangling of the enclosing function.

...
In any case, I don't think that mangled names are useful for other reasons I mentioned.

With such support, the following:

plugin = dll::import_variable_alias( lib_path / "my_plugin_sum", "plugin", dll::load_mode::append_decorations );

can become:

BOOST_DLL_IMPORT( lib_path / "my_plugin_sum", BOOST_DLL_AUTO_ALIAS(my_namespace::plugin), // import a variable BOOST_DLL_AUTO_ALIAS(my_namespace::function), // import a function BOOST_DLL_AUTO_ALIAS(my_namespace::object), // import a class implementation ... )

TypeIndex approach works for object types, but not for function names. It will return function signature, not its real name: BOOST_DLL_AUTO_ALIAS(my_namespace::function) == "void(int)" // not a mangled "my_namespace::adl_barrier::function(int)" string I see no way to extract function name from compiler internals. Implementing mangling for variables only is a task that does not worth wasting time, because exporting functions is a much more popular case than exporting variables. Any attempt to implement mangling of strings from scratch (from "my_namespace::function" for example) will cause many hard detectable errors. It would be also hard for user to use such macro, because provided string must specify full typename with namespace for every input variable. For a function that accepts some boost::variant or boost::mpl::vector variables such string could be more than 10 Kbytes. Portability and reliability of such method will not match high standards of Boost. -- Best regards, Antony Polukhin

Niall Douglas

7:58 p.m.

On 4 Oct 2014 at 16:07, Antony Polukhin wrote:

...

BOOST_DLL_AUTO_ALIAS(my_namespace::function) == "void(int)" // not a mangled "my_namespace::adl_barrier::function(int)" string

I see no way to extract function name from compiler internals. Implementing mangling for variables only is a task that does not worth wasting time, because exporting functions is a much more popular case than exporting variables.

I'd like to see everything possible supported. You could define a dummy type where the variable lives in its namespaces, and then deconstruct the variable mangling from that. Append something like $$$___$$$ to the variable name in the dummy type, that makes it very easy to remove in the mangling. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Mathias Gaunard

12:42 p.m.

On 04/10/2014 13:24, Niall Douglas wrote:

...

Andrey, please list here three C++ compilers which had at least one version release in the past five years which doesn't provide a magic macro expanding out to the mangling of the enclosing function.

What about MinGW?

Niall Douglas

10:34 p.m.

On 4 Oct 2014 at 14:42, Mathias Gaunard wrote:

...

On 04/10/2014 13:24, Niall Douglas wrote:

...
Andrey, please list here three C++ compilers which had at least one version release in the past five years which doesn't provide a magic macro expanding out to the mangling of the enclosing function.

What about MinGW?

I just ran the following program on MinGW: #include <iostream> #include <typeinfo> template<class T> void doprint(T a) { std::cout << __func__ << std::endl; std::cout << __FUNCTION__ << std::endl; std::cout << __PRETTY_FUNCTION__ << std::endl; struct $$$ { }; std::cout << typeid($$$).name() << std::endl; } template<class T> struct doprint$$$___$$$ { }; int main(void) { doprint(1); std::cout << typeid(doprint$$$___$$$<int>).name() << std::endl; return 0; } It prints: doprint doprint void doprint(T) [with T = int] Z7doprintIiEvT_E3$$$ 16doprint$$$___$$$IiE Something I was wrong about is that there is no equivalent to __FUNCDSIG__ in GCC - I had thought that __FUNCTION__ produced mangled symbols, and I was wrong on that - I can find no magic macro for GCC which can. You can however still fall back onto typeid, albeit this forces a dependency on RTTI. I suppose one could just parse one's own ELF binary headers to look up mangled symbols you yourself instantiated, but that is getting a bit extreme. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Antony Polukhin

5 Oct 5 Oct

5:28 a.m.

2014-10-05 2:34 GMT+04:00 Niall Douglas :

...

On 4 Oct 2014 at 14:42, Mathias Gaunard wrote:

...
On 04/10/2014 13:24, Niall Douglas wrote:

...
Andrey, please list here three C++ compilers which had at least one version release in the past five years which doesn't provide a magic macro expanding out to the mangling of the enclosing function.

What about MinGW?

I just ran the following program on MinGW:

#include <iostream> #include <typeinfo>

template<class T> void doprint(T a) { std::cout << __func__ << std::endl; std::cout << __FUNCTION__ << std::endl; std::cout << __PRETTY_FUNCTION__ << std::endl; struct $$$ { }; std::cout << typeid($$$).name() << std::endl; } template<class T> struct doprint$$$___$$$ { };

int main(void) { doprint(1); std::cout << typeid(doprint$$$___$$$<int>).name() << std::endl; return 0; }

It prints:

doprint doprint void doprint(T) [with T = int] Z7doprintIiEvT_E3$$$ 16doprint$$$___$$$IiE

That's exactly what I'm talking about. typeid returns *types*, while Boost.DLL works with exported *instances*. In other words: #include <typeinfo> #include <iostream> #include namespace foo { namespace adl_barrier { boost::variant export_me_function(boost::variant, boost::variant) { return 0; } boost::variant export_me_variable; } // namespace adl_barrier using adl_barrier::export_me_function; using adl_barrier::export_me_variable; } int main() { std:: cout << typeid(foo::export_me_function).name() << std::endl; std:: cout << typeid(foo::export_me_variable).name() << std::endl; } The program from above will output: FN5boost7variantIisNS_6detail7variant5void_ES3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_EES4_S4_E N5boost7variantIisNS_6detail7variant5void_ES3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_S3_EE However we need the following: _ZN3foo11adl_barrier18export_me_functionEN5boost7variantIisNS1_6detail7variant5void_ES5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_EES6_ _ZN3foo11adl_barrier18export_me_variableE -- Best regards, Antony Polukhin

Mathias Gaunard

5:23 p.m.

On 05/10/2014 07:28, Antony Polukhin wrote:

...

That's exactly what I'm talking about. typeid returns *types*, while Boost.DLL works with exported *instances*.

For importing, all you have is name + type anyway. The question is: is there an easy scheme that allows you to compute the name of the symbol from that information? If there isn't, I suppose the current approach of aliasing is suitable. Personally, I only use C functions in my ABI, but if we ever wanted to make this standard-proof, it would need to have no such restrictions.

Gavin Lambert

9:56 p.m.

On 6/10/2014 06:23, Mathias Gaunard wrote:

...

Personally, I only use C functions in my ABI, but if we ever wanted to make this standard-proof, it would need to have no such restrictions.

Given that neither C++ mangling nor any other details of the C++ ABI are standardised (AFAIK), I don't think anything involving C++ *could* be standard-proof.

Mathias Gaunard

10:45 p.m.

On 05/10/2014 23:56, Gavin Lambert wrote:

...

On 6/10/2014 06:23, Mathias Gaunard wrote:

...
Personally, I only use C functions in my ABI, but if we ever wanted to make this standard-proof, it would need to have no such restrictions.

Given that neither C++ mangling nor any other details of the C++ ABI are standardised (AFAIK), I don't think anything involving C++ *could* be standard-proof.

I'm talking about a possible submission to the standard library. There is no need to specify a mangling ABI to specify a standard library for this.

Andrew Marlow

6 Oct 6 Oct

9:07 a.m.

For anyone who is interested in boost.DLL, as a user, implementor, reviewer or just curious, I encourage them to look at ACE DLL. ACE has been there, done that. I know that ACE code looks horrible due to its age and the need to support ancient compilers, and there are ugly macros all over the place. However, ACE has solved this problem so I think developers that want to solve the problem again should look at how it has been done before. For example, I see from the boost.DLL example that the entire pathname, including any OS-specific suffixes etc, is supplied to the ctor. ACE takes a different view and it is instructive to look at the comments in the code to see why. I am thinking of different versions of UNIX in particular and how they handle shared libraries differently. ACE has also thought about shared library lifetimes quite thoroughly and caters for a lifetime that might be greater than the immediate object that is doing symbol lookup for example. This is in the DLL_Manager class. On 5 October 2014 23:45, Mathias Gaunard wrote:

...

On 05/10/2014 23:56, Gavin Lambert wrote:

...
On 6/10/2014 06:23, Mathias Gaunard wrote:

...
Personally, I only use C functions in my ABI, but if we ever wanted to make this standard-proof, it would need to have no such restrictions.

Given that neither C++ mangling nor any other details of the C++ ABI are standardised (AFAIK), I don't think anything involving C++ *could* be standard-proof.

I'm talking about a possible submission to the standard library. There is no need to specify a mangling ABI to specify a standard library for this.

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/ mailman/listinfo.cgi/boost

-- Regards, Andrew Marlow http://www.andrewpetermarlow.co.uk

Antony Polukhin

9:38 a.m.

2014-10-06 13:07 GMT+04:00 Andrew Marlow :

...

For anyone who is interested in boost.DLL, as a user, implementor, reviewer or just curious, I encourage them to look at ACE DLL. ACE has been there, done that.

I've took a look at it and saw no advantages over Boost.DLL. Moreover, it looks overcomplicated at some places (DLL_Manager class is redundant, OS takes care of this; not unloading plugins looks like a strategy from pure C that is not good in C++; no UTF16 support and so on...)

...

I know that ACE code looks horrible due to its age and the need to support ancient compilers, and there are ugly macros all over the place. However, ACE has solved this problem so I think developers that want to solve the problem again should look at how it has been done before.

For example, I see from the boost.DLL example that the entire pathname, including any OS-specific suffixes etc, is supplied to the ctor. ACE takes a different view and it is instructive to look at the comments in the code to see why. I am thinking of different versions of UNIX in particular and how they handle shared libraries differently.

Use `load_mode::append_decorations` to avoid specifying platform specific decorations. Not specifying a path leads to many problems. Take a look at the WinAPIs mess of "add-search-path/remove-search-path/add-maybe-before-search-path/add-search-path-before-before". It's hell, global variables are not good in modern multi threaded programs.

...

ACE has also thought about shared library lifetimes quite thoroughly and caters for a lifetime that might be greater than the immediate object that is doing symbol lookup for example. This is in the DLL_Manager class.

Please take a closer look to the Boost.DLL library and tell me what is exactly missing. -- Best regards, Antony Polukhin

Bjorn Reese

9:25 a.m.

On 10/06/2014 12:45 AM, Mathias Gaunard wrote:

...

I'm talking about a possible submission to the standard library.

http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2014/n4028.pdf

Antony Polukhin

5 Oct 5 Oct

6:12 a.m.

2014-10-05 2:34 GMT+04:00 Niall Douglas :

...

On 4 Oct 2014 at 14:42, Mathias Gaunard wrote:

...
On 04/10/2014 13:24, Niall Douglas wrote:

...
Andrey, please list here three C++ compilers which had at least one version release in the past five years which doesn't provide a magic macro expanding out to the mangling of the enclosing function.

What about MinGW?

I just ran the following program on MinGW:

#include <iostream> #include <typeinfo>

template<class T> void doprint(T a) { std::cout << __func__ << std::endl; std::cout << __FUNCTION__ << std::endl; std::cout << __PRETTY_FUNCTION__ << std::endl; struct $$$ { }; std::cout << typeid($$$).name() << std::endl; } template<class T> struct doprint$$$___$$$ { };

int main(void) { doprint(1); std::cout << typeid(doprint$$$___$$$<int>).name() << std::endl; return 0; }

It prints:

doprint doprint void doprint(T) [with T = int] Z7doprintIiEvT_E3$$$ 16doprint$$$___$$$IiE

I've enchanted the previous example and the results are not very good: #include <typeinfo> #include <iostream> #include namespace foo { namespace adl_barrier { BOOST_SYMBOL_EXPORT boost::variant export_me_function(boost::variant, boost::variant) { return 0; } BOOST_SYMBOL_EXPORT boost::variant export_me_variable; } // namespace adl_barrier using adl_barrier::export_me_function; using adl_barrier::export_me_variable; } template & var> struct printer {}; template (&var)(boost::variant, boost::variant)> struct printer2 {}; int main() { std:: cout << typeid(printer2foo::export_me_function).name() << std::endl; std:: cout << typeid(printerfoo::export_me_variable).name() << std::endl; } Code from above will output: Linux: 8printer2ILZN3foo11adl_barrier18export_me_functionEN5boost7variantIisNS2_6detail7variant5void_ES6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_EES7_EE 7printerILZN3foo11adl_barrier18export_me_variableEEE Windows: .?AU?$printer2@$1?export_me_function@adl_barrier@foo@ @YA?AV?$variant@HFUvoid_@0detail@boost@@U1023@U1023@U1023@U1023@U1023@U1023 @U1023@U1023@U1023@U1023@U10 23@U1023@U1023@U1023@U1023@U1023@U1023@@boost@@V45@0@Z@@ .?AU?$printer@$E?export_me_variable@adl_barrier@foo@@3V?$variant@HFUvoid_ @0detail@boost@@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023 @U1023@U 1023@U1023@U1023@U1023@U1023@U1023@@boost@@A@@ While we still need the following: Linux: _ZN3foo11adl_barrier18export_me_functionEN5boost7variantIisNS1_6detail7variant5void_ES5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_EES6_ _ZN3foo11adl_barrier18export_me_variableE Windows: ?export_me_function@adl_barrier@foo@@YA?AV?$variant@HFUvoid_@0detail@boost @@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023 @U1023@U1 023@U1023@U1023@U1023@@boost@@V34@0@Z ?export_me_variable@adl_barrier@foo@@3V?$variant@HFUvoid_@0detail@boost @@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023 @U1023@U1023 @U1023@U1023@U1023@@boost@@A Looks like such method is not very good, function definition has changed (Linux:5 was transformed into 6, 1 was tranformed into 2; Windows: V34 changed into V45)... I'm afraid that there'll be more surprises -- Best regards, Antony Polukhin

Niall Douglas

6 Oct 6 Oct

11:51 p.m.

On 5 Oct 2014 at 10:12, Antony Polukhin wrote: Let me reorg this for you ...

...

Code from above will output: Linux: 8printer2ILZN3foo11adl_barrier18export_me_functionEN5boost7variantIisNS2_6detail7variant5void_ES6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_S6_EES7_EE 7printerILZN3foo11adl_barrier18export_me_variableEEE

While we still need the following: Linux: _ZN3foo11adl_barrier18export_me_functionEN5boost7variantIisNS1_6detail7variant5void_ES5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_S5_EES6_ _ZN3foo11adl_barrier18export_me_variableE

Code from above will output: Windows: .?AU?$printer2@$1?export_me_function@adl_barrier@foo@ @YA?AV?$variant@HFUvoid_@0detail@boost@@U1023@U1023@U1023@U1023@U1023@U1023 @U1023@U1023@U1023@U1023@U10 23@U1023@U1023@U1023@U1023@U1023@U1023@@boost@@V45@0@Z@@ .?AU?$printer@$E?export_me_variable@adl_barrier@foo@@3V?$variant@HFUvoid_ @0detail@boost@@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023 @U1023@U 1023@U1023@U1023@U1023@U1023@U1023@@boost@@A@@

While we still need the following: Windows: ?export_me_function@adl_barrier@foo@@YA?AV?$variant@HFUvoid_@0detail@boost @@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023 @U1023@U1 023@U1023@U1023@U1023@@boost@@V34@0@Z ?export_me_variable@adl_barrier@foo@@3V?$variant@HFUvoid_@0detail@boost @@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023@U1023 @U1023@U1023 @U1023@U1023@U1023@@boost@@A

Looks like such method is not very good, function definition has changed (Linux:5 was transformed into 6, 1 was tranformed into 2;

Yes, that's the name compression scheme used to avoid repetition, so they use numbers to refer to earlier decls to avoid spelling them out in full each time. When wrapped with printer, you can safely subtract one from all of those numbers as you know printer will consume the first number. For Itanium, an S prefix means "use the n-th decl from the beginning".

...

Windows: V34 changed into V45)... I'm afraid that there'll be more surprises

For the same reason, earlier mangled decls int and short are offset by one, hence the V34 => V45. There is an alternative, but it is slightly evil: how about one only provides type safety for imported functions and variables which are in at least one namespace level? One then can replace one of those namespaces with printer<>, and therefore not modify the remainder of the mangling? To test that, I modified your original program like this: #include <typeinfo> #include <iostream> #include <utility> struct void_ { }; template struct variant : std::pair { variant() { } variant(A a, B b) : std::pair(a, b) {} }; namespace foo { namespace adl_barrier { __declspec(dllexport) variant export_me_function(variant, variant) { return variant(0, 0); } __declspec(dllexport) variant export_me_variable; } // namespace adl_barrier using adl_barrier::export_me_function; using adl_barrier::export_me_variable; } // These being the dummy types purely used to get the mangling from them namespace $$magic$$_adl_barrier { extern variant export_me_function(variant, variant); extern variant export_me_variable; } template & var> struct $$magic$$_vprinter {}; template (&var)(variant, variant)> struct $$magic$$_fprinter {}; int main() { std:: cout << typeid($$magic$$_fprinter<$$magic$$_adl_barrier::export_me_function>). name() << std::endl; std:: cout << typeid($$magic$$_vprinter<$$magic$$_adl_barrier::export_me_variable>). name() << std::endl; } Prints: 18$$magic$$_fprinterILZN21$$magic$$_adl_barrier18export_me_functionE7v ariantIis5void_S2_S2_ES3_EE 18$$magic$$_vprinterILZN21$$magic$$_adl_barrier18export_me_variableEEE Exported: __ZN3foo11adl_barrier18export_me_functionE7variantIis5void_S2_S2_ES3_ __ZN3foo11adl_barrier18export_me_variableE ... which looks bang on. Simply replace the 8fprinter with 3foo and remove the $$magic$$_. In case that seems too hacky, another option is to name the decl for the printer to be identical to the first namespace, that way the decl gets reused instead of incrementing the back reference count. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Antony Polukhin

7 Oct 7 Oct

12:58 p.m.

2014-10-07 3:51 GMT+04:00 Niall Douglas :

...

<...> Prints:

18$$magic$$_fprinterILZN21$$magic$$_adl_barrier18export_me_functionE7v ariantIis5void_S2_S2_ES3_EE 18$$magic$$_vprinterILZN21$$magic$$_adl_barrier18export_me_variableEEE

Exported:

__ZN3foo11adl_barrier18export_me_functionE7variantIis5void_S2_S2_ES3_ __ZN3foo11adl_barrier18export_me_variableE

... which looks bang on. Simply replace the 8fprinter with 3foo and remove the $$magic$$_.

Still looks like something error-prone. Actually, user could always get the list of mangled symbols using the boost::dll::library_info class methods on a compiled binary. So instead of using mangling voodoo macro and explicitly declaring exports in import module, user can find all the export_me_* mangled names and import them using any of the interfaces. Something close can be found in this example: http://apolukhin.github.io/Boost.DLL/boost_dll/tutorial.html#boost_dll.tutor... ... some dream of a nicest solution: Add attribute [[abi_name "new_abi_name"]], remove all the *_alias methods and member function, remove _ALIAS macro. Now user is free to do following things: // in plugin: namespace foo { namespace adl_barrier { [[abi_name "export_me_function_1"]] BOOST_SYMBOL_EXPORT boost::variant export_me_function( boost::variant, boost::variant) { return 0; } [[abi_name "export_me_variable_1"]] BOOST_SYMBOL_EXPORT boost::variant export_me_variable; } // namespace adl_barrier } // in importer: dll::shared_library lib(lib_path); typedef boost::variant vis_t; auto f = lib.get("export_me_function_1"); auto v = lib.get("export_me_variable_1"); Code from above looks nice, but requires abi_name attribute that was not even proposed to C++ standardization and until now existed only in my head. -- Best regards, Antony Polukhin

Niall Douglas

5:31 p.m.

On 7 Oct 2014 at 16:58, Antony Polukhin wrote:

...

...
<...> Prints:

18$$magic$$_fprinterILZN21$$magic$$_adl_barrier18export_me_functionE7v ariantIis5void_S2_S2_ES3_EE 18$$magic$$_vprinterILZN21$$magic$$_adl_barrier18export_me_variableEEE

Exported:

__ZN3foo11adl_barrier18export_me_functionE7variantIis5void_S2_S2_ES3_ __ZN3foo11adl_barrier18export_me_variableE

... which looks bang on. Simply replace the 8fprinter with 3foo and remove the $$magic$$_.

Still looks like something error-prone.

Not for the Itanium ABI which is very well understood and publicly documented. You can write a simple [1] Itanium type mangler using pure C++ metaprogramming, it's trivial in fact. The MSVC ABI is considerably harder. This is why I am advocating getting the compiler to do the mangling for you, then you transform its result according to some simple heuristics. [1]: By simple I mean only types with no repetition compression.

...

Actually, user could always get the list of mangled symbols using the boost::dll::library_info class methods on a compiled binary. So instead of using mangling voodoo macro and explicitly declaring exports in import module, user can find all the export_me_* mangled names and import them using any of the interfaces. Something close can be found in this example: http://apolukhin.github.io/Boost.DLL/boost_dll/tutorial.html#boost_dll.tutor...

Oh sure. But I think what we'd all love is for some long overdue type safety when binding symbols at runtime in C++. Otherwise out of step binaries make segfaults, security holes and buffer overruns instead of link failures.

...

... some dream of a nicest solution: Add attribute [[abi_name "new_abi_name"]], remove all the *_alias methods and member function, remove _ALIAS macro. Now user is free to do following things:

// in plugin: namespace foo { namespace adl_barrier {

[[abi_name "export_me_function_1"]] BOOST_SYMBOL_EXPORT boost::variant export_me_function( boost::variant, boost::variant) { return 0; }

[[abi_name "export_me_variable_1"]] BOOST_SYMBOL_EXPORT boost::variant export_me_variable; } // namespace adl_barrier }

// in importer: dll::shared_library lib(lib_path); typedef boost::variant vis_t; auto f = lib.get("export_me_function_1"); auto v = lib.get("export_me_variable_1");

Code from above looks nice, but requires abi_name attribute that was not even proposed to C++ standardization and until now existed only in my head.

Personally speaking, if a proper Componentisation ABI is considered still too hard for us, I'd simply standardise on the Itanium ABI. Your DLL library could provide a portable method for auto generating aliases with the Itanium ABI. As I mentioned, it is fairly trivial to write some metaprogramming which will output an Itanium mangling for some input type. You just then need to regex in the fully qualified name of the item, and finally run a pass collapsing repetition (a regex matcher will suffiice). Itanium is very logical, no surprises. I keep a quick cheat sheet document on symbol mangling at https://github.com/ned14/NiallsCPP11Utilities/blob/master/demanglers/c alling_conventions.pdf, you may find it useful. The official http://mentorembedded.github.io/cxx-abi/abi.html#mangling isn't bad either. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Mathias Gaunard

8 Oct 8 Oct

9:23 a.m.

On 07/10/14 19:31, Niall Douglas wrote:

...

Your DLL library could provide a portable method for auto generating aliases with the Itanium ABI. As I mentioned, it is fairly trivial to write some metaprogramming which will output an Itanium mangling for some input type. You just then need to regex in the fully qualified name of the item, and finally run a pass collapsing repetition (a regex matcher will suffiice). Itanium is very logical, no surprises. I keep a quick cheat sheet document on symbol mangling at https://github.com/ned14/NiallsCPP11Utilities/blob/master/demanglers/c alling_conventions.pdf, you may find it useful. The official http://mentorembedded.github.io/cxx-abi/abi.html#mangling isn't bad either.

Since you seem to know the subject matter very well yourself, wouldn't it be better if you could contribute that component?

Niall Douglas

11:35 a.m.

On 8 Oct 2014 at 11:23, Mathias Gaunard wrote:

...

...
Your DLL library could provide a portable method for auto generating aliases with the Itanium ABI. As I mentioned, it is fairly trivial to write some metaprogramming which will output an Itanium mangling for some input type. You just then need to regex in the fully qualified name of the item, and finally run a pass collapsing repetition (a regex matcher will suffiice). Itanium is very logical, no surprises. I keep a quick cheat sheet document on symbol mangling at https://github.com/ned14/NiallsCPP11Utilities/blob/master/demanglers/c alling_conventions.pdf, you may find it useful. The official http://mentorembedded.github.io/cxx-abi/abi.html#mangling isn't bad either.

Since you seem to know the subject matter very well yourself, wouldn't it be better if you could contribute that component?

Well, I do have a lot of demands on my time these next few months. Currently three new Boost libraries will be appearing from me by March 2015, so I am not volunteering for even more work. That said, given how trivial the above metaprogrammed Itanium type mangler is, I would be happy to collaborate with someone else on an implementation. You'll just need to be patient with my rate of progress, I only get Saturdays most weeks for non-work work items and that is usually spent on closing Boost.Thread defects or trying to find some time to work on AFIO. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

Antony Polukhin

13 Oct 13 Oct

8:16 a.m.

2014-10-08 15:35 GMT+04:00 Niall Douglas :

...

On 8 Oct 2014 at 11:23, Mathias Gaunard wrote:

...
...
Your DLL library could provide a portable method for auto generating aliases with the Itanium ABI. As I mentioned, it is fairly trivial to write some metaprogramming which will output an Itanium mangling for some input type. You just then need to regex in the fully qualified name of the item, and finally run a pass collapsing repetition (a regex matcher will suffiice). Itanium is very logical, no surprises. I keep a quick cheat sheet document on symbol mangling at https://github.com/ned14/NiallsCPP11Utilities/blob/master/demanglers/c alling_conventions.pdf, you may find it useful. The official http://mentorembedded.github.io/cxx-abi/abi.html#mangling isn't bad either.

Since you seem to know the subject matter very well yourself, wouldn't it be better if you could contribute that component?

Well, I do have a lot of demands on my time these next few months. Currently three new Boost libraries will be appearing from me by March 2015, so I am not volunteering for even more work.

That said, given how trivial the above metaprogrammed Itanium type mangler is, I would be happy to collaborate with someone else on an implementation. You'll just need to be patient with my rate of progress, I only get Saturdays most weeks for non-work work items and that is usually spent on closing Boost.Thread defects or trying to find some time to work on AFIO.

I'd appreciate any help. Task of name mangling does not seem review blocking and can be easily done later at any time. Here's just an idea how importing could be done in a nice way without aliases if we had a fully functional mangler: dll::shared_library lib(path_to_lib); typedef boost::variant vis_t; // mangle_get method has signature of the method and it's full name, // so a Mangler could create a correct mangled name without forward declaration auto f = lib.mangle_get("foo::adl_barrier:: export_me_function"); // mangle_get method has type of the variable and it's full name, // so a Mangler could create a correct mangled name without forward declaration auto v = lib.mangle_get("foo::adl_barrier::export_me_variable"); -- Best regards, Antony Polukhin

Niall Douglas

8:36 p.m.

On 13 Oct 2014 at 12:16, Antony Polukhin wrote:

...

...
Well, I do have a lot of demands on my time these next few months. Currently three new Boost libraries will be appearing from me by March 2015, so I am not volunteering for even more work.

That said, given how trivial the above metaprogrammed Itanium type mangler is, I would be happy to collaborate with someone else on an implementation. You'll just need to be patient with my rate of progress, I only get Saturdays most weeks for non-work work items and that is usually spent on closing Boost.Thread defects or trying to find some time to work on AFIO.

I'd appreciate any help. Task of name mangling does not seem review blocking and can be easily done later at any time.

Here's just an idea how importing could be done in a nice way without aliases if we had a fully functional mangler:

dll::shared_library lib(path_to_lib); typedef boost::variant vis_t;

// mangle_get method has signature of the method and it's full name, // so a Mangler could create a correct mangled name without forward declaration auto f = lib.mangle_get("foo::adl_barrier:: export_me_function");

I'd be much happier with: auto f = BOOST_DLL_MANGLE_GET(lib, (vis_t), ((foo), (adl_barrier), (export_me_function)), (vis_t, vis_t)); That way we can use macro metaprogramming too. And use any compile-time mangling facilities I'd really like compiler vendors to add (e.g. http://llvm.org/bugs/show_bug.cgi?id=21263).

...

// mangle_get method has type of the variable and it's full name, // so a Mangler could create a correct mangled name without forward declaration auto v = lib.mangle_get("foo::adl_barrier::export_me_variable");

You might find the following of interest Antony. If you remember my local namespace binding tool I am intending to split off Boost libraries so they are standalone, such forked Boost library headers might take something resembling the following preamble: #include "local-bind-cpp-library/include/import.hpp" #define BOOST_SPINLOCK_V1 (boost), (spinlock), (v1, inline) #ifndef BOOST_SPINLOCK_V1_STL11_IMPL #define BOOST_SPINLOCK_V1_STL11_IMPL std #endif #define BOOST_SPINLOCK_V1_NAMESPACE BOOST_LOCAL_BIND_NAMESPACE (BOOST_SPINLOCK_V1) #define BOOST_SPINLOCK_V1_NAMESPACE_BEGIN BOOST_LOCAL_BIND_NAMESPACE_BEGIN(BOOST_SPINLOCK_V1) #define BOOST_SPINLOCK_V1_NAMESPACE_END BOOST_LOCAL_BIND_NAMESPACE_END (BOOST_SPINLOCK_V1) #define BOOST_STL11_ATOMIC_MAP_NAMESPACE_BEGIN BOOST_LOCAL_BIND_NAMESPACE_BEGIN(BOOST_SPINLOCK_V1, (stl11, inline)) #define BOOST_STL11_ATOMIC_MAP_NAMESPACE_END BOOST_LOCAL_BIND_NAMESPACE_END (BOOST_SPINLOCK_V1, (stl11, inline)) #define BOOST_STL11_CHRONO_MAP_NAMESPACE_BEGIN BOOST_LOCAL_BIND_NAMESPACE_BEGIN(BOOST_SPINLOCK_V1, (stl11, inline), (chrono)) #define BOOST_STL11_CHRONO_MAP_NAMESPACE_END BOOST_LOCAL_BIND_NAMESPACE_END (BOOST_SPINLOCK_V1, (stl11, inline), (chrono)) #define BOOST_STL11_MUTEX_MAP_NAMESPACE_BEGIN BOOST_LOCAL_BIND_NAMESPACE_BEGIN(BOOST_SPINLOCK_V1, (stl11, inline)) #define BOOST_STL11_MUTEX_MAP_NAMESPACE_END BOOST_LOCAL_BIND_NAMESPACE_END (BOOST_SPINLOCK_V1, (stl11, inline)) #define BOOST_STL11_THREAD_MAP_NAMESPACE_BEGIN BOOST_LOCAL_BIND_NAMESPACE_BEGIN(BOOST_SPINLOCK_V1, (stl11, inline)) #define BOOST_STL11_THREAD_MAP_NAMESPACE_END BOOST_LOCAL_BIND_NAMESPACE_END (BOOST_SPINLOCK_V1, (stl11, inline)) #include BOOST_LOCAL_BIND_INCLUDE_STL11(BOOST_SPINLOCK_V1_STL11_IMPL, atomic) #include BOOST_LOCAL_BIND_INCLUDE_STL11(BOOST_SPINLOCK_V1_STL11_IMPL, chrono) #include BOOST_LOCAL_BIND_INCLUDE_STL11(BOOST_SPINLOCK_V1_STL11_IMPL, mutex) #include BOOST_LOCAL_BIND_INCLUDE_STL11(BOOST_SPINLOCK_V1_STL11_IMPL, thread) BOOST_SPINLOCK_V1_NAMESPACE_BEGIN BOOST_LOCAL_BIND_DECLARE(BOOST_SPINLOCK_V1) ... normal header stuff ... BOOST_SPINLOCK_V1_NAMESPACE_END You can see a real example at https://github.com/ned14/boost.spinlock/blob/expected_future/include/s pinlock.hpp#L68 What the above does is to allow the following: 1. External code can choose the C++ STL 11 implementation, boost or std, using BOOST_SPINLOCK_V1_STL11_IMPL. Those are bound directly into the boost::spinlock::v1 namespace. 2. Multiple versions of Boost.Spinlock can coexist in the same process thanks to C++ 11's inline namespaces 3. Multiple variants of Boost.Spinlock (e.g. one with boost, one with std) can also coexist in the same process. There is a minor bug in the above macro metaprogramming about that, but it's on my todo, it'll work right soon. I also have realised that local namespace bindings are really your library *exactly* specifying its dependencies, so I now have a new way of importing a bind-capable library directly into your library without affecting other libraries i.e. Library A can import Boost.Spinlock v1.01 using the std C++ 11 STL while Library B can import Boost Spinlock v1.02 using the boost C++ 11 STL and it all "just works" plus Library A's configuration does not interfere with Library B's configuration. To therefore import locally, not globally to all the code which includes your header file, you do: #define BOOST_SPINLOCK_MAP_NAMESPACE_BEGIN BOOST_LOCAL_BIND_NAMESPACE_BEGIN(MY_NAMESPACE, (spinlock)) #define BOOST_SPINLOCK_MAP_NAMESPACE_END BOOST_LOCAL_BIND_NAMESPACE_END (MY_NAMESPACE, (spinlock)) #include "boost/spinlock.hpp" This gains us a poor man's implementation of C++ 17 Modules today using namespace trickery. I think it's pretty neato. There are quite a few opportunites for Boost.DLL here, because I tag bind-capable namespaces so they can be identified by examining the binary. I see no reason why the bindings generator couldn't also spit out an inspection function, then a program can very easily inspect a library and say *exactly* what its dependencies are and moreover, *exactly* what depends on that library. That helps solve the packaging and distribution problem. More info explaining the above macro programming at the newly updated Readme at https://github.com/ned14/local-bind-cpp-library. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/

arvid

5 Oct 5 Oct

6:10 p.m.

On 2014-10-02 05:45, Antony Polukhin wrote:

...

2014-10-02 16:18 GMT+04:00 Mathias Gaunard : [...] - I did not find any rationale for the alias mechanism. Surely you can work

...
with C++ symbols without needing to alias them with a C name?

Yes, you can. Though C++ name will be mangled and some `void boost::foo(std::sting)` will change to something like `N5boostN3foosE`. Importing function by `N5boostN3foosE` name does not looks user friendly, especially assuming the fact that different compilers have different mangling scheme.

The alias name - is a not mangled name for C++ symbol.

In my mind, the single most useful feature of a library like this would be to add type-safety to GetProcAddress(), essentially. i.e. specify the the type and name, the library mangles it and loads it and returns the correct function pointer type. It's not clear to me why you need to wrap the functions in a std::function (but I've only skimmed the documentation still). -- Arvid Norberg

Rob Stewart

8:49 p.m.

On October 5, 2014 2:10:24 PM EDT, arvid wrote:

...

On 2014-10-02 05:45, Antony Polukhin wrote:

...
2014-10-02 16:18 GMT+04:00 Mathias Gaunard : [...] - I did not find any rationale for the alias mechanism. Surely you can work

...
with C++ symbols without needing to alias them with a C name?

Yes, you can. Though C++ name will be mangled and some `void boost::foo(std::sting)` will change to something like `N5boostN3foosE`. Importing function by `N5boostN3foosE` name does not looks user friendly, especially assuming the fact that different compilers have different mangling scheme.

The alias name - is a not mangled name for C++ symbol.

In my mind, the single most useful feature of a library like this would be to add type-safety to GetProcAddress(), essentially.

i.e. specify the the type and name, the library mangles it and loads it and returns the correct function pointer type.

There is definitely value in a portable interface over an inherently unsafe interface. Any name mangling functionality would be an additional benefit. ___ Rob (Sent from my portable computation engine)

3659

Age (days ago)

3709

Last active (days ago)

List overview

Download

40 comments

10 participants

participants (10)

Andrew Marlow
Andrey Semashev
Antony Polukhin
arvid
Bjorn Reese
Gavin Lambert
Klaim - Joël Lamotte
Mathias Gaunard
Niall Douglas
Rob Stewart