[RPC] Compile-time reflection for RPC and mocking

Peter Bindels

8 Dec 2014 8 Dec '14

2:27 p.m.

Hi all, I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms. Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as the arguments (basically the mangled name of the interface function) to ensure there are no mismatches. I would like to take this oppurtunity to propose two things: - I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about the argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the link code), but I don't know whether that's true. On the second note, I also proposed a Mock library a few years ago. This basically fell flat on the code that I suggested being very unsafe and using many hackish methods to mock functions. Using the compile-time reflection proposed for the second bullet I would be able to replace the entire hackishness from that with 100% compliant code, with only one minor regression in functionality. That should make it Boost-OK at least from a language standards perspective. Any ideas? Reactions on RPC or Mocking? Best regards, Peter Bindels

Show replies by date

Edward Diener

8 Dec 8 Dec

7:43 p.m.

On 12/8/2014 9:27 AM, Peter Bindels wrote:

...

Hi all,

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as the arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about the argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the link code), but I don't know whether that's true.

On the second note, I also proposed a Mock library a few years ago. This basically fell flat on the code that I suggested being very unsafe and using many hackish methods to mock functions. Using the compile-time reflection proposed for the second bullet I would be able to replace the entire hackishness from that with 100% compliant code, with only one minor regression in functionality. That should make it Boost-OK at least from a language standards perspective.

Any ideas? Reactions on RPC or Mocking?

I worked on a programming project under Windows a number of years back involving RPC and I remember how difficult it was to get everything correct using standard C++, VC++, and IDL. So I can definitely see the use for a Boost RPC which would make implementing RPC interfaces using template programming much easier.

Damien Buhl

7:50 p.m.

Hi Peter, Were you at the meetingcpp in Berlin ? I also was there, sad that we didn't got in contact, because I've been working on such kind of things for a while now (many years, from compiler plugins for gcc, clang to a c++-only / macro / template metaprogramming version). I would find it nice to see something good solving the issue of remote procedure call and data field synchronization, because I've a not-so-much working solution that I want to opensource at some point when it will be good enough, but it would be of great benefit to get others ideas/impl, because I've been essentially working alone and I'm at the 5th rewrite already. :p In any case I would be really interested on such a library. For example you could use mechanisms like BOOST_FUSION_ADAPT_STRUCT to "automatically" introspects classes in order to generate as well serialized output and function calls. We could even add extensions to fusion to allow more introsprection of adapted types. What I want when I'll get my last changes on BOOST_FUSION_ADAPT_STRUCT working on MSVC too, is to improve it in order to have it automatically generate the boilerplate code to implement compile time reflections proposals to iterate over structs or adt fields, bases etc. And to implement an API near to the proposals ones. I'm wondering how I could use Abel's Sinckovicks metaparse library to improve the emulation for compile time reflection support, so that it don't need to specify fields manually. I think there already is Boost.Rpc in the sandbox, but I feel it didn't evolved much lastly or am I wrong ? Do you meant basing your work on it ? Regarding Boost HPX, I believe they already is an internal library for this kind of automatic remote rpc, did you mean to use their serializers based on spirit and so ? I've always been dreaming of things like : auto remote_obj = std::make_remotely_shared<MyClass>(); remote_obj->call_remote_service(); remote_obj->remote_field = "assign it remotely"; So yes I would definitely love to see others people working on these points, and could even give a hand because I anyway invest weekly time in this topic alone in my cellar :) . Cheers, -- Damien Buhl On 12/08/2014 03:27 PM, Peter Bindels wrote:

...

Hi all,

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as the arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about the argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the link code), but I don't know whether that's true.

On the second note, I also proposed a Mock library a few years ago. This basically fell flat on the code that I suggested being very unsafe and using many hackish methods to mock functions. Using the compile-time reflection proposed for the second bullet I would be able to replace the entire hackishness from that with 100% compliant code, with only one minor regression in functionality. That should make it Boost-OK at least from a language standards perspective.

Any ideas? Reactions on RPC or Mocking?

Best regards, Peter Bindels

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Oleg Labutin

8:46 p.m.

No any questions, currenlty I'd like to say about point. Basically in a C++ we should have one mechanism to implement compile time reflection This is registering type, and functions-type. And compile time reflection which was proposed, in hl library it is only this step by some approaches. could I point C++ code struct my_reflected_type { void (my_function)(int (first), int (second)) { } Field(int, my_field0); Field(std::string, my_field1); }; REGISTER_TYPE_SERVICE(classes_stg, my_reflected_type); struct my_reflected_type0 { void (foo)(my_reflected_type (first), int (second)) { } void (on_service_call)(my_reflected_type (first)) { std::cout << first.my_field0 << std::endl; } Field(int, my_field0); Field(std::string, my_field1); }; REGISTER_TYPE_SERVICE(classes_stg, my_reflected_type0); void main_call(std::wstring const& json) { my_reflected_type0().call_by_json (getjson(json), 0); // here // get boost::v_item classes typedef typename extract >::type classes; // here // get boost::v_item ,0>, 0> accessors typedef typename extract_class ::type accessors; typename find_meth_by_accessor::type accessor; } It's really working, without fusion, we can make fields and methods in a different scope . Thanks 2014-12-08 21:50 GMT+02:00 Damien Buhl :

...

Hi Peter,

Were you at the meetingcpp in Berlin ? I also was there, sad that we didn't got in contact, because I've been working on such kind of things for a while now (many years, from compiler plugins for gcc, clang to a c++-only / macro / template metaprogramming version).

I would find it nice to see something good solving the issue of remote procedure call and data field synchronization, because I've a not-so-much working solution that I want to opensource at some point when it will be good enough, but it would be of great benefit to get others ideas/impl, because I've been essentially working alone and I'm at the 5th rewrite already. :p

In any case I would be really interested on such a library. For example you could use mechanisms like BOOST_FUSION_ADAPT_STRUCT to "automatically" introspects classes in order to generate as well serialized output and function calls. We could even add extensions to fusion to allow more introsprection of adapted types.

What I want when I'll get my last changes on BOOST_FUSION_ADAPT_STRUCT working on MSVC too, is to improve it in order to have it automatically generate the boilerplate code to implement compile time reflections proposals to iterate over structs or adt fields, bases etc. And to implement an API near to the proposals ones.

I'm wondering how I could use Abel's Sinckovicks metaparse library to improve the emulation for compile time reflection support, so that it don't need to specify fields manually.

I think there already is Boost.Rpc in the sandbox, but I feel it didn't evolved much lastly or am I wrong ? Do you meant basing your work on it ?

Regarding Boost HPX, I believe they already is an internal library for this kind of automatic remote rpc, did you mean to use their serializers based on spirit and so ?

I've always been dreaming of things like : auto remote_obj = std::make_remotely_shared<MyClass>();

remote_obj->call_remote_service(); remote_obj->remote_field = "assign it remotely";

So yes I would definitely love to see others people working on these points, and could even give a hand because I anyway invest weekly time in this topic alone in my cellar :) .

Cheers, -- Damien Buhl

...
Hi all,

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as

...
arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about

On 12/08/2014 03:27 PM, Peter Bindels wrote: the the

...
argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the link code), but I don't know whether that's true.

On the second note, I also proposed a Mock library a few years ago. This basically fell flat on the code that I suggested being very unsafe and using many hackish methods to mock functions. Using the compile-time reflection proposed for the second bullet I would be able to replace the entire hackishness from that with 100% compliant code, with only one minor regression in functionality. That should make it Boost-OK at least from a language standards perspective.

Any ideas? Reactions on RPC or Mocking?

Best regards, Peter Bindels

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Damien Buhl

8:54 p.m.

Thanks Oleg for pointing it out, I remember the discussion about it on this list for some month. Is there any documentation available or example we can compile and play with about your hl library ? Cheers ;) On 12/08/2014 09:46 PM, Oleg Labutin wrote:

...

No any questions, currenlty I'd like to say about point. Basically in a C++ we should have one mechanism to implement compile time reflection This is registering type, and functions-type. And compile time reflection which was proposed, in hl library it is only this step by some approaches.

could I point C++ code

struct my_reflected_type { void (my_function)(int (first), int (second)) {

}

Field(int, my_field0); Field(std::string, my_field1); };

REGISTER_TYPE_SERVICE(classes_stg, my_reflected_type);

struct my_reflected_type0 { void (foo)(my_reflected_type (first), int (second)) {

}

void (on_service_call)(my_reflected_type (first)) { std::cout << first.my_field0 << std::endl; }

Field(int, my_field0); Field(std::string, my_field1); };

REGISTER_TYPE_SERVICE(classes_stg, my_reflected_type0);

void main_call(std::wstring const& json) { my_reflected_type0().call_by_json (getjson(json), 0);

// here // get boost::v_item classes typedef typename extract >::type classes;

// here // get boost::v_item ,0>, 0> accessors typedef typename extract_class ::type accessors;

typename find_meth_by_accessor::type accessor; }

It's really working, without fusion, we can make fields and methods in a different scope .

Thanks

2014-12-08 21:50 GMT+02:00 Damien Buhl :

...
Hi Peter,

Were you at the meetingcpp in Berlin ? I also was there, sad that we didn't got in contact, because I've been working on such kind of things for a while now (many years, from compiler plugins for gcc, clang to a c++-only / macro / template metaprogramming version).

I would find it nice to see something good solving the issue of remote procedure call and data field synchronization, because I've a not-so-much working solution that I want to opensource at some point when it will be good enough, but it would be of great benefit to get others ideas/impl, because I've been essentially working alone and I'm at the 5th rewrite already. :p

In any case I would be really interested on such a library. For example you could use mechanisms like BOOST_FUSION_ADAPT_STRUCT to "automatically" introspects classes in order to generate as well serialized output and function calls. We could even add extensions to fusion to allow more introsprection of adapted types.

What I want when I'll get my last changes on BOOST_FUSION_ADAPT_STRUCT working on MSVC too, is to improve it in order to have it automatically generate the boilerplate code to implement compile time reflections proposals to iterate over structs or adt fields, bases etc. And to implement an API near to the proposals ones.

I'm wondering how I could use Abel's Sinckovicks metaparse library to improve the emulation for compile time reflection support, so that it don't need to specify fields manually.

I think there already is Boost.Rpc in the sandbox, but I feel it didn't evolved much lastly or am I wrong ? Do you meant basing your work on it ?

Regarding Boost HPX, I believe they already is an internal library for this kind of automatic remote rpc, did you mean to use their serializers based on spirit and so ?

I've always been dreaming of things like : auto remote_obj = std::make_remotely_shared<MyClass>();

remote_obj->call_remote_service(); remote_obj->remote_field = "assign it remotely";

So yes I would definitely love to see others people working on these points, and could even give a hand because I anyway invest weekly time in this topic alone in my cellar :) .

Cheers, -- Damien Buhl

...
Hi all,

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as

...
arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about

On 12/08/2014 03:27 PM, Peter Bindels wrote: the the

...
argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the link code), but I don't know whether that's true.

On the second note, I also proposed a Mock library a few years ago. This basically fell flat on the code that I suggested being very unsafe and using many hackish methods to mock functions. Using the compile-time reflection proposed for the second bullet I would be able to replace the entire hackishness from that with 100% compliant code, with only one minor regression in functionality. That should make it Boost-OK at least from a language standards perspective.

Any ideas? Reactions on RPC or Mocking?

Best regards, Peter Bindels

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Oleg Labutin

9:16 p.m.

Currently hl availble on https://github.com/hl-plus/. And certain module in which used compile time reflection is https://github.com/hl-plus/hl/blob/master/include/hl_ipc/luna_service_backen... (Obviously register_service(Service* svc) should be called for all sevices by iteration type list.) But I guess it very specific and related to luna (dbus based calling) service. And I hope, I can make more suitable for explanation simple project for demostrate tomorrow. Thanks 2014-12-08 22:54 GMT+02:00 Damien Buhl :

...

Thanks Oleg for pointing it out, I remember the discussion about it on this list for some month.

Is there any documentation available or example we can compile and play with about your hl library ?

Cheers ;)

...
No any questions, currenlty I'd like to say about point. Basically in a C++ we should have one mechanism to implement compile time reflection This is registering type, and functions-type. And compile time reflection which was proposed, in hl library it is only this step by some approaches.

could I point C++ code

struct my_reflected_type { void (my_function)(int (first), int (second)) {

}

Field(int, my_field0); Field(std::string, my_field1); };

REGISTER_TYPE_SERVICE(classes_stg, my_reflected_type);

struct my_reflected_type0 { void (foo)(my_reflected_type (first), int (second)) {

}

void (on_service_call)(my_reflected_type (first)) { std::cout << first.my_field0 << std::endl; }

Field(int, my_field0); Field(std::string, my_field1); };

REGISTER_TYPE_SERVICE(classes_stg, my_reflected_type0);

void main_call(std::wstring const& json) { my_reflected_type0().call_by_json (getjson(json), 0);

// here // get boost::v_item classes typedef typename extract >::type classes;

// here // get boost::v_item ,0>, 0> accessors typedef typename extract_class ::type accessors;

typename find_meth_by_accessor::type accessor; }

It's really working, without fusion, we can make fields and methods in a different scope .

Thanks

2014-12-08 21:50 GMT+02:00 Damien Buhl :

...
Hi Peter,

Were you at the meetingcpp in Berlin ? I also was there, sad that we didn't got in contact, because I've been working on such kind of things for a while now (many years, from compiler plugins for gcc, clang to a c++-only / macro / template metaprogramming version).

I would find it nice to see something good solving the issue of remote procedure call and data field synchronization, because I've a not-so-much working solution that I want to opensource at some point when it will be good enough, but it would be of great benefit to get others ideas/impl, because I've been essentially working alone and I'm at the 5th rewrite already. :p

In any case I would be really interested on such a library. For example you could use mechanisms like BOOST_FUSION_ADAPT_STRUCT to "automatically" introspects classes in order to generate as well serialized output and function calls. We could even add extensions to fusion to allow more introsprection of adapted types.

What I want when I'll get my last changes on BOOST_FUSION_ADAPT_STRUCT working on MSVC too, is to improve it in order to have it automatically generate the boilerplate code to implement compile time reflections proposals to iterate over structs or adt fields, bases etc. And to implement an API near to the proposals ones.

I'm wondering how I could use Abel's Sinckovicks metaparse library to improve the emulation for compile time reflection support, so that it don't need to specify fields manually.

I think there already is Boost.Rpc in the sandbox, but I feel it didn't evolved much lastly or am I wrong ? Do you meant basing your work on it ?

Regarding Boost HPX, I believe they already is an internal library for this kind of automatic remote rpc, did you mean to use their serializers based on spirit and so ?

I've always been dreaming of things like : auto remote_obj = std::make_remotely_shared<MyClass>();

remote_obj->call_remote_service(); remote_obj->remote_field = "assign it remotely";

So yes I would definitely love to see others people working on these points, and could even give a hand because I anyway invest weekly time in this topic alone in my cellar :) .

Cheers, -- Damien Buhl

...
Hi all,

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as

...
arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about

On 12/08/2014 03:27 PM, Peter Bindels wrote: the the

...
argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the

...
...
...
code), but I don't know whether that's true.

On the second note, I also proposed a Mock library a few years ago. This basically fell flat on the code that I suggested being very unsafe and using many hackish methods to mock functions. Using the compile-time reflection proposed for the second bullet I would be able to replace

On 12/08/2014 09:46 PM, Oleg Labutin wrote: link the

...
...
...
entire hackishness from that with 100% compliant code, with only one minor regression in functionality. That should make it Boost-OK at least from a language standards perspective.

Any ideas? Reactions on RPC or Mocking?

Best regards, Peter Bindels

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Oleg Labutin

9:29 p.m.

Thanks ;) 2014-12-08 23:16 GMT+02:00 Oleg Labutin :

...

Currently hl availble on https://github.com/hl-plus/. And certain module in which used compile time reflection is

https://github.com/hl-plus/hl/blob/master/include/hl_ipc/luna_service_backen... (Obviously register_service(Service* svc) should be called for all sevices by iteration type list.)

But I guess it very specific and related to luna (dbus based calling) service. And I hope, I can make more suitable for explanation simple project for demostrate tomorrow.

Thanks

2014-12-08 22:54 GMT+02:00 Damien Buhl :

...
Thanks Oleg for pointing it out, I remember the discussion about it on this list for some month.

Is there any documentation available or example we can compile and play with about your hl library ?

Cheers ;)

On 12/08/2014 09:46 PM, Oleg Labutin wrote:

...
No any questions, currenlty I'd like to say about point. Basically in a C++ we should have one mechanism to implement compile time reflection This is registering type, and functions-type. And compile time reflection which was proposed, in hl library it is only this step by some approaches.

could I point C++ code

struct my_reflected_type { void (my_function)(int (first), int (second)) {

}

Field(int, my_field0); Field(std::string, my_field1); };

REGISTER_TYPE_SERVICE(classes_stg, my_reflected_type);

struct my_reflected_type0 { void (foo)(my_reflected_type (first), int (second)) {

}

void (on_service_call)(my_reflected_type (first)) { std::cout << first.my_field0 << std::endl; }

Field(int, my_field0); Field(std::string, my_field1); };

REGISTER_TYPE_SERVICE(classes_stg, my_reflected_type0);

void main_call(std::wstring const& json) { my_reflected_type0().call_by_json (getjson(json), 0);

// here // get boost::v_item classes typedef typename extract >::type classes;

// here // get boost::v_item ,0>, 0> accessors typedef typename extract_class ::type accessors;

typename find_meth_by_accessor::type accessor; }

It's really working, without fusion, we can make fields and methods in a different scope .

Thanks

2014-12-08 21:50 GMT+02:00 Damien Buhl :

...
Hi Peter,

Were you at the meetingcpp in Berlin ? I also was there, sad that we didn't got in contact, because I've been working on such kind of things for a while now (many years, from compiler plugins for gcc, clang to a c++-only / macro / template metaprogramming version).

I would find it nice to see something good solving the issue of remote procedure call and data field synchronization, because I've a not-so-much working solution that I want to opensource at some point when it will be good enough, but it would be of great benefit to get others ideas/impl, because I've been essentially working alone and I'm at the 5th rewrite already. :p

In any case I would be really interested on such a library. For example you could use mechanisms like BOOST_FUSION_ADAPT_STRUCT to "automatically" introspects classes in order to generate as well serialized output and function calls. We could even add extensions to fusion to allow more introsprection of adapted types.

What I want when I'll get my last changes on BOOST_FUSION_ADAPT_STRUCT working on MSVC too, is to improve it in order to have it automatically generate the boilerplate code to implement compile time reflections proposals to iterate over structs or adt fields, bases etc. And to implement an API near to the proposals ones.

I'm wondering how I could use Abel's Sinckovicks metaparse library to improve the emulation for compile time reflection support, so that it don't need to specify fields manually.

I think there already is Boost.Rpc in the sandbox, but I feel it didn't evolved much lastly or am I wrong ? Do you meant basing your work on it ?

Regarding Boost HPX, I believe they already is an internal library for this kind of automatic remote rpc, did you mean to use their serializers based on spirit and so ?

I've always been dreaming of things like : auto remote_obj = std::make_remotely_shared<MyClass>();

remote_obj->call_remote_service(); remote_obj->remote_field = "assign it remotely";

So yes I would definitely love to see others people working on these points, and could even give a hand because I anyway invest weekly time in this topic alone in my cellar :) .

Cheers, -- Damien Buhl

...
Hi all,

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as

On 12/08/2014 03:27 PM, Peter Bindels wrote: the

...
arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would

collapse >> to >>> RPC::Register<T>(); To do so, it would need two language additions. It >>> needs access to a compile-time iterator over functions and traits about >> the >>> argument types, return type, name and whether it is a virtual function. >> For >>> the prototype this can be manually generated for an interface to show it >>> works. Additionally, it would need to be able to use such a definition to >>> instantiate a virtual function from a base class - basically, hooking up >> an >>> std::function<X> to a virtual function X directly. I think this is >>> well-implementable (probably reusing most code from lambdas for the link >>> code), but I don't know whether that's true. >>> >>> On the second note, I also proposed a Mock library a few years ago. This >>> basically fell flat on the code that I suggested being very unsafe and >>> using many hackish methods to mock functions. Using the compile-time >>> reflection proposed for the second bullet I would be able to replace the >>> entire hackishness from that with 100% compliant code, with only one >> minor >>> regression in functionality. That should make it Boost-OK at least from a >>> language standards perspective. >>> >>> Any ideas? Reactions on RPC or Mocking? >>> >>> Best regards, >>> Peter Bindels >>> >>> _______________________________________________ >>> Unsubscribe & other changes: >> http://lists.boost.org/mailman/listinfo.cgi/boost >>> >> >> _______________________________________________ >> Unsubscribe & other changes: >> http://lists.boost.org/mailman/listinfo.cgi/boost >> > > _______________________________________________ > Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost >

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Rob Stewart

11:34 p.m.

On December 8, 2014 4:29:34 PM EST, Oleg Labutin wrote:

...

Thanks ;)

2014-12-08 23:16 GMT+02:00 Oleg Labutin :

[snip]

...

...
2014-12-08 22:54 GMT+02:00 Damien Buhl :

[snip]

...

...
...
On 12/08/2014 09:46 PM, Oleg Labutin wrote:

[snip]

...

...
...
...
2014-12-08 21:50 GMT+02:00 Damien Buhl :

[snip]

...

...
...
...
...
On 12/08/2014 03:27 PM, Peter Bindels wrote:

[snip, including many ML footers] When posting to the Boost Developers mailing list, please follow the policies at http://www.boost.org/community/policy.html#quoting. Note particularly not to top post or over-quote. ___ Rob (Sent from my portable computation engine)

Bjorn Reese

9 Dec 9 Dec

11:10 a.m.

On 12/08/2014 03:27 PM, Peter Bindels wrote:

...

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses

We are seeing a general tendency away from RPC towards either ReST for client-server architectures or messaging middlewares for peer-to-peer architectures. If you are interested in working on alternative proposals, then one possibility could be to add a client-side for the Boost.Http project: https://github.com/BoostGSoC14/boost.http If you like to work with binary protocols, adding CoAP to the above project could also be useful.

...

macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize.

We already have Boost.Serialization and Boost.Fusion that are capable of serialization.

Peter Bindels

2:40 p.m.

On 9 December 2014 at 12:10, Bjorn Reese wrote:

...

On 12/08/2014 03:27 PM, Peter Bindels wrote:

- I can work on this idea to work it out into a full RPC mechanism, to be

...
proposed for Boost.RPC, if there is enough interest. Currently it uses

We are seeing a general tendency away from RPC towards either ReST for client-server architectures or messaging middlewares for peer-to-peer architectures.

Who's "We" ? Given the powerful abilities that you can get with async and future/promise, coupled with easy remoting with some rpc in between that gives the same interface, would lead me to believe RPC is due for a comeback. At least, for computing distribution (think threading++), not for using services.

...

If you are interested in working on alternative proposals, then one possibility could be to add a client-side for the Boost.Http project:

https://github.com/BoostGSoC14/boost.http

Sounds like an interesting proposal, but quite different. Also, it looks like a client library for that would be more like a separate HTTP client project, not as a companion project. How's the boost acceptance status of this project? We already have Boost.Serialization and Boost.Fusion that are capable of

...

serialization.

True, and that would probably be my next guess for integrating it into boost - switch out some simplified local hacks for battle-tested Boost code. For the current level of "it's a bit of a hack" that doesn't necessarily make sense.

Bjorn Reese

11 Dec 11 Dec

12:02 p.m.

On 12/09/2014 03:40 PM, Peter Bindels wrote:

...

Who's "We" ? Given the powerful abilities that you can get with async and

"We" are anybody who follows the general developments in cloud computing and Internet-of-Things, and anybody who has witnessed the stagnation of RPC over the past decade. I am not claiming the demise of RPC, but rather pointing out that there are more fertile ground out there. RPC is inherently restricted to the request-reply interaction pattern, but there are many more interaction patterns that are useful; for instance, the publish-subscribe pattern is widely used in sensor networks. You can quite easily build the various interaction patterns on top of messaging; trying to build them on top of RPC is going to be quite a challenge.

...

future/promise, coupled with easy remoting with some rpc in between that gives the same interface, would lead me to believe RPC is due for a

Regarding the API, I suggest that you investigate asio::async_result rather than focusing exclusively on promise/future.

...

Sounds like an interesting proposal, but quite different. Also, it looks like a client library for that would be more like a separate HTTP client project, not as a companion project. How's the boost acceptance status of this project?

It was a GSoC project this summer. It has not been submitted for review yet.

Oleg Labutin

12 Dec 12 Dec

1:26 a.m.

...

RPC is inherently restricted to the request-reply interaction pattern, but there are many more interaction patterns that are useful; for instance, the publish-subscribe pattern is widely used in sensor networks. You can quite easily build the various interaction patterns on top of messaging; trying to build them on top of RPC is going to be quite a challenge.

Strange but situation is that RPC and questions above is a different things, we can talk about cloud computance and use RPC as technical approach why not? RPC is not restricted by points about which you told, we can use subscribing, object channel and moveable instance , and it can be RPC. It can be ORPC, it can be distributed shared network instance, but we can talk about it in context rpc - it's technical approaches to solve. Currently we talk about RPC (ORPC) by compile time reflection. Why compile time? It can make computation during compiling and resolve some questions on a C++ side and communicate through different's technologies, but on a C++ side it'll have one view. We talk about transformation src to any protocols which used on a remote side.

Oleg Labutin

19 Dec 19 Dec

10:49 p.m.

I'am bored ..Why didnn't your understood tha c++ is looser because haven't reflection &&& Stupid MFC and e.g. ))) that's all 2014-12-12 3:26 GMT+02:00 Oleg Labutin :

...

...
RPC is inherently restricted to the request-reply interaction pattern, but there are many more interaction patterns that are useful; for instance, the publish-subscribe pattern is widely used in sensor networks. You can quite easily build the various interaction patterns on top of messaging; trying to build them on top of RPC is going to be quite a challenge.

Strange but situation is that RPC and questions above is a different things, we can talk about cloud computance and use RPC as technical approach why not? RPC is not restricted by points about which you told, we can use subscribing, object channel and moveable instance , and it can be RPC. It can be ORPC, it can be distributed shared network instance, but we can talk about it in context rpc - it's technical approaches to solve.

Currently we talk about RPC (ORPC) by compile time reflection. Why compile time? It can make computation during compiling and resolve some questions on a C++ side and communicate through different's technologies, but on a C++ side it'll have one view. We talk about transformation src to any protocols which used on a remote side.

Hartmut Kaiser

9 Dec 9 Dec

1:47 p.m.

...

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as the arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about the argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the link code), but I don't know whether that's true.

On the second note, I also proposed a Mock library a few years ago. This basically fell flat on the code that I suggested being very unsafe and using many hackish methods to mock functions. Using the compile-time reflection proposed for the second bullet I would be able to replace the entire hackishness from that with 100% compliant code, with only one minor regression in functionality. That should make it Boost-OK at least from a language standards perspective.

Any ideas? Reactions on RPC or Mocking?

In our experience, there is no need for having yet another special reflection solution if you use Boost.Serialization (or similar). However, those serialization libraries use some kind of reflection internally. Since you mentioned HPX, here is how we implemented the generic RPC solution we have. In essence, in order to marshal a function to another machine (which can't be done directly, we let the user create a special unique type representing the function. This type is then serialized (together with the function arguments). This allows to invoke the function after de-serialization on the receiving end. Here is a sketch of what's going on: template struct action; template struct action { tuple arguments; // construction, serialization, etc. // this is called on the receiving end R invoke() const { return invoke_fused(f, arguments); } }; int foo(double) { return 42; } typedef action foo_action; Now, 'foo_action' can be used to invoke 'foo' on a destination machine: future<int> f = async(foo_action(), target_address, 42.0); where async() is a special overload implementing the actual serialization of foo_action and the arguments, and 'target_address' represents the node where foo() should be invoked on. Note that the return value is sent back using a similar mechanism targeting the promise instance which originally generated the future<int> we got back from our async. The benefit of this is that you can wrap any existing function without changing it by just adding the typedef defining the needed action. In HPX this is done using a macro: HPX_ACTION(foo); // defines 'foo_action' HTH Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

Thomas Heller

5:18 p.m.

Hey there, On Monday, December 08, 2014 15:27:33 Peter Bindels wrote:

...

Hi all,

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>.

(This is one of the HPX developers speaking) Please bear with me, I didn't attend meetingcpp and don't know which questions have been answered already or not. I just would like to add my two cents here. The biggest question I currently have here is: How do you distinguish if it is remote or local? The HPX answer to that problem is a distributed database which is responsible for mapping Global Identifiers (GIDs) to actual objects. We call that the Active Global Address Service (AGAS). The GID therefor acts as your proxy which is, in HPX, kinda like a void pointer to a possibly remote object where you can invoke so called actions on. Actions our mechanisms to do RPC, more on that below... By meaning "return a future<T>", do you refer to std::future<T> or your own class behaving like a future<T>?

...

It does what the discussion mentions - serialize the name of the interface and function, as well as the arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

You say "basically the mangled name", what does that mean exactly? How do you get unique names?

...

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize.

As has been noted in a different mail in that thread, boost already has a solution for that: Boost.Serialization. We currently use it but were not very happy with its performance (and still aren't), but that's a different issue ... I think the interface that Boost.Serialization exports is great to implement compile time reflection without compiler support. It has its caveats, but it is a decent, and accepted way of accomplishing this task.

...

- Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about the argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the link code), but I don't know whether that's true.

Mechanisms for that are already there: Boost.Serialization. Nevertheless I'd be very interested to see the code of your prototype. Here is more on actions: In order to turn a regular function into a "RPC"able action, you essentially need a way for the communicating partners which function should get executed on the other hand. The simplest way to do that is by it's function pointer and the arguments that are supposed to be passed to that function. The function pointer, and it's type is the hard part here. We currently solve this by having a template (instantiated on all communicating partners) which is serialized over a polymorphic base pointer. As such you only need to register this function as a action with your serialization library and call it a day. Side note: We haven't found a truly portable way to automate that, in addition, with this techniques, we are not able to send lambdas over the wire, and the committee had come to the conclusion that it isn't possible with the current standard, IIUC. <snip>

...

Any ideas? Reactions on RPC or Mocking?

I like the idea of having a generic RPC library lying around somewhere. The requirements, i can think of right now, for us in order to use it would be: 1) Extension mechanism for the actual network communication 2) Extension mechanism for the actual execution on the remote end 3) Extension mechanism for the future which gets returned 4) High performance of the serialization with zero copy support (preferably API compatible with Boost.Serialization) With that being said, HPX is a little more than just a library for doing RPC, there is a whole bunch of stuff we implemented to make it convenient and fast to use (and we still aren't really there yet ...). It would be great though if we could out source some of our infrastructure to 3rd parties to ease our maintenance burden ;)

...

Best regards, Peter Bindels

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Hartmut Kaiser

10 Dec 10 Dec

8:36 p.m.

...

I've searched through the Boost mailing list history to find related emails and the last I found was from January where Oleg Labutin and Hartmut Kaiser discussed compile-time reflection in the context of RPC mechanisms.

Based on Hartmut's speech last Saturday introducing HPX and explaining a bit about async/future/promise I tried to implement a simple RPC mechanism that uses then to decouple work and the result I have so far is very promising. I have a simple interface with a call that returns a future, and the caller does not distinguish between calling the actual object and calling on a proxy, as both return a future<T>. It does what the discussion mentions - serialize the name of the interface and function, as well as the arguments (basically the mangled name of the interface function) to ensure there are no mismatches.

I would like to take this oppurtunity to propose two things:

- I can work on this idea to work it out into a full RPC mechanism, to be proposed for Boost.RPC, if there is enough interest. Currently it uses macros to generate and wrap functions, and a simple serializer that serializes in the same way that Protobuf and Thrift also serialize. - Given a compile-time reflection method, the entire proxy class and dispatch class can be auto-generated from the interface class - the full code for registering an interface with the RPC mechanism would collapse to RPC::Register<T>(); To do so, it would need two language additions. It needs access to a compile-time iterator over functions and traits about the argument types, return type, name and whether it is a virtual function. For the prototype this can be manually generated for an interface to show it works. Additionally, it would need to be able to use such a definition to instantiate a virtual function from a base class - basically, hooking up an std::function<X> to a virtual function X directly. I think this is well-implementable (probably reusing most code from lambdas for the link code), but I don't know whether that's true.

On the second note, I also proposed a Mock library a few years ago. This basically fell flat on the code that I suggested being very unsafe and using many hackish methods to mock functions. Using the compile-time reflection proposed for the second bullet I would be able to replace the entire hackishness from that with 100% compliant code, with only one minor regression in functionality. That should make it Boost-OK at least from a language standards perspective.

Any ideas? Reactions on RPC or Mocking?

3630

Age (days ago)

3641

Last active (days ago)

List overview

Download

15 comments

8 participants

participants (8)

Bjorn Reese
Damien Buhl
Edward Diener
Hartmut Kaiser
Oleg Labutin
Peter Bindels
Rob Stewart
Thomas Heller