On May 18, 2013, at 11:54 PM, "Vicente J. Botet Escriba" wrote:

...

Le 17/05/13 23:26, Charousset, Dominik a écrit :

...

Hi,

is there any interest in a library for actor programming?

The library we've developed is a C++11 open source library named "libcppa". It is currently released under LGPL2, but re-releasing the library under the boost license is not an issue. You'll find all important links and ressources either under http://libcppa.org or on the GitHub project page: https://github.com/Neverlord/libcppa, including a user manual as HTML or PDF. We've also submitted a paper to the C++Now conference.

What is actor programming? For those of you who aren't yet familiar with the actor model: actors basically represent tasks or services that communicate via message passing. In libcppa, actors are very lightweight (you can spawn literally millions of them) and scheduled cooperatively using a thread pool (although actors are allowed to opt-out of the cooperative scheduling). Some of the main strengths of this programming paradigm are: (1) network transparency, (2) race conditions are avoided by design, (3) a strong failure model that enables developers to build reliable distributed systems, and (4) the actor model addresses both concurrency (multiple actors on one host) and distribution (any number of actors on any number of hosts).

A message is simply a tuple of values that is pattern matched at the receiver. For example, " on_arg_match >> [](const string& what) {...}" defines a message handler that is called whenever a tuple with one single string element arrives. "on_arg_match" simply defines a pattern that matches the signature of the given lambda.

Please have a look at the examples folder on GitHub (https://github.com/Neverlord/libcppa/tree/master/examples) or at the user manual (http://neverlord.github.io/libcppa/manual/) if you want to see more examples.

Hi,

This is really a lot of good and interesting work on your library. Thanks.

...

Moving all the library to the Boost standards would be quite a lot of work however. Replacing some classes by the corresponding Boost ones.

I would see already several Boost libraries in your libcppa library: * Atoms * CowTuples * Dynamically Typed Tuples - type erased tuples * Tuples Pattern matching * Actors I agree. At least separating pattern matching (for dynamically typed tuples) from the actual actor implementation would make sense.

...

There is something that I would like to see on the Actors architecture. The interface of an actor is not really defined other than by their behavior. It can receive any dynamic tuple. This is a lot. I would like a more statically typed interface. An actor could define statically the kind of messages/signals it is allowed to handle and make it part of the interface. There was some discussion about this topic after the talk at the C++Now conference. I do agree that it is desirable to enable the compiler to verify the correctness of actor programs at compile time. However, in order to be able to ensure the correctness of a program, one would have to define not only all possible input messages, but the response types depending on the input as well. Consider this example:

actor_type::with<int>, reacts_to::withstd::string> my_actor() { return ( on(atom("add"), arg_match) >> [](int a, int b) { return a+b; }, on(atom("hello")) >> [] { return "world"; } ); }

It's easy to check whether the interface is fulfilled. On each send, the compiler is able to check whether the given message matches actor_type and if the sender of the message (more precisely, the receiver of the response message) can handle the result. However, an approach like this cannot allow an actor to change its behavior. Hence, something as simple as the classical dining philosophers example (https://github.com/Neverlord/libcppa/blob/master/examples/message_passing/di...) is not possible. What do you mean by change its behavior? So, what if we allow actors to set a subset of given actor_type as behavior? Consider this example:

struct my_actor : actor_type, reacts_to::with<int>, reacts_to::withstd::string> { void init() { // check whether given match_expr defines a subset of defined actor_type become ( on(atom("init")) >> [=] { become ( on(atom("add"), arg_match) >> [=](int a, int b) { return a+b; }, on(atom("hello")) >> [=] { return "world"; } ); } ); } }

This approach only allows class-based actors, because we need to evaluate the type information provided by 'this'. We would enable the compiler to check for *some* errors, but we could not verify, at compile time, that an actor actually implements its own interface, because we have no way the check whether an actor implements a message handler for each message it claims to be able to receive. Good point. I think that your approach is to let the message on the mail box until a match is requested. On the systems I use to work an asynchronous message is simply destroyed if not handled explicitly, so in my case the message is always implemented implicitly. Another history is for the synchronous communication (reply?). I would say that this is similar to callback interface.

Even worse, both approaches do not allow actors to send/receive messages before replying to a request (unless message passing would be blocking, which would put an end to scalability). If we go back to the reply() API currently found in libcppa, we couldn't match request and response type. You lost me.

In any case, we would no longer be able to converts threads to actors on-the-fly: Humm, this would be unfortunate. int main() { auto worker = spawn(...); // Um... what is the type of 'self'? Is it a valid receiver for the response message? send(worker, ...); receive ( // again: what is 'self' allowed to receive/reply? ... ); } Couldn't spawn have a template parameter giving the provided protocol? Self could be typed with the supported protocol(s). Self could be passed by the underlying system as a parameter of the spawned function, as e.g. Boost.Coroutine does. This avoids the use of thread specific data to get

Le 20/05/13 01:48, Dominik Charousset a écrit : the current self as I expect you do. Up to the user to manage with the self parameter.

Long story short: a type-safe interface would be less powerful and would require more boilerplate code. However, perhaps there is a middle ground for doing type checks at runtime when compiled in debug mode?

int my_actor() { // macro, defined as nothing if not compiled in debug mode assert_protocol(reacts_to::with<int>, reacts_to::withstd::string); }

I don't think I would use this kind of assertions.

Perhaps it would be possible to include the first presented approach for type-safe actors along with the (dynamic) default actor API. In this way, we could compose type-safe subsystems of actors whenever a problem can be solved using the limited API. Yeah, providing both approaches would be great.

...

While I find your tuple pattern matching quite interesting I'm looking for a more low level interface for the Actors part, that doesn't force to use tuples. It provides just a way to identify uniquely the message type and the associated data.

on<messageType> >> [](associatedDateType const& data) { /*...*/},

Of course the user could already use {atom("messageName"), associatedDataType()}, but this forces the user to use a cow data type that could be too costly for some applications. As most the communications are point to point, the ownership of the message is quite clear in these cases, so no need for COW. I guess this is more of an optimization issue. We have to wrap the values into a mailbox_element anyways before putting the message to an actor's mailbox (http://libcppa.blogspot.de/2011/04/mailbox-part-1.html). This element might as well *be* the tuple in case of 1:1 communication. When forwarding the message, we then could simply transfer ownership of the mailbox element. Only when accessing self->last_dequeued(), we would have to make sure to provide COW semantics. When there are a lot of exchanged messages optimizing the communication seems important to me. I would prefer move semantics in this particular case. COW doesn't provides the efficiency I'm locking for. This doesn't mean that your approach is not coherent, but at a higher level.

...

Last but not least your library works only on some C++11 compilers. I have nothing against having C++11 specific libraries in Boost, even the opposite (I proposed it long time ago) but the fact is that currently, the Boost libraries provide at least a minimal interface on C++98 compilers. I don't know what the Boost community thinks about this point now. Welcome, keep up the good work and good luck with your library, Vicente Thank you. At the "Future of Boost" session at the C++Now conference, most developers agreed that it makes sense to embrace C++11 and to use the momentum of the new standard. I hope this is a broad consensus among all Boost developers.

Great. Glad to hear we are ready to move. Best, Vicente

On May 20, 2013, at 3:31 AM, Vicente J. Botet Escriba wrote:

Le 20/05/13 01:48, Dominik Charousset a écrit :

...

On May 18, 2013, at 11:54 PM, "Vicente J. Botet Escriba" wrote:

...

Le 17/05/13 23:26, Charousset, Dominik a écrit :

...

Hi,

is there any interest in a library for actor programming?

The library we've developed is a C++11 open source library named "libcppa". It is currently released under LGPL2, but re-releasing the library under the boost license is not an issue. You'll find all important links and ressources either under http://libcppa.org or on the GitHub project page: https://github.com/Neverlord/libcppa, including a user manual as HTML or PDF. We've also submitted a paper to the C++Now conference.

What is actor programming? For those of you who aren't yet familiar with the actor model: actors basically represent tasks or services that communicate via message passing. In libcppa, actors are very lightweight (you can spawn literally millions of them) and scheduled cooperatively using a thread pool (although actors are allowed to opt-out of the cooperative scheduling). Some of the main strengths of this programming paradigm are: (1) network transparency, (2) race conditions are avoided by design, (3) a strong failure model that enables developers to build reliable distributed systems, and (4) the actor model addresses both concurrency (multiple actors on one host) and distribution (any number of actors on any number of hosts).

A message is simply a tuple of values that is pattern matched at the receiver. For example, " on_arg_match >> [](const string& what) {...}" defines a message handler that is called whenever a tuple with one single string element arrives. "on_arg_match" simply defines a pattern that matches the signature of the given lambda.

Please have a look at the examples folder on GitHub (https://github.com/Neverlord/libcppa/tree/master/examples) or at the user manual (http://neverlord.github.io/libcppa/manual/) if you want to see more examples.

Hi,

This is really a lot of good and interesting work on your library. Thanks.

...

Moving all the library to the Boost standards would be quite a lot of work however. Replacing some classes by the corresponding Boost ones.

I would see already several Boost libraries in your libcppa library: * Atoms * CowTuples * Dynamically Typed Tuples - type erased tuples * Tuples Pattern matching * Actors I agree. At least separating pattern matching (for dynamically typed tuples) from the actual actor implementation would make sense.

...

There is something that I would like to see on the Actors architecture. The interface of an actor is not really defined other than by their behavior. It can receive any dynamic tuple. This is a lot. I would like a more statically typed interface. An actor could define statically the kind of messages/signals it is allowed to handle and make it part of the interface. There was some discussion about this topic after the talk at the C++Now conference. I do agree that it is desirable to enable the compiler to verify the correctness of actor programs at compile time. However, in order to be able to ensure the correctness of a program, one would have to define not only all possible input messages, but the response types depending on the input as well. Consider this example:

actor_type::with<int>, reacts_to::withstd::string> my_actor() { return ( on(atom("add"), arg_match) >> [](int a, int b) { return a+b; }, on(atom("hello")) >> [] { return "world"; } ); }

It's easy to check whether the interface is fulfilled. On each send, the compiler is able to check whether the given message matches actor_type and if the sender of the message (more precisely, the receiver of the response message) can handle the result. However, an approach like this cannot allow an actor to change its behavior. Hence, something as simple as the classical dining philosophers example (https://github.com/Neverlord/libcppa/blob/master/examples/message_passing/di...) is not possible. What do you mean by change its behavior?

To change the behavior of an actor means to replace the current message handler by another one, i.e., call become().

...

So, what if we allow actors to set a subset of given actor_type as behavior? Consider this example:

struct my_actor : actor_type, reacts_to::with<int>, reacts_to::withstd::string> { void init() { // check whether given match_expr defines a subset of defined actor_type become ( on(atom("init")) >> [=] { become ( on(atom("add"), arg_match) >> [=](int a, int b) { return a+b; }, on(atom("hello")) >> [=] { return "world"; } ); } ); } }

This approach only allows class-based actors, because we need to evaluate the type information provided by 'this'. We would enable the compiler to check for *some* errors, but we could not verify, at compile time, that an actor actually implements its own interface, because we have no way the check whether an actor implements a message handler for each message it claims to be able to receive. Good point. I think that your approach is to let the message on the mail box until a match is requested. On the systems I use to work an asynchronous message is simply destroyed if not handled explicitly, so in my case the message is always implemented implicitly. Another history is for the synchronous communication (reply?). I would say that this is similar to callback interface.

Yes, each message remains in the mailbox until the actor handled it. However, it's easy to define a catch-all rule to discard unexpected messages: void my_actor() { become( on(atom("foo")) >> [] { reply(atom("bar")); }, others() >> [] { cerr << "unexpected message: " << to_string(self->last_dequeued()) << endl; ); }

...

Even worse, both approaches do not allow actors to send/receive messages before replying to a request (unless message passing would be blocking, which would put an end to scalability). If we go back to the reply() API currently found in libcppa, we couldn't match request and response type.

You lost me.

The compiler needs to know what response type belongs to what request type. The only way to do this in C++ is to return the response from the request handler, so it becomes part of the signature. Using reply() wouldn't work, because we cannot determine from what handler it was called.

...

In any case, we would no longer be able to converts threads to actors on-the-fly:

...

int main() { auto worker = spawn(...); // Um... what is the type of 'self'? Is it a valid receiver for the response message? send(worker, ...); receive ( // again: what is 'self' allowed to receive/reply? ... ); } Couldn't spawn have a template parameter giving the provided protocol? Self could be typed with the supported protocol(s). Self could be passed by the underlying system as a parameter of the spawned function, as e.g. Boost.Coroutine does. This avoids the use of thread specific data to get

Humm, this would be unfortunate. the current self as I expect you do. Up to the user to manage with the self parameter.

The 'self' pointer has to use a thread-local variable, because (1) scheduled actors migrate between threads (see Oliver Kowalke's response) and (2) threads are converted implicitly as soon as they use actor-related messages such as 'send'.

...

Long story short: a type-safe interface would be less powerful and would require more boilerplate code. However, perhaps there is a middle ground for doing type checks at runtime when compiled in debug mode?

int my_actor() { // macro, defined as nothing if not compiled in debug mode assert_protocol(reacts_to::with<int>, reacts_to::withstd::string); } I don't think I would use this kind of assertions.

Well, it could make it easier to debug an actor-based application. Another possibility is to perform static analysis (maybe using a Clang plugin).

...

Perhaps it would be possible to include the first presented approach for type-safe actors along with the (dynamic) default actor API. In this way, we could compose type-safe subsystems of actors whenever a problem can be solved using the limited API. Yeah, providing both approaches would be great.

Agreed. It should be straightforward to implement, too.

...

...

While I find your tuple pattern matching quite interesting I'm looking for a more low level interface for the Actors part, that doesn't force to use tuples. It provides just a way to identify uniquely the message type and the associated data.

on<messageType> >> [](associatedDateType const& data) { /*...*/},

Of course the user could already use {atom("messageName"), associatedDataType()}, but this forces the user to use a cow data type that could be too costly for some applications. As most the communications are point to point, the ownership of the message is quite clear in these cases, so no need for COW. I guess this is more of an optimization issue. We have to wrap the values into a mailbox_element anyways before putting the message to an actor's mailbox (http://libcppa.blogspot.de/2011/04/mailbox-part-1.html). This element might as well *be* the tuple in case of 1:1 communication. When forwarding the message, we then could simply transfer ownership of the mailbox element. Only when accessing self->last_dequeued(), we would have to make sure to provide COW semantics. When there are a lot of exchanged messages optimizing the communication seems important to me. I would prefer move semantics in this particular case. COW doesn't provides the efficiency I'm locking for. This doesn't mean that your approach is not coherent, but at a higher level.

libcppa already uses move whenever possible. As long as you send a message to one actor (using move on your arguments), you won't make copy of any value, unless you do something silly like 'send_tuple(buddy, self->last_dequeued())'.

...

...

Last but not least your library works only on some C++11 compilers. I have nothing against having C++11 specific libraries in Boost, even the opposite (I proposed it long time ago) but the fact is that currently, the Boost libraries provide at least a minimal interface on C++98 compilers. I don't know what the Boost community thinks about this point now. Welcome, keep up the good work and good luck with your library, Vicente Thank you. At the "Future of Boost" session at the C++Now conference, most developers agreed that it makes sense to embrace C++11 and to use the momentum of the new standard. I hope this is a broad consensus among all Boost developers.

Great. Glad to hear we are ready to move.

Best, Vicente

There was some discussion about this topic after the talk at the C++Now conference. I do agree that it is desirable to enable the compiler to verify the correctness of actor programs at compile time. However, in order to be able to ensure the correctness of a program, one would have to define not only all possible input messages, but the response types depending on the input as well. Consider this example:

actor_type::with<int>, reacts_to::withstd::string> my_actor() { return ( on(atom("add"), arg_match) >> [](int a, int b) { return a+b; }, on(atom("hello")) >> [] { return "world"; } ); }

It's easy to check whether the interface is fulfilled. On each send, the compiler is able to check whether the given message matches actor_type and if the sender of the message (more precisely, the receiver of the response message) can handle the result. However, an approach like this cannot allow an actor to change its behavior. Hence, something as simple as the classical dining philosophers example (https://github.com/Neverlord/libcppa/blob/master/examples/message_passing /dining_philosophers.cpp) is not possible.

So, what if we allow actors to set a subset of given actor_type as behavior? Consider this example:

struct my_actor : actor_type, reacts_to::with<int>, reacts_to::withstd::string> { void init() { // check whether given match_expr defines a subset of defined actor_type become ( on(atom("init")) >> [=] { become ( on(atom("add"), arg_match) >> [=](int a, int b) { return a+b; }, on(atom("hello")) >> [=] { return "world"; } ); } ); } }

This approach only allows class-based actors, because we need to evaluate the type information provided by 'this'. We would enable the compiler to check for *some* errors, but we could not verify, at compile time, that an actor actually implements its own interface, because we have no way the check whether an actor implements a message handler for each message it claims to be able to receive.

Even worse, both approaches do not allow actors to send/receive messages before replying to a request (unless message passing would be blocking, which would put an end to scalability). If we go back to the reply() API currently found in libcppa, we couldn't match request and response type.

In any case, we would no longer be able to converts threads to actors on- the-fly:

int main() { auto worker = spawn(...); // Um... what is the type of 'self'? Is it a valid receiver for the response message? send(worker, ...); receive ( // again: what is 'self' allowed to receive/reply? ... ); }

Long story short: a type-safe interface would be less powerful and would require more boilerplate code. However, perhaps there is a middle ground for doing type checks at runtime when compiled in debug mode?

int my_actor() { // macro, defined as nothing if not compiled in debug mode assert_protocol(reacts_to::with<int>, reacts_to::withstd::string); }

Perhaps it would be possible to include the first presented approach for type-safe actors along with the (dynamic) default actor API. In this way, we could compose type-safe subsystems of actors whenever a problem can be solved using the limited API.

Even if I have to admit that I don't understand all implication of what you're saying, I would like to assert that it is possible to expose a fully compile-time type-safe way of invoking remote functions (actors). Here is what we do in HPX (https://github.com/STEllAR-GROUP/hpx/): int foo(std::string s) { return boost::lexical_cast<int>(s); } typedef hpx::make_action::type foo_action_type; foo_action_type foo_action; // 'synchronous' invocation cout << foo_action(remote_locality_id, "42"); // prints 42 // asynchronous invocation hpx::future<int> f = hpx::async(foo_action, remote_locality_id, "42"); // do other stuff cout << f.get(); // prints 42 // fails compiling cout << foo_action(remote_locality_id, 42); The same (similar) works for invoking member functions of remote objects. Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu

On Sun, May 19, 2013 at 5:48 PM, Dominik Charousset < dominik.charousset@haw-hamburg.de> wrote:

On May 18, 2013, at 11:54 PM, "Vicente J. Botet Escriba" < vicente.botet@wanadoo.fr> wrote:

...

Le 17/05/13 23:26, Charousset, Dominik a écrit :

...

Hi,

is there any interest in a library for actor programming?

The library we've developed is a C++11 open source library named "libcppa".

...

There is something that I would like to see on the Actors architecture. The interface of an actor is not really defined other than by their behavior. It can receive any dynamic tuple. This is a lot. I would like a more statically typed interface. An actor could define statically the kind of messages/signals it is allowed to handle and make it part of the interface.

There was some discussion about this topic after the talk at the C++Now conference.

To continue that discussion...

I do agree that it is desirable to enable the compiler to verify the correctness of actor programs at compile time. However, in order to be able to ensure the correctness of a program, one would have to define not only all possible input messages, but the response types depending on the input as well.

Yes, and when writing code with strong compiler guarantees, this is a desirable feature.

However, an approach like this cannot allow an actor to change its behavior.

I think there is a large subset of use-cases where there is a willingness to disallow an actor to "change its behavior" (aka, change its type), in order to gain the benefit of compile-time guarantees.

Hence, something as simple as the classical dining philosophers example ( https://github.com/Neverlord/libcppa/blob/master/examples/message_passing/di...) is not possible.

I don't think it's been demonstrated that the 'typed' and 'untyped' approaches are mutually exclusive. For a Boost.Actor library, I would really like to see static types being well supported. That would likely be the only type of Actor library I would personally use in practice. Strong compiler guarantees enable safe refactoring of large applications, which is where C++ shines. I wouldn't object to there being a trap door to type-unsafety to fit some need, though its use should rightfully be discouraged. Long story short: a type-safe interface would be less powerful and would

require more boilerplate code.

The same argument can be used for any dynamic vs. static type argument. The user base of this library has already taken a stance on the issue by their choice of C++. That may be why there is so much push back to the current dynamic approach of libcppa.

However, perhaps there is a middle ground for doing type checks at runtime when compiled in debug mode?

int my_actor() { // macro, defined as nothing if not compiled in debug mode assert_protocol(reacts_to::with<int>, reacts_to::withstd::string); }

This mimics the python approach to check types at runtime. The whole benefit of compile time safety is that we get guarantees about code correctness *before* we run. -- David Sankel

On 10/06/2013 13:59, Dominik Charousset wrote:

On 08.06.2013, at 20:11, David Sankel wrote:

...

The same argument can be used for any dynamic vs. static type argument. The user base of this library has already taken a stance on the issue by their choice of C++. That may be why there is so much push back to the current dynamic approach of libcppa. I think providing a strongly typed subset of the library along with the "fully dynamic" version should be a good middle ground. Type safe actors will be more or less lifted functions, where we can derive all the type information we need from the signature. If you are going to be strongly typed then I would suggest using some kind of message and actor interface compiler and generating code. Using something that is strongly typed with C++ at both ends of the wire is going to be fragile in terms of requiring 'big bang' deployments and also extremely constraining through an inability to communicate with other language runtimes.

This mimics the python approach to check types at runtime. The whole benefit of compile time safety is that we get guarantees about code correctness*before* we run.

(to David - sorry!) I don't think that the limited correctness you get from that is very useful in a concurrent system. It may ensure that teh form of messages is compatible, but not what they mean, nor any ordering guarantees, and the flip-side is that you have the schema migration issues. IMO its why object databases never went anywhere, and the lingua-franca of SQL (and blob databases with JSON-like data) have been supreme. James

On 11/06/2013 13:03, Brandon Kohn wrote:

On 6/10/2013 4:30 PM, james wrote:

...

I don't think that the limited correctness you get from that is very useful in a concurrent system.

Why not? I think the development crews that worked on the mars orbiter mission which failed would beg to differ.

Regards,

Brandon

Personal experience? Its a question of whether you are building and releasing all components at once, and whether they will all be built in the same team, and with the same technology. I have found systems with soft data structures to be more easily enhanced than ones built from IDL, and have not had many problems from mismatched syntaxes. Rigid syntax (whether from IDL, or some XML form check) always seem to end up subverted with additional key/value data or (even worse) data embedded into 'structured comments'. There's a reason for that and it all goes back to schema evolution, and that's why systems like protobuf tend to sacrifice clean syntax for extensibility.. The critical size for a homogeneous system that must be built and released from the same sources to guarantee wire compatibility seems to be quite small in practice. James

on Tue Jun 11 2013, David Sankel wrote:

As an aside, the whole concept of 'schema evolution' has really piqued my curiosity. Is there some academic key-word I can search with to learn more about data structure evolution in systems that need to maintain some kind of backwards compatibility?

You find this term all over discussions about applications—especially web apps—that use persistent databases, but it also applies to anything that saves and reads files whose format might need to change over time, i.e. standard desktop apps. -- Dave Abrahams

On 06.07.2013, at 15:36, Dave Abrahams wrote:

on Tue Jun 11 2013, David Sankel wrote:

...

On Tue, Jun 11, 2013 at 3:31 PM, james wrote:

...

On 11/06/2013 13:03, Brandon Kohn wrote:

...

On 6/10/2013 4:30 PM, james wrote:

...

I don't think that the limited correctness you get from that is very useful in a concurrent system.

Why not? I think the development crews that worked on the mars orbiter mission which failed would beg to differ.

[I]t all goes back to schema evolution, and that's why systems like protobuf tend to sacrifice clean syntax for extensibility..

I think that goes back to my original point. For a large subset of users, schema evolution is a non-issue—the entire system is deployed at once. A Boost.Actor library should have a good story for these users IMO.

+1

-- Dave Abrahams

So... It's been a while and did indeed cause more implementation work than I had expected, but typed actors are finally ready to fly. A short example: ------------------------------------------------ // no class => actor from match expression auto p0 = spawn_typed( on_arg_match >> [](int a, int b) { return static_cast<double>(a) * b; }, on_arg_match >> [](double a, double b) { return make_cow_tuple(a * b, a / b); } ); // equal class-based implementation class foo : public typed_actor::with<double>,replies_to::with> { protected: // override inherited, pure virtual member function behavior_type make_behavior() final { return ( on_arg_match >> [](int a, int b) { return static_cast<double>(a) * b; }, on_arg_match >> [](double a, double b) { return make_cow_tuple(a * b, a / b); }; ); } }; ... auto p0 = spawn_typed<foo>(); ------------------------------------------------ The newly spawned actor addressed by "p0" has the type typed_actor_ptr::with<double>,replies_to::with>. A value returned from a message handler is implicitly used as response message. This now true for all types of actors (and the old 'reply' function is hence deprecated). Typed actors are not allowed to use the "become()" API, because this would change its type. And without proper compiler support, we cannot traverse the AST to look "inside" the message handlers and parse each call to "become()" to extract all possible message types. Furthermore, typed actors are not allowed to use guards. Either it handles any input of a type or none. The "replies_to<..>::with<..>" syntax may be a bit verbose, but I think a self-explanatory type is worth some typing. This type is of course assignable to any of its subtypes: ------------------------------------------------ typed_actor_ptr::with<double>> p1 = p0; // ok typed_actor_ptr::with> p1 = p0; // ok typed_actor_ptr p3 = p0; // compiler error ------------------------------------------------ When using a typed interface, the compiler is now able to type-check messaging: ------------------------------------------------ send(p0, 42); // <- compiler error send(p0, 42, 24); // <- ok sync_send(p0, 1, 2, 3).then ... // <- compiler error sync_send(p0, 1, 2).then( [](float) { ... } // <- compiler error: expected double ); sync_send(p0, 1, 2).then( [](double d) { ... } // <- ok ); ------------------------------------------------ By the way, another big change in the library is that match expressions now handle result types properly: ------------------------------------------------ auto res = match(something) ( on("hello") >> [] { return 42; }, on("goodbye") >> [] { return "why are you leaving?"; } ); if (res.is<int>()) { ... } else { ... } ------------------------------------------------ That's also a plus when releasing the pattern matching as standalone boost library. Of course, "res" can be inspected using a visitor as well. It is basically a variant type that is either none_t or any result type from one of the message handlers (void is mapped to void_t to be able to distinct between "no match" and "invoked a void function"). There are currently two "missing" features for typed actors: typed remote actors and priorities. However, this is merely ongoing implementation work and is not by design. Since type-safety was the major (the only?) concern regarding the library, I would be happy to hear feedback and suggestions. Since most of the "heavy" implementation work is done, I would focus on integrating the library into boost (i.e. use other boost components wherever possible) and using boost namespaces next.

On 08.05.2014 19:30, Dominik Charousset wrote:

Hi all,

attached you'll find the first preliminary submission for Boost.Actor as ZIP file along with the Manual in PDF version. You currently still need CMake, a Boost.Jam build file will follow. The project is hosted at GitHub: https://github.com/Neverlord/boost.actor

As far as I understand a library with LGPL can not be part of boost (see www.boost.org/development/requirements.html ). So the it should be changed.

To state the obvious: we [1] took our time, but not because we are not taking the boost submission serious. In fact, the opposite is true. We did refine libcppa (which is the base for Boost.Actor) over the last months based on community feedback - feedback from this mailing list, last year's C++Now, GitHub, and libcppa's mailing list. However, we are not just trying to lift our library into Boost. We want Boost.Actor to fit into Boost as good as possible - and we did significant changes to the API (see below), which also took time. Furthermore, we think performance is key when providing a native actor system. Boost.Actor/libcppa has now reached a level of performance with the latest version that lives up to our expectations - and hopefully to the expectations of (future) users as well. A brief performance evaluation [2] is attached. The graphs still mention 'libcppa' instead of Boost.Actor.

I hope the lack of Boost.Jam support is not too much of an issue - I thought it would be a good idea to post the preliminary submission before the C++Now (you can meet me there!), to get people time to play with Boost.Actor beforehand.

*Differences between Boost.Actor and libcppa*

If you don't have any experience with libcppa, you can simply skip this. The most visible change is the removal of any_tuple and cow_tuple. The COW tuple implementation of libcppa is in our opinion not general enough and is too intermeshed with the type system. Instead, Boost.Actor provides the class `message` that is an opaque container from a user's point of view. Also, the pattern matching has been reduced to a minimal DSL for defining message handlers. The separate DSL for guard expressions is gone completely (a similar functionality still exists though). Since our pattern matching only worked with messages/any_tuples, we don't see too much use of it outside of Boost.Actor. We are still hoping for native pattern matching in C++ (N3449 proposed this a while ago) in which case our DSL would become unnecessary anyways. As a side-effect of removing all these parts, Boost.Actor has become a streamlined version of libcppa that is easier to grasp and that will be easier to maintain i

n the long run as well.

*Boost.Actor depends on...*

variant, optional, none, string_algorithms, intrusive_ptr, thread (currently as header-only dependency), context/coroutine (for cooperatively scheduled blocking actors). Some examples also require program_options.

*Boost.Actor does NOT use...*

(1) MPL. Boost.Actor uses a ludicrous amount of metaprogramming based on boost::actor::detail::type_list. I've tried to port it to MPL and wasted almost two days doing so. MPL is a great C++ library. Unfortunately, C++98, not C++11. Even after adding variadic helpers to reduce the amount of template instantiations, it would still explode and fail to compile. I've stopped after writing half the size of the current type_list utility functions and still wasn't able to reduce the amount of template instantiations significantly enough. That being said, I am looking forward to the GSoC project for MPL11.

(2) Serialization. The first and most obvious problem I have with this library is the lack of a portable binary archive. String serialization is sufficient for a lot of applications, but the performance hit for both networking overhead and the string serialization itself is not acceptable in an actor system. Furthermore, simply adding such a binary archive would not be enough. The library does not give you a platform-independent representation of the type names, which is crucial. Also, one cannot create an instance of a type given only its type name and then deserialize its values from a network stream (I was thinking of ETI::key_register using the uniform names and ETI::find, but it only solves a part of the problem). Adding all of these capabilities on top of boost.serialization plus writing a portable binary archive (or maybe use something like EOS - http://epa.codeplex.com/) would probably result in more code and most likely in a more complicated design, i.e., would be harder

to maintain. Maybe I miss the forest for the trees here and some experienced boost.serialization user knows a straightforward way I don't see. However, maybe there is interest in extending boost.serialization. I'd be happy to work on patches for adding uniform names, etc.

(3) Boost.Asio. There is definitely potential to port the low-level parts of the middleman to Asio, I think. These changes would be transparent to the user, though, since users really only use the broker and can forget about the rest.

*Separating parts of Boost.Actor?*

Vicente J. Botet Escriba suggested to separate some parts of library. Namely the constexpr atom() [the other parts he suggested were removed, btw]. Aside from that, there are some C++11 utility functions that I think would be useful for other developers as well. Namely int_list [3] which can be used to create indices for variadic types such as tuples to allow users to apply the values of a tuple to a functor with apply_args [4]. type_list [5] is a powerful tool too, but I think this functionality goes to MPL11 anyways. Something that might be worth lifting to the boost namespace is unit [6], which makes metaprogramming (for me) a lot easier by providing an easy way of lifting void to something useful and don't need to worry whether or not it might end up in a tuple or variant. MPL already has void_t, but it doesn't seem to be indented to be used the way I use unit. For binary serialization of floats, there is IEEE 754 packing/unpacking [6] that is based on http://beej.us/guide/bgn

et/examples/pack2.c What is the difference from your int_list and std::integer_sequence (http://en.cppreference.com/w/cpp/utility/integer_sequence) ?

Sorry for the wall of text, the Mail got longer and longer somehow ...

[1] "We" are a working group at the University of Applied Sciences Hamburg: http://inet.cpt.haw-hamburg.de/

[2] The sources for all benchmark programs can be found here: https://github.com/Neverlord/cppa-benchmarks, please keep in mind that the evaluation uses libcppa, so the API used in the benchmarks is different.

[3] https://github.com/Neverlord/boost.actor/blob/a0cab41e2946df7d751c59dfead0c2...

[4] https://github.com/Neverlord/boost.actor/blob/ea53a515e09a98baa173ad379127c3...

[5] https://github.com/Neverlord/boost.actor/blob/58d6177837788b2e54b8547531bda9...

[6] https://github.com/Neverlord/boost.actor/blob/644a47760d6b55eef438a81d7bcdc2...

[7] https://github.com/Neverlord/boost.actor/blob/ea53a515e09a98baa173ad379127c3...

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Thu, May 8, 2014 at 12:30 PM, Dominik Charousset < dominik.charousset@haw-hamburg.de> wrote:

Hi all,

attached you'll find the first preliminary submission for Boost.Actor as ZIP file along with the Manual in PDF version. You currently still need CMake, a Boost.Jam build file will follow. The project is hosted at GitHub: https://github.com/Neverlord/boost.actor

To state the obvious: we [1] took our time, but not because we are not taking the boost submission serious. In fact, the opposite is true. We did refine libcppa (which is the base for Boost.Actor) over the last months based on community feedback - feedback from this mailing list, last year's C++Now, GitHub, and libcppa's mailing list. However, we are not just trying to lift our library into Boost. We want Boost.Actor to fit into Boost as good as possible - and we did significant changes to the API (see below), which also took time. Furthermore, we think performance is key when providing a native actor system. Boost.Actor/libcppa has now reached a level of performance with the latest version that lives up to our expectations - and hopefully to the expectations of (future) users as well. A brief performance evaluation [2] is attached. The graphs still mention 'libcppa' instead of Boost.Actor.

I hope the lack of Boost.Jam support is not too much of an issue - I thought it would be a good idea to post the preliminary submission before the C++Now (you can meet me there!), to get people time to play with Boost.Actor beforehand.

*Differences between Boost.Actor and libcppa*

If you don't have any experience with libcppa, you can simply skip this. The most visible change is the removal of any_tuple and cow_tuple. The COW tuple implementation of libcppa is in our opinion not general enough and is too intermeshed with the type system. Instead, Boost.Actor provides the class `message` that is an opaque container from a user's point of view. Also, the pattern matching has been reduced to a minimal DSL for defining message handlers. The separate DSL for guard expressions is gone completely (a similar functionality still exists though). Since our pattern matching only worked with messages/any_tuples, we don't see too much use of it outside of Boost.Actor. We are still hoping for native pattern matching in C++ (N3449 proposed this a while ago) in which case our DSL would become unnecessary anyways. As a side-effect of removing all these parts, Boost.Actor has become a streamlined version of libcppa that is easier to grasp and that will be easier to maintain in the long run as well.

*Boost.Actor depends on...*

variant, optional, none, string_algorithms, intrusive_ptr, thread (currently as header-only dependency), context/coroutine (for cooperatively scheduled blocking actors). Some examples also require program_options.

*Boost.Actor does NOT use...*

(1) MPL. Boost.Actor uses a ludicrous amount of metaprogramming based on boost::actor::detail::type_list. I've tried to port it to MPL and wasted almost two days doing so. MPL is a great C++ library. Unfortunately, C++98, not C++11. Even after adding variadic helpers to reduce the amount of template instantiations, it would still explode and fail to compile. I've stopped after writing half the size of the current type_list utility functions and still wasn't able to reduce the amount of template instantiations significantly enough. That being said, I am looking forward to the GSoC project for MPL11.

(2) Serialization. The first and most obvious problem I have with this library is the lack of a portable binary archive. String serialization is sufficient for a lot of applications, but the performance hit for both networking overhead and the string serialization itself is not acceptable in an actor system. Furthermore, simply adding such a binary archive would not be enough. The library does not give you a platform-independent representation of the type names, which is crucial. Also, one cannot create an instance of a type given only its type name and then deserialize its values from a network stream (I was thinking of ETI::key_register using the uniform names and ETI::find, but it only solves a part of the problem). Adding all of these capabilities on top of boost.serialization plus writing a portable binary archive (or maybe use something like EOS - http://epa.codeplex.com/) would probably result in more code and most likely in a more complicated design, i.e., would be harder to maintain. Maybe I miss the forest for the trees here and some experienced boost.serialization user knows a straightforward way I don't see. However, maybe there is interest in extending boost.serialization. I'd be happy to work on patches for adding uniform names, etc.

(3) Boost.Asio. There is definitely potential to port the low-level parts of the middleman to Asio, I think. These changes would be transparent to the user, though, since users really only use the broker and can forget about the rest.

*Separating parts of Boost.Actor?*

Vicente J. Botet Escriba suggested to separate some parts of library. Namely the constexpr atom() [the other parts he suggested were removed, btw]. Aside from that, there are some C++11 utility functions that I think would be useful for other developers as well. Namely int_list [3] which can be used to create indices for variadic types such as tuples to allow users to apply the values of a tuple to a functor with apply_args [4]. type_list [5] is a powerful tool too, but I think this functionality goes to MPL11 anyways. Something that might be worth lifting to the boost namespace is unit [6], which makes metaprogramming (for me) a lot easier by providing an easy way of lifting void to something useful and don't need to worry whether or not it might end up in a tuple or variant. MPL already has void_t, but it doesn't seem to be indented to be used the way I use unit. For binary serialization of floats, there is IEEE 754 packing/unpacking [6] that is based on http://beej.us/guide/bgnet/examples/pack2.c

Sorry for the wall of text, the Mail got longer and longer somehow ...

[1] "We" are a working group at the University of Applied Sciences Hamburg: http://inet.cpt.haw-hamburg.de/

[2] The sources for all benchmark programs can be found here: https://github.com/Neverlord/cppa-benchmarks, please keep in mind that the evaluation uses libcppa, so the API used in the benchmarks is different.

[3] https://github.com/Neverlord/boost.actor/blob/a0cab41e2946df7d751c59dfead0c2...

[4] https://github.com/Neverlord/boost.actor/blob/ea53a515e09a98baa173ad379127c3...

[5] https://github.com/Neverlord/boost.actor/blob/58d6177837788b2e54b8547531bda9...

[6] https://github.com/Neverlord/boost.actor/blob/644a47760d6b55eef438a81d7bcdc2...

[7] https://github.com/Neverlord/boost.actor/blob/ea53a515e09a98baa173ad379127c3...

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

I'm a bit of a lurker on this list but your post got me curious. The actor model looks quite appealing. However a bit of self study led me to http://www.theron-library.com/. Their code examples look more like something I would use. I'd be curious to know how you see the differences. Cheers, THK -- http://www.keittlab.org/

(Disclaimer: I'm a libcppa user for quite a while now.)

attached you'll find the first preliminary submission for Boost.Actor as ZIP file along with the Manual in PDF version.

I would like to express my full support for Boost.Actor. The number one reason why I believe we need the library in Boost, is that the library raises the level abstraction for concurrent and network-transparent message passing in a type-safe manner. It provides a framework allowing users to efficiently harness (i) all available cores on a modern multi-core machine, (ii) general-purpose clusters, and (iii) the compute power of graphics card via OpenCL [1]. My own experiences with libcppa: - Quick round-trips on mailing list and during bug reports - Low CPU overhead (<=5%) - Works robust and stable since 0.9 In the past, I attempted to build a similar abstraction out of 0mq and Boost.Asio, with mildly put, limited success. Therefore I am thrilled to see someone attempting to implement the actor model natively The Right Way. By adopting the library in Boost, users would receive a tool to express their concurrent and distributed programs while focussing on the challenges of their domain, as opposed to grappling with low-level intricacies of data structure synchronization and threading. Personally, I found it immensely productive to not having to worry about data races anymore, as these are eradicated by design. Matthias [1] This is a feature of libcppa and not planned for Boost.Actor, although the functionality should remain in some form.

On Sat, May 17, 2014 at 10:17:25AM +0200, Bjorn Reese wrote:

I have a couple of major concerns with the current submission, and I am going to suggest some substantial changes. I hope that it does not discourage you too much.

I am going to suggest that:

1. The library is broken into more fundamental building-blocks (which is what both Boost and the C++ standard is all about.) 2. A more flexible data flow architecture is adopted. 3. More use of existing Boost libraries.

I recognize three more fundamental building-blocks in the current submission: active objects, messaging middleware, and data flow. I am not against a higher level actor API, but the fundamentals need to be in place first.

Thank you for taking your time for this thorough comment. However, I have to say I disagree on many levels. First of all: C++ is not about having a low level of abstraction. C++ is about having the highest level of abstraction possible without sacrificing performance. What you are suggesting is to not having an actor library in Boost. You want to have a low-level active object library with low-level networking primitives.

Building-blocks ---------------

Boost.Actor implements a distributed mailbox-based actor model. While this is a building-block to some users, it is not fundamental. It conflates the actor with distribution.

An actor *is* the fundamental primitive! We are talking about an implementation of the actor model. The whole point of the actor model is to have software entities that abstract over physical deployment. Actors are *not* the same thing as active objects. There is obviously overlap of these two models, but the actor modle is more general, i.e., describes a higher level of abstraction.

I suggest that you start with a non-distributed actor model. This is simply an active object with an incoming message queue. This can be used its own right without distribution and mailboxes. Many applications have classes with a working thread inside them, and active objects should strive to replace these classes [1].

A non-distributed actor model? I think what you really want to say here is: you want users to be able to use the lightweight actor implementation and the work-stealing scheduler without having link-dependencies to networking infrastructure they don't use. Can you agree on that? That's a fair point and useful indeed. Well, here's the thing: there is nothing baked into the actor primitive that would require such a dependency. The software design is fully modular. Separate the middleman, move publish/remote_actor to a different namespace, ship it separately, done. All you have to do to extend boost.actor is to provide an implementation for actor_proxy. Behind the scenes, you can do all kinds of stuff: networking, OpenCL-binding (that's how it's done in licppa), you name it.

Active objects have two important variation points: scheduling and the queue type. Regarding scheduling there is a C++ standards proposal that should be considered [2]. There is a GSoC project about this [3]. For the distributed case, boost::asio::io_service also has to be considered.

I know the Executor proposal and I think it's too basic to be useful for anything other than implementing std::async. ThreadPool implementations in general do work-sharing, whereas boost.actor implements work-stealing. The latter yields superior performance in almost all use cases. I wouldn't mind moving the scheduler to it's own namespace to allow other projects to build on top of that. The scheduler uses the interfaces resumable and execution_unit, so there aren't any actor-specific types.

There is also work done on message queues. We already have some in Boost.Lockfree, or the wait-free multi-producer queue in the Boost.Atomic examples, as well as sync_queue in Boost.Thread.

Show me a queue that outperforms single_reader_queue and I'll take it. Keep in mind though, that the enqueue operation uses only *a single* compare-and-swap operation [1]. I don't see how you can outperform that. Did you have a look at the performance evaluation? In particular the N:1 communication? This queue scales up to 63 concurrent writers without a measurable perforance hit. I'm not passionate about implementations details, though. Show me a queue that performs even better in boost.actor and I'll take it.

Architecture ------------

Once we have got active objects, the question is how do we connect them? The variation points here are routing and transmission.

Again, this library is not about active objects. It's about actors.

The mailbox approach is too simple for many applications. Partly because it is too limited in some regards (e.g. push-only) and too flexible in other regards (e.g. you cannot have fine-grained access control or restricted visibility.)

There are several flow-based approaches that should be considered: Boost.Iostreams has all the required concepts in place. There was a Boost.Dataflow GSoC project [4] some years ago. There is a C++ standards proposal [5] about C++ pipelines. See also the ZeroMQ guide [6] for various examples.

All of that is true and it's damn good that libcppa/boost.actor it is the way it is. I'm sorry, but again: this is an actor library. If you want to fiddle with low-level networking, this is not the library you are looking for. There was a talk at this year's C++Now about libcppa and VAST. VAST is a distributed, interactive database allowing you to do full-text search over gigabytes (that's only what's working right now, VAST aims for scanning petabytes!) of data in realtime, i.e., sub-second round-trip times. Matthias Vallentin gave a great talk about his design - purely based on libcppa actors! The flow-control needed to do the indexing in realtime (which btw is a constant stream of events) is build *on top* of actors. Not the other way around. Matthias tried a ZeroMQ design first, you might want to ask him how well that went... Having a low level of abstraction does by no means implies good performance or scalability. This is a fundamental design decision and I want people to write code on a sane, reasonable level of abstraction. If you can't reason about your code, it doesn't matter how "efficient" your building blocks are. The actor model is so appealing because it *takes away* the complexity of distributed runtime environments. And guess what? You can get insane performance out of actor systems with less headache. If you don't believe me that actor systems scale, go have a look at the selection of Production Users at http://akka.io/ and see for yourself. Those companies pick Scala and Java over C++ for *performance-critical applications* because of the actor model.

Boost.Actor implements its own network protocol, but you often need to integrate with an existing protocol, such as MQTT [7] or DDS [8].

You can integrate any network protocol by using brokers: http://neverlord.github.io/boost.actor/manual/#sec45 There's an example how to integrate Google Protobuf in libcppa: https://github.com/Neverlord/libcppa/tree/master/examples/remote_actors

We can add distribution by having proxies. The proxies can hide the details about routing (e.g. actors may change location due to load balancing or migration,) and network protocol.

That's exactly how it's done.

Library reuse -------------

Although Boost.Actor reuses other Boost libraries, it has implemented quite a lot that either exists in other Boost libraries, or that could be moved to those.

You have already mentioned that you do not use MPL, Serialization, and Asio, so I will not delve into these, other than saying that I believe that having your own socket implementation instead of using Boost.Asio is a show-stopper.

Aren't peer reviews about interface design, documentation, and testing? I cannot believe an implementation detail can be a show-stopper. To be quite frank, I just don't care about it. I do care about performance. As long as Asio delivers equal or better performance, I'll migrate sooner rather than later. But to me, this is an unimportant implementation detail.

Apart from these three, there are other libraries that should be considered. Boost.Actor has:

o Own continuations instead of Boost.Thread (future::then)

Futures don't deal with messages and know nothing about the scheduling in boost.actor. The syntax is similar, but the continuations used in boost.actor are syntactic sugar for the message pasing underneath.

o Own producer-consumer queue instead of Boost.Lockfree

The producer-consumer queue used in the scheduler ist based on an excellent Dr. Dobb's article of Herb Sutter [2] and performs reasonably well. Maybe there's interest in adding it to Boost.

o Own logging framework instead of Boost.Log, although I would prefer not having logging in a library at all.

As a user, you won't have logging. It's purely for debugging purposes and not compiled unless you define the macros to do so. It really should be in the detail namespace though.

o Own UUID instead of Boost.UUID

Can this library give me the UUID of the first hard drive? That's the only use case I have for this. The generators in the documentation don't mention anything like this.

o Own time duration instead of Boost.Chrono

The reason is the same why I don't ues std::chrono::duration: they are templated. I need a generic duration type that has the unit as member rather than as template parameter and also can be invalid. Maybe I could replace this with optionalstd::chrono::milliseconds in the future, though it would mean to hardcode the maximum resolution.

Then there are code that could be refactored to other Boost libraries so they can be used in other contexts. For example:

o Stacktrace dumper o RIPEMD hash function o MAC address

Agree, except for the stacktrace dumper. Unless someone else refactores it to work on Windows. I hope I could convince you that you are not requesting changes to boost.actor. What you want is a different library entirely. An actor *is* the fundamental building block, that's what actor programming is all about. [1] http://libcppa.blogspot.de/2011/04/mailbox-part-1.html [2] http://www.drdobbs.com/parallel/writing-a-generalized-concurrent-queue/21160...

On Sat, May 17, 2014 at 04:06:52PM -0500, Larry Evans wrote:

The way I read Bjorn's statement:

I am not against a higher level actor API, but the fundamentals need to be in place first.

is that Bjorn's not objecting to a high level of abstraction; however, he wants a lower level first on which to build the higher level. IOW, he's not saying, as you suggest:

to not having an actor library in Boost

At least that's the way I read his post, FWIW.

Fair enough. I'm not subjecting the idea of having buiding blocks generalized to allow others to build on top of it. I hope my response didn't sound disrespectful. Still, I don't think active objects are building blocks for actors, they're a different kind of abstraction. Bjorn also mentioned data flow as building block. You can see the messages going through an actor system as some kind of dataflow, but there are no streams. This is about message oriented programming, not about streams and dataflows. It's a different paradigm. The middleman operates on messages and (logical) actor addresses, not on data flows. I don't know if the middleman implementation has parts that are worth separating. It (1) manages connections (you can do that with Asio already), (2) dispatches messages based on logical actor addresses, (3) manages proxy instances for remote actors, and (4) keeps a lookup table for type IDs for each connection it has; this is only an optimization though, nothing essential. That's about it. The serialization layer used by the middleman might be useful to others, but it is strongly coupled with uniform_type_info. If there's any interest in lifting this thing to its own library, I have no objections. Please let's take the discussion to a different level. With this preliminary submission, I wanted to get a feeling if there is interest in a Boost.Actor library and if it can get enough momentum to eventually find a review manager. I guess this is the question I should have asked in my original post: "Do you think Boost should provide a high-level programming paradigm such as the actor model and if you do, do you think the proposed design is the right one?" I know that this is a large library. The more important it is in my opinion to talk about concepts, API, and possibly documentation. Maybe I'm naive, but separating parts of the library or tweaking implementation details should be the last step, should it not?

However, I believe the actor model is not the *best* possible approach to message passing. The model is rather intricate, with monitors, links, handles, timeouts, priorities, groups, and so on. To me it seems a bit like the OO of concurrency: well-designed and insightful, but needlessly complicated compared to a more general and powerful paradigm such as generic programming.

In my eyes, the well-defined failure semantics with links/monitors do not convolute the design, but rather make the important aspect of error handling explicit. Moreover, priorities, links, monitors are all *opt-in* and concepts orthogonal to each other. A user can ignore them if desired. To stick with your analogy, it sounds to me that this modular behavior is what you'd expect from "concurrent generic programming."

I think the *right* design would be a concurrent equivalent of generic programming, where the only fundamental building blocks should be a well-designed statically typed SPSC queue, move semantics, a low-level thread launching utility (such as boost::thread) and a concise generic EDSL for the linking of nodes with queues.

The notion of *right* is very subjective, in my eyes. For example, I personally don't want threads to be the concurrency building block in my application. I would like to run as many threads as I have cores on my machine, and a scheduler that maps logical tasks to a thread pool. Today, a thread is what C++ programmers choose as concurrency primitive. But it's a hardware abstraction and does not scale. (You cannot spawn millions of threads efficiently.) Your application may offer a much higher degree of logical parallelism, for whatever notion of task you choose. We have to start appreciating that other languages have had tremendous success with the actor model. Skala/Akka, Clojure, Erlang, all show that the this is an industrial-strength abstraction of not only concurrency but also network transparency. (When programming for cloud/cluster applications, one has to consider the latter; see below.)

start(readfile(input) | runlengthenc | huffmanenc | writefile(output));

You describe a classic pipes-and-filters notion of concurrency here, where presumably you'd expect your data to flow asynchronously through the filters. Effectively, this is just syntactic sugar for message passing, where nodes represent actors taking one type of message, transforming it, and spitting out another (except for the sink). Such an EDSL is orthogonal to the underlying mechanism for message passing.

I dislike the option to throw out static typing but I realise that's a matter of taste.

Yeah, I agree with you. Static typing is what makes C++ powerful. We have to understand though, that from the perspective of a single actor, message handling is *always* type-safe. It's only when you build larger systems and want to test whether the protocol match. And libcppa offers that, it's just more boilerplate. For rapid prototyping, I can understand that one may want a weaker notion of protocol compatibility, though.

I'm a bit skeptical about the necessity and usefulness of built-in network transparency, but you might be able to convince me that it needs to be there.

I feel quite the opposite: network transparency is an essential aspect of any message passing abstraction. When developing cluster-scale applications, I would like to write my application logic once and consider deployment an orthogonal problem. Wiring components without needing to touch the implementation is a *huge* advantage. It enables implementing complex and dynamic behaviors of distributed systems, for example spawn new nodes if the system sense a compute bottleneck. Matthias

...

In my eyes, the well-defined failure semantics with links/monitors do not convolute the design, but rather make the important aspect of error handling explicit.

I will immediately concede that this is important, but I don't think it is the only possible way.

The hierarchical “supervision trees” (as they are called in Erlang) have proven to be a solid foundation to build applications on. That being said, Boost.Actor is in a state where it provides the basics of an actor system (Links, Monitors, etc), but is by no means “complete”. That’s why I think this is the perfect time to submit the library: there’s still room for input and contributions from the community and this is what we are looking forward to.

...

For example, I personally don't want threads to be the concurrency building block in my application. I would like to run as many threads as I have cores on my machine, and a scheduler that maps logical tasks to a thread pool. Today, a thread is what C++ programmers choose as concurrency primitive. But it's a hardware abstraction and does not scale. (You cannot spawn millions of threads efficiently.) Your application may offer a much higher degree of logical parallelism, for whatever notion of task you choose.

In the approach I proposed threads would be fundamental building blocks of the framework, but they do not need to be building blocks in your application. In fact, there is a fairly straightforward way to implement a worker pool with a scheduler as an abstraction on top of the fundamental building blocks. Your application could create the same network of nodes and queues and feed it into the abstraction of the scheduled worker pool instead of directly into a thread launcher, or even take a hybrid approach.

In a sense, Boost.Actor is already a hybrid system. It uses a work-stealing scheduler underneath for its cooperatively scheduling, which of course uses threads. If you have actors that really need to have their own thread (e.g. because they are calling blocking APIs), you can simply opt-out of the cooperative scheduling by using spawn<detached>(...).

...

We have to start appreciating that other languages have had tremendous success with the actor model. Skala/Akka, Clojure, Erlang,

I do appreciate that! In fact this is the main reason I believe the actor model is *good*, and learning about Erlang and the actor model caused me to look into SPSC queues. I just think it is possible to do *even better*.

I don't think there is is a “best” solution when it comes to concurrency. There are amazing Haskell applications using data flow paradigms. Clojure has agents concurrently working on transactional data structures which is just great for simulations (have a look at the Clojure ant simulation from Rich Hickey). The actor model has a unique and powerful way to describe both concurrent and distributed applications. That’s why I prefer it for most applications.

...

I feel quite the opposite: network transparency is an essential aspect of any message passing abstraction. When developing cluster-scale applications, I would like to write my application logic once and consider deployment an orthogonal problem. Wiring components without needing to touch the implementation is a *huge* advantage. It enables implementing complex and dynamic behaviors of distributed systems, for example spawn new nodes if the system sense a compute bottleneck.

In other words, it is very powerful to work with nodes/workers/actors without needing to know whether they are on the same processor or a remote one. I understand this and I agree that network transparency has value. What I'm rather skeptical about is that it needs to be built-in by default; I would prefer it to be opt-in.

It’s actually easy to separate the middleman and ship the network abstraction separately. The networking parts are not built into the actor class(es). Maybe we could use the namespace boost::actor_io for this and move the publish/remote_actor function pair as well as the brokers to this namespace (and provide it as separate library to link against).

I think separating sublibraries would be wise. In addition to actor programming, libcppa appears to provide three other facilities: CoW tuples, guards and pattern matching, and platform-independent RTTI. It would seem reasonable (and modular) to me to treat each as an independent sublibrary and review them as such.

Boost.Actor is actually a redesign of libcppa, not just a port to Boost. The CowTuples are gone. I found them to be too entangled with the RTTI system and it’s just not needed for the actor programming. The guard expressions are gone, too. You still can do filtering by providing a “T -> optional<T>” functor. The guard expression library seemed neat at the time, but at the end of the day it’s making easy things complicated and had a rather lengthy implementation. The pattern matching has been reduced to be a factory for message handlers. It’s only working on messages, so there’s no point in separating it IMO. My hope is that the proposal for Type Switch / Native Pattern Matching will someday make its way to the standard, rendering any library implementation obsolete.

Nat Goodspeed wrote:

...

I think the *right* design would be a concurrent equivalent of generic programming, where the only fundamental building blocks should be a well-designed statically typed SPSC queue, move semantics, a low-level thread launching utility (such as boost::thread) and a concise generic EDSL for the linking of nodes with queues. All further abstractions can be built on top of those ingredients (a bit like the STL is built on top of templates). It should allow me to do something like this:

start(readfile(input) | runlengthenc | huffmanenc | writefile(output));

At some point I want to publish a proposal for a library that does exactly this, but I need to find more time to work on it. The good news is that all necessary ingredients appear to already exist with C++11, Boost.Proto and Boost.Lockfree.

[OT, sorry...] Julian, are you familiar with this pipeline proposal? http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2013/n3534.html

No I wasn't! Thanks a lot for making me aware of it. It is almost exactly what I had in mind, except that the authors seem to not have realised the full generality of the approach. This worries me. Has the proposal progressed far towards standardization already? If I want to react to this proposal so that the standards committee knows about it, what should I do? Cheers, Julian

On 06/11/2013 11:31 PM, james wrote:

I have found systems with soft data structures to be more easily enhanced than ones built from IDL, and have not had many problems from mismatched syntaxes. Rigid syntax (whether from IDL, or some XML form check) always seem to end up subverted with additional key/value data or (even worse) data embedded into 'structured comments'. There's a reason for that and it all goes back to schema evolution, and that's why systems like protobuf tend to sacrifice clean syntax for extensibility..

JSON (combined with REST) is probably a better example, because protobuf is also built from an IDL (the .proto file) and knowledge of the IDL is needed in order to correctly interpret some fields.

On 03.06.2013, at 15:09, Brandon Kohn wrote:

On 5/17/2013 5:26 PM, Charousset, Dominik wrote:

...

is there any interest in a library for actor programming? I'm quite interested in this.

The library we've developed is a C++11 open source library named "libcppa". It is currently released under LGPL2, but re-releasing the library under the boost license is not an issue. You'll find all important links and ressources either under http://libcppa.org or on the GitHub project page: https://github.com/Neverlord/libcppa, including a user manual as HTML or PDF. We've also submitted a paper to the C++Now conference. I agree that embracing C++11 is great, but would also be curious if a significant amount of functionality could also work with earlier versions/compilers by using the boost equivalents. This is perhaps a bit selfish of me because I work in multi-agent simulation, and our current code base is locked in msvc-9.0.

The project originally started as C++03 project under the name 'acedia' (http://sourceforge.net/projects/acedia/), but you really end up tilting at windmills and the resulting code is neither clean nor readable. We switched to C++11 for that reason when GCC's "C++0x" support included lambdas and variadic templates (we've started development on a GCC 4.5 beta release). Variadic templates and lambda expressions are ubiquitously used in libcppa. I guess you could "back-port" the core functionality, but you would destroy maintainability in the process. By replacing every peace of code in libcppa that uses C++11 features with a C++03 workaround, you'll increase the code size by an order of magnitude, while reducing the readability to zero. Let alone behavior definitions. Without lambdas, you will end up using boost::bind *a lot*. Due to the lack of constexpr, you will have to evaluate each "atom(..)" call at runtime. That combined with the fact that boost::bind relies on heap allocations (while lambdas just use the stack), the back-port would perform very poorly.

Let me know if you'd like any assistance. Your library looks really nice, and I'd be happy to help.

Support is always appreciated. :) Just send me a mail what parts you are interested in (development, testing/debugging, documentation, help in porting it to windows either once MS finally has sufficient C++11 support or using MinGW for example). Best regards, Dominik

On 6/5/2013 9:48 AM, Hartmut Kaiser wrote:

Just to reiterate, HPX (https://github.com/STEllAR-GROUP/hpx/) is a very similar library exposing actor oriented paradigms to the programmer (amongst other things), just wrapped up into a C++11 compliant interface (future, async, thread, bind, function, tuple, etc.). It's usable with C++03 compliant compilers (I have not tried msvc-9.0, though). While not part of Boost, it strongly adheres to Boost coding guidelines and style and it is maintained by long standing Boost authors.

Thanks for posting this Hartmut. I was not aware of this library. It looks very interesting. Cheers, Brandon

On 06/24/2013 01:34 PM, Dominik Charousset wrote:

Actually, it is just a simple 6-bit encoding using a lookup table [1]. In this way, the mapping is guaranteed to be collision free and invertible.

Base-64 is also just a simple 6-bit encoding using a lookup table ;-) Looking at the code you referenced, it appears that your encoding algorithm is the same as the base-64 algorithm, but your lookup table is slightly different (yours includes space and underscore instead of plus and slash, and the rest are offset by one.) Thanks for the answer.

Yes, sh is an alias for bash on my test machines. I fixed the configure script in the master branch. The FindBoost script supports hints about search paths, in particular BOOST_LIBRARYDIR might be useful: http://www.cmake.org/cmake/help/git-master/module/FindBoost.html. Hope that helps. ________________________________________ From: Boost [boost-bounces@lists.boost.org] on behalf of Larry Evans [cppljevans@suddenlink.net] Sent: Wednesday, May 14, 2014 21:23 To: boost@lists.boost.org Subject: Re: [boost] Is there any interest in a library for actor programming? On 05/14/14 13:48, Larry Evans wrote:

On 05/17/13 16:26, Charousset, Dominik wrote:> Hi,

...

is there any interest in a library for actor programming?

The library we've developed is a C++11 open source library named "libcppa". It is currently released under LGPL2, but re-releasing the library under the boost license is not an issue. You'll find all important links and ressources either under http://libcppa.org or on the GitHub project page: https://github.com/Neverlord/libcppa,

[snip] I tried following the steps described here:

https://github.com/Neverlord/boost.actor

in the README.md; however, apparently my bash interpreter is not the same as yours:

~/prog_dev/boost/sandbox/boost.actor $ echo $BOOST_ROOT /home/evansl/prog_dev/boost/boost-modular/modular-boost ~/prog_dev/boost/sandbox/boost.actor $ ./configure ./configure: 94: ./configure: [[: not found -- The C compiler identification is GNU Changed 1st line of configure to:

#!/bin/bash and now there's no error with the [[...]]. Apparently, on your system, sh is same as bash; however, on mine, I guess they're different. However, still getting the errors about not finding boost libraries: CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1202 (message): Unable to find the requested Boost libraries. Boost version: 1.56.0 Boost include path: /home/evansl/prog_dev/boost/boost-modular/modular-boost The following Boost libraries could not be found: boost_system boost_context boost_coroutine boost_thread _______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 05/15/14 15:14, Larry Evans wrote:

On 05/14/14 14:58, Charousset, Dominik wrote:

...

Yes, sh is an alias for bash on my test machines. I fixed the configure script in the master branch.

The FindBoost script supports hints about search paths, in particular BOOST_LIBRARYDIR might be useful: http://www.cmake.org/cmake/help/git-master/module/FindBoost.html. Hope that helps.

I tried the attached wrapper around boost.actor/configure; however, despite setting all the relevant environment variables, cmake still fails to find the boost libraries:

~/prog_dev/boost/sandbox $ . boost.actor.configure.sh ("BOOST_ROOT=" "/home/evansl/prog_dev/boost/boost-modular/modular-boost") ("Boost_DIR=" "/home/evansl/prog_dev/boost/boost-modular/modular-boost") ("BOOST_INCLUDEDIR=" "/home/evansl/prog_dev/boost/boost-modular/modular-boost") ("BOOST_LIBRARYDIR=" "/home/evansl/prog_dev/boost/boost-modular/modular-boost") -- The C compiler identification is GNU -- The CXX compiler identification is GNU -- Check for working C compiler: /usr/bin/gcc -- Check for working C compiler: /usr/bin/gcc -- works -- Detecting C compiler ABI info -- Detecting C compiler ABI info - done -- Check for working CXX compiler: /usr/bin/c++ -- Check for working CXX compiler: /usr/bin/c++ -- works -- Detecting CXX compiler ABI info -- Detecting CXX compiler ABI info - done -- Found g++ version 4.8

-- Looking for C++ include valgrind/valgrind.h -- Looking for C++ include valgrind/valgrind.h - found -- Build type: CMake Error at /usr/share/cmake-2.8/Modules/FindBoost.cmake:1202 (message): Unable to find the requested Boost libraries.

Boost version: 1.56.0

Boost include path: /home/evansl/prog_dev/boost/boost-modular/modular-boost

The following Boost libraries could not be found:

boost_system boost_context boost_coroutine boost_thread

No Boost libraries were found. You may need to set BOOST_LIBRARYDIR to the directory containing Boost libraries or BOOST_ROOT to the location of Boost.

Do I need to create a link library for each of these boost libraries? I assumed the above error messages meant it could find the include files for the libraries, but I noticed system and context have build/Jamfiles which, after a brief glance, look like they create a link library.

...

I created the boost libraries and tried the boost.actor configure again and it worked. However, the subsequent make failed because compiler could not find memcmp, as shown by the terminal session at the end of this post. Note that I did do a `git diff` which showed the only' difference was in the ./configure. Also, there were several warnings.

Is there something I should do to get the make to work?

[snip] I modified the configure arguments as shown in 1st attachment. Now, at the `make` step, it aborts at the same place, but with a message about incompatible args instead of not finding the memcmp function. The complete terminal session is in 2nd attachment.

Thank you for your patience. The function memcmp is defined in <cstring>, which was simply not included. I don’t know why I never got this error (I’ve tested on SuSE 13.1, Ubuntu 13.10, Fedora, Mac OS, MinGW @ Win7). Btw, you don’t have to pass both GCC and Clang to the configure script. I would recommend using the compiler you’ve used to build the Boost libraries to avoid running into ABI incompatibilities. I have added the missing include in the master branch and fixed the mismatched-tags warnings as well. Hope that fixes your issues. Oh, and don’t be surprised that almost each file has changed if you’re running git pull: the license has been changed from LGPL to the Boost Software License. Best regards, Dominik

[ 20%] Building CXX object CMakeFiles/libboost_actor.dir/src/cs_thread.cpp.o /home/evansl/prog_dev/boost/sandbox/boost.actor.dir/boost.actor/src/cs_thread.cpp:154:61: error: no member named 'minimum_stacksize' in 'boost::coroutines::basic_standard_stack_allocatorboost::coroutines::stack_traits' auto mss = static_cast( stack_allocator::minimum_stacksize() ~~~~~~~~~~~~~~~~~^ /home/evansl/prog_dev/boost/sandbox/boost.actor.dir/boost.actor/src/cs_thread.cpp:158:13: error: assigning to 'context' (aka 'void **') from incompatible type 'fcontext_t' (aka 'void *') ctx = ctxn::make_fcontext(sinf.sp, sinf.size, cst_trampoline); ^ ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~ /home/evansl/prog_dev/boost/sandbox/boost.actor.dir/boost.actor/src/cs_thread.cpp:160:24: error: member reference base type 'ctxn::fcontext_t' (aka 'void *') is not a structure or union ctx->fc_stack.sp, ~~~^ ~~~~~~~~ 3 errors generated. make[3]: *** [CMakeFiles/libboost_actor.dir/src/cs_thread.cpp.o] Error 1 make[3]: Leaving directory `/home/evansl/prog_dev/boost/sandbox/boost.actor.dir/boost.actor/build' make[2]: *** [CMakeFiles/libboost_actor.dir/all] Error 2 make[2]: Leaving directory `/home/evansl/prog_dev/boost/sandbox/boost.actor.dir/boost.actor/build' make[1]: *** [all] Error 2 make[1]: Leaving directory `/home/evansl/prog_dev/boost/sandbox/boost.actor.dir/boost.actor/build' make: *** [all] Error 2 ~/prog_dev/boost/sandbox/boost.actor.dir/boost.actor $

Please, how should I correct the errors?

What Version of Boost are you using? Did they change the Boost.Context API again? Disabling it should help: ./configure --disable-context-switching This option turns off the context-switching actor implementation, i.e., actors that use the blocking API but still participate in the cooperative scheduling.

On May 24, 2014, at 5:37 PM, Larry Evans wrote:

...

What Version of Boost are you using?

I'm using git. Maybe this output will tell you what I'm using:

I’ve disabled the context-switching feature per default in the master branch. This also gets rid of the context and coroutine dependency for now.

/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/src/get_root_uuid.cpp:79:10: fatal error: 'boost/actor/detail/algorithm.hpp' file not found #include "boost/actor/detail/algorithm.hpp" ^ 1 error generated.

What should be done about this problem?

That header was a leftover from libcppa. I’ve fixed this issue in master and tested a Linux build of Boost.Actor on Ubuntu 13.10 OpenSuSE 13.1. Have fun. :)

On 05/25/14 10:01, Larry Evans wrote:

On 05/24/14 15:32, Dominik Charousset wrote:

...

On May 24, 2014, at 5:37 PM, Larry Evans wrote:

...

...

What Version of Boost are you using?

I'm using git. Maybe this output will tell you what I'm using:

I’ve disabled the context-switching feature per default in the master branch. This also gets rid of the context and coroutine dependency for now.

...

/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/src/get_root_uuid.cpp:79:10: fatal error: 'boost/actor/detail/algorithm.hpp' file not found #include "boost/actor/detail/algorithm.hpp" ^ 1 error generated.

What should be done about this problem?

That header was a leftover from libcppa. I’ve fixed this issue in master and tested a Linux build of Boost.Actor on Ubuntu 13.10 OpenSuSE 13.1. Have fun. :)

Thanks Dominik. That solved the problem and now `make` completes with just warnings:

~/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor $ make make[1]: Entering directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build'

make[2]: Entering directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build'

make[3]: Entering directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build'

Scanning dependencies of target libboost_actor make[3]: Leaving directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build'

make[3]: Entering directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build'

[ 0%] Building CXX object CMakeFiles/libboost_actor.dir/src/middleman_event_handler_epoll.cpp.o [ 1%] Building CXX object CMakeFiles/libboost_actor.dir/src/abstract_actor.cpp.o In file included from /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/src/abstract_actor.cpp:31:

/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/./boost/actor/actor_addr.hpp:116:1: warning: 'hash' defined as a class template here but previously declared as a struct template [-Wmismatched-tags] class hashboost::actor::actor_addr { ^ /home/evansl/dwnlds/gcc/4.8.1/install/lib/gcc/x86_64-unknown-linux-gnu/4.8.1/../../../../include/c++/4.8.1/system_error:112:5: note: did you mean class here? struct hash; ^ . . . make[3]: Leaving directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build'

[100%] Built target typed_calculator make[2]: Leaving directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build'

make[1]: Leaving directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build'

~/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor $

Unfortunately, `make test` failed all the tests. Adding a -VV flag to the /usr/bin/ctest invocation, as shown here: # Special rule for the target test test: $(CMAKE_COMMAND) -E cmake_echo_color --switch=$(COLOR) --cyan "Running tests..." /usr/bin/ctest -VV --force-new-ctest-process $(ARGS) .PHONY : test in the */build/Makefile resulted in output: ~/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor $ make test make[1]: Entering directory `/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build' Running tests... UpdateCTestConfiguration from :/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/DartConfiguration.tcl UpdateCTestConfiguration from :/home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/DartConfiguration.tcl Test project /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build Constructing a list of tests Done constructing a list of tests Checking test dependency graph... Checking test dependency graph end test 1 Start 1: ripemd_160 1: Test command: /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/bin/test_ripemd_160 1: Test timeout computed to be: 9.99988e+06 1: /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/bin/test_ripemd_160: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.18' not found (required by /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/lib/libboost_actor.so.0) 1: /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/bin/test_ripemd_160: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.17' not found (required by /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/lib/libboost_actor.so.0) 1: /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/bin/test_ripemd_160: /usr/lib/x86_64-linux-gnu/libstdc++.so.6: version `GLIBCXX_3.4.19' not found (required by /home/evansl/prog_dev/boost/boost-modular/modular-boost/sandbox/rw/sandbox/boost.actor.dir/boost.actor/build/lib/libboost_actor.so.0) 1/16 Test #1: ripemd_160 .......................***Failed 0.00 sec How do you allow the test runs to find the libstdc++.so.6 files? Maybe modify the LD_LIBRARY_PATH? Right now, it's empty. -regards, Larry