On 4 Jul 2014 at 11:55, Benedek Thaler wrote:
Thanks for the further encouragement. I'll definitely take a closer look at HPX in the future. However, the deadline of GSoC is closing in, I have to found a simpler solution for now. Hopefully, having a more solid base will enable us to build something truly great later.
This is why I originally suggested HPX for this. Hartmut, who is one of the oldest core Boost people still here, did a really great job on HPX which is pretty much the final say on how to do task execution at large scale. A Pipelines implementation matching your proposal wouldn't be trivial on HPX, but it would be tractable before GSoC ends. Your only alternative to deliver before GSoC ends I suspect is to fire a thread at every component in the pipeline as without async i/o, you can't leverage coroutines. As you've probably realised by now, pipeline processors need to communicate in two directions, so end stage processors need to be able to tell the first data source "I need one more byte or else I can't continue" and such. Thing is, that one extra byte at the end of the pipeline may turn in Kb of extra bytes as each processor imposes its own needs on the forward propagation. This is the kind of stuff HPX takes care of for you. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/