On 8 Jul 2014 at 21:23, christophe.j.henry@googlemail.com wrote:
My only qualm, really, is that proposed Boost.Asynchronous does not appear to me to implement pull-push pipeline management which is the hard part of a Pipelines implementation. I am under the possibly mistaken understanding that HPX does.
What do you mean with pull-push? Maybe I'm making it myself too easy but I'd see every pipeline stage as a scheduler, say, for Asynchronous a stealing threadpool scheduler(with one or more threads), every stage getting a job transforming input data and posting to the queue of the next scheduler a functor doing the next stage transformation, etc. Then I'd create a composite in one line of code to make sure work-stealing happens and that would be it for the infrastructure.
Am I missing something?
We almost certainly are meaning different things. I meant something like: auto p = Paragraphs << TextFile << HTMLFile << "http://www.boost.org/" Paragraphs::Words::iterator it=p[4].match("Niall"); while(it) std::cout << "'Niall' found at offset " << it->offset << std::endl; Here a pipeline extracting paragraphs from the text content of the HTML file from boost.org is constructed on the first line. No processing actually happens yet. The second line configures more pipeline, an iterator matching all instances of the word 'Niall' in the fifth paragraph of the extracted text. Still no processing happens yet. The third line loops the iterator, printing the offsets of 'Niall' in the fifth paragraph. At this point the Paragraphs issues a pull forward that it wants the fifth paragraph to TextFile, TextFile tells HTMLFile it only wants the fifth <p>...</p>, HTMLFile only bothers fetching as much of the content as will supply the fifth <p> content. It then pushes what content it also retrieved back down the pipeline as maybe empty paragraphs are to be disregarded etc and only the lower stage processors will know anything about that. That's what I meant by push-pull pipeline management. Oh, and it should cope with live content changes, so if the source HTML changes the entire pipeline is flushed of state until the next pull or is dynamically updated with the new content and all current code using it is told. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/