On Thu, Aug 28, 2008 at 1:19 PM, Manuel Jung
I checked out the whole dataflow from sandbox. Example compiles and runs fine.
Great!
[snip]
// our network // increase >>= increase2 will be in its own thread // increase3 will be in its own thread // increase4 will be in its own thread source | (make_delay(pool, increase) >>= increase2) | (make_delay(pool, increase3) >>= make_delay(pool, increase4));
// submit the first task submit(pool, source);
This seems very usable. I also like that you are able to add filters to different pools.
Yeah, that turned out to be a nice advantage.
Like you said, delay might not be the best name, but works for now. A couple of comments:
* The example is hard-wired to a void(int) signature. It wouldn't be hard to generalize, if there is interest.
I need to have it generalize. Would be great if you could do it, otherwise, i'll try at the weekend.
It just need slight tweaks in things that are already there. I might not find time until Sunday, but once I get to it it shouldn't take me long.
Hmm. thread_safe_signals are needed too. There will be new signal connections after the threading.
Frank Mori Hess has integrated a change that I believe is the only change necessary to make thread_safe_signals work with Dataflow.Signals. When I work on the generalization, I will also test it with thread_safe_signals.
* I don't use chained_submit - instead, the delay object will do a regular submit when it gets its input. This was more in line with the push-oriented design of Boost.Signals. To use chained_submits and futures would require either using a pull-based network (each component taking in the input as the argument and returning the result), or propagating the final result back through return values.
That is ok with me. I would prefer all the tasks to be added at the beginning (because of an easier priority handling of the queue), but i can this way too, i think.
To do the chaining at the beginning, we would need have access to the network graph. We can have that using the Dataflow.Blueprint layer, so maybe it's something we can explore in the future.
Seems very great so far. This way you can do late evaluated dataflow networks :-). Just create a pool and make a dataflow network (even a cyclic! :-)) and you could evaluate it step for step, depending by the threadpools queueing pollicy.
Yeah, this was a really good (and concrete) suggestion, thanks for bringing it up. I'm excited about the cyclic possibilities too :-)
Thanks for what you did so far :-). I like the dataflow lib. Can i help writing some code, the generalization for all data types or something else? I just don't know the inside of the dataflow library that good, it would take much longer, if i try to do it... but im of course willing to help. Maybe an other example?
I really appreciate your offer to help. The Dataflow.Signals layer is coming up for review on September 1st, so it would be great if we could continue developing and discussing this example through the review period. As someone that has used / is using the lib (or at least looking at using it), your feedback would be very much appreciated and helpful during the review! If you're interested in joining the development, I have recently started a rewrite of the Dataflow library (the rewrite is to address some things that I already know I want to redesign as well as any feedback received during the review). I am doing the rewrite in a very step-by-step fashion, starting with the library foundations and only moving on when things are properly designed and documented. I can keep you informed of the progress if you'd like, and you would be more than welcome to join in at any point. Best, Stjepan