On 27 Aug 2015 at 8:25, Sebastian Theophil wrote:
Ok, let's revisit the original pattern code I mentioned:
EXAMPLE A:
shared_future h=async_file("niall.txt"); // Perform these in any order the OS thinks best for(size_t n=0; n<100; n++) async_read(h, buffer[n], 1, n*4096);
Niall,
is parallel read (parallel writing maybe?) the only use-case where you want a shared_future?
The problem is continuations. You may only add one continuation to a unique future. If you schedule 100 continuations onto the same future, that is a problem.
If I understand Thomas correctly, he doubts you need the shared_future semantics because one async operation hands down a single handle to the next continuation.
Essentially something like:
async_file(“niall.txt”) .async_read(buffer, length_to_read) .async_truncate(length_to_read) .async_close()
Your counter example was an asynchronous *and* parallel read where you need to share the file handle (or rather the future<handle>) between parallel reads. Shouldn’t this be abstracted away in the API somehow? I can’t think of many file operations you want to do N times in parallel. Truncating a file in parallel several times doesn™t seem to make much sense :-)
You want to relax ordering on file i/o as much as possible for optimum performance. It therefore should be assumed to be a very common pattern, and during post-debugging optimisation you the programmer is going to be _relaxing_ ordering as much as possible without breaking your concurrency visibility.
So why not make it:
async_file(“niall.txt”) // Read 100 times asynchrnously and in parallel and provide lambda returning n-th buffer and offset: .async_parallel_read(100, [&](int nCount) { return std::make_pair(buffer[n], n*4096; }) .async_truncate(length_to_read) .async_close()
The 100 reads are internally not ordered but they can only begin once the file has been opened, they consume this handle together, and only after all reads are complete can we truncate.
But isn't this just shared_future?
Is this not what you’re trying to do?
I'm all for unique futures over shared futures where the normal use case is a 1 to 1 mapping of inputs to outputs. However the case of one future to many futures, and many futures to one future comes up a lot in file system programming. Sufficiently so that I chose non-consuming (shared future) semantics as the default for afio::future<>.get_handle(), but left consuming (unique future) semantics for fetching any T in afio::future<T>.get(). Even in another thread with Gavin Lambert I showed how non-consuming futures are needed to ignore errors as AFIO defaults to error propagation. As futures either carry a handle or an error, as I showed in that thread you can use the depends(precondition, output) function to substitute a future output when a future precondition becomes ready. That lets you always close or delete files even if an error occurs. If AFIO defaulted to unique future (consuming) semantics for get_handle(), the user would have to explicitly convert the future to a shared future in their code. I am questioning if that extra hassle for the user is worth it. Thomas has also argued that each async_* function should return only a future specific to its return type, so instead of afio::future<T> always transporting a future handle in addition to any future T, you must supply (I would suppose) two futures to every async_* function, one for the handle, the other as the precondition. I asked if he could propose an API based on that model which is substantially superior to the one used by AFIO, because if he can I'll take it. I can imagine several API models based on unique futures as Thomas advocated. Indeed, I whiteboarded design options about a year ago, as I did genuinely try quite hard to make the current API entirely unique future based. I came to the conclusion that it forced a whole load of extra typing onto the library user for what I felt was very little gain, and it made the AFIO API considerably more complex to both understand and use correctly which had bad consequences on client code complexity and maintainability. I therefore eliminated that API design from consideration. Now, Thomas and his former mentor are world experts in this field. I take their opinions seriously, even if his former mentor has never given my opinions or code on this stuff anything but ridicule and mockery as you have seen in other threads. But I also suspect that their disagreement with my approach comes from one of theory versus boots-on-the-ground, and my view is that C++ is a dirty mongrel language and therefore gets dirty mongel design patterns. In the end, I ask this of C++ library design (in order of priority): (i) Is it fun to program? (ii) Will it bite you in the ass? (iii) Is it very testable and easy to debug and maintainable? (iv) Is it flexibly reusable to solve unanticipated problems? (v) Does it perform well in space and time? I don't care if I trample all over the priniciples of text book proper library design if I get a "yes" to all those five questions. In the end, the C++ compiler to me is just a fancy assembler macro engine, and hence why I upset some people with my "mongrel" design principles. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/