On 31 Aug 2015 at 17:20, Andreas Schäfer wrote:
- The naive OpenMP example is using a thread pool, AFIO is using these elaborate, efficient futures, continuations etc. I'd have expected this to be more efficient, especially given the nature of the OpenMP example (which does a sequential directory walk before spawning threads).
More code executed usually means slower performance.
- Even if we assume that AFIO can't be faster than an OpenMP code in this setting: why is the overhead so high? What is making it >2x slower?
I have been as transparent and honest about the inefficiencies in AFIO as I can be. I have listed copious benchmarks and graphs in the AFIO documentation which show AFIO in as poor a light as I can make it. I have identified another round of low hanging fruit which I feel lightweight futures will help me solve, and I am about to go implement. What more do you want here? In the end AFIO does a lot more than a naïve OpenMP solution, and I never expect it to be quicker without a true kernel async backend to offset the extra work performed. It executes more syscalls and does more processing. In exchange, you get much superior guarantees and reliability. If you don't want those extra guarantees, then don't use AFIO. As I have suggested on multiple occasions now, ASIO has a perfectly fine async file i/o library.
What AFIO provides is a *portable* API for async file system and file i/o. It provides the least benefit on Linux of any of the platforms, but that is not AFIO's fault.
Sorry, but that's not true. It remains AFIO's fault as long as you can't explain where this slowdown comes from. Why don't you, say, default to a naive, simple OpenMP implementation on Linux?
I have repeatedly explained where I believe slowdowns come from. You keep choosing to not listen, twist and cherry pick, and harrass me again and again over the same items. I could guess at your motives given you are also in German HPC, but I will leave it at that. I have now had enough of this harrassment and repeatedly explaining the same thing over and over. You have your answers and you have made your vote. Please now be quiet and let others speak. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/