On 18:19 Mon 31 Aug , Niall Douglas wrote:
On 31 Aug 2015 at 17:20, Andreas Schäfer wrote:
- The naive OpenMP example is using a thread pool, AFIO is using these elaborate, efficient futures, continuations etc. I'd have expected this to be more efficient, especially given the nature of the OpenMP example (which does a sequential directory walk before spawning threads).
More code executed usually means slower performance.
This is a vague answer and doesn't give me any insight WRT the problem at hand.
- Even if we assume that AFIO can't be faster than an OpenMP code in this setting: why is the overhead so high? What is making it >2x slower?
I have been as transparent and honest about the inefficiencies in AFIO as I can be. I have listed copious benchmarks and graphs in the AFIO documentation which show AFIO in as poor a light as I can make it. I have identified another round of low hanging fruit which I feel lightweight futures will help me solve, and I am about to go implement. What more do you want here?
I want an explanation. Saying that were open or that more code was being executed is not particularly enlightening. Since you wrote that both examples essentially execute the same file system calls. I'd like to know which functions within AFIO burn those extra cycles. If it was my code I'd do a profiling run to assess exactly this. Actually I don't want to know this for myself, but I believe it would be valuable to you to steer your future development activities.
In the end AFIO does a lot more than a naïve OpenMP solution, and I never expect it to be quicker without a true kernel async backend to offset the extra work performed. It executes more syscalls and does more processing. In exchange, you get much superior guarantees and reliability.
Why syscalls are executed in addition? I was assuming both codes would run the same file system syscalls.
If you don't want those extra guarantees, then don't use AFIO. As I have suggested on multiple occasions now, ASIO has a perfectly fine async file i/o library.
So AFIO's unique selling point is now supposed to be only reliability WRT race conditions? I'll try to remember this.
What AFIO provides is a *portable* API for async file system and file i/o. It provides the least benefit on Linux of any of the platforms, but that is not AFIO's fault.
Sorry, but that's not true. It remains AFIO's fault as long as you can't explain where this slowdown comes from. Why don't you, say, default to a naive, simple OpenMP implementation on Linux?
I have repeatedly explained where I believe slowdowns come from.
Your speculations, as far as I can remember, were: a) no good API in Linux <4.0, b) more code being executed, c) more syscalls being issued. Did I miss any other explanation? I'm asking this because profiling is part of my job. I do it to find out why my (or other people's) code is slow. The more I do it the less I trust my own speculations. Measurements trump intuition.
You keep choosing to not listen, twist and cherry pick, and harrass me again and again over the same items.
I ask the same questions from different angles to get answers. Mistaking that for harassment is troubling.
I could guess at your motives given you are also in German HPC, but I will leave it at that.
What is that supposed to mean?
I have now had enough of this harrassment and repeatedly explaining the same thing over and over. You have your answers and you have made your vote. Please now be quiet and let others speak.
It's sad that you chose to view my questions as harassment while they were really an opportunity for you to resolve bugs (the segfault on my system) and demonstrate your technical excellence by lecturing me on your design. Kind regards -Andreas -- ========================================================== Andreas Schäfer HPC and Grid Computing Department of Computer Science 3 Friedrich-Alexander-Universität Erlangen-Nürnberg, Germany +49 9131 85-27910 PGP/GPG key via keyserver http://www.libgeodecomp.org ========================================================== (\___/) (+'.'+) (")_(") This is Bunny. Copy and paste Bunny into your signature to help him gain world domination!