Limiting the discussion to just the problem of race free filesystem on POSIX, imagine the problem of opening the sibling of a file in a directory whose path is constantly changing. POSIX does not provide an API for opening the parent directory of an open file descriptor. To work around this, you must first get the canonical current path of the open file descriptor, strip off the end, open that directory and then use that as a base directory for opening a file with the same leafname as your original file. You then loop all that if you fail to open that leafname, or the leafname opened has a different inode. Once you have the correct parent directory, you can open the sibling. This is an example of where caching the stat_t of handle during open saves syscalls and branches in more performance important APIs later on.
Sorry for asking a question again (and I'm sure Niall will ignore it because he judges it as 'disrespectful, petty, or juvenile', but perhaps somebody else can answer this):
You could treat me with the respect due to someone working in this field for several years the same as the respect I treat you for your ample experience in your domain. If I treated you the way you treat me, you would explode as we have seen before with you shouting at other people on this list who you felt were being disrespectful to you.
I am not incompetent. If I chose a design, it was for a reason. It may have been a suboptimal choice, but it is not due to lack of experience nor a lack of prior research. I have been programming for over twenty years, and more than fifteen commercially. I have a very different style and approach to you, but that does not make it inferior or to be mocked or attacked just because I am imprecise in your opinion.
In another mail you said AFIO wouldn't cache anything, thus must do a lot of syscalls over and over again, which is the reason for it to be slow.
We don't second guess the OS. If the OS lies to us - and it does, frequently - we pass through those lies exactly. Similarly, if you loop requesting the length of a file, you'll get back exactly what the OS tells us each time. We always ask the OS for the length if you ask AFIO for the length. No caching.
However the semantics emulation and workaround code still needs to correctly emulate any missing semantics on some platform in order to provide a universal file system model to the AFIO user. AFIO provides race-free sibling lookup which works just fine on Windows, but requires some extra work on POSIX, so on POSIX it runs the algorithm described above.
The difference is not unlike the difference between FreeBSD and Linux with respect to POSIX. POSIX defines a universal portable standard, and both Linux and FreeBSD implement most of that standard. FreeBSD implements POSIX far more accurately than Linux, and has correspondingly lower peak performance in everything from mutexs to opening files.
However, once one includes the cost of the workarounds for Linux's divergences from POSIX in application code, much of that performance gap is eliminated. FreeBSD also scales out much better than Linux because the lower peak performance in a 1-4 CPU machine is really a situation with a higher fixed cost but a less steep slope on the variable cost curve, so as you ramp up the hardware FreeBSD scales better.
AFIO is similar to POSIX here. It provides a universal standard model. All platforms require workarounds to match that model, but the workarounds vary between even Linux and FreeBSD. If AFIO must execute extra syscalls on some particular platform in some particular case, you should accept that as a sunk cost on that platform and not worth worrying about in the bigger picture as the simplified application code with fewer workarounds is likely to perform better, be easier to maintain, and transparently improves as OS facilities improve. Your application code is also much more portable.
Thanks! Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu