On 1 Sep 2015 a 11:41, Joel FALCOU wrote:
As a concluding note, I *really* want Niall to keep going on making AFIO the reference async file I/O library. Just take the needed time to reflect on the afformentionned PAI and implementation issues and remember that there is no rush to get there.
Thanks for the review Joel. This final point is an interesting one, and I think it is worth discussing a bit further, though everyone note I start my holidays away from Boost on Thursday and you won't see me again here until GSoC starts up again in January 2016. Race free filesystem is far more useful to the C++ community than async file i/o which really does not deliver what people think it does without completely rearchitecting their application. Race free filesystem would be a big value add for reliability and predictability for any code using the file system because it very substantially reduces the chance for surprise and surface for security attacks. As Beman mentioned a long time ago, for race free filesystem we need to standardise on some method of representing open file handles, and then somehow figure out how that fits nicely with STL iostreams. This is a big mess of difficult and hard, because everybody will have an opinion right from Boost up to WG21, and moreover most people's opinions will have no technical basis for being better than most other people's opinions. There are many, technical merit equivalent, ways to skin that cat, and piloting a compromise suitable to enough people to be standardisation capable would be a big multi-year commitment. Nevertheless, such a library would be of general value to the C++ community, and moreover WG21 is keen on the idea. It would probably be just as slow as AFIO's file path consuming APIs i.e. a lot slower than Boost.Filesystem or STL iostreams path operations, and therefore probably cannot wholly replace the existing solutions (unless POSIX fixes the missing functionality we have to workaround with inode comparing loops - I really wish POSIX would take note of Windows more on file system, Windows has for the most part a far superior implementation). But I think the knock on consequences on modernising STL iostreams to better fit today's C++ world would be enormously valuable, and indeed the source of much disagreement and debate. If anyone wished to volunteer to design and write such a library and take it eventually for standardisation, I would be more than happy to act as an advisor with hard experience from the coal face, but not as an implementor. You may be surprised to hear that I don't much care for conflict, and getting that library past review would involve a whole ton of conflict and I just don't have the energy given my plans for the next few years. Other fish to fry, as I will explain. Regarding AFIO, as you all now realise it is not the library you thought it would be, but my big hope was and is that this perhaps too early review has given you all a chance to ponder what a next generation standardised C++ file system and file i/o library ought to be, and that might bear fruit in any later review of AFIO in months or years to come. Many of the reviews felt something like what ASIO provides for seekable async streams makes a lot more sense through virtue of being much ligher weight - yet, as I countered, if you want that then ASIO already has that ready to go for you. I aim for AFIO to bring something very new to the table: a toolkit of fundamental portable primitives for bare metal programming of file systems where "bare metal" refers to the direct, unmodified exposure of portable filing system semantics to the programmer rather than "bare metal" in terms of least implementation. Where a particular platform differs from the standardised abstract portable model presented by AFIO to the programmer, emulation is performed to make up the shortfall, and hence the apparent complexity and overheads relative to a least implementation solution. That comes, unavoidably, with a very different weighting of what complexities and relative tradeoffs various operations have than you are used to, and it will wreck you head initially. But I am confident that after a few months of it getting into your head space, it'll start to make a lot more sense especially given the relative fixed and variable costs of operations on the file system relative to one another, and the next time you see AFIO it'll hopefully all just make more sense why it does what it does in the way it does it. *Especially* when you see the standardised C++ transactional versioned key-value store based on AFIO I have planned, and the enormous world of new C++ program designs you can do when you can just *assume* you have always reliable transactional persistent versioned storage built into the language runtime and which works perfectly on everything from embedded C++ right up to the largest mainframes, works during process bootstrap before main() is called, copes well with power loss and store damage, and has no issue with concurrent users on heterogeneous platforms making scaling up from tiny SSDs to large distributed SANs trivial. Anyone who has ever programmed on Zope (https://en.wikipedia.org/wiki/Zope) will need no convincing of just what a paradigm shift a standardised scalable transactional persistent store in the runtime affords, and this is why I came out of lurking on boost-dev to develop these libraries after being adjunct to the community here since 2002 or so. I have had enough dealing with inherently unreliable data storage in my applications. I want truly reliable storage supplied as standard in every major programming language runtime available. That's my end goal, and AFIO is but a vital step along that path. Anyway, I'll stop boring you now. I'll be around here until Thursday morning British Standard Time for any questions etc, then I'll be deactivating my boost-dev subscriptions until Jan 2016 to take a long overdue break from Boost library development which has sucked down something close to 700 hours since February. Private email will continue to work of course, and I'll be tipping away at no more than eight hours per week on Boost.Outcome and the AFIO engine rewrite until they're done. I'll monitor the Fiber mini-review discussion starting shortly, but I can already give my vote: it's a YES to acceptance (I have been following Fiber closely to ensure lightweight futures will work with it. It's a definite yes, and my congrats to Oliver for doing such a great job). Thank you *everybody* for the reviews of AFIO. They have been enormously valuable, and I am tremendously grateful to you all for taking the time to review my library. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/