[ASIO] directory monitor ASIO extension
Hi, I am going through code of directory monitor ASIO extension by Boris Schäling (http://www.highscore.de/boost/dir_monitor.zip) and I am wondering: what is the benefit of making this piece of code / library an ASIO extension. It just feels like it could be easier to use and implement if it wasn't? I hope I am missing something tho and looking here for enlightenment ;) Regards, Szymon Gatner
if you're trying to write platform independent code and already using
boost, then it's a strong incentive right there, to avoid #if PLATFORM do
this #else do something else, #endif, etc.
A previous employer of mine had used Boris's directory monitor service
code. However there were some issues which we had to work around. I don't
remember what they were. Unfortunately I no longer work with that employer
so don't have access to the source code to jog my memory. It might have
simply been the way in which we were using the code.
On Tue, Aug 5, 2014 at 8:36 AM, Szymon Gatner
Hi,
I am going through code of directory monitor ASIO extension by Boris Schäling (http://www.highscore.de/boost/dir_monitor.zip) and I am wondering: what is the benefit of making this piece of code / library an ASIO extension.
It just feels like it could be easier to use and implement if it wasn't? I hope I am missing something tho and looking here for enlightenment ;)
Regards, Szymon Gatner
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
-- Keith Bennett
if you're trying to write platform independent code and already using boost, then it's a strong incentive right there, to avoid #if PLATFORM do this #else do something else, #endif, etc.
That is what I am not seeing, extensions will often require platform depended code anyway, like this directory monitor - has native Windows and POSIX calls.
A previous employer of mine had used Boris's directory monitor service code. However there were some issues which we had to work around. I don't remember what they were. Unfortunately I no longer work with that employer so don't have access to the source code to jog my memory. It might have simply been the way in which we were using the code.
OK ;)
On Tue, Aug 5, 2014 at 10:46 AM, Szymon Gatner
That is what I am not seeing, extensions will often require platform depended code anyway, like this directory monitor - has native Windows and POSIX calls.
True enough. But at the time there was a difference between us writing and maintaining platform dependent code verses that fact being hidden away in a third party library. So effectively our code would be write-once for all platforms and then the extension would hide away the details for each platform: theoretically reducing our time to market. Then if we had any particular issues, we had the option of either fixing it ourselves or contacting the original author for support. Or if it was egregious enough, we could look for another third party library. Only finally after exhausting those, would we look at writing platform dependent code ourselves. -- Keith Bennett
On Tue, 05 Aug 2014 15:36:05 +0200, Szymon Gatner
I am going through code of directory monitor ASIO extension by Boris Schäling (http://www.highscore.de/boost/dir_monitor.zip) and I am wondering: what is the benefit of making this piece of code / library an ASIO extension.
Keith has already answered your question. :) I just want to let you know that others have continued working on the directory monitor. There is for example this project at GitHub: https://github.com/berkus/dir_monitor Please note that the directory monitor (it's from 2008) can be implemented much better today. For example, have a look at this implementation (from 2011): https://github.com/boostcon/2011_presentations/blob/master/wed/asio_extensio... The directory monitor from 2008 uses a background thread. The (Windows-only) implementation from 2011 is integrated into Boost.Asio's event loop and uses no extra thread anymore. Boris
Hey Boris!
Please note that the directory monitor (it's from 2008) can be implemented much better today. For example, have a look at this implementation (from 2011): <https://github.com/boostcon/2011_presentations/blob/ master/wed/asio_extensions/directory_monitor.cpp>
thanks for answering. I actually saw both implementations (2008 ans 2011) and my question arose from differences from them. Version from 2011 is single class and NOT an ASIO extension - there is no deriving from basic_io_object or io_service::service. Hence my question: what is the benefit of deriving from those classes and making your class an ASIO extension vs 2011 version where io_service is just used "externally" for synchronization purposes.
On Tue, 05 Aug 2014 21:05:30 +0200, Szymon Gatner
[...]Version from 2011 is single class and NOT an ASIO extension - there is no deriving from basic_io_object or io_service::service. Hence my question: what is the benefit of deriving from those classes and making your class an ASIO extension vs 2011 version where io_service is just used "externally" for synchronization purposes.
I think the short answer is that you shouldn't derive from basic_io_object or create your own service unless you have to. :) Before the I/O object windows::object_handle was added to Boost.Asio, there was no way to use Windows event objects as there was no code in Boost.Asio which would call Windows functions like WaitForSingleObject() or RegisterWaitForSingleObject(). It wouldn't have been possible to reuse an existing I/O object to add support for event objects. Thus, windows::object_handle was created which is derived from basic_io_object, comes with its own service (where those Windows API functions are called) and integrated pending asynchronous operations into Boost.Asio's event loop (so Boost.Asio doesn't have to create its own background threads). It's a perfect fit into Boost.Asio's framework. Today if you ever need to wait for events asynchronously, you can reuse windows::object_handle and can ignore the entire framework Boost.Asio set up for I/O objects. Boris
2014-08-05 21:42 GMT+02:00 Boris Schäling
On Tue, 05 Aug 2014 21:05:30 +0200, Szymon Gatner
wrote: [...]Version from 2011 is single class and NOT an ASIO extension - there
is no
deriving from basic_io_object or io_service::service. Hence my question: what is the benefit of deriving from those classes and making your class an ASIO extension vs 2011 version where io_service is just used "externally" for synchronization purposes.
I think the short answer is that you shouldn't derive from basic_io_object or create your own service unless you have to. :) [...] Today if you ever need to wait for events asynchronously, you can reuse windows::object_handle and can ignore the entire framework Boost.Asio set up for I/O objects.
Answer I was looking for, thanks a lot! Regards, Szymon Gatner
On 08/05/2014 03:36 PM, Szymon Gatner wrote:
I am going through code of directory monitor ASIO extension by Boris
You may also be interested in AFIO (F for file.) It has an experimental directory monitor living in some branch whose name I do not know. https://github.com/BoostGSoC/boost.afio.git
On 6 Aug 2014 at 0:17, Bjorn Reese wrote:
On 08/05/2014 03:36 PM, Szymon Gatner wrote:
I am going through code of directory monitor ASIO extension by Boris
You may also be interested in AFIO (F for file.) It has an experimental directory monitor living in some branch whose name I do not know.
Ah yes. The directory monitor isn't ready for production use. By production use I mean: * Can cope with 1m entry directories easily as the rest of AFIO can. * Can handle 50,000 entries changing every second in a 1m entry directory without losing change deltas and without racing on individual stats on a 2Ghz single core, with approximate linear scaling with additional cores i.e. four cores can monitor four 1m entry directories seeing 5% change. * Works as expected on remotely mounted filing systems like NFS. * Uses native OS features to avoid polling when possible. * Works reliably on Microsoft Windows (hard), Linux (slightly easier), BSD and OS X (easiest by far of the lot). Paul Kirth presented on this at C++ Now 2014. It is a surprisingly tough engineering problem, and requires a design approaching ideal for it to work at all. tl;dr; it'll be a while longer. Paul is still kicking at it. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On 6 Aug 2014 at 1:54, Niall Douglas wrote:
I am going through code of directory monitor ASIO extension by Boris
You may also be interested in AFIO (F for file.) It has an experimental directory monitor living in some branch whose name I do not know.
Ah yes. The directory monitor isn't ready for production use. By production use I mean:
Sorry, I forgot (it's nearly 2am) to mention that AFIO itself is production ready and has been in the review queue since October 2013. You can see its CI test dashboard at the bottom of the github page plus a link to its documentation. There is a slight correction to the URL: https://github.com/BoostGSoC13/boost.afio You *will* find AFIO's directory enumeration facilities to be faster than anything else, including the system APIs. AFIO goes direct to the kernel and skips the intervening baggage, hence happily working with 1m or even 10m entry directories on BSD, Linux and Windows with ease. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
2014-08-06 2:54 GMT+02:00 Niall Douglas
On 6 Aug 2014 at 0:17, Bjorn Reese wrote:
On 08/05/2014 03:36 PM, Szymon Gatner wrote:
I am going through code of directory monitor ASIO extension by Boris
You may also be interested in AFIO (F for file.) It has an experimental directory monitor living in some branch whose name I do not know.
Ah yes. The directory monitor isn't ready for production use. By production use I mean:
* Can cope with 1m entry directories easily as the rest of AFIO can.
* Can handle 50,000 entries changing every second in a 1m entry directory without losing change deltas and without racing on individual stats on a 2Ghz single core, with approximate linear scaling with additional cores i.e. four cores can monitor four 1m entry directories seeing 5% change.
I should be OK :) What are the advantages over Boris' version tho?
Paul Kirth presented on this at C++ Now 2014. It is a surprisingly tough engineering problem, and requires a design approaching ideal for it to work at all.
I actually just watched that presentation yesterday. Pretty great piece of code. Regards, Szymon Gatner
On 6 Aug 2014 at 8:58, Szymon Gatner wrote:
Ah yes. The directory monitor isn't ready for production use. By production use I mean:
* Can cope with 1m entry directories easily as the rest of AFIO can.
* Can handle 50,000 entries changing every second in a 1m entry directory without losing change deltas and without racing on individual stats on a 2Ghz single core, with approximate linear scaling with additional cores i.e. four cores can monitor four 1m entry directories seeing 5% change.
I should be OK :) What are the advantages over Boris' version tho?
Boris's won't deadlock as frequently as the AFIO one :). If code isn't in the master or stable branch in AFIO, I wouldn't use it. You could use AFIO's directory enumeration which is in stable combined with Boris' directory monitoring. It depends on if you need unracy metadata support on Windows as Filesystem is currently racy there.
Sorry, I forgot (it's nearly 2am) to mention that AFIO itself is production ready and has been in the review queue since October 2013.
Why isn't it under review yet?
There is a large backlog, some libraries have been waiting in the queue for years. Part of this is due to sickness in the Boost culture and the ongoing decline of Boost for two years now, part is simply because no one wants nor needs a portable asynchronous file i/o library and therefore no one is interested in volunteering as review manager. Besides, it wouldn't pass a review right now because its API would be considered unusable as it was designed for maximum speed, and therefore isn't "Boosty enough" - people would ask "where are the iterators?" etc etc. This is also because the API is designed as a stable binary ABI to be assembled by the forthcoming monadic continuations framework being led out by Vicente. Still, if you need maximum portable file i/o performance now, it works very well and will continue to work into the future. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
participants (5)
-
Bjorn Reese
-
Boris Schäling
-
Keith Bennett
-
Niall Douglas
-
Szymon Gatner