[threads] Win32 Events on POSIX
Hello, I want to port win32 events on Posix using Boost. To avoid the re-invented the wheel I use PEvents: https://github.com/neosmart/pevents which is on MIT licence. But with my current knowledge I do some mistakes because I have deadlock. Maybe someone with bigger knowledge about differencies in boost and pevents might help with this ? I know that there is also thread-safe boost.signals2 with observer pattern but it does not provide WaitForMultipleObjects functionality. Best regards.
I want to port win32 events on Posix using Boost. To avoid the re-invented the wheel I use PEvents:
https://github.com/boostorg/sync/ functionally it should mostly be done, but it will need a bit of polishing and documentation (andrey will have a better overview on the state of the project, i haven't been able to spend much time on this project for some time)
On Sunday 15 February 2015 13:36:26 Tim Blechmann wrote:
I want to port win32 events on Posix using Boost. To avoid the re-invented
the wheel I use PEvents: https://github.com/boostorg/sync/
functionally it should mostly be done, but it will need a bit of polishing and documentation (andrey will have a better overview on the state of the project, i haven't been able to spend much time on this project for some time)
You are mostly correct. I'll add that relative timed functions use realtime clock now instead of monotonic.
Thank you for the reply. I see that you have almost all functionality done, I miss only something equivalent to WaitForMultipleObjects it is in your plans for the future functionality ? Best regards 2015-02-15 9:19 GMT+01:00 Andrey Semashev <andrey.semashev@gmail.com>:
On Sunday 15 February 2015 13:36:26 Tim Blechmann wrote:
I want to port win32 events on Posix using Boost. To avoid the re-invented
the wheel I use PEvents: https://github.com/boostorg/sync/
functionally it should mostly be done, but it will need a bit of polishing and documentation (andrey will have a better overview on the state of the project, i haven't been able to spend much time on this project for some time)
You are mostly correct. I'll add that relative timed functions use realtime clock now instead of monotonic.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Monday 16 February 2015 08:30:58 Fu ji wrote:
Thank you for the reply. I see that you have almost all functionality done, I miss only something equivalent to WaitForMultipleObjects it is in your plans for the future functionality ?
On POSIX systems this is only achievable with select() & co., which in turn implies that synchronization primitives must be fd-based. I currently have no plans for implementing that, mostly because I recon that such primitives would have worse performance in the normal use (i.e. when a single primitive is used for thread synchronization).
On 16 Feb 2015 at 11:58, Andrey Semashev wrote:
On Monday 16 February 2015 08:30:58 Fu ji wrote:
Thank you for the reply. I see that you have almost all functionality done, I miss only something equivalent to WaitForMultipleObjects it is in your plans for the future functionality ?
On POSIX systems this is only achievable with select() & co., which in turn implies that synchronization primitives must be fd-based. I currently have no plans for implementing that, mostly because I recon that such primitives would have worse performance in the normal use (i.e. when a single primitive is used for thread synchronization).
You can implement an entirely user space wait_for_all and wait_for_any easily enough. Indeed, one of the things myself and Vicente discusssed for the mooted Boost.Thread v5 rewrite was to move thread scheduling completely into user space whereby the sole and only way of kernel sleeping a thread (apart from i/o) is a thread locally stored condvar. Thread v5 would keep a registry of threads and their thread local condvars, and would wake a thread when needed. With regards to the OP, he may find my C11 permit object of use as that does allow wait composure and works on all platforms, including Windows. See https://ci.nedprod.com/view/Boost%20Thread-Expected-Permit/job/Permit% 20Test%20Linux%20GCC%204.8/lastSuccessfulBuild/artifact/latex/pthread_ permit_reference.pdf specifically the API pthread_permit_select(). The OP should note that pthread_permit_select() is not particularly efficient as it does not allocate memory. I am unaware of an algorithm which permits O(1) waiter complexity and does not require dynamic memory allocation. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On February 16, 2015 3:58:08 AM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 08:30:58 Fu ji wrote:
Thank you for the reply. I see that you have almost all functionality done, I miss only something equivalent to WaitForMultipleObjects it is in your plans for the future functionality ?
On POSIX systems this is only achievable with select() & co., which in turn implies that synchronization primitives must be fd-based. I currently have no plans for implementing that, mostly because I recon that such primitives would have worse performance in the normal use (i.e. when a single primitive is used for thread synchronization).
Performance isn't the only criteria by which to judge an API. The ability to wait on multiple synchronization primitives can be helpful in some cases regardless of performance. Obviously if performance is an issue, that must be documented. Using the Windows API one also can wait on other handles, like process handles, and not just sync primitives. The same is possible using select if one opens a pipe or socket to the other process, since closing the other process will close its end of the IPC. If a select()-based wait on sync primitives is possible, then a portable WaitForMultipleObjects() is possible and would be handy in one's toolbox. ___ Rob (Sent from my portable computation engine)
On Monday 16 February 2015 06:47:00 Rob Stewart wrote:
On February 16, 2015 3:58:08 AM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 08:30:58 Fu ji wrote:
Thank you for the reply. I see that you have almost all
functionality done,
I miss only something equivalent to WaitForMultipleObjects it is in
your
plans for the future functionality ?
On POSIX systems this is only achievable with select() & co., which in turn implies that synchronization primitives must be fd-based. I currently have no plans for implementing that, mostly because I recon that such primitives would have worse performance in the normal use (i.e. when a single primitive is used for thread synchronization).
Performance isn't the only criteria by which to judge an API. The ability to wait on multiple synchronization primitives can be helpful in some cases regardless of performance. Obviously if performance is an issue, that must be documented.
Using the Windows API one also can wait on other handles, like process handles, and not just sync primitives. The same is possible using select if one opens a pipe or socket to the other process, since closing the other process will close its end of the IPC.
I think sigwait or signalfd are better suited for this.
If a select()-based wait on sync primitives is possible, then a portable WaitForMultipleObjects() is possible and would be handy in one's toolbox.
In my experience performance tends to be the defining factor in thread synchronization primitives. I have a few real world cases where I would like to use WaitForMultipleObjects but at the same time I want the best performance there, including when there is only one thread and no actual synchronization needed. I chose not to use select/poll/epoll in those cases for this reason, and I would have made the same decision for WaitForMultipleObjects if it was a wrapper around them. I realize that my experience is not the whole world, and there may be situations where performance is not a concern. However, I feel that in such cases specialized solutions or Boost.ASIO would play better.
On February 16, 2015 8:21:01 AM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On February 16, 2015 3:58:08 AM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 08:30:58 Fu ji wrote:
Thank you for the reply. I see that you have almost all
functionality done,
I miss only something equivalent to WaitForMultipleObjects it is in your plans for the future functionality ?
On POSIX systems this is only achievable with select() & co., which in turn implies that synchronization primitives must be fd-based. I currently have no plans for implementing that, mostly because I recon that such primitives would have worse performance in the normal use (i.e. when a single
is used for thread synchronization).
Performance isn't the only criteria by which to judge an API. The ability to wait on multiple synchronization primitives can be helpful in some cases regardless of performance.
Using the Windows API one also can wait on other handles, like
On Monday 16 February 2015 06:47:00 Rob Stewart wrote: primitive process
handles, and not just sync primitives. The same is possible using select if one opens a pipe or socket to the other process, since closing the other process will close its end of the IPC.
I think sigwait or signalfd are better suited for this.
Those only apply to monitoring signals. The desire here was to wait on multiple things including sync primitives.
If a select()-based wait on sync primitives is possible, then a portable WaitForMultipleObjects() is possible and would be handy in one's toolbox.
In my experience performance tends to be the defining factor in thread synchronization primitives. I have a few real world cases where I would like to use WaitForMultipleObjects but at the same time I want the best performance there, including when there is only one thread and no actual synchronization needed. I chose not to use select/poll/epoll in those cases for this reason, and I would have made the same decision for WaitForMultipleObjects if it was a wrapper around them.
I realize that my experience is not the whole world, and there may be situations where performance is not a concern. However, I feel that in such cases specialized solutions or Boost.ASIO would play better.
Those would only apply to files, sockets, etc. You may be right that performance is paramount for many use cases, but that doesn't mean others require it, so denying support for those use cases may be heavy handed. Having said that, my use cases for WFMO()'s have been limited, so it could be just as well to not provide it. There are other ways to make things work in the absence of WFMO(). ___ Rob (Sent from my portable computation engine)
On Monday 16 February 2015 17:04:34 Rob Stewart wrote:
On February 16, 2015 8:21:01 AM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 06:47:00 Rob Stewart wrote:
The same is possible using select if one opens a pipe or socket to the other process, since closing the other process will close its end of the IPC.
I think sigwait or signalfd are better suited for this.
Those only apply to monitoring signals. The desire here was to wait on multiple things including sync primitives.
I was suggesting a way to monitor a process, as a better alternative to pipes.
However, I feel that in such cases specialized solutions or Boost.ASIO would play better.
Those would only apply to files, sockets, etc.
Boost.ASIO has support for any fd-based entities, and also can be used to invoke functions in the working threads. This covers pretty much everything you may need in order to synchronize with multiple event sources.
In the future i should think about changing the code design and use boost.signals2 but in meantime i would like to have some workaround so i ported: https://github.com/neosmart/pevents and it already works well but when we go out of scope I have probably some heap corruption. (HEAP[app.exe]: HEAP: Free Heap block 363ef10 modified at 363ef38 after it was freed). I attached source code and also test case, maybe I am blind because I don't know what is wrong. Best regards 2015-02-17 5:00 GMT+01:00 Andrey Semashev <andrey.semashev@gmail.com>:
On Monday 16 February 2015 17:04:34 Rob Stewart wrote:
On February 16, 2015 8:21:01 AM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 06:47:00 Rob Stewart wrote:
The same is possible using select if one opens a pipe or socket to the other process, since closing the other process will close its end of the IPC.
I think sigwait or signalfd are better suited for this.
Those only apply to monitoring signals. The desire here was to wait on multiple things including sync primitives.
I was suggesting a way to monitor a process, as a better alternative to pipes.
However, I feel that in such cases specialized solutions or Boost.ASIO would play better.
Those would only apply to files, sockets, etc.
Boost.ASIO has support for any fd-based entities, and also can be used to invoke functions in the working threads. This covers pretty much everything you may need in order to synchronize with multiple event sources.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On February 17, 2015 3:52:18 AM EST, Fu ji <fujimailing@gmail.com> wrote:
i ported: https://github.com/neosmart/pevents and it already works well but when we go out of scope I have probably some heap corruption. (HEAP[app.exe]: HEAP: Free Heap block 363ef10 modified at 363ef38 after it was freed). I attached source code and also test case, maybe I am blind because I don't know what is wrong.
Please don't top post or over-quote on this list. Read the policies at http://www.boost.org/community/policy.html#quoting If your code used C++ rather than C, the problem would be easier to resolve. Constructors, destructors, and other member functions would ensure proper lifetime management, encapsulate a lot of logic, and declutter the API functions. One thing I did notice: your wait loops can easily exceed the supplied timeouts. ___ Rob (Sent from my portable computation engine)
Sorry for policy violation, with loops you mean that I wait for example 800 miliseconds for each event on condition variable ? 2015-02-17 11:02 GMT+01:00 Rob Stewart <rob.stewart@verizon.net>:
On February 17, 2015 3:52:18 AM EST, Fu ji <fujimailing@gmail.com> wrote:
i ported: https://github.com/neosmart/pevents and it already works well but when we go out of scope I have probably some heap corruption. (HEAP[app.exe]: HEAP: Free Heap block 363ef10 modified at 363ef38 after it was freed). I attached source code and also test case, maybe I am blind because I don't know what is wrong.
Please don't top post or over-quote on this list. Read the policies at http://www.boost.org/community/policy.html#quoting
If your code used C++ rather than C, the problem would be easier to resolve. Constructors, destructors, and other member functions would ensure proper lifetime management, encapsulate a lot of logic, and declutter the API functions.
One thing I did notice: your wait loops can easily exceed the supplied timeouts.
___ Rob
(Sent from my portable computation engine)
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On February 17, 2015 5:11:21 AM EST, Fu ji <fujimailing@gmail.com> wrote:
Sorry for policy violation,
You're sorry for violating the policy, but in your reply, you repeat the violations? That suggests either a failure to understand or that you're not actually sorry.
with loops you mean that I wait for example 800 miliseconds for each event on condition variable ?
In UnlockedWaitForEvent(), for example, you wait the prescribed time each time through the do-while loop. ___ Rob (Sent from my portable computation engine)
On February 16, 2015 11:00:25 PM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 17:04:34 Rob Stewart wrote:
On February 16, 2015 8:21:01 AM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 06:47:00 Rob Stewart wrote:
The same is possible using select if one opens a pipe or socket to the other process, since closing the other process will close its end of the IPC.
I think sigwait or signalfd are better suited for this.
Those only apply to monitoring signals. The desire here was to wait on multiple things including sync primitives.
I was suggesting a way to monitor a process, as a better alternative to pipes.
Okay, but I was suggesting the idea of doing that together with waiting on synchronization primitives, which is the domain of WFMO().
However, I feel that in such cases specialized solutions or Boost.ASIO would play better.
Those would only apply to files, sockets, etc.
Boost.ASIO has support for any fd-based entities, and also can be used to invoke functions in the working threads. This covers pretty much everything you may need in order to synchronize with multiple event sources.
It doesn't cover sync primitives. ___ Rob (Sent from my portable computation engine)
Ok guys after small debugging session it work's, I attached working code, maybe someone want's to use it. But I see that I started really interesting conversation. It is really hard to port all funtionality of WFMO because you can put as parameters: Change notification, Console input, Event, Memory resource notification, Mutex, Process, Semaphore, Thread, Waitable timer. So we can create unified API but that kind of flexibility always provide performance penalty. I am not sure if something equivalent in Unix can be implemented on low level. 2015-02-17 10:29 GMT+01:00 Rob Stewart <rob.stewart@verizon.net>:
On February 16, 2015 11:00:25 PM EST, Andrey Semashev < andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 17:04:34 Rob Stewart wrote:
On February 16, 2015 8:21:01 AM EST, Andrey Semashev <andrey.semashev@gmail.com> wrote:
On Monday 16 February 2015 06:47:00 Rob Stewart wrote:
The same is possible using select if one opens a pipe or socket to the other process, since closing the other process will close its end of the IPC.
I think sigwait or signalfd are better suited for this.
Those only apply to monitoring signals. The desire here was to wait on multiple things including sync primitives.
I was suggesting a way to monitor a process, as a better alternative to pipes.
Okay, but I was suggesting the idea of doing that together with waiting on synchronization primitives, which is the domain of WFMO().
However, I feel that in such cases specialized solutions or Boost.ASIO would play better.
Those would only apply to files, sockets, etc.
Boost.ASIO has support for any fd-based entities, and also can be used to invoke functions in the working threads. This covers pretty much everything you may need in order to synchronize with multiple event sources.
It doesn't cover sync primitives.
___ Rob
(Sent from my portable computation engine)
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 17 Feb 2015 at 10:41, Fu ji wrote:
But I see that I started really interesting conversation. It is really hard to port all funtionality of WFMO because you can put as parameters:
Change notification, Console input, Event, Memory resource notification, Mutex, Process, Semaphore, Thread, Waitable timer.
So we can create unified API but that kind of flexibility always provide performance penalty. I am not sure if something equivalent in Unix can be implemented on low level.
BSD and OS X implements that via kqueues. Linux is actually the odd man out here, epoll just isn't as useful. I believe Boost.Thread v5 intends to implement a unified wait composure mechanism atop the C++ 11 STL which could also be hooked by a Fiber implementation. Indeed, proposed Boost.Fiber I assume does already implement a unified wait composure system for Fiber wait objects. The real bear in all these unified wait composure systems is merging i/o waits into the same unified wait composure system. ASIO can be persuaded via async_result, AFIO will get support in v1.4, but in the end most C++ is still using iostreams etc. That means if you block on i/o, your Fibers and composed waits all block too for that thread until the i/o returns. Probably that means someone would have to reimplement iostreams using an async engine under the bonnet. Good luck finding commercial funding for that. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
participants (5)
-
Andrey Semashev -
Fu ji -
Niall Douglas -
Rob Stewart -
Tim Blechmann