boost thread strange behaviour
hi , i am trying to learn executors and executors thread pools using with boost::async but i found strange behabiour when i make thread pool adaptor ea and submit tasks to it and make ea00 and submit tasks to it if i started ea with async , it automatically starts ea00 the attached file has the code i thought that async will only start the passed thread pool"adaptor" But i think there is underlying thing which gather all threads of different pools into single entity and starts them with any call to boost::async is this a bug??? i run this from windows 7 64 by visual studio 2017
On Sat, 29 Feb 2020 at 08:09, Adham Sabry via Boost
hi , i am trying to learn executors and executors thread pools using with boost::async
I like this (HO) Concurrency.TS implementation: https://stlab.cc/ (Adobe/Sean Parent/Felix Petriconi). The release has at least one bug, which is fixed in master (1.6), 2.0 is on the way, I am guessing it's a milestone. degski -- @realdegski https://brave.com/google-gdpr-workaround/ "We value your privacy, click here!" Sod off! - degski "Anyone who believes that exponential growth can go on forever in a finite world is either a madman or an economist" - Kenneth E. Boulding "Growth for the sake of growth is the ideology of the cancer cell" - Edward P. Abbey
thanks .i have already studied boost thread and i just want to make sure
that this is bug so as to avoid using boost::async with thread pool
On Sat, Feb 29, 2020 at 4:45 PM degski via Boost
On Sat, 29 Feb 2020 at 08:09, Adham Sabry via Boost
wrote:
hi , i am trying to learn executors and executors thread pools using with boost::async
I like this (HO) Concurrency.TS implementation: https://stlab.cc/ (Adobe/Sean https://stlab.cc/(Adobe/Sean Parent/Felix Petriconi).
The release has at least one bug, which is fixed in master (1.6), 2.0 is on the way, I am guessing it's a milestone.
degski -- @realdegski https://brave.com/google-gdpr-workaround/ "We value your privacy, click here!" Sod off! - degski "Anyone who believes that exponential growth can go on forever in a finite world is either a madman or an economist" - Kenneth E. Boulding "Growth for the sake of growth is the ideology of the cancer cell" - Edward P. Abbey
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On Sat, 29 Feb 2020 at 09:51, Adham Sabry via Boost
thanks .i have already studied boost thread and i just want to make sure that this is bug so as to avoid using boost::async with thread pool
The library is a prototype of things that expectedly will be part of C++23, so by the time C++23 arrives (the library tracks the developing std, so by the time its frozen, it IS the std), all you need to do, is change the namespace and the 'include', and delete the 'old' lib (headers). degski -- @realdegski https://brave.com/google-gdpr-workaround/ "We value your privacy, click here!" Sod off! - degski "Anyone who believes that exponential growth can go on forever in a finite world is either a madman or an economist" - Kenneth E. Boulding "Growth for the sake of growth is the ideology of the cancer cell" - Edward P. Abbey
On 29/02/2020 23:22, Adham Sabry wrote:
i am trying to learn executors and executors thread pools using with boost::async but i found strange behabiour when i make thread pool adaptor ea and submit tasks to it and make ea00 and submit tasks to it if i started ea with async , it automatically starts ea00 the attached file has the code i thought that async will only start the passed thread pool"adaptor" But i think there is underlying thing which gather all threads of different pools into single entity and starts them with any call to boost::async is this a bug??? i run this from windows 7 64 by visual studio 2017
Perhaps I'm missing something, since I haven't played with Boost executors before. But this looks entirely normal to me. Creating a basic_thread_pool will presumably start some background threads that are sleeping on work. When you submit_some(ea00) it queues some work on ea00 that will execute at some unspecified later time (but typically relatively soon), whenever one of the sleeping threads in ea00 wakes up. Your use of boost::async(ea, &f1) is entirely unrelated to this -- except that it causes the test_executor_adaptor() thread to pause long enough that you're not destroying the ea00 thread pool before it gets around to executing the work. You can see this by commenting out the boost::async and just sleeping instead; the submitted work gets called in the background. There is no "gathering from multiple pools into a single entity". Each thread pool is entirely separate at all times.
thanks for your reply where can i find the duration of sleep that basic_thread_pool does then wakes up???it sleeps till some methods like submit are called.i thought submit just make tasks without starting the thread and async is the one which launch thread but i was totally wrong drived by asio paradigm what is the reason of wake up??is it timed internally or it needs explicit launching?we can do both using schedulers although i do not know how also why async leads to pausing test_executor_adaptor???here async return is assigned to variable t2 so it will not block in async destructor,but will work asynchronously till the scope is ended and then block in destructor of async???this i do not know answer to it????why async pause ??? what do you mean by sleeping?you meant to sleep between submit calls so they are finished thanks again you helped me in 3 years misunderstood behavior On Wed, Mar 4, 2020 at 8:25 AM Gavin Lambert via Boost < boost@lists.boost.org> wrote:
On 29/02/2020 23:22, Adham Sabry wrote:
i am trying to learn executors and executors thread pools using with boost::async but i found strange behabiour when i make thread pool adaptor ea and submit tasks to it and make ea00 and submit tasks to it if i started ea with async , it automatically starts ea00 the attached file has the code i thought that async will only start the passed thread pool"adaptor" But i think there is underlying thing which gather all threads of different pools into single entity and starts them with any call to boost::async is this a bug??? i run this from windows 7 64 by visual studio 2017
Perhaps I'm missing something, since I haven't played with Boost executors before. But this looks entirely normal to me.
Creating a basic_thread_pool will presumably start some background threads that are sleeping on work.
When you submit_some(ea00) it queues some work on ea00 that will execute at some unspecified later time (but typically relatively soon), whenever one of the sleeping threads in ea00 wakes up.
Your use of boost::async(ea, &f1) is entirely unrelated to this -- except that it causes the test_executor_adaptor() thread to pause long enough that you're not destroying the ea00 thread pool before it gets around to executing the work.
You can see this by commenting out the boost::async and just sleeping instead; the submitted work gets called in the background.
There is no "gathering from multiple pools into a single entity". Each thread pool is entirely separate at all times.
_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost
On 6/03/2020 22:47, Adham Sabry wrote:
where can i find the duration of sleep that basic_thread_pool does then wakes up???it sleeps till some methods like submit are called.i thought submit just make tasks without starting the thread and async is the one which launch thread but i was totally wrong drived by asio paradigm what is the reason of wake up??is it timed internally or it needs explicit launching?we can do both using schedulers although i do not know how also why async leads to pausing test_executor_adaptor???here async return is assigned to variable t2 so it will not block in async destructor,but will work asynchronously till the scope is ended and then block in destructor of async???this i do not know answer to it????why async pause ??? what do you mean by sleeping?you meant to sleep between submit calls so they are finished
There is no specific time. The threadpool threads are asleep waiting for work to be submitted. As soon as you actually submit work (whether directly via submit() or indirectly via async()), the thread is eligible to wake up (in fact it might wake up and complete the work before submit even returns) -- that part is entirely up to your OS's thread scheduler. If it helps: async() is essentially internally just a call to submit() with some additional wrapping to give you a future representing the completion of the operation. [This is not precisely accurate but it's a good way to think about it for understanding purposes.] My point about the async calls is that because you're calling async and then future.get(), you're blocking the original thread waiting for something else, which allows the threadpool work to complete in the background, regardless of which threadpool the work was submitted to. If you completely removed the future.get() calls (even if you still called async), then you might end up destroying the threadpool before it actually did the work, thus it would appear that submit() did nothing. (And the asyncs would also end up doing nothing.) But because like I said this is up to your OS's thread scheduler, it might sometimes make some progress on some of the work before actually being destroyed, which can be confusing. If you have a sleep (or any other work running long enough) in there, so that you don't destroy the threadpool too soon, then it will complete in the background. Usually you need to structure your code in a way that assures that the things you want to complete have actually completed before you destroy them. Waiting on futures is an easy way to do that in a toy example, but it can get more complex in a real application. (I personally consider futures almost entirely useless in a real application -- but that doesn't make learning about them useless.) I'd recommend finding a good book on the topic to read. (Though books specifically on executors are probably rare, as it's relatively new ground.) Threading and async in C++ can be very hard to get correct unless you have the fundamentals down strongly.
Mere moments ago, quoth I:
My point about the async calls is that because you're calling async and then future.get(), you're blocking the original thread waiting for something else, which allows the threadpool work to complete in the background, regardless of which threadpool the work was submitted to.
If you completely removed the future.get() calls (even if you still called async), then you might end up destroying the threadpool before it actually did the work, thus it would appear that submit() did nothing. (And the asyncs would also end up doing nothing.) But because like I said this is up to your OS's thread scheduler, it might sometimes make some progress on some of the work before actually being destroyed, which can be confusing.
If you have a sleep (or any other work running long enough) in there, so that you don't destroy the threadpool too soon, then it will complete in the background.
For another example, if you submitted some long-running tasks to ea and some short-running tasks to ea00, and then only waited on the futures from ea00, then they'd complete quickly and you'd exit those blocks and destroy both threadpools -- but ea's work is very likely to be incomplete or not even started yet. Conversely, submitting the same tasks to the same pools but instead waiting on futures from ea instead would increase your chances that both pools completed their work but it's still not entirely impossible that only ea finished and ea00 still had some unfinished work. (You'd have to have quite poor luck with the scheduler though.) Finally, submitting a task to ea, then async'ing a task also to ea, then waiting on that second task's future: this would guarantee that both tasks finished *only if there was exactly one thread in the pool*. If there's more than one thread then you could still get a situation where the second task finishes before the first. If you want to guarantee that everything completes before you stop then you need to wait for completion of everything in some fashion or another.
participants (3)
-
Adham Sabry
-
degski
-
Gavin Lambert