I'm looking at the boost.fiber scheduling customization options. I have a few comments.
My first concern is that scheduler is a global (per thread) property. This makes sense, but it limits its usability in libraries, unless the library completely owns a thread. It would be nice if schedulers where schedulable entities themselves, so that they could be nested (more at the end).
Also the description of the scheduler interface does not specify any thread safety requirements. I assume that at least awakened must be thread safe as the scheduling might be caused by a signal coming from another thread. Any requirements should be specified properly. This leads to two additional points.
First of all, there should be a way to retrieve the scheduler associated with a fiber: I haven't looked at the source, but the association must exist internally, otherwise cross thread scheduling wouldn't work.
Second, there does not seem to be a way to allow signaled fibers to run in the context of the signaling thread. This seems an important optimization.I understand this is an explicit decision as currently it is not possible to portably migrate fibers, but the option should be left to the user if he knows it is safe in their setup. Possibly require 'awakened(fiber*x)' to call x->get_scheduler()->awakened(x) if it does not support running the fiber. My preference is to add a queryable boolean flag to each schedulable entity that states whether it is allowed to move from its scheduler (this makes a difference when you have multiple nested schedulers). Among other things this would also prevent work stealing. The flag would by default be set to prevent migration of course.
Now, what do nested schedulers give you? In addition to composability, you can have the equivalent of an asio strand without explicit mutual exclusion. Let say you have a bunch of fibers that all access the same resource; you do not care where they run, as long as they never run concurrently. By binding all of them to the same scheduler, the guarantee is implicit.
It should be possible to implement nested schedulers by having each fiber set the current scheduler pointer if it differs and store the previous one. On a scheduling event, if the current scheduler runs out of runnables, the signal/yield code would restore the previous scheduler and try again. Of course handling priorities and non strict fifo scheduling might be more complex. Note that the multi scheduler logic need to be implemented by Boost.Fiber itself.
Finally, it might be possible (but I haven't tested this) to make fibers migrable on platforms where it normally isn't safe by allocating an actual thread for each one. The thread would be stopped as soon as it has been created, but a thread control block plush TLS entries would have been allocated. The existing context switch would need to be augmented with the ability to swap the thread context pointer.
+1 to everything! Regards Hartmut --------------- http://boost-spirit.com http://stellar.cct.lsu.edu