boost.spawn: review requested

Oliver Kowalke

28 Nov 2021 28 Nov '21

6:49 a.m.

Hi, I'd like to propose the new library boost.spawn (https://olk.github.io/) for review. boost.asio's boost::asio::spawn() uses the deprecated and inefficient boost.coroutine library. boost.spawn provides the same functionality while using boost.context's fiber directly. Oliver

Show replies by date

Andrzej Krzemienski

28 Nov 28 Nov

12:18 p.m.

Hi Oliver, Thank you for the submission. Some quick high-level feedback, after reading the docs. Is Boost.Spawn intended to be useful on its own, or is it only usable in Boost.ASIO? The provided link points to the docs. Is there an associated source code? Or are you just checking if there is sufficient interest? The "Effects" clause of the third overload has " on behalf of spawned spawn", which is supposed to be " on behalf of spawned fiber", I guess. Regards, &rzej; niedz., 28 lis 2021 o 07:50 Oliver Kowalke via Boost napisał(a):

...

Hi, I'd like to propose the new library boost.spawn (https://olk.github.io/) for review. boost.asio's boost::asio::spawn() uses the deprecated and inefficient boost.coroutine library. boost.spawn provides the same functionality while using boost.context's fiber directly.

Oliver

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Oliver Kowalke

1:27 p.m.

...

Is Boost.Spawn intended to be useful on its own, or is it only usable in Boost.ASIO?

It can only be used with boost .asio (or with the forthcoming C++ std::io_context). The lib is intended to replace the boost::asio:::spawn().

...

The provided link points to the docs. Is there an associated source code? Or are you just checking if there is sufficient interest?

Oh, I forgot the link: https://github.com/olk/spawn

...

The "Effects" clause of the third overload has " on behalf of spawned spawn", which is supposed to be " on behalf of spawned fiber", I guess.

right regards, Oliver

Andrey Semashev

4:34 p.m.

On 11/28/21 16:27, Oliver Kowalke via Boost wrote:

...

...
Is Boost.Spawn intended to be useful on its own, or is it only usable in Boost.ASIO?

It can only be used with boost .asio (or with the forthcoming C++ std::io_context). The lib is intended to replace the boost::asio:::spawn().

In this case why should it be a separate library and not just part of Boost.ASIO? What would be the value for Boost users or other Boost libraries?

Oliver Kowalke

4:50 p.m.

Am So., 28. Nov. 2021 um 17:34 Uhr schrieb Andrey Semashev via Boost < boost@lists.boost.org>:

...

In this case why should it be a separate library and not just part of Boost.ASIO?

I tried to pass a fix to boost.asio that was not taken up by the author and unfortunately no response...

...

What would be the value for Boost users or other Boost libraries?

spawn_fiber() is more efficient than boost::asio::spawn() spawn_fiber() is prepared to work with the forthcoming std::io_context async. operations are written as synchronous code.

Richard Hodges

4:53 p.m.

On Sun, 28 Nov 2021 at 17:34, Andrey Semashev via Boost < boost@lists.boost.org> wrote:

...

On 11/28/21 16:27, Oliver Kowalke via Boost wrote:

...
...
Is Boost.Spawn intended to be useful on its own, or is it only usable in Boost.ASIO?

It can only be used with boost .asio (or with the forthcoming C++ std::io_context). The lib is intended to replace the boost::asio:::spawn().

In this case why should it be a separate library and not just part of Boost.ASIO? What would be the value for Boost users or other Boost libraries?

Getting code into Asio would require Chris Kohlhoff's buy-in.

...

I tried to pass a fix to boost.asio that was not taken up by the author and unfortunately no response...

You need to submit the PR here: https://github.com/chriskohlhoff/asio/

...

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Oliver Kowalke

5:17 p.m.

Am So., 28. Nov. 2021 um 17:53 Uhr schrieb Richard Hodges via Boost < boost@lists.boost.org>:

...

On Sun, 28 Nov 2021 at 17:34, Andrey Semashev via Boost < boost@lists.boost.org> wrote:

...
On 11/28/21 16:27, Oliver Kowalke via Boost wrote:

...
...
Is Boost.Spawn intended to be useful on its own, or is it only usable

in

...
Boost.ASIO?

It can only be used with boost .asio (or with the forthcoming C++ std::io_context). The lib is intended to replace the boost::asio:::spawn().

In this case why should it be a separate library and not just part of Boost.ASIO? What would be the value for Boost users or other Boost libraries?

Getting code into Asio would require Chris Kohlhoff's buy-in.

...
I tried to pass a fix to boost.asio that was not taken up by the author and unfortunately no response...

You need to submit the PR here:

https://github.com/chriskohlhoff/asio/

I wrote an email... unfortunately no response

Andrey Semashev

5:17 p.m.

On 11/28/21 19:53, Richard Hodges via Boost wrote:

...

On Sun, 28 Nov 2021 at 17:34, Andrey Semashev via Boost < boost@lists.boost.org> wrote:

...
On 11/28/21 16:27, Oliver Kowalke via Boost wrote:

...
...
Is Boost.Spawn intended to be useful on its own, or is it only usable in Boost.ASIO?

It can only be used with boost .asio (or with the forthcoming C++ std::io_context). The lib is intended to replace the boost::asio:::spawn().

In this case why should it be a separate library and not just part of Boost.ASIO? What would be the value for Boost users or other Boost libraries?

Getting code into Asio would require Chris Kohlhoff's buy-in.

Using this library in Boost.ASIO also requires Chris' cooperation, doesn't it? And if this library is not useful outside Boost.ASIO and Chris is not interested, I just don't see the point.

Oliver Kowalke

5:23 p.m.

Am So., 28. Nov. 2021 um 18:18 Uhr schrieb Andrey Semashev via Boost < boost@lists.boost.org>:

...

On 11/28/21 19:53, Richard Hodges via Boost wrote:

...
On Sun, 28 Nov 2021 at 17:34, Andrey Semashev via Boost < boost@lists.boost.org> wrote:

...
On 11/28/21 16:27, Oliver Kowalke via Boost wrote:

...
...
Is Boost.Spawn intended to be useful on its own, or is it only usable

in

...
Boost.ASIO?

It can only be used with boost .asio (or with the forthcoming C++ std::io_context). The lib is intended to replace the boost::asio:::spawn().

In this case why should it be a separate library and not just part of Boost.ASIO? What would be the value for Boost users or other Boost libraries?

Getting code into Asio would require Chris Kohlhoff's buy-in.

Using this library in Boost.ASIO also requires Chris' cooperation, doesn't it?

no response to email

...

And if this library is not useful outside Boost.ASIO and Chris is not interested, I just don't see the point.

Don't get it ... 'not useful outside boost.asio' ... You can create asynchronous code that looks synchronous on behalf of boost::asio::io_context or the forthcoming C++ std::io_context.

Oliver Kowalke

5:30 p.m.

Am So., 28. Nov. 2021 um 18:18 Uhr schrieb Andrey Semashev via Boost < boost@lists.boost.org>:

...

And if this library is not useful outside Boost.ASIO and Chris is not interested, I just don't see the point.

another point: boost.asio still uses boost.coroutine which is deprecated and I really want to remove it from boost but is not possible because of boost::asio::spawn() boost.spawn is the alternative to asio's spawn() (which could be removed from boost.asio in favour of boost.spawn)

Richard Hodges

5:36 p.m.

On Sun, 28 Nov 2021 at 18:30, Oliver Kowalke via Boost < boost@lists.boost.org> wrote:

...

Am So., 28. Nov. 2021 um 18:18 Uhr schrieb Andrey Semashev via Boost < boost@lists.boost.org>:

...
And if this library is not useful outside Boost.ASIO and Chris is not interested, I just don't see the point.

another point: boost.asio still uses boost.coroutine which is deprecated and I really want to remove it from boost but is not possible because of boost::asio::spawn() boost.spawn is the alternative to asio's spawn() (which could be removed from boost.asio in favour of boost.spawn)

Asio's spawn is extremely popular and in use in a number of code bases. I really don't think removing from boost is a sensible option unless there is a clear, simple replacement that works for C++03.

...

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Oliver Kowalke

6:18 p.m.

...

Asio's spawn is extremely popular and in use in a number of code bases.

I know but it still uses the deprecated boost.coroutine

...

I really don't think removing from boost is a sensible option unless there is a clear, simple replacement that works for C++03.

boost.spawn can be extended to support C++03 too

Robert Ramey

7:49 p.m.

On 11/28/21 9:30 AM, Oliver Kowalke via Boost wrote:

...

Am So., 28. Nov. 2021 um 18:18 Uhr schrieb Andrey Semashev via Boost < boost@lists.boost.org>:

...
And if this library is not useful outside Boost.ASIO and Chris is not interested, I just don't see the point.

another point: boost.asio still uses boost.coroutine which is deprecated and I really want to remove it from boost but is not possible because of boost::asio::spawn() boost.spawn is the alternative to asio's spawn() (which could be removed from boost.asio in favour of boost.spawn)

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Couldn't one achieve the desired effect by just updating the implementation of boost.coroutine with the implementation of boost.spawn? Robert Ramey

Oliver Kowalke

29 Nov 29 Nov

5:30 a.m.

Robert Ramey via Boost schrieb am So., 28. Nov. 2021, 20:54:

...

Couldn't one achieve the desired effect by just updating the implementation of boost.coroutine with the implementation of boost.spawn?

No, I've already provided boost.coroutine2 - that's why boost.context was deprecated. Unfortunately we can't remove it form the boost collection because boost.asio has a dependency to boost.coroutine. The coroutine libraries are too much heavyweight for the purpose of spawn() (for instance push_type and pull_type etc.). boost.spawn uses boost.context's fiber directly - without the boilerplate code required by coroutine libs (take a look at boost/spawn/impl/spawn.hpp).

Niall Douglas

2:49 p.m.

On 28/11/2021 13:27, Oliver Kowalke via Boost wrote:

...

...
Is Boost.Spawn intended to be useful on its own, or is it only usable in Boost.ASIO?

It can only be used with boost .asio (or with the forthcoming C++ std::io_context). The lib is intended to replace the boost::asio:::spawn().

For this to be valid as a standalone Boost library, in my opinion it needs to also support at the very least C++ coroutines if on C++ 20. (FYI std::io_context is currently up in the air at WG21, something quite different to what ASIO does may be coming instead) Niall

Oliver Kowalke

3:03 p.m.

Am Mo., 29. Nov. 2021 um 15:49 Uhr schrieb Niall Douglas via Boost < boost@lists.boost.org>:

...

For this to be valid as a standalone Boost library, in my opinion it needs to also support at the very least C++ coroutines if on C++ 20.

Sorry, but I think this does not make sense because boost::asio::spawn() a boost.spawn do only support stackful context switching. It was long discussed in the C++ standardisation meetings if a unified syntax for stackless and stackful 'coroutines' is possible - the outcome was, no its impossible (think on function coloring).

Niall Douglas

4:02 p.m.

On 29/11/2021 15:03, Oliver Kowalke via Boost wrote:

...

Am Mo., 29. Nov. 2021 um 15:49 Uhr schrieb Niall Douglas via Boost < boost@lists.boost.org>:

...
For this to be valid as a standalone Boost library, in my opinion it needs to also support at the very least C++ coroutines if on C++ 20.

Sorry, but I think this does not make sense because boost::asio::spawn() a boost.spawn do only support stackful context switching. It was long discussed in the C++ standardisation meetings if a unified syntax for stackless and stackful 'coroutines' is possible - the outcome was, no its impossible (think on function coloring).

Y'see, I'd look at this as an opportunity to extend C++ Coroutines with stackfulness emulation, and that would be exactly the value in a standalone Boost library here. But otherwise I gotta agree with Vinnie here, if you require Boost.Context and Boost.ASIO and you only work with those and Chris K isn't keen on adding this into Boost.ASIO, the next most logical place for something as small as this is Boost.Context surely? Niall

Peter Dimov

28 Nov 28 Nov

3:36 p.m.

Oliver Kowalke wrote:

...

Hi, I'd like to propose the new library boost.spawn (https://olk.github.io/) for review. boost.asio's boost::asio::spawn() uses the deprecated and inefficient boost.coroutine library. boost.spawn provides the same functionality while using boost.context's fiber directly.

The documentation of spawn_fiber often says "must have signature void(basic_yield_context<Handler>)" on signatures where there's no Handler.

Oliver Kowalke

4:47 p.m.

Am So., 28. Nov. 2021 um 16:36 Uhr schrieb Peter Dimov via Boost < boost@lists.boost.org>:

...

The documentation of spawn_fiber often says "must have signature void(basic_yield_context<Handler>)" on signatures where there's no Handler.

void do_echo(boost::spawn::yield_context yield) { try { char data[128]; for (;;) { std::size_t length = my_socket.async_read_some(boost::asio::buffer(data), yield); boost::asio::async_write(my_socket, boost::asio::buffer(data, length), yield); } } catch (std::exception const& e) { // ... }}boost::spawn_fiber(my_strand, do_echo); do_echo() is the handler that is passed to spawn_fiber()

Boris Kolpackov

29 Nov 29 Nov

2:54 p.m.

Oliver Kowalke via Boost writes:

...

I'd like to propose the new library boost.spawn [...]

FWIW (and I may be out of context), but "spawn" seems like a too generic of a name. I, for example, thought this is about posix_spawn(). Maybe call it spawn_fiber or some such?

Andrey Semashev

6:52 p.m.

On 11/29/21 17:54, Boris Kolpackov via Boost wrote:

...

Oliver Kowalke via Boost writes:

...
I'd like to propose the new library boost.spawn [...]

FWIW (and I may be out of context), but "spawn" seems like a too generic of a name. I, for example, thought this is about posix_spawn(). Maybe call it spawn_fiber or some such?

Vinnie Falco

7:02 p.m.

On Mon, Nov 29, 2021 at 10:52 AM Andrey Semashev via Boost wrote:

...

...
"spawn" seems like a too generic of a name.

How about Boost.Asio.Spawn? This is more correcteror. Thanks

Oliver Kowalke

7:22 p.m.

Vinnie Falco via Boost schrieb am Mo., 29. Nov. 2021, 20:02:

...

On Mon, Nov 29, 2021 at 10:52 AM Andrey Semashev via Boost wrote:

...
...
"spawn" seems like a too generic of a name.

How about Boost.Asio.Spawn? This is more correcteror.

I think he means the function name.

...

Vinnie Falco

7:31 p.m.

On Mon, Nov 29, 2021 at 11:23 AM Oliver Kowalke via Boost wrote:

...

I think he means the function name.

We shouldn't use types from Asio's detail namespace: https://github.com/olk/spawn/blob/216f23a11bf4fe854c105fa2eadfb0ba998529bc/i... Regards

Oliver Kowalke

8:04 p.m.

Am Mo., 29. Nov. 2021 um 20:31 Uhr schrieb Vinnie Falco via Boost < boost@lists.boost.org>:

...

On Mon, Nov 29, 2021 at 11:23 AM Oliver Kowalke via Boost wrote:

...
I think he means the function name.

We shouldn't use types from Asio's detail namespace:

< https://github.com/olk/spawn/blob/216f23a11bf4fe854c105fa2eadfb0ba998529bc/i...

...

Which one? yield_context and basic_yield_context reside in namespace boost::spawn?

Vinnie Falco

8:10 p.m.

On Mon, Nov 29, 2021 at 12:05 PM Oliver Kowalke via Boost wrote:

...

Which one? yield_context and basic_yield_context reside in namespace boost::spawn?

I misread, detail::net::executor is yours not asio's (via detail/net.hpp) Thanks

Oliver Kowalke

7:21 p.m.

Andrey Semashev via Boost schrieb am Mo., 29. Nov. 2021, 19:52:

...

On 11/29/21 17:54, Boris Kolpackov via Boost wrote:

...
Oliver Kowalke via Boost writes:

...
I'd like to propose the new library boost.spawn [...]

FWIW (and I may be out of context), but "spawn" seems like a too generic of a name. I, for example, thought this is about posix_spawn(). Maybe call it spawn_fiber or some such?

+1

As I answered posix_spawn() is wrong because the lib supports Windows too. Boost.spawn already names the function spawn_fiber() ... I don't get your point or what I've to change.

...

Vinnie Falco

3:56 p.m.

On Sat, Nov 27, 2021 at 10:50 PM Oliver Kowalke via Boost wrote:

...

I'd like to propose the new library boost.spawn (https://olk.github.io/) for review. boost.asio's boost::asio::spawn() uses the deprecated and inefficient boost.coroutine library. boost.spawn provides the same functionality while using boost.context's fiber directly.

I like the direction this library takes, and I have a use for something like this myself in Boost.JSON. Specifically, to implement a generator which does pull parsing of JSON. This will make the implementation of parsing directly into structs more simple and robust. I also like the simplification over push/pull, which never really made sense to me. However, this library should not depend on Asio. The fiber functionality provided by boost.spawn is useful enough that people will want to use it in more situations than just Asio. What will standalone Asio use? It is worth considering a standalone version of this library (or perhaps just not depend on any other Boost libraries). Have you considered just rolling this into Boost.Context? That library is sort of not particularly useful on its own. I would like to see an example of a generator using boost.spawn. A simple fibonacci producer would suffice. This library also needs more examples that don't require Asio. Thanks

Oliver Kowalke

4:03 p.m.

Am Mo., 29. Nov. 2021 um 16:56 Uhr schrieb Vinnie Falco via Boost < boost@lists.boost.org>:

...

However, this library should not depend on Asio. The fiber functionality provided by boost.spawn is useful enough that people will want to use it in more situations than just Asio. What will standalone Asio use? It is worth considering a standalone version of this library (or perhaps just not depend on any other Boost libraries).

I'm confused by the request for a standalone version. The spawned fibers must be scheduled for execution. boost.fiber does this without the need of boost.asio. boost.spawn itself relies on boost::asio's io_context (demultiplexer, event scheduler) and the async result/handler framework.

...

Have you considered just rolling this into Boost.Context?

Yes, I had a commit that has been reverted because of dependency cycles between the boost libs.

...

That library is sort of not particularly useful on its own.

boost.context must be considered as a foundation library for other high-level constructs (like boost.coroutine2 or boost.fiber).

...

I would like to see an example of a generator using boost.spawn. A simple fibonacci producer would suffice. This library also needs more examples that don't require Asio.

I think this is a misunderstanding of what boost.spawn is intended...

Vinnie Falco

4:29 p.m.

On Mon, Nov 29, 2021 at 8:09 AM Oliver Kowalke via Boost wrote:

...

I think this is a misunderstanding of what boost.spawn is intended...

Upon reviewing Boost.Coroutine2, Boost.Fiber, and taking another look at Boost.Spawn, I believe you are correct. We have a problem with not being able to deprecate Boost.Coroutine because of Asio. The Asio author is silent on pull requests and emails. Boost.Spawn appears to be a mechanism for facilitating progress on this matter, in a fashion similar to adding an improved string_view to Boost.Utility. In light of this, I endorse Boost.Spawn for review. Thanks

Glen Fernandes

5:08 p.m.

On Mon, Nov 29, 2021 at 11:30 AM Vinnie Falco wrote:

...

Upon reviewing Boost.Coroutine2, Boost.Fiber, and taking another look at Boost.Spawn, I believe you are correct.

We have a problem with not being able to deprecate Boost.Coroutine because of Asio. The Asio author is silent on pull requests and emails. Boost.Spawn appears to be a mechanism for facilitating progress on this matter, in a fashion similar to adding an improved string_view to Boost.Utility.

In light of this, I endorse Boost.Spawn for review.

Boost.Coroutine is deprecated, but cannot be removed because of Asio. The discussion above implies that Boost.Spawn allows Boost.Coroutine to be removed - but I don't understand how because Boost.Asio still depends on Boost.Coroutine And Boost.Asio cannot depend on Boost.Spawn since the latter depends on Boost.Asio, no? According to Oliver's e-mail above Spawn uses io_context from Asio, and that would result in a dependency cycle. What am I missing? Glen

Oliver Kowalke

5:16 p.m.

Am Mo., 29. Nov. 2021 um 18:09 Uhr schrieb Glen Fernandes via Boost < boost@lists.boost.org>:

...

Boost.Coroutine is deprecated, but cannot be removed because of Asio. The discussion above implies that Boost.Spawn allows Boost.Coroutine to be removed - but I don't understand how because Boost.Asio still depends on Boost.Coroutine

My suggestion is that we convince Chris to remove boost::asio::spawn and add a notice/link to boost.spawn in the asio documentation.

...

And Boost.Asio cannot depend on Boost.Spawn since the latter depends on Boost.Asio, no? According to Oliver's e-mail above Spawn uses io_context from Asio, and that would result in a dependency cycle.

What am I missing?

I previously made a commit in boost.context that implemented spawn_fiber() (but introduced a dependency to boost.asio because of io_context). Via boost::asio::spawn() boost.asio has itself a dependency to boost.coroutine. Boost.coroutine depends on boost.context as a foundation for context switching.

Glen Fernandes

5:41 p.m.

On Mon, Nov 29, 2021 at 12:17 PM Oliver Kowalke wrote:

...

Am Mo., 29. Nov. 2021 um 18:09 Uhr schrieb Glen Fernandes wrote:

...
Boost.Coroutine is deprecated, but cannot be removed because of Asio. The discussion above implies that Boost.Spawn allows Boost.Coroutine to be removed - but I don't understand how because Boost.Asio still depends on Boost.Coroutine

My suggestion is that we convince Chris to remove boost::asio::spawn and add a notice/link to boost.spawn in the asio documentation.

Understood - thanks. Glen

Niall Douglas

8:30 p.m.

On 29/11/2021 16:29, Vinnie Falco via Boost wrote:

...

On Mon, Nov 29, 2021 at 8:09 AM Oliver Kowalke via Boost wrote:

...
I think this is a misunderstanding of what boost.spawn is intended...

Upon reviewing Boost.Coroutine2, Boost.Fiber, and taking another look at Boost.Spawn, I believe you are correct.

We have a problem with not being able to deprecate Boost.Coroutine because of Asio. The Asio author is silent on pull requests and emails. Boost.Spawn appears to be a mechanism for facilitating progress on this matter, in a fashion similar to adding an improved string_view to Boost.Utility.

In light of this, I endorse Boost.Spawn for review.

It seems to me unwise to create standalone libraries just to shim around a process and coordination problem. As an alternative, how about announcing the complete removal of Boost.Coroutine with two major release notice (including this release)? That should spur Boost.ASIO being upgraded to Boost.Coroutine2. Niall

Oliver Kowalke

8:38 p.m.

Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost < boost@lists.boost.org>:

...

As an alternative, how about announcing the complete removal of Boost.Coroutine with two major release notice (including this release)? That should spur Boost.ASIO being upgraded to Boost.Coroutine2.

That had already happen some years ago - including warnings during compilations - but didn't help. Ask Vini... At the end I was forced to remove the compiler (preprocessor) warnings.

Niall Douglas

8:47 p.m.

On 29/11/2021 20:38, Oliver Kowalke via Boost wrote:

...

Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost < boost@lists.boost.org>:

...
As an alternative, how about announcing the complete removal of Boost.Coroutine with two major release notice (including this release)? That should spur Boost.ASIO being upgraded to Boost.Coroutine2.

That had already happen some years ago - including warnings during compilations - but didn't help. Ask Vini... At the end I was forced to remove the compiler (preprocessor) warnings.

Alas I'm probably one of the very few here who would be absolutely fine with removing completely Boost.Coroutine after two major releases, and whatever didn't fix itself and broke with that sufficient amount of warning well that's on them in my book. I assume there's no way you can shim Coroutine's API into a Coroutine2 implementation, then you can drop Coroutine? Niall

Oliver Kowalke

9:49 p.m.

Niall Douglas via Boost schrieb am Mo., 29. Nov. 2021, 21:47:

...

On 29/11/2021 20:38, Oliver Kowalke via Boost wrote:

...
Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost < boost@lists.boost.org>:

...
As an alternative, how about announcing the complete removal of Boost.Coroutine with two major release notice (including this release)? That should spur Boost.ASIO being upgraded to Boost.Coroutine2.

That had already happen some years ago - including warnings during compilations - but didn't help. Ask Vini... At the end I was forced to remove the compiler (preprocessor) warnings.

Alas I'm probably one of the very few here who would be absolutely fine with removing completely Boost.Coroutine after two major releases, and whatever didn't fix itself and broke with that sufficient amount of warning well that's on them in my book.

I think this not a good idea... boost::asio:::spawn() seams to be often used. I assume there's no way you can shim Coroutine's API into a Coroutine2

...

implementation, then you can drop Coroutine2

...

Unfortunately no, boost.coroutine is C++03 while boost.coroutine2 requires C++11.

Vinnie Falco

10:48 p.m.

On Mon, Nov 29, 2021 at 1:50 PM Oliver Kowalke via Boost wrote:

...

I think this not a good idea... boost::asio:::spawn() seams to be often used.

Yep. Unfortunately, Beast uses it in several examples: https://github.com/boostorg/beast/blob/94e6f143295e38f6e7e7d2f60937d4f76ac1d... Thanks

Andrzej Krzemienski

30 Nov 30 Nov

7:21 p.m.

pon., 29 lis 2021 o 21:39 Oliver Kowalke via Boost napisał(a):

...

Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost < boost@lists.boost.org>:

...
As an alternative, how about announcing the complete removal of Boost.Coroutine with two major release notice (including this release)? That should spur Boost.ASIO being upgraded to Boost.Coroutine2.

That had already happen some years ago - including warnings during compilations - but didn't help. Ask Vini... At the end I was forced to remove the compiler (preprocessor) warnings.

Please correct me if I am wrong, but my understanding is that Boost.ASIO works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2 requires C++11. If Boost.ASIO were to switch from BoostCoroutine to Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my work still have to use C++03.) "Boost.Spawn" could be an addition to Boost.ASIO, but not a replacement, right? Regards, &rzej;

...

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Oliver Kowalke

7:26 p.m.

Andrzej Krzemienski schrieb am Di., 30. Nov. 2021, 20:21:

...

pon., 29 lis 2021 o 21:39 Oliver Kowalke via Boost napisał(a):

...
Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost < boost@lists.boost.org>:

...
As an alternative, how about announcing the complete removal of Boost.Coroutine with two major release notice (including this release)? That should spur Boost.ASIO being upgraded to Boost.Coroutine2.

That had already happen some years ago - including warnings during compilations - but didn't help. Ask Vini... At the end I was forced to remove the compiler (preprocessor) warnings.

Please correct me if I am wrong, but my understanding is that Boost.ASIO works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2 requires C++11. If Boost.ASIO were to switch from BoostCoroutine to Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my work still have to use C++03.)

"Boost.Spawn" could be an addition to Boost.ASIO, but not a replacement, right?

boost.spawn can not replace boost.asio - only it's functionality boost::asio::spawn()

...

Andrey Semashev

8:06 p.m.

On 11/30/21 22:26, Oliver Kowalke via Boost wrote:

...

Andrzej Krzemienski schrieb am Di., 30. Nov. 2021, 20:21:

...
pon., 29 lis 2021 o 21:39 Oliver Kowalke via Boost napisał(a):

...
Am Mo., 29. Nov. 2021 um 21:30 Uhr schrieb Niall Douglas via Boost < boost@lists.boost.org>:

...
As an alternative, how about announcing the complete removal of Boost.Coroutine with two major release notice (including this release)? That should spur Boost.ASIO being upgraded to Boost.Coroutine2.

That had already happen some years ago - including warnings during compilations - but didn't help. Ask Vini... At the end I was forced to remove the compiler (preprocessor) warnings.

Please correct me if I am wrong, but my understanding is that Boost.ASIO works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2 requires C++11. If Boost.ASIO were to switch from BoostCoroutine to Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my work still have to use C++03.)

"Boost.Spawn" could be an addition to Boost.ASIO, but not a replacement, right?

boost.spawn can not replace boost.asio - only it's functionality boost::asio::spawn()

I don't think Andrzej was suggesting that. I think the question is fair - if Boost.Spawn (though I'd really prefer a more specific name) is C++11, how Boost.ASIO can switch to it while still supporting C++03? And if Boost.Spawn is updated to also support C++03, how is it different from Boost.Coroutine?

Peter Dimov

7:28 p.m.

Andrzej Krzemienski wrote:

...

Please correct me if I am wrong, but my understanding is that Boost.ASIO works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2 requires C++11. If Boost.ASIO were to switch from BoostCoroutine to Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my work still have to use C++03.)

It's a bit more complex than that. Boost.Coroutine depends on Boost.Context, which is C++11. However, in order to not break C++03, Coroutine doesn't use the public API of Context, but the private "fcontext" parts that are kept C++03 compatible only because of that. This incidentally means that if you compile Context with the `ucontext` backend instead of the `fcontext` one, Coroutine breaks. So it's basically a mess and needs to be cleaned up. Since asio::spawn is a separate part of Asio, moving just it to C++11 wouldn't be as disruptive a change as requiring C++11 for all of Asio.

Andrey Semashev

8 p.m.

On 11/30/21 22:28, Peter Dimov via Boost wrote:

...

Andrzej Krzemienski wrote:

...
Please correct me if I am wrong, but my understanding is that Boost.ASIO works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2 requires C++11. If Boost.ASIO were to switch from BoostCoroutine to Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my work still have to use C++03.)

It's a bit more complex than that. Boost.Coroutine depends on Boost.Context, which is C++11. However, in order to not break C++03, Coroutine doesn't use the public API of Context, but the private "fcontext" parts that are kept C++03 compatible only because of that. This incidentally means that if you compile Context with the `ucontext` backend instead of the `fcontext` one, Coroutine breaks.

So it's basically a mess and needs to be cleaned up. Since asio::spawn is a separate part of Asio, moving just it to C++11 wouldn't be as disruptive a change as requiring C++11 for all of Asio.

I wonder how much of Boost.Context actually requires C++11. From my admittedly naive perspective, it should mostly be a bunch of assembler bits and/or interface glue for system APIs like ucontext, none of which inherently requires C++11.

Oliver Kowalke

9:40 p.m.

Andrey Semashev via Boost schrieb am Di., 30. Nov. 2021, 21:00:

...

On 11/30/21 22:28, Peter Dimov via Boost wrote:

...
Andrzej Krzemienski wrote:

...
Please correct me if I am wrong, but my understanding is that Boost.ASIO works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2 requires C++11. If Boost.ASIO were to switch from BoostCoroutine to Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my work still have to use C++03.)

It's a bit more complex than that. Boost.Coroutine depends on Boost.Context, which is C++11. However, in order to not break C++03, Coroutine doesn't use the public API of Context, but the private "fcontext" parts that are kept C++03 compatible only because of that. This incidentally means that if you compile Context with the `ucontext` backend instead of the `fcontext` one, Coroutine breaks.

So it's basically a mess and needs to be cleaned up. Since asio::spawn is a separate part of Asio, moving just it to C++11 wouldn't be as disruptive a change as requiring C++11 for all of Asio.

I wonder how much of Boost.Context actually requires C++11. From my admittedly naive perspective, it should mostly be a bunch of assembler bits and/or interface glue for system APIs like ucontext, none of which inherently requires C++11

It's described in the C++ proposal ... lifecycle, stack must be managed + ownership and state (for instance resuming an already running fiber).

Andrey Semashev

1 Dec 1 Dec

12:10 p.m.

On 12/1/21 00:40, Oliver Kowalke via Boost wrote:

...

Andrey Semashev via Boost schrieb am Di., 30. Nov. 2021, 21:00:

...
On 11/30/21 22:28, Peter Dimov via Boost wrote:

...
Andrzej Krzemienski wrote:

...
Please correct me if I am wrong, but my understanding is that Boost.ASIO works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2 requires C++11. If Boost.ASIO were to switch from BoostCoroutine to Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my work still have to use C++03.)

It's a bit more complex than that. Boost.Coroutine depends on Boost.Context, which is C++11. However, in order to not break C++03, Coroutine doesn't use the public API of Context, but the private "fcontext" parts that are kept C++03 compatible only because of that. This incidentally means that if you compile Context with the `ucontext` backend instead of the `fcontext` one, Coroutine breaks.

So it's basically a mess and needs to be cleaned up. Since asio::spawn is a separate part of Asio, moving just it to C++11 wouldn't be as disruptive a change as requiring C++11 for all of Asio.

I wonder how much of Boost.Context actually requires C++11. From my admittedly naive perspective, it should mostly be a bunch of assembler bits and/or interface glue for system APIs like ucontext, none of which inherently requires C++11

It's described in the C++ proposal ... lifecycle, stack must be managed + ownership and state (for instance resuming an already running fiber).

IMHO, fiber execution, and the fiber class itself, should be a separate library, which may require C++11. It was my understanding that this is what Boost.Coroutine and Boost.Coroutine2 are for. Boost.Context, on the other hand, would only contain low level code for saving and restoring thread context and nothing else. This kind functionality doesn't seem like dependent on C++11 features.

Andrzej Krzemienski

8:20 a.m.

wt., 30 lis 2021 o 20:28 Peter Dimov via Boost napisał(a):

...

Andrzej Krzemienski wrote:

...
Please correct me if I am wrong, but my understanding is that Boost.ASIO works under C++03, and so does Boost.Coroutine. Whereas Boost.Coroutine2 requires C++11. If Boost.ASIO were to switch from BoostCoroutine to Boost.Coroutine2, it would break its C++03 guarantee. (Some teams in my work still have to use C++03.)

It's a bit more complex than that. Boost.Coroutine depends on Boost.Context, which is C++11. However, in order to not break C++03, Coroutine doesn't use the public API of Context, but the private "fcontext" parts that are kept C++03 compatible only because of that. This incidentally means that if you compile Context with the `ucontext` backend instead of the `fcontext` one, Coroutine breaks.

So it's basically a mess and needs to be cleaned up. Since asio::spawn is a separate part of Asio, moving just it to C++11 wouldn't be as disruptive a change as requiring C++11 for all of Asio.

Thanks for this explanation. I am not familiar with the details of the issue, so I basically work based on the information in this thread. I am trying to figure out whether the proposed Boost.Spawn better fits inside Boost.ASIO or in a separate library. My current understanding is that a C++03 user that does not use the `ucontext` backend can successfully use `boost::asio:::spawn()` today, and if the implementation of `boost::asio:::spawn()` were simply upgraded to use the proposed implementation, the said C++03 user would be hurt. Is this correct? If this is correct, it means that the new implementation cannot fully replace `boost::asio:::spawn()`, but it can be used as an addition, so that the users can be offered two functions: boost::asio::spawn_fiber() // preferred if you are C++11 or higher boost::asio::spawn() // dangerous, but works with C++03 Would this be a correct summary? Regards, &rzej;

...

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

Casey Bodley

29 Nov 29 Nov

6:11 p.m.

On Sun, Nov 28, 2021 at 1:50 AM Oliver Kowalke via Boost wrote:

...

Hi, I'd like to propose the new library boost.spawn (https://olk.github.io/) for review. boost.asio's boost::asio::spawn() uses the deprecated and inefficient boost.coroutine library. boost.spawn provides the same functionality while using boost.context's fiber directly.

Oliver

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

thanks for proposing this, Oliver! as a satisfied user, i'd like to express my support. the Ceph project has been maintaining a fork of this work at https://github.com/ceph/spawn/, and it's been running in production for nearly 2 years now in our Ceph Object Gateway [1] component for us, the crucial feature of this fork is its ability to customize the StackAllocator. the original boost::asio::spawn() offers limited customization through boost::coroutines::attributes [2], but that only covers the stack size and unwind behavior. Oliver's fork allows you to pass in any object meeting the requirements of boost::context's StackAllocator concept [3], and the context library provides several options here. you can read a bit more about our use case in the "rgw: coroutine stack protection" thread [4] on the Ceph development mailing list another minor feature that this adds over boost::asio::spawn() is support for completion handlers with more than one non-error-code argument. for example, when a handler's signature is (error_code, int, std::string), the async initiating function will return those non-error-code values as a std::tuple [1] https://docs.ceph.com/en/pacific/radosgw/ [2] https://www.boost.org/doc/libs/1_77_0/libs/coroutine/doc/html/coroutine/attr... [3] https://www.boost.org/doc/libs/1_77_0/libs/context/doc/html/context/stack.ht... [4] https://www.spinics.net/lists/dev-ceph/msg00704.html

Christopher Kohlhoff

1 Dec 1 Dec

1:22 p.m.

On Sun, Nov 28, 2021, at 5:49 PM, Oliver Kowalke via Boost wrote:

...

Hi, I'd like to propose the new library boost.spawn (https://olk.github.io/) for review. boost.asio's boost::asio::spawn() uses the deprecated and inefficient boost.coroutine library. boost.spawn provides the same functionality while using boost.context's fiber directly.

In a nutshell: I am not going to remove spawn() from Asio, nor am I going to keep it in its current form forever. This boost.spawn library might well have utility for some folks right now, but it doesn't reflect the design direction that I have in mind for asio::spawn(). Specifically, the current problem is the code breakage inherent in upgrading asio::spawn() from coroutine-v1 to anything else. This breakage relates to: - C++03 support - coroutine-v1 structures that are exposed through the API I have opted to defer that breakage for as long as possible, until I decide on the form, function, language requirement, and level of backward compatibility of a post-breakage asio::spawn() API. The proposal in this thread is to work around me by providing a library as an alternative to asio::spawn(), because I have not yet upgraded asio::spawn(). As part of providing this alternative library, the proposal is that I should also remove asio::spawn()... and in doing so break user code. If there is to be breakage in asio::spawn(), then that breakage creates the opportunity for introducing improvements. I do have some ideas of what these will be... performance, usability, consistency and integration with more recent additions to Asio. Of course this entails spending some time on design work and implementation, but I intend to do this and it will be happening at some point relatively soon. A one-to-one substitution of boost.context in place of boost.coroutine-v1 won't achieve this. Chris

Oliver Kowalke

1:56 p.m.

Am Mi., 1. Dez. 2021 um 14:31 Uhr schrieb Christopher Kohlhoff via Boost < boost@lists.boost.org>:

...

The proposal in this thread is to work around me by providing a library as an alternative to asio::spawn(), because I have not yet upgraded asio::spawn().

Well I tried to solve this with you in 2017 ... My intend is to get boost.coroutine removed from the boost libs.

Oliver Kowalke

6 Dec 6 Dec

4:11 p.m.

Am Mi., 1. Dez. 2021 um 14:31 Uhr schrieb Christopher Kohlhoff via Boost < boost@lists.boost.org>:

...

On Sun, Nov 28, 2021, at 5:49 PM, Oliver Kowalke via Boost wrote:

...
Hi, I'd like to propose the new library boost.spawn (https://olk.github.io/) for review. boost.asio's boost::asio::spawn() uses the deprecated and inefficient boost.coroutine library. boost.spawn provides the same functionality while using boost.context's fiber directly.

In a nutshell: I am not going to remove spawn() from Asio, nor am I going to keep it in its current form forever. This boost.spawn library might well have utility for some folks right now, but it doesn't reflect the design direction that I have in mind for asio::spawn().

Specifically, the current problem is the code breakage inherent in upgrading asio::spawn() from coroutine-v1 to anything else. This breakage relates to:

- C++03 support - coroutine-v1 structures that are exposed through the API

I have opted to defer that breakage for as long as possible, until I decide on the form, function, language requirement, and level of backward compatibility of a post-breakage asio::spawn() API.

The proposal in this thread is to work around me by providing a library as an alternative to asio::spawn(), because I have not yet upgraded asio::spawn(). As part of providing this alternative library, the proposal is that I should also remove asio::spawn()... and in doing so break user code.

If there is to be breakage in asio::spawn(), then that breakage creates the opportunity for introducing improvements. I do have some ideas of what these will be... performance, usability, consistency and integration with more recent additions to Asio. Of course this entails spending some time on design work and implementation, but I intend to do this and it will be happening at some point relatively soon. A one-to-one substitution of boost.context in place of boost.coroutine-v1 won't achieve this.

When do you plan to push your boost.asio commit?

1086

Age (days ago)

1094

Last active (days ago)

List overview

Download

49 comments

12 participants

participants (12)

Andrey Semashev
Andrzej Krzemienski
Boris Kolpackov
Casey Bodley
Christopher Kohlhoff
Glen Fernandes
Niall Douglas
Oliver Kowalke
Peter Dimov
Richard Hodges
Robert Ramey
Vinnie Falco