On Wed, Oct 11, 2023, 10:10 PM Christian Mazakas via Boost < boost@lists.boost.org> wrote:

...

Finally, as a personal note on the review, it was notable how much negativity there currently is against C++ coroutines in general from the C++ community.

I don't think this is a very accurate description at all.

I think most people are incredibly excited about coroutines. They're just less excited about an Asio wrapper.

Though even Rust suffers the problem that in the ecosystem, Futures are tied to their runtimes. In practice, this isn't overwhelmingly a showstopper but I think this is the next area of exploration and development: building a common language for disparate runtimes to use for cross-communication.

What is the issue with the proposed standard executor ts? Does it not provide that?

On 11/10/2023 14:08, Klemens Morgenstern wrote:

...

I find the suggestions that C++ 20 coroutines needs to be somehow in the name convincing. Exactly what form it should take I’ll leave up to the author and community to negotiate, but “Boost.Async” is definitely not a name with good buy-in in the Boost community. It cannot be named “Boost.Async” if it is to enter Boost.

How do we judge what is acceptable to the boost community? Because there might always be someone to object to it.

I'll compose a list and start a separate thread.

I don't claim it'll be easy, but there was strong consensus that the current name must go.

There is left_select, which goes strictly left to right. Would this criteria be satisfied if I renamed left_select to select, and the current select to `unordered_select` or `random_select` ?

The review objections where to naming that operation "select" at all. Select for 99% of C and C++ users is something which does not cancel the other waiters on return. Several mentioned how surprising it was, for them Select is something which returns when something is ready and then code might do stuff like cancel things. I don't think there is anything wrong with the current semantics of the operation, except its name. `select_cancelling_others()` would be fine, but I'm sure a shorter name is possible without the BSD sockets type connotations.

...

- Andrzej took issue with the design of the generators, and I can see why. However, out in real world code land, C++ coroutine generators have surprisingly large amount of design variation. I’ve seen ones which you loop on testing them for boolean true, I’ve seen others which loop on their iterators, I’ve even seen ones which don’t implement either boolean true nor iterators and require you to call a `value()` member function which returns an Optional. Why? No idea, people seem to innovate uselessly here for some reason. I think it wise to adhere to `std::generator` from the standard, and do exactly what it does. From my reading of Async’s `generator`, whilst it has extensions e.g. the invocable call operator, the only thing missing is the iterator interface to make it implement `std::generator` and I don’t see any obvious reason why an iterator interface can’t be added here?

The reason is that we don't have "async for". I would need to resume the generator on increment, but if `itr++` is an awaitable expression that needs to be co_awaited, it's not an iterator. If there was an async for (or a common pattern) I'd make generator work with it.

Y'see I don't see the need for async for. That's separate to this. `std::generator` provides iterators. When you iterate one of its iterators the coroutine gets resumed and it yields a new value. The reason `std::generator` is like this is because it can be called from non-coroutine code, so `co_await` isn't available. Implied in this is iterator implementations must pump the coroutine by hand. This is very straightforward for the simple case, it gets harder if the generator suspends on say i/o. Then your generator iterator implementation needs to pump the event loop too. This isn't a hard to surmount problem to solve for Boost.Async I think. And I think a Boost level library does need to implement what the C++ standard demands, even if it's a bit dumb. Down the road if WG21 ever gets round to async for, support for that can be added later. I have also wondered if a `BOOST_CO_ASYNC_FOR` macro couldn't implement async for right now. And finally, I have a gut feeling without proof nor evidence that it *may* be possible to detect from a macro whether it is being invoked by a coroutine or not, and if so, "do the right thing" based on caller context. So a `BOOST_ASYNC_FOR()` macro which does the right kind of for loop for iterating a generator depending on caller may be achievable. Niall

...

...

The reason is that we don't have "async for". I would need to resume the generator on increment, but if `itr++` is an awaitable expression that needs to be co_awaited, it's not an iterator. If there was an async for (or a common pattern) I'd make generator work with it.

Y'see I don't see the need for async for. That's separate to this.

`std::generator` provides iterators. When you iterate one of its iterators the coroutine gets resumed and it yields a new value.

The reason `std::generator` is like this is because it can be called from non-coroutine code, so `co_await` isn't available. Implied in this is iterator implementations must pump the coroutine by hand.

This is very straightforward for the simple case, it gets harder if the generator suspends on say i/o. Then your generator iterator implementation needs to pump the event loop too. This isn't a hard to surmount problem to solve for Boost.Async I think. And I think a Boost level library does need to implement what the C++ standard demands, even if it's a bit dumb.

Ah, gotcha I thought you wanted that to be in async, i.e. with a co_await somewhere. Yeah that could be done & might actually be something useful for non-pushable generators. I'll look into it.

So the earlier versions of N4775 that had `for co_await` would have made the `++itr` awaitable. With std::generator the increment also rethrows exceptions. That means I could in theory have an async_for do a `co_await async::next(itr)` instead of `co_await ++itr` and have the same iterator type work. There is however a problem with "pumping the event-loop". I have the executor but I might not know the event-loop it's referring to (it might be a thread_pool) or something custom. Now I would either need to somehow create an interface to allow the user to supply a pump function or require that he uses asio::io_context for that to work. I don't like either, I think I'll hold off until there's a user asking for it.

On 11/10/2023 16:53, David Sankel wrote:

On Wed, Oct 11, 2023 at 6:25 AM Niall Douglas via Boost mailto:boost@lists.boost.org> wrote:

Proposed Boost.Async by Klemens Morgenstern was reviewed between 8th and 18th August, and again between 22nd September and 2nd October. Reviews or substantially detailed comments came from...

Thanks Niall. Could you please indicate the number of formal Boost reviews that were made and their tally of accept/reject?

I listed the reviews or feedback substantial enough to be worth naming their authors at the top. I intentionally did not list whether they recommended accept or reject. If you want to know that, you can look up the review. I believe all but one was accept, but some gave no formally written recommendation so I had to infer from the body of the feedback. In any case, it does not matter, approval or rejection isn't based on vote counts. Niall

On 11/10/2023 17:57, Richard Hodges via Boost wrote:

I must claim some culpability for this. I was an early adopter of the library, which itself is a distant descendant of work I and later Klemens did to replicate Go's channel and select() in C++ I/O loops.

The Go select() function does exactly what Klemens' does - suspends until the first source/coroutine has a value available (ties broken by pseudo random selection in order to prevent starvation of the right-most sources), and then continues, leaving the values in the unselected coroutines available for subsequent select() calls.

Being a user of Asio for the past decade, It didn't occur to me that people still actually used BSD select()...

Accepting that the name select() is not acceptable, my motivation for encouraging Klemens to have the default xxx() function take a random ready value, and the left_xxx() function to take the leftmost, was that taking the leftmost ready value is very likely to lead to starvation of code that responds to the the right-most sources delivering values if the leftmost sources are constantly ready. xxx() is therefore the safer unsurprising default for an inexperienced or tired developer.

If I were designing that API, I'd personally provide a user supplied callable which gets invoked to decide on the tie break, and I'd default its implementation to one choosing randomly. Then users can choose whatever suits them, and no need for left_xxx() functions etc. Niall

On 11/10/2023 19:02, Richard Hodges via Boost wrote:

...

If I were designing that API, I'd personally provide a user supplied callable which gets invoked to decide on the tie break, and I'd default its implementation to one choosing randomly. Then users can choose whatever suits them, and no need for left_xxx() functions etc.

This is probably why it’s best I stick to hacking together trading systems.

Wouldn’t it be difficult to default the argument to prefer_left() or similar though, given that the parameters are a variadic pack of sources?

Not having a default would be unacceptable to me as it would involve: - more typing, and - having to read the documentation prior to hacking the code together.

I guess it depends on what you're used to, but in principle I have no idea with convenience wrapper functions which save obtuse syntax for a base implementation function. You can design APIs to take variadic packs of variadic packs if you want. I actually have an API like that in my work code base, I couldn't think of anything less worse, so I went with it. Niall