Boost.HTTPKit, a new library from the makers of Beast!
During the formal review of Beast the desire was expressed to have
parsing and serialization available independently of Asio.
Unfortunately the obstacles to this are the concrete functions and
types required for interoperability with the buffer sequence concepts
defined in Asio.
I have drawn up a solution to this problem which involves creating two
brand new libraries.
The first new library is called Boost.Buffers, and I described it
partially in an earlier list posting. This library will contain copies
of just the header files from Asio needed to implement the buffer
sequence concepts. A preprocessor switch will allow users of
Boost.Buffers to decide whether to use the copies of the headers, or
to just include the Asio headers directly:
https://github.com/vinniefalco/buffers/blob/30ef7031ec0909972a720c0cdd8d6c6e...
If an author wants to develop a library which uses just the buffer
sequence concepts from Asio, then instead of writing:
#include
2017-10-04 12:22 GMT-03:00 Vinnie Falco via Boost
This solution satisfies long-stated needs of users to have HTTP parsing and serialization without Boost.Asio.
Could you compare this to my parser[1]? I've been writing this project since the last year[2]. It could be that I'm slow, but I'm starting to think you like to rush design decisions. [1] https://vinipsmaker.github.io/asiohttpserver/ [2] https://gist.github.com/vinipsmaker/4998ccfacb971a0dc1bd -- Vinícius dos Santos Oliveira https://vinipsmaker.github.io/
On Wed, Oct 4, 2017 at 9:44 AM, Vinícius dos Santos Oliveira
Could you compare this to my parser[1]?
HTTP parsing is a pretty well defined problem and if you want to have decent performance, there aren't a whole lot of different ways to write it. Looking over your code I don't see any obvious implementation mistakes although as far as the interface goes I would have made different choices, as you can see by looking at the declaration: https://github.com/boostorg/beast/blob/7fe74b1bf544a64ecf8985fde44abf88f9902... The biggest, obvious difference is that beast::http::basic_parser and beast::http::serializer are already in Boost, and have gotten extensive testing and vetting from stakeholders. It is also running on production servers that handle hundreds of millions of dollars worth of financial transactions per month.
It could be that I'm slow
I don't know if I would say that you're slow as I have not observed your workflow but a comparison is probably unfair, as I have the opportunity to work on Beast and related projects such as Boost.Buffers and Boost.HTTPKit full-time.
but I'm starting to think you like to rush design decisions.
Now that is really unfair, and ignores the enormous amount of hours not just from me but from all of the other people who participated in the design of Beast (note that I invited you to be one of those participants). Nothing about Beast was rushed, in fact for 3 months in 2016 I was blocked on the parser design because I could not figure out an elegant interface that worked with the stream algorithms and allowed the users to supply their own buffer for the body. I tried several different designs, if you look in the commit log you can find those alternate designs. Here are just a few of the design collaborations that helped to get Boost.Beast where it is at today: Progressive body reading (122 comments!) https://github.com/boostorg/beast/issues/154 Split parsing / headers first https://github.com/boostorg/beast/issues/132 Expect: 100-continue design https://github.com/boostorg/beast/issues/123 Message container constructors https://github.com/boostorg/beast/issues/13 Fields concept and allocator support https://github.com/boostorg/beast/issues/124 Refactor Body types https://github.com/boostorg/beast/issues/580 Asio deallocate-before-invoke contract https://github.com/boostorg/beast/issues/215 And this does not include over a hundred hours working with engineers from Ripple (Beast was originally designed for them) on shared code pads to create the initial container and stream algorithm ideas. Quite a lot of time went into the design of Beast in fact there was far more time spent designing than actual coding. And I couldn't have done it without the very helpful interaction with actual users of the library; Beast was shaped by the needs of stakeholders. As a reminder to everyone, the Beast pull request / issues queue is open to all and I welcome anyone who wants to participate in the design of the library to jump right in and comment on any of the issues or pull requests. You can see that there are a number of open issues that have unresolved design questions: https://github.com/boostorg/beast/issues?q=is%3Aissue+is%3Aopen+label%3ADesi... Thanks!
2017-10-04 15:20 GMT-03:00 Vinnie Falco
Now that is really unfair, and ignores the enormous amount of hours not just from me but from all of the other people who participated in the design of Beast
If anything, it was my response that was rushed. Sorry about that. Now, moving on... given that you have __not__ answered how your parser's design[1] compares to the parser I've developed, I wrote an “implementing Boost.Beast parser interface” in my documentation: https://vinipsmaker.github.io/asiohttpserver/#_implementing_ boost_beast_parser_interface This tutorial is full of “design implications” blocks where I take the time to dissect what it means to go with each choice. (note that I invited
you to be one of those participants).
Yes, just bad timing. [1] at most it hides behind a “multiple people/stakeholders agree with me” __shield__ -- Vinícius dos Santos Oliveira https://vinipsmaker.github.io/
How does the community feel about:
* Boost.Buffers as a solution to accessing buffer concepts without Asio?
* Boost.Buffers offering Beast's buffer sequence adapters and dynamic buffers?
I reiterate from my Beast review that the best design for these above is to use: * span<T> http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/p0122r5.pdf * Ranges TS http://www.open-std.org/jtc1/sc22/wg21/docs/papers/2017/n4685.pdf Both of these can integrate with the Networking TS, but not the other way round due to ASIO's legacy design. You chose, against my advice, to base Beast on the outdated and hard coded buffer sequence design in ASIO. It was not a showstopper for Beast because Beast was so closely tied into ASIO, so I recommended acceptance for Beast despite this design choice. But for a standalone library, things are different. I would advise strongly against accepting a Boost.Buffers library unless it is based on span<T> and Ranges and is forward looking into C++ 20 design patterns, not backwards at C++ 98 era design patterns like ASIO's buffer sequences. Regarding HTTPKit etc, I haven't looked into it much, but if I remember I had some issues with your implementation and design there too specifically the lack of use of zero copy views. Again, as part of Beast wholly dependent on ASIO and thus limited by ASIO's limits, that's acceptable. As a standalone library where a much wider use case applies, I think I would need to set a much higher bar in any review. But I haven't looked into it, you may have completely changed the design and implementation since I last looked. Niall -- ned Productions Limited Consulting http://www.nedproductions.biz/ http://ie.linkedin.com/in/nialldouglas/
On Wed, Oct 4, 2017 at 4:18 PM, Niall Douglas via Boost
I reiterate from my Beast review that the best design for these above is to use:
* span<T> * Ranges TS
First of all, thanks for investing the time to respond, your feedback
is appreciated. I think this might be one of those rare cases where I
am in partial agreement. A design that uses standard library types is
better than one which requires `std::experimental::net::const_buffer`
or `boost::asio::const_buffer`. However, a couple of points:
* Ranges TS is unnecessary, as ConstBufferSequence and
MutableBufferSequence are already modeled after ForwardRange.
* `span<byte>` and `span<byte const>` are not great choices because
`span` and `byte` are not part of Boost and not part of the standard
library. The closest we can get and still have something compatible
with C++11 is `pair
You chose, against my advice, to base Beast on the outdated and hard coded buffer sequence design in ASIO.
I'll say the same thing I said the last time you brought this up. If you feel strongly about it, then you need to write a paper. Otherwise, your advice to me effectively becomes "ignore existing and emerging standards and invent a new, incompatible concept." That's way too much risk for me to take on, and I have no evidence that my users want such a thing. All the feedback I have received thus far cites compatibility with Asio as a primary motivator for adoption of Beast. Ignoring this seems...reckless.
But for a standalone library, things are different. I would advise strongly against accepting a Boost.Buffers library unless it is based on span<T> and Ranges and is forward looking into C++ 20 design patterns, not backwards at C++ 98 era design patterns like ASIO's buffer sequences.
If you feel that span<T>, Ranges, and C++20 design patterns are important then you should be enthusiastic about my Boost.Buffers proposal, because it physically separates the buffer concepts from the networking algorithms. It effectively "factors out the buffers from Asio." In other words it completes the first step of achieving your goal - it separates buffers from the rest of Asio where it can be acted on independently. Note that I am in favor of Paul's proposal to change the requirements of ConvertibleToConstBuffer, in order to relax the dependency on a concrete type. I believe this is in line with your goals as well.
Regarding HTTPKit etc, I haven't looked into it much, but if I remember I had some issues with your implementation and design there too specifically the lack of use of zero copy views.
In the version of Beast that was accepted into Boost, beast::http::basic_parser was already "zero-copy" (when presented with input as a single contiguous buffer) and beast::http::serializer was "zero-copy". They still are.
As a standalone library where a much wider use case applies, I think I would need to set a much higher bar in any review.
So, I think my comments earlier apply here as well. The primary consumers of beast::http::basic_parser and beast::http::serializer are using Asio buffer concepts (Beast stream algorithms being the best examples). Factoring out the buffer concepts from Asio into a new library Boost.Buffers, and then factoring out the parser and serializer from Beast into a new library Boost.HTTPKit which uses Boost.Buffers without needing Boost.Asio as a dependency seems like a very pragmatic way forward which gives stakeholders something they've been asking for, doesn't reinvent established buffer concepts, doesn't force Boost.Asio as a dependency, and yet is compatible with Boost.Asio for the users that want it. Thanks
participants (3)
-
Niall Douglas
-
Vinnie Falco
-
Vinícius dos Santos Oliveira