On Sun, Sep 22, 2019 at 6:44 PM Vinnie Falco wrote:

...

Errata, the sentence should read: "The library enforces a simple invariant: All elements which are part of the same JSON "document" (i.e. all share a common ancestor) are guaranteed to use the same allocator." Thanks

On Sun, Sep 22, 2019 at 6:12 PM Rene Rivera wrote:

Why does it need to depend on Beast? That's not the kind of dependency I'd expect from a JSON library.

There are these four Beast includes which are leftovers from the initial development which will eventually be removed so that the library only depends on basic Boost facilities (Align, Assert, Config, Container, Utility): #include #include #include #include static_string is being developed into its own proposed library: https://github.com/vinniefalco/fixed_string clamp.hpp provides a trivial inline function that I will simply copy over. The flat_buffer will be replaced with a straightforward memory allocation. The dependency on file.hpp will be removed by using std facilities to read from the file. Thanks

On Mon, Sep 23, 2019 at 5:19 AM Dominique Devienne via Boost wrote:

Hi,

What about performance? Have you heard of https://github.com/lemire/simdjson? Where would your library fall in that benchmark at the above link? Finally, there are many corner cases in JSON parsing. Is there the equivalent of Autobabhn for WebSocket, but for JSON parsing? Any plans to integrate with such infrastructure, assuming there's one?

There's also https://github.com/miloyip/nativejson-benchmark which benchmarks both performance and conformance.

The already mentioned and popular nlohmann/json has a convenient API, but fairs poorly in that benchmark for example.

Also, in client/server communications, it's less often a few huge JSON documents, but rather lots of small documents, so the constant "startup" time of the parser matters too, and in that same vein, a PULL-parser that allows to build the native data-structures directly, rather than the DOM-like approach of fully converting the document to a built-in JSON object, and then convert that to the native-structure, avoids the temp document, which is especially useful for large documents.

It looks like nlohmann/json now has this kind of API too: 1. https://tinyurl.com/nl-json-parse 2. https://tinyurl.com/nl-json-parse-callback Glen

On 23. Sep 2019, at 13:06, Glen Fernandes via Boost wrote:

On Mon, Sep 23, 2019 at 6:44 AM Glen Fernandes wrote:

...

On Mon, Sep 23, 2019 at 5:19 AM Dominique Devienne via Boost

...

...

Also, in client/server communications, it's less often a few huge JSON documents, but rather lots of small documents, so the constant "startup" time of the parser matters too, and in that same vein, a PULL-parser that allows to build the native data-structures directly, rather than the DOM-like approach of fully converting the document to a built-in JSON object, and then convert that to the native-structure, avoids the temp document, which is especially useful for large documents.

It looks like nlohmann/json now has this kind of API too: 1. https://tinyurl.com/nl-json-parse 2. https://tinyurl.com/nl-json-parse-callback

That is the use case that I find more interesting. If you want to decouple your data representation in your program from the serialization format.

e.g. In memory if your data structure is something like: vector > >.

This could be expressed in json by something like: [ { "key": [1, 0.01], "def": [9, 0.37], ... }, { "xyz": [5, 1.25], "abc": [2, 4.68], ... }, ... ]

Your load it from some JSON file. Your don't want to store/use it in your program as a SomeLibrary::JsonArray. Past the point of serialization it should be your own data structures. Of course your could always convert it from SomeLIbrary::JsonArray to your own structures, but that's overhead you don't need if there's hundreds of megabytes worth of content in that data.

Just throwing our library in the ring: https://github.com/taocpp/json We have a SAX-style interface at the core, parsers only generate events, from there you can generate a DOM object or some direct UDT structure like mentioned above. (Uses some macros to register types). Or you directly pretty print it or convert to CBOR, MsgPack, etc. You could even generate an nlohmann-DOM-value with our parsers. Benchmarks in the nativejson-benchmark showed that we were faster than nlohmann's builtin parser (that was some time ago, might have changed in the meantime) Currently, our parser(s) do not support incremental parsing, though. Daniel

Glen

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On 23. Sep 2019, at 13:27, Glen Fernandes wrote:

On Mon, Sep 23, 2019 at 7:15 AM Daniel Frey wrote:

...

Just throwing our library in the ring: https://github.com/taocpp/json

We have a SAX-style interface at the core, parsers only generate events, from there you can generate a DOM object or some direct UDT structure like mentioned above. (Uses some macros to register types). Or you directly pretty print it or convert to CBOR, MsgPack, etc.

You could even generate an nlohmann-DOM-value with our parsers. Benchmarks in the nativejson-benchmark showed that we were faster than nlohmann's builtin parser (that was some time ago, might have changed in the meantime)

Looks interesting. Missing a Boost license though. :)

Thanks, and making it dual-licensed is certainly not a problem if that's an actual problem. 😊

Glen

Hi Vinnie! I know that it is not standard JSON, but do you plan to support comments (// ..., /* ... */)? It would be useful for storing configuration with comments, for example. Thanks, Jarda po 23. 9. 2019 v 15:36 odesílatel Vinnie Falco via Boost < boost@lists.boost.org> napsal:

On Mon, Sep 23, 2019 at 4:06 AM Glen Fernandes via Boost wrote:

...

Your don't want to store/use it in your program as a SomeLibrary::JsonArray. Past the point of serialization it should be your own data structures.

This is accomplished by making your own class derived from `json::basic_parser`, and implementing the abstract virtual "event" members:

< https://github.com/vinniefalco/json/blob/25ddea5f4d088f3e56911bf9f97549eeb93...

...

Thanks

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Mon, Sep 23, 2019 at 10:39 AM Vinnie Falco via Boost wrote:

On Mon, Sep 23, 2019 at 7:51 AM JF via Boost wrote:

...

I know that it is not standard JSON, but do you plan to support comments (// ..., /* ... */)? It would be useful for storing configuration

I used the same approach with the JSON parser that I used with Beast's HTTP parser. That is, a strict parser which strives to adhere to the letter of the spec.

I agree with this approach. The popular nlohmann::json project made the same design decision, and I think their reasoning is quite solid. See here: https://github.com/nlohmann/json#comments-in-json -Vicram

On 9/23/19 1:05 PM, Glen Fernandes via Boost wrote:

That is the use case that I find more interesting. If you want to decouple your data representation in your program from the serialization format.

Which is what Boost.Serialization archives gives us.

On Mon, Sep 23, 2019 at 2:19 AM Dominique Devienne via Boost wrote:

What about performance? Have you heard of https://github.com/lemire/simdjson? Where would your library fall in that benchmark at the above link?

simdjson gives the highest performance, but the resulting output is in a specialized format which is read-only. My library has roughly the same or better performance as RapidJSON. Thanks

On Tue, Oct 22, 2019 at 10:18 AM Mateusz Loskot via Boost wrote:

Does it mean you have run it under Milo's nativejson-benchmark?

I tried to integrate it into that benchmark but to be honest, that project looks like a hot mess. I don't have the resources to try to get my library working with it.

Can you post the results?

I have the results from a simple benchmarking program that I wrote, which parses a bunch of random JSON. There are two data sets, a small one and a big one: small data set: rapidjson parse 55952763 bytes in 230ms rapidjson parse 55952763 bytes in 229ms rapidjson parse 55952763 bytes in 228ms nlohmann parse 55952763 bytes in 514ms nlohmann parse 55952763 bytes in 514ms nlohmann parse 55952763 bytes in 533ms Boost.JSON parse 55952763 bytes in 234ms Boost.JSON parse 55952763 bytes in 233ms Boost.JSON parse 55952763 bytes in 233ms large data set: rapidjson parse 488889121 bytes in 1793ms rapidjson parse 488889121 bytes in 1791ms rapidjson parse 488889121 bytes in 1789ms nlohmann parse 488889121 bytes in 3921ms nlohmann parse 488889121 bytes in 3942ms nlohmann parse 488889121 bytes in 3965ms Boost.JSON parse 488889121 bytes in 1754ms Boost.JSON parse 488889121 bytes in 1761ms Boost.JSON parse 488889121 bytes in 1764ms The benchmark program is here: https://github.com/vinniefalco/json/blob/master/bench/bench.cpp I will likely expand on it to measure more things and report the results in the documentation (a work in progress).

I have not found enough time to pull your library in.

I still have a bit of work to do with the treatment of floating point numbers, and there are a couple of bugs with the handling of utf code points. Those will be done very soon. Thanks

I have the results from a simple benchmarking program that I wrote, which parses a bunch of random JSON. There are two data sets, a small one and a big one:

small data set: rapidjson parse 55952763 bytes in 230ms rapidjson parse 55952763 bytes in 229ms rapidjson parse 55952763 bytes in 228ms nlohmann parse 55952763 bytes in 514ms nlohmann parse 55952763 bytes in 514ms nlohmann parse 55952763 bytes in 533ms Boost.JSON parse 55952763 bytes in 234ms Boost.JSON parse 55952763 bytes in 233ms Boost.JSON parse 55952763 bytes in 233ms

large data set: rapidjson parse 488889121 bytes in 1793ms rapidjson parse 488889121 bytes in 1791ms rapidjson parse 488889121 bytes in 1789ms nlohmann parse 488889121 bytes in 3921ms nlohmann parse 488889121 bytes in 3942ms nlohmann parse 488889121 bytes in 3965ms Boost.JSON parse 488889121 bytes in 1754ms Boost.JSON parse 488889121 bytes in 1761ms Boost.JSON parse 488889121 bytes in 1764ms

Impressive Vinnie. Not at all a small accomplishment to match RapidJSON. Congrats! Niall

On 10/22/19 15:40, Robert Ramey via Boost wrote:

On 10/22/19 1:45 PM, Niall Douglas via Boost wrote:

...

...

I have the results from a simple benchmarking program that I wrote, which parses a bunch of random JSON. There are two data sets, a small one and a big one:

<snip impressive comparisons of parse times>

...

Impressive Vinnie. Not at all a small accomplishment to match RapidJSON. Congrats!

Nail,

I'd love to see the result you'd get by using one of number of JSON parsers implemented by Boost.Spirit.  A number of them pop up by googling "C++ JSON spirit" including one by our own Michael Caisse.  I'd also be curious to know the compile time as well as the runtime for the programs above.

The Michael Caisse one isn't made for speed. It does something else. It will perform poorly in comparison I suspect. -- Michael Caisse Ciere Consulting ciere.com

On 9/23/19 9:11 AM, Vinnie Falco via Boost wrote:

On Mon, Sep 23, 2019 at 8:58 AM Bjorn Reese via Boost wrote:

...

...online parser... A push parser (SAX)... A tree parser (DOM)

I have no experience with these terms other than occasionally coming across them in my Google searching adventures. The parsers that I have written take as input one or more buffers of contiguous characters, and produce as "output" a series of calls to abstract member functions which are implemented in the derived class. These calls represent tokens or events, such as "key string", "object begin", "array end". So what would we call this in the taxonomy above?

Hmmmm - sounds like a job for Boost Spirit ! This is what I used 15+ years ago to create a special purpose for the limited subset of XML that boost serialization library. In all that time, through innumerable variations of compilers, linkers, C++ versions, ... everything - modification has been required on only a couple of cases. And by separating syntax from actions, it has permitted other collaborators to discover and suggest fixes to weird corner cases. The code seems pretty efficient too - at least no one has complained about that aspect - or another other either. The only complain would be slow compile time. But since the serialization library is compiled and the code is only compiled when the grammar changes - it's not really an issue. This software and its approach has been under appreciated - need a good CppCon talk on this subject. Food for thought. Robert Ramey

Thanks

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Tue, Sep 24, 2019 at 1:50 AM Phil Endecott via Boost wrote:

Real-soon-now we'll have coroutines, and the two parties (the parser and its user) won't have to choose anymore about who pushes and who pulls.

A design criteria for this library is to require only C++11

Rather than that, how about a co-routine that yields a variant of those types?

If you mean something like json::value jv = co_await parse(source); I think that the inversion of the flow of control resulting from this design has excessive impact on the rest of the code. If we wanted to retrieve a JSON document from a socket for example, then `source` would have to be a coroutine. In the current design, the parser does not impose any model as to how the data buffers are acquired. I could be wrong though, do you have a more complete example? Thanks

On Mon, Sep 23, 2019 at 6:12 PM Glen Fernandes via Boost < boost@lists.boost.org> wrote:

Dominique explained some of the pull (stax) / push (sax) terminology to me off-list, and I agree. This does appear to be the more appealing underlying facility.

I didn't realize it was off-list, usually plain Reply goes to the list. But doesn't matter, Bjorn explained it better than me anyway. On Mon, Sep 23, 2019 at 6:11 PM Vinnie Falco via Boost < boost@lists.boost.org> wrote:

On Mon, Sep 23, 2019 at 8:58 AM Bjorn Reese via Boost wrote:

...

...online parser... A push parser (SAX)... A tree parser (DOM)

I have no experience with these terms other than occasionally coming across them in my Google searching adventures. The parsers that I have written take as input one or more buffers of contiguous characters, and produce as "output" a series of calls to abstract member functions which are implemented in the derived class. These calls represent tokens or events, such as "key string", "object begin", "array end". So what would we call this in the taxonomy above?

That's a PUSH parser IMHO. The doc on Qt's XML PULL parser should make that clearer perhaps: https://doc.qt.io/qt-5/qxmlstreamreader.html#details Many of these terms originated in the XML world, and many (like SAX) from the Java world too. To give you a feel for it, here's my PUSH parser API: class JSONHandler { public: ... virtual bool handle_object_begin(); virtual bool handle_object_key(const std::string& key); virtual bool handle_object_end(); virtual bool handle_array_begin(); virtual bool handle_array_end(); virtual bool handle_number(int); virtual bool handle_number(int64_t); virtual bool handle_number(uint64_t); virtual bool handle_number(double value); virtual bool handle_string(const std::string& value); virtual bool handle_boolean(bool value); virtual bool handle_null(); ... }; bool json_parse(const char* json_utf8_text, size_t len, JSONHandler& handler); While that's my PULL parser API: enum JSONParsingEventType { //! Special end-of-document token. JSON_END = 0, // Value tokens. JSON_NULL, JSON_TRUE, JSON_FALSE, JSON_STRING, JSON_NUMBER, JSON_OBJECT_BEGIN, JSON_OBJECT_KEY, JSON_OBJECT_END, JSON_ARRAY_BEGIN, JSON_ARRAY_END, ... }; class JSONReader { public: JSONReader( const char* json_utf8_text, size_t len, const JSONParserOptions& options = JSONParserOptions() ); ~JSONReader(); JSONParsingEventType peek() const; JSONParsingEventType next(); JSONParsingEventType current() const; size_t skip_next(); size_t skip_current(); JSONToken token(); size_t depth(); size_t count(); bool is_integral(); int get_int(); int64_t get_int64_t(); uint64_t get_uint64_t(); float get_float(); double get_double(); std::string get_string(); std::string get_string_or_null(); bool get_boolean(); std::string get_key(); bool is_key(const char* key); bool is_key(const char* key, size_t len); ... }; where JSONToken is basically a std::string_view-like object into the raw JSON doc bytes, with low-level info for more control, about seeing a numeric sign, fractional point, or exponent, or about strings having escaped characters, including unicode ones, i.e. can't be used as-is, must be decoded according to JSON rules to get back UTF-8 text. The former parser "pushes" information at you, the client code. The parser does the looping. While in the latter, the client code is in the driver seat and does the loop, and controls the parser, extracting information out of it. There's also no inheritance necessary with a PULL parser, virtual or static-CRTP. As Bjorn wrote, a PULL parser is the lowest level building block, and the most convenient one to use. A PULL parser is typically passed around to code decoding various data structures, to instantiate them and their "children/descendants" from the infoset in the JSON doc. To make that safe from misbehaving code, I added concepts like "scopes" and "savepoints", so that the function you pass the reader to cannot step out of the current object, and to allow the caller code to recover by "rewinding" the doc to before the misbehaving reader, skip that object, and try the next one. Which means I also basically support incremental parsing too, even though I don't have an API for it, as obvious from above. Many parsers also have safeguards and "limits" in terms of depth of the stack, or maximum size allowed for strings, which are configured here via the JSONParserOptions struct. Anyways, I'm just showing this to illustrate differences between parsers. There are much better and faster parsers than mine. I learned a lot building them though, it was fun. Mine is comparable to nlohmann in terms of performance, i.e. not that fast :). --DD

On 9/23/19 6:11 PM, Vinnie Falco wrote:

On Mon, Sep 23, 2019 at 8:58 AM Bjorn Reese via Boost wrote:

...

...online parser... A push parser (SAX)... A tree parser (DOM)

I have no experience with these terms other than occasionally coming across them in my Google searching adventures. The parsers that I have written take as input one or more buffers of contiguous characters, and produce as "output" a series of calls to abstract member functions which are implemented in the derived class. These calls represent tokens or events, such as "key string", "object begin", "array end". So what would we call this in the taxonomy above?

So if I understand your model correctly, it parses as many token as possible, and each token results in a callback. When you reach the end of the buffer, you manually suspend parsing and resume when more data is fed into it. That sounds like a variation of a push parser. It may be easier to understand the parser models in C++ terms, although these terms do not accurately cover the parser models. A pull parser is equivalent to a forward iterator. The user tells it when to advance to the next element. A pull parser usually has a richer interface than a C++ iterator because we are iterating over heterogenous elements. We therefore need to query multiple attributes of the current element, such as * The type of the current element (e.g. integer, string, start of array) * The converted value (e.g. text-to-integer conversion, or JSON string to UTF-8 string conversion) * The unconverted value (a string view of the input corresponding to the current value) A push parser is equivalent to a loop with a visitor. The loop traverses the entire input, and for each recognized element it calls the visitor. Assuming you already have a rich iterator, then it is straight-forward to use this for the loop and call a visitor in each iteration. That is, push parsers are easily build on top of pull parsers. You can find a simple example of this here: https://github.com/breese/trial.protocol/tree/develop/example/json/push_pars... A more elaborate example shows an alternative implementation of boost::property_tree JSON parser using a pull parser: https://github.com/breese/trial.protocol/blob/develop/example/json/property_...

On 30/09/2019 23:14, Julien BLANC wrote:

"Bjorn Reese" – 29 septembre 2019 12:56

...

That is, push parsers are easily build on top of pull parsers.

Do you have an example of an efficient implementation of a pull parser on top of async io ? I find push parsers really more straightforward in that case.

Usually that requires the ReadNextToken call (whatever its name) to be async as well (whether that's via callbacks, futures, coroutines, or some other async framework). Failing that, it would have to return some special error code that means "I need more input". Usually (outside of a proper async framework) the caller pulling tokens from the parser is directly responsible for feeding it new input as well.

On Tue, Sep 24, 2019 at 4:29 AM Julien Blanc via Boost wrote:

You may have a look at https://github.com/Julien-Blanc-tgcm/jbc-json .

Do I understand this correctly, that jbc-json parses one character at a time and stores function pointers in the stack, calling through the function pointer for each character? https://github.com/Julien-Blanc-tgcm/jbc-json/blob/ccd8f482fd285eb52ad4350b7... Thanks

On Tue, Sep 24, 2019 at 2:29 AM Mathias Gaunard wrote:

Features that matter to me: - no allocation, ever - parsing/generating data incrementally from/into segmented buffers without copying - full arbitrary-precision decimal support without imposing a decimal representation on me - not imposing any string or blob type - fast conversion between numbers and text, with correct shortest representation for floating-point

These are all features of a JSON parser. As I said earlier, I think the parser is the least interesting part of a JSON library. There are already a bunch of parsers that do exactly what you are asking above. Since parsers are unlikely to appear in public interfaces, there is little value in standardizing on a particular one. Library interoperability is not enhanced in this case. What we are missing, is a robust *container* for storing JSON values. Having a great container for representing all or part of a JSON document does make it easier to write interoperable library components that work with JSON. A library that implements the JSON-RPC specification [1] may have a function with this signature: // Validates and extracts a conforming JSON-RPC request from the // specified JSON object. Throws an exception if the request is not // conforming rpc_call get_rpc_request (boost::json::value& jv); For this to work, boost::json::value needs to be a good general purpose JSON container that satisfies most users. We can never satisfy ALL users; some design choices must represent tradeoffs between conflicting goals. The library I am providing, to propose to Boost eventually, places emphasis on the design of the JSON container because this is the surface which will be exposed between libraries. It is this part that hopefully will stimulate the growth of an ecosystem of libraries which use JSON and interoperate. A final example, someone may use Boost.Beast and Boost.JSON to implement a generic RPC client or server library, taking care of the socket and connection management, making requests and receiving responses. Users of such a library will care little about the parser it uses, other than that it exists and works. But they will care very much about the types used to represent JSON data, since they will be interacting with it regularly to implement their business logic. Regards [1] https://www.jsonrpc.org/specification

On Tue, Sep 24, 2019 at 6:56 AM Dominique Devienne via Boost wrote:

A boost::json::value with by-value semantic, proper R-value support, a nice API, that's compact and efficient, would have value.

Yes!

How different from boost::variant is it though?

Do you mean literally boost::variant, std::vector<...>, number_type, bool, null_type> ? I'm not even sure what to say to that... lol... the difference between the declaration above and the provided json::value is like night and day. `variant` is missing almost all of these member functions: https://github.com/vinniefalco/json/blob/dd53665862d626b6cc308203fdb6d580457... Surely you mean something else?? Regards

On 10/22/19 11:03 AM, Vinnie Falco via Boost wrote:

On Tue, Oct 22, 2019 at 10:40 AM Robert Ramey via Boost wrote:

...

Did you look into using boost.spirit for parsing?

I didn't even consider it. All of the parsers that I work with accept untrusted inputs,

I don't buy this as a reason - in fact I'd call it reason to specify and enforce a formal grammar.

so when writing a parser I prefer to have no external dependencies. Spirit is an enormous dependency and scares off potential users.

This I can appreciate and sympathize with.. It points to the problem that C++ and boost have been wrestling with forever - dependency management. I'm pretty doubtful that anyone can write a demonstrably correct parser without using such a tool. So if security is an issue perhaps you might want to write a test program based on spirit. This would give one confidence that your library won't introduce security holes - at least on the inputs tested. This might be appreciated. Since users don't typically build/run tests (though I've advocated that they should!) there wouldn't be any kind of dependency issue for them. And it would give you the option of writing 1000++ test cases without having to check them all by hand. Just food for thought - feeding the beast. Robert Ramey

Thanks

_______________________________________________ Unsubscribe & other changes: http://lists.boost.org/mailman/listinfo.cgi/boost

On Tue, Oct 22, 2019 at 12:55 PM Mateusz Loskot via Boost wrote:

I'd consider covering the thing with https://google.github.io/oss-fuzz/ instead.

My strategy for ensuring correctness is two-fold. First, as with Beast, it will be reviewed by an external company (they will do the fuzzing). Second, are the special tests I write so that I have confidence everything works. This methodology is as follows: * Create a set of representative test vectors (examples of correct and invalid inputs) I have written my own inputs, and I have imported these test: https://github.com/nst/JSONTestSuite/tree/master/test_parsing Then, for each test vector: * Parse the input as one string and verify the output * Loop over every possible location that the input may be split into two pieces, parse the input as two individual pieces, verify the output. Code: https://github.com/vinniefalco/json/blob/cb348218345cfe2bea09d4a8ca8ea4c0f13... Then, for each possible split point also perform these algorithms: * Using a special allocator (`fail storage`) which throws after N calls to allocate, attempt to parse the input in a loop where N starts out at 1 and is incremented on each allocation failure. The test succeeds if the loop exits after a maximum number of iterations and the output is verified correct. Code for this failing allocator is here: https://github.com/vinniefalco/json/blob/cb348218345cfe2bea09d4a8ca8ea4c0f13... * Using a special parser (`fail_parser`) which returns an error after N calls to the parser's SAX API, attempt to parse the input in a loop where N starts out as 1 and is incremented on each failure. The test succeeds if the loop exits after a maximum number of iterations and the output is verified correct. Code for this failing allocator is here: https://github.com/vinniefalco/json/blob/cb348218345cfe2bea09d4a8ca8ea4c0f13... These tests are run under valgrind, address sanitizer, undefined behavior sanitizer, and code coverage. Then I look at the code coverage to find uncovered or partially covered lines, and devise individual tests to ensure that code is exercised. By now, there are only a handful of such lines if that. With these techniques I achieve close to 100% code coverage and very high confidence that every path through the parser is correct. After a bunch of testing (which consists of telling users it is "ready" and seeing what they report back) I submit it to the external code auditing company to get a report. After fixing any issues they raise in the report, my strategy changes: touch the code as little as possible. If this code used an external dependency, and that upstream code changed, then transitively it means my code changed - for this reason I avoid using external code like Spirit (or regex) even if it means I have to duplicate stuff. Thanks

On 2019-10-22 23:21, Vinnie Falco via Boost wrote:

On Tue, Oct 22, 2019 at 12:55 PM Mateusz Loskot via Boost wrote:

...

I'd consider covering the thing with https://google.github.io/oss-fuzz/ instead.

My strategy for ensuring correctness is two-fold. First, as with Beast, it will be reviewed by an external company (they will do the fuzzing). Second, are the special tests I write so that I have confidence everything works.

[snip]

With these techniques I achieve close to 100% code coverage and very high confidence that every path through the parser is correct. After a bunch of testing (which consists of telling users it is "ready" and seeing what they report back) I submit it to the external code auditing company to get a report. After fixing any issues they raise in the report, my strategy changes: touch the code as little as possible. If this code used an external dependency, and that upstream code changed, then transitively it means my code changed - for this reason I avoid using external code like Spirit (or regex) even if it means I have to duplicate stuff.

Although I appreciate that you achieve a great degree of code stability with your approach, I find it counter-productive since it encourages constantly reinventing the wheel and monolithic design. Imagine if every library in Boost had its own copy of e.g. shared_ptr. No matter how well tested and reliable each copy is, that would be a nightmare for users. IMO, we should reuse the well designed, reviewed and tested code that we already have (and Boost.Spirit is an example of such). Otherwise all that work someone put in that code was done in vain.

On 10/22/19 2:03 PM, Vinnie Falco via Boost wrote:

Furthermore json::basic_parser is optimized to work well for incremental inputs, Spirit on the other hand is not. But I'm no Spirit expert so if someone wants to show me how a Spirit based parser can match the hand-rolled version feature for feature at the same or better performance, that would be quite interesting.

First of all I'm not sure what you by incremental inputs. Maybe it means the same as in the serialization xml parse which I think does one phrase on demand rather than setting it up and letting'er rip. I've conceded to your decision about not using spirit rather than your hand rolled version as a component of your JSON parser. My suggestion is that you consider it for use in your test suite where dependencies are not an issue but provable correctness is. It's much easier to verify the correctness of a grammar in bnf or peg than it is to verify the correctness of a hand rolled parser. So you can make a test program in spirit which whose correctness can be statically verified and use it test same strings you use to test your hand rolled version and verify you get the exact same results. This avoids having to manually verify the correct results for every test string. And makes the process of adding a new test string trivial rather than onerous. You've described in detail your plans for ensuring that your parser is correct. The way you do it it's a huge amount of work involving many people and subject to human frailties. And the correctness of your checking can't really be verified by someone other than the person that does it. Adding more people and more work doesn't really increase confidence. Using an alternative method which is more provably correct is much more likely to smoke out a bug. And it's a lot less work too! Robert Ramey

You guys are overthinking it. You know how fast this JSON parser is, you like the API (or you don't), it works correctly => you use it (or you don't). If correctness is critical for you, wait a few months until the rate of commits declines. File bugs if you find them. Who cares _how_ it works? On Tue, Oct 22, 2019 at 2:42 PM Robert Ramey via Boost < boost@lists.boost.org> wrote:

On 10/22/19 2:03 PM, Vinnie Falco via Boost wrote:

...

Furthermore json::basic_parser is optimized to work well for incremental inputs, Spirit on the other hand is not. But I'm no Spirit expert so if someone wants to show me how a Spirit based parser can match the hand-rolled version feature for feature at the same or better performance, that would be quite interesting.

First of all I'm not sure what you by incremental inputs. Maybe it means the same as in the serialization xml parse which I think does one phrase on demand rather than setting it up and letting'er rip.

I've conceded to your decision about not using spirit rather than your hand rolled version as a component of your JSON parser. My suggestion is that you consider it for use in your test suite where dependencies are not an issue but provable correctness is. It's much easier to verify the correctness of a grammar in bnf or peg than it is to verify the correctness of a hand rolled parser. So you can make a test program in spirit which whose correctness can be statically verified and use it test same strings you use to test your hand rolled version and verify you get the exact same results. This avoids having to manually verify the correct results for every test string. And makes the process of adding a new test string trivial rather than onerous.

You've described in detail your plans for ensuring that your parser is correct. The way you do it it's a huge amount of work involving many people and subject to human frailties. And the correctness of your checking can't really be verified by someone other than the person that does it. Adding more people and more work doesn't really increase confidence. Using an alternative method which is more provably correct is much more likely to smoke out a bug. And it's a lot less work too!

On Tue, Oct 22, 2019 at 3:13 PM Robert Ramey via Boost < boost@lists.boost.org> wrote:

Now this particular case - providing a provable correct parser for a simple language - IS a toy problem.

It does not follow that the only valid reason to do it is to play games.

Implementing a parser derived from a formal grammar CAN be done quite easily with boost spirit - if only for the the testing portion of the project. (Of course if spirit were used in the main product, much less testing would be required!).

both Vinnies method and your method create huge amount of extra work for various persons involved in the process. I'm not over thinking it, you're not thinking big enough. It's not just use cranking up some code it's sucking in all the other participants to do our work for us.

I get your point that it's easier to reason about correctness if you can trust the individual components. This reflects a "white box testing" mentality. But if Spirit is bug-free, logically it does not follow that a library that uses it has better chance of being bug-free, compared to a library that does not. Either way, you ought to test, ignoring any and all knowledge of its internals. Consider also even if using tested components can get you to bug-free implementation quicker (which isn't proven in this case), on balance it is better if you don't depend on another library. I'd be surprised if you think otherwise.

On Tue, Oct 22, 2019 at 8:39 PM Robert Ramey via Boost < boost@lists.boost.org> wrote:

On 10/22/19 3:42 PM, Emil Dotchevski via Boost wrote:

...

I get your point that it's easier to reason about correctness if you can trust the individual components. Right. This reflects a "white box testing" mentality. I have heard the term - but I never knew what it meant

It means, you approach testing with knowledge of how the program works. In this case, you reason about the correctness of a JSON parser based on the correctness of Spirit. For example, you might skip testing certain input sets because you'd reason that their impact is limited to Spirit, and you *know* it works. In contrast, "black box" testing specifically ignores such reasoning and focuses on testing the correct behavior not knowing how the program works. Ideally, whether or not you can rely on a component to be bug-free should have no impact on the thoroughness of the tests.

...

But if Spirit is bug-free, logically it does not follow that a library that uses it has better chance of being bug-free, compared to a library that does not.

...

on balance it is better if you don't depend on another library.

Also I disagree with this. All things being equal, I'd rather depend on someone else's work than my own. I want to focus on only those things that are unique to my situation. And I get no satisfaction from doing something that someone else has already done better.

What I mean is that your users would appreciate that your library has one fewer dependencies.

On Tue, Oct 22, 2019 at 1:50 PM Andrey Semashev via Boost < boost@lists.boost.org> wrote:

IMO, we should reuse the well designed, reviewed and tested code that we already have (and Boost.Spirit is an example of such). Otherwise all that work someone put in that code was done in vain.

Not always. The exception is when a library can have zero dependencies, which is very valuable for low level libraries. For example, in LEAF I need a piece of mp11, but I've copied that piece over in LEAF so there's no dependency on anything. If you have one dependency, you might as well have 2 or 3.

On 10/22/19 12:54 PM, Mateusz Loskot via Boost wrote:

I'd consider covering the thing with https://google.github.io/oss-fuzz/ instead.

That's a separate issue. That can generate test cases, but can't check that the test is parsed correctly.

Best regards,