Em dom., 24 de nov. de 2019 às 12:30, Vinnie Falco via Boost < boost@lists.boost.org> escreveu:
I'm not seeing where trial.protocol has incremental algorithms, perhaps you can show me? trial::protocol::json::basic_reader constructs with the complete input:
< https://github.com/breese/trial.protocol/blob/4bdf90747944f24b61aa9dbde92d8f...
There is no API to provide additional buffers. By "incremental" I mean an "online algorithm", i.e. the entire input does not need to be presented at once. For example, this is what it might look like using boost.json to incrementally parse a JSON from a socket:
json::value parse( net::ip::tcp::socket& sock ) { error_code ec; json::parser p; p.start(); for(;;) { char buf[4096]; auto const n = sock.read_some( net::mutable_buffer(buf, sizeof(buf)), ec); if(ec == net::error::eof) break; if(ec) throw system_error(ec); p.write(buf, n, ec); if(ec) throw system_error(ec); } p.finish(); return p.release(); }
Serialization functions similarly. The caller provides a buffer, and the implementation attempts to fill the buffer with serialized JSON. If the buffer is not big enough, subsequent calls may be made to retrieve the rest of the serialized output.
There are a few remarks that ought to be added to these statements. What I have in mind for an incremental parser is an in situ/in-place algorithm in which case there is no auxiliary data structure (i.e. only a small and constant amount of space may be used). These are my expectations when I see a library which advertises incremental parsing. I don't expect the library to internally buffer everything I feed it to if it advertises itself as an incremental parser. Yet, this is what I see in your example itself, which is using the DOM object itself as a buffer: https://github.com/vinniefalco/json/blob/04fe8c2ba8c3414e51a44017638688063e1... This is from your example, not from the library. The library at least offers basic_parser which does meet my expectations (for an incremental parser). JSON allows recursion and it is not really possible to parse arbitrary JSON values w/o at least a stack of states which both Trial.Protocol and your library use. So even if they aren't strictly in situ parsers either, this unavoidable violation is acceptable. And now back to your expectation, there are other concerns that I want to bring to the table. The property you want to offer is useful to streamable formats, but JSON has never been a streamable format. No libraries out there offer such functionality (please correct me if I'm wrong) and therefore, as an interchange format for a multitude of services, the messages designed around JSON have not been designed to rely on the property to stream any information. If streaming is required, you're better off with an extra communication channel to transfer the streamable data. The context switch to parse small chunks of independent JSON values would actually hurt the cache. With all that in mind, even if streamable JSONs was a property we wanted to tackle, the design of Trial.Protocol can easily accommodate such feature with very small changes. My HTTP (which is a streamable format) parser was inspired by Trial.Protocol and this was the first divergence I had to tackle (which was pretty easy to solve). -- Vinícius dos Santos Oliveira https://vinipsmaker.github.io/