Re: [boost] [Re] [Re] JSON Parsing Library for GSoC 14
There are many libraries that parse JSON to maps and vectors. IIRC Microsoft’s REST SDK includes a JSON library as well (http://casablanca.codeplex.com). There are at least 15 other C++ libraries like that and we don’t need to write another one. But there are only two libraries AFAIK that parse JSON to C++ structs using serialization adaptors: - https://github.com/Loki-Astari/ThorsSerializer - http://fossil.wanderinghorse.net/repos/nosjob/index.cgi/index I consider the ability to read and write C++ structs essential for any library admitted to boost. Shouldn't such a library be an extension to boost.serialization? Boost.Serialization already defines a way for each data structure to define which data should be written/read. The Boost.Serialization docs say that the library even offers methods & macros to output a class member together with its member variable name as a name value pair. Exactly what a JSON library would need. There has been a discussion about last year’s GSoC mentioning further desirable properties of a JSON library: http://boost.2283326.n4.nabble.com/JSON-Parser-GSoC-2013-td4645117.html Regards Sebastian -- Dr. Sebastian Theophil | stheophil@think-cell.com Senior Software Engineer We are looking for C++ Developers: http://www.think-cell.com/career think-cell Software GmbH | Chausseestr. 8/E | 10115 Berlin | Germany http://www.think-cell.com | phone +49 30 666473-10 | US phone +1 800 891 8091 Amtsgericht Berlin-Charlottenburg, HRB 85229 | European Union VAT Id DE813474306 Directors: Dr. Markus Hannebauer, Dr. Arno Schödl
On 03/03/2014 10:22 AM, Sebastian Theophil wrote:
But there are only two libraries AFAIK that parse JSON to C++ structs using serialization adaptors:
Here is one that integrates with Boost.Serialization: http://protoc.sourceforge.net/ You can parse JSON directly into C++ structures: std::string input("[1,2,3]"); struct my_struct { int x, y, z; } result; protoc::json::iarchive archive(input.begin(), input.end()); archive >> result; or std::string input("[1,2,3]"); std::vector<int> result; protoc::json::iarchive archive(input.begin(), input.end()); archive >> result; You can also parse generic JSON into a DOM based on boost::variant. I currently use dynamic-cpp: http://dynamic-cpp.googlecode.com/ Parsing into this DOM follows the same pattern as above: std::string input("[1,2,[3,4]]"); dynamic::var result; protoc::json::iarchive archive(input.begin(), input.end()); archive >> dom; The protoc project has a couple of other advantages: (1) it supports streaming, which means that you can parse or construct JSON without requiring all data to be in memory at once, and (2) it supports various binary encodings with a structure similar to JSON. I really ought to write the documentation of this library...
There has been a discussion about last year’s GSoC mentioning further desirable properties of a JSON library: http://boost.2283326.n4.nabble.com/JSON-Parser-GSoC-2013-td4645117.html
Indeed, and most of these desiderata still applies. With the protoc library, we can focus on the integration with other Boost libraries, such as Boost.Fusion, or replacing the JSON implementation of Boost.PropertyTree. Another suitable GSoC project (for the student who wants to do meta- programming) could be to write a more generic version of dynamic-cpp. I have already written parts of it.
2014-03-06 18:34 GMT+08:00 Bjorn Reese
On 03/03/2014 10:22 AM, Sebastian Theophil wrote:
But there are only two libraries AFAIK that parse JSON to C++ structs
using serialization adaptors:
Here is one that integrates with Boost.Serialization:
http://protoc.sourceforge.net/
You can parse JSON directly into C++ structures:
std::string input("[1,2,3]"); struct my_struct { int x, y, z; } result;
protoc::json::iarchive archive(input.begin(), input.end()); archive >> result;
or
std::string input("[1,2,3]"); std::vector<int> result;
protoc::json::iarchive archive(input.begin(), input.end()); archive >> result;
You can also parse generic JSON into a DOM based on boost::variant. I currently use dynamic-cpp:
http://dynamic-cpp.googlecode.com/
Parsing into this DOM follows the same pattern as above:
std::string input("[1,2,[3,4]]"); dynamic::var result;
protoc::json::iarchive archive(input.begin(), input.end()); archive >> dom;
The protoc project has a couple of other advantages: (1) it supports streaming, which means that you can parse or construct JSON without requiring all data to be in memory at once
I'm confused. Your json::iarchive takes iterators not istream. I see there's stream_oarchive but no stream_iarchive, something I missed? and (2) it supports various
binary encodings with a structure similar to JSON.
I really ought to write the documentation of this library...
After a glance at some of the tests, say, the 'person' example in oarchive_suite.cpp, I wonder why it outputs value as array instead of object: {"name": "Kant", "age": 127}
There has been a discussion about last year's GSoC mentioning further
desirable properties of a JSON library: http://boost.2283326.n4. nabble.com/JSON-Parser-GSoC-2013-td4645117.html
Indeed, and most of these desiderata still applies.
With the protoc library, we can focus on the integration with other Boost libraries, such as Boost.Fusion, or replacing the JSON implementation of Boost.PropertyTree.
Another suitable GSoC project (for the student who wants to do meta- programming) could be to write a more generic version of dynamic-cpp. I have already written parts of it.
What's the benefit of dynamic-cpp over Boost.Any? If you want more generic, I guess that's what Boost.TypeErasure is for.
On 03/06/2014 02:35 PM, TONGARI J wrote:
I'm confused. Your json::iarchive takes iterators not istream. I see there's stream_oarchive but no stream_iarchive, something I missed?
I have not needed stream_iarchive so far. But you are right, it should be added.
After a glance at some of the tests, say, the 'person' example in oarchive_suite.cpp, I wonder why it outputs value as array instead of object:
{"name": "Kant", "age": 127}
NVP are intended for XML tags. As JSON has no tags, the name is ignored. This was an arbitrary design choice that I am willing to reconsider.
Another suitable GSoC project (for the student who wants to do meta- programming) could be to write a more generic version of dynamic-cpp. I have already written parts of it.
What's the benefit of dynamic-cpp over Boost.Any? If you want more generic, I guess that's what Boost.TypeErasure is for.
Good question, and you can add Boost.Variant to that list. Alex Fabijanic has written a good two-part article on the topic in ACCU Overload: http://accu.org/index.php/journals/1855 http://accu.org/index.php/journals/1841 With regards to using dynamic-cpp (or a generalization thereof) as a JSON DOM, when you read a generic JSON document, you normally want to investigate the type of its elements. That is a challenge when the type has been erased. I personally use dynamic-cpp because I prefer its syntax.
On 03/06/2014 04:38 PM, Bjorn Reese wrote:
On 03/06/2014 02:35 PM, TONGARI J wrote:
I'm confused. Your json::iarchive takes iterators not istream. I see there's stream_oarchive but no stream_iarchive, something I missed?
I have not needed stream_iarchive so far. But you are right, it should be added.
It just occurred to me that the term "streaming" is ambiguous. When we talk about streaming parsers there are two interpretations: 1. The parser will read from an input stream and deliver the results in a DOM (or via a builder design pattern like SAX-style events.) The parser may block while waiting for the next data on the input stream. 2. The parser only returns the next element, so you have to put a loop around it to get a full parse. This is the StAX-style parsing [1]. When I mentioned that protoc is a streaming parser, I was thinking about the second category, whereas the stream archives belong to the first category, so I see the confusion now. I am going to change my vocabulary accordingly and refer to the second category as pull parsers from now on. [1] http://en.wikipedia.org/wiki/StAX
participants (3)
-
Bjorn Reese
-
Sebastian Theophil
-
TONGARI J