Re: [Boost-users] [Spirit] Qi lexeme only taking the first word

this has an attribute of vector>> or something similar.

Where are you getting that from? It makes no sense whatsoever given the struct full_it_t { std::string val; }, which is similarly mapped, and ruled, I've just had a look and the synthesized attribute is actually boost::fusion::vector,std::vector >,std::allocator > > > > Cleaned up that is, boost::fusion::vector>> So almost exactly what I said. You can get this using spirit::traits::attribute_of. It makes perfect sense. This is the synthesized attribute of the RHS parser, not the attribute you have passed it. Your struct full_it_t { std::string val; } appears to spirit as boost::fusion::vectorstd::string this is precisely what the BOOST_FUSION_ADAPT_STRUCT is for. The RHS parsers is the result of the various parsers you use. In your case... Sequence Parser (a >> b) Expression --> Attribute a: A, b: B --> (a >> b): tuple Kleene Parser (*a) Expression --> Attribute a: A --> *a: vector<A> Character Parser (char_, lit) Expression --> Attribute ns::char_ --> The character type of the Character Encoding Namespace, ns. With your expression, where id's attribute is string, you have id >> *(char_('.') >> id) gives tuple>> This is then assigned to the target attribute of the LHS rule. which in your setup is, boost::fusion::vector ________________________________ From: Boost-users on behalf of Michael Powell via Boost-users Sent: 07 November 2018 02:08 To: boost-users@lists.boost.org Cc: Michael Powell Subject: Re: [Boost-users] [Spirit] Qi lexeme only taking the first word On Tue, Nov 6, 2018 at 8:12 PM rmawatson rmawatson wrote:

It's been a long while since I've used spirit::qi. But What it looks like is happeneing in your setup is something liek this,

When you have:

qi::rule full_id;

the attribute is vector<string>

When it matches

id >> *(char_('.') >> id)

this has an attribute of vector>> or something similar.

Where are you getting that from? It makes no sense whatsoever given the struct full_it_t { std::string val; }, which is similarly mapped, and ruled, etc.

spirit appears to compare your target attribute with the synthesised attribute of the parser and for any (trailing?) members of the synthesised attribute that do not match in your attribute, it marks them as unused_type and they are not assigned.

Would I need to do some grouping or something to persuade Spirit to treat the struct as I've defined and adapted it?

You can see overload of assign to is used in your example if you breakpoint it -> boost\spirit\home\qi\detail\assign_to.hpp line 399.

It appears in boost\spirit\home\qi\operator\sequence_base.hpp line 74, where the predicate traits::attribute_not_unused is passed to spirit::any_if (boost\spirit\home\support\algorithm\any_if.hpp line 186.) it will basically discard attributes where the LHS sequence is not matched with the RHS.

You can see this in your example by adding an additional member to

struct full_id_t { std::string val; std::vectorstd::string others; };

BOOST_FUSION_ADAPT_STRUCT(AST::full_id_t, val, others)

Your missing bits will appear in this std::vector, as they are now not silently discarded. http://coliru.stacked-crooked.com/a/51f16c6deff45309

I think what the problem fundamentally is the attribute propagation is different when you have a string to when you have a vector<string> as in your two examples. the first kicks in whatever logic exists to flatten the LHS attribute into a string, the second takes the first element, assigns it and marks the rest as unused.

One thing you can do is use qi::asstd::string()[ id >> *(char_('.') >> id) ] to force conversion of synthesised attribute to a string to happen before it is assigned to your attribute. http://coliru.stacked-crooked.com/a/6a060343a390f037

I've only had a quick look and this is pretty half hearted analysis. You'll really have to dig deep to find out exactly what is going on, but I suspect this is somewhat along the right lines. ________________________________ From: Boost-users on behalf of Michael Powell via Boost-users Sent: 06 November 2018 23:03 To: boost-users@lists.boost.org Cc: Michael Powell Subject: Re: [Boost-users] [Spirit] Qi lexeme only taking the first word

On Tue, Nov 6, 2018 at 5:40 PM Michael Powell wrote:

...

On Tue, Nov 6, 2018 at 5:01 PM Michael Powell wrote:

...

Hello,

I've got a couple of rules that are perplexing to me. First,

rule id %= lexeme[qi::alpha >> *char_("A-Za-z0-9_")];

In and of itself, id is working fine. Then I've got a "full id":

rule full_id %= id >> *(char_('.') >> id);

Where:

struct full_id_t { std::string val; };

full_id_t::val is quite intentional for reasons elsewhere in the grammar.

The perplexity comes in, it seems lexeme is only shaving off the first word as the val.

For instance, parsing "two.oranges.red.test", I receive back "two" in the AST.

Perhaps I should defer specifying the lexeme part of id until later?

I elaborated a little on the "simple" full id sub-grammar, but I cannot repro using the GCC compiler. I'm wondering if this has anything to do with the VS2017 fpos issue?

http://coliru.stacked-crooked.com/a/adeb42ce2f19b0fd

Or there may be insufficient context in the web compiler to adequately demo.

I got a repro:

http://coliru.stacked-crooked.com/a/069a44296240be7e

Although the reasons as to why I do not know.

It is a difference in attribute synthesis. When full_id synthesizes a std::string(), the conversion to full_id_t() "just works" magically. I'm guessing by happy accident based on the std::string val being the only member (adaptation, etc).

But when I change the synthesis to be its "true" type, that is, AST::full_id_t(), suddenly I see the same behavior.

Really and truly, I do not know why. Everything else being equal why would one approach be any different than the other?

Anyone with some Spirit, Fusion, AST, insights?

Thanks!

For now, I'll run with it as has been exposed here, but it's a bit troubling to me not knowing the difference.

...

...

Thoughts? Suggestions?

Thank you!

Best regards,

Michael Powell

---

Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users

Boost-users mailing list Boost-users@lists.boost.org https://lists.boost.org/mailman/listinfo.cgi/boost-users