[Proto]: How to use complex grammars in a domain/extension
Hi, I am trying to create a DSEL with Boost.Proto, which (among other things) should allow expressions like i == j + 3 && t != "foo" and want to prevent disallowed operators to be generated by Proto. Please consider the code below. // ----------------------------------------------------------------------- #include<boost/proto/proto.hpp> using namespace boost; typedef proto::terminal<int>::type terminal; struct addition: proto::or_ < terminal, proto::plus<addition, addition> > {}; struct equation: proto::equal_to<addition, addition> {}; struct condition: proto::or_ < equation, proto::logical_and<equation, equation> > {}; template<class Expr> struct extension; struct my_domain: proto::domain < proto::pod_generator< extension>, condition > {}; template<class Expr> struct extension { BOOST_PROTO_EXTENDS( Expr , extension<Expr> , my_domain ) }; int main() { terminal c; BOOST_PROTO_ASSERT_MATCHES(c + c, addition); // OK BOOST_PROTO_ASSERT_MATCHES(c == c, equation); // OK BOOST_PROTO_ASSERT_MATCHES(c == c, condition); // OK BOOST_PROTO_ASSERT_MATCHES(c == c&& c == c, condition); // OK extension<terminal> i; // I was hoping for the following to be legal i + i; // ERROR: operator+ not found i == i; // ERROR: operator== not found i == i&& i == i; // ERROR: operator== not found } // ----------------------------------------------------------------------- In my_domain, I can replace condition by addition, which allows i + i, but then the == operator is still missing (and the &&, too, probably). How can I solve this? Thanks and regards, Roland
On 20/09/10 19:59, Roland Bock wrote:
Hi,
I am trying to create a DSEL with Boost.Proto, which (among other things) should allow expressions like
i == j + 3 && t != "foo"
and want to prevent disallowed operators to be generated by Proto.
Please consider the code below.
// ----------------------------------------------------------------------- #include<boost/proto/proto.hpp>
using namespace boost;
typedef proto::terminal<int>::type terminal;
struct addition: proto::or_ < terminal, proto::plus<addition, addition>
{};
struct equation: proto::equal_to<addition, addition> {};
struct condition: proto::or_ < equation, proto::logical_and<equation, equation>
{};
In my_domain, I can replace condition by addition, which allows i + i, but then the == operator is still missing (and the &&, too, probably).
i+i doesn't match your grammar at all. an equation is addition == addition which is not matchable by i+i if equation was: struct equation: proto::or_< addition, proto::equal_to<addition, addition> > {}; then it'll work.
On 09/20/2010 08:04 PM, joel falcou wrote:
On 20/09/10 19:59, Roland Bock wrote:
Hi,
I am trying to create a DSEL with Boost.Proto, which (among other things) should allow expressions like
i == j + 3 && t != "foo"
and want to prevent disallowed operators to be generated by Proto.
Please consider the code below.
// ----------------------------------------------------------------------- #include<boost/proto/proto.hpp>
using namespace boost;
typedef proto::terminal<int>::type terminal;
struct addition: proto::or_ < terminal, proto::plus<addition, addition>
{};
struct equation: proto::equal_to<addition, addition> {};
struct condition: proto::or_ < equation, proto::logical_and<equation, equation>
{};
In my_domain, I can replace condition by addition, which allows i + i, but then the == operator is still missing (and the &&, too, probably).
i+i doesn't match your grammar at all. an equation is addition == addition which is not matchable by i+i if equation was:
struct equation: proto::or_< addition, proto::equal_to<addition, addition> > {};
then it'll work. Joel,
ok, got it! I needed to prevent i && i == i // that's done and I need to find a bug in the larger SQL grammar, but I am getting there. Proto takes some time to comprehend, but it sure is fun to use it :-) Regards, Roland
On 09/20/2010 08:04 PM, joel falcou wrote:
then it'll work.
Sorry, I need to ask again: // ------------------------------------------------- #include<boost/proto/proto.hpp> using namespace boost; typedef proto::terminal<proto::_>::type terminal; struct addition: proto::or_ < terminal, proto::plus<addition, addition> > {}; struct equation: proto::or_ < proto::equal_to<addition, addition> > {}; template<class Expr> struct extension; struct my_domain : proto::domain< proto::pod_generator< extension> , equation > {}; template<class Expr> struct extension { BOOST_PROTO_EXTENDS( Expr , extension<Expr> , my_domain ) }; int main() { extension<proto::terminal<int>::type> i; i == i; // Error } // ------------------------------------------------- I know it has to be obvious, but I just can't see it... Regards, Roland
On 20/09/10 22:57, Roland Bock wrote:
typedef proto::terminal<proto::_>::type terminal;
struct addition: proto::or_ < terminal, proto::plus<addition, addition>
{};
terminal is a terminal type, not a terminal grammar: struct addition : proto::or_< proto::terminal<proto::_> , proto::plus<addition,addition> > {};
On 09/21/2010 08:54 AM, Joel Falcou wrote:
On 20/09/10 22:57, Roland Bock wrote:
typedef proto::terminal<proto::_>::type terminal;
struct addition: proto::or_ < terminal, proto::plus<addition, addition>
{};
terminal is a terminal type, not a terminal grammar:
struct addition : proto::or_< proto::terminal<proto::_> , proto::plus<addition,addition>
{}; That did not make a difference:
// ----------------------------------------------------------------- #include<complex> #include<boost/proto/proto.hpp> using namespace boost; struct addition : proto::or_ < proto::terminal<proto::_> , proto::plus<addition,addition> > {}; struct equation: proto::or_ < proto::equal_to<addition, addition> > {}; template<class Expr> struct extension; struct my_domain : proto::domain< proto::pod_generator< extension> , equation > {}; template<class Expr> struct extension { BOOST_PROTO_EXTENDS( Expr , extension<Expr> , my_domain ) }; int main() { extension<proto::terminal<int>::type> i; i == i; // ERROR } // ----------------------------------------------------------------- Regards, Roland
The following code works here: // ----------------------------------------------------------------------- #include<boost/proto/proto.hpp> using namespace boost; typedef proto::terminal<int>::type terminal; struct addition: proto::or_ < proto::terminal<proto::_>, proto::plus<addition, addition> > {}; struct equation: proto::or_ < addition, proto::equal_to<addition, addition> > {}; template<class Expr> struct extension; struct my_domain: proto::domain < proto::pod_generator< extension>, equation > {}; template<class Expr> struct extension { BOOST_PROTO_EXTENDS( Expr , extension<Expr> , my_domain ) }; int main() { extension<terminal> i; i == i; } Seems proto::or_ wiht *1* condition acts weird. Eric, am I right or am I hallucinating and missing the obvious ?
It bugged me for a while too ... the only fix I can come with is the same as yours joel : struct equation: proto::or_ < addition ,proto::equal_to<addition, addition>
{};
But it's wrong, as in the grammar is right w/o it ... So I figured out it's a "bug" in proto? Or I'm missing the obvious too...
Mathieu - wrote:
It bugged me for a while too ... the only fix I can come with is the same as yours joel :
struct equation: proto::or_ < addition ,proto::equal_to<addition, addition>
{};
But it's wrong, as in the grammar is right w/o it ... So I figured out it's a "bug" in proto? Or I'm missing the obvious too...
other "fix" (workaround): struct equation: proto::or_ < proto::terminal<proto::_>, proto::equal_to<addition, addition> > {}; I think the problem is, that proto's operator enable/disable facilities only "saw" the equal_to operator, but obviously no terminal type for it's right and left hand side. It looks to me as a bug in proto as well. because with my "fix" something like this: i == i + i didn't work anymore. So, it looks like proto's enable facilities do not look "deep enough" in the grammar. that is, it somehow misses the template arguments to equal_to.
On 09/21/2010 09:49 AM, Joel Falcou wrote:
The following code works here:
// ----------------------------------------------------------------------- [...] Yes, I know, but that is not the grammar I need. I want to combine my terminal similar to std::string, so that:
std::string a, b; a; // ok a + b; // ok a == b; // ok a == b + b; // ok a && a == b; // ERROR
Seems proto::or_ wiht *1* condition acts weird. Eric, am I right or am I hallucinating and missing the obvious ?
You can remove the proto::or_ like struct equation: proto::equal_to<addition, addition> {}; It does not change the result. So it does not look like a problem in proto::or_, but maybe a bug in how proto determines the operators of extensions? Regards, Roland
On 9/21/2010 4:11 AM, Roland Bock wrote:
On 09/21/2010 09:49 AM, Joel Falcou wrote:
The following code works here:
// ----------------------------------------------------------------------- [...] Yes, I know, but that is not the grammar I need. I want to combine my terminal similar to std::string, so that:
std::string a, b;
a; // ok a + b; // ok a == b; // ok a == b + b; // ok a && a == b; // ERROR
Seems proto::or_ wiht *1* condition acts weird. Eric, am I right or am I hallucinating and missing the obvious ?
You can remove the proto::or_ like
struct equation: proto::equal_to<addition, addition> {};
It does not change the result. So it does not look like a problem in proto::or_, but maybe a bug in how proto determines the operators of extensions?
struct equation: proto::or_< proto::equal_to<addition, addition>, proto::terminal<proto::_> > {}; ? -- Eric Niebler BoostPro Computing http://www.boostpro.com
Roland Bock wrote:
On 09/21/2010 08:54 AM, Joel Falcou wrote:
On 20/09/10 22:57, Roland Bock wrote:
typedef proto::terminal<proto::_>::type terminal;
struct addition: proto::or_ < terminal, proto::plus<addition, addition>
{};
terminal is a terminal type, not a terminal grammar:
struct addition : proto::or_< proto::terminal<proto::_> , proto::plus<addition,addition>
{}; That did not make a difference:
// ----------------------------------------------------------------- <snip>
Solved the mistery. here is the code, explanation comes afterward: #include<boost/proto/proto.hpp> using namespace boost; typedef proto::terminal<int>::type terminal; struct addition: proto::or_ < proto::plus<addition, addition>, proto::terminal<proto::_> > {}; struct equation: proto::or_ < proto::equal_to<addition, addition> > {}; template<class Expr> struct extension; struct my_domain: proto::domain < proto::pod_generator< extension>, // we need both grammars in our domain grammar proto::or_<equation, addition>, proto::default_domain > {}; template<class Expr> struct extension { BOOST_PROTO_EXTENDS( Expr , extension<Expr> , my_domain ) }; template <typename Grammar, typename Expr> void matches(Expr const&) { std::cout << std::boolalpha << proto::matches<Expr, Grammar>::value << "\n"; } int main() { extension<terminal> i; extension<terminal> j; matches<equation>(i); // 1) false matches<equation>(i == j); // 2) true matches<equation>(i == i + i); // 3) true matches<equation>(i + i == i); // 4) true matches<equation>(i + i == i + i); // 5) true matches<equation>(i + i); // 6) false } Ok, what happened, why does this work, and why didn't the previous attempt work: the equation grammar obviously doesn't match case 1) and 6). And that is exactly why proto didn't generate any operator+ overload. In order for a proto overload to be created the following conditions must be true: 1) the operands must be in a compatible domain 2) the left hand operand and the right hand operand must match the grammar specified in the domain 3) the resulting expression must match the grammar specified in the domain. In our example 1) is always true. 2) and 3) are not true for expressions like 1) and 6). That means, that any binary operator gets disabled whenever the LHS or the RHS is a terminal<_>. because of the equation grammar, terminals or additions can not stand alone, that is why operator creation failed. Makes sense! In order to enable the operators, we just say, our domain can have addition _or_ equation. That's it, operator+ and operator== get enabled. We can still detect invalid expression with proto::matches! So, everything works as expected!
Regards,
Roland
On 09/21/2010 11:55 AM, Thomas Heller wrote:
<snip> Solved the mistery. here is the code, explanation comes afterward:
<snip> So, everything works as expected!
Thomas, wow, this was driving me crazy, thanks for solving and explaining in all the detail! :-) I am still not sure if this isn't a conceptual problem, though: The "equation" grammar is perfectly OK. But without or-ing it with the "addition" grammar, the extension does not allow ANY valid expression to be created. I wonder if that is a bug or a feature? Thanks and regards, Roland
#include<boost/proto/proto.hpp>
using namespace boost;
typedef proto::terminal<int>::type terminal;
struct addition: proto::or_ < proto::plus<addition, addition>, proto::terminal<proto::_> > {};
struct equation: proto::or_ < proto::equal_to<addition, addition> > {};
template<class Expr> struct extension;
struct my_domain: proto::domain < proto::pod_generator< extension>, // we need both grammars in our domain grammar proto::or_<equation, addition>, proto::default_domain > {};
template<class Expr> struct extension { BOOST_PROTO_EXTENDS( Expr , extension<Expr> , my_domain ) };
template<typename Grammar, typename Expr> void matches(Expr const&) { std::cout<< std::boolalpha << proto::matches<Expr, Grammar>::value<< "\n"; }
int main() { extension<terminal> i; extension<terminal> j;
matches<equation>(i); // 1) false matches<equation>(i == j); // 2) true matches<equation>(i == i + i); // 3) true matches<equation>(i + i == i); // 4) true matches<equation>(i + i == i + i); // 5) true matches<equation>(i + i); // 6) false }
(proto lists members, see this thread for the rest of the discussion, which has been happening on -users: http://lists.boost.org/boost-users/2010/09/62747.php) On 9/21/2010 9:19 AM, Roland Bock wrote:
On 09/21/2010 11:55 AM, Thomas Heller wrote:
<snip> Solved the mistery. here is the code, explanation comes afterward:
<snip> So, everything works as expected!
Thomas,
wow, this was driving me crazy, thanks for solving and explaining in all the detail! :-)
I am still not sure if this isn't a conceptual problem, though:
The "equation" grammar is perfectly OK. But without or-ing it with the "addition" grammar, the extension does not allow ANY valid expression to be created. I wonder if that is a bug or a feature?
It's a feature. Imagine this grammar: struct IntsOnlyPlease : proto::or_< proto::terminal<int> , proto::nary_expr<proto::_, proto::vararg<IntsOnlyPlease> > > {}; And a integer terminal "i" in a domain that obeys this grammar. Now, what should this do: i + "hello!"; You want it to fail because if it doesn't, you would create an expression that doesn't conform to the domain's grammar. Right? Proto accomplishes this by enforcing that all operands must conform to the domain's grammar *and* the resulting expression must also conform. Otherwise, the operator is disabled. Anything else would be fundamentally broken. -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 9/21/2010 9:51 AM, Eric Niebler wrote:
(proto lists members, see this thread for the rest of the discussion, which has been happening on -users: http://lists.boost.org/boost-users/2010/09/62747.php)
On 9/21/2010 9:19 AM, Roland Bock wrote:
On 09/21/2010 11:55 AM, Thomas Heller wrote:
<snip> Solved the mistery. here is the code, explanation comes afterward:
<snip> So, everything works as expected!
Thomas,
wow, this was driving me crazy, thanks for solving and explaining in all the detail! :-)
I am still not sure if this isn't a conceptual problem, though:
The "equation" grammar is perfectly OK. But without or-ing it with the "addition" grammar, the extension does not allow ANY valid expression to be created. I wonder if that is a bug or a feature?
It's a feature. Imagine this grammar:
struct IntsOnlyPlease : proto::or_< proto::terminal<int> , proto::nary_expr<proto::_, proto::vararg<IntsOnlyPlease> > > {};
And a integer terminal "i" in a domain that obeys this grammar. Now, what should this do:
i + "hello!";
You want it to fail because if it doesn't, you would create an expression that doesn't conform to the domain's grammar. Right? Proto accomplishes this by enforcing that all operands must conform to the domain's grammar *and* the resulting expression must also conform. Otherwise, the operator is disabled.
Anything else would be fundamentally broken.
This explanation is incomplete. Naturally, this operator+ would be disabled anyway because the resulting expression doesn't conform to the grammar regardless of whether the LHS and RHS conform. It's a question of *when* the operator gets disabled. For a full explanation, see this bug report: https://svn.boost.org/trac/boost/ticket/2407 The answer is simple and logically consistent: make sure *every* valid expression in your domain (including lone terminals) is accounted for by your grammar. -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 09/21/2010 04:01 PM, Eric Niebler wrote:
On 9/21/2010 9:51 AM, Eric Niebler wrote:
(proto lists members, see this thread for the rest of the discussion, which has been happening on -users: http://lists.boost.org/boost-users/2010/09/62747.php)
On 9/21/2010 9:19 AM, Roland Bock wrote:
On 09/21/2010 11:55 AM, Thomas Heller wrote:
<snip> Solved the mistery. here is the code, explanation comes afterward:
<snip> So, everything works as expected!
Thomas,
wow, this was driving me crazy, thanks for solving and explaining in all the detail! :-)
I am still not sure if this isn't a conceptual problem, though:
The "equation" grammar is perfectly OK. But without or-ing it with the "addition" grammar, the extension does not allow ANY valid expression to be created. I wonder if that is a bug or a feature?
It's a feature. Imagine this grammar:
struct IntsOnlyPlease : proto::or_< proto::terminal<int> , proto::nary_expr<proto::_, proto::vararg<IntsOnlyPlease> > > {};
And a integer terminal "i" in a domain that obeys this grammar. Now, what should this do:
i + "hello!";
You want it to fail because if it doesn't, you would create an expression that doesn't conform to the domain's grammar. Right? Proto accomplishes this by enforcing that all operands must conform to the domain's grammar *and* the resulting expression must also conform. Otherwise, the operator is disabled.
Anything else would be fundamentally broken.
This explanation is incomplete. Naturally, this operator+ would be disabled anyway because the resulting expression doesn't conform to the grammar regardless of whether the LHS and RHS conform. It's a question of *when* the operator gets disabled. For a full explanation, see this bug report:
https://svn.boost.org/trac/boost/ticket/2407
The answer is simple and logically consistent: make sure *every* valid expression in your domain (including lone terminals) is accounted for by your grammar.
OK, it is sinking in, slowly :-) Maybe I missed the equivalent part in the documentation, but the list given by Thomas would make a very good section.
1) the operands must be in a compatible domain 2) the left hand operand and the right hand operand must match the grammar specified in the domain 3) the resulting expression must match the grammar specified in the domain.
Also, maybe as an illustrating example: This grammar proto::plus<proto::terminal<int>, proto::terminal<int> > used in a domain would not allow i + i; with i being an int-terminal. Thanks for all the answers! Regards from Munich, Roland
On 9/21/2010 10:19 AM, Roland Bock wrote:
On 09/21/2010 04:01 PM, Eric Niebler wrote:
The answer is simple and logically consistent: make sure *every* valid expression in your domain (including lone terminals) is accounted for by your grammar.
OK, it is sinking in, slowly :-)
Maybe I missed the equivalent part in the documentation, but the list given by Thomas would make a very good section.
1) the operands must be in a compatible domain 2) the left hand operand and the right hand operand must match the grammar specified in the domain 3) the resulting expression must match the grammar specified in the domain.
Yes, agreed. Something like this should be in the docs. Would you mind filing a trac ticket (http://svn.boost.org) so this doesn't get lost?
Also, maybe as an illustrating example:
This grammar
proto::plus<proto::terminal<int>, proto::terminal<int> >
used in a domain would not allow
i + i;
with i being an int-terminal.
Yes, that also should be noted.
Thanks for all the answers!
No problem. -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 9/21/2010 5:55 AM, Thomas Heller wrote:
Solved the mistery. here is the code, explanation comes afterward:
#include<boost/proto/proto.hpp>
using namespace boost;
typedef proto::terminal<int>::type terminal;
struct addition: proto::or_ < proto::plus<addition, addition>, proto::terminal<proto::_> > {};
struct equation: proto::or_ < proto::equal_to<addition, addition> > {};
template<class Expr> struct extension;
struct my_domain: proto::domain < proto::pod_generator< extension>, // we need both grammars in our domain grammar proto::or_<equation, addition>,
That will also work because addition (contrary to its name) will also match lone terminals.
proto::default_domain > {};
template<class Expr> struct extension { BOOST_PROTO_EXTENDS( Expr , extension<Expr> , my_domain ) };
template <typename Grammar, typename Expr> void matches(Expr const&) { std::cout << std::boolalpha << proto::matches<Expr, Grammar>::value << "\n"; }
int main() { extension<terminal> i; extension<terminal> j;
matches<equation>(i); // 1) false matches<equation>(i == j); // 2) true matches<equation>(i == i + i); // 3) true matches<equation>(i + i == i); // 4) true matches<equation>(i + i == i + i); // 5) true matches<equation>(i + i); // 6) false }
Ok, what happened, why does this work, and why didn't the previous attempt work:
the equation grammar obviously doesn't match case 1) and 6). And that is exactly why proto didn't generate any operator+ overload. In order for a proto overload to be created the following conditions must be true:
1) the operands must be in a compatible domain 2) the left hand operand and the right hand operand must match the grammar specified in the domain
Precisely.
3) the resulting expression must match the grammar specified in the domain.
Right-o.
In our example 1) is always true. 2) and 3) are not true for expressions like 1) and 6). That means, that any binary operator gets disabled whenever the LHS or the RHS is a terminal<_>. because of the equation grammar, terminals or additions can not stand alone, that is why operator creation failed. Makes sense! In order to enable the operators, we just say, our domain can have addition _or_ equation. That's it, operator+ and operator== get enabled. We can still detect invalid expression with proto::matches!
So, everything works as expected!
Thomas wins the prize. This is not a bug in Proto. -- Eric Niebler BoostPro Computing http://www.boostpro.com
On 9/21/2010 3:15 AM, Roland Bock wrote:
That did not make a difference:
// ----------------------------------------------------------------- #include<complex> #include<boost/proto/proto.hpp>
using namespace boost;
struct addition : proto::or_ < proto::terminal<proto::_> , proto::plus<addition,addition>
{};
struct equation: proto::or_ < proto::equal_to<addition, addition>
{};
template<class Expr> struct extension;
struct my_domain : proto::domain< proto::pod_generator< extension> , equation > {};
template<class Expr> struct extension { BOOST_PROTO_EXTENDS( Expr , extension<Expr> , my_domain ) };
int main() { extension<proto::terminal<int>::type> i;
Trouble here. The "equation" grammar only matches things of the form "x == y". The terminal "i" *by itself* is not of that form. You won't be able to use this terminal at all.
i == i; // ERROR
As expected. You need to change your grammar to allow lone terminals. -- Eric Niebler BoostPro Computing http://www.boostpro.com
participants (2)
-
Eric Niebler -
joel falcou -
Joel Falcou
-
Mathieu - -
Roland Bock -
Thomas Heller