Plus operator in boost qi doesn't work as expected - boost

Here's a sample code
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <iostream>
#include <string>
namespace qi = boost::spirit::qi;
template <typename Iterator>
struct input : qi::grammar<Iterator, unsigned()>
{
input() : input::base_type(start)
{
using qi::lit;
using qi::double_;
start = lit("ADD")
>> +(
+lit(" ")
>> double_
>> +lit(" ")
>> double_
);
}
qi::rule<Iterator, unsigned()> start;
};
int main()
{
input<std::string::const_iterator> input_parser; // Our grammar
std::string str = "ADD 1132.324 2342.234";
unsigned result;
std::string::const_iterator iter = str.begin();
std::string::const_iterator end = str.end();
bool r = qi::parse(iter, end, input_parser, result);
}
I get the following error.
/usr/local/include/boost/spirit/home/qi/nonterminal/rule.hpp:201:
instantiated from
‘boost::spirit::qi::rule&
boost::spirit::qi::rule::operator=(const Expr&)
[with Expr =
boost::proto::exprns_::expr&,
const
boost::proto::exprns_::expr&>,
1l>&, const
boost::spirit::terminal&>,
2l>&, const
boost::proto::exprns_::expr&>,
1l>&>, 2l>&, const
boost::spirit::terminal&>,
2l>&>, 1l>&>, 2l>, Iterator =
__gnu_cxx::__normal_iterator,
std::allocator > >, T1 =
unsigned int ()(), T2 =
boost::fusion::unused_type, T3 =
boost::fusion::unused_type, T4 =
boost::fusion::unused_type]’
mini.c++:34: instantiated from
‘input::input() [with
Iterator =
__gnu_cxx::__normal_iterator,
std::allocator > >]’
mini.c++:49: instantiated from here
/usr/local/include/boost/spirit/home/qi/operator/plus.hpp:62:
error: no type named ‘type’ in
‘struct
boost::spirit::traits::container_valueâ€
The weird part is, if I use the minus (-) operation i.e.
start = lit("ADD")
>> -(
+lit(" ")
>> double_
>> +lit(" ")
>> double_
);
... it compiles perfectly fine!
Compiling on gcc 4.3.4.

The attribute exposed by your start rule is unsigned, while the plus-parser exposes a container type holding the attributes of the wrapped elements. Here are the attribute propagation rules from the docs:
a: A --> +a: vector<A>
(i.e. if the parser a exposes an attribute of type A, a parser +a will expose a (standard) container holding instances of A, for instance std::vector<A>).
In your case, the embedded parser exposes double's. For this reason, you need to change the code in order to work as expected:
template <typename Iterator>
struct input : qi::grammar<Iterator, std::vector<double>()>
{
input() : input::base_type(start)
{
using qi::lit;
using qi::double_;
start = lit("ADD") >> +(+lit(" ") >> double_ >> +lit(" ") >> double_);
}
qi::rule<Iterator, std::vector<double>()> start;
};
int main()
{
input<std::string::const_iterator> input_parser; // Our grammar
std::string str = "ADD 1132.324 2342.234";
std::vector<double> result;
std::string::const_iterator iter = str.begin();
std::string::const_iterator end = str.end();
bool r = qi::parse(iter, end, input_parser, result);
}
But unfortunately, things are not as easy as it seems. The current version of Spirit has a bug preventing the code to work (even the soon to be released Boost V1.46 will still have this bug, but it's fixed in SVN trunk). The problem is, that plus does not 'flatten' the embedded elements into the provided container, resulting in every second double parsed by the code above to be lost.
The workaround is to avoid sequences exposing more than one attribute inside plus:
start = lit("ADD") >> +(+lit(" ") >> double_);
verifying afterwards that the number of elements parsed was even.
A side note: you seem to want to skip whitespace in between elements in the input. This can be achieved easier by using a skip parser:
template <typename Iterator>
struct input : qi::grammar<Iterator, std::vector<double>(), qi::space_type>
{
input() : input::base_type(start)
{
using qi::lit;
using qi::double_;
start = lit("ADD") >> +double_;
}
qi::rule<Iterator, std::vector<double>(), qi::space_type> start;
};
int main()
{
input<std::string::const_iterator> input_parser; // Our grammar
std::string str = "ADD 1132.324 2342.234";
std::vector<double> result;
std::string::const_iterator iter = str.begin();
std::string::const_iterator end = str.end();
bool r = qi::phrase_parse(iter, end, input_parser, qi::space, result);
}
which at the same time circumvents the issue described above.

Related

Expecting sequences and alternations of char_ parsers to synthesize a string

In the following test case, the alternation of one alpha and a sequence bombs with a long error dump basically saying static assertion failed: The parser expects tuple-like attribute type. Intuitively, I expected the entire rule to produce a string but that's not what happens. I either have to change the left-side of the alternation to +alpha (making both sides vectors) or go the path of semantic actions, at least for the lone char in the alternation (append to _val). Or, change the lone left-side char_ to string. Anyways, I can't figure out what's the proper simple way of parsing a string as trivial as this, any hint is appreciated. TIA.
#include <iostream>
#include <boost/spirit/home/x3.hpp>
namespace x3 = boost::spirit::x3;
namespace grammar {
using x3::char_;
using x3::alpha;
using x3::xdigit;
const auto x =
x3::rule< struct x_class, std::string > { "x" } =
char_('/') > alpha >> *(alpha | (char_('#') > xdigit));
} // namespace grammar
int main () {
std::string input{ "/Foobar#F" }, attr;
auto iter = input.begin ();
if (phrase_parse (iter, input.end (), grammar::x, x3::space, attr)) {
std::cout << attr << std::endl;
}
return 0;
}
I hate this behaviour too. Qi was much more natural in this respect.
I honestly don't always know how to "fix" it although
in this case it seems you can use raw[] - simplifying the grammar as well
sometimes it helps to avoid mixing operator> and operator>>
Here's what I'd do for your grammar:
Live On Coliru
#include <iostream>
#include <boost/spirit/home/x3.hpp>
namespace x3 = boost::spirit::x3;
namespace grammar {
const auto x =
x3::rule<struct x_class, std::string> { "x" } =
x3::raw [ '/' > x3::alpha >> *(x3::alpha | ('#' > x3::xdigit)) ];
}
int main () {
std::string input{ "/Foobar#F" }, attr;
auto iter = input.begin ();
if (phrase_parse (iter, input.end (), grammar::x, x3::space, attr)) {
std::cout << attr << std::endl;
}
}
Prints
/Foobar#F

How to define the AST for this boost spirit rule?

I have a more complex rule, but this one will suffice for this question (I hope). Consider the rule:
result = double_ >> *(char_ > int_);
where result is declared in terms of a struct result in namespace ast:
qi::rule<Iterator, ast::result(), qi::space_type> result;
Then how does ast::result have to look like?
According to the boost::spirit docs (http://www.boost.org/doc/libs/develop/libs/spirit/doc/html/spirit/abstracts/attributes/compound_attributes.html), the attribute of
char_ > int_ is tuple<char, int> or std::pair<char, int>.
So, I tried:
namespace ast
{
using second_type = std::vector<std::pair<char, int>>;
struct result
{
double first;
second_type second;
};
} // namespace ast
in addition to
BOOST_FUSION_ADAPT_STRUCT(
ast::result,
(double, first),
(ast::second_type, second)
)
But this gives me the compile error:
error: no matching function for call to 'std::pair<char, int>::pair(const char&)'
This rule is simple, creating the AST struct that the result will be stored in should be simple too... but how?
Here is a complete test program with my attempt:
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <iostream>
#include <string>
#include <vector>
namespace ast
{
using second_type = std::vector<std::pair<char, int>>;
struct result
{
double first;
second_type second;
friend std::ostream& operator<<(std::ostream& os, result const& result);
};
std::ostream& operator<<(std::ostream& os, second_type::value_type val)
{
return os << val.first << ' ' << val.second;
}
std::ostream& operator<<(std::ostream& os, result const& result)
{
os << result.first;
for (auto& i : result.second)
os << ' ' << i;
return os;
}
} // namespace ast
BOOST_FUSION_ADAPT_STRUCT(
ast::result,
(double, first),
(ast::second_type, second)
)
namespace client
{
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
template <typename Iterator>
class test_grammar : public qi::grammar<Iterator, ast::result(), qi::space_type>
{
private:
qi::rule<Iterator, ast::result(), qi::space_type> result;
public:
test_grammar() : test_grammar::base_type(result, "result_grammar")
{
using qi::double_;
using qi::char_;
using qi::int_;
result = double_ >> *(char_ > int_);
}
};
} // namespace client
int main()
{
std::string const input{"3.4 a 5 b 6 c 7"};
using iterator_type = std::string::const_iterator;
using test_grammar = client::test_grammar<iterator_type>;
namespace qi = boost::spirit::qi;
test_grammar program;
iterator_type iter{input.begin()};
iterator_type const end{input.end()};
ast::result out;
bool r = qi::phrase_parse(iter, end, program, qi::space, out);
if (!r || iter != end)
{
std::cerr << "Parsing failed." << std::endl;
return 1;
}
std::cout << "Parsed: " << out << std::endl;
}
SirGuy changed the AST to suit the default synthesized attributes. At the cost of, indeed complicating the AST.
However, you could leverage attribute compatibity rules by adapting std::pair. In fact, that is as simple as including 1 header:
#include <boost/fusion/include/std_pair.hpp>
Then, everything compiles without change, printing:
Parsed: 3.4 a 5 b 6 c 7
I made the following changes:
#include <boost/fusion/tuple.hpp>
using second_type = std::vector<boost::fusion::tuple<char, int>>;
std::ostream& operator<<(std::ostream& os, second_type::value_type val)
{
return os << boost::fusion::get<0>(val) << ' ' << boost::fusion::get<1>(val);
}
and the result compiled for me. There are most certainly other solutions available too.

Boost Spirit: Sub-grammar appending to string?

I am toying with Boost.Spirit. As part of a larger work I am trying to construct a grammar for parsing C/C++ style string literals. I encountered a problem:
How do I create a sub-grammar that appends a std::string() result to the calling grammar's std::string() attribute (instead of just a char?
Here is my code, which is working so far. (Actually I already got much more than that, including stuff like '\n' etc., but I cut it down to the essentials.)
#define BOOST_SPIRIT_UNICODE
#include <string>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
using namespace boost;
using namespace boost::spirit;
using namespace boost::spirit::qi;
template < typename Iterator >
struct EscapedUnicode : grammar< Iterator, char() > // <-- should be std::string
{
EscapedUnicode() : EscapedUnicode::base_type( escaped_unicode )
{
escaped_unicode %= "\\" > ( ( "u" >> uint_parser< char, 16, 4, 4 >() )
| ( "U" >> uint_parser< char, 16, 8, 8 >() ) );
}
rule< Iterator, char() > escaped_unicode; // <-- should be std::string
};
template < typename Iterator >
struct QuotedString : grammar< Iterator, std::string() >
{
QuotedString() : QuotedString::base_type( quoted_string )
{
quoted_string %= '"' >> *( escaped_unicode | ( char_ - ( '"' | eol ) ) ) >> '"';
}
EscapedUnicode< Iterator > escaped_unicode;
rule< Iterator, std::string() > quoted_string;
};
int main()
{
std::string input = "\"foo\u0041\"";
typedef std::string::const_iterator iterator_type;
QuotedString< iterator_type > qs;
std::string result;
bool r = parse( input.cbegin(), input.cend(), qs, result );
std::cout << result << std::endl;
}
This prints fooA -- the QuotedString grammar calls the EscapedUnicode grammar, which results in a char being added to the std::string attribute of QuotedString (the A, 0x41).
But of course I would need to generate a sequence of chars (bytes) for anything beyond 0x7f. EscapedUnicode would neet to produce a std::string, which would have to be appended to the string generated by QuotedString.
And that is where I've met a roadblock. I do not understand the things Boost.Spirit does in concert with Boost.Phoenix, and any attempts I have made resulted in lengthy and pretty much undecipherable template-related compiler errors.
So, how can I do this? The answer need not actually do the proper Unicode conversion; it's the std::string issue I need a solution for.
A few points applied:
please do not blanket using namespace in relation to highly generic code. ADL will ruin your day unless you control it
Operator %= is auto-rule assignment, meaning that automatic attribute propagation will be forced even in the presence of semantic actions. You don't want that because the attribute exposed by uint_parser will not be (correctly) automatically propagated if you want to encode into multi-byte string representation.
The input string
std::string input = "\"foo\u0041\"";
needed to be
std::string input = "\"foo\\u0041\"";
otherwise the compiler did the escape handling before the parser even runs :)
Here come the specific tricks for the meat of the task:
You will want to change the rule's declared attribute to something that Spirit will automatically "flatten" in simple sequences. E.g.
quoted_string = '"' >> *(escaped_unicode | (qi::char_ - ('"' | qi::eol))) >> '"';
Will not append because the first branch of the alternate results in a sequence of char, and the second in a single char. The following spelling of the equivalent:
quoted_string = '"' >> *(escaped_unicode | +(qi::char_ - ('"' | qi::eol | "\\u" | "\\U"))) >> '"';
subtly triggers the appending heuristic in Spirit, so we can achieve what we want without involving Semantic Actions.
The rest is straight-forward:
implement the actual encoding with a Phoenix function object:
struct encode_f {
template <typename...> struct result { using type = void; };
template <typename V, typename CP> void operator()(V& a, CP codepoint) const {
// TODO implement desired encoding (e.g. UTF8)
bio::stream<bio::back_insert_device<V> > os(a);
os << "[" << std::hex << std::showbase << std::setw(std::numeric_limits<CP>::digits/4) << std::setfill('0') << codepoint << "]";
}
};
boost::phoenix::function<encode_f> encode;
This you can then use like:
escaped_unicode = '\\' > ( ("u" >> uint_parser<uint16_t, 16, 4, 4>() [ encode(_val, _1) ])
| ("U" >> uint_parser<uint32_t, 16, 8, 8>() [ encode(_val, _1) ]) );
Because you mentioned you don't care about the specific encoding, I elected to encode the raw codepoint in 16bit or 32bit hex representation like [0x0041]. I pragmatically used Boost Iostreams which is capable of directly writing into the attribute's container type
Use BOOST_SPIRIT_DEBUG* macros
Live On Coliru
//#define BOOST_SPIRIT_UNICODE
//#define BOOST_SPIRIT_DEBUG
#include <string>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
// for demo re-encoding
#include <boost/iostreams/device/back_inserter.hpp>
#include <boost/iostreams/stream.hpp>
#include <iomanip>
namespace qi = boost::spirit::qi;
namespace bio = boost::iostreams;
namespace phx = boost::phoenix;
template <typename Iterator, typename Attr = std::vector<char> > // or std::string for that matter
struct EscapedUnicode : qi::grammar<Iterator, Attr()>
{
EscapedUnicode() : EscapedUnicode::base_type(escaped_unicode)
{
using namespace qi;
escaped_unicode = '\\' > ( ("u" >> uint_parser<uint16_t, 16, 4, 4>() [ encode(_val, _1) ])
| ("U" >> uint_parser<uint32_t, 16, 8, 8>() [ encode(_val, _1) ]) );
BOOST_SPIRIT_DEBUG_NODES((escaped_unicode))
}
struct encode_f {
template <typename...> struct result { using type = void; };
template <typename V, typename CP> void operator()(V& a, CP codepoint) const {
// TODO implement desired encoding (e.g. UTF8)
bio::stream<bio::back_insert_device<V> > os(a);
os << "[0x" << std::hex << std::setw(std::numeric_limits<CP>::digits/4) << std::setfill('0') << codepoint << "]";
}
};
boost::phoenix::function<encode_f> encode;
qi::rule<Iterator, Attr()> escaped_unicode;
};
template <typename Iterator>
struct QuotedString : qi::grammar<Iterator, std::string()>
{
QuotedString() : QuotedString::base_type(start)
{
start = quoted_string;
quoted_string = '"' >> *(escaped_unicode | +(qi::char_ - ('"' | qi::eol | "\\u" | "\\U"))) >> '"';
BOOST_SPIRIT_DEBUG_NODES((start)(quoted_string))
}
EscapedUnicode<Iterator> escaped_unicode;
qi::rule<Iterator, std::string()> start;
qi::rule<Iterator, std::vector<char>()> quoted_string;
};
int main() {
std::string input = "\"foo\\u0041\\U00000041\"";
typedef std::string::const_iterator iterator_type;
QuotedString<iterator_type> qs;
std::string result;
bool r = parse( input.cbegin(), input.cend(), qs, result );
std::cout << std::boolalpha << r << ": '" << result << "'\n";
}
Prints:
true: 'foo[0x0041][0x00000041]'

Boost spirit: assertion fails under Windows but not Linux. Why?

[Replaced code by a complete program and versions update]
The code below fails under Windows with the following message (sorry manualy translated from
French, I don't know how to force Visual C++ to work in English). This is
under Windows. It works under Linux. You can compile with
Linux: gcc LimaTokenizerParser.cpp -o LimaTokenizerPars -lboost_system-mt -lstdc++
Windows(fails): cl LimaTokenizerParser.cpp /Ic:\boost\path\include /EHsc
I use Boost 1.50 with Visual C++ 2010 under Windows and Boost 1.48
and gcc 4.6.3 under Linux.
The assertion in boost grammar.hpp is accompanied by this comment:
// If you see the assertion below failing then the start rule
// passed to the constructor of the grammar is not compatible with
// the grammar (i.e. it uses different template parameters).
but I don't understand how to understand it in my case...
Any idea ?
Regards,
Gaël
Code:
LimaTokenizerParser.cpp
#include "SpiritTokenizerParser.hpp"
#include <iostream>
#include <fstream>
void readStream(std::istream &is,std::string &dest)
{
while (is.good() && !is.eof())
{
const int bufferSize = 10240;
char buffer[bufferSize];
is.read(buffer,bufferSize);
const int size = is.gcount();
if (size>0)
dest.insert(dest.end(),buffer,buffer+size);
}
}
int main(int argc, char* argv[])
{
if (argc != 2)
{
std::cerr << "Needs exactly one argument" << std::endl;
return 1;
}
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
using ascii::space;
typedef std::string::const_iterator iterator_type;
typedef tokenizer<iterator_type> tokenizer;
// #ERROR DOES NOT WORK ON WINDOWS PORT
tokenizer tokenizer_parser;
std::string str;
std::ifstream file(argv[1]);
readStream(file, str);
std::string::const_iterator iter = str.begin();
std::string::const_iterator end = str.end();
tokenizer_automaton automaton;
bool r = false;
// #ERROR DOES NOT WORK ON WINDOWS PORT
r = phrase_parse(iter, end, tokenizer_parser, skipper, automaton);
if (r && iter == end)
{
std::cout << "Parsing succeeded: "<<automaton.size()<<" states" << std::endl;
}
else
{
std::string rest(iter, end);
std::cout << "Parsing failed. Stopped at: \": " << rest << "\"\n";
}
return 0;
}
SpiritTokenizerParser.hpp
#ifndef SPIRITTOKENIZERPARSER_HPP
#define SPIRITTOKENIZERPARSER_HPP
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix_core.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <boost/spirit/include/phoenix_object.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/fusion/include/io.hpp>
#include <string>
namespace qi = boost::spirit::qi;
namespace ascii = boost::spirit::ascii;
#define skipper qi::space | ascii::char_('#') >> *(ascii::char_ - qi::eol) >> qi::eol
typedef BOOST_TYPEOF(skipper) skipper_type;
enum transitions {
STORE,
FLUSH,
TOKEN,
EXIT,
LTOKEN
};
typedef std::vector<std::string> tokenizer_precondition;
typedef std::vector<std::string> tokenizer_postcondition;
struct tokenizer_transition
{
std::vector<tokenizer_precondition> preconditions;
std::vector<std::string> event;
std::vector<tokenizer_postcondition> postconditions;
transitions transition;
std::string target;
std::vector<std::string> statuses;
};
struct tokenizer_state
{
std::string id;
std::vector<tokenizer_transition> transitions;
};
typedef std::vector<tokenizer_state> tokenizer_automaton;
BOOST_FUSION_ADAPT_STRUCT(
tokenizer_transition,
(std::vector<tokenizer_precondition>, preconditions)
(std::vector<std::string>, event)
(std::vector<tokenizer_postcondition>, postconditions)
(transitions, transition)
(std::string, target)
(std::vector<std::string>, statuses)
)
BOOST_FUSION_ADAPT_STRUCT(
tokenizer_state,
(std::string, id)
(std::vector<tokenizer_transition>, transitions)
)
using ascii::space_type;
template <typename Iterator>
struct tokenizer : qi::grammar<Iterator, tokenizer_automaton(), skipper_type>
{
struct transitionsymbol_ : qi::symbols<char, unsigned>
{
// > is +1 store
// / is +1 flush (forget the current token)
// = is +1 token
// ^ is exit
transitionsymbol_()
{
add
(">", STORE)
("/", FLUSH)
("=", TOKEN)
("^", EXIT)
;
}
} transitionsymbol;
tokenizer() : tokenizer::base_type(start)
{
using qi::alnum;
using qi::lexeme;
using ascii::char_;
start %= *state ;
state %= '(' >> identifier >> ')' >> '{' >> *transition >> '}';
transition %= '-' >> *precondition >> event >> *postcondition >> transitionsymbol >> identifier >> -('(' >> identifier % ',' >> ')');
identifier %= lexeme[+(alnum | char_('_'))];
precondition %= '[' >> (identifier % '|') >> ']';
event %= identifier % '|';
postcondition %= identifier % '|';
}
qi::rule<Iterator, tokenizer_automaton(), skipper_type> start;
qi::rule<Iterator, tokenizer_state(), skipper_type> state;
qi::rule<Iterator, tokenizer_transition(), skipper_type> transition;
qi::rule<Iterator, std::string(), skipper_type> identifier;
qi::rule<Iterator, std::vector<std::string>(), skipper_type> precondition;
qi::rule<Iterator, std::vector<std::string>(), skipper_type> event;
qi::rule<Iterator, std::vector<std::string>(), skipper_type> postcondition;
};
//]
#endif // SPIRITTOKENIZERPARSER_HPP
Messages:
LimaTokenizerParser.cpp
c:\Program Files\boost\boost_1_50\include\boost/spirit/home/qi/nonterminal/grammar.hpp(77) : error C2664: 'boost::mpl::assertion_failed'<FF>: can not convert parameter 1 from 'boost::mpl::failed ************(__cdecl boost::spirit::qi::grammar<Iterator,T1,T2>::{ctor}::incompatible_start_rule::* ***********)(boost::spirit::qi::rule<Iterator,T1,boost::spirit::qi::alternative<Elements>>)' en 'boost::mpl::assert<false>::type'
with
[
Iterator=iterator_type,
T1=tokenizer_automaton (void),
T2=skipper_type,
Elements=boost::fusion::cons<boost::spirit::qi::char_class<boost::spirit::tag::char_code<boost::spirit::tag::space,boost::spirit::char_encoding::standard>>,boost::fusion::cons<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::literal_char<boost::spirit::char_encoding::ascii,false,false>,boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::difference<boost::spirit::qi::char_class<boost::spirit::tag::char_code<boost::spirit::tag::char_,boost::spirit::char_encoding::ascii>>,boost::spirit::qi::eol_parser>>,boost::fusion::cons<boost::spirit::qi::eol_parser,boost::fusion::nil>>>>,boost::fusion::nil>>
]
No constructor could take the source type, or the constructor overload resolution was ambiguous
e:\projets\amose\sources\lima_linguisticprocessing\src\linguisticprocessing\core\flattokenizer\SpiritTokenizerParser.hpp(89)<FF>: see reference to function template instantiation 'boost::spirit::qi::grammar<Iterator,T1,T2>::grammar<Iterator,tokenizer_automaton(void),boost::spirit::qi::alternative<Elements>,boost::spirit::unused_type,boost::spirit::unused_type>(const boost::spirit::qi::rule<Iterator,T1,boost::spirit::qi::alternative<Elements>> &,const std::string &)' en cours de compilation
with
[
Iterator=iterator_type,
T1=tokenizer_automaton (void),
T2=skipper_type,
Elements=boost::fusion::cons<boost::spirit::qi::char_class<boost::spirit::tag::char_code<boost::spirit::tag::space,boost::spirit::char_encoding::standard>>,boost::fusion::cons<boost::spirit::qi::sequence<boost::fusion::cons<boost::spirit::qi::literal_char<boost::spirit::char_encoding::ascii,false,false>,boost::fusion::cons<boost::spirit::qi::kleene<boost::spirit::qi::difference<boost::spirit::qi::char_class<boost::spirit::tag::char_code<boost::spirit::tag::char_,boost::spirit::char_encoding::ascii>>,boost::spirit::qi::eol_parser>>,boost::fusion::cons<boost::spirit::qi::eol_parser,boost::fusion::nil>>>>,boost::fusion::nil>>
]
TokenizerParser.hpp(88)<FF>: during the compilation of the member function ':: tokenizer tokenizer <iterator> (void)' model class
with
[
Iterator=iterator_type
]
LimaTokenizerParser.cpp(33)<FF>: see reference to class template instantiation 'tokenizer <iterator>' being compiled
with
[
Iterator=iterator_type
]
When posting to the Spirit list (yep, I saw this posted there), it is best to provide something minimal. Making something minimal helps a lot. It avoids the visual clutter that tends to waste time when understanding the code. It is just a matter of elimination anyway that anyone asking for support could do. Attached is what I mean by minimal. If I've seen something like this instead, it would have taken me just a few secs, instead of 10s of minutes to understand the problem.
Anyway, here's the code distilled and as minimal as it can be while still exhibiting the problem:
#include <boost/config/warning_disable.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <string>
namespace qi = boost::spirit::qi;
typedef qi::space_type skipper_type;
template <typename Iterator>
struct tokenizer : qi::grammar<Iterator, skipper_type>
{
tokenizer() : tokenizer::base_type(start)
{
}
qi::rule<Iterator, skipper_type> start;
};
int main()
{
typedef std::string::const_iterator iterator_type;
typedef tokenizer<iterator_type> tokenizer;
// #ERROR DOES NOT WORK ON WINDOWS PORT
tokenizer tokenizer_parser;
return 0;
}
I notice that if I rename "skipper_type" to "skipper_type_" (note trailing underscore). Then it compiles fine! This leads me to believe that it is an MSVC bug.
Solution: just change the name skipper_type.

Assigning data to a given element in a vector within a rule

I'm trying to set up a parser which, given a value, can assign it to a certain element of a vector, but I'm not entirely sure how to implement it.
Let's say the following piece of code parses the string (0){**+*+}. It should increment bar.a[0] once for every +, and bar.b[0] once for every *. The issue I'm encountering is that I'm not sure how to get a reference to a vector's element using _a:
#include <boost/spirit/include/qi.hpp>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
#include <boost/spirit/include/phoenix_operator.hpp>
#include <vector>
//The struct containing the vector.
struct testStruct {
std::vector<int> a, b;
};
BOOST_FUSION_ADAPT_STRUCT (
testStruct,
(std::vector<int>, a)
(std::vector<int>, b)
)
namespace qi = boost::spirit::qi;
namespace phoenix = boost::phoenix;
namespace ascii = boost::spirit::ascii;
template<typename Iterator>
struct foo : qi::grammar<Iterator, testStruct(), qi::locals<unsigned>, ascii::space_type> {
foo() : foo::base_type(start) {
using namespace qi::labels;
using qi::eps;
using qi::lit;
using qi::uint_;
using phoenix::at_c;
start = lit('(')
>> uint_ [_a = _1]
>> ')'
>> '{'
>> starsOrPlus(
at_c<_a>(at_c<0>(_val)), //This is where I'm not sure what to do.
at_c<_a>(at_c<1>(_val))
)
>> '}'
;
starsOrPlus = eps [_r1 = 0]
[_r2 = 0]
>> (
* (
(
+lit('+') [_r1 += 1]
)
^ (
+lit('*') [_r2 += 1]
)
)
)
;
}
qi::rule<Iterator, testStruct(), qi::locals<unsigned>, ascii::space_type> start;
//Parses stars and pluses. Sets the first uint8_t to the number of *, and the
//second to the number of +.
qi::rule<Iterator, void(int&, int&), ascii::space_type> starsOrPlus;
};
//Main program
int main() {
std::string testString = "(2){**++*+}";
typedef foo<std::string::const_iterator> foo;
foo grammar;
testStruct bar;
std::string::const_iterator iter = testString.begin();
std::string::const_iterator end = testString.end();
bool parsed = phrase_parse(iter, end, grammar, ascii::space, bar);
if (parsed) {
//Do something with the data...
}
return 0;
}
This fails to compile with the following errors:
main.cpp||In constructor 'foo<Iterator>::foo()':|
main.cpp|36|error: 'boost::spirit::_a' cannot appear in a constant-expression|
main.cpp|36|error: no matching function for call to 'at_c(boost::phoenix::actor<boost::phoenix::composite<boost::phoenix::at_eval<0>, boost::fusion::vector<boost::spirit::attribute<0>, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_> > >&)'|
main.cpp|37|error: 'boost::spirit::_a' cannot appear in a constant-expression|
main.cpp|37|error: no matching function for call to 'at_c(boost::phoenix::actor<boost::phoenix::composite<boost::phoenix::at_eval<1>, boost::fusion::vector<boost::spirit::attribute<0>, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_, boost::fusion::void_> > >&)'|
So, clearly, I can't use a placeholder value within at_c. I'm also aware that, even if I could, there would also be the issue of re-sizing the vector if the given position is out of range.
How would I implement something like this? Am I just going about this entirely the wrong way?
This should get you started:
#include <vector>
#include <string>
#include <iostream>
#include <boost/fusion/include/adapt_struct.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>
#include <boost/spirit/include/phoenix_fusion.hpp>
struct testStruct
{
std::vector<int> a, b;
};
BOOST_FUSION_ADAPT_STRUCT
(
testStruct,
(std::vector<int>, a)
(std::vector<int>, b)
)
namespace bp = boost::phoenix;
namespace bs = boost::spirit;
namespace bsq = bs::qi;
template<typename Iterator>
struct foo : bsq::grammar<Iterator, testStruct(), bsq::locals<unsigned> >
{
foo() : foo::base_type(start)
{
using namespace bs::labels;
using bp::at_c;
using bp::resize;
using bs::lit;
using bs::uint_;
start
= '('
>> uint_
[
_a = _1,
resize(at_c<0>(_val), _1 + 1),
resize(at_c<1>(_val), _1 + 1)
]
>> "){"
>> starsOrPlus(at_c<0>(_val)[_a], at_c<1>(_val)[_a])
>> '}'
;
starsOrPlus
= *(
lit('+')[_r1 += 1]
| lit('*')[_r2 += 1]
)
;
}
bsq::rule<Iterator, testStruct(), bsq::locals<unsigned> > start;
bsq::rule<Iterator, void(int&, int&)> starsOrPlus;
};
void printvec(std::vector<int> const& vec)
{
bool first = true;
for (std::vector<int>::const_iterator it = vec.begin(), it_end = vec.end();
it != it_end;
++it)
{
if (first)
first = false;
else
std::cout << ", ";
std::cout << *it;
}
}
int main()
{
foo<std::string::const_iterator> grammar;
testStruct bar;
std::string const input = "(2){**++*}";
std::string::const_iterator first = input.begin(), last = input.end();
if (bsq::parse(first, last, grammar, bar) && first == last)
{
std::cout << "bar.a: ";
printvec(bar.a);
std::cout << "\nbar.b: ";
printvec(bar.b);
std::cout << '\n';
}
else
std::cout << "parse failed\n";
}
The notable changes here are:
The vectors inside of testStruct must be resized to a size sufficient for the desired index to be valid
operator[] is used instead of boost::phoenix::at_c to access the index of a vector, just as one would in "normal" code
Note that I took out the skip parser to simplify things (and because it didn't appear to be necessary); add it back if you need to -- it had no real relevance here.

Resources