Re: [Boost-users] [xpressive] Using it with input iterators or how far the backtracing will go back

On 11/9/2010 3:25 PM, Alex Dubov wrote:

Joel de Guzman writes:

...

On 11/9/2010 3:03 AM, Eric Niebler wrote:

...

(cross-posting to spirit-devel)

Just hazarding a guess here, but the multi_pass_iterator probably works by caching all the read data so far. Otherwise, how would it support making a copy of the begin iterator and dereferencing it at a later point in time? If that solution wasn't good enough for you before, you probably don't want multi_pass_iterator+Spirit.

Can the Spirit guys chime in on this?

The multi_pass iterator uses a sliding window that can be flushed when you reach a deterministic point, a point of no return where back-tracking is no longer an option. In this case, saved iterators are invalidated.

I was under impression that for most grammars explicit flush is not even necessary. multi_pass maintains (at least, this follows from documentation) reference counts for all of its copies, so the moment specific "past" iterator position is not longer referenced by the parser the storage will be truncated automatically (when employing "split_std_deque" storage policy). There's also a faster "fixed_size_queue" policy, but it requires knowing in advance how far the backtracking can possibly go for a given grammar.

Yep. That is true. You know spirit and your multi_pass well :-) Also, with Qi, we rigged it up such that we automatically flush on expectation points; e.g. *(a > b). This is very efficient and a must especially when parsing multi-gigabytes of data. Regards, -- Joel de Guzman http://www.boostpro.com http://spirit.sf.net