Re: [Boost-users] Tokenizer Question

At 06:37 2005-06-13, Robert Zeh wrote:

"Tom Browder" writes:

...

The accumulated times (sec) are surprising:

Boost Victor's Mine ===== ===== ==== no loop 1.50 38.84 20.13 loop 131.91 38.89 23.91

Granted, I didn't do the tests multiple times, but it seems to me that the Boost tokenizer is great if you don't need to iterate through it, but it is the pits if you do.

-Tom

I'll send you my code and results if interested.

-Tom

I'd be interested in seeing profiling output from your test --- when the tokenizer is reading from an input stream it builds up the strings character by character with operator +=.

It can avoid creating the tokens character by character when it isn't dealing with input streams (it checks for an input stream by checking for std::input_iterator_tag, which I assume is set for istringstream).

I'm still stunned that the range constructor is slower than Tom's loop. Substantially slower.

Robert Zeh http://home.earthlink.net/~rzeh

_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users

Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"