At 19:19 2005-06-11, you wrote:
for tokenizing on whitespace, simple stream input (>>) to a std::string suffices.
My own tokenizer does just that--and puts the tokens into a deque.
IMO, it's hardly worth troubling yourself with a tokenizer for whitespace.
Well, not really. When parsing line-oriented output and semi-known structured lines it's handy to be able to sometimes work with a line's tokens as if they were in a vector or deque.
string yourline; istringstream is(yourline); deque<string> yourvec((istream_iteratorstd::string(is)), istream_iteratorstd::string()); voila, a deque it would be interesting to profile that against the hypothetical indexable tokenizer.
In fact, I was going to add a suggestion that the tokenizer also have the [] operator so that the individual tokens could be addressed as tok[1], etc.
-Tom
_______________________________________________ Boost-users mailing list Boost-users@lists.boost.org http://lists.boost.org/mailman/listinfo.cgi/boost-users
Victor A. Wagner Jr. http://rudbek.com The five most dangerous words in the English language: "There oughta be a law"