Tokenizer

I have a question about boost::tokenizer. I have a Semicolon separated string, something like: "All;;;;;;Bus;;;;;;BRT;;;;;;All" When I use boost tokenizer, I get "All", "Bus", "BRT",and "All". I want to get the the empty string between the semicolons, seems like the tokenizer takes consecutive semicolons as a single token; is there any way I can change this to represent different strings? boost::char_separator<char> sep(";"); typedef boost::tokenizer tokenizer; tokenizer Id_tokens(IdString, sep); for(tokenizer::iterator Id_tok_iter= Id_tokens.begin(), End_Id_tok_iter = Id_tokens.end(); Id_tok_Iter != End_Id_tok_iter) { //Do Something } Another question, is there some way to assign a different string once a tokenizer is constructed? tokenizer Id_tokens(IdString,sep); Id_tokens.assign(NewString,sep); //Is this possible with the boost tokenizer? I couldn't find any examples to this end in the places I searched. Thank you, Vijay