String and StringTokenizer

Jerome Sturt

12y1.7kVisual Basic .NET

What is difference between String and StringTokenizer?

4 Replies

Know the answer? Post it — somebody with the same question will find it here.

Lakshmanan Sethu Sankaranarayan12y ago
@Vulpes, yeah it is related to Java.
Munesh Sharma12y ago
Obviously StringTokenizer as it is suggested by its name tokenizes a String supplied to it as an argument to its constructor and the character based on which tokens of that string are to be made.The default tokenizing character is space " ".

You can think of the StringTokenizer as a specialized form of StreamTokenizer, where each token that is returned is a String. Depending upon what you want to do with the resultant tokens and/or how complex of a parse job you want to deal with lets you decide which to use.
The StringTokenizer parses a string into a series of tokens. It is then your responsibility to figure out what each token returned is, beyond the definition of a token separated by delimiters. Usually, you just treat things as words when using StringTokenizer.
On the other hand, a StreamTokenizer allows you to ask is the next token a number, word, quoted string, end-of-file, end-of-line, comment, or whitespace. In this case, the StreamTokenizer is smarter, though can be considerably harder to use.
Vulpes12y ago
@Jerome,

Note that this question relates to Java, not C# (and Lakshmanan has answered accordingly) because .NET doesn't have a StringTokenizer class.
Lakshmanan Sethu Sankaranarayan12y ago
Hi Please find answers below -> String.split() and Pattern.split() give you an easy syntax for doing the latter, but that's essentially all that they do. If you want to parse the resulting strings, or change the delimiter halfway through depending on a particular token, they won't help you with that. -> StringTokenizer is even more restrictive than String.split(), and also a bit fiddlier to use. It is essentially designed for pulling out tokens delimited by fixed substrings. Because of this restriction, it's about twice as fast as String.split(). (See my comparison of String.split() and StringTokenizer.) It also predates the regular expressions API, of which String.split() is a part. You'll note from my timings that String.split() can still tokenize thousands of strings in a few milliseconds on a typical machine. In addition, it has the advantage over StringTokenizer that it gives you the output as a string array, which is usually what you want. Using an Enumeration, as provided by StringTokenizer, is too "syntactically fussy" most of the time. From this point of view, StringTokenizer is a bit of a waste of space nowadays, and you may as well just use