Introduction
A simple tokenizer in csharp without using regex or MatchCollections. It accepts single string input. The string can contain multiple tokens or tags to be parsed. The delimiter can be any character. It works by create a new instance and use GetToken to get the first token back, use MoveToNext furhter on. The first GetToken call will automatically position to the beginning of the input string. There is no to call MoveToNext to do the same job! The characters to be bypassed can be added or modified to the char[] BYPASSCODE in unicode, hex, or character formats.