Quote:
I want you to write a function that takes a String and tokenizes it in VB.Net. This function would take a String as input and return an array of Strings as output.
The tokenization process should consider each non-alpha numeric character as a single token.
It should consider whole numbers and mixed numbers like 12.5 as a single tokens.
It should consider any pair of carriage return and linefeed characters as a single token.
It should consider any consecutive series of whitespace characters as a single token of one or more whitespace characters.
It should consider any consecutive series of alphanumeric characters as a single token. The underscore character should also be considered and alpha-numeric character.
If any single quote character is encountered, the single quote and everything after it up to the first newline character would be considered a single token. The pair of carriage return and linefeed pair should not be included in this token.
If any double quote character is encountered, the double quote character and everything thing after it until the next lone double quote character is considered one token. Special care needs to be taken for pairs of double quotes that come after a lone double quote character. Pairs of double quote characters between single double quote characters are also just part of the token.
Here are some examples of tokenization to help you understand:
[Number = 10] = [Number][ ][=][ ][10]
[Dim My_Str As String = "Dog ""Cat"" Dog"] = [Dim][ ][My_Str][ ][As][ ]String[ ][=][ ]["Dog ""Cat"" Dog"]
[Math.Add(12.5,300)] = [Math][.][Add][(][12.5][,][300]
The string inside the braces on the left hand side of the equal sign represents the input and the contents between each pair of open and closing braces on the right hand side of the equal sign represents each token.
Here are a few notes about this:
Do not use Regex for this.
The function itself could be part of a class.
The class could contain helper functions.
Use helper functions to avoid nesting loops within loops. If it becomes necessary to embed a loop within a loop. The inner loop should be implemented as a helper function instead.
And the other for processing the tokens:-
Quote:
I want you to write a function in VB.Net that takes an array of Strings that represent tokens, wrap then in tags, concatenate them and return the result.
The tags are colour tags. For example, lets say "Cat" is the token, it could become "Cat".
The colour of the tags depends on the token type. Here are the different token types:
Keyword tokens: As, For, Next, End, Class, Structure, Dim, Private, Public Friend, Sub, Function, Property, If, Else, Implements, Inherits, DirectCast, CType, Handles
Operator tokens: +, -, /, *, =, ^, (, )
Type tokens: Single, Double, String, Boolean, Integer, Long, Short, Byte
The keyword tokens should have the colour #A52A2A.
The operator tokens should have the colour #0000FF.
The type tokens should have the colour #4B0082.
Tokens that begin with a single quote should be coloured as #006400
Tokens that begin with a double quotes should be coloured as #8B4513
Tokens that begin with white space, carriage return or line feed should not be changed and concatenated as is.
Before returning the final concatenated String, it should itself be wrapped in a colour tag which has the effect of giving the whole text a default colour. That could should be #000080.
Those two queries got me like 80% of the way there. From there I kept asking it to refactor the code repeatedly until I got something I was satisfied with which was the actual code I posted above.