[VB6] Faster RTB Code Formatting

[dilettante]

The scenario here is a program that allows user scripting in VBScript. This program needs a script IDE to make it easier for users to write, test, and modify their scripts. Instead of a simple text editor, we want to format entered VBScript code in a manner similar to the way VB6.EXE does it. This means more than simple "colorization" but also includes correction of whitespace and punctuation as well as re-formatting statements as they are manually altered.

Performance is a factor because in the real code far more is going on and it occurs during user editing as well as upon loading.


Demo

Stripped down implementation for illustration. Only about 5% of the real thing is included here.

Does not take into account predefined constants, comments, line continuations, or string literals. Does not attempt to correct whitespace and punctuation usage. Also does not include the logic to handle statement by statement user edits.

Merely does some one-pass simplified colorizing of the entire script after loading. Just enough to look interesting.

This is done using four different methods in four different Forms varying only as needed to add optimizations in steps:

1. RTB.Visible = True (default).
2. Let RTB.Visble = False at start of formatting, then True again when done. Flawed, see MS KB 189483.
3. Let RTB.Visible = False at start and make ShowWindow calls per KB article.
4. Use TOM with Freeze and ITextRange objects.

The four Forms are MDI child Forms within a parent Form. They are run in sequence and timings are displayed for each child Form upon completion. They all use the same large sample script (testcase.txt).

Requirements

To use TOM with the VB6 RichTextBox, Windows 2000 or later is required. This causes RichTextBox to use Riched20.dll in Microsoft Rich Edit 1.0 emulation mode, thus giving us the TOM interfaces to use. TOM is not supported by Riched32.dll.


Links

BUG: Cannot Set RichTextBox Visible Property to False

Text Object Model

[georgekar]

I have a small program to compare two pieces of vb code. Top first piece is colorized (We can add some keywords)..
http://www.vbforums.com/showthread.p...highlight=test
I place your code and converted in dt...(i haven't measure but you just no see any delay...).
No using of any rich edit control. Have some problems, but for the checking is very good.
If I found some extra time maybe I can get the "colour" method for this usercontrol and put it in the more sophisticated glist4..

I run your code. You have to make it to colorize code as you type.

[dilettante]

I run your code. You have to make it to colorize code as you type.

No, this is a stripped down demo.

The full code (actually a code editing UserControl) does the reformatting as the user types, just like in the VB6 IDE.

The point of this thread is that the usual approaches to doing this run too slowly for interactive use.

There is far more to it than the colorizing of the text. For example whenever the user makes a change to a declarative statement the identifiers need to be re-cased, example:

WAS

Dim This, That, Theother

CHANGED TO

Dim This, That, TheOther

All occurrences of "Theother" need to be recased to match the new "TheOther" since Dim statements are declarative.

Or the user types something like:

wscript.echo(3*somenumber)

... where SomeNumber was declared elsewhere. This gets reformatted as:

WScript.Echo (3 * SomeNumber)

... with colorization but that's the trivial part. Operators need to have spaces inserted around them, the foolish use of parentheses after a subroutine/method call needs to be separated by a space, identifier case needs to be corrected, etc. including detecting and handling comments and string literals.

[georgekar]

But you want something that is too much. In vb ide all identifiers feed a list. So when colorize a line you change identifiers with that in the list. When you change the first occur of this identifier then the change is done anywhere. How you know that this is the first if you add code before, so you have a shifted offset? This easy...you tag this word...and the tag is hidden. So you want a structure to hold the paragraphs, colors and tags, and the display and manipulation part, a user control.

[dilettante]

But you want something that is too much.

Hardly too much, since all real IDEs do that... and more (such as IntelliSense).

In vb ide all identifiers feed a list.

And your IDE must do the same thing.

So when colorize a line you change identifiers with that in the list. When you change the first occur of this identifier then the change is done anywhere.

No, case-changes get done whenever you make a change to any line containing a "declaring" statement, not just the first one.


My point is the process is a lot of work to do properly. Or I should say my real point is you must be able to do this very quickly because your IDE has to do quite a lot of parsing and reparsing as the user changes the code statement by statement.

[georgekar]

Is not a lot of work. You have to separate the data from the view. The data need to have more info, so we can handle the names as you want. So you must make a simple routine that take string in tags and export a colored string in where you want to view.

This is the MyMid$ from the "tester" as you can find in #2
In a$ is color information and string. Color information is for background and foreground. So with MyMid$ I can get any part with data and the color information also. And if I cut somewhere in the middle of a string that has the same color information then a new header produced. So this is the "RAW" string format used in tester.

So myMID$ extract a RAW string form a char position (as text only) for a specific number of chars.

Function MyMiD$(a$, ByVal from As Long, Optional ByVal some As Long = -1)
' MyMid$ work like Mid$
' bl=0 and bl=1 are skipped
Dim I As Long, pad$, bl As Long, co As Long, npad$, fin As Long
Dim oldco As Long, oldbl As Long, oldpad$, olen As Long

If Len(a$) < 3 Or some = 0 Then
MyMiD$ = "": Exit Function
Else
 I = 1
Do While I <= Len(a$)

bl = Asc(Mid$(a$, I, 1))
If bl > 1 Then
co = Asc(Mid$(a$, I + 1, 1))

oldpad$ = oldpad$ + Mid$(a$, I + 2, bl - 2)
I = I + bl
If from <= Len(oldpad$) Then Exit Do
Else
' bl=1 also stripted
I = I + 1
End If
Loop
pad$ = Mid$(oldpad$, from)
If some = -1 Then
If pad$ = "" Then
MyMiD$ = "": Exit Function
Else
MyMiD$ = Chr(Len(pad$) + 2) + Chr(co) + pad$ + Mid$(a$, I)
End If
ElseIf some <= Len(pad$) Then
MyMiD$ = Chr(some + 2) + Chr(co) + Left$(pad$, some)
Else
some = some - Len(pad$)
pad$ = Chr(Len(pad$) + 2) + Chr(co) + pad$
oldpad$ = ""
Do While I <= Len(a$)
bl = Asc(Mid$(a$, I, 1))
If bl > 1 Then
co = Asc(Mid$(a$, I + 1, 1))
oldpad$ = Mid$(a$, I + 2, bl - 2)

If some <= Len(oldpad$) Then
Exit Do
Else
oldpad$ = ""   
End If
pad$ = pad$ + Mid$(a$, I, bl)
some = some - bl + 2
I = I + bl
Else
' also here BL=1 is splitted
I = I + 1
End If
Loop
If some <= Len(oldpad$) Then

MyMiD$ = pad$ + Chr(2 + some) + Chr(co) + Mid$(oldpad$, 1, some)
ElseIf oldpad$ = "" And Len(pad$) <> 2 Then ' i will see if i want "" or chr$(2)+chr$(0) for nothing
MyMiD$ = pad$
End If
End If

End If

Here is the Stripline and I have comments about structure.
So StripLine give as the text without the color information

Public Function StripLine(a$, Optional lfonly As Boolean = False) As String
' take a raw string and give a pure string..without color information
' Not oly one line but also lines.
' Structure of raw string:
' first byte is length of a colored blick or zero. Zero means paragraph break
' next byte is the color information, 0-15 for foreground and 4 bits most significant 1-15  for background
' color 0 means use textcolor and no fill box behind letter.
' We can have transparent control but some text with fill box, for annotations purposes
'
Dim I As Long, pad$, bl As Long, co As Long

If Len(a$) < 3 Then
StripLine = ""
Else
 I = 1
' byte len
Do While I <= Len(a$)
bl = Asc(Mid$(a$, I, 1))
If bl > 1 Then
co = Asc(Mid$(a$, I + 1, 1))
pad$ = pad$ + Mid$(a$, I + 2, bl - 2)
I = I + bl
Else
If bl = 0 Then
I = I + 1
If lfonly Then
pad$ = pad$ + vbLf
Else
pad$ = pad$ + vbCrLf
End If
Else
' just skip BL=1 not yet implemented
I = I + 1
End If

End If
Loop
StripLine = pad$

End If
End Function

I don't say to use them, but to get an idea what you can done if you use strings as structures.
For your application is better to use id number as indexes in an array of identifiers. Whatever you do you need functions like MyMid$ and StripLine..

[georgekar]

In Tester (the program in #2) we have auto colorized code, but this isn't for any other purpose than we change some code from the code we see in the bottom box. So it isn't an IDE..at all.
There are some problems with drag and drop..but for the testing purposes is ok. This tester help me a lot...to find out what are the differences in two versions...of a form or module. So color is essential for this kind of tool. We can recolor in few milliseconds thousands of lines. Also we can mark as equal non equal lines, or mark for skip some special lines...Then we press a button and we went to a new "problem" area.

[dilettante]

This thread is about using TOM over other techniques in order to rapidly work through the data in a RichTextBox.

An IDE with colorization is merely an example. You could also be doing spelling correction or language translation or almost anything else involving automatic actions during editing of text.

[georgekar]

you are right...

[DrUnicode]

TOM (Text Object Model) is also a great way to handle Unicode in RichTextBox without the need for API hacks.

Public Property Get Text() As String
   Text = m_Tom.TextDocument.Range(0, Len(RTB.Text)).Text
End Property

Public Property Let Text(ByVal New_Text As String)
   m_Tom.TextDocument.Range(0, Len(RTB.Text)).Text = New_Text
   m_Text = New_Text
   PropertyChanged "Text"
End Property

Public Property Get SelText() As String
   SelText = m_Tom.TextDocument.Selection
End Property

Public Property Let SelText(ByVal New_Text As String)
   m_Tom.TextDocument.Selection = New_Text
End Property

[dilettante]

Here is another example "bCompRtf."

Far faster and neater using "TOM" than even a mix of RichEdit SendMessage calls and RichTextBox properties. Though you'll note that a few things still needed messages, for example setting tabs with TOM must be done "paragraph" by "paragraph."

I needed a binary file compare utility that could create a "report" easily emailed or uploaded.

These generated RTF files get huge (and take a while) if you compare files much greater than 25KB. You can simplify the mismatch highlighting (see comments in Form1) which shrinks them, and in any case even the big ones ZIP down to acceptable propertions for emailing.

This displays 8 bytes across, but you could modify it to 16 fairly easily... but then you might choose a smaller font because things get w-i-d-e.


Updated:

Minor bug fix.