Re: [VB6] StringBuilder - Fast string concatenation
I should add that for my initial assessment of Dragokas' code with the smaller buffer being around 2x as fast compared to what I was using before (a variation on my posted Array approach) - This is still the case when doing a smaller # of smaller string concats (for example, when building something like an SQL string).
Code:
Private Sub Form_Click()
AutoRedraw = True: Cls
Dim x&, y&, Arr$()
Dim SB1 As clsStringBuilder, SB2 As vbRichClient5.cStringBuilder, SB3 As Collection, SB4 As CStringBuilderJ
Dim s$
New_c.Timing True
Set SB1 = New clsStringBuilder
For x = 1 To 1000
SB1.Clear
For y = 1 To 100
SB1.Append String$(100, "A")
SB1.Append vbNewLine
Next y
Next x
s = s & vbCrLf & "Short Dragokas" & vbTab & Len(SB1.ToString)
s = s & vbTab & New_c.Timing
New_c.Timing True
Set SB2 = New_c.StringBuilder
For x = 1 To 1000
SB2.Clear
For y = 1 To 100
SB2.AppendNL String$(100, "A")
Next y
Next x
s = s & vbCrLf & "Short RC5-SB" & vbTab & Len(SB2.ToString)
s = s & vbTab & New_c.Timing
New_c.Timing True
For x = 1 To 1000
Set SB3 = New Collection ' No Clear/RemoveAll method on VB collection
For y = 1 To 100
SB3.Add String$(100, "A")
SB3.Add vbNewLine
Next y
Next x
s = s & vbCrLf & "Short VB6 Collection" & vbTab & Len(ConcatCollection(SB3))
s = s & vbTab & New_c.Timing
Set SB3 = Nothing
New_c.Timing True
Set SB4 = New CStringBuilderJ
For x = 1 To 1000
SB4.Clear
For y = 1 To 100
SB4.Append String$(100, "A"), appendmod_CrLf
Next y
Next x
s = s & vbCrLf & "Short Array" & vbTab & Len(SB4.Value)
s = s & vbTab & New_c.Timing
New_c.Timing True
Dim SB5 As cArrayList
Set SB5 = New_c.ArrayList(vbString)
For x = 1 To 1000
SB5.RemoveAll
For y = 1 To 100
SB5.Add String$(100, "A")
Next y
SB5.Add vbNewLine
Next x
s = s & vbCrLf & "Short RC5-ArrList" & vbTab & Len(SB5.Join(vbCrLf))
s = s & vbTab & New_c.Timing
txtText1.Text = s
Set SB5 = Nothing
End Sub
Re: [VB6] StringBuilder - Fast string concatenation
@Olaf - since the RC5ArrayList approach seems fastest, would there be worth it to update the RC5 StringBuilder class to use it? Or are there features of the StringBuilder class that would make that too painful or not worthwhile (like the UTF8 and XML functions)?
Re: [VB6] StringBuilder - Fast string concatenation
@Dragokas, I still don't think the timing is accurate. I see you're using a different timer. So to more accurately portray the work, each builder should output its final string before the cTim(x).Freeze method call.
VB6 is my therapy.
Sometimes when you're trying to solve a problem you need to talk to the
Re: [VB6] StringBuilder - Fast string concatenation
killian353535, no, there is all okay with that.
if .Freeze is called first, .GetTime and .GetTimeLastStep methods doesn't call QueryPerformanceCounter API anymore until .Start method will be called.
Re: [VB6] StringBuilder - Fast string concatenation
@Dragokas, that is the problem For instance, many of the versions don't assemble the string until the end of the description string.
For example, the collection version doesn't include the time required to actually assemble the final string.
Code:
cTim(3).Freeze
s = s & vbCrLf & "SB3 Collection" & vbTab & cTim(3).GetTimeLastStep & vbTab & Len(ConcatCollection(SB3))
Pretty much all versions have some work to be done when getting the length to include in the description string. That work should be included in the time since that is the only time we build the final string.
VB6 is my therapy.
Sometimes when you're trying to solve a problem you need to talk to the
Re: [VB6] StringBuilder - Fast string concatenation
killian353535, I see. Sorry. You are right. Even more, it is not accurate even if set .Freeze (or New_c.Timing) as inline after Len(.ToString), because ~ 30 msec. took an execution of this line before reach the code of counting average values.
I fixed it in new test + added VBCorLib ArrList.
StringBuilder by VolteFace updated to v2.2
' .AppendLine method has been added (same as .Append, but also adds CrLf characters to the end of string)
' Speed is improved (reallocation of buffer is now *= 1.6 instead of += CHUNK_SIZE ).
Re: [VB6] StringBuilder - Fast string concatenation
Dragokas, thank you for the all the work! It's been great to see all the thought processes. I find it interesting that the RC5 ArrayList solution is the fastest, since it wasn't even originally thought of. I like being surprised In the end it seems so close that any of them would perform plenty fast for given situations.
VB6 is my therapy.
Sometimes when you're trying to solve a problem you need to talk to the
Re: [VB6] StringBuilder - Fast string concatenation
Originally Posted by jpbro
@Olaf - since the RC5ArrayList approach seems fastest, would there be worth it to update the RC5 StringBuilder class to use it?
I was quite surprised myself - but (as seen in my post #29), the RC5-StringBuilder was implemented straight-forward (not yet optimized).
I've optimized it now (new version 5.0.65 is online) - and the new implementation (still using the same principle as before)
is now slightly faster than the ArrayList in nearly every scenario (just changed the LeftHand-Mid to typelib-based RtlMoveMemory and the
reallocations to SysReAllocStringLen in the Append-Method).
It now works at a speed-level, that the small Temp-Array-Buffering-trick (to gather the Column-Values and avoid high-frequent COM-calls to .Append)
is only worthwhile for larger Column-Aggregation-Counts (as e.g. 32) - below that, direct Appends work faster as it seems.
I wrote a small Test-App, which can now apply different Row- and ColumnCounts to the TestRoutines
(to test the SBs in midsized and smallsized scenarios as well - included is the version v2.2 from Dragokas, as posted in #46). SBTests.zip
Here is a ScreenShot:
The above shows the already well-known scenario (100,000 rows x 32 cols) -
but a midsized one and a small one are included as well.
The "Pure" routines do not make use of the tmp-Array-trick (feeding the SBs directly).
Re: [VB6] StringBuilder - Fast string concatenation
@Olaf - those optimizations are performing very nicely here in my tests. In fact, it looks like I will replace my Array/Join based StringBuilder with the RC5 StringBuilder now. While the Array/Join approach performs respectably close on the smaller tests, it performs increasingly poorly on larger "pure" tests.
Re: [VB6] StringBuilder - Fast string concatenation
Threads like this make me miss VB6. I used to have a lot of fun getting my hands dirty with stuff like this. Modern frameworks have everything now. I rarely get a chance to get down and dirty like this.
C++ programmers will dismiss you as a cretinous simpleton for your inability to keep track of pointers chained 6 levels deep and Java programmers will pillory you for buying into the evils of Microsoft. Meanwhile C# programmers will get paid just a little bit more than you for writing exactly the same code and VB6 programmers will continue to whitter on about "footprints". - FunkyDexter
There's just no reason to use garbage like InputBox. - jmcilhinney
The threads I start are Niya and Olaf free zones. No arguing about the benefits of VB6 over .NET here please. Happiness must reign. - yereverluvinuncleber
Re: [VB6] StringBuilder - Fast string concatenation
The reason why scripting languages like Python and JavaScript are becoming more and more popular is that they have a large number of modern frameworks and third-party libs. VB6 is both an interpreted language and a compiled language, and its potential should be more powerful than scripting languages if VB6 also has similar modern frameworks and third-party libs. Sadly, stupid Microsoft gave up VB6. But until now there are still many people who love VB6 very much. Thank you, Dragokas and other enthusiastic people.
Last edited by dreammanor; Dec 17th, 2017 at 11:34 PM.
Re: [VB6] StringBuilder - Fast string concatenation
Like this;:
Public Function GetBetween(ByRef sStart As String, _
ByRef sStop As String, _
Optional RemoveSpaces As Boolean = False, _
Optional ByRef first As Long = 1) As String
Dim Temp$
Dim second&
first = sb.Find(1, sStart)
If first > 0 Then
second = sb.Find(first + Len(sStart), sStop)
End If
If second > first Then
first = first + Len(sStart)
Temp$ = sb.ToStringMid(first, second - 1 - first)
If RemoveSpaces Then Temp$ = Trim$(Temp$)
GetBetween = Temp$
End If
End Function
ex:.../...
sb.Append "Very large concatenation string is a real programming intention "
Text1 = GetBetween("real", "intention", True) 'Return -> programming
.../...
Re: [VB6] StringBuilder - Fast string concatenation
[QUOTE=Dragokas;5168793]v.2.3.
It's well-tested class for concatenating strings like:
Code:
s = s & "one line"
s = s & "two line"
...
s = s & "N line"
but much much more faster than VB runtime do it.
Cases when it is needed sometimes:
Well, e.g., I'm using it to prepare some really huge logs of program in a single 'String' variable (e.g. up to 1 MB.) rather then writing them line by line to file (for performance acceleration purposes, if debug mode of my app. is disabled).
Don't know what else good cases. Better, don't store large data in this way, use suitable tools: arrays, databases ...
Using examples:
Code:
Option Explicit
Private Sub Form_Load()
'init
Dim sb As clsStringBuilder
Set sb = New clsStringBuilder
'concatenation
sb.Append "Very "
sb.Append "many "
sb.Append "pieces "
sb.Append "of data"
'result
Debug.Print "All data: "; sb.ToString
Debug.Print "Len of data: "; sb.length
'removing 5 characters from position # 1
sb.Remove 1, 5
Debug.Print "New data: "; sb.ToString
'inserting string in position # 1
sb.Insert 1, "Not "
Debug.Print "New data: "; sb.ToString
'overwrite part of the text from position # 1 (same like MID(str,x) = "...")
sb.Overwrite 1, "How"
Debug.Print "New data: "; sb.ToString
'getting 2 first characters
Debug.Print "2 left chars: "; sb.ToStringLeft(2)
'getting 2 characters from the end
Debug.Print "2 end chars: "; sb.ToStringRight(2)
'getting 2 characters from the middle, beginning from position 6
Debug.Print "2 middle chars: "; sb.ToStringMid(6, 2)
'getting a pointer to a NUL terminated string to use somehow (e.g. write on disk by ptr, WriteFile, e.t.c.)
'this method is much faster than .ToString()
'warning: you should use this pointer before calling next any method of StringBuilder, that may cause changing its data
Debug.Print "ptr to string: "; sb.ToStringPtr
'replacing the data (same as Clear + Append)
sb.StringData = "Anew"
Debug.Print "New data: "; sb.ToString
'go to the next line (append CrLf)
sb.AppendLine ""
'append second line with CrLf at the end
sb.AppendLine "Second line"
'append third line without CrLf
sb.Append "Third Line"
Debug.Print sb.ToString
'clear all data
sb.Clear
Debug.Print "Len of data (after clear): "; sb.length
'Search samples (by default search is case sensitive)
'Set new data
sb.StringData = "|textile|Some|text|to|search"
Debug.Print "New data: "; sb.ToString
'Simple search ('text' will be found inside 'textile' word)
Debug.Print "Position of 'text': " & sb.Find(1, "text")
'Simple search (start search from position 3)
Debug.Print "Position of 'text': " & sb.Find(3, "text")
Debug.Print "'some' (case sensitive): " & sb.Find(1, "some")
Debug.Print "'some' (case insensitive): " & sb.Find(1, "some", , vbTextCompare)
'Search by delimiter
Debug.Print "searching for |text|: " & sb.Find(1, "text", "|")
'Search for empty string, saved with delimiter |
Debug.Print "empty string (delim = '|'): " & sb.Find(1, "", "|")
'Undo operations
sb.StringData = "Some data "
Debug.Print "Orig. string: " & sb.ToString
sb.Append "remove"
Debug.Print "After Append: " & sb.ToString
'or you can use .UndoAppend
sb.Undo
Debug.Print "After Undo: " & sb.ToString
sb.Insert 6, "bad "
Debug.Print "After Insert: " & sb.ToString
'or you can use .UndoInsert
sb.Undo
Debug.Print "After Undo: " & sb.ToString
sb.Overwrite 1, "Here"
Debug.Print "After Overwrite: " & sb.ToString
'or you can use .UndoOverwrite
sb.Undo
Debug.Print "After Undo: " & sb.ToString
sb.Remove 2, 5
Debug.Print "After Remove: " & sb.ToString
'or you can use .UndoRemove
sb.Undo
Debug.Print "After Undo: " & sb.ToString
'when you finished work with the class
Set sb = Nothing
Unload Me
End Sub
Hi, Dragokas!
Congratulations on your code. He really is very good and very fast!
I would like to suggest the function GetString (Index As Long) As String. The purpose is returns the string at an indexed location within the internal collection. For example:
Code:
MyStr.Append "One"
MyStr.Append "Two"
MyStr.Append "Tree"
Debug.Print MyStr.GetString(1) 'Returns the string named "Two"
Re: [VB6] StringBuilder - Fast string concatenation
Thank you Dragokas.
I saw "HUGE" improvements in speed when i incorporated your code.
I have written a project where multiple clients push satellite data to the server so there is a central pool of whats currently available.
I was using huge amounts of
Code:
string = string + string
to build a packet that would be issued out. This is in two formats, http as there is a web portal and then a client/server packet.
You start out not worrying about it and then it becomes noticeable if you don't do anything about it. I always believe in seeing if someone else out there has done the work and in your case Kudos.
To give you one example, things which took a couple seconds now take 100th of a second.
It was easy to port your class in, and quickly utilize it. Thanks again!!! Your work is much appreciated.
Re: [VB6] StringBuilder - Fast string concatenation
UTF8?
All internal VB6 strings are unicode.
The conversion from unicode to an UTF8 byte array is only performed when you save it to disk.
When you get/read an byte array containing utf8 the you need to convert it back to unicode.
There are multiple samples on the forum on how to these conversions
Re: [VB6] StringBuilder - Fast string concatenation
Black_Storm, it would be better you describe your entire use case. Some code from you can explain.
Also, are you sure that you provide a substring in the same utf8 format?
Generally, the class is not intended for utf8.
He also asked for UTF8 controls and much more.
He should deal with his data as normal strings and only convert them when receiving the data.
Code:
' StrConvUTF8.bas
Option Explicit
Private Declare Function MultiByteToWideChar Lib "kernel32" (ByVal CodePage As Long, ByVal dwFlags As Long, ByVal lpMultiByteStr As Long, ByVal cchMultiByte As Long, ByVal lpWideCharStr As Long, ByVal cchWideChar As Long) As Long
Private Declare Sub PutMem4 Lib "msvbvm60" (ByVal Ptr As Long, ByVal Value As Long)
Private Declare Function SysAllocStringLen Lib "oleaut32" (ByVal Ptr As Long, ByVal Length As Long) As Long
Private Declare Function WideCharToMultiByte Lib "kernel32" (ByVal CodePage As Long, ByVal dwFlags As Long, ByVal lpWideCharStr As Long, ByVal cchWideChar As Long, ByVal lpMultiByteStr As Long, ByVal cchMultiByte As Long, ByVal lpDefaultChar As Long, lpUsedDefaultChar As Long) As Long
Public Function StrConvFromUTF8(Text As String) As String
' get length
Dim lngLen As Long, lngPtr As Long: lngLen = LenB(Text)
' has any?
If lngLen Then
' create a BSTR over twice that length
lngPtr = SysAllocStringLen(0, lngLen * 1.25)
' place it in output variable
PutMem4 VarPtr(StrConvFromUTF8), lngPtr
' convert & get output length
lngLen = MultiByteToWideChar(65001, 0, ByVal StrPtr(Text), lngLen, ByVal lngPtr, LenB(StrConvFromUTF8))
' resize the buffer
StrConvFromUTF8 = Left$(StrConvFromUTF8, lngLen)
End If
End Function
Public Function StrConvToUTF8(Text As String) As String
' get length
Dim lngLen As Long, lngPtr As Long: lngLen = LenB(Text)
' has any?
If lngLen Then
' create a BSTR over twice that length
lngPtr = SysAllocStringLen(0, lngLen * 1.25)
' place it in output variable
PutMem4 VarPtr(StrConvToUTF8), lngPtr
' convert & get output length
lngLen = WideCharToMultiByte(65001, 0, ByVal StrPtr(Text), Len(Text), ByVal lngPtr, LenB(StrConvToUTF8), ByVal 0&, ByVal 0&)
' resize the buffer
StrConvToUTF8 = LeftB$(StrConvToUTF8, lngLen)
End If
End Function
Re: [VB6] StringBuilder - Fast string concatenation
Updated.
' v2.6
' Fixed .Undo method incorrecly work after .Append / .AppendLine (thanks xiaoyao for pointing me about that back in 2021
'
' v2.5
' Improved speed by replacing some RtlMoveMemory by __vbaCopyBytes
'
' v2.4
' Added check & Err.raise to all methods to ensure user doesn't accidentally call this class within Forms' Terminate event.