[VB6] A faster way to loop through string content

[LaVolpe]

What you are about to see isn't new, but may be new to you?

If you have a need to parse long strings, to loop through each character of long strings or basically a scenario like:
For i = 1 To Len(strStuff), then consider this option.

Instead of:

    Dim c As Long

    For c = 1 To Len(strStuff) ' -- strStuff passed to routine externally
        Select Case AscW(Mid$(strStuff, c, 1))
        ' ....
        End Select
    Next

This:

    Dim c As Long
    Dim tSA as SafeArray1DStruct, iData() As Integer

    pvOverlayArrayOnString True, tSA, iData(), strStuff
    On Error GoTo ReleaseOverlay
    For c = 1 To Len(strStuff) ' -- strStuff passed to routine externally
        Select Case iData(c)
        ' ....
        End Select
    Next
ReleaseOverlay:
    pvOverlayArrayOnString False, tSA, iData()
    If Err Then
        Stop
        Err.Clear
    End If
    On Error GoTo 0

Why the error trapping in second example? Simple, we have an integer array and string referencing the exact same memory. If VB destroys the array and also destroys the string -- crash when VB tries to destroy the same memory twice. By using the On Error GoTo statement, we can avoid this by ensuring the array no longer references the same memory before VB gets its hands on it. And at same time, clue you into a potential bug in your loop.

The code you need is shown below. But just FYI. I used a simple string "Now is the time for all good men to come to the aid of their country". Then I used VB and the hack above to get the Ascii value of each character a total of 500,000 times. The difference, uncompiled & compiled with optimizations.

Uncompiled
VB: 3.03 seconds
Hack: 0.46 seconds

Compiled with array bounds & overflow checks turned off. Better still
VB: 2.46 seconds
Hack: 0.03 seconds

APIs:

Private Declare Sub CopyMemory Lib "kernel32.dll" Alias "RtlMoveMemory" (ByRef Destination As Any, ByRef Source As Any, ByVal length As Long)
Private Declare Function VarPtrArray Lib "msvbvm60.dll" Alias "VarPtr" (ByRef Ptr() As Any) As Long

Private Type SafeArray1DStruct
    cDims As Integer
    fFeatures As Integer
    cbElements As Long
    cLocks As Long
    pvData As Long
    rgSABelements As Long
    rgSABlBound As Long
End Type

The reusable routine:

Private Sub pvOverlayArrayOnString(bOverlay As Boolean, tSA As SafeArray1DStruct, _
                                    dstArray() As Integer, Optional strSource As String = vbNullString)

    ' Pass a SafeArray1DStruct, the string to be overlayed and the integer array for the overlay
    ' i.e.,
    '   Dim tSA As SafeArray1DStruct, strSample As String, iData() As Integer
    '   pvOverlayArrayOnString True, tSA, iData(), strSample
    ' now you can loop same as a string, i.e., For c = 1 to Len(strSample) ...
    
    ' To release the overlay and prevent crash when your source string and array are destroyed,
    '   pass the same tSA structure and integer array
    '   pvOverlayArrayOnString False, tSA, iData()

    If bOverlay = False Then
        If tSA.pvData = 0 Then 
            Erase dstArray()
        Else
            CopyMemory ByVal VarPtrArray(dstArray()), 0&, 4&
        End If
        
    ElseIf LenB(strSource) = 0 Then         ' passing a null string?
        tSA.pvData = 0
        ReDim dstArray(0 To 0)              ' return something equivalent to ""
        
    Else
        With tSA
            .cDims = 1                      ' nr of dimensions
            .cbElements = 2                 ' integer array
            .rgSABlBound = 1                ' one-bound same as string char indexing
            .pvData = StrPtr(strSource)     ' memory address
            .rgSABelements = Len(strSource) ' number array items
        End With
        CopyMemory ByVal VarPtrArray(dstArray()), VarPtr(tSA), 4&
    End If
    
End Sub

Long story short. Useful technique for string parsing engines. Or routines that must navigate lengthy strings within loops. This technique really has no drawbacks as long as you are not editing the string during your parsing. Otherwise, in the same routine, you can simultaneously access the string data via the Integer array and VB string functions like InStr(), etc. This is because the array overlays the exact same data. Any changes you make to array data automatically affects the underlying string. But if you make changes using VB string functions and the end result is a new string, then you must remove the overlay because it now references released memory.

Standard warning: Never manually resize or erase the array you pass to the overlay function. The array was not sized by VB, VB can only do bad things with it. Erasing the array via VB will destroy the string & when VB attempts to release the string -- crash.

[LaVolpe]

And again, FYI. I have a "fun" project that must parse lengthy files, dozens at a time. My first attempt was to use standard string parsing as shown at top of previous post, i.e., Asc or AscW on Mid$(string, pos, 1). I noticed some slow downs in the routines when particularly large files were in play. Changed the code to use the array overlay hack and the the routines went faster on those larger files than when the smaller files were used with the previous code.

This is not for everyone, I sure wouldn't consider doing overlays for some simple routine that parses individual small strings. The extra effort isn't worth it in my opinion.

[LaVolpe]

If interested, I tried to see if a byte-matching InStr() could be improved by using the Integer array overlay.

Sadly no. Uncompiled the For:Next loop (with IF to check matching) to move along each byte was so slow due to VB safety & overflow checks. When compiled, and removing those safety checks, the routine did out-perform VB, but only marginally.

The main reason why retrieving the AscW() value of any string character, via the Integer array, is so much faster is simply because Left$(), Mid$(), Right$() and many other string functions return a string. So AscW(Mid$(s, x, 1)) results in VB creating a 1-char string, populating it, then passing it to AscW(), then destroying the string. The Integer array option is simply a low-level compare of 2 bytes against another 2 bytes -- no string functions needed.

However and just FYI: The same result as AscW(Left$(someString,1), would be AscW(someString) and that option is lightning fast because VB simply grabs the 1st 2 bytes of the string, it doesn't create a string; it's those other functions that are the bottleneck. Too bad we don't have a built-in AscWMid(s, charOffset) function that would be basically the Integer array option posted above.

[xxdoc123]

Option Explicit
Const strStuff As String = "123456789"
Private Sub Command1_Click()
    Dim c As Long
    Dim tSA As SafeArray1DStruct, iData() As Integer

    pvOverlayArrayOnString True, tSA, iData(), strStuff
    On Error GoTo ReleaseOverlay
    For c = 1 To Len(strStuff) ' -- strStuff passed to routine externally
       
       Debug.Print "sece===" & iData(c)
        ' ....
       
    Next
ReleaseOverlay:
    pvOverlayArrayOnString False, tSA, iData()
    If Err Then
        Stop
        Err.Clear
    End If
    On Error GoTo 0
End Sub

Private Sub Command2_Click()
    Dim c As Long

    For c = 1 To Len(strStuff) ' -- strStuff passed to routine externally
        Debug.Print Asc(Mid$(strStuff, c, 1))
        ' ....
        
    Next
End Sub

 49 
 50 
 51 
 52 
 53 
 54 
 55 
 56 
 57 

sece===49
sece===57
sece===99
sece===101
sece===61
sece===61
sece===61
sece===54
sece===52

[Victor Bravo VI]

Hi! There is an alternative implementation here that wraps the logic in a class.

[LaVolpe]

xxdoc123 -- good one, but an oddball case.

Constants can't be changed. So VB has a mechanism to prevent it. Can only guess what's going on -- StrPtr is on the stack (temp memory) and is populated by VB each time the constant is referenced, but quickly overwritten on return due to stack operations. I can run your loop, inside another short loop, and get different results. Almost as though the constant isn't returning a true BStr, but a pseudo-BStr at a fixed stack location?

Likewise, wouldn't work on some string that has characters being changed dynamically, say by a timer.

Trying to do this with constants (which I don't think anyone would), won't work. Again, good one

[LaVolpe]

Hi! There is an alternative implementation here that wraps the logic in a class.

Knew this wasn't new. Didn't know about Bonnie's posting. See nothing different in the logic & wouldn't have posted this if I knew her's was there. Added more/different details/thoughts than what's mentioned on her thread, and maybe that could be worth something to someone?

[Elroy]

Constants can't be changed. So VB has a mechanism to prevent it. Can only guess what's going on -- StrPtr is on the stack (temp memory) and is populated by VB each time the constant is referenced, but quickly overwritten on return due to stack operations.

Not exactly on-point, but I always thought copies were made anytime Constants were passed as arguments (i.e., a temp variable created), similar to the way Fixed-Length-Strings are done (converted to a temp BSTR). And, I thought this was true even when calling VarPtr(). And that's why you couldn't change them even if you were willing to start messing around with memory. It's just difficult to find their "true" memory location.

EDIT1: I suppose you can change Fixed-Length-Strings, when passed ByRef. However, they do get converted to temp BSTR. And then, upon return, there's some strange LSET-type code executed to put the altered BSTR back into the original Fixed-Length-String.

[cokecanyo]

I came across this years ago and have been using it as an alternative to AscW(Mid$()). It's slower in IDE but a lot quicker when compiled. Not sure where I found it or how it compares to your code but thought I'd offer it as another solution.

Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)

''''''''''''''''''''''''''''''''''''''''''''''''''''''
' Visual Basic is one of the few languages where you
' can't extract a character from or insert a character
' into a string at a given position without creating
' another string.
'
' The following Property fixes that limitation.
'
' Twice as fast as AscW and Mid$ when compiled.
'        iChr = AscW(Mid$(sStr, lPos, 1))
'        iChr = MidI(sStr, lPos)
''''''''''''''''''''''''''''''''''''''''''''''''''''''
Public Property Get MidI(sStr As String, ByVal lPos As Long) As Long
    CopyMemory MidI, ByVal StrPtr(sStr) + lPos + lPos - 2&, 2&
End Property

''''''''''''''''''''''''''''''''''''''''''''''''''''''
'        Mid$(sStr, lPos, 1) = Chr$(iChr)
'        MidI(sStr, lPos) = iChr
''''''''''''''''''''''''''''''''''''''''''''''''''''''
Public Property Let MidI(sStr As String, ByVal lPos As Long, ByVal iChr As Long)
    CopyMemory ByVal StrPtr(sStr) + lPos + lPos - 2&, iChr, 2&
End Property

''''''''''''''''''''''''''''''''''''''''''''''''''''''
'        iChr = AscW(LCase$(Mid$(sStr, lPos, 1)))
'        iChr = MidLcI(sStr, lPos)
''''''''''''''''''''''''''''''''''''''''''''''''''''''
Public Property Get MidLcI(sStr As String, ByVal lPos As Long) As Long
    CopyMemory MidLcI, ByVal StrPtr(sStr) + lPos + lPos - 2&, 2&
    If MidLcI > 64 And MidLcI < 91 Then MidLcI = MidLcI + 32
End Property

''''''''''''''''''''''''''''''''''''''''''''''''''''''
'        Mid$(sStr, lPos, 1) = LCase$(Chr$(iChr))
'        MidLcI(sStr, lPos) = iChr
''''''''''''''''''''''''''''''''''''''''''''''''''''''
Public Property Let MidLcI(sStr As String, ByVal lPos As Long, ByVal iChr As Long)

    If iChr > 64 And iChr < 91 Then iChr = iChr + 32
    CopyMemory ByVal StrPtr(sStr) + lPos + lPos - 2&, iChr, 2&

End Property