dcsimg
Results 1 to 9 of 9

Thread: [VB6] A faster way to loop through string content

  1. #1

    Thread Starter
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    17,450

    Lightbulb [VB6] A faster way to loop through string content

    What you are about to see isn't new, but may be new to you?

    If you have a need to parse long strings, to loop through each character of long strings or basically a scenario like:
    For i = 1 To Len(strStuff), then consider this option.

    Instead of:
    Code:
        Dim c As Long
    
        For c = 1 To Len(strStuff) ' -- strStuff passed to routine externally
            Select Case AscW(Mid$(strStuff, c, 1))
            ' ....
            End Select
        Next
    This:
    Code:
        Dim c As Long
        Dim tSA as SafeArray1DStruct, iData() As Integer
    
        pvOverlayArrayOnString True, tSA, iData(), strStuff
        On Error GoTo ReleaseOverlay
        For c = 1 To Len(strStuff) ' -- strStuff passed to routine externally
            Select Case iData(c)
            ' ....
            End Select
        Next
    ReleaseOverlay:
        pvOverlayArrayOnString False, tSA, iData()
        If Err Then
            Stop
            Err.Clear
        End If
        On Error GoTo 0
    Why the error trapping in second example? Simple, we have an integer array and string referencing the exact same memory. If VB destroys the array and also destroys the string -- crash when VB tries to destroy the same memory twice. By using the On Error GoTo statement, we can avoid this by ensuring the array no longer references the same memory before VB gets its hands on it. And at same time, clue you into a potential bug in your loop.

    The code you need is shown below. But just FYI. I used a simple string "Now is the time for all good men to come to the aid of their country". Then I used VB and the hack above to get the Ascii value of each character a total of 500,000 times. The difference, uncompiled & compiled with optimizations.
    Uncompiled
    VB: 3.03 seconds
    Hack: 0.46 seconds

    Compiled with array bounds & overflow checks turned off. Better still
    VB: 2.46 seconds
    Hack: 0.03 seconds

    APIs:
    Code:
    Private Declare Sub CopyMemory Lib "kernel32.dll" Alias "RtlMoveMemory" (ByRef Destination As Any, ByRef Source As Any, ByVal length As Long)
    Private Declare Function VarPtrArray Lib "msvbvm60.dll" Alias "VarPtr" (ByRef Ptr() As Any) As Long
    
    Private Type SafeArray1DStruct
        cDims As Integer
        fFeatures As Integer
        cbElements As Long
        cLocks As Long
        pvData As Long
        rgSABelements As Long
        rgSABlBound As Long
    End Type
    The reusable routine:
    Code:
    Private Sub pvOverlayArrayOnString(bOverlay As Boolean, tSA As SafeArray1DStruct, _
                                        dstArray() As Integer, Optional strSource As String = vbNullString)
    
        ' Pass a SafeArray1DStruct, the string to be overlayed and the integer array for the overlay
        ' i.e.,
        '   Dim tSA As SafeArray1DStruct, strSample As String, iData() As Integer
        '   pvOverlayArrayOnString True, tSA, iData(), strSample
        ' now you can loop same as a string, i.e., For c = 1 to Len(strSample) ...
        
        ' To release the overlay and prevent crash when your source string and array are destroyed,
        '   pass the same tSA structure and integer array
        '   pvOverlayArrayOnString False, tSA, iData()
    
        If bOverlay = False Then
            If tSA.pvData = 0 Then 
                Erase dstArray()
            Else
                CopyMemory ByVal VarPtrArray(dstArray()), 0&, 4&
            End If
            
        ElseIf LenB(strSource) = 0 Then         ' passing a null string?
            tSA.pvData = 0
            ReDim dstArray(0 To 0)              ' return something equivalent to ""
            
        Else
            With tSA
                .cDims = 1                      ' nr of dimensions
                .cbElements = 2                 ' integer array
                .rgSABlBound = 1                ' one-bound same as string char indexing
                .pvData = StrPtr(strSource)     ' memory address
                .rgSABelements = Len(strSource) ' number array items
            End With
            CopyMemory ByVal VarPtrArray(dstArray()), VarPtr(tSA), 4&
        End If
        
    End Sub
    Long story short. Useful technique for string parsing engines. Or routines that must navigate lengthy strings within loops. This technique really has no drawbacks as long as you are not editing the string during your parsing. Otherwise, in the same routine, you can simultaneously access the string data via the Integer array and VB string functions like InStr(), etc. This is because the array overlays the exact same data. Any changes you make to array data automatically affects the underlying string. But if you make changes using VB string functions and the end result is a new string, then you must remove the overlay because it now references released memory.

    Standard warning: Never manually resize or erase the array you pass to the overlay function. The array was not sized by VB, VB can only do bad things with it. Erasing the array via VB will destroy the string & when VB attempts to release the string -- crash.
    Last edited by LaVolpe; Aug 21st, 2018 at 03:52 PM. Reason: warning
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

  2. #2

    Thread Starter
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    17,450

    Re: [VB6] A faster way to loop through string content

    And again, FYI. I have a "fun" project that must parse lengthy files, dozens at a time. My first attempt was to use standard string parsing as shown at top of previous post, i.e., Asc or AscW on Mid$(string, pos, 1). I noticed some slow downs in the routines when particularly large files were in play. Changed the code to use the array overlay hack and the the routines went faster on those larger files than when the smaller files were used with the previous code.

    This is not for everyone, I sure wouldn't consider doing overlays for some simple routine that parses individual small strings. The extra effort isn't worth it in my opinion.
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

  3. #3

    Thread Starter
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    17,450

    Re: [VB6] A faster way to loop through string content

    If interested, I tried to see if a byte-matching InStr() could be improved by using the Integer array overlay.

    Sadly no. Uncompiled the For:Next loop (with IF to check matching) to move along each byte was so slow due to VB safety & overflow checks. When compiled, and removing those safety checks, the routine did out-perform VB, but only marginally.

    The main reason why retrieving the AscW() value of any string character, via the Integer array, is so much faster is simply because Left$(), Mid$(), Right$() and many other string functions return a string. So AscW(Mid$(s, x, 1)) results in VB creating a 1-char string, populating it, then passing it to AscW(), then destroying the string. The Integer array option is simply a low-level compare of 2 bytes against another 2 bytes -- no string functions needed.

    However and just FYI: The same result as AscW(Left$(someString,1), would be AscW(someString) and that option is lightning fast because VB simply grabs the 1st 2 bytes of the string, it doesn't create a string; it's those other functions that are the bottleneck. Too bad we don't have a built-in AscWMid(s, charOffset) function that would be basically the Integer array option posted above.
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

  4. #4
    Hyperactive Member
    Join Date
    Aug 2016
    Posts
    278

    Re: [VB6] A faster way to loop through string content

    Code:
    Option Explicit
    Const strStuff As String = "123456789"
    Private Sub Command1_Click()
        Dim c As Long
        Dim tSA As SafeArray1DStruct, iData() As Integer
    
        pvOverlayArrayOnString True, tSA, iData(), strStuff
        On Error GoTo ReleaseOverlay
        For c = 1 To Len(strStuff) ' -- strStuff passed to routine externally
           
           Debug.Print "sece===" & iData(c)
            ' ....
           
        Next
    ReleaseOverlay:
        pvOverlayArrayOnString False, tSA, iData()
        If Err Then
            Stop
            Err.Clear
        End If
        On Error GoTo 0
    End Sub
    
    Private Sub Command2_Click()
        Dim c As Long
    
        For c = 1 To Len(strStuff) ' -- strStuff passed to routine externally
            Debug.Print Asc(Mid$(strStuff, c, 1))
            ' ....
            
        Next
    End Sub
    Code:
     49 
     50 
     51 
     52 
     53 
     54 
     55 
     56 
     57 
    
    sece===49
    sece===57
    sece===99
    sece===101
    sece===61
    sece===61
    sece===61
    sece===54
    sece===52

  5. #5
    Lively Member
    Join Date
    Aug 2017
    Posts
    117

    Re: [VB6] A faster way to loop through string content

    Hi! There is an alternative implementation here that wraps the logic in a class.

  6. #6

    Thread Starter
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    17,450

    Re: [VB6] A faster way to loop through string content

    xxdoc123 -- good one, but an oddball case.

    Constants can't be changed. So VB has a mechanism to prevent it. Can only guess what's going on -- StrPtr is on the stack (temp memory) and is populated by VB each time the constant is referenced, but quickly overwritten on return due to stack operations. I can run your loop, inside another short loop, and get different results. Almost as though the constant isn't returning a true BStr, but a pseudo-BStr at a fixed stack location?

    Likewise, wouldn't work on some string that has characters being changed dynamically, say by a timer.

    Trying to do this with constants (which I don't think anyone would), won't work. Again, good one
    Last edited by LaVolpe; Aug 21st, 2018 at 07:25 PM.
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

  7. #7

    Thread Starter
    VB-aholic & Lovin' It LaVolpe's Avatar
    Join Date
    Oct 2007
    Location
    Beside Waldo
    Posts
    17,450

    Re: [VB6] A faster way to loop through string content

    Quote Originally Posted by Victor Bravo VI View Post
    Hi! There is an alternative implementation here that wraps the logic in a class.
    Knew this wasn't new. Didn't know about Bonnie's posting. See nothing different in the logic & wouldn't have posted this if I knew her's was there. Added more/different details/thoughts than what's mentioned on her thread, and maybe that could be worth something to someone?
    Insomnia is just a byproduct of, "It can't be done"

    Classics Enthusiast? Here's my 1969 Mustang Mach I Fastback. Her sister '67 Coupe has been adopted

    Newbie? Novice? Bored? Spend a few minutes browsing the FAQ section of the forum.
    Read the HitchHiker's Guide to Getting Help on the Forums.
    Here is the list of TAGs you can use to format your posts
    Here are VB6 Help Files online


    {Alpha Image Control} {Memory Leak FAQ} {Unicode Open/Save Dialog} {Resource Image Viewer/Extractor}
    {VB and DPI Tutorial} {Manifest Creator} {UserControl Button Template} {stdPicture Render Usage}

  8. #8
    PowerPoster Elroy's Avatar
    Join Date
    Jun 2014
    Location
    Near Nashville TN
    Posts
    4,579

    Re: [VB6] A faster way to loop through string content

    Quote Originally Posted by LaVolpe View Post
    Constants can't be changed. So VB has a mechanism to prevent it. Can only guess what's going on -- StrPtr is on the stack (temp memory) and is populated by VB each time the constant is referenced, but quickly overwritten on return due to stack operations.
    Not exactly on-point, but I always thought copies were made anytime Constants were passed as arguments (i.e., a temp variable created), similar to the way Fixed-Length-Strings are done (converted to a temp BSTR). And, I thought this was true even when calling VarPtr(). And that's why you couldn't change them even if you were willing to start messing around with memory. It's just difficult to find their "true" memory location.

    EDIT1: I suppose you can change Fixed-Length-Strings, when passed ByRef. However, they do get converted to temp BSTR. And then, upon return, there's some strange LSET-type code executed to put the altered BSTR back into the original Fixed-Length-String.
    Last edited by Elroy; Aug 22nd, 2018 at 10:49 AM.
    Any software I post in these forums written by me is provided “AS IS” without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. Please understand that I’ve been programming since the mid-1970s and still have some of that code. My contemporary VB6 project is approaching 1,000 modules. In addition, I have a “VB6 random code folder” that is overflowing. I’ve been at this long enough to truly not know with absolute certainty from whence every single line of my code has come, with much of it coming from programmers under my employ who signed intellectual property transfers. I have not deliberately attempted to remove any licenses and/or attributions from any software. If someone finds that I have inadvertently done so, I sincerely apologize, and, upon notice and reasonable proof, will re-attach those licenses and/or attributions. To all, peace and happiness.

  9. #9
    Registered User
    Join Date
    Aug 2018
    Posts
    1

    Re: [VB6] A faster way to loop through string content

    I came across this years ago and have been using it as an alternative to AscW(Mid$()). It's slower in IDE but a lot quicker when compiled. Not sure where I found it or how it compares to your code but thought I'd offer it as another solution.

    Code:
    Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)
    
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''
    ' Visual Basic is one of the few languages where you
    ' can't extract a character from or insert a character
    ' into a string at a given position without creating
    ' another string.
    '
    ' The following Property fixes that limitation.
    '
    ' Twice as fast as AscW and Mid$ when compiled.
    '        iChr = AscW(Mid$(sStr, lPos, 1))
    '        iChr = MidI(sStr, lPos)
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''
    Public Property Get MidI(sStr As String, ByVal lPos As Long) As Long
        CopyMemory MidI, ByVal StrPtr(sStr) + lPos + lPos - 2&, 2&
    End Property
    
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''
    '        Mid$(sStr, lPos, 1) = Chr$(iChr)
    '        MidI(sStr, lPos) = iChr
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''
    Public Property Let MidI(sStr As String, ByVal lPos As Long, ByVal iChr As Long)
        CopyMemory ByVal StrPtr(sStr) + lPos + lPos - 2&, iChr, 2&
    End Property
    
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''
    '        iChr = AscW(LCase$(Mid$(sStr, lPos, 1)))
    '        iChr = MidLcI(sStr, lPos)
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''
    Public Property Get MidLcI(sStr As String, ByVal lPos As Long) As Long
        CopyMemory MidLcI, ByVal StrPtr(sStr) + lPos + lPos - 2&, 2&
        If MidLcI > 64 And MidLcI < 91 Then MidLcI = MidLcI + 32
    End Property
    
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''
    '        Mid$(sStr, lPos, 1) = LCase$(Chr$(iChr))
    '        MidLcI(sStr, lPos) = iChr
    ''''''''''''''''''''''''''''''''''''''''''''''''''''''
    Public Property Let MidLcI(sStr As String, ByVal lPos As Long, ByVal iChr As Long)
    
        If iChr > 64 And iChr < 91 Then iChr = iChr + 32
        CopyMemory ByVal StrPtr(sStr) + lPos + lPos - 2&, iChr, 2&
    
    End Property

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width