Page 2 of 2 FirstFirst 12
Results 41 to 54 of 54

Thread: Simple and fast, lightweight HashList-Class (no APIs)

  1. #41
    Addicted Member
    Join Date
    Mar 2019
    Posts
    243

    Resurrection - performance vs dictionary

    Hi Guys,

    I was hoping to be able to use this as a faster string,string value pair kind of thing than scripting.dictionary so I altered the code to be string,string and for me it is never faster than scripting.dictionary with the data I am using.

    The attached zip contains the demo project and a file called connect.log which is a text file containing the data. The project reads the text file and loads the key/values into two arrays.

    It then populates both the dictionary and the clsHashList from the array entries.

    It can then get each item using the array entry as the key

    There are about 8500 items in the connect.log file.

    For me (slow dev box) the results are (timed by timegettime)


    Dictionay Load 19 ms
    Get each dictionary item 19ms

    Load Hash 26ms
    Get each hash item 33ms

    I need to check each key against another dictionary in my production app rather than returning all key/value pairs hence the approach.

    What can I do to speed this up if anything? Everybody else in the thread seems to get much better performance vs dictionary.

    cheers
    Attached Files Attached Files

  2. #42
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,168

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    Have you tried it compiled?
    Because it seems you have tested it from the IDE with an uncompiled Hash library

  3. #43
    Addicted Member
    Join Date
    Mar 2019
    Posts
    243

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    Compiled for fast code with all optimizations on.

  4. #44
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,168

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    Did you use cHashD.zip from post #1 or did you use the code between the code blocks in the post?

  5. #45
    Addicted Member
    Join Date
    Mar 2019
    Posts
    243

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    I used the code blocks. I guess that was a mistake on my behalf?

  6. #46
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,168

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    I don't know, but all the speed tests were done using the code provided in the cHashD.zip file

  7. #47
    PowerPoster Arnoutdv's Avatar
    Join Date
    Oct 2013
    Posts
    5,168

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    See the remarks in the original post:
    Quote Originally Posted by Schmidt View Post
    ..

    Edit: Whilst the above Demo was more a "Concept-Showcase"... below is a performance-optimized version,
    acting more like the MS-Scripting-Dictionary now (with a Param-wise inversed Add-Method, but feel free
    to change it at will - the code-volume is still bearable with about 230 lines in
    cHashD).
    In the meantime I've also added support for Integer-, Double- and Object-Keys - but I think I'll leave it at that -
    since the Class is easy understandable and expandable...

    Attachment 140909

    Performance of Case-Insensitive-TextcompareMode for Scripting-Dictionary and cHashD


    Performance of Binary-TextcompareModes for Scripting-Dictionary and cHashD


    Olaf

  8. #48
    Addicted Member
    Join Date
    Mar 2019
    Posts
    243

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    I have modified the class to be string,string so no use of variants.

    Would the code below be correct for adding remove? I am not sure I have it correct..

    EDIT - Actually I think this is completely wrong. I am not sure how to implement remove.

    Code:
    Public Function Remove(key As String) As Boolean
    
    10        On Error GoTo errorHandler
              
              Dim HashValue As Long, UB As Long, index As Long
    
    20        HashValue = CalcHash(key)
    
    30        index = FindIndex(key, HashTable(HashValue))
           
    40        If index >= 0 Then
           
    50           mKeys(index) = ""
    
    60           With HashTable(HashValue)
              
    70                If .Count > 0 Then
                      
    80                   .DataIndexes(.Count) = 0
                   
    90                   .Count = .Count - 1
               
    100                End If
                       
    110          End With
            
    120          mCount = mCount - 1
              
    130       Else
    
    140          Err.Raise 32811
             
    150       End If
    
    160       Exit Function
              
    errorHandler:
    
    170       errorDisplay Err.description, "hash_Remove", Erl
              
    180       Resume endofitall
              
    endofitall:
    
    End Function
    Last edited by vbwins; May 23rd, 2022 at 06:47 AM.

  9. #49

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    6,050

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    There's an additional zip-file download (with integrated Remove) in post #30:
    https://www.vbforums.com/attachment....7&d=1590156833

    Olaf

  10. #50
    New Member
    Join Date
    May 2021
    Posts
    11

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    Quote Originally Posted by Schmidt View Post
    There's an additional zip-file download (with integrated Remove) in post #30:
    https://www.vbforums.com/attachment....7&d=1590156833

    Olaf
    Please excuse if its me being stupid, but when I use the above object to store arrays, collections or other cHashDs then I get an error with the Keys and Items method.

    The statement which appear to be causing the issue is


    Code:
    Do While s.Values(j) = NoEntry

    I haven't yet tested the Pairs method.

  11. #51

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    6,050

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    Quote Originally Posted by freeflow View Post
    Please excuse if its me being stupid, but when I use the above object to store arrays, collections or other cHashDs then I get an error with the Keys and Items method.

    The statement which appear to be causing the issue is


    Code:
    Do While s.Values(j) = NoEntry

    I haven't yet tested the Pairs method.
    A "complete example" would be nice (which demonstrates the issue).
    From a your short snippet (as it is), I can only recommend:
    - to store the resut of the Values-method in its own (TmpValsArr) variable
    - and then loop over that TmpValsArr instead
    .. you're currently copying the whole Values-Array again and again (in each iteration)

    HTH

    Olaf

  12. #52
    New Member
    Join Date
    May 2021
    Posts
    11

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    OKk here you go. The code below respect that Items and Keys return an Array of Variants rather than a variant containing an array.

    Code:
    Public Sub TestcHashDItems()
        Debug.Print "Retrieving Items from cHashD"
        ' Using CHashD with remove
        Dim myItems As Variant
        Dim mycHashD As cHashD
        Set mycHashD = New cHashD
        '############################################################################
        ' Test NUmbers
        '############################################################################
        mycHashD.Add "One", 10
        mycHashD.Add "Two", 20
        mycHashD.Add "Three", 30
        
        On Error Resume Next
        myItems = Array(Empty, Empty, Empty)
        myItems = mycHashD.Items
        Debug.Print
        Debug.Print "Items are 'Integers'"
        If Err.Number <> 0 Then
                Debug.Print "Error:", Err.Description
        Else
            Debug.Print "Expecting 10,20,30     Found  " ; myItems(0), myItems(1), myItems(2)
        End If
        On Error GoTo 0
        
        '############################################################################
        ' Test Strings
        '############################################################################
        Set mycHashD = New cHashD
        mycHashD.Add "One", "Hello"
        mycHashD.Add "Two", "There"
        mycHashD.Add "Three", "World"
     
        On Error Resume Next
        myItems = Array(Empty, Empty, Empty)
        myItems = mycHashD.Items
        Debug.Print
        Debug.Print "Items are Strings"
        If Err.Number <> 0 Then
                Debug.Print "Error:", Err.Description
        Else
            Debug.Print "Expecting Hello There World     Found  " ; myItems(0), myItems(1), myItems(2)
        End If
        On Error GoTo 0
        
        '############################################################################
        ' Test Arrays
        '############################################################################
        mycHashD.Clear
        mycHashD.Add "One", Array(1, 2, 3)
        mycHashD.Add "Two", Array(4, 5, 6)
        mycHashD.Add "Three", Array(7, 8, 9)
        
        On Error Resume Next
        myItems = Array(Empty, Empty, Empty)
        myItems = mycHashD.Items  ' At this point the debug window in Twinbasic is showing myitems contains "Hello There World"
        Debug.Print
        Debug.Print "Items are arrays"
        If Err.Number <> 0 Then
            Debug.Print "Error:", Err.Description
        Else
            Debug.Print "Expecting 1 5 9     Found  " ; myItems(0)(0), myItems(1)(1), myItems(2)(2)
        End If
        On Error GoTo 0
         
         '############################################################################
         ' Test collection objects
         '############################################################################
         Dim myColl As Collection
         Set myColl = New Collection
         myColl.Add "Item1"
         myColl.Add "Item2"
         myColl.Add "Item3"
         
         Set mycHashD = New cHashD
         mycHashD.Add "One", myColl
         mycHashD.Add "Two", myColl
         mycHashD.Add "Three", myColl
         
         On Error Resume Next
         myItems = Array(Empty, Empty, Empty)
         myItems = mycHashD.Items
         Debug.Print
         Debug.Print "Items are Collections"
         If Err.Number <> 0 Then
                 Debug.Print "Error:", Err.Description
         Else
             Debug.Print "Expecting Item1 Item2 Item 3     Found" ; myItems(0)(1), myItems(1)(2), myItems(2)(3)
         End If
         On Error GoTo 0
         
         
              
         '############################################################################
         ' Test cHashD objects
         '############################################################################
         Dim myTest As cHashD
         Set myTest = New cHashD
         myTest.Add "TestOne", "Item1"
         myTest.Add "TestTwo", "Item2"
         myTest.Add "TestThree", "Item3"
         
         Set mycHashD = New cHashD
         mycHashD.Add "One", myTest
         mycHashD.Add "Two", myTest
         mycHashD.Add "Three", myTest
         
         On Error Resume Next
         myItems = Array(Empty, Empty, Empty)
         myItems = mycHashD.Items
         Debug.Print
         Debug.Print "Items are cHashDs"
         If Err.Number <> 0 Then
                 Debug.Print "Error:", Err.Description
         Else
             Debug.Print "Expecting Item1 Item2 Item3     Found  " ; myItems(0)("TestOne"), myItems(1)("TestTwo"), myItems(2)("TestThree")
         End If
         On Error GoTo 0
         
     End Sub
     
     Public Sub TestcHashDKeys()
        Debug.Print "Retreiving Keys from cHashD"
        Debug.Print
        ' Using CHashD with remove
        Dim myItems As Variant
        Dim mycHashD As cHashD
        Set mycHashD = New cHashD
        '############################################################################
        ' Test NUmbers
        '############################################################################
        mycHashD.Add "One", 10
        mycHashD.Add "Two", 20
        mycHashD.Add "Three", 30
        
        On Error Resume Next
        myItems = Array(Empty, Empty, Empty)
        myItems = mycHashD.Keys
        Debug.Print
        Debug.Print "Items are 'Integers'"
        If Err.Number <> 0 Then
                Debug.Print "Error:", Err.Description
        Else
            Debug.Print "Expecting One Two Three    Found  " ; myItems(0), myItems(1), myItems(2)
        End If
        On Error GoTo 0
        
        '############################################################################
        ' Test Strings
        '############################################################################
        Set mycHashD = New cHashD
        mycHashD.Add "One", "Hello"
        mycHashD.Add "Two", "There"
        mycHashD.Add "Three", "World"
     
        On Error Resume Next
        myItems = Array(Empty, Empty, Empty)
        myItems = mycHashD.Keys
        Debug.Print
        Debug.Print "Items are Strings"
        If Err.Number <> 0 Then
                Debug.Print "Error:", Err.Description
        Else
            Debug.Print "Expecting One Two Three     Found  " ; myItems(0), myItems(1), myItems(2)
        End If
        On Error GoTo 0
        
        '############################################################################
        ' Test Arrays
        '############################################################################
        mycHashD.Clear
        mycHashD.Add "One", Array(1, 2, 3)
        mycHashD.Add "Two", Array(4, 5, 6)
        mycHashD.Add "Three", Array(7, 8, 9)
        
        On Error Resume Next
        myItems = Array(Empty, Empty, Empty)
        myItems = mycHashD.Keys
        Debug.Print
        Debug.Print "Items are arrays"
        If Err.Number <> 0 Then
            Debug.Print "Error:", Err.Description
        Else
            Debug.Print "Expecting One Two Three     Found  " ; myItems(0), myItems(1), myItems(2)
        End If
        On Error GoTo 0
         
         '############################################################################
         ' Test collection objects
         '############################################################################
         Dim myColl As Collection
         Set myColl = New Collection
         myColl.Add "Item1"
         myColl.Add "Item2"
         myColl.Add "Item3"
         
         Set mycHashD = New cHashD
         mycHashD.Add "One", myColl
         mycHashD.Add "Two", myColl
         mycHashD.Add "Three", myColl
         
         On Error Resume Next
         myItems = Array(Empty, Empty, Empty)
         myItems = mycHashD.Keys
         Debug.Print
         Debug.Print "Items are Collections"
         If Err.Number <> 0 Then
                 Debug.Print "Error:", Err.Description
         Else
             Debug.Print "Expecting One Two Three    Found  " ; myItems(0), myItems(1), myItems(2)
         End If
         On Error GoTo 0
         
         
              
         '############################################################################
         ' Test cHashD objects
         '############################################################################
         Dim myTest As cHashD
         Set myTest = New cHashD
         myTest.Add "TestOne", "Item1"
         myTest.Add "TestTwo", "Item2"
         myTest.Add "TestThree", "item"
         
         Set mycHashD = New cHashD
         mycHashD.Add "One", myTest
         mycHashD.Add "Two", myTest
         mycHashD.Add "Three", myTest
         
         On Error Resume Next
         myItems = Array(Empty, Empty, Empty)
         myItems = mycHashD.Keys
         Debug.Print
         Debug.Print "Items are cHashDs"
         If Err.Number <> 0 Then
                 Debug.Print "Error:", Err.Description
         Else
             Debug.Print "Expecting One Two Three Found " ; myItems(0), myItems(1), myItems(2)
         End If
         On Error GoTo 0
         
     End Sub
    Output is

    Code:
    Retrieving Items from cHashD
    
    Items are 'Integers'
    Expecting 10,20,30     Found   10          20            30 
    
    Items are Strings
    Expecting Hello There World     Found  Hello            There         World
    
    Items are arrays
    Error:        Type mismatch
    
    Items are Collections
    Error:        Application-defined or object-defined error
    
    Items are cHashDs
    Expecting Item1 Item2 Item3     Found  Item1            Item2         Item3
    (time taken: 2.7723064s)
    Executing 'NewProject.Pootle.TestcHashDKeys'...
    Retreiving Keys from cHashD
    2
    
    Items are 'Integers'
    Expecting One Two Three    Found  One     Two           Three
    
    Items are Strings
    Expecting One Two Three     Found  One    Two           Three
    
    Items are arrays
    Error:        Type mismatch
    
    Items are Collections
    Error:        Application-defined or object-defined error
    
    Items are cHashDs
    Expecting One Two Three Found One         Two           Three
    (time taken: 0.0009554s)

  13. #53

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    6,050

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    Quote Originally Posted by freeflow View Post
    OKk here you go.
    Thanks, I see what you mean now...

    Easy Fixes (for the faster cHashD-Zip, linked in Post #30):

    1) Replace the "NoEntry"-Const-line with this line:
    Private Const NoEntry As Variant = Empty

    2) Replace all "Do While" lines (in Keys-, Items- and Pairs- Props) with this one:
    Do While IsEmpty(mKeys(j)): j = j + 1: Loop

    And that's it - all your Tests should pass now...
    (I've applied that fix also to the RC6-built-in cHashD in preparation for the next release)

    Quote Originally Posted by freeflow View Post
    The code below respect that Items and Keys return an Array of Variants rather than a variant containing an array.
    Currently no, to respect the Items() and Keys() return-type -
    (and increase performance in enumerations and indexed access) -
    you should define the myItems-Variable in your TestCode:
    - not as: Dim myItems As Variant
    - but as: Dim myItems() As Variant

    HTH

    Olaf

  14. #54
    New Member
    Join Date
    May 2021
    Posts
    11

    Re: Simple and fast, lightweight HashList-Class (no APIs)

    Olaf, many thanks for taking the time to look at this. First my apologies, I edited my code and forgot to post the final version hence the mixup with Variant().

    I can confirm that implementing your recommended changes means that the class now works as expected when storing arrays and collections.

Page 2 of 2 FirstFirst 12

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width