Array ReDim performance

[Merri]

Continuing from String Optimization thread.

I repeat: When you need to write a solution where you can't possibly know how many items your array will eventually need -- anywhere from hundreds to millions -- what is the best way to grow that array inside a loop?

Regarding this I made a new benchmark. I got interested on when things start to change so that another method becomes better than the others.

I ended up with this function:

Private Function BestSize(ByVal Items As Long) As Long
    If Items < 3 Then
        BestSize = 3
    ElseIf Items < 2000 Then
        BestSize = Items * 2 - 1
    ElseIf Items < 20000 Then
        BestSize = Items * 1.5 - 1
    Else
        BestSize = Items * 1.25 - 1
    End If
End Function

The idea is to pass UBound value when doing ReDim preserve. On my tests it seems to be the fastest on reaching 20000 items. After that it hits memory save mode and increases by a lower margin. Doubling would be in most cases quite insane on memory usage.

This was quite hard for me to benchmark. Since I was running a lot of other applications while testing I sometimes got insanely fast results and other times things got very slow in comparison. The final size of the array also seemed to affect results in some ways when running multiple comparative tests. In conclusion, OS memory management has a great impact on final result.


If somebody has a cleaner testing environment you're welcome to test. The program I used can be found at http://kontu.selfip.info/vb6/project.../Array_growth/

[techgnome]

the trick to this is determining the threashold and knowing when to grow it (do you grow it when the array is full or close to being full - think FILL 90% when building indexes in SQL) ... and by how much - I'm usually skepticle of growths that are pergentage based on existing sizes... I'd hate to grow it by an extra 25% when I only needed maybe 10 more elements.... but then at the time Iwouldn't know that I only needed 10 more ... Perhapse a shrinking of the array follwing the filling would be beneficial.

-tg