[RESOLVED] Avoiding Memory Leaks - CopyMemory

[Quiver318]

Hello all,

1. Am I correct in assuming that there is no need to clean up after using CopyMemory API when it is used to copy between two VB6 declared arrays? It does not have it's own secret cache, or anything, does it?

2. Is CopyMemory a problem in any version of Windows now? I thought I had read that it was replaced by another method.

Form level general declarations

Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" (Destination As Any, Source As Any, ByVal Length As Long)

Dim PicNew()  As RGBQUAD      '// Holds new picture for manipuation
Dim PicOrg()  As RGBQUAD       '// Holds original jpg picture source
Dim OrgLng    As Long             '// Holds the picture size for CopyMemory API

Form level code below needs two picture boxes, one named PicNew and one named PicOrg. PicOrg needs an embedded JPG picture.

' // Create a buffer for photos using VB6
 ReDim PicNew(0 To Pic.ScaleWidth - 1, 0 To Pic.ScaleHeight - 1)
 ReDim PicOrg(0 To Pic.ScaleWidth - 1, 0 To Pic.ScaleHeight - 1)

' // Get the size of the original photo for CopyMemory to use
OrgLng = (UBound(PicOrg, 1) + 1) * (UBound(PicOrg, 2) + 1) * 4

' // Copy from one VB6 declared array to another
CopyMemory PicNew(0, 0), PicOrg(0, 0), OrgLng

Thanks!

Quiver

[Milk]

There will be no leak issues copying from one declared array to another. If the copied data comprises of reference types then you have to manually dereference one of them and allow the other to be cleaned up by the garbage collector. You are not using reference types so not an issue for you.

I am not aware of issues with RtlMoveMemory on newer versions of windows, MSDN makes no mention of any.

[jemidiah]

CopyMemory doesn't need to be cleaned up if you're copying between variables you own, since the VB garbage collector manages them entirely. I have no idea how a special cache might help CopyMemory without being an awful headache to programmers, though it certainly benefits from hardware caching. I've never heard of CopyMemory being replaced.

One thing I should mention is that I don't believe all VB6 arrays are laid out sequentially in memory, so CopyMemory may or may not be useful. I imagine for something named RGBQuad this isn't an issue.

Edit: Actually, Milk makes a good point about reference types. CopyMemory could screw up the garbage collector's reference counting in that case, I think.

[Quiver318]

Thank you both kindly for the replies, Milk and Jemidiah. I believe this answers my question, exactly.

Quiver

[LaVolpe]

One thing I should mention is that I don't believe all VB6 arrays are laid out sequentially in memory, so CopyMemory may or may not be useful. I imagine for something named RGBQuad this isn't an issue.

Edit: Actually, Milk makes a good point about reference types. CopyMemory could screw up the garbage collector's reference counting in that case, I think.

FYI. Array's elements are always contiguous.
Using copy array on arrays of: strings, variants, UDTs (depending on what they contain), objects (i.e., fonts, stdPictures) requires caution and more work to avoid both memory leaks and potential of crashing during garbage collection.

[Quiver318]

FYI. Array's elements are always contiguous.
Using copy array on arrays of: strings, variants, UDTs (depending on what they contain), objects (i.e., fonts, stdPictures) requires caution and more work to avoid both memory leaks and potential of crashing during garbage collection.

Thank you so much, LaVolpe. Your comments and knowledge are always appreciated.

[jemidiah]

In another thread I was looking at an in-memory UDT array, where the UDT contained variable-length strings and further UDTs. It didn't seem to be laid out sequentially, which I couldn't account for and nobody responded to. I could dig up the example again; maybe I just misread the memory, though the strange thing was I could clearly read off the first array element, but subsequent ones were completely garbled. My best guess was that VB used a linked list instead of a sequential array, but I didn't want to go hunting around in memory to confirm the suspicion.

[LaVolpe]

In another thread I was looking at an in-memory UDT array, where the UDT contained variable-length strings and further UDTs. It didn't seem to be laid out sequentially, which I couldn't account for and nobody responded to....

The UDT is layed out contiguously. But one has to keep in mind that variable length strings and other things that require pointers, only use 4 bytes of the UDT. One cannot think of a UDT and all of its members layed out in the same contiguous memory as a nice neat package, because it may not be. However, and this is what matters, from one UDT to the next, they will be the same distance apart in memory, next to each other.
So, VarPtr(myUdt(1)) - VarPtr(myUdt(0)) = LenB(myUdt(0))

In your case, those strings were 4 byte pointers to other memory locations.
Here is an entertaining & somewhat in-depth thread on CopyMemory that I participated in, 2 years ago.

[jemidiah]

Thanks for the link. I understand that references (eg. variable length string members) in UDTs are just pointers so that the UDT itself should be fixed-length and it should be possible to lay out an array of UDTs contiguously in memory. That it didn't appear to be laid out in this manner is what surprised me. I'll dig up the thread and see if I can reproduce it, or find out if I was just seeing things.

[jemidiah]

Here we go. I don't believe I was seeing things.

The attached project uses a UDT

Private Type MyUDT
    b1 As Integer
    b2 As Integer
'    s As String
End Type

It really uses two versions of this UDT--one with s and one without. This is either 4 or 8 bytes in length, to avoid any possible alignment issues. I make an array of 21 of these UDTs and populate two of them (the rest are default initialized). I then copy that memory into a byte array and output the byte array to a textbox (in hex).

In the first case, without s, I populate with

    With MyUDTs(0)
        .b1 = &H2357
        .b2 = &H1248
    End With
    With MyUDTs(12)
        .b1 = &H1234
        .b2 = &H5678
    End With

which gives a textbox of

 57 23 48 12
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 34 12 78 56
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0
 0 0 0 0

This is precisely as I expect, considering I'm using a little endian machine. You can clearly see the values I stuck in, and there are 0's everywhere else for the default initialized data. However, adding in the s garbles things. Initializing with

    With MyUDTs(0)
        .b1 = &H2357
        .b2 = &H1248
        .s = "this is my string"
    End With
    With MyUDTs(12)
        .b1 = &H1234
        .b2 = &H5678
        .s = "and this is too"
    End With

produces

 57 23 48 12 CC CC EF 5
 0 88 C8 5 0 88 C8 5
 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0
 D0 87 C8 5 F4 F8 17 0
 C4 F9 17 0 1 0 0 0
 3E F6 E5 75 78 40 59 77
 FE 4 2E 0 A0 19 C0 F
 0 0 0 0 0 0 0 0
 1 0 0 0 0 0 0 0
 60 E3 EA 5 C E4 EC 5
 70 E5 EA 5 C5 0 0 0
 0 E4 0 0 0 D0 C0 F
 0 0 0 0 0 0 0 0
 0 0 0 0 0 0 0 0
 53 E4 EA 5 18 19 C0 5
 A0 FD 17 0 E0 A6 BD F
 A0 19 C0 F 60 F7 17 0
 E8 F8 17 0 22 E4 EA 5
 0 D0 C0 F 70 E5 EA 5
 0 0 0 0 0 0 0 0

You can see MyUDTs(0).b1 and .b2 in the first 4 bytes of the first line. Presumably the next 4 bytes are just the pointer to the .s string. However, the next 4 bytes should be 0, since they were default initialized--they're not. In fact, the next 8 bytes repeat the 4 byte sequence "0 88 C8 5", which is probably a memory address. I haven't investigated further, but it doesn't appear to be contiguous.

Edit: I just noticed that the MyUDTs(0).s pointer address, 0x05EFCCCC, and the mysterious repeated sequence, 0x05C88800, are quite close in memory, which gives even more credence to the thought that the repeated sequence is a pointer.

Further edit: Also, sorry to the OP for hijacking this thread, though you seem to have had your questions satisfactorily answered.

[LaVolpe]

Two things.

1. Your array size is 1 too big. At 8 bytes per UDT & 21 array items, that's 168 bytes & your array is 169
If you resize your array to one less, add back the +1 that I removed from the CopyMemory lines below.
2. But here's the problem. Just one thing wrong with CopyMemory & all bets are off.

Instead of: CopyMemory MyBytes(0), MyUDTs(0), UBound(MyBytes)
Use: CopyMemory MyBytes(0), MyUDTs(0).b1, UBound(MyBytes)

Edited: Honestly don't quite fully understand why. It has something to do with the way VB is passing the reference maybe?
VarPtr(myUDTs(0)) = VarPtr(myUDTs(0).b1)
Likewise, this works... CopyMemory MyBytes(0), ByVal VarPtr(MyUDTs(0)), UBound(MyBytes)
as well as this: CopyMemory MyBytes(0), ByVal VarPtr(MyUDTs(0).b1), UBound(MyBytes)

[jemidiah]

1. Your array size is 1 too big.

Good catch, though it doesn't effect the test.

2. But here's the problem. Just one thing wrong with CopyMemory & all bets are off.

I don't see what I wrote as an error considering, as you mentioned, VarPtr(myUDTs(0)) = VarPtr(myUDTs(0).b1). But, it does work with that change. I'm confused as to why it works, as well. This behavior is very screwy. The issue only comes up when the string is added, too. Why would VB's reference passing depend on the contents of the variable in any way?

VB is creating a copy of MyUDTs(0) for some reason. I tried swapping to MyUDTs(12), and the general structure of the output is very similar, though not quite identical; MyUDTs(12) also appears as the first 8 bits. I suppose I'd have to say VB is making a copy of MyUDTs(n) and passing a pointer to that copy, and that this behavior is only triggered under mysterious conditions having to do with the UDT's structure and perhaps the phase of the moon. I'm gonna classify this behavior as a bug. Since it's a pretty advanced (and obscure) scenario, I could see it slipping under the radar all these years.


At least VB does just use contiguous arrays after all, even for UDTs .

[LaVolpe]

I've never come across that before; but then again I always pass UDTs ByVal VarPtr() vs ByRef to CopyMemory; just a force of habit. But from your test. Not using ByVal, VB is obviously not passing the VarPtr of the UDT array member. One would need a debugger to see what address it is passing instead. But then one would have to be extremely curious to go that route too. Well, lesson learned for all hopefully.

As far as the array size goes. Though it didn't effect the test; it could have resulted in a crash if that last byte that attempted to be copied happened to be uninitialized or restricted memory.

[jemidiah]

One would need a debugger to see what address it is passing instead.

I don't see how even having the address would be helpful. Most likely VB is just copying MyUDTs(0) and passing the copy's address, in which case the address would be an unassuming heap location. I suppose there's a chance it's using the stack, which would be a little more interesting, though still apparently unhelpful.

As far as the array size goes. Though it didn't effect the test; it could have resulted in a crash if that last byte that attempted to be copied happened to be uninitialized or restricted memory.

So actually in this case, I had used

CopyMemory MyBytes(0), MyUDTs(0), UBound(MyBytes)

which copies the correct number of bytes, 21*8 = 168, since UBound(MyBytes) = 168. However MyBytes has 169 elements, making the last one unused. It couldn't have led to a crash, but it is slightly inefficient, and a conceptual error. I should have subtracted 1 from the upper bound of MyBytes and used UBound(MyBytes)+1 in the CopyMemory call (or equivalently, the full UBound(MyBytes) - LBound(MyBytes) + 1).

Regardless, it's good to note the mistake in case someone reads this thread in the future and uses my code for non-throw-away purposes.


So, the moral of the story is, don't rely on ByRef when using CopyMemory; use ByVal and VarPtr instead.

[Milk]

I am getting a little out of touch but, I assume you both know that when passing strings to API, VB makes a temporary ANSI copy of the string and passes the temporary copy instead. By passing the first element ByRef VB knows you are passing a reference to something which contains a string so it makes a temporary ANSI copy of the first element (and just the first element) and passes that. Hence apparent garbage. If you followed that string pointer I bet you find an ANSI string.

The moral of the story should perhaps be, be careful with ByRef strings and API.

[LaVolpe]

Yep, Milk. That's probably it. VB passing a temporary buffer.

[jemidiah]

I actually never knew VB made a copy like that, but it makes sense since I've never had to deal with string conversion for API calls. I tried following the pointer at location 4-7 in my byte array by reading it into a long and reading the data at that location into a further byte array, and it led to garbage instead of an ANSI string. There's a chance VB destroys the buffer after the API call or perhaps the buffer is on the stack and so gets destroyed by subsequent calls.

In any case, I don't find it worthwhile investigating further. My curiosity has been satisfied well enough by the reasonable explanation that VB makes an ANSI-compatible version of the UDT on the stack when passed ByRef, in the background adding conversion code before and after the call. When ByVal VarPtr's are used instead, the ANSI-compatibility mechanism doesn't get triggered. When MyUDTs(0).b1 is passed ByRef instead, the mechanism also doesn't get triggered since no compatibility conversion is needed.

This has been interesting!

[LaVolpe]

...When MyUDTs(0).b1 is passed ByRef instead, the mechanism also doesn't get triggered since no compatibility conversion is needed.

Just to clarify for anyone else reading this in the future....

Passing MyUDTs(0).b1 requires no conversion because what is being passed is a pointer to that .b1 member, an Integer datatype. And .b1 happens to be the first member of the UDT. But passing MyUDTs(0) causes VB to look at the entire UDT and because of strings and/or other pointer-related members, prepares the temporary buffer and passes that.

If .b1 was a String vs. Integer, then passing MyUDTs(0).b1 should result in garbage also or a crash in worse-case scenario.

In short, passing any array item ByRef can cause unexpected results or crashes unless you know for sure, there will be no VB pre-handling involved. Better rule of thumb is to use template of: ByVal VarPtr(arrayItem(n)) vs. just arrayItem(n)