IDE crashing after running a program with subclassing

I have a VB6 program that uses subclassing. As a compiled exe it works fine.

If I run it within the VB IDE it also works fine, but on closing the program the IDE crashes, sometimes immediately and sometimes after a minute or two. I know that I can’t use things like STOP, END, Break, etc within the IDE as these always crashes the IDE, but this is not the problem. The program is closed in the correct way.

This means that each time make any modifications to the program I have to compile it, and then run the exe to test the modifications.
This can get quite laborious....

The subclassed windows are correctly released when the program closes, so at this point there is no longer any subclassing active.
Has anyone any ideas as to why the IDE crashes?

An extract of part of the subclassing code:

In the form load event.
SetWindowSubclass Me.hwnd, AddressOf SubClassProc, ID

In the form unload event.
RemoveWindowSubclass, Me.hwnd, AddressOf SubClassProc, ID

Public Function SubClassProc(ByVal hwnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, _
ByVal uIdSubclass As Long, ByVal dwRefData As Long) As Long
On Error Resume Next

Select Case uMsg
Case WM_NOTIFY
(call some code)

Case UserCustomMesssage
(call some code)
End Select

SubClassProc = DefSubclassProc(hwnd, uMsg, wParam, lParam)
End Function

[...]

Quote:

---

Originally Posted by TedH

This means that each time make any modifications to the program I have to compile it, and then run the exe to test the modifications.
This can get quite laborious....

---

...well, you might mitigate this a little bit:

Make your modifications, edit and continue like normal.
Then before you close your app, copy the code over to a text editor like Notpad++ (switch to Modul-view first)
Then let it crash, start anew, then copy the code back and go on.

At least a work-around for the time beeing ;)

..

Thanks.
I assume the program was ok with the exe, but you might be right about it crashing silently.

I will download that subclassing library and give it a try.
The program contains a lot of API calls, and there are some "CopyMemory" calls in it. I will check these again.

http://www.vbforums.com/showthread.p...ws-and-classes

Hi Ted,

What's most obvious to me is that you've got an extra comma in your call to RemoveWindowSubclass. I'm not sure you're getting it removed.

Quote:

---

Originally Posted by TedH

RemoveWindowSubclass, Me.hwnd, AddressOf SubClassProc, ID

---

That actually makes me wonder how you've got RemoveWindowSubclass declared. I wouldn't mind seeing that too. Here are mine:

Code:

---


Private Declare Function SetWindowSubclass Lib "comctl32.dll" Alias "#410" (ByVal hWnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long, Optional ByVal dwRefData As Long) As Long
Private Declare Function GetWindowSubclass Lib "comctl32.dll" Alias "#411" (ByVal hWnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long, pdwRefData As Long) As Long
Private Declare Function RemoveWindowSubclass Lib "comctl32.dll" Alias "#412" (ByVal hWnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long) As Long
Private Declare Function DefSubclassProc Lib "comctl32.dll" Alias "#413" (ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long) As Long


---

Good Luck,
Elroy

Private Declare Function DefSubclassProc Lib "comctl32.dll" (ByVal hwnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long) As Long
Private Declare Function SetWindowSubclass Lib "comctl32.dll" (ByVal hwnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long, ByVal dwRefData As Long) As Long
Private Declare Function RemoveWindowSubclass Lib "comctl32.dll" (ByVal hwnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long) As Long

GetWindowSubclass not used.

However, the subclassing all works fine with the program running from an EXE or within the IDE. Whilst running, various windows are opened and subclassed, and later de-subclassed and closed with no apparent problems.
It's only when I close the program (when running it in the IDE) that the IDE crashes.

As mentioned in post #2, this may not be truly subclass related, but CopyMemory related. When we use subclassing, we almost always use CopyMemory to get subclass parameter data. If you are not using CopyMemory at all, then ignore this reply.

Quote:

---

Originally Posted by LaVolpe

As mentioned in post #2, this may not be truly subclass related, but CopyMemory related. When we use subclassing, we almost always use CopyMemory to get subclass parameter data. If you are not using CopyMemory at all, then ignore this reply.

---

It does use CopyMemory.

Private Type ENLINK
hwndFrom As Long
idFrom As Long
Code As Long
msg As Long
wParam As Long
lParam As Long
cpMin As Long
cpMax As Long
End Type

....

Dim udtENLINK As ENLINK

Case WM_NOTIFY
CopyMemory udtENLINK, ByVal lParam, Len(udtENLINK)
If udtENLINK.Code = EN_LINK Then
....

If that's the only usage of CopyMemory in your project, nothing there to be concerned with unless lParam is zero. You should validate that.

Subclassing a VB richtext control or an API-generated one?

Just saw something and maybe you can clarify.

You are using CopyMemory with the WM_NOTIFY message. That message specifically defines the lParam as a NMHDR structure. That structure only contains 3 Longs, not 8. This could be the cause of your crashes.

Also, there's still something wrong here. Ted, you say this is your SetWindowSubclass declaration ...

Code:

---


Private Declare Function SetWindowSubclass Lib "comctl32.dll" (ByVal hwnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long, ByVal dwRefData As Long) As Long


---

... and that this is your call to it ...

Code:

---


SetWindowSubclass Me.hwnd, AddressOf SubClassProc, ID


---

The way you've got it set up, I don't know how you're even getting it compiled. Your SetWindowSubclass declaration has four arguments, and your call only has three. Also, I wasn't aware that the SetWindowSubclass and related API calls had a non-numeric exposed entry-point. Maybe they do on newer versions of the comctl32.dll, not sure.

Also, it sure looks like LaVolpe is onto something as well.

Good Luck,
Elroy

Quote:

---

Originally Posted by Elroy

I wasn't aware that the SetWindowSubclass and related API calls had a non-numeric exposed entry-point

---

As of XP. Win2K and earlier (if applicable), ordinal only. Regarding the missing parameter; probably a typo in his posting; else if that parameter not optional (as he posted as his declaration), VB wouldn't have run the project... Wouldn't focus on that.

@LaVolpe: Yeah, I sort of suspected that the posts were full of typos, but it's hard to diagnose when you're not sure what you're looking at.

@TedH: Also, just an FYI, you got away with it this time, but it's highly advisable to use LenB() and not Len() to get the size of a UDT. In this specific case, it doesn't matter because they're all longs, and UDTs have four-byte padding, but it will often matter for other UDTs.

Good Luck Getting It Worked Out,
Elroy

Quote:

---

Originally Posted by Elroy

Also, there's still something wrong here. Ted, you say this is your SetWindowSubclass declaration ...

Code:

---


Private Declare Function SetWindowSubclass Lib "comctl32.dll" (ByVal hwnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long, ByVal dwRefData As Long) As Long


---

... and that this is your call to it ...

Code:

---


SetWindowSubclass Me.hwnd, AddressOf SubClassProc, ID


---

The way you've got it set up, I don't know how you're even getting it compiled. Your SetWindowSubclass declaration has four arguments, and your call only has three. Also, I wasn't aware that the SetWindowSubclass and related API calls had a non-numeric exposed entry-point. Maybe they do on newer versions of the comctl32.dll, not sure.

Also, it sure looks like LaVolpe is onto something as well.

Good Luck,
Elroy

---

Sorry, that was a cut&past error.
it is actually SetWindowSubclass Me.hwnd, AddressOf SubClassProc, ID, 0

Quote:

---

Originally Posted by Elroy

@LaVolpe: Yeah, I sort of suspected that the posts were full of typos, but it's hard to diagnose when you're not sure what you're looking at.

@TedH: Also, just an FYI, you got away with it this time, but it's highly advisable to use LenB() and not Len() to get the size of a UDT. In this specific case, it doesn't matter because they're all longs, and UDTs have four-byte padding, but it will often matter for other UDTs.

Good Luck Getting It Worked Out,
Elroy

---

Thanks,
Point noted!

Quote:

---

Originally Posted by LaVolpe

Just saw something and maybe you can clarify.

You are using CopyMemory with the WM_NOTIFY message. That message specifically defines the lParam as a NMHDR structure. That structure only contains 3 Longs, not 8. This could be the cause of your crashes.

---

I am getting confused here.

Windows defines the ENLINK structure as such.
Contains information about an EN_LINK notification code from a rich edit control.
typedef struct {
NMHDR nmhdr;
UINT msg;
WPARAM wParam;
LPARAM lParam;
CHARRANGE chrg;
} ENLINK;

The first part is indeed a NMHDR Structure. The next 3 contain further info about the notification, which my code makes use of.
As in: If udtENLINK.msg = WM_LBUTTONDOWN Then LinkProcess(hwnd, udtENLINK.cpMin, udtENLINK.cpMax)

This does work, so I assumed it must be correct?

I suppose it SHOULD have been defined as:
Type NMHDR
hwndFrom As Long
idFrom As Long
code As Long
End Type

Type CHARRANGE
cpMin As Long
cpMax As Long
End Type

Private Type ENLINK
tNMHDR As NMHDR
msg As Long
wParam As Long
lParam As Long
chrg As CHARRANGE
End Type

Quote:

---

Originally Posted by TedH

I am getting confused here.

Windows defines the ENLINK structure as such.
Contains information about an EN_LINK notification code from a rich edit control.
typedef struct {
NMHDR nmhdr;
UINT msg;
WPARAM wParam;
LPARAM lParam;
CHARRANGE chrg;
} ENLINK;

The first part is indeed a NMHDR Structure. The next 3 contain further info about the notification, which my code makes use of.
As in: If udtENLINK.msg = WM_LBUTTONDOWN Then LinkProcess(hwnd, udtENLINK.cpMin, udtENLINK.cpMax)

---

Granted, but not every WM_NOTIFY message is going to contain an ENLINK structure or nothing other than just the NMHDR. WM_NOTIFY is more or less generic and can be referencing structures smaller in size than ENLINK.

You should be able to test this in your subclass:

Code:

---

If udtENLINK.Code <> EN_LINK Then Debug.Print "WM_NOTIFY and not a EN_LINK"

---

Edited. To be clear, this line is likely the problem: CopyMemory udtENLINK, ByVal lParam, Len(udtENLINK)
Why? You are always copying 32 bytes (8 longs) even if the structure pointed to by lParam has less.

However, I'd suspect an immediate crash if trying to read unallocated/unauthorized memory vs. a delayed crash as you are describing. In any case, that part of the code is incorrect.

Suggest this instead:

Code:

---

Case WM_NOTIFY
CopyMemory udtENLINK, ByVal lParam, 12 ' copy just the first 3 longs
If udtENLINK.Code = EN_LINK Then
    CopyMemory udtENLINK.msg, ByVal lParam + 12, Len(udtENLINK) - 12
...

---

Many on this site might cringe at using lParam+12 and it may carry a small risk in some scenarios of the pointer math overflowing. If worried about it, you can just simply recopy the structure, in total this time, after you verify it is an ENLINK structure

Code:

---

Case WM_NOTIFY
CopyMemory udtENLINK, ByVal lParam, 12 ' copy just the first 3 longs
If udtENLINK.Code = EN_LINK Then
    CopyMemory udtENLINK, ByVal lParam, Len(udtENLINK)
...

---

I have now changed the code to

Dim udtENLINK As ENLINK, tNMHDR As NMHDR
...
Case WM_NOTIFY
CopyMemory tNMHDR, ByVal lParam, Len(tNMHDR)
If tNMHDR.code = EN_LINK Then
CopyMemory udtENLINK, ByVal lParam, Len(udtENLINK)
If udtENLINK.msg = WM_LBUTTONDOWN Then ....

Seems to work ok, so far.

As a general question,
If a CopyMemory does copy more bytes than it should, does simply copying extra bytes cause a problem?
Would it not just copy what happened to be there (which may be rubbish), or would it upset Windows with some sort of invalid memory access problem?

I edited my reply to address that. VB uses stdCall convention which means the callee (code placing stuff in memory) is responsible for cleanup. Therefore, if you can copy past the allocated memory without errors, your code won't be releasing the memory the memory just copied. This is why I'd expect an immediate vs delayed crash if attempting to read restricted/unallocated memory. The crashes you are experiencing could be exactly what I just described, but it is hit or miss depending on what memory block the callee has placed the data.

CopyMemory is fairly low level memory movement. Do it wrong and a crash is not far behind.