[VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass method
So there's quite a few posts about specific questions where code like this is shown, but I really thought it would be better to have a subclassing tutorial type codebank entry for it to make it more easily searchable, and a better answer for when somebody hasn't used subclassing before.
Since Windows XP, there's been some great subclassing setups posted showing how to use SetWindowSubclass as a newer, better subclassing method than previous SetWindowLong-based methods. Karl E. Peterson's HookXP and LaVolpe's Subclassing With Common Controls Library being two top notch examples, and this method was first shown to me by Bonnie West over in my ITaskBarList3 demo. But before delving into using interfaces etc, (and even still for subclassing things other than forms and the like), it's helpful to show how to do the very simplest of subclassing: When you have any object, and just want to have a code to handle a message without much fuss.
The subclassing example I picked for this is validating whether text typed into a textbox is a valid filename, and blocking the input altogether if the keystroke or paste operation contains an illegal character or is too long. (You could do this without subclassing in most scenarios, but I thought it was a nice and simple way to get the idea across.)
All it requires on the form is a single textbox.
Once you have that, you'll need to create the function that handles its messages in a module. All such functions, usually referred to as the WndProc, have the same arguments. Also, they all must unsubclass when the window is being destroyed (or before) otherwise the program will crash. So before adding any code for handling messages, the basic prototype looks like this:
Code:
Public Function EditWndProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, ByVal uIdSubclass As Long, ByVal dwRefData As Long) As Long
Select Case uMsg
'[other messages will go here later]
Case WM_DESTROY
Call UnSubclass(hWnd, PtrEditWndProc)
End Select
EditWndProc = DefSubclassProc(hWnd, uMsg, wParam, lParam)
End Function
Private Function PtrEditWndProc() As Long
PtrEditWndProc = FARPROC(AddressOf EditWndProc)
End Function
The DefSubclassProc call passes everything back to the default handler if you're not completely handling it. The PtrEditWndProc is there because you can't use the AddressOf operator to get the address of the function it's in.
Now that you have a function to handle the messages, you can add the code to start the subclass back in the form:
Code:
Private Sub Form_Load()
Call Subclass(Text1.hWnd, AddressOf EditWndProc)
End Sub
And now you have a basic subclass all set up and ready to go. You don't need an Unsubclass in Form_Unload.
Here's the message handlers used to validate input for a textbox looking for a valid file name (and appears right after Select Case uMsg:
Code:
Case WM_CHAR
Dim lLen As Long
lLen = SendMessageW(hWnd, WM_GETTEXTLENGTH, 0, ByVal 0&) + 1
If lLen > 260 Then
Beep
wParam = 0
Call ShowBalloonTipEx(hWnd, "", "Maximum number of characters has been reached. The total length of the file name cannot exceed 260 characters.", TTI_NONE) ' TTI_ERROR)
Exit Function
End If
Select Case wParam
Case 47, 92, 60, 62, 58, 42, 124, 63, 34 'Illegal chars /\<>:*|?"
Beep
Call ShowBalloonTipEx(hWnd, "", "File names may not contain any of the following characters:" & vbCrLf & " / \ < > : ? * | " & Chr$(34), TTI_NONE) ' TTI_ERROR)
wParam = 0
End Select
Case WM_PASTE
Dim iCheck As Integer
iCheck = IsClipboardValidFileName()
If iCheck = 0 Then
Beep
Call ShowBalloonTipEx(hWnd, "", "File names may not contain any of the following characters:" & vbCrLf & " / \ < > : ? * | " & Chr$(34), TTI_NONE) ' TTI_ERROR)
Exit Function
ElseIf iCheck = -1 Then
Beep
Call ShowBalloonTipEx(hWnd, "", "The file name you have entered is too long. The total length of the file name cannot exceed 260 characters.", TTI_NONE) ' TTI_ERROR)
Exit Function
End If
Here's what the full project looks like with all the supporting codes and declares added in:
Form1
Code:
Option Explicit
Private Sub Form_Load()
Call Subclass(Text1.hWnd, AddressOf EditWndProc)
End Sub
mSubclass
Code:
Option Explicit
Private Declare Function DefSubclassProc Lib "comctl32.dll" Alias "#413" (ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long) As Long
Private Declare Function SetWindowSubclass Lib "comctl32.dll" Alias "#410" (ByVal hWnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long, Optional ByVal dwRefData As Long) As Long
Private Declare Function RemoveWindowSubclass Lib "comctl32.dll" Alias "#412" (ByVal hWnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long) As Long
Private Declare Function SendMessageW Lib "user32" (ByVal hWnd As Long, ByVal wMsg As Long, ByVal wParam As Long, lParam As Any) As Long
Private Type EDITBALLOONTIP
cbStruct As Long
pszTitle As Long
pszText As Long
ttiIcon As BalloonTipIconConstants ' ; // From TTI_*
End Type
Private Enum BalloonTipIconConstants
TTI_NONE = 0
TTI_INFO = 1
TTI_WARNING = 2
TTI_ERROR = 3
End Enum
Private Const WM_CHAR = &H102
Private Const WM_PASTE = &H302
Private Const WM_DESTROY = &H2
Private Const WM_GETTEXTLENGTH = &HE
Private Const ECM_FIRST As Long = &H1500
Private Const EM_SHOWBALLOONTIP As Long = (ECM_FIRST + 3)
Public Function Subclass(hWnd As Long, lpfn As Long) As Long
Subclass = SetWindowSubclass(hWnd, lpfn, 0)
End Function
Public Function UnSubclass(hWnd As Long, lpfn As Long) As Long
'Only needed if you want to stop the subclassing code and keep the program running.
'Otherwise, the WndProc function should call this on WM_DESTROY
UnSubclass = RemoveWindowSubclass(hWnd, lpfn, 0)
End Function
Public Function EditWndProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, ByVal uIdSubclass As Long, ByVal dwRefData As Long) As Long
Select Case uMsg
Case WM_CHAR
Dim lLen As Long
lLen = SendMessageW(hWnd, WM_GETTEXTLENGTH, 0, ByVal 0&) + 1
If lLen > 260 Then
Beep
wParam = 0
Call ShowBalloonTipEx(hWnd, "", "Maximum number of characters has been reached. The total length of the file name cannot exceed 260 characters.", TTI_NONE) ' TTI_ERROR)
Exit Function
End If
Select Case wParam
Case 47, 92, 60, 62, 58, 42, 124, 63, 34 'Illegal chars /\<>:*|?"
Beep
Call ShowBalloonTipEx(hWnd, "", "File names may not contain any of the following characters:" & vbCrLf & " / \ < > : ? * | " & Chr$(34), TTI_NONE) ' TTI_ERROR)
wParam = 0
End Select
Case WM_PASTE
Dim iCheck As Integer
iCheck = IsClipboardValidFileName()
If iCheck = 0 Then
Beep
Call ShowBalloonTipEx(hWnd, "", "File names may not contain any of the following characters:" & vbCrLf & " / \ < > : ? * | " & Chr$(34), TTI_NONE) ' TTI_ERROR)
Exit Function
ElseIf iCheck = -1 Then
Beep
Call ShowBalloonTipEx(hWnd, "", "The file name you have entered is too long. The total length of the file name cannot exceed 260 characters.", TTI_NONE) ' TTI_ERROR)
Exit Function
End If
Case WM_DESTROY
Call UnSubclass(hWnd, PtrEditWndProc)
End Select
EditWndProc = DefSubclassProc(hWnd, uMsg, wParam, lParam)
End Function
Private Function PtrEditWndProc() As Long
PtrEditWndProc = FARPROC(AddressOf EditWndProc)
End Function
Private Function FARPROC(pfn As Long) As Long
FARPROC = pfn
End Function
Private Sub ShowBalloonTipEx(hWnd As Long, sTitle As String, sText As String, btIcon As BalloonTipIconConstants)
Dim lR As Long
Dim tEBT As EDITBALLOONTIP
tEBT.cbStruct = LenB(tEBT)
tEBT.pszText = StrPtr(sText)
tEBT.pszTitle = StrPtr(sTitle)
tEBT.ttiIcon = btIcon
lR = SendMessageW(hWnd, EM_SHOWBALLOONTIP, 0, tEBT)
End Sub
Public Function IsClipboardValidFileName() As Integer
Dim i As Long
Dim sz As String
Dim sChr As String
'there's a couple scenarios for invalid file names i've trimmed out
'to keep this example as simple as possible, look into them if you're
'going to use this code in an actual rename procedure
sz = Clipboard.GetText
If Len(sz) > 260 Then
IsClipboardValidFileName = -1
Exit Function
End If
IsClipboardValidFileName = 1
If InStr(sz, "*") Or InStr(sz, "?") Or InStr(sz, "<") Or InStr(sz, ">") Or InStr(sz, "|") Or InStr(sz, Chr$(34)) Then
IsClipboardValidFileName = 0
End If
End Function
It's still a little complicated, but this is the very simplest way to get subclassing going.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
This is a good technique we've had available since IE 4.x came out. It wasn't documented and the entrypoints were not exported by name until comctl32.dll version 6.0 in Windows XP came out, so many people seem to be unaware of it even today.
Even Karl E. Peterson blamed this on the blinders worn by so many in the VB community, which tends to have a lot of folks who reject change and turned their backs on Windows XP for many years... until it became their new Mother and then Vista became the new whipping boy.
But by accessing these by ordinal as shown above they work as far back as Windows 95 as long as the IE 4.x Desktop Update is installed.
Last edited by dilettante; May 11th, 2015 at 10:56 AM.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Good article! I will say a couple of my words.
It's just a call SetProp (GlobalAddAtom ("CC32SubclassInfo"), NewWndProc) with additional checks. Also, addition in the special table of parameter information. When processing each message is searched in the table and additional checks that slows down the program. So why additional load on the CPU? SetWindowSubclass - an add-on SetWindowLong. If you chase performance, it is best to discard any wrappers on standard functions.
If you need modularity, it would be possible to make subclassing in a class module. Then you can pass any number of parameters.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by The trick
If you chase performance, it is best to discard any wrappers on standard functions.
Raymond Chen gave a very good reason why one should prefer the newer subclassing APIs: they're safer.
Originally Posted by The trick
If you need modularity, it would be possible to make subclassing in a class module.
As demonstrated here, SetWindowSubclass makes it easier (and neater) to redirect a subclass procedure to the desired Object module.
On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Bonnie West
Raymond Chen gave a very good reason why one should prefer the newer subclassing APIs: they're safer.
I did not see the advantages. With the same success I can instead cause DefSubclassProc - CallWindowProc.
Originally Posted by Bonnie West
As demonstrated here, SetWindowSubclass makes it easier (and neater) to redirect a subclass procedure to the desired Object module.
To transfer user information there are many ways. SetProp, GWL_USERDATA, etc. Just SetWindowSubclass uses SetProp to save the previous window procedure. I also developed a class for safe subclassing using only one class.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
All that's great, but the idea here is to introduce people to writing subclassing code in the simplest way possible. I'm a big fan of approaching things with complex methods when simpler ones exist (see: half my code bank submissions), but went the opposite way here.
Although something I'm curious about, have you found any way to subclass that doesn't crash the IDE with most errors? (besides an external dll)
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by The trick
I did not see the advantages.
Try the attached test projects below to see what I mean by "safer".
On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Bonnie West
Try the attached test projects below to see what I mean by "safer".
Bonnie
I have been using Paul Caton,s SubClassing technique with a formless window to receive system messages. That technique involves injecting assembler code with debug code to avoid problems in the IDE, but SetWindowSubclass seems a lot simpler. Would that work for such things as my NewSocket.cls? I thought I would seek your opinion before I venture forth.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by couttsj
... but SetWindowSubclass seems a lot simpler. Would that work for such things as my NewSocket.cls? I thought I would seek your opinion before I venture forth.
I haven't taken a close look at your NewSocket.cls yet, but I believe it should be possible to convert your existing subclassing logic to one that uses SetWindowSubclass instead. I can't think of a reason why such a conversion cannot be done.
Last edited by Bonnie West; May 28th, 2015 at 11:24 PM.
On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Bonnie West
I haven't taken a close look at your NewSocket.cls yet, but I believe it should be possible to convert your existing subclassing logic to one that uses SetWindowSubclass instead. I can't think of a reason why such a conversion cannot be done.
Because of it's complexity, I would like to leave the NewSocket.cls as much intact as possible. I created a new companion Module with the Subclass code within, and I can create and utilize a message handler (WndProc) within the module without a problem. But how do utilize a message handler from within the Class?
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by couttsj
But how do utilize a message handler from within the Class?
The new subclassing APIs adds a couple more parameters to the subclass procedure's signature:
Code:
Public Function StaticSubclassProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, _
ByVal uIdSubclass As ucSetWindowSubclass, ByVal dwRefData As Long) As Long
You can pass a pointer to the Class to either of them:
In the subclass procedure, you can alter the corresponding parameter's type so that VB automatically casts the passed Class pointer as a new reference to the Class:
Code:
Public Function StaticSubclassProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, _
ByVal uIdSubclass As ucSetWindowSubclass, ByVal dwRefData As Long) As Long
In the Class module, the real message handler procedure is given the scope Friend (because Friend procedures are faster than Public procedures):
Code:
Friend Function SubclassProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, ByVal dwRefData As Long) As Long
The static subclass procedure can now be redirected to the real message handler procedure of the appropriate Class instance:
I believe this method of redirecting subclassing to the desired Object module is quite similar to how the various ASM subclassing thunks out there generally works. Although it offers absolutely no IDE protection, it is however very simple to understand and very lightweight.
On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Bonnie West
[LIST]
I believe this method of redirecting subclassing to the desired Object module is quite similar to how the various ASM subclassing thunks out there generally works. Although it offers absolutely no IDE protection, it is however very simple to understand and very lightweight.
Let me see if I have this correct. In the companion Module, I define a new function called StaticSubclassProc with a couple of extra parameters (uIdSubclass & dwRefData). The function itself returns a reference to the Message Handling routine in the Class Module, and when AddressOf is applied to the result, it Subclasses the message handling routine in the Class Module.
When I do that, VB tells me that the data type "ucSetWindowSubclass" is not defined. Do I have to add something extra to the Class Module?
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by couttsj
In the companion Module, I define a new function called StaticSubclassProc with a couple of extra parameters (uIdSubclass & dwRefData).
... in addition, of course, to the 4 standard window procedure parameters (hWnd, uMsg, wParam & lParam). The "Static" prefix refers to the fact that that procedure resides in a static module (a.k.a. standard module).
Originally Posted by couttsj
When I do that, VB tells me that the data type "ucSetWindowSubclass" is not defined. Do I have to add something extra to the Class Module?
Sorry, I forgot to mention that those examples were from the test project in post #8. ucSetWindowSubclass is the name of a UserControl in that project. Replace that data type with the name of your actual Class module that will be handling the messages.
Basically, the idea here is that you'll be passing a pointer to your Class instance (ObjPtr(Me)) and VB will automatically treat that pointer as a new reference to the Class instance. This trick is essentially similar to the ObjectFromPointer routines you may already have seen.
Originally Posted by couttsj
The function itself returns a reference to the Message Handling routine in the Class Module, and when AddressOf is applied to the result, it Subclasses the message handling routine in the Class Module.
Um, no. The StaticSubclassProc function just returns the return value of a particular Windows message. The AddressOf operator, as its name implies, returns the starting location in memory of the specified procedure, which must reside in a static module.
From the point of view of the new subclassing APIs, the StaticSubclassProc function is the real subclass procedure, i.e., it has the required function signature. What we're trying to do here, however, is that whenever that callback function gets called by the OS, we invoke the method of a Class where we perform the actual message handling. When that method (which has nearly the same parameters as the real subclass procedure that the APIs expect) exits, it passes the return value of the current message to the real subclass procedure, which in turn passes it along to the rest of the subclass chain.
This redirection is necessary because it isn't possible for a method of a Class to become a callback procedure that can be directly called by the APIs. The reason is because all methods of a Class has a hidden first parameter that is a pointer to the Class instance and a similarly hidden HRESULT return value (which returns an error value, if any). Additionally, functions also have one more hidden "out" parameter where its actual declared return value is returned. So, as you can see, the hidden parameters make it impossible to construct the proper callback signature, hence the need for redirection or ASM thunks.
On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Bonnie West
This redirection is necessary because it isn't possible for a method of a Class to become a callback procedure that can be directly called by the APIs. The reason is because all methods of a Class has a hidden first parameter that is a pointer to the Class instance and a similarly hidden HRESULT return value (which returns an error value, if any). Additionally, functions also have one more hidden "out" parameter where its actual declared return value is returned. So, as you can see, the hidden parameters make it impossible to construct the proper callback signature, hence the need for redirection or ASM thunks.
Thanks Bonnie. Good information, especially the part above. I don't know how Emiliano Scavuzzo did it, but he doesn't appear to use redirection. I do know that he requires the Message Handler routine to be the very first one in the class. I will investigate further.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by couttsj
I do know that he requires the Message Handler routine to be the very first one in the class.
If that message handler routine is also required to be Public, then that's a strong indicator that he's likely using ASM thunking. You'll probably find that the ASM machine code he's using is stored as a string of hex characters.
On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Bonnie West
If that message handler routine is also required to be Public, then that's a strong indicator that he's likely using ASM thunking. You'll probably find that the ASM machine code he's using is stored as a string of hex characters.
Oh it is definitely Assembler code, and it has been a long time since I have done any Assembler coding. I have the actual source code, which I can send you if you would like.
In addition to several patches to protect the IDE, what appears to be occurring is that upon the first Socket connection, several more patches are made to the code. Some of it provides index pointers to socket ownership tables (one for address resolution, and one for socket messages), and the Message Handling location is given as a 28 byte offset to the socket owner. That sort of explains why the Message Handling routine has to be the very first one in the class.
I made many changes to the code to provide support for IPv6, but the original subclassing remained relatively intact(except for using "ws2_32.dll" instead of "ws_32.dll"). So it seems that using SetWindowSubclass is out of the question at this point, unless I want to do a lot of extra work. Thanks again for the feedback.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by fafalone
With something that incredibly complex to begin with I don't know that there's any value in changing the specific subclassing method anyway.
In hindsight you are absolutely correct, but the effort was not without a small reward. I was able to replace this inefficient code:
Code:
nLen = Len(sHex)
'Convert the string from hex pairs to bytes and store in the ASCII string opcode buffer
For i = 1 To nLen Step 2
sCode = sCode & ChrB$(Val("&H" & Mid$(sHex, i, 2)))
Next i
nLen = LenB(sCode)
nAddrSubclass = GlobalAlloc(0, nLen) 'Allocate fixed memory for machine code buffer
Call CopyMemory(ByVal nAddrSubclass, ByVal StrPtr(sCode), nLen)
with:
Code:
nLen = Len(sHex) / 2
ReDim bTmp(nLen - 1)
For i = 0 To UBound(bTmp)
bTmp(i) = Val("&H" & Mid$(sHex, i * 2 + 1, 2))
Next i
nAddrSubclass = GlobalAlloc(0, nLen) 'Allocate fixed memory for machine code buffer
Call CopyMemory(ByVal nAddrSubclass, ByVal VarPtr(bTmp(0)), nLen)
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
On the subject of not crashing the IDE:
Just to add more information, If you look at the assembly stubs created. the majority of them take action based on vba6.dll#EbMode()
If it's in Run Mode, it calls the subclassed handler. If it's in break mode, it passes the call to the previous WndProc, and if the IDE is stopped it unsubclasses the window, and then forwards to the previous WndProc.
Some of the subclassing tehniques actually go a step farther and hook the actual the IDE Stop/Resettng function, and unsbuclasses everything before passing control back to the IDE.
If you query EbMode, and move the subclassing to a DLL, you've got 99% IDE protection. I think there are still some cases where the IDE crashes when you hit END.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
fafalone, thanks for this excellent write-up. Very helpful. Like others, I've been (blindly) using Paul Caton's methods for years but have been meaning to switch various projects to this CCL technique.
I'm currently working on some code related to subclassing WM_SETTINGCHANGE, but before moving to SetWindowSubclass, I wanted to ask about this comment from the MSDN article on window subclassing:
Note: All strings passed to the [subclass] procedure are Unicode strings even if Unicode is not specified as a preprocessor definition.
I didn't notice any mention of this in your write-up, but do you know - is MSDN accurate? If I subclass something like an ANSI VB window, and am looking at messages where the w/lParams are string pointers, will those pointers always be to DBCS strings?
In my case, at least, this has some implications for retrieving those strings (e.g. lstrlenA vs lstrlenW, etc), so I'm curious if others have firsthand experience. If MSDN does end up being accurate, maybe worth mentioning that caveat in your article, "just in case"...?
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Tanner_H
I didn't notice any mention of this in your write-up, but do you know - is MSDN accurate? If I subclass something like an ANSI VB window, and am looking at messages where the w/lParams are string pointers, will those pointers always be to DBCS strings?
Not DBCS (which was a form of Asian-language extension for ANSI encoding that mixes one and two-byte characters, designed for early pre-Unicode Win9x) but the UCS-2/UTF-16LE that Windows refers to as "Unicode."
A double-byte character set (DBCS), also known as an "expanded 8-bit character set", is an extended single-byte character set (SBCS), implemented as a code page. DBCSs were originally developed to extend the SBCS design to handle languages such as Japanese and Chinese. Some characters in a DBCS, including the digits and letters used for writing English, have single-byte code values. Other characters, such as Chinese ideographs or Japanese kanji, have double-byte code values. A DBCS can correspond either to a Windows code page or an OEM code page. A DBCS code page can also include a non-native code page, for example, an EBCDIC code page. For definitions of these code pages, see Code Pages.
Things like the ListView that COMCTL32.OCX wraps can be shifted in and out of ANSI mode, so depending on the mode you might get/put one encoding or the other.
So I think the bad news is that "it depends."
Last edited by dilettante; Sep 19th, 2015 at 01:15 AM.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
More detail:
For a ListView WM_NOTIFY, the lParam points to a instance of the NMHDR struct. If its .code = LVN_GETDISPINFO, you need to write an LV_DISPINFO struct at the lParam pointer and when (.mask And LVIF_TEXT) is non-0 the requested text you supply is normally ANSI.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Tanner_H
If I subclass something like an ANSI VB window, and am looking at messages where the w/lParams are string pointers, will those pointers always be to DBCS strings?
Through testing, I've found out that string parameters passed to subclass procedures will be ANSI or Unicode depending on whether the subclass procedure is being called by an ANSI or Unicode subclass. The system apparently performs the necessary conversion between ANSI and Unicode when it calls an ANSI or Unicode subclass procedure (i.e., the system converts ANSI strings to Unicode when calling a Unicode subclass procedure and vice versa). Thus, there is no need to be concerned about receiving ANSI string parameters when the subclass is set by a Unicode API function.
Originally Posted by Tanner_H
... is MSDN accurate?
I have observed via API Monitor that the SetWindowSubclass API actually calls the SetWindowLongW function internally. Subclasses set using it will therefore always receive Unicode strings.
On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
@dilettante, Bonnie - thanks much. Appreciate your expertise. Dilettante in particular, you'd think I could keep the various Unicode designations straight after all these years. Thanks again.
Originally Posted by Bonnie West
I have observed via API Monitor that the SetWindowSubclass API actually calls the SetWindowLongW function internally. Subclasses set using it will therefore always receive Unicode strings
Perfect, thank you, Bonnie. That makes a lot of sense, given the other warning on the MSDN subclassing page:
Note: ComCtl32.dll version 6 is Unicode only. The common controls supported by ComCtl32.dll version 6 should not be subclassed (or superclassed) with ANSI window procedures.
This is a "5.0" ListView (as in COMCTL32.OCX) being used with version 6.0 ComCtl32.dll and sure enough it operates in ANSI mode by default even so. Whether the VB-supplied OCX put it into that mode for normal use or not I don't know, but it definately operates in ANSI and you'd have to send and handle WM_NOTIFYFORMAT notifications (these go both ways) and reply that you want to use Unicode. This impacts the encoding of WM_NOTIFY message strings.
That might be one of a very few oddball cases. But when in doubt research may be necessary.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Even with a directly created ListView (CreateWindowEx) I had to use WM_NOTIFYFORMAT to get all Unicode messages and strings. The issue came up during label editing; I wasn't getting Unicode support somewhere I can't remember (probably the NMLVDISPINFO item text), which I figured was because there was both LVN_[BEGIN|END]LABELEDIT A and W, and I was only getting A. So I replied with NFR_UNICODE, started receiving the LVN_xLABELEDITW messages, and got full unicode support for label editing.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
There is one more caveat with SetWindowSubclass when using ASM thunks. It appears it keeps an internal table, keyed on (wnd class, wnd proc address) pair that is a form of cache which does not get cleared upon RemoveWindowSubclass.
Normally when you subclass TextBoxes or Forms with direct AddressOf your callback function this table contains 10s of entries at most. But when ASM thunks are being allocated from heap the second member of the pair changes with each Subclass call, so you get lot's of entries in the internal table over the lifetime of your process that never get cleared.
You can use Bear to monitor subclassing internal table size.
The nasty thing is that even if you use old SetWindowLong(GWL_WNDPROC) approash but use ASM thunks for the wndproc (Matt Curland's PushParamThunk for instance for get an extra parameter with `This` pointer on the callback) you risk some 3rd party or 1st party control to try to subclass with SetWindowSubclass and leak. Case in point -- standard ComCtl tooltips do it on Vista and newer for tree/list views.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
NOTE: I just discovered that specifying a value for the uIdSubclass parameter when subclassing and unsubclassing is required in some (all?) circumstances if you're going to remove the subclass during runtime.
I had a VB ListView subclassed and couldn't change the imagelist assignment via API, because I found it to still be calling WndProc code that blocked LVM_SETIMAGELIST, even after I had called UnSubclass. I set uIdSubclass to the hWnd and the issue was resolved.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
interesting. I had no idea you could subclass without specifying uIdSubclass - as it's the combination of the SubClassProc and uIdSubclass that define a unique subclassing instance. Were you just setting uIdSubclass to zero? interesting that it breaks when using 0 as the Id.
Also maybe its just a misunderstanding, but what scenario wouldn't you unsubclass during runtime? or when you say runtime, do you just mean during the life of the window not counting shutdown. Standard operation is still to unsubclass before window destruction.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
I had always been using 0 for uIdSubclass; I do *a lot* of subclassing and never had a problem, even with things like subclassing different hWnds to the same WndProc, until I needed to remove a subclass during runtime- something, as you suggest, is a very uncommon issue.
In this particular scenario, VB doesn't recognize an API HIMAGELIST assigned to the OCX-based native comctl ListView and problems can arise, so once you set an API imagelist, you subclass and prevent VB from clearing your imagelist assignment. The unsubclassing during runtime came up with the Thumbnail ListView I just posted, where it needs to switch from being assigned the system imagelist to the custom imagelist with the thumbnails when the user switches to thumbnail mode-- and this is the first project I've made that uses the OCX ListView in addition to the SetWindowSubclass-based subclassing (almost always any ListView I make that does anything beyond basics is made with CreateWindowEx, but I didn't want to further complicate the already hard to follow demo). In retrospect I suppose I could have just set a flag as there's nothing else to reset in the stripped down demo version, but why mess with what works.
Last edited by fafalone; Aug 26th, 2016 at 04:38 PM.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Is it possible to use SetWindowSubclass in excel project to safely subclass any build-in control?
By "safely" I mean that when I hit "stop" button Excel will not crash immediately.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Im a little confused... Wich one is the better subclassing to use? SetWindowSubclass or using thunking in using classes with assembler patches that check eBmode?
Every project I see seems to use a diferent way to subclass so its really confusing for new people to understand which is the better way to do it
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
What's better depends on your circumstance. I use both this method and the thunk-based ones depending on if I just need a simple subclass on a normal window, or need more advanced functionality that justifies taking the time to implement a more complex solution.
If you're really new to it and not worrying about the harder problems like class instances and usercontrols, whatever is the simplest method to get the messages you need to process should come first.
Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho
Originally Posted by Bonnie West
Try the attached test projects below to see what I mean by "safer".
I believe this method of redirecting subclassing to the desired Object module is quite similar to how the various ASM subclassing thunks out there generally works. Although it offers absolutely no IDE protection, it is however very simple to understand and very lightweight.
The original version would crash, I made some modifications and tried it with the new version
comctl32_SetWindowSubclass.zip
Code:
Public Function StaticSubclassProc(ByVal Hwnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, ByVal uIdSubclass As ucSetWindowSubclass, ByVal dwRefData As Long) As Long
On Error GoTo ERR
StaticSubclassProc = uIdSubclass.SubclassProc(Hwnd, uMsg, wParam, lParam, dwRefData)
Exit Function
ERR:
Debug.Print "StaticSubclassProc err:" & ERR.Description & ",obj=" & ObjPtr(uIdSubclass)
StaticSubclassProc = DefSubclassProc(Hwnd, uMsg, wParam, lParam)
Call RemoveWindowSubclass(Hwnd, GetCallBackAddress, ObjPtr(uIdSubclass))
End Function
about:
It has been fixed, the IDE protection is completely fine, and the end button will not crash.
Last edited by xiaoyao; Jun 17th, 2023 at 06:20 PM.
Why would I want to try *non*-crash-free subclasser when there is already a crash-free IDE-safe self-contained one? MST has a mouse tracking sample too. . .