dcsimg
Results 1 to 36 of 36

Thread: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass method

  1. #1

    Thread Starter
    PowerPoster
    Join Date
    Jul 2010
    Location
    NYC
    Posts
    2,344

    [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass method

    So there's quite a few posts about specific questions where code like this is shown, but I really thought it would be better to have a subclassing tutorial type codebank entry for it to make it more easily searchable, and a better answer for when somebody hasn't used subclassing before.

    Since Windows XP, there's been some great subclassing setups posted showing how to use SetWindowSubclass as a newer, better subclassing method than previous SetWindowLong-based methods. Karl E. Peterson's HookXP and LaVolpe's Subclassing With Common Controls Library being two top notch examples, and this method was first shown to me by Bonnie West over in my ITaskBarList3 demo. But before delving into using interfaces etc, (and even still for subclassing things other than forms and the like), it's helpful to show how to do the very simplest of subclassing: When you have any object, and just want to have a code to handle a message without much fuss.

    The subclassing example I picked for this is validating whether text typed into a textbox is a valid filename, and blocking the input altogether if the keystroke or paste operation contains an illegal character or is too long. (You could do this without subclassing in most scenarios, but I thought it was a nice and simple way to get the idea across.)

    All it requires on the form is a single textbox.

    Once you have that, you'll need to create the function that handles its messages in a module. All such functions, usually referred to as the WndProc, have the same arguments. Also, they all must unsubclass when the window is being destroyed (or before) otherwise the program will crash. So before adding any code for handling messages, the basic prototype looks like this:

    Code:
    Public Function EditWndProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, ByVal uIdSubclass As Long, ByVal dwRefData As Long) As Long
    
    Select Case uMsg
    
        '[other messages will go here later]
    
       Case WM_DESTROY
         Call UnSubclass(hWnd, PtrEditWndProc)
    
    End Select
       
    EditWndProc = DefSubclassProc(hWnd, uMsg, wParam, lParam)
    
    End Function
    Private Function PtrEditWndProc() As Long
    PtrEditWndProc = FARPROC(AddressOf EditWndProc)
    End Function
    The DefSubclassProc call passes everything back to the default handler if you're not completely handling it. The PtrEditWndProc is there because you can't use the AddressOf operator to get the address of the function it's in.

    Now that you have a function to handle the messages, you can add the code to start the subclass back in the form:

    Code:
    Private Sub Form_Load()
    Call Subclass(Text1.hWnd, AddressOf EditWndProc)
    End Sub
    And now you have a basic subclass all set up and ready to go. You don't need an Unsubclass in Form_Unload.
    Here's the message handlers used to validate input for a textbox looking for a valid file name (and appears right after Select Case uMsg:

    Code:
       Case WM_CHAR
         Dim lLen As Long
         lLen = SendMessageW(hWnd, WM_GETTEXTLENGTH, 0, ByVal 0&) + 1
         If lLen > 260 Then
                 Beep
                 wParam = 0
                 Call ShowBalloonTipEx(hWnd, "", "Maximum number of characters has been reached. The total length of the file name cannot exceed 260 characters.", TTI_NONE) ' TTI_ERROR)
                 Exit Function
         End If
         Select Case wParam
             Case 47, 92, 60, 62, 58, 42, 124, 63, 34 'Illegal chars /\<>:*|?"
                 Beep
                 Call ShowBalloonTipEx(hWnd, "", "File names may not contain any of the following characters:" & vbCrLf & " / \ < > : ? * | " & Chr$(34), TTI_NONE) ' TTI_ERROR)
                 wParam = 0
         End Select
         
         Case WM_PASTE
             Dim iCheck As Integer
             iCheck = IsClipboardValidFileName()
             If iCheck = 0 Then
                 Beep
                 Call ShowBalloonTipEx(hWnd, "", "File names may not contain any of the following characters:" & vbCrLf & " / \ < > : ? * | " & Chr$(34), TTI_NONE) ' TTI_ERROR)
                 Exit Function
             ElseIf iCheck = -1 Then
                 Beep
                 Call ShowBalloonTipEx(hWnd, "", "The file name you have entered is too long. The total length of the file name cannot exceed 260 characters.", TTI_NONE) ' TTI_ERROR)
                 Exit Function
    
             End If
    Here's what the full project looks like with all the supporting codes and declares added in:

    Form1
    Code:
    Option Explicit
    
    Private Sub Form_Load()
    Call Subclass(Text1.hWnd, AddressOf EditWndProc)
    End Sub
    mSubclass
    Code:
    Option Explicit
    
    Private Declare Function DefSubclassProc Lib "comctl32.dll" Alias "#413" (ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long) As Long
    Private Declare Function SetWindowSubclass Lib "comctl32.dll" Alias "#410" (ByVal hWnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long, Optional ByVal dwRefData As Long) As Long
    Private Declare Function RemoveWindowSubclass Lib "comctl32.dll" Alias "#412" (ByVal hWnd As Long, ByVal pfnSubclass As Long, ByVal uIdSubclass As Long) As Long
    Private Declare Function SendMessageW Lib "user32" (ByVal hWnd As Long, ByVal wMsg As Long, ByVal wParam As Long, lParam As Any) As Long
    
    Private Type EDITBALLOONTIP
        cbStruct As Long
        pszTitle As Long
        pszText As Long
        ttiIcon As BalloonTipIconConstants ' ; // From TTI_*
    End Type
    Private Enum BalloonTipIconConstants
       TTI_NONE = 0
       TTI_INFO = 1
       TTI_WARNING = 2
       TTI_ERROR = 3
    End Enum
    
    Private Const WM_CHAR = &H102
    Private Const WM_PASTE = &H302
    Private Const WM_DESTROY = &H2
    Private Const WM_GETTEXTLENGTH = &HE
    
    Private Const ECM_FIRST As Long = &H1500
    Private Const EM_SHOWBALLOONTIP As Long = (ECM_FIRST + 3)
    
    Public Function Subclass(hWnd As Long, lpfn As Long) As Long
    Subclass = SetWindowSubclass(hWnd, lpfn, 0)
    End Function
    Public Function UnSubclass(hWnd As Long, lpfn As Long) As Long
    'Only needed if you want to stop the subclassing code and keep the program running.
    'Otherwise, the WndProc function should call this on WM_DESTROY
    UnSubclass = RemoveWindowSubclass(hWnd, lpfn, 0)
    End Function
    
    Public Function EditWndProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, ByVal uIdSubclass As Long, ByVal dwRefData As Long) As Long
    Select Case uMsg
    
       Case WM_CHAR
         Dim lLen As Long
         lLen = SendMessageW(hWnd, WM_GETTEXTLENGTH, 0, ByVal 0&) + 1
         If lLen > 260 Then
                 Beep
                 wParam = 0
                 Call ShowBalloonTipEx(hWnd, "", "Maximum number of characters has been reached. The total length of the file name cannot exceed 260 characters.", TTI_NONE) ' TTI_ERROR)
                 Exit Function
         End If
         Select Case wParam
             Case 47, 92, 60, 62, 58, 42, 124, 63, 34 'Illegal chars /\<>:*|?"
                 Beep
                 Call ShowBalloonTipEx(hWnd, "", "File names may not contain any of the following characters:" & vbCrLf & " / \ < > : ? * | " & Chr$(34), TTI_NONE) ' TTI_ERROR)
                 wParam = 0
         End Select
         
         Case WM_PASTE
             Dim iCheck As Integer
             iCheck = IsClipboardValidFileName()
             If iCheck = 0 Then
                 Beep
                 Call ShowBalloonTipEx(hWnd, "", "File names may not contain any of the following characters:" & vbCrLf & " / \ < > : ? * | " & Chr$(34), TTI_NONE) ' TTI_ERROR)
                 Exit Function
             ElseIf iCheck = -1 Then
                 Beep
                 Call ShowBalloonTipEx(hWnd, "", "The file name you have entered is too long. The total length of the file name cannot exceed 260 characters.", TTI_NONE) ' TTI_ERROR)
                 Exit Function
    
             End If
             
       Case WM_DESTROY
         Call UnSubclass(hWnd, PtrEditWndProc)
    
    End Select
       
    EditWndProc = DefSubclassProc(hWnd, uMsg, wParam, lParam)
    
    End Function
    Private Function PtrEditWndProc() As Long
    PtrEditWndProc = FARPROC(AddressOf EditWndProc)
    End Function
    
    Private Function FARPROC(pfn As Long) As Long
      FARPROC = pfn
    End Function
    
    Private Sub ShowBalloonTipEx(hWnd As Long, sTitle As String, sText As String, btIcon As BalloonTipIconConstants)
    Dim lR As Long
    Dim tEBT As EDITBALLOONTIP
    tEBT.cbStruct = LenB(tEBT)
    tEBT.pszText = StrPtr(sText)
    tEBT.pszTitle = StrPtr(sTitle)
    tEBT.ttiIcon = btIcon
    lR = SendMessageW(hWnd, EM_SHOWBALLOONTIP, 0, tEBT)
    End Sub
    Public Function IsClipboardValidFileName() As Integer
    Dim i As Long
    Dim sz As String
    Dim sChr As String
    'there's a couple scenarios for invalid file names i've trimmed out
    'to keep this example as simple as possible, look into them if you're
    'going to use this code in an actual rename procedure
    sz = Clipboard.GetText
    
    If Len(sz) > 260 Then
        IsClipboardValidFileName = -1
        Exit Function
    End If
    IsClipboardValidFileName = 1
    
    If InStr(sz, "*") Or InStr(sz, "?") Or InStr(sz, "<") Or InStr(sz, ">") Or InStr(sz, "|") Or InStr(sz, Chr$(34)) Then
        IsClipboardValidFileName = 0
    End If
    End Function
    It's still a little complicated, but this is the very simplest way to get subclassing going.
    Attached Files Attached Files

  2. #2
    PowerPoster
    Join Date
    Feb 2006
    Posts
    20,662

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    This is a good technique we've had available since IE 4.x came out. It wasn't documented and the entrypoints were not exported by name until comctl32.dll version 6.0 in Windows XP came out, so many people seem to be unaware of it even today.

    Even Karl E. Peterson blamed this on the blinders worn by so many in the VB community, which tends to have a lot of folks who reject change and turned their backs on Windows XP for many years... until it became their new Mother and then Vista became the new whipping boy.

    But by accessing these by ordinal as shown above they work as far back as Windows 95 as long as the IE 4.x Desktop Update is installed.
    Last edited by dilettante; May 11th, 2015 at 10:56 AM.

  3. #3
    Frenzied Member
    Join Date
    Feb 2015
    Posts
    1,362

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Good article! I will say a couple of my words.
    It's just a call SetProp (GlobalAddAtom ("CC32SubclassInfo"), NewWndProc) with additional checks. Also, addition in the special table of parameter information. When processing each message is searched in the table and additional checks that slows down the program. So why additional load on the CPU? SetWindowSubclass - an add-on SetWindowLong. If you chase performance, it is best to discard any wrappers on standard functions.
    If you need modularity, it would be possible to make subclassing in a class module. Then you can pass any number of parameters.

    Thanks again for the article!
    Анатолий.

  4. #4
    Default Member Bonnie West's Avatar
    Join Date
    Jun 2012
    Location
    InIDE
    Posts
    4,057

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by The trick View Post
    If you chase performance, it is best to discard any wrappers on standard functions.
    Raymond Chen gave a very good reason why one should prefer the newer subclassing APIs: they're safer.

    Quote Originally Posted by The trick View Post
    If you need modularity, it would be possible to make subclassing in a class module.
    As demonstrated here, SetWindowSubclass makes it easier (and neater) to redirect a subclass procedure to the desired Object module.
    On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
    Declare Sub CrashVB Lib "msvbvm60" (Optional DontPassMe As Any)

  5. #5
    Frenzied Member
    Join Date
    Feb 2015
    Posts
    1,362

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by Bonnie West View Post
    Raymond Chen gave a very good reason why one should prefer the newer subclassing APIs: they're safer.
    I did not see the advantages. With the same success I can instead cause DefSubclassProc - CallWindowProc.
    Quote Originally Posted by Bonnie West View Post
    As demonstrated here, SetWindowSubclass makes it easier (and neater) to redirect a subclass procedure to the desired Object module.
    To transfer user information there are many ways. SetProp, GWL_USERDATA, etc. Just SetWindowSubclass uses SetProp to save the previous window procedure. I also developed a class for safe subclassing using only one class.

  6. #6

    Thread Starter
    PowerPoster
    Join Date
    Jul 2010
    Location
    NYC
    Posts
    2,344

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    All that's great, but the idea here is to introduce people to writing subclassing code in the simplest way possible. I'm a big fan of approaching things with complex methods when simpler ones exist (see: half my code bank submissions), but went the opposite way here.

    Although something I'm curious about, have you found any way to subclass that doesn't crash the IDE with most errors? (besides an external dll)

  7. #7

  8. #8
    Default Member Bonnie West's Avatar
    Join Date
    Jun 2012
    Location
    InIDE
    Posts
    4,057

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by The trick View Post
    I did not see the advantages.
    Try the attached test projects below to see what I mean by "safer".
    Attached Files Attached Files
    On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
    Declare Sub CrashVB Lib "msvbvm60" (Optional DontPassMe As Any)

  9. #9

  10. #10
    Fanatic Member
    Join Date
    Dec 2012
    Posts
    777

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by Bonnie West View Post
    Try the attached test projects below to see what I mean by "safer".
    Bonnie
    I have been using Paul Caton,s SubClassing technique with a formless window to receive system messages. That technique involves injecting assembler code with debug code to avoid problems in the IDE, but SetWindowSubclass seems a lot simpler. Would that work for such things as my NewSocket.cls? I thought I would seek your opinion before I venture forth.

    J.A. Coutts

  11. #11
    Default Member Bonnie West's Avatar
    Join Date
    Jun 2012
    Location
    InIDE
    Posts
    4,057

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by couttsj View Post
    ... but SetWindowSubclass seems a lot simpler. Would that work for such things as my NewSocket.cls? I thought I would seek your opinion before I venture forth.
    I haven't taken a close look at your NewSocket.cls yet, but I believe it should be possible to convert your existing subclassing logic to one that uses SetWindowSubclass instead. I can't think of a reason why such a conversion cannot be done.
    Last edited by Bonnie West; May 28th, 2015 at 11:24 PM.
    On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
    Declare Sub CrashVB Lib "msvbvm60" (Optional DontPassMe As Any)

  12. #12
    Fanatic Member
    Join Date
    Dec 2012
    Posts
    777

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by Bonnie West View Post
    I haven't taken a close look at your NewSocket.cls yet, but I believe it should be possible to convert your existing subclassing logic to one that uses SetWindowSubclass instead. I can't think of a reason why such a conversion cannot be done.
    Because of it's complexity, I would like to leave the NewSocket.cls as much intact as possible. I created a new companion Module with the Subclass code within, and I can create and utilize a message handler (WndProc) within the module without a problem. But how do utilize a message handler from within the Class?

    J.A. Coutts

  13. #13
    Default Member Bonnie West's Avatar
    Join Date
    Jun 2012
    Location
    InIDE
    Posts
    4,057

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by couttsj View Post
    But how do utilize a message handler from within the Class?
    • The new subclassing APIs adds a couple more parameters to the subclass procedure's signature:
      Code:
      Public Function StaticSubclassProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, _
                                         ByVal uIdSubclass As ucSetWindowSubclass, ByVal dwRefData As Long) As Long
    • You can pass a pointer to the Class to either of them:
      Code:
      m_Subclassed = SetWindowSubclass(m_hWndParent, AddressOf StaticSubclassProc, ObjPtr(Me), 0&)
    • In the subclass procedure, you can alter the corresponding parameter's type so that VB automatically casts the passed Class pointer as a new reference to the Class:
      Code:
      Public Function StaticSubclassProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, _
                                         ByVal uIdSubclass As ucSetWindowSubclass, ByVal dwRefData As Long) As Long
    • In the Class module, the real message handler procedure is given the scope Friend (because Friend procedures are faster than Public procedures):
      Code:
      Friend Function SubclassProc(ByVal hWnd As Long, ByVal uMsg As Long, ByVal wParam As Long, ByVal lParam As Long, ByVal dwRefData As Long) As Long
    • The static subclass procedure can now be redirected to the real message handler procedure of the appropriate Class instance:
      Code:
      StaticSubclassProc = uIdSubclass.SubclassProc(hWnd, uMsg, wParam, lParam, dwRefData)


    I believe this method of redirecting subclassing to the desired Object module is quite similar to how the various ASM subclassing thunks out there generally works. Although it offers absolutely no IDE protection, it is however very simple to understand and very lightweight.
    On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
    Declare Sub CrashVB Lib "msvbvm60" (Optional DontPassMe As Any)

  14. #14
    Fanatic Member
    Join Date
    Dec 2012
    Posts
    777

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by Bonnie West View Post
    [LIST]
    I believe this method of redirecting subclassing to the desired Object module is quite similar to how the various ASM subclassing thunks out there generally works. Although it offers absolutely no IDE protection, it is however very simple to understand and very lightweight.
    Let me see if I have this correct. In the companion Module, I define a new function called StaticSubclassProc with a couple of extra parameters (uIdSubclass & dwRefData). The function itself returns a reference to the Message Handling routine in the Class Module, and when AddressOf is applied to the result, it Subclasses the message handling routine in the Class Module.

    When I do that, VB tells me that the data type "ucSetWindowSubclass" is not defined. Do I have to add something extra to the Class Module?

    J.A. Coutts

  15. #15
    Default Member Bonnie West's Avatar
    Join Date
    Jun 2012
    Location
    InIDE
    Posts
    4,057

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by couttsj View Post
    In the companion Module, I define a new function called StaticSubclassProc with a couple of extra parameters (uIdSubclass & dwRefData).
    ... in addition, of course, to the 4 standard window procedure parameters (hWnd, uMsg, wParam & lParam). The "Static" prefix refers to the fact that that procedure resides in a static module (a.k.a. standard module).

    Quote Originally Posted by couttsj View Post
    When I do that, VB tells me that the data type "ucSetWindowSubclass" is not defined. Do I have to add something extra to the Class Module?
    Sorry, I forgot to mention that those examples were from the test project in post #8. ucSetWindowSubclass is the name of a UserControl in that project. Replace that data type with the name of your actual Class module that will be handling the messages.

    Basically, the idea here is that you'll be passing a pointer to your Class instance (ObjPtr(Me)) and VB will automatically treat that pointer as a new reference to the Class instance. This trick is essentially similar to the ObjectFromPointer routines you may already have seen.

    Quote Originally Posted by couttsj View Post
    The function itself returns a reference to the Message Handling routine in the Class Module, and when AddressOf is applied to the result, it Subclasses the message handling routine in the Class Module.
    Um, no. The StaticSubclassProc function just returns the return value of a particular Windows message. The AddressOf operator, as its name implies, returns the starting location in memory of the specified procedure, which must reside in a static module.

    From the point of view of the new subclassing APIs, the StaticSubclassProc function is the real subclass procedure, i.e., it has the required function signature. What we're trying to do here, however, is that whenever that callback function gets called by the OS, we invoke the method of a Class where we perform the actual message handling. When that method (which has nearly the same parameters as the real subclass procedure that the APIs expect) exits, it passes the return value of the current message to the real subclass procedure, which in turn passes it along to the rest of the subclass chain.

    This redirection is necessary because it isn't possible for a method of a Class to become a callback procedure that can be directly called by the APIs. The reason is because all methods of a Class has a hidden first parameter that is a pointer to the Class instance and a similarly hidden HRESULT return value (which returns an error value, if any). Additionally, functions also have one more hidden "out" parameter where its actual declared return value is returned. So, as you can see, the hidden parameters make it impossible to construct the proper callback signature, hence the need for redirection or ASM thunks.
    On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
    Declare Sub CrashVB Lib "msvbvm60" (Optional DontPassMe As Any)

  16. #16
    Fanatic Member
    Join Date
    Dec 2012
    Posts
    777

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by Bonnie West View Post
    This redirection is necessary because it isn't possible for a method of a Class to become a callback procedure that can be directly called by the APIs. The reason is because all methods of a Class has a hidden first parameter that is a pointer to the Class instance and a similarly hidden HRESULT return value (which returns an error value, if any). Additionally, functions also have one more hidden "out" parameter where its actual declared return value is returned. So, as you can see, the hidden parameters make it impossible to construct the proper callback signature, hence the need for redirection or ASM thunks.
    Thanks Bonnie. Good information, especially the part above. I don't know how Emiliano Scavuzzo did it, but he doesn't appear to use redirection. I do know that he requires the Message Handler routine to be the very first one in the class. I will investigate further.

    J.A. Coutts

  17. #17
    Default Member Bonnie West's Avatar
    Join Date
    Jun 2012
    Location
    InIDE
    Posts
    4,057

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by couttsj View Post
    I do know that he requires the Message Handler routine to be the very first one in the class.
    If that message handler routine is also required to be Public, then that's a strong indicator that he's likely using ASM thunking. You'll probably find that the ASM machine code he's using is stored as a string of hex characters.
    On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
    Declare Sub CrashVB Lib "msvbvm60" (Optional DontPassMe As Any)

  18. #18
    Fanatic Member
    Join Date
    Dec 2012
    Posts
    777

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by Bonnie West View Post
    If that message handler routine is also required to be Public, then that's a strong indicator that he's likely using ASM thunking. You'll probably find that the ASM machine code he's using is stored as a string of hex characters.
    Oh it is definitely Assembler code, and it has been a long time since I have done any Assembler coding. I have the actual source code, which I can send you if you would like.

    In addition to several patches to protect the IDE, what appears to be occurring is that upon the first Socket connection, several more patches are made to the code. Some of it provides index pointers to socket ownership tables (one for address resolution, and one for socket messages), and the Message Handling location is given as a 28 byte offset to the socket owner. That sort of explains why the Message Handling routine has to be the very first one in the class.

    I made many changes to the code to provide support for IPv6, but the original subclassing remained relatively intact(except for using "ws2_32.dll" instead of "ws_32.dll"). So it seems that using SetWindowSubclass is out of the question at this point, unless I want to do a lot of extra work. Thanks again for the feedback.

    J.A. Coutts

  19. #19

  20. #20
    Fanatic Member
    Join Date
    Dec 2012
    Posts
    777

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by fafalone View Post
    With something that incredibly complex to begin with I don't know that there's any value in changing the specific subclassing method anyway.
    In hindsight you are absolutely correct, but the effort was not without a small reward. I was able to replace this inefficient code:
    Code:
        nLen = Len(sHex)
        'Convert the string from hex pairs to bytes and store in the ASCII string opcode buffer
        For i = 1 To nLen Step 2
            sCode = sCode & ChrB$(Val("&H" & Mid$(sHex, i, 2)))
        Next i
        nLen = LenB(sCode)
        nAddrSubclass = GlobalAlloc(0, nLen)             'Allocate fixed memory for machine code buffer
        Call CopyMemory(ByVal nAddrSubclass, ByVal StrPtr(sCode), nLen)
    with:
    Code:
        nLen = Len(sHex) / 2
        ReDim bTmp(nLen - 1)
        For i = 0 To UBound(bTmp)
            bTmp(i) = Val("&H" & Mid$(sHex, i * 2 + 1, 2))
        Next i
        nAddrSubclass = GlobalAlloc(0, nLen)            'Allocate fixed memory for machine code buffer
        Call CopyMemory(ByVal nAddrSubclass, ByVal VarPtr(bTmp(0)), nLen)
    J.A. Coutts

  21. #21
    PowerPoster
    Join Date
    Jun 2015
    Posts
    2,224

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    On the subject of not crashing the IDE:
    Just to add more information, If you look at the assembly stubs created. the majority of them take action based on vba6.dll#EbMode()
    If it's in Run Mode, it calls the subclassed handler. If it's in break mode, it passes the call to the previous WndProc, and if the IDE is stopped it unsubclasses the window, and then forwards to the previous WndProc.

    Some of the subclassing tehniques actually go a step farther and hook the actual the IDE Stop/Resettng function, and unsbuclasses everything before passing control back to the IDE.

    If you query EbMode, and move the subclassing to a DLL, you've got 99% IDE protection. I think there are still some cases where the IDE crashes when you hit END.

  22. #22
    Fanatic Member
    Join Date
    Aug 2013
    Posts
    806

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    fafalone, thanks for this excellent write-up. Very helpful. Like others, I've been (blindly) using Paul Caton's methods for years but have been meaning to switch various projects to this CCL technique.

    I'm currently working on some code related to subclassing WM_SETTINGCHANGE, but before moving to SetWindowSubclass, I wanted to ask about this comment from the MSDN article on window subclassing:

    Note: All strings passed to the [subclass] procedure are Unicode strings even if Unicode is not specified as a preprocessor definition.
    I didn't notice any mention of this in your write-up, but do you know - is MSDN accurate? If I subclass something like an ANSI VB window, and am looking at messages where the w/lParams are string pointers, will those pointers always be to DBCS strings?

    In my case, at least, this has some implications for retrieving those strings (e.g. lstrlenA vs lstrlenW, etc), so I'm curious if others have firsthand experience. If MSDN does end up being accurate, maybe worth mentioning that caveat in your article, "just in case"...?
    Check out PhotoDemon, a pro-grade photo editor written completely in VB6. (Full source available at GitHub.)

  23. #23
    PowerPoster
    Join Date
    Feb 2006
    Posts
    20,662

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by Tanner_H View Post
    I didn't notice any mention of this in your write-up, but do you know - is MSDN accurate? If I subclass something like an ANSI VB window, and am looking at messages where the w/lParams are string pointers, will those pointers always be to DBCS strings?
    Not DBCS (which was a form of Asian-language extension for ANSI encoding that mixes one and two-byte characters, designed for early pre-Unicode Win9x) but the UCS-2/UTF-16LE that Windows refers to as "Unicode."

    Double-byte Character Sets

    A double-byte character set (DBCS), also known as an "expanded 8-bit character set", is an extended single-byte character set (SBCS), implemented as a code page. DBCSs were originally developed to extend the SBCS design to handle languages such as Japanese and Chinese. Some characters in a DBCS, including the digits and letters used for writing English, have single-byte code values. Other characters, such as Chinese ideographs or Japanese kanji, have double-byte code values. A DBCS can correspond either to a Windows code page or an OEM code page. A DBCS code page can also include a non-native code page, for example, an EBCDIC code page. For definitions of these code pages, see Code Pages.

    Things like the ListView that COMCTL32.OCX wraps can be shifted in and out of ANSI mode, so depending on the mode you might get/put one encoding or the other.

    So I think the bad news is that "it depends."
    Last edited by dilettante; Sep 19th, 2015 at 01:15 AM.

  24. #24
    PowerPoster
    Join Date
    Feb 2006
    Posts
    20,662

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    More detail:

    For a ListView WM_NOTIFY, the lParam points to a instance of the NMHDR struct. If its .code = LVN_GETDISPINFO, you need to write an LV_DISPINFO struct at the lParam pointer and when (.mask And LVIF_TEXT) is non-0 the requested text you supply is normally ANSI.

  25. #25
    Default Member Bonnie West's Avatar
    Join Date
    Jun 2012
    Location
    InIDE
    Posts
    4,057

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Quote Originally Posted by Tanner_H View Post
    If I subclass something like an ANSI VB window, and am looking at messages where the w/lParams are string pointers, will those pointers always be to DBCS strings?
    Through testing, I've found out that string parameters passed to subclass procedures will be ANSI or Unicode depending on whether the subclass procedure is being called by an ANSI or Unicode subclass. The system apparently performs the necessary conversion between ANSI and Unicode when it calls an ANSI or Unicode subclass procedure (i.e., the system converts ANSI strings to Unicode when calling a Unicode subclass procedure and vice versa). Thus, there is no need to be concerned about receiving ANSI string parameters when the subclass is set by a Unicode API function.

    Quote Originally Posted by Tanner_H View Post
    ... is MSDN accurate?
    I have observed via API Monitor that the SetWindowSubclass API actually calls the SetWindowLongW function internally. Subclasses set using it will therefore always receive Unicode strings.
    On Local Error Resume Next: If Not Empty Is Nothing Then Do While Null: ReDim i(True To False) As Currency: Loop: Else Debug.Assert CCur(CLng(CInt(CBool(False Imp True Xor False Eqv True)))): Stop: On Local Error GoTo 0
    Declare Sub CrashVB Lib "msvbvm60" (Optional DontPassMe As Any)

  26. #26
    Fanatic Member
    Join Date
    Aug 2013
    Posts
    806

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    @dilettante, Bonnie - thanks much. Appreciate your expertise. Dilettante in particular, you'd think I could keep the various Unicode designations straight after all these years. Thanks again.

    Quote Originally Posted by Bonnie West
    I have observed via API Monitor that the SetWindowSubclass API actually calls the SetWindowLongW function internally. Subclasses set using it will therefore always receive Unicode strings
    Perfect, thank you, Bonnie. That makes a lot of sense, given the other warning on the MSDN subclassing page:

    Note: ComCtl32.dll version 6 is Unicode only. The common controls supported by ComCtl32.dll version 6 should not be subclassed (or superclassed) with ANSI window procedures.
    Check out PhotoDemon, a pro-grade photo editor written completely in VB6. (Full source available at GitHub.)

  27. #27
    PowerPoster
    Join Date
    Feb 2006
    Posts
    20,662

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    But as I said, that Unicode assumption breaks down in certain cases.

    See Virtual 5.0 ListView in the CodeBank.

    This is a "5.0" ListView (as in COMCTL32.OCX) being used with version 6.0 ComCtl32.dll and sure enough it operates in ANSI mode by default even so. Whether the VB-supplied OCX put it into that mode for normal use or not I don't know, but it definately operates in ANSI and you'd have to send and handle WM_NOTIFYFORMAT notifications (these go both ways) and reply that you want to use Unicode. This impacts the encoding of WM_NOTIFY message strings.

    That might be one of a very few oddball cases. But when in doubt research may be necessary.

  28. #28

    Thread Starter
    PowerPoster
    Join Date
    Jul 2010
    Location
    NYC
    Posts
    2,344

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Even with a directly created ListView (CreateWindowEx) I had to use WM_NOTIFYFORMAT to get all Unicode messages and strings. The issue came up during label editing; I wasn't getting Unicode support somewhere I can't remember (probably the NMLVDISPINFO item text), which I figured was because there was both LVN_[BEGIN|END]LABELEDIT A and W, and I was only getting A. So I replied with NFR_UNICODE, started receiving the LVN_xLABELEDITW messages, and got full unicode support for label editing.

  29. #29
    Frenzied Member wqweto's Avatar
    Join Date
    May 2011
    Posts
    1,585

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    There is one more caveat with SetWindowSubclass when using ASM thunks. It appears it keeps an internal table, keyed on (wnd class, wnd proc address) pair that is a form of cache which does not get cleared upon RemoveWindowSubclass.

    Normally when you subclass TextBoxes or Forms with direct AddressOf your callback function this table contains 10s of entries at most. But when ASM thunks are being allocated from heap the second member of the pair changes with each Subclass call, so you get lot's of entries in the internal table over the lifetime of your process that never get cleared.

    You can use Bear to monitor subclassing internal table size.

    The nasty thing is that even if you use old SetWindowLong(GWL_WNDPROC) approash but use ASM thunks for the wndproc (Matt Curland's PushParamThunk for instance for get an extra parameter with `This` pointer on the callback) you risk some 3rd party or 1st party control to try to subclass with SetWindowSubclass and leak. Case in point -- standard ComCtl tooltips do it on Vista and newer for tree/list views.

    cheers,
    </wqw>

  30. #30

    Thread Starter
    PowerPoster
    Join Date
    Jul 2010
    Location
    NYC
    Posts
    2,344

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    NOTE: I just discovered that specifying a value for the uIdSubclass parameter when subclassing and unsubclassing is required in some (all?) circumstances if you're going to remove the subclass during runtime.

    I had a VB ListView subclassed and couldn't change the imagelist assignment via API, because I found it to still be calling WndProc code that blocked LVM_SETIMAGELIST, even after I had called UnSubclass. I set uIdSubclass to the hWnd and the issue was resolved.

  31. #31
    PowerPoster
    Join Date
    Jun 2015
    Posts
    2,224

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    interesting. I had no idea you could subclass without specifying uIdSubclass - as it's the combination of the SubClassProc and uIdSubclass that define a unique subclassing instance. Were you just setting uIdSubclass to zero? interesting that it breaks when using 0 as the Id.

    Also maybe its just a misunderstanding, but what scenario wouldn't you unsubclass during runtime? or when you say runtime, do you just mean during the life of the window not counting shutdown. Standard operation is still to unsubclass before window destruction.

  32. #32

    Thread Starter
    PowerPoster
    Join Date
    Jul 2010
    Location
    NYC
    Posts
    2,344

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    I had always been using 0 for uIdSubclass; I do *a lot* of subclassing and never had a problem, even with things like subclassing different hWnds to the same WndProc, until I needed to remove a subclass during runtime- something, as you suggest, is a very uncommon issue.

    In this particular scenario, VB doesn't recognize an API HIMAGELIST assigned to the OCX-based native comctl ListView and problems can arise, so once you set an API imagelist, you subclass and prevent VB from clearing your imagelist assignment. The unsubclassing during runtime came up with the Thumbnail ListView I just posted, where it needs to switch from being assigned the system imagelist to the custom imagelist with the thumbnails when the user switches to thumbnail mode-- and this is the first project I've made that uses the OCX ListView in addition to the SetWindowSubclass-based subclassing (almost always any ListView I make that does anything beyond basics is made with CreateWindowEx, but I didn't want to further complicate the already hard to follow demo). In retrospect I suppose I could have just set a flag as there's nothing else to reset in the stripped down demo version, but why mess with what works.

  33. #33
    New Member
    Join Date
    Jan 2014
    Posts
    3

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Is it possible to use SetWindowSubclass in excel project to safely subclass any build-in control?
    By "safely" I mean that when I hit "stop" button Excel will not crash immediately.

  34. #34

  35. #35
    Member shagratt's Avatar
    Join Date
    Jul 2019
    Posts
    48

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    Im a little confused... Wich one is the better subclassing to use? SetWindowSubclass or using thunking in using classes with assembler patches that check eBmode?
    Every project I see seems to use a diferent way to subclass so its really confusing for new people to understand which is the better way to do it

  36. #36

    Thread Starter
    PowerPoster
    Join Date
    Jul 2010
    Location
    NYC
    Posts
    2,344

    Re: [VB6] Simple, basic subclassing tutorial using the easier SetWindowSubclass metho

    What's better depends on your circumstance. I use both this method and the thunk-based ones depending on if I just need a simple subclass on a normal window, or need more advanced functionality that justifies taking the time to implement a more complex solution.

    If you're really new to it and not worrying about the harder problems like class instances and usercontrols, whatever is the simplest method to get the messages you need to process should come first.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width