Subclassing an external window

[Megatron]

I find this question comes up quite a bit. I posted a solution in the API forum, but thought it would nice to have one here as well.

This example will subclass "Notepad" and catch the WM_RBUTTONDOWN event. We will display our own custom messagebox, instead of the default context menu that usually pops up.

So that said, open Notepad, and don't close it until I tell you to.

Now, let's start off with the gross part: C coding. Start a new Win32 Dynamic Link Library project in C++.

Insert the following code into it. I've tried to comment all the way through so you know what's going on.

#include <windows.h>
HHOOK hHook;

////////////////////////////////////////////////////////////////////////////
// GetMsgProc
// 
// This callback function will be imported from our VB app
////////////////////////////////////////////////////////////////////////////
extern "C" __declspec(dllexport) LRESULT CALLBACK GetMsgProc(INT nCode, 
															   WPARAM wParam, 
															   LPARAM lParam)
{
	// The lParam parameter contains a structure that provides information 
	// about the message being sent. 
	MSG* pCwp = (MSG*)lParam;

	// Here, we find the window we want to subclass. In this example, I'll use the
	// typing area of notepad
	HWND hEdit = FindWindowEx( FindWindow("Notepad", NULL), 0, "Edit", NULL );
	
	// We want to catch the WM_RBUTTONDOWN event, and we only want to trap it
	// if it was sent to the window we found earlier
	if(  (pCwp->message == WM_RBUTTONDOWN) && (pCwp->hwnd=hEdit) )
	{
		// The message has been caught -- do stuff here.

		// NOTE: If you do not need to replace/intercept the message, rather
		// just "know when it happens" you can send a message back to your main
		// window, and have any processing done there.
		MessageBox( hEdit, "You clicked it", "Bingo", MB_OK );
		return 0;
	}

	// Call the next hook.. Don't miss this step
	return CallNextHookEx( hHook, nCode, wParam, lParam );

}

All we're really doing is writing our GetMsgProc callback function, and specifying that the function is for export. This allows external applications to import the procedure.

So once that's created, add the following code to your VB Project.

Private Declare Function LoadLibrary Lib "kernel32" Alias "LoadLibraryA" (ByVal lpLibFileName As String) As Long
Private Declare Function GetProcAddress Lib "kernel32" (ByVal hModule As Long, ByVal lpProcName As String) As Long
Private Declare Function SetWindowsHookEx Lib "user32" Alias "SetWindowsHookExA" (ByVal idHook As Long, ByVal lpfn As Long, ByVal hmod As Long, ByVal dwThreadId As Long) As Long
Private Declare Function UnhookWindowsHookEx Lib "user32" (ByVal hHook As Long) As Long
Private Declare Function FreeLibrary Lib "kernel32" (ByVal hLibModule As Long) As Long
Private Const WH_GETMESSAGE = 3
Private hHook As Long
Private hModule As Long
 
Private Sub Form_Load()
        
    'Receives the address of the GetMsgProc function, which will be imported
    Dim lpfn As Long
    
    'Loads our DLL into memory. Change the path to the path of your DLL
    hModule = LoadLibrary("C:\MyPath\Subclassdll.dll")
    
    If hModule <> 0 Then
        'Retrieve the address of the GetMsgProc function in our DLL
        lpfn = GetProcAddress(hModule, "_GetMsgProc@12")
        
        'Install the WH_GETMESSAGE hook
        If lpfn <> 0 Then hHook = SetWindowsHookEx(WH_GETMESSAGE, lpfn, hModule, 0)
    Else
        MsgBox "Cannot load module"
    End If
    
End Sub
 
Private Sub Form_Unload(Cancel As Integer)
    'Uninstall the hook, and free the module in use
    If hHook <> 0 Then UnhookWindowsHookEx (hHook)
    FreeLibrary hModule
End Sub

The comments should explian what's going on in the code. I also want to draw your attention to the "_GetMsgProc@12" line. This is the function we're importing, but notice that's not what we originally called it? C modified the function name when you compiled your DLL. I found out the 'true name' via Dependancy walker. (Note: All hook functions will have the same prefix and suffix, since they contain the exact same parameters -- but that's going a little off topic).

Now, run your VB app, and try right-clicking in notepad. Close your VB application via the close-button (i.e. not "End" command, or "Stop" button on the toolbar). Also, close the VB app before you close notepad.

Save your work often.

If you break any of these rules, you'll discover why so many people recommand against subclassing external applications.

(You may close Notepad now)