Subclass External Programs done for you

[moeur]

This code uses a C++ dll and an ActiveX VB control to hook external programs.
Note: The source is included for both in case you want to change it or just learn from it.

There is another post on this subject that deals with a system wide hook. Here I am posting complete code that allows the user to perform a thread-specific hook on an external program from their VB program and respond to any messages sent to that external program from within the VB Program.

In this method the hook is set in the C++ code so we don't need to mess with passing procedure addresses back and forth. This code handles two types of hooks, and forwards messages along with their complete data structures back to your VB code. This code also allows the VB user to change messages where permitted. See the next post in this thread for more details on how the code works.

There are two parts to the code: a DLL written in C++ (don't worry, it's already compiled for you) that is injected into the thread of the target program, and an ActiveX control that handles communication between the DLL and the developer's VB program.

These components can be used as-is to trap any message sent to external programs, or since I am including the source, can be modified. They handle two types of hooks:

WH_CALLWNDPROC - For sent messages
WH_GETMESSAGE - For posted messages, messages can be changed before being sent on to App

After hooking the target App and setting the messages to monitor, an event will be triggered in the control for each trapped message. The data for each message is supplied to the events in it's original data structure.

After unzipping the files, register the dll and the ocx component. Instructions for using the ActiveX control are included in an html file 'ReadMe.htm'

If anyone finds a problem with this code then let me know so I can fix it.

The following is an example of how to use the Hook Control in a VB App. For another example see Adding Submenus to External Programs.

Example:
This example hooks the text area in Notepad

start a VB project, add the hookcontrol to your tool box and drag it over to your own app.
Add two command buttons cmdHook and cmdUnHook
Before running your app, start Notepad.exe.

Option Explicit
 
Const ParentCaption = "Untitled - Notepad"
Const ChildClassName = "Edit"
Const ChildCaption = ""
 
Private Const WM_LBUTTONDOWN = &H201
Private Const WM_LBUTTONUP = &H202
Private Const WM_SETTEXT = &HC
Private Const WM_CHAR = &H102
Private Const WM_KEYDOWN = &H100
Private Const WM_KEYUP = &H101
 
 
Private Sub cmdHook_Click()
'This routine sets the hook to monitor specified messages
  Dim lhWnd As Long
  With HookControl
 
    'handle to main window
    lhWnd = .GetTopLevelHandle(ParentCaption)
    While lhWnd = 0
     If MsgBox("Open Notepad", vbOKCancel) = vbCancel Then Exit Sub
     lhWnd = .GetTopLevelHandle(ParentCaption)
    Wend
    
 
    'monitor the following message(s)
    .AddMessage WM_CHAR, "WM_CHAR"
    
    'handle to the textbox we want to hook
    .TargethWnd = .GetChildHandle(ChildCaption, ChildClassName, lhWnd)
 
    'Set the hooks
    If .SetHook Then
      cmdHook.Enabled = False
      cmdUnHook.Enabled = True
    Else
      cmdUnHook.Enabled = False
      cmdHook.Enabled = True
      MsgBox "Error Installing Hook."
    End If
 
  End With
 
End Sub
 
 
Public Sub cmdUnHook_Click()
'This routine removes the hook
  If HookControl.RemoveAllHooks Then
    cmdUnHook.Enabled = False
    cmdHook.Enabled = True
  Else
    cmdHook.Enabled = False
    cmdUnHook.Enabled = True
    MsgBox "Error Uninstalling Hook."
  End If
 
End Sub
 
 
'Here is where the posted messages will arrive
'we cannot discard posted messages, but can change them
Private Sub HookControl_PostedMessage( _
    uMsg As Long, _
    wParam As Long, _
    lParam As Long _
)
  Dim Msg
 
  'display the messages as they arrive
  For Each Msg In HookControl.Messages
    If uMsg = Msg.Value Then
        Debug.Print "Posted: " & Msg.Label & "  " & wParam
        Exit For
    End If
  Next
 
  'Here is an example of how to change a message
  'Change all a's  to ‘X’
  If uMsg = WM_CHAR And wParam = Asc("a") Then wParam = Asc("X")
 
  'change message to WM_NULL if key is "s" so Notepad ignores it
  If uMsg = WM_CHAR And wParam = Asc("s") Then uMsg = WM_NULL
 
End Sub
 
 
'reset our buttons if notepad is closed and hook is released
Private Sub HookControl_UnHook()
  cmdHook.Enabled = True
  cmdUnHook.Enabled = False
End Sub

NOTES:
Fixed problem with multiple instances (2/22/2005)
Added Thread local storage to dll to help with multiple instances (2/27/2005)
Fixed problem with decleration in dll that was causing control to crash (3/08/2005)
Fixed problem that prevented control from being used in an array. (10/12/2005)
Uploaded the latest dll source code

[moeur]

The above code is perhaps a little daunting so here is a simplified version for illustrative purposes.
We'll create a project that hooks posted messages and allows the VB user to change those messages before they are passed on to the original process.
First create the C++ dll. It will have three routines accessible to VB, so create your .def file

//HookDemo.def
LIBRARY MainHook
EXPORTS
	InstallFilterDLL	@1
	UnInstallFilterDLL	@2
	SetSharedData		@5

In your source file, do some basic steup

//HookDemo.cpp
#include <windows.h>
#include "WINUSER.H"    
/*---------------------------------------------
          Shared Variables
This data is shared between both prcesses.  The
VB App has access to these variables through the
function SetSharedData
---------------------------------------------*/
#pragma data_seg(".shared")
	bool ChangeMessage = false;
	int Shared_uMsg = 0;
	int Shared_wParam = 0;
	int Shared_lParam = 0;
	HWND hWndVB = 0; //handle to subclassed VB Window
	HWND hWndCtrl = 0;//handle to the window we want to monitor
#pragma data_seg()
#pragma comment(linker, "/SECTION:.shared,RWS")

//---------------------------------------------
// Global Variables, specific to each process
//---------------------------------------------
	HHOOK	hmsgHooks = 0;		// Hook handle for WH_GETMESSAGE
	HINSTANCE	hInstance;	// Global instance handle for DLL

//--------------------------------------------
//        DLL entry-point 
//--------------------------------------------
BOOL WINAPI DllMain(
    HINSTANCE hinstDLL,  // handle to DLL module
    DWORD fdwReason,     // reason for calling function
    LPVOID lpvReserved )  // reserved
{
    // Perform actions based on the reason for calling.
    switch( fdwReason ) 
    { 
        case DLL_PROCESS_ATTACH:
         // Initialize once for each new process.
         // Return FALSE to fail DLL load.
			hInstance = hinstDLL;//save dll handle for each process
            break;
	
        case DLL_THREAD_ATTACH:
         // Do thread-specific initialization.
            break;

        case DLL_THREAD_DETACH:
         // Do thread-specific cleanup.
            break;

        case DLL_PROCESS_DETACH:
         // Perform any necessary cleanup.
            break;
    }
    return TRUE;  // Successful DLL_PROCESS_ATTACH.
}

OK, Now let's create those three routines
First a routine for setting the hook

/*  set WH_GETMESSAGE hook
  Private Declare Function InstallFilterDLL Lib "C:\bin\HookDemo.dll" ( _
    ByVal dwThreadID As Long, _
	ByVal ExtrnHandle As Long, _
    ByVal VBHandle As Long _
) As Long
*/
__declspec(dllexport) int _stdcall InstallFilterDLL(DWORD dwThreadId, HWND ExtrnHandle, HWND VBhandle)
{
	if (hmsgHooks==0)
			hWndVB = VBhandle;\\save handle to subclassed VB Window
			hWndCtrl = ExtrnHandle;//Handle to external window
			hmsgHooks = SetWindowsHookEx(WH_GETMESSAGE,(HOOKPROC) GetMsgProc, (HINSTANCE) hInstance, dwThreadId);
		if (hmsgHooks==0) return GetLastError();
	return 0;
}

Next a routine for removing the hook

/*  Remove the WH_GETMESSAGE hook
  Private Declare Function UnInstallFilterDLL Lib "C:\bin\HookDemo.dll" () As Long
*/
 __declspec(dllexport) int UnInstallFilterDLL(void)
{
	LRESULT result;
	if (hmsgHooks != 0){
		result = UnhookWindowsHookEx(hmsgHooks);
		if (result == 0) return GetLastError();
		hmsgHooks = 0;
	}
	return 0;
}

And a routine so that VB can change the message data

/* SetSharedData Allows the VB App to alter the shared data
  Private Declare Sub SetSharedData Lib "C:\bin\HookDemo.dll" ( _
    ByVal uMsg As Long, _
    ByVal wParam As Long, _
    ByVal lParam As Long _
)
*/
__declspec(dllexport) void _stdcall SetSharedData(int uMsg, WPARAM wParam, LPARAM lParam)
{
	Shared_uMsg = uMsg;
	Shared_wParam = wParam;
	Shared_lParam = lParam;
	ChangeMessage = true;
}

Finally we have to create a routine to forward the messages to the VB App

/*---------------------------------------------------------------------------
				 Filter function for the WH_GETMESSAGE

  The GetMsgProc hook procedure can examine or modify the message. After the hook 
procedure returns control to the system, the GetMessage or PeekMessage function 
returns the message, along with any modifications, to the application that 
originally called it.
Allows changing the message, but not removal (use WM_NULL instead)
---------------------------------------------------------------------------*/

LRESULT CALLBACK GetMsgProc(int nCode, WPARAM wParam, LPARAM lParam)
{
	MSG *lpMsg;
	//return immediately for negative nCodes
	if (nCode >= 0){
		if (nCode==HC_ACTION){
			lpMsg = (MSG *) lParam;
			//see if this is the window we are monitoring
			if (lpMsg->hwnd == hWndCtrl){
				//forward the message to VB App
				ChangeMessage=false;
				SendMessage(hWndVB, lpMsg->message, lpMsg->wParam, lpMsg->lParam);
				if (ChangeMessage==true){//Did VB App make changes to the data?
					lpMsg->message = Shared_uMsg;
					lpMsg->wParam = Shared_wParam;
					lpMsg->lParam = Shared_lParam;
				}//end if change message
			}//end if correct hWnd
		}//end if nCode==Action
	}//end if nCode >=0			
	return(CallNextHookEx(hmsgHooks, nCode, wParam, lParam));
}

This is all you need, now here is how to use it from Visual Basic
You need to subclass something in order to retrieve the messages sent back from the dll.
In this example I have chosen to subclass a checkbox.
I've omitted the sublcassing code since this is explained in detail at various spots in this forum.
First, declare some API functions and our functions

Option Explicit
Private Declare Function FindWindow Lib "user32" Alias "FindWindowA" ( _
  ByVal lpClassName As String, _
  ByVal lpWindowName As String  _
) As Long
Private Declare Function GetWindowThreadProcessId Lib "user32" ( _
  ByVal hwnd As Long,  _
  lpdwProcessId As Long _
) As Long
Private Declare Function FindWindowEx Lib "user32" Alias "FindWindowExA" ( _
  ByVal hWnd1 As Long, _
  ByVal hWnd2 As Long, _
  ByVal lpsz1 As String, _
  ByVal lpsz2 As String _
) As Long
 
Private Const GWLP_WNDPROC = (-4)
Private Const WM_CHAR = &H102
 
'Make sure to change dll path to yours
Private Declare Sub SetSharedData Lib "C:\bin\HookDemo.dll" ( _
    ByVal uMsg As Long, _
    ByVal wParam As Long, _
    ByVal lParam As Long _
)
 
 Private Declare Function InstallFilterDLL Lib "C:\bin\HookDemo.dll" ( _
    ByVal dwThreadID As Long, _
    ByVal ExtrnHandle As Long, _
    ByVal VBHandle As Long _
) As Long
 
 Private Declare Function UnInstallFilterDLL Lib "C:\bin\HookDemo.dll" () As Long

This following routine sets the hook

Private Sub cmdSetHook_Click()
Dim hWndParent As Long
Dim hWndChild As Long
Dim ThreadID As Long
Dim ParentCaption As String
Dim ChildClass As String
Dim ChildCaption As String
 
  ParentCaption = "Untitled - Notepad"
  ChildClass = "Edit"
  ChildCaption = ""
 
  'subclass the checkbox
  Set CSubClsApp = New CSubclass
  CSubClsApp.hwnd = Check1.hwnd
  CSubClsApp.EnableSubclass
 
  'get handle to parent window
  hWndParent = FindWindow(vbNullString, ParentCaption)
  While hWndParent = 0
    If MsgBox("Open Notepad", vbOKCancel) = vbCancel Then Exit Sub
    hWndParent = FindWindow(vbNullString, ParentCaption)
  Wend
 
  'get handle to child window
  hWndChild = FindWindowEx(hWndParent, 0, ChildClass, ChildCaption)
 
  'get ThreadID
  ThreadID = GetWindowThreadProcessId(hWndChild, 0)
 
  'set the hook
  If InstallFilterDLL(ThreadID, hWndChild, Check1.hwnd) Then
    MsgBox "Cannot Install Hook"
    CSubClsApp.DisableSubclass
  End If
 
End Sub

Of Course we don't want our program to crash

Private Sub Form_QueryUnload(Cancel As Integer, UnloadMode As Integer)
  UnInstallFilterDLL
  CSubClsApp.DisableSubclass
End Sub

Finally, we need to write code to handle the hook messages

Public Function NewWndProc( _
  ByVal hwnd As Long, _
  ByVal uMsg As Long, _
  ByVal wParam As Long, _
  ByVal lParam As Long _
) As Long
'Do your message handling here
'If you want to change the data, do it like this
    If uMsg = WM_CHAR Then
        'change all keyboard input to 'X'
        wParam = Asc("X")
        'if you want to discard message, change uMsg to WM_NULL
        SetSharedData uMsg, wParam, lParam
    End If
        'Pass message to the default window procedure
        NewWndProc = CallWindowProc(CSubClsApp.OrigWndProc, hwnd, uMsg, wParam, lParam)
End Function

And that's it. This code will hook Notepad and change all keyboard input to 'X's
Next you might want to add a hook to trap sent messages.
This way you can respond to WM_DESTROY messages telling you the external app is shutting down.

Attached is the full code for the above demo.
Note: missing files added to attached file 2/16/05

[EJ12N]

Really nice!, just awesome thanks a lot

[moeur]

Handling Multiple Instances
I received some feedback on the Hook Control code's inability to handle all cases of multiple instances. I had originally tested the program by hooking several controls in a target program by using several instance of the control in a single VB App. There are, however two cases I didn't plan for:

1. Multiple VB programs hooking a single target program
2. One VB program hooking multiple target processes

Finding the resulting solution took me into the world of multi-threading and shared data since the dll is loaded by each process that either makes a call to the exported routines (VB Apps) or is hooked (target Apps). The solution lay in making sure of the following:

1. No thread is hooked more than once
2. A VB App can't remove a hook unless no other VB App is using it
3. Each thread keeps it's own list of messages to monitor, and VB App callback handles

With this in mind, the following had to be implemented.
1. Each Target process needs to keep track of a few things:
.....Handles back to the hooking VB Apps
.....Lists of messages to monitor
.....Associated lists of handles of intended windows
2. We must only allow the VB Apps to set a particular hook only once. The next time a VB App requests a hook, we need to stop it from actually setting the hook, but instead have it increment a counter of how many times a hook setting was requested. This is neccessary when it comes time to remove the hook. When a VB App requests the hook be removed, we can just decrement the counter until no more VB Apps remain that are expecting the hook to be set.

The VB Apps need to share the following info with each other
1. which hooks have been set
2. The original hook handles for removing the hook
3. A count of how many VB Apps have requested the hook

The target processes share nothing with each other.

So, the dll can be divided into two parts: Exported routines which are called from the VB Apps and need to share data, and the Hook procedures which share data with no one (with one exception).

Sharing Data between Apps
There are two ways I know of to share data between processes.
1. Create named data sections using the #pragma statement
2. Use memory-mapped files
The first technique is the simplest.

//Shared Data Section.  Make sure to initialize all variables
//declared here or they will not be shared
#pragma data_seg(".shared")
    int ThreadCount = 0;//shared between VB Apps
    int TestData1=0;
    int TestData2=0;
#pragma data_seg()
#pragma comment(linker, "/SECTION:.shared,RWS")

There is one drawback to this method; accessing pointers in the shared data segment is unreliable. Since I want to share an array of a user defined structures for the VB Apps, I had to use the second method of sharing data.
My Data structure is

//This structure is shared between all instances of VB Apps
struct sHookData{
    int Counts;
    HHOOK Handle;
};
struct sThreadData{
    int ThreadID;
    sHookData SendHook;
    sHookData PostHook;
    sHookData kbHook;
    sHookData MouseHook;
};

The code used to create the named file mapping object is placed in the dll entry point

#define SHMEMSIZE 4096 //size of shared memory block
static LPVOID lpvMem = NULL;      // pointer to shared memory
static HANDLE hMapObject = NULL;  // handle to file mapping
HINSTANCE	hInstance = 0;	// Global instance handle for DLL
//-------------------------
// The DLL entry-point function
//-------------------------
BOOL APIENTRY DllMain(
    HINSTANCE hinstDLL,  // handle to DLL module
    DWORD fdwReason,     // reason for calling function
    LPVOID lpvReserved )  // reserved
{
    BOOL fInit, fIgnore; 
    // Perform actions based on the reason for calling.
    switch( fdwReason ) 
    { 
       case DLL_PROCESS_ATTACH:
         // Initialize once for each new process.
         // Return FALSE to fail DLL load.
			hInstance = hinstDLL;//save dll handle for each process

        // Create a named file mapping object.
             hMapObject = CreateFileMapping( 
                INVALID_HANDLE_VALUE, // use paging file
                NULL,                 // default security attributes
                PAGE_READWRITE,       // read/write access
                0,                    // size: high 32-bits
                SHMEMSIZE,            // size: low 32-bits
                "dllmemfilemap");     // name of map object
            if (hMapObject == NULL) 
                return FALSE; 
 
            // The first process to attach initializes memory.
             fInit = (GetLastError() != ERROR_ALREADY_EXISTS); 
 
            // Get a pointer to the file-mapped shared memory.
             lpvMem = MapViewOfFile( 
                hMapObject,     // object to map view of
                FILE_MAP_WRITE, // read/write access
                0,              // high offset:  map from
                0,              // low offset:   beginning
                0);             // default: map entire file
            if (lpvMem == NULL) 
                return FALSE; 
 
            // Initialize memory if this is the first process.
             if (fInit) 
                memset(lpvMem, '\0', SHMEMSIZE); 
			 
			 //pointer to our shared data structures
			 ThreadData = (sThreadData*) lpvMem;
            break;
	
        case DLL_THREAD_ATTACH:
         // Do thread-specific initialization.
            break;

        case DLL_THREAD_DETACH:
         // Do thread-specific cleanup.
            break;

        case DLL_PROCESS_DETACH:
            // Unmap shared memory from the process's address space.
            fIgnore = UnmapViewOfFile(lpvMem); 
 
            // Close the process's handle to the file-mapping object.
            fIgnore = CloseHandle(hMapObject); 
            break;
    }
    return TRUE;  // Successful DLL_PROCESS_ATTACH.
    UNREFERENCED_PARAMETER(hinstDLL); 
    UNREFERENCED_PARAMETER(lpvReserved); 
}

To access the shared data, we just need to use the pointer created above as a pointer to our data structures

sThreadData* ThreadData;
ThreadData = (sThreadData*) lpvMem;

One thing not accounted for above is the matter of how to get the the handle of the original hook procedure from the VB App to the new hook procedure which resides in the target process. We need this value in the new hook procedure so that we can contiunue the hook chain and pass control to the next hook procedure in the chain.

I handle this by sending a user-defined message from the VB App thread to the target thread. This message carries the old hook procedure handle with it.
In the InstallFilterDLL routine

SendMessage(hWndDLL, UM_INITHOOK,(WPARAM)WH_CALLWNDPROC, (LPARAM)ThreadData[i].SendHook.Handle);

sends the handle in the lParam variable. When the target recieves the message it intercepts it and sets a global variable to this value

case UM_INITHOOK://store hook handle
  //wParam - HookType
  //lParam - HookHandle
  switch (lpMsg->wParam){
  case WH_CALLWNDPROC:
    if (hhookHooks==0)
	hhookHooks = (HHOOK)lpMsg->lParam;
    break;
  case WH_GETMESSAGE:
    if (hmsgHooks==0)
	hmsgHooks = (HHOOK)lpMsg->lParam;
    break;
  case WH_KEYBOARD:
    if (hkbdHooks==0)
	hkbdHooks = (HHOOK)lpMsg->lParam;
    break;
  case WH_MOUSE:
    if (hmouseHooks==0)
	hmouseHooks = (HHOOK)lpMsg->lParam;
    break;
  }

I'm not sure this is the best way to handle this problem, but it will suffice for now.

[moeur]

Thread Local Storage

One more scenerio that is not handled by the code above is the case of hooking two seperate threads that belong to the same process. Why this is a problem, is that each process only loads one copy of the dll. If this process has multiple threads they will all share the global data inside the dll. We need some mechanism of keeping a thread's global data private to that thread. This mechanism is called Thread Local Storage (TLS).

Setting up TLS
To set up TLS we need to do something each time a new process or thread attaches the dll
So, in the dllMain function

	
case DLL_PROCESS_ATTACH:
  //----------------------------------------------------
  //      Set up Thread Local Storage to keep data
  //       specific to each thread
  //----------------------------------------------------
  // Allocate a TLS index.
   if ((dwTlsIndex = TlsAlloc()) == TLS_OUT_OF_INDEXES) 
      return FALSE; 
  // No break: Initialize the index for first thread.

  //     Do thread-specific initialization.
  //----------------------------------------------------
  case DLL_THREAD_ATTACH:
    // Initialize the TLS index for this thread.
    lpvData = (LPVOID) LocalAlloc(LPTR, sizeof(sMessageData)*MAX_HOOKS+20); 
    if (lpvData != NULL) 
       fIgnore = TlsSetValue(dwTlsIndex, lpvData); 
    break;

And we need cleanup

  //     Do thread-specific cleanup.
  //----------------------------------------------------
  case DLL_THREAD_DETACH:
    // Release the allocated memory for this thread.
    lpvData = TlsGetValue(dwTlsIndex); 
    if (lpvData != NULL) 
        LocalFree((HLOCAL) lpvData); 
    break;

  //     Do process clean up
  //----------------------------------------------------
  case DLL_PROCESS_DETACH:
    //----------------------------------------------------
    //       Clean up Thread Local Storage
    //----------------------------------------------------
    // Release the allocated memory for this thread.
    lpvData = TlsGetValue(dwTlsIndex); 
    if (lpvData != NULL) 
       LocalFree((HLOCAL) lpvData); 
    // Release the TLS index.
    TlsFree(dwTlsIndex);     
    break;

Accessing the TLS data
Since each pair of hooks is associated with one thread, the data that we want to keep private is data associated with each hook. For instance the message queues

struct sMessageData{
	HWND hWndVB;//handle back to hooking VB App
	HWND hWndChild;//handle of window message is intended for
	int MsgCNT;//number of messages to monitor
	int Message[MAX_MSGS];
};

As well as the hook return handles and a count of hook requests.

The code inside the hook functions is run when a message arrives. This code is run within the target thread, so we need access to the thread specific data we stored for that particular thread. To Access this data we can do the following:

In the code above, a global variable (dwTlsIndex) was initialzed to act as a process-wide index to the TLS. Once inside a thread you can get a pointer to your local storage area using this global index.

LPVOID lpvData;
lpvData =  TlsGetValue(dwTlsIndex);

Now we can access the data; for instance our message data structures

sMessageData* TLSMessageData = (sMessageData*)(((int*)lpvData)+5);

And the other data

HHOOK& TLSsendHHook = *((HHOOK*)lpvData + 4);
HHOOK& TLSpostHHook = *((HHOOK*)lpvData+1);
//Set our thread local variable to this shared value
int& TLSHookCount = *((int*)lpvData);

Passing data from VB thread to target thread
As mentioned in an earlier post, I had a problem of getting the original hook handle that is returned when the hook is set, from the VB thread to the proper target thread. I solved this by sending a Windows message between the threads. I've changed that now in the following way.
In the VB thread the handles are stored in memory that is shared between all processes along with the ThreadID associated with the hook.

In the target thread (i.e. the hook procedure) we can examine the shared data, look for our threadID number and get the associated hook handles which we need for the CallNextHookEX function.

  //see if TLS return hook handle has been initialized
  if (TLSsendHHook==0){//if not then get value from file mapping
    //get current threadID
    ThreadID = (int)GetCurrentThreadId();
    //find matching array element
    for (i=0;i<ThreadCount;i++)
	if (ThreadData[i].ThreadID==ThreadID) break;
    if (i==ThreadCount) return 1;//something went wrong
    TLSsendHHook = ThreadData[i].SendHook.Handle;
  }

All these changes were brought about by feedback from people using the code. So, if you find a problem, let me know.

[Jacob123]

I currently am having my own DLL injected into a game.

I have a client that I made in VB. I want this client to communicate with this injected DLL. I made this dll.

For instance, the DLL reads my current ingame coordinates. I want the DLL to send my client these coordinates.
I know how to get the coordinates, but how would I go about actually sending them to the client?

Could I somehow use this example for what i'm trying to do?

[Jacob123]

Couldnt I just could subclass my game's window, and just send WM_COPYDATA with the coordinates as a parameter?

[moeur]

Couldnt I just could subclass my game's window, and just send WM_COPYDATA with the coordinates as a parameter?

Subclassing your game's window will allow you to send data from your client to the game, but you can't send data the other way since you don't have a window in the client.
See your original thread at
http://www.vbforums.com/showthread.p...=1#post1939741
for the technique I recommend.

[Max_aka_NOBODY]

Hey, moeur. This code of yours looks absolutely stunning, especially since I got into programming in the first place to find something like this, but I'm having a problem. When trying to register the DLL, it gives me a "DLLRegistryServer entry point not found" error message, and the app fails to find it. Your test application attched with the control works very well though. I'm on WinXP SP1. Thanx for any assistance in advance, and sorry for cluttering your thread.

[moeur]

sorry for cluttering your thread

This is the reason for the thread.

The dll does not have any registration requirements. Place it in your Windows\System32 directory and your App should be able to find it.
Alternatively, you could place it in the same directory as your exe, but then if you run a program from the IDE it won't see the dll.

[Max_aka_NOBODY]

Thanx. I ended up putting the DLL in the same folder. Now there's another problem, which is that the example code you provided in the beginning of the thread, while working great on retrieving data, crashes instantly on any attempt to change it, with 'The instruction at "0x********" referenced memory at "0x00000102". Memory could not be "written".' message.... What might have caused this?

[moeur]

Which code are you referring to?
Code in the zip file?
When you say trying to change it, is it a posted message?

[Max_aka_NOBODY]

I'm referring to the code you posted in the beginning of the thread, which catches a posted message from Notepad and sends "X" instead of "a".

[moeur]

Can you do me a favor and attach your project that crashes?

[Max_aka_NOBODY]

Yes. It's the same code you posted though...

[moeur]

One trick I use to help debug Hooked code is to compile it before running it. If you set Option Explicit then the compile will find bugs in your code that are hard to find when you run it.

In this case I saw that WM_NULL was not defined

Private Const WM_NULL = 0

[Max_aka_NOBODY]

No, that isn't the issue. I was using the 'Declares' module supplied with the test app, which has that and all the other used constants declared.

[moeur]

It seems that everytime I fixed one thing I break another.
I found the problem and uploaded the fixed code to the original zip link above.
The changes are all in the dll file
Thanks for the testing
Let me know if therre are more problems

[Chris49ers]

Hi Moeur,

Great Code!

I am having the following problem.

I am using your code to successfully hook into the EM_REPLACESEL message sent to a RICHEDIT control in another application (different process).

I get SentMessage events correctly. For this event, the Lparam in question represents a pointer to a string in the other process's memory space.

I'm having two problems:

1. The lparam value returned from your code, for example, 1310180, does not match the lparam value returned from Spy++ (for example, 00E97C80 [Hex], which is 15301760 in Decimal). I have confirmed multiple times that the actual EVENTS/MESSAGES captured are the same, but I cannot get the lparam values to match up.

2. Like Spy++ provides, I need the actual value of the string pointed to by the lparam string pointer. Is there a way to get this with your code? Spy++ refers to this as "decoding" the lparam value (i.e., getting the value in memory pointed to by the lparam pointer).

Finally, what do you see involved in porting this over to C#?


Thanks very much for your effort with this--this is stellar work.


Best regards,

Chris

[moeur]

To Get the string value pointed to by lParam you can use a function like this.

Option Explicit
Public Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" ( _
    pDest As Any, pSrc As Any, _
    ByVal ByteLen As Long _
)
 
Public Function StrFromPtr(pStr As Long) As String
Dim bTemp As Byte
Dim i As Long
Dim x As String
x = ""
    For i = 0& To 255&
        CopyMemory bTemp, ByVal pStr + i, 1
        If bTemp = 0 Then Exit For
        x = x & Chr(bTemp)
    Next i
StrFromPtr = x
End Function

For Unicode you'll have to make a few adjustments.
Is this what you're looking for?

[Chris49ers]

I saw that function in your code. In theory, it should do OK. However, the actual location of this memory (the string) is in the other application's process space...

I've never tried it, but I don't think CopyMemory can look into another process's memory space? That's what I'm trying to do...


Edit: I think the thinking here is that each process gets its own memory space, and the pointer to the string is a "relative" reference relative to the current process--not absolute. So the same address in two different memory spaces will point to entirely different (and not necessarily valid) things...

My suspiscion, though, is that since the DLL is injected into the other process's space, that it could decode the lparam there (in the C++ code) and then return this string value back up to VB as a standard character array. (We could put the constraint on it that it fit in some buffer of size, say, 4096 bytes.)

So, one solution is as follows... You could have a boolean value that indicates whether or not to "decode" the wparam and lparam, and if so, as what type. (Initially, you could just have a bit that dictates whether or not to decode lparam as a string.) This could then be passed up to the receiving vb function whose signature would change to accomodate the string.

Of course, in C++, deferencing a pointer to a string value is straightforward; getting it into VB is tricky but well understood (although I personally would really have no clue where to begin).

If someone wanted to make modifications to the C++ file, what is the procedure to recompile and export a DLL?


Thanks,

Chris

[moeur]

You are correct in your thinking about processes and memory spaces. The address in lParam is valid only inside your local process, but it does contain the string you want. Just use the above procedure to extract it. This is, BTW, why you see different numbers in HookControl and SPY++ for lParam, they are both local to their own process.

There is no reason to decode the string in the dll since it has been forwarded to your VB app. Maybe I don't understand what you are trying to do or what the problem is.

If someone wanted to make modifications to the C++ file, what is the procedure to recompile and export a DLL?

If you have Visual C++, just double click on the HookDll\MainHook.dsw file and the whole project will be loaded into the IDE ready for recompiling.

[Chris49ers]

Hi Moeur! Thanks so much for the reply!

You're right, I was able to use the function you provided to decode the string pointed to by the lparam. I was convinced that I didn't have the string in memory of my local process, but apparently you've taken care of this with your code. As you point out this also explains the discrepancy between your lparam values and Spy++'s.

Thanks again for your help here.

So I have the precise capability I need here in VB, but my project is in C#. Last night I spent three hours carefully translating the code that I could, but I'm stuck on a number of small issues (VB's AddressOf capability, for one).

Can you think of any reason why this would not work in C# / .NET? I know I can make all of the correct calls to SendMessage with native P/Invoking, and the same applies for the exported methods in MainHook.dll. I *think* C# is capable of subclassing VB-style (not to be confused with subtyping, btw), but I am not sure...

Any pointers here? Has anyone looked into porting this over to C#?


Thanks again, Moeur...

--Chris

[Jordan]

Absolutely excellent code! I've been looking for a VB external subclassing tool forever. What are the chances that you add system wide hooking as well? I personally need system wide WH_SHELL, WH_KEYBOARD, and WH_CALLWNDPROC hooks. Adding that capability would really complete this package.

Also, is it possible with the current code to override the default action? For example if I were monitoring WM_KEYDOWN could I block the event from occuring?

[moeur]

What are the chances that you add system wide hooking as well?

That should be a seperate project.
You can take a look at this link for info on how to do a system-wide hook.
Also there are a couple of system-wide hooks you can do with 100% VB 6 code: WH_MOUSE_LL, WH_KEYBOARD_LL, WH_JOURNALRECORD, and WH_JOURNALPLAYBACK.

Also, is it possible with the current code to override the default action?

Yes, posted messages can be changed, but not discarded. You can however change a posted message to WM_NULL which essentially discards it. You are in luck, WM_KEYDOWN is a posted message.

[Johno]

I desperately need a working external subclass like this one, for use in VB.NET. I am assuming that the issue is, the "addressof" operator, it is different in .NET. Could this be part of the reasons why it does not work in .NET? If so, could it be made possible to post your own addressof value? That would be fantastic. It would have to be given a custom data type though, because the addressof function needs a delegate type.

[moeur]

Unfortunately, I am not familiar with .NET stuff.
Does VB.NET allow you to use VB6 ActiveX controls?
If you want to port this code to .NET then the only issues I can think of are:
Calling dll functions, same as vb6?
Subclassing is done differently in .NET
Here is what msdn says about converting VB6 to VB.NET. They specifically mention subclassing, API calls and message queues.
I will be glad to help where I can, but most .NET questions will have to be asked in the .NET forum.

[Johno]

Hi Moeur,
I did try in the .NET forum, but it seems you're the only person who really knows anything about this subclassing, in either language!

The reason I believe that the library can work, is that I used a VB6 control another time (SetCWPMSGHook). That worked, but it seemed to have a problem when you tried to re-hook a program after unhooking it, which I believe wasn't just a .NET problem.

The only distinct difference in the subclassing, is the use of AddressOf. If the DLL uses this function, I am assuming that is the cause of the program. .NET is actually very similar to VB6 as a whole. Except that Integers have a much larger capacity, so a lot of the time it is better to replace longs with integers.

Could we possibly chat on MSN Messenger? As I'm unaware of when you have replied to my posts.

Cheers.
John

[maxm007]

Hi thank you for that useful post. My current project requires me to be notified AFTER an external window has received and processed message. More specifically , I need to know when a window has repainted so I can paint an overlay grid over it.
Using your control my overlay gets painted over of course. Could you give me some pointers of how I should modify your code to also support a WH_CALLWNDPROCRET hook?

[moeur]

support a WH_CALLWNDPROCRET hook?

I will change the code to handle this. I'll post back when it is finished.

[moeur]

For those of you who have asked how to port the code to .NET I have some info.
First of all, there is an excellent book on Subclassing and hooking in VB which includes two chapters on .NET.
http://www.oreilly.com/catalog/subhookvb/index.html

Fortunately for us the example code is posted online. Here are examples of mouse and keyboard hooks in .NET as well as sublcassing examples.
http://examples.oreilly.com/subhookvb/Chapter22-23.zip

Good luck.

[moeur]

Here is a Beta version of code that handles WH_CALLWNDPROCRET hook. Let me know of bugs.

[stevesny]

Hi Moeur!

As has already been said -- but I'll use a different adjective -- this is superlative code!

In testing with a VB6 application, it works great under Windows NT4, 2000, and XP. But has problems under 95, 98, and ME (no error messages, but no callbacks occur).

Your original HookDemo2 does work under 95, 98, and ME. So, after looking at the differences in your C code, I modified the HookDemo2 VB test program to conform with the new InstallFilterDLL in MainHook.dll. The call to InstallFilterDLL then passed, but the callback wasn't called.

If you don't have access to 95, 98, or ME, I'd be happy to help test and provide feedback.

I also have a suggestion. Since subclassing an external window with no defined message has no use, you might want to consider making the default (if no messages are defined) passing all messages to the callback. That way, a VB programmer could use debug.print to view the messages that occur in response to an event, and gain insight into what to modify or discard to achieve a desired effect.

Again, great work!

Best regards,

Steve

[moeur]

Since subclassing an external window with no defined message has no use, you might want to consider making the default (if no messages are defined) passing all messages to the callback

An excellent suggestion. I'll incorporate this idea.

Your original HookDemo2 does work under 95, 98, and ME

Can you give me the link to the code that works under 95? I'll check it out.

Thanks for the feedback and the new adjective

[stevesny]

I've attached HookDemo2.zip, which I downloaded from a link near the top of this thread. Under Windows 95, 98, and ME, if Notepad is not already open when the "Set Hook" button is pressed in the HookDemo2 executable, Notepad tends to produce a fault.

Again, if I can be of any help in testing, please let menow.

Steve

[moeur]

can you show me the changes you made that caused HookDemo2 to stop working?

Thanks

[moeur]

Under Windows 95, 98, and ME, if Notepad is not already open when the "Set Hook" button is pressed in the HookDemo2 executable, Notepad tends to produce a fault.

This makes no sense. The code for getting the handle to notepad is

'get handle to parent window
  hWndParent = FindWindow(vbNullString, ParentCaption)
  While hWndParent = 0
    If MsgBox("Open Notepad", vbOKCancel) = vbCancel Then Exit Sub
    hWndParent = FindWindow(vbNullString, ParentCaption)
  Wend

As you can see, if notepad isn't open then Findwindow returns zero
Then a message box appears,
if the user presses OK, then findwindow is called again.
There is no difference between the first call to findwindow and the second.

[stevesny]

can you show me the changes you made that caused HookDemo2 to stop working?


Thanks

Hi,

The HookDemo2 test program did and still does work with HookDemo.dll. Since the test program with HookDemo.dll works on 95, 98, and ME, and the test programs with MainHook.dll don't, I thought I'd see if I could isolate the problem to either the OCX or the DLL.

To do this, I modified the HookDemo2 test to work with MainHook.dll, by changing the call to InstallFilterDLL to pass the arguments required by MainHook.dll. That was the only change. After so doing, the call to InstallFilterDLL passed (i.e. no "Cannot Install Hook" message). However, the callback was not called. This may argue for the fact that the issue lies in the DLL rather than the OCX.

Interestingly, there's an article at http://codeproject.com/system/hooksys.asp, with the comment "Injecting DLL by using CreateRemoteThread() API function
Well, this is my favorite one. Unfortunately it is supported only by NT and Windows 2K operating systems. It is bizarre, that you are allowed to call (link with) this API on Win 9x as well, but it just returns NULL without doing anything."

Even though you don't use CreateRemoteThread() (at least I didn't see it), there may be some call in either the DLL or OCX which appears to work on 9x, but doesn't.

Regarding the Notepad fault with HookDemo2, you're right, it doesn't make sense. It may be just one of those "VB things", and is really of no consequence.

Anyway, I hope my explanation was understandable. If not, please let me know.

Steve

[moeur]

OK, I think I know why mainhook.dll does not work with Win9x.
Besides the WH_GETMESSAGE hook that Hookdemo2 installs, this dll installs and relies heavily on a WH_CALLWNDPROC hook.

The WH_CALLWNDPROC hook behaves differently under Win9x than it does under NT/2000 and XP. In the case of Win9x the hook procedure is called in the context of the thread that called Sendmessage, in the case of the later OS's, the hook procedure is called in the context of the hooked application's thread.

In my scheme, I communicate with the dll that has been injected into the external App by sending messages from VB to the external thread. This triggers the WH_CALLWNDPROC hook procedure and in the case of Win9x, this takes place in the VB thread.

If my guess is correct that this is what is causing the problem, I would have to rewrite the dll to make allowances for this. This is not a simple fix and I don't have Win9x running on any of my machines anymore so I wouldn't have a platform to develop this code on.

What I would rather do is work with you to alter the much simpler HookDemo.dll so that it has the functionality that you require.

What do you think?

[stevesny]

OK, I think I know why mainhook.dll does not work. What I would rather do is work with you to alter the much simpler HookDemo.dll so that it has the functionality that you require.

What do you think?

Hi,

This is indeed very nice of you! It seems that HookDemo.dll does not allow for the ability to subclass multiple windows, which is what I need to do. And also, for performance reasons, it would be nice to only pass messages that are specified -- unless none are specified, in which case, all messages. I really don't care about calling it through an OCX.

Can this accomplised using the techniques in HookDemo.dll?

Thanks very much,

Steve