[vb6] IDE-Safety Thunks for subclassing, hooking and more: A new breed

[LaVolpe]

Attached project is something I've been thinking about for a couple years, just never found the time to put it together. See post #2 for tips, tricks, other info.

Disclaimer. This is not compatible with VBA (i.e., Office modules). Not tested with VB5.

Suggestion. If you like these thunks, recommend placing a copy of the class file into your templates folder shown below. If that is done, whenever you right click in your project explorer to add a new class, the thunks class will be an option to select. When updates are made to this project, update that file also.
C:\Program Files\Microsoft Visual Studio\VB98\Template\Classes

The suite of thunks included in the sample project are wrapped in VB COM objects. This means that the thunks have properties and methods exposed just like any VB object. Granted, the object methods are late-bound (no intellisense) vs. early-bound (faster & has intellisense). Intellisense can be achieved with use of wrappers. Early bound calls can be achieved by creating/using a TLB designed to mimic the thunks.

The documentation for each thunk and more is included with the attached Readme files. Those files are important for you to download if you want to play with the thunks outside of the sample projects. Also included are the NASM scripts used to compile the thunks and directions on how to compile them if you want to manipulate the scripts.

What is a thunk? It is simply executable code placed in process memory, unknown to VB. Typically, thunks are used to enhance VB in one way or another beyond what can be done within VB and without using an external DLL. Thunks are not scary if you have the source code and can verify what is being done. I don't expect everyone to know how to read assembly; therefore, every thunk source is heavily commented so that anyone could follow the logic and see there is nothing malicious. The fact that I've included the source code should also put your mind at ease.

The key to strong resistance to crashing when in IDE is the fact that the thunk is wrapped in a COM object. By being a declared COM object, VB will always unload it. Since it knows it is being unloaded, it can manage its own destruction, including its own clean-up and removal from process memory. Also, being "external" to VB, a thunk can ask VB whether the IDE is paused or not which can't be done from "internal" code. When paused, the thunk simply passes the message on or eats it, as applicable & defined by you. Minimally, a thunk can be IDE-safe if it can:
- determine when not to callback to the IDE; that is, when the IDE cannot respond
- ensure it can unload/disengage before anything it calls back to is unloaded & becomes dead code

Thunks included, along with a brief description:

1) Subclassing. Uses the common controls subclassing library. The thunk can subclass multiple hWnds simultaneously.

2) Hooks. Windows offers various hooks, i.e., keyboard, mouse, CBT and more. A sample project shows how subclassing and hooking are used together to create owner-drawn VB comboboxes.

3) Timer. Probably of little use, but provided anyway. The only real advantage of a thunk timer is the ability to add it to a VB class and extended interval values.

4) Custom Window Classes. For those of you that are API-nuts and like to create pure or semi-pure custom windows, the thunk helps manage the class procedure.

5) Callbacks. Both standard Windows (stdCall) and C-style (cDecl) formats are supported. A sample project shows both.

6) COM object hooks. Ability to hook/subclass a COM object, either one exposed by VB or a VTable-only interface. An example shows how to subclass the stdPicture object to hook its rendering.

7) GDI+ token/handle management. Ensures GDI+ is shut down properly and GDI+ handles are destroyed even if the IDE is terminated unexpectedly. Sample included in using GDI+ to load and display a PNG inside of a VB image control and animating a GIF.

8) API caller. This thunk ensures one or more APIs are called after the project unloads, regardless of how it is unloaded. Such a thunk can be used to ensure objects created with GDI or files opened with CreateFile APIs are destroyed/released. This thunk also enables CDecl APIs to be called on demand.

9) VTable caller. Similar to DispCallFunc API, the thunk enables calling methods directly to a VTable-only interface.

10) VTable-only Implementation. A way to have a VB class, form, etc implement a VTable-only interface.



Each thunk type is included in the sample projects. Also included is a sample to allow you to attempt to crash the IDE due to "End" statements. There may be a scenario that includes various levels of usercontrols, but haven't discovered one yet that crashes the IDE due to "End". If you discover a scenario, let me know so I can find a workaround as needed.

Update History
 5 July: one minor patch, one minor enhancement. See post #51
    -- Subclassing thunk did not honor return value when EatMessage returned as True
    -- Hooking thunk now allows global low-level keyboard and mouse hooks
 4 May: Added new thunk, see thunk 10 above. See post #43 for change details.
    -- fixed logic flaw when VTable-caller tasker is passed no parameters
    -- fixed logic flaw in scanning host ordinals that could fail in some scenarios
    -- rewrote host ordinal scanning logic, now 14x faster, no restriction on method counts any longer
    -- error reporting when creating taskers now will include reason for error
    -- additional IDE-safety check made to warn you if "Break on Unhandled Errors" in use
18 Apr: Several changes, primarily for version protection, see post #38
    -- Also patched COM-hook thunk
    -- Thunks now Base64 strings vs array - dramatic compile size difference
12 Apr: Major changes, see post #36 below for details
 5 Apr: updated documentation only. Resolved mislabeling of functions as properties/subs and vice versa
 3 Apr 2020: modified thunks to accept optional TLB VTable-only interfaces. Minor tweaks.
 3 Sep: re-upload. Failed to parse thunk 4-byte blocks with values of zero. Needed to include 
         them as it is possible their memory can be allocated with junk if not overwritten.
 1 Sep: Complete revision
    - uses private heap vs VirtualAlloc for thunks, overall memory reduction
    - includes two new thunks: APICaller and VTableCaller thunks
    - documentation completely redone, each thunk has its own readme file
    - thunks can be optionally used with wrappers for early-bound, intellisense capability
22 Jun: Enhancements
    - replaced COM hook thunk with another that can hook multiple objects vs. just one
    - added a new thunk to help make GDI+ IDE-safe and added a sample of its usage
    - by request, increased default max method count expected in any VB code page to 700
    - other minor enhancements/changes made
 3 Jun: Bug reported when compiled. Fixed (see post #4)
 2 Jun 2019: initial release

Note: In the future, changes only to the clsThunks class and not the sample project, will have the class attached separately. Will not repost the test project unless I change something in it. If the class is a separate attachment, it should be used in place of the one in the test project.

Latest changes have these zips updated: clsThunks.zip, documentation.zip. No others were updated.