[vb6] Fix Palette when IDE Manifested - AddIn

This is just a simple add-in to fix the popup palette in the IDE when the IDE is manifested for common controls, i.e., themed.

Updated 2 Aug 2020. Added ability for elevated IDE to receive dragged files from Windows Explorer. Most of us run the IDE elevated. That prevents us from dragging files from Explorer to our Project Window tree (as of Win8). With a few lines of code, no subclassing, that is now re-enabled. I simply added those lines of code to the add-in. Won't break binary compatibility with previous version. Won't step on some other add-in that may do the same thing. The fix is applied whether or not the IDE is manifested for common controls.

Leandro has a similar project that offers other features. However, this add-in simply restores the palette to how it would look if IDE were not manifested, no modifications. Personally, I just wanted the palette back but didn't want any extra options.

Quickly, how it works. When manifested, extra messages are sent and among them is WM_ERASEBKGND. Since the palette is drawn vs some image control, that message is part of the reason the palette is being erased. The add-in will find the popup palette, subclass it, eat that extra message and trigger a refresh. To prevent sending excessive refresh messages, the subclassing will only perform those tasks when the palette is shown, not the system colors. The entire add-in is just a couple dozen lines of code, so code comments are sparse.

Recommend compiling the add-in into the same folder as your VB executable.

With decades of coding under my belt, this is the first add-in that I've ever created. So, if something is wrong or it doesn't work in a specific scenario, please post the problem.

As you can see from the controls in this screenshot, the IDE is manifested and the palette is restored.

Attachment 176891

If you are going to recompile the add-in, you first need to remove it from the IDE, otherwise, you'll likely get access-denied errors. To do that, close all VB instances except the one you are using for recompiling the add-in. Open the Add-In manager from your IDE menu bar. Select the add-in, ensure all checkboxes at bottom of window are unchecked. Close that manager window & recompile. When done, simply go back and check the boxes you unchecked earlier.

Edited: A few people had problems compiling the add-in DLL. If so, remove binary compatibility requirement as shown in post #18 below. And if that is done, you can delete the ManifestPalFix.binaryCompat file in the zip.

Since the question will likely be asked... Using this add-in does nothing if the IDE is not manifested for common control; other than applying the drag/drop fix noted in first posting.

The first thing the add-in does is test if ComCtl32 dll is loaded into the process. If not, common controls isn't there and no subclassing occurs, no action taken. If it is there, then it checks for a function that is only exported when v6 of ComCtl32 is in the process. If it isn't exported, no subclassing, no action taken.

Here are those initial checks:

Code:

---

    lValue = GetModuleHandle("comctl32.dll")
    If lValue = 0 Then Exit Sub
    If GetProcAddress(lValue, "GetWindowSubclass") = 0 Then Exit Sub

---

Very helpful. Will use this Add-In now all over the place. ;)
Even when it's not often actually needed it just feels better to have this fixed.

Hi La Volpe,
Leandro and you ... two radical solutions in two days !
what else !
Great again

Quote:

---

Originally Posted by darjeeling

Leandro and you ... two radical solutions in two days !
what else !

---

His idea motivated me. Our solutions are completely different though. I use subclassing. He does not and offers other options. Not trying to compete. I just didn't want "options", only wanted the original palette being drawn and am quite happy with the results.

Gotta be honest though. Been using the IDE without the popup palette painting correctly for over a decade now & kinda forgot what it looked like. But in that time, memorized a lot of color values. Guess I can release them to make room for other stuff (Kelly Bundy, Season 8, Ep.26, Married w/Children)

Edited: Wouldn't recommend running the two add-ins together :eek:

very good job!, if I remember correctly it is the same solution that is used with the option buttons in windows xp

Quote:

---

Originally Posted by LeandroA

very good job!, if I remember correctly it is the same solution that is used with the option buttons in windows xp

---

You may have a better memory. I thought it was that the WM_PRINTCLIENT message wasn't being handled? But am not sure any longer.

Quote:

---

Originally Posted by LaVolpe

You may have a better memory. I thought it was that the WM_PRINTCLIENT message wasn't being handled? But am not sure any longer.

---

For manifested frame controls to prevent option and check buttons widget images getting garbled *and* to prevent flicker on mouse hovering I use this in subclassproc

Code:

---

        Case WM_PRINTCLIENT, WM_MOUSELEAVE
            ControlSubclassProc = DefWindowProc(hWnd, wMsg, wParam, lParam)
            Exit Function

---

Edit: Btw, in this line

Code:

---

        lValue = SetWindowSubclass(hWnd, AddressOf WindowProc, hWnd Xor lValue, lValue)

---

the Xor usage is a nice trick but mostly useless.

If you always pass 0 for uIdSubclass there is *no* chance for collision with anyone else's add-in using the same 0 for uIdSubclass because the subclass entries are keyed both on uIdSubclass *and* pfnSubclass so to collide the other add-in has to use the same AddressOf WindowProc which is very unlikely.

Quote:

---

Originally Posted by MSDN

uIdSubclass

Type: UINT_PTR

The subclass ID. This ID together with the subclass procedure uniquely identify a subclass. To remove a subclass, pass the subclass procedure and this value to the RemoveWindowSubclass function. This value is passed to the subclass procedure in the uIdSubclass parameter.

---

cheers,
</wqw>

Quote:

---

Originally Posted by wqweto

Code:

---

        Case WM_PRINTCLIENT, WM_MOUSELEAVE
            ControlSubclassProc = DefWindowProc(hWnd, wMsg, wParam, lParam)
            Exit Function

---

---

Hello. Do you know what is the difference between calling DefWindowProc and not subclassing the messages at all?

Quote:

---

Originally Posted by Eduardo-

Hello. Do you know what is the difference between calling DefWindowProc and not subclassing the messages at all?

---

If the custom subclassproc swallows the messages they will just stop working.For instance WM_PRINTCLIENT is used by AnimateWindow API so it will break for no apparent reason.

Generally you don't want to *not* forward system messages to *next* window proc which you don't explicitly respond to and/or implement. Calling DefWindowProc here instead just skips the VB's window proc which is buggy.

cheers,
</wqw>

Quote:

---

Originally Posted by wqweto

Calling DefWindowProc here instead just skips the VB's window proc which is buggy.

---

Ah, OK, I understand it now. Thank you.

Quote:

---

Originally Posted by wqweto

the Xor usage is a nice trick but mostly useless.

If you always pass 0 for uIdSubclass there is *no* chance for collision with anyone else's add-in using the same 0 for uIdSubclass because the subclass entries are keyed both on uIdSubclass *and* pfnSubclass so to collide the other add-in has to use the same AddressOf WindowProc which is very unlikely.

---

I am fully aware. This is more or less a force of habit. I prefer not use zero as the subclass-ID. I typically use the hWnd itself, or Xor hWnd w/AddressOf, or use the Xor method to stuff a secondary value (static) with the subclass along with dwRefData, depending on the scenario. When I was first building it, I was thinking of using dwRefData for the m_SendPrint value instead of using a variable and using the Xor trick to carry the ListBox hWnd into the subclass procedure vs. externally caching it. But later changed my mind about using dwRefData for the m_SendPrint value.

Now, before anyone says "well in that case, then why do you Xor w/hwnd, it wouldn't be needed - just use the value as-is?", let me say that is true, but its usage is more or less a "signature" I use.

Would be nice if this thread would also be copied (or moved?) to the "UtilityBank - IDE Add-Ins" forum.

Forgive my ignorance but how do I apply this fix? I've downloaded the ZIP, extracted the contents but not sure where to go from there. I loaded the VBP but that doesn't do anything... Help!

Quote:

---

Originally Posted by AAraya

Forgive my ignorance but how do I apply this fix? I've downloaded the ZIP, extracted the contents but not sure where to go from there. I loaded the VBP but that doesn't do anything... Help!

---

OK, I gather I need to compile the DLL first. When I try to do that however I get this error dialog:

Quote:

---

Compile error:
Public declaration does not description of event or procedure having same name.

---

This happens in the Connect Designer AddinInstance_OnConnection Sub

OK, got this working. I had to change the Version Compatibility setting from "Binary Compatibility" to "No Compatibility" to successfully compile the DLL which I then registered using regsvr32. Works great now! Thanks, this palette issue was a minor annoyance to deal with!!!

Quote:

---

Compile error:
Public declaration does not description of event or procedure having same name.

---

Quote:

---

This happens in the Connect Designer AddinInstance_OnConnection Sub

---

Quote:

---

OK, got this working. I had to change the Version Compatibility setting from "Binary Compatibility" to "No Compatibility" to successfully compile the DLL which I then registered using regsvr32. Works great now! Thanks, this palette issue was a minor annoyance to deal with!!!

---

I tried these steps, but for me it didn't resolve to remove the binary compatibility.

Quote:

---

Originally Posted by Episcopal

I tried these steps, but for me it didn't resolve to remove the binary compatibility.

---

Just to be very explicit about what settings I changed...

Project Properties>Component tab>Version Compatibility section - I changed the option from "Binary Compatibility" to "No Compatibility". Save the new Project Properties and then compile the DLL.

Quote:

---

Project Properties>Component tab>Version Compatibility section - I changed the option from "Binary Compatibility" to "No Compatibility". Save the new Project Properties and then compile the DLL.

---

but that's exactly what I did ... rsrsrs

Just rename .binaryCompat to .dll and put in syswow64 folder and register.
It's already pre-compiled.

Hi LaVolpe, happy to come across this topic as that color box has been an annoyance since some time now. Wasn't aware of that it had to do with the IDE being manifested and wasn't even aware that it was as I cannot recall I have done this myself! Suspect it was done by something I installed... and me not paying attention. It may explain why the VB IDE caption bar comes up totally blank when I open some projects, but others not.

Anyway, trying to load your Add-in project (Win10 didn't want to register the renamed .dll file as Krool suggested above) I run in to what to me appears to be a problem. I get a dialog saying "Please wait while Windows installs and configures Visual Studio 2013" ? Yes I bought and installed that once up on a time, but never use it. Is this merely related to the references in your project file and is "safe" to let complete? For now I have canceled those dialogs (3 or 4) as they come up.

EDIT: OK found that vb6.manifest.manifest file and it's possible I put it there and those brain cells simply died by the chock of using a "VBNet Manifest for VB5 IDE", which may explains the VS2013 reaction as well.
EDIT2: Nope, the VS2013 "issue" had noting to do with the manifest file. Renamed it and tried again with same result, also blank IDE caption bar.
So after cancel all the install dialogs I try to compile the dll and geta compile error" "Procedure declaration does not match description of event or procedure having the same name" in the Connect!AddinInstance_OnConnection(...) Sub. Something is off here? Not sure what or why.

Ok, Changing from Binary Compatibility to No Compatibility fixed the compile issue. Obviously the ManifestPalFix.binaryCompat file included differs in interface with current code.

Quote:

---

Originally Posted by Krool

Just rename .binaryCompat to .dll and put in syswow64 folder and register.
It's already pre-compiled.

---

Yes ... I know that ... but for me an error occurred even using the method of post # 18

Wow! after years of not having the palette... thanks.
looks like vb6 is still very alive and well .

Updated to re-enable another lost function... being able to drag files from Explorer to our project's explorer window tree. See comments near top of 1st posting.

edited. Without using the updated add-in, if you want to play with the logic...
-- use at least Win8 which is where I think the drag/drop restrictions began?

1. Run your IDE elevated and start a new test project
2. Try to drag a cls, bas, or frm file from Windows Explorer to your project's Explorer tree
-- the drag/drop icon is correct, but trying to drop the file has no effect
3. Now in that test form, add this code, run project, close form & return to IDE, then repeat the test

Code:

---

Private Declare Function ChangeWindowMessageFilter Lib "user32" (ByVal Message As Long, ByVal dwFlag As Long) As Long
Const WM_DROPFILES As Long = &H233&
Const WM_COPYDATA      As Long = &H4A&
Const WM_COPYGLOBALDATA As Long = &H49&

Private Sub Form_Load()
    ChangeWindowMessageFilter WM_DROPFILES, 1
    ChangeWindowMessageFilter WM_COPYGLOBALDATA, 1
    ChangeWindowMessageFilter WM_COPYDATA, 1
End Sub

---

The above test is for that IDE instance only and only until VB is unloaded.

Now this does not allow your running project to receive dropped files when that project is run from an elevated IDE or run elevated from its compiled exe. In order for that to happen, you'll need to subclass your form/controls that will act as drop targets and handle the WM_DROPFILES message. An example can be found in this thread, starting with post #7

Quote:

---

Originally Posted by LaVolpe

Updated to re-enable another lost function... being able to drag files from Explorer to our project's explorer window tree. See comments near top of 1st posting.

---

I was used to drag files from Explorer and already changed my habit to right-click load..
However, it's nice now being able to drag files again in an elevated IDE on Windows 10. :)

Thanks for that. And good to integrate this into this add-in as "side-effect fix".

Quote:

---

And good to integrate this into this add-in as "side-effect fix".

---

Unsure if you were making a suggestion? :confused:

Quote:

---

Originally Posted by LaVolpe

Unsure if you were making a suggestion? :confused:

---

No suggestion. Just wanted to say it was a good idea to include it in this add-in and not release another stand-alone add-in.

Ah, gotcha. Like you, I was right clicking on the tree, adding files as needed, for the past several years. It's only been a week or so and I'm already "re-learning" the easier drag/drop method. Doesn't take long to forget that adjusted, forced, behavior ;)

Quote:

---

Originally Posted by LaVolpe

Been using the IDE without the popup palette painting correctly for over a decade now & kinda forgot what it looked like. But in that time, memorized a lot of color values. Guess I can release them to make room for other stuff (Kelly Bundy, Season 8, Ep.26, Married w/Children)

---

HAHA - I had to look it up. The Bundy Bounce. Awesome. Thanks for that.:bigyello:

Thank you LaVolpe. This has resolved a long standing problem for me.

Cheers