FYI: IDE Options via Code

This was a curiosity thing and thought I throw out there so others can improve upon it and/or fill in the gaps (missing stuff). Not really sure how helpful this could be.

The code uses an undocumented but exported vb6a.dll function. This means it likely only exists where vb6.exe exists. The function name is EbGetOptions and has a similar EbSetOptions.

In the comments, you'll see some of the gaps noted. This also doesn't get every option, notably all those checkbox items in Docking & Environment tabs of the Options dialog.

Declarations

Code:

---

Private Declare Function EbGetOptions Lib "vba6.dll" (ByVal dontKnow As Long, ByRef pbBuffer As Any) As Long
Private Declare Function EbSetOptions Lib "vba6.dll" (ByVal dontKnow As Long, ByRef pbBuffer As Any) As Long

Private Type TextCategoriesStruct
    NormalText As Long
    SelectionText As Long
    SyntaxError As Long
    ExecutionPoint As Long
    BreakPoint As Long
    Comment As Long
    KeyWord As Long
    Identifier As Long
    Bookmark As Long
    CallReturn As Long
End Type

Private Type VbIdeOptionsStruct            ' 56 dwords, 224 bytes
    nBits As Long
        ' mask auto-indent                  0x0001
        ' mask break on all errors          0x0002
        ' mask auto-syntax                  0x0004
        ' mask option explicit              0x0008
        ' ??                                0x0010
        ' mask break in classes            0x0020
        ' mask break unhandled errors      0x0022 = 0x00 or 0x22 (none/both break all + in classes)
        ' mask procedure separator          0x0040
        ' mask compile on demand            0x0080
        ' mask default to full mode view    0x0100
        ' mask background compile          0x0200
        ' ??                                0x0400
        ' mask margin indicator bar        0x0800
    nTabWCx As Long                        ' tab width
    nFontSize As Long                      ' font point size (whole number only)
    pFontName As String                    ' BSTR wide-char format
    unk1 As Long
    nForeColors As TextCategoriesStruct
    reserved1(0 To 5) As Long              ' ?? reserved for expansion ??
    nBackColors As TextCategoriesStruct
    reserved2(0 To 5) As Long              ' ?? reserved for expansion ??
    nIndicatorColors As TextCategoriesStruct
    reserved3(0 To 5) As Long              ' ?? reserved for expansion ??
    unk2 As Long
    LCID As Long
    nBits2 As Long
        ' mask drag/drop text editing      0x0001
        ' mask auto-list members            0x0002
        ' mask auto-quick info              0x0004
        ' mask auto-data tips              0x0008
End Type

---

Sample call. GetOptions always returns zero, maybe a sub vs. function. SetOptions set eax before exiting, but unsure of its meaning: 0 = S_OK?

Code:

---

    Dim udt As VbIdeOptionsStruct
    EbGetOptions 0, ByVal VarPtr(udt)  ' use VarPtr so VB doesn't do Unicode>ANSI convesion

---

Play with EbSetOptions at your own risk. I did not try to decipher where the settings are cached: registry, ini, elsewhere. Maybe other's know. If unsure, don't mess with setting options, or at least back stuff up.

If anyone wants to pursue this, maybe for some addin, here are a couple lessons learned...

1. The font name cannot be set to null if using EbSetOptions - crash.

2. The value -1 in a text color = Auto. Looks like forecolor auto is black and the other 2 colors auto are white.

3. Those 3 reserved#() blocks of 6 longs have the value -1 for most of them, but not all, on my IDE. Don't know if those are used internally by VB

4. Those two unk# members use is, well, unknown. I was thinking that it might be the checked items and/or some of the other option button selections that I couldn't find in the structure. But checking/unchecking & requerying IDE options was fruitless. Maybe those "missing" options, i.e., gaps, need to be discovered by another API call?

5. Setting the error breaking option during runtime may not be a good thing to do. After setting it, then requerying its value, it isn't as expected. This may be because VB allows you to toggle that mode at runtime (context menu > "Toggle"). The settings may be cached then Xor'd or Or'd with the original settings producing unexpected results?

6. The 1st API parameter seems to not be used. When I looked at the decompiled ASM, I couldn't see anywhere that it was referenced. But I am not an ASM guru either & could have missed it. The API call triggers 100s of ASM instructions and dozens of nested calls to subroutines.

7. The passed 224 byte structure's content when retrieving options doesn't matter. The 1st action that EbGetOption does with it is filling the udt with zeroes.

And all said and done, would API behavior change, or 1st param be required, if using that call on an install that had no service pack or less than SP6? That would be interesting to know, again out of curiosity for me, but for others that actually may find a need for this code.

update: I visually scanned the ASM for the original vba6.dll that comes with VB installs. It doesn't appear to use that 1st parameter either.

Every time I see the word "unk", I think of the scene in The Good, The Bad, and The Ugly where "The Ugly" finds the tombstone and tries to pronounce the name on it. I'll leave it at that. :)

EDIT: Here it is. It's about 45 seconds in.