Results 1 to 8 of 8

Thread: How to debug vb6 DLL in Visual Studio

  1. #1

    Thread Starter
    Lively Member
    Join Date
    Feb 2020
    Posts
    70

    How to debug vb6 DLL in Visual Studio

    I tried debugging VB6 DLL in Visual Studio 2017 using different methods.

    Initially, attempted by enabling native Edit and Continue in options as mentioned in the link below
    https://stackoverflow.com/questions/...al-studio-2012

    Also attempted debugging by selecting “Start external program” in debugging window then entering the vb6.exe file path and the full .vbp file in the “command-line argument” , as discussed here:
    https://stackoverflow.com/questions/...et-application

    Also tried the link below , but was not successful.
    https://learn.microsoft.com/en-us/pr...ectedfrom=MSDN

    The control was not getting shifted to the vb6 dll even if there was a breakpoint added in vb6 code. Hence I am not able to debug the vb6 dll from Visual Studio. Please note that the code was working fine . I am having issues while debug only.

    How can I debug a VB6 dll in Visual Studio 2017?

  2. #2
    Wall Poster TysonLPrice's Avatar
    Join Date
    Sep 2002
    Location
    Columbus, Ohio
    Posts
    3,969

    Re: How to debug vb6 DLL in Visual Studio

    Quote Originally Posted by IT_Researcher View Post
    I tried debugging VB6 DLL in Visual Studio 2017 using different methods.

    Initially, attempted by enabling native Edit and Continue in options as mentioned in the link below
    https://stackoverflow.com/questions/...al-studio-2012

    Also attempted debugging by selecting “Start external program” in debugging window then entering the vb6.exe file path and the full .vbp file in the “command-line argument” , as discussed here:
    https://stackoverflow.com/questions/...et-application

    Also tried the link below , but was not successful.
    https://learn.microsoft.com/en-us/pr...ectedfrom=MSDN

    The control was not getting shifted to the vb6 dll even if there was a breakpoint added in vb6 code. Hence I am not able to debug the vb6 dll from Visual Studio. Please note that the code was working fine . I am having issues while debug only.

    How can I debug a VB6 dll in Visual Studio 2017?
    A fellow worker was having that issue and I sent him my settings. He said it worked for him:
    Attached Images Attached Images  
    Please remember next time...elections matter!

  3. #3
    PowerPoster wqweto's Avatar
    Join Date
    May 2011
    Location
    Sofia, Bulgaria
    Posts
    6,167

    Re: How to debug vb6 DLL in Visual Studio

    Quote Originally Posted by IT_Researcher View Post
    I tried debugging VB6 DLL in Visual Studio 2017 using different methods.
    You can debug only OS processes (not DLLs). You need to "host" your DLL in a process. You have to instument an Std-EXE project in VB6 which calls into methods of instances from the DLL. You can create such host EXE in VS (C++ or C#) too but it's more error prone.

    Once you have this test bed project you just start it (from Explorer) and choose Debug->Attach to Process... menu in VS, open VB6 source with File->Open..., put breakpoints in VB6 sources and then click buttons/menus on the debugged process UI which hit breakpoint in methods you want to debug. Later when you finish testing you can save OPT file (or similar) with your debug session settings.

    Quote Originally Posted by IT_Researcher View Post
    The control was not getting shifted to the vb6 dll even if there was a breakpoint added in vb6 code. Hence I am not able to debug the vb6 dll from Visual Studio. Please note that the code was working fine . I am having issues while debug only.
    To be able to trace VB6 source code of the DLL you'll need the DLL being compiled with symbolic debug info. This info is stored in a separate PDB file next to the DLL file. There is a checkbox in VB6 project properties on the Compile tab named "Create Symbolic Debug Info" which should be set.

    Quote Originally Posted by IT_Researcher View Post
    How can I debug a VB6 dll in Visual Studio 2017?
    You can't actually :-))

    It is VS2015 the last version which can parse PDB debug info as created by VB6 ancient linker.

    cheers,
    </wqw>

  4. #4

    Thread Starter
    Lively Member
    Join Date
    Feb 2020
    Posts
    70

    Re: How to debug vb6 DLL in Visual Studio

    Quote Originally Posted by wqweto View Post
    You can debug only OS processes (not DLLs). You need to "host" your DLL in a process. You have to instument an Std-EXE project in VB6 which calls into methods of instances from the DLL. You can create such host EXE in VS (C++ or C#) too but it's more error prone.

    Once you have this test bed project you just start it (from Explorer) and choose Debug->Attach to Process... menu in VS, open VB6 source with File->Open..., put breakpoints in VB6 sources and then click buttons/menus on the debugged process UI which hit breakpoint in methods you want to debug. Later when you finish testing you can save OPT file (or similar) with your debug session settings.


    To be able to trace VB6 source code of the DLL you'll need the DLL being compiled with symbolic debug info. This info is stored in a separate PDB file next to the DLL file. There is a checkbox in VB6 project properties on the Compile tab named "Create Symbolic Debug Info" which should be set.


    You can't actually :-))

    It is VS2015 the last version which can parse PDB debug info as created by VB6 ancient linker.

    cheers,
    </wqw>
    Based on the above reply, I have attempted a trial, but did not work. I am unable to debug the VB6 DLL. I have detailed my attempts below. Please review them and let me know if any changes are required in the steps, or why it is not working.



    Trial 1:
    DLL and Exe both made in VB6
    Registered the DLL
    Called a Dll Function from Form in VB6 exe
    vb6 exe project opened in vb6 IDE and executed . Breakpoint kept in dll function call.
    opened a new .net application in VS 2010 PRO, attached the vb6exe process to dotnet application
    VB6 dll project file Opened And BreakPoints set.
    VB6 dll File could not be Debugged.

  5. #5
    PowerPoster wqweto's Avatar
    Join Date
    May 2011
    Location
    Sofia, Bulgaria
    Posts
    6,167

    Re: How to debug vb6 DLL in Visual Studio

    That's possible but too complicated. I already told you you can debug only OS processes (not DLLs) so it's possible to debug VB6.EXE in this way, it acting like a host process for your DLL.

    - compile Ax-DLL with symbolic info (it's important to have .PDB file for each .DLL file you wish to debug in native debugger)
    - if Ax-DLL already compiled then just register it
    - open Std-EXE project in VB6 IDE (without adding the Ax-DLL to a project group or anything)
    - start project in VB6 IDE and execute functionality so that the Ax-DLL is loaded in VB6.exe process
    - start VS2010 and attach to running VB6.exe process (start debugging VB6 IDE)
    - in VS2010 load a source file (File->Open) of the Ax-DLL
    - in VS2010 put a breakpoint in a procedure in this source file
    - in running Std-EXE project execute functionality (click menu) for which this procedure in Ax-DLL is called

    It's much easier to use Std-EXE as host without debugging VB6 IDE like this

    - compile Ax-DLL
    - compile Std-EXE
    - exit VB6 IDE
    - start Std-EXE binary
    - start VS2010 and attach Std-EXE binary for debug
    - load Ax-DLL source file and put breakpoints
    - click something on Std-EXE menus so that a breakpoint is hit

    cheers,
    </wqw>

  6. #6

    Thread Starter
    Lively Member
    Join Date
    Feb 2020
    Posts
    70

    Re: How to debug vb6 DLL in Visual Studio

    Quote Originally Posted by wqweto View Post
    That's possible but too complicated. I already told you you can debug only OS processes (not DLLs) so it's possible to debug VB6.EXE in this way, it acting like a host process for your DLL.

    - compile Ax-DLL with symbolic info (it's important to have .PDB file for each .DLL file you wish to debug in native debugger)
    - if Ax-DLL already compiled then just register it
    - open Std-EXE project in VB6 IDE (without adding the Ax-DLL to a project group or anything)
    - start project in VB6 IDE and execute functionality so that the Ax-DLL is loaded in VB6.exe process
    - start VS2010 and attach to running VB6.exe process (start debugging VB6 IDE)
    - in VS2010 load a source file (File->Open) of the Ax-DLL
    - in VS2010 put a breakpoint in a procedure in this source file
    - in running Std-EXE project execute functionality (click menu) for which this procedure in Ax-DLL is called

    It's much easier to use Std-EXE as host without debugging VB6 IDE like this

    - compile Ax-DLL
    - compile Std-EXE
    - exit VB6 IDE
    - start Std-EXE binary
    - start VS2010 and attach Std-EXE binary for debug
    - load Ax-DLL source file and put breakpoints
    - click something on Std-EXE menus so that a breakpoint is hit

    cheers,
    </wqw>
    Thank you for the reply; it works for VB6 exe calling VB6 DLL but ,I want to debug a VB6 DLL that is being called in a .NET exe.
    Also when I debug VB6 DLL that is being called by VB6 exe , unable to get the values when I hover over the function or variable. How can I achieve this?

  7. #7
    PowerPoster wqweto's Avatar
    Join Date
    May 2011
    Location
    Sofia, Bulgaria
    Posts
    6,167

    Re: How to debug vb6 DLL in Visual Studio

    > How can I achieve this?

    Not possible for variables of VARIANT or BSTR data-type, only for basic types which C/C++ debugger can understand like Long, Integer and Boolean and C/C++ arrays of these (but not SAFEARRAYS i.e. dynamic arrays in VBx).

    > I want to debug a VB6 DLL that is being called in a .NET exe

    Btw, I cannot see why native debugger would distinguish .Net executables as anything but a standard OS process which can be interrupted and debugged the same way.

    cheers,
    </wqw>

  8. #8
    King of sapila
    Join Date
    Oct 2006
    Location
    Greece
    Posts
    6,763

    Re: How to debug vb6 DLL in Visual Studio

    I'm not certain about this but that how it works with C++ dll's but would be turning the target framework to x64 and using dllimport with some modification on p.e. string to intptr or whatever vb6 uses could be a starting point?
    ἄνδρα μοι ἔννεπε, μοῦσα, πολύτροπον, ὃς μάλα πολλὰ
    πλάγχθη, ἐπεὶ Τροίης ἱερὸν πτολίεθρον ἔπερσεν·

Tags for this Thread

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width