Switching between Window UI and Console UI-VBForums
Results 1 to 18 of 18

Thread: Switching between Window UI and Console UI

  1. #1

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Switching between Window UI and Console UI

    Is there a way to determine if a program is running in a Window environment or in a Console environment?

    Or perhaps I should ask if there is a way to know how a program is launched, either from a DOS prompt, or from a Windows program?

    Thanks.

  2. #2
    PowerPoster
    Join Date
    Feb 2006
    Posts
    17,714

  3. #3

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    Thanks dilettante - the link you provided is right in line with what I was searching for.

    It's not common, but at times I will open a Dos Command Window to do some work, and once done I return to the normal Window GUI.

    In doing this I have realized that I have a handful of utility programs written in support of a GUI, but could actually make use of the same programs while working at a command line interface. My first thought was to simply create two programs with two different interfaces, but I never like having duplicate functions/programs which will then both have to be maintained and supported. That's when I started to think about one program able to support two different UI's.

    Obviously this isn't for programs that have extensive interaction between screen displays and user input. It's only for relatively simply utilities like a clean-disk type of program.

    Thanks.

  4. #4
    PowerPoster
    Join Date
    Feb 2006
    Posts
    17,714

    Re: Switching between Window UI and Console UI

    Put most of the logic into a DLL Project. Then you can make a Console UI program and a Windows UI program, both making use of the larger DLL.

  5. #5
    Addicted Member
    Join Date
    Jun 2015
    Posts
    140

    Re: Switching between Window UI and Console UI

    the msblog they did it by checking to see if the program was started with command line options. I like to drag and drop files onto my exes to start them with a specific file or register them as shell handlers in the registry which would trip up this mechanism.

    You can do it by looking at the process parent id and then using that to get the parent exe name.
    If the parent process is cmd.exe you were launched from a console.

    The following example uses the open source proclib library

    https://github.com/dzzie/libs/blob/m...rocessInfo.cls

    Code:
    Private Declare Function GetCurrentProcessId Lib "kernel32" () As Long
    
    Private Sub Form_Load()
        
        Dim ppid As Long, myPid As Long
        Dim proc As New proc_lib.CProcessLib
        Dim cp As CProcess
        
        myPid = GetCurrentProcessId()
        Debug.Print "myPid=" & myPid
        
        If Not proc.GetProcess(myPid, cp) Then
            Debug.Print "Failed to get process"
            Exit Sub
        End If
        
        ppid = cp.ParentPID
        Debug.Print "parent Pid=" & ppid
        
        If Not proc.GetProcess(ppid, cp) Then
            Debug.Print "Failed to get parent process"
            Exit Sub
        End If
        
        MsgBox "Parent process name: " & cp.Path
            
    End Sub
    Last edited by dz32; Nov 11th, 2017 at 07:18 AM.

  6. #6

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    dilettante - The DLL approach is exactly the way I've done it in the past, and while it works, I'm left with two programs for performing one job. So I figured that if it wasn't too difficult to create a single program that could run in both 'modes' it would be one less thing to keep track of.

    dz32 - Thanks for the example code, I have a question about checking against the name 'cmd.exe', Has Microsoft used the same name throughout all of the Windows versions? And what name comes back when checking a programs parent when the system is booted in DOS mode? - I would have to check that one.

    I haven't tried it yet, but I was thinking about getting the Windows Desktop handle and checking that against my programs parent handle. If the handles match, then my program was started from Explorer when a user clicked on the Exe, or from the Start-Run prompt, or from even a shortcut.

    If my program is started from a DOSBOX the handles shouldn't match. I'm not quite sure what would happen with calling GetDesktopWindow when a System is Booted in DOS mode. I'm assuming it would fail in which case the handles would not match and all would be OK.

    The only time I can think of when this would fail is if some program were to launch my program with a Shell function and not passing in the Desktop handle to the call, which I don't see happening.

    I was hopping for some obscure / hidden API that might already exist out there to detect the current UI, but that's not looking very hopeful right now.

    Thanks

  7. #7
    Addicted Member
    Join Date
    Jun 2015
    Posts
    140

    Re: Switching between Window UI and Console UI

    cmd.exe should be stable back to win2k and forward to win10, win98 might be different

    there are alternative shells like cygwin that would twist up this detection too
    you will have to figure out what environments you want to support.

    Maybe you can check if the stdout file handle is writable to see if a console was allocated.

    I do not believe a normal vb6 app will run if the box was booted into dos mode.
    There is a flag pe header-> optional header -> subsystem which probably requires be set to console
    for DOS mode. If I needed tools to run with the system booted into dos/recovery mode I would do it in C or similar.

    cant comment on the other thoughts, requires testing for all the environments you wish to support.
    Good luck

  8. #8

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    I can't ever see a time where I would need to support a booted DOS mode, I was simply thinking that if the call to GetDesktopWindow were to fail under such a situation, it would still properly fall back and assume a UI environment. Whether the code would actually run or not, I doubt it, I can't foresee ever having a situation where it would be needed.

  9. #9

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    Update: I found a couple of things relating to how a program is launched, one is an External reference to a variable that can be accessed from a C program. Being external, I don't know 'who' sets its initial value? If it comes from some C library function which is linked in during a build, then even if it could be accessed in VB, there's noway to know if it would be set and usable in a VB program.

    I don't know if its of any use to anybody, but I thought I'd include it if anybody's interested.


    Another method to determine the application type is to use the semi-documented extern "C" int __app_type variable. It's value is 1 for console applications and 2 for GUI applications. This method will not work for DLL modules.

    I'm thinking about slapping together a C DLL that's callable from VB to see if it would return an accurate value. However the same problem might exist where it would all depend on what critter actually sets this variable. If it's part of a C programs initialization process, then even a DLL might not work.

    The site where the reference above came from goes on talking about what Dz suggested in checking IO handles to determine how a program was launched.

    The most promising was an API call GetConsoleWindow() which is supposed to fail when there's no console window, and succeeds when one is found! The only problem is that I can't get it work!
    Last edited by stuck-n-past; Nov 11th, 2017 at 08:01 PM.

  10. #10
    Fanatic Member
    Join Date
    Aug 2013
    Posts
    795

    Re: Switching between Window UI and Console UI

    @stuck - unfortunately, I don't think that int _app_type check will help. From your subsequent comments, I guess you discovered that that's just a helper function provided by the VS compiler so that proper flags can be sent to the C runtime.

    As dz32 suggests, checking the parent process for "cmd.exe" is a better bet. You don't need a 3rd-party library to do this (although the APIs are a wee bit involved). As a "quick and dirty" test, you can also grab this value from WMI, looks like:

    https://stackoverflow.com/questions/...533287#2533287
    Check out PhotoDemon, a pro-grade photo editor written completely in VB6. (Full source available at GitHub.)

  11. #11

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    Hey Tanner, thanks for the post, Yea I've learned a lot as I've been looking into this, and I think most of what's out there seems to be miss-represented.

    From everything I've tried and tested, the checks or validations to determine the run-mode only looks to be reflecting how an executable was compiled / linked. And from further investigation it looks as most only indicate the run mode based on whether the API AttachConsole has been called, which defeats the whole idea.

    My plan of a quick check to see if my programs parent handle matched the desktops handle turned more involved once I learned that getting the parent of a console program isn't as straight forward as I thought it would be.

    dz - I checked out the 3rd party code from your post, but I found that there are additional dependencies required to support the CProcess class, and wasn't sure what all it would take to resolve, so I didn't go very far with it.

  12. #12
    Addicted Member
    Join Date
    Jun 2015
    Posts
    140

    Re: Switching between Window UI and Console UI

    yeah the processlib class has a mountain of process related code, I only used the lib for simplicity and you could extract what you needed. Here are the relevant bits as a self contained project. I only extracted, you can boil this down more
    Attached Files Attached Files
    Last edited by dz32; Nov 13th, 2017 at 07:38 PM.

  13. #13

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    Hey dz thanks for the zip file, I'll check it out.

    In checking for a parent being cmd.exe or explorer.exe (Desktop), is there any correlation between parent and child Windows / Forms vs. parent and child PID's? Or does every parent/child window have to belong to the same thread or process.
    Last edited by stuck-n-past; Nov 13th, 2017 at 03:03 PM.

  14. #14
    PowerPoster
    Join Date
    Feb 2006
    Posts
    17,714

    Re: Switching between Window UI and Console UI

    The parent process can be anything. It might be Task Scheduler, IIS, Telnet client, or it might be a program of your own.

  15. #15

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    Well as usual, I'm twisted around with nomenclature and alike with ProcessID, ThreadID, and Handles.

    I'm a bit confused about what GetWindowThreadProcessId is returning. Passing it the handle from a VB Form, I get a PID of VB6.exe which I would expect, but then for the ThreadID which is supposed to be who created the Form, I get an ID that's not displayed in the TaskManager or has already terminated.

    In all of VB's management code, is it possible that some initialization code is creating the Form and then ending?

  16. #16

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    Hello, I have things working between Windows GUI and DOS with some oddities.

    Everything on the Windows side appears to be fine, and when running the same program in DOS, the program runs, it displays the output properly, but the program doesn't appear to ever end?

    I can hit return and the DOS prompt will echo back as if all is well, just as if the program had ended properly, but then checking with the task manager, it shows that my program is still running and chewing up the processor at 100%

    When using the Main subroutine as the startup method in VB, are there any special requirements for terminating the program?
    Last edited by stuck-n-past; Nov 15th, 2017 at 12:26 AM.

  17. #17
    Fanatic Member
    Join Date
    Aug 2013
    Posts
    795

    Re: Switching between Window UI and Console UI

    Congrats on getting things working!

    Quote Originally Posted by stuck-n-past View Post
    When using the Main subroutine as the startup method in VB, are there any special requirements for terminating the program?
    Nope. Of course, when Main() exits, VB still has to perform some of its own cleanup before shutting down the app's message pump. I don't know if this interferes with various console workarounds as I haven't tried it. (Maybe someone else has...?)

    As a test, you could always try ExitProcess or TerminateProcess to see if it changes anything. Just note all the usual caveats about potential leaks, etc.

    Barring that, I'd probably investigate Karl Peterson's console project here to see how he does it.
    Check out PhotoDemon, a pro-grade photo editor written completely in VB6. (Full source available at GitHub.)

  18. #18

    Thread Starter
    Fanatic Member
    Join Date
    Nov 2014
    Posts
    523

    Re: Switching between Window UI and Console UI

    Tanner, thanks for the link to the console code, I'll take a look at it.

    I did add a call to ExitProcess so that I can set the DOS errorlevel in case my program is run from a batch file. Typically I end a VB program through the Form_Unload event, but that's not the case for a console program so I also added the dreaded 'End' statement to try and finish things cleanly when in 'DOS' mode.

    With the program using the Sub-Main as a starting point and then branching off to either a Windows GUI or a DOS Command line UI, I'm not completely sure if I'm starting things properly.

    In a normal VB project the main form is automatically loaded, but in this project I'm loading it myself. I first just made use of the Load command, but quickly learned that this also required a call to the 'Show' command placed in the Form_Load event. So I switched to using the API ShowWindow, but I'm still not sure if that's the proper way when loading a primary Form of a VB project manually, or if I'm leaving something out.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Featured


Click Here to Expand Forum to Full Width

Survey posted by VBForums.