-
Nov 17th, 2020, 11:40 AM
#1
[RESOLVED] VB6 command-line executable sending "result" code back to caller
I'm calling a VB6 program (from command line with an argument) with Shell from the VBA.
I'd like to return a single boolean (or even just a bit) back to the VBA caller letting it know whether or not the call was successful.
What would people recommend as the easiest way to return this "result" code? I'd really rather not waste the time with writing out some file, and I'd rather not use subclassing in the VBA. I just need some shared memory I can set from VB6 and then check once that WaitForTermination breaks loose in the VBA.
Just as an FYI, I intended to use something like the following (along with "Shell") to call the VB6 program from the VBA.
Code:
Public Sub WaitForTermination(lProcessIdFromShell As Long)
' This procedure should be used with some caution.
' The program will not be able to refresh its forms and controls
' while waiting. The best way to use this is to make sure that
' the program has nothing visible before calling this procedure.
'
' The following command is the best way to use this procedure:
' WaitForTermination Shell( sTheShellCommandString )
Const SYNCHRONIZE As Long = &H100000
Const INFINITE As Long = &HFFFF
Dim lProcessHwnd As Long
'
If lProcessIdFromShell = 0 Then Exit Sub ' Make sure there is a process to wait on.
lProcessHwnd = OpenProcess(SYNCHRONIZE, 0, lProcessIdFromShell)
If lProcessHwnd = 0 Then Exit Sub ' Make sure we can open the process.
WaitForSingleObject lProcessHwnd, INFINITE ' Wait for process to terminate.
CloseHandle lProcessHwnd
End Sub
Thanks,
Elroy
EDIT: In fact, it'd be really nice if that WaitForTermination could just return this Boolean (or Long).
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Nov 17th, 2020, 11:46 AM
#2
Re: VB6 command-line executable sending "result" code back to caller
-
Nov 17th, 2020, 12:30 PM
#3
Re: VB6 command-line executable sending "result" code back to caller
OptionBase1, that looks perfect.
The call to ExitProcess paired with a call to GetExitCodeProcess looks like exactly what I was after. I'm trying to finish something else up first, but I'm assuming the return from Shell is the same PID that goes into GetExitCodeProcess? I haven't tested it, but I threw this together thinking it's what I'm after:
Code:
Option Explicit
'
Private Declare Function CloseHandle Lib "kernel32" (ByVal hObject As Long) As Long
Private Declare Function OpenProcess Lib "kernel32" (ByVal dwDesiredAccess As Long, ByVal bInheritHandle As Long, ByVal dwProcessId As Long) As Long
Private Declare Function WaitForSingleObject Lib "kernel32" (ByVal hHandle As Long, ByVal dwMilliseconds As Long) As Long
Private Declare Function GetExitCodeProcess Lib "kernel32" (ByVal hProcess As Long, lpExitCode As Long) As Long
'
' And this one is used by the "Shelled To" program when terminating.
Private Declare Sub ExitProcess Lib "kernel32" (ByVal uExitCode As Long)
Public Function ShellAndWait(sFormattedProgramSpecAndArgs As String) As Long
Const SYNCHRONIZE As Long = &H100000
Const INFINITE As Long = -1&
Dim lProcessHwnd As Long
Dim lProcessIdFromShell As Long
Dim lExitReturn As Long
'
' Shell and save PID.
lProcessIdFromShell = Shell(sFormattedProgramSpecAndArgs)
If lProcessIdFromShell = 0 Then Exit Function ' Make sure there is a process to wait on.
'
' Get handle to PID.
lProcessHwnd = OpenProcess(SYNCHRONIZE, 0, lProcessIdFromShell)
If lProcessHwnd = 0 Then Exit Function ' Make sure we can open the process.
'
' Wait for PID to terminate.
WaitForSingleObject lProcessHwnd, INFINITE ' Wait for process to terminate.
CloseHandle lProcessHwnd
'
' And now fetch the return from ExitProcess.
' Return -999 if GetExitCodeProcess fails.
lExitReturn = GetExitCodeProcess(lProcessIdFromShell, ShellAndWait)
If lExitReturn = 0& Then ShellAndWait = -999&
End Function
CORRECTION: I had lProcessHwnd in the GetExitCodeProcess call, and I'm pretty sure lProcessIdFromShell goes in there.
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Nov 17th, 2020, 12:59 PM
#4
Re: VB6 command-line executable sending "result" code back to caller
Code:
Function GetProcExitCode(ByVal nProcessID As Long) As Long
Dim h As Long
Const PROCESS_QUERY_INFORMATION = 1024
h = OpenProcess(PROCESS_QUERY_INFORMATION, 0, nProcessID)
Call GetExitCodeProcess(h, GetProcExitCode)
Call CloseHandle(h)
End Function
-
Nov 17th, 2020, 01:28 PM
#5
Re: VB6 command-line executable sending "result" code back to caller
Ok, so it IS a handle and not the PID.
I've reworked it, and I believe I've got it now. Still haven't tested though, but will shortly.
Code:
Option Explicit
'
Private Declare Function CloseHandle Lib "kernel32" (ByVal hObject As Long) As Long
Private Declare Function OpenProcess Lib "kernel32" (ByVal dwDesiredAccess As Long, ByVal bInheritHandle As Long, ByVal dwProcessId As Long) As Long
Private Declare Function WaitForSingleObject Lib "kernel32" (ByVal hHandle As Long, ByVal dwMilliseconds As Long) As Long
Private Declare Function GetExitCodeProcess Lib "kernel32" (ByVal hProcess As Long, lpExitCode As Long) As Long
'
' And this one is used by the "Shelled To" program when terminating.
Private Declare Sub ExitProcess Lib "kernel32" (ByVal uExitCode As Long)
Public Function ShellWaitAndReturnExitCode(sFormattedProgramSpecAndArgs As String) As Long
' Returns -999 if there's any failure within this procedure.
'
Const PROCESS_QUERY_INFORMATION As Long = 1024&
Const SYNCHRONIZE As Long = &H100000
Const INFINITE As Long = -1&
Dim lProcessHandle As Long
Dim lProcessIdFromShell As Long
Dim lExitApiReturn As Long
'
' Shell and save PID.
lProcessIdFromShell = Shell(sFormattedProgramSpecAndArgs)
If lProcessIdFromShell = 0& Then ' Make sure there is a process to wait on.
ShellWaitAndReturnExitCode = -999& ' Failure.
Exit Function
End If
'
' Get handle to PID.
lProcessHandle = OpenProcess(SYNCHRONIZE, 0, lProcessIdFromShell)
If lProcessHandle = 0& Then ' Make sure we can open the process.
ShellWaitAndReturnExitCode = -999& ' Failure.
Exit Function
End If
'
' Wait for PID to terminate.
WaitForSingleObject lProcessHandle, INFINITE ' Wait for process to terminate.
CloseHandle lProcessHandle ' Done with this handle.
'
' And now fetch the return from ExitProcess.
lProcessHandle = OpenProcess(PROCESS_QUERY_INFORMATION, 0&, lProcessIdFromShell)
If lProcessHandle = 0& Then ' Make sure we can still access the PID.
ShellWaitAndReturnExitCode = -999& ' Failure.
Exit Function
End If
lExitApiReturn = GetExitCodeProcess(lProcessHandle, ShellWaitAndReturnExitCode)
CloseHandle lProcessHandle ' Done with this handle.
If lExitApiReturn = 0& Then ' Make sure we can still access the process.
ShellWaitAndReturnExitCode = -999&
Exit Function
End If
'
' If we fall through, we're all set and have the exit code.
End Function
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Nov 17th, 2020, 01:32 PM
#6
Re: VB6 command-line executable sending "result" code back to caller
WOW, you've got to be careful with ExitProcess too. When executing from the IDE, it doesn't just terminate the execution ... it terminates the entire IDE.
So, for others using this, that's a caveat to watch out for.
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Nov 17th, 2020, 01:36 PM
#7
Re: VB6 command-line executable sending "result" code back to caller
Originally Posted by Elroy
WOW, you've got to be careful with ExitProcess too. When executing from the IDE, it doesn't just terminate the execution ... it terminates the entire IDE.
So, for others using this, that's a caveat to watch out for.
Yes, perhaps you would like to add a check to an InIDE function before calling it.
-
Nov 17th, 2020, 03:23 PM
#8
Re: VB6 command-line executable sending "result" code back to caller
Originally Posted by Eduardo-
Yes, perhaps you would like to add a check to an InIDE function before calling it.
Code:
Public Sub ExitWithCode(iCode As Long)
If InIDE Then
MsgBox "Exited with code: " & Format$(iCode)
End
Else
ExitProcess iCode
End If
End Sub
Public Function InIDE(Optional ByRef b As Boolean = True) As Boolean
' NEVER specify the Optional b when calling.
If b = True Then Debug.Assert Not InIDE(InIDE) Else b = True
End Function
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Nov 17th, 2020, 04:23 PM
#9
Re: VB6 command-line executable sending "result" code back to caller
Ok, finally got to the testing of fetching the return code. The code above "sort of" worked, but it occasionally failed (unable to get the handle when trying to get the exit code). It was failing about 30% of the time. I think the PID was disappearing from Windows before the handle was fetched.
Therefore, I combined the OpenProcess flags, and don't get a handle twice anymore. Now, I can't seem to make it fail:
Code:
Option Explicit
'
Private Declare Function CloseHandle Lib "kernel32" (ByVal hObject As Long) As Long
Private Declare Function OpenProcess Lib "kernel32" (ByVal dwDesiredAccess As Long, ByVal bInheritHandle As Long, ByVal dwProcessId As Long) As Long
Private Declare Function WaitForSingleObject Lib "kernel32" (ByVal hHandle As Long, ByVal dwMilliseconds As Long) As Long
Private Declare Function GetExitCodeProcess Lib "kernel32" (ByVal hProcess As Long, lpExitCode As Long) As Long
'
Private Function ShellWaitAndReturnExitCode(sFormattedProgramSpecAndArgs As String) As Long
' Returns -999 if there's any failure within this procedure.
'
Const PROCESS_QUERY_INFORMATION As Long = &H400&
Const SYNCHRONIZE As Long = &H100000
Const INFINITE As Long = -1&
Dim lProcessHandle As Long
Dim lProcessIdFromShell As Long
Dim lExitApiReturn As Long
'
' Shell and save PID.
lProcessIdFromShell = Shell(sFormattedProgramSpecAndArgs, vbNormalFocus)
If lProcessIdFromShell = 0& Then ' Make sure there is a process to wait on.
ShellWaitAndReturnExitCode = -999& ' Failure.
Exit Function
End If
'
' Get handle to PID.
lProcessHandle = OpenProcess(SYNCHRONIZE Or PROCESS_QUERY_INFORMATION, 0&, lProcessIdFromShell)
If lProcessHandle = 0& Then ' Make sure we can open the process.
ShellWaitAndReturnExitCode = -999& ' Failure.
Exit Function
End If
'
' Wait for PID to terminate.
WaitForSingleObject lProcessHandle, INFINITE
'
' And now fetch the return from ExitProcess.
lExitApiReturn = GetExitCodeProcess(lProcessHandle, ShellWaitAndReturnExitCode)
'
' Done with the PID's handle.
CloseHandle lProcessHandle
'
' Make sure GetExitCodeProcess worked correctly.
If lExitApiReturn = 0& Then
ShellWaitAndReturnExitCode = -999&
Exit Function
End If
'
' If we fall through, we're all set and have the exit code.
End Function
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Nov 17th, 2020, 07:03 PM
#10
Re: [RESOLVED] VB6 command-line executable sending "result" code back to caller
A simpler way (avoiding all the APIs and potential coding-mistakes) -
would be the usage of a System-COM-Object:
Code:
Public Function RunAndWaitForExitCode(Cmd As String) As Long
RunAndWaitForExitCode = CreateObject("WScript.Shell").Run(Cmd, 0, True)
End Function
Personally I'd communicate from (potential 64Bit-)VBA with a VB6-exe either via Sockets,
or making it an ActiveX.exe - then communicating via a well-defined COM-Interface (which would work even earlybound in the VBA-Client).
HTH
Olaf
-
Nov 17th, 2020, 09:10 PM
#11
Re: [RESOLVED] VB6 command-line executable sending "result" code back to caller
Lots of IPC alternatives, from shared memory to window messages. Easy enough to pass a window handle value as a command-line argument, then the worker process can send a custom message back. Call RegisterWindowMessage() or just use WM_USER/WM_APP.
-
Nov 18th, 2020, 05:49 AM
#12
Re: [RESOLVED] VB6 command-line executable sending "result" code back to caller
Originally Posted by Schmidt
(which would work even earlybound in the VBA-Client).
That would be helpful only for intellisense in the IDE. The performance of an early-bound out-of-process call is more influenced by it being out-of-process than early-bound.
I'm not even sure we can call it early-bound as the vtable call is calling into a proxy object anyway.
@Elroy: Btw, here is an API based shell-and-wait that I have handy here
Code:
Option Explicit
Private Declare Function ShellExecuteEx Lib "shell32" Alias "ShellExecuteExA" (lpExecInfo As SHELLEXECUTEINFO) As Long
Private Declare Function WaitForSingleObject Lib "kernel32" (ByVal hHandle As Long, ByVal dwMilliseconds As Long) As Long
Private Declare Function GetExitCodeProcess Lib "kernel32" (ByVal hProcess As Long, lpExitCode As Long) As Long
Private Declare Function CloseHandle Lib "kernel32" (ByVal hObject As Long) As Long
Private Type SHELLEXECUTEINFO
cbSize As Long
fMask As Long
hWnd As Long
lpVerb As String
lpFile As String
lpParameters As String
lpDirectory As Long
nShow As Long
hInstApp As Long
' optional fields
lpIDList As Long
lpClass As Long
hkeyClass As Long
dwHotKey As Long
hIcon As Long
hProcess As Long
End Type
Public Function ShellWait( _
sFile As String, _
sParameters As String, _
Optional ByVal StartHidden As Boolean, _
Optional Verb As String, _
Optional ExitCode As Long) As Boolean
Const SW_HIDE As Long = 0
Const SW_SHOWDEFAULT As Long = 10
Const SEE_MASK_NOCLOSEPROCESS As Long = &H40
Const SEE_MASK_NOASYNC As Long = &H100
Const SEE_MASK_FLAG_NO_UI As Long = &H400
Const INFINITE As Long = -1
Dim uShell As SHELLEXECUTEINFO
With uShell
.cbSize = Len(uShell)
.fMask = SEE_MASK_NOCLOSEPROCESS Or SEE_MASK_NOASYNC Or SEE_MASK_FLAG_NO_UI
.lpVerb = Verb
.lpFile = sFile
.lpParameters = sParameters
.nShow = IIf(StartHidden, SW_HIDE, SW_SHOWDEFAULT)
End With
If ShellExecuteEx(uShell) <> 0 Then
Call WaitForSingleObject(uShell.hProcess, INFINITE)
Call GetExitCodeProcess(uShell.hProcess, ExitCode)
Call CloseHandle(uShell.hProcess)
'--- success
ShellWait = True
Else
ExitCode = Err.LastDllError
End If
End Function
Can be used to start a process elevated too with Verb:="runas". Can be tweaked to wait until a timeout too.
cheers,
</wqw>
Last edited by wqweto; Nov 18th, 2020 at 05:59 AM.
-
Nov 18th, 2020, 09:13 AM
#13
Re: [RESOLVED] VB6 command-line executable sending "result" code back to caller
Originally Posted by wqweto
That would be helpful only for intellisense in the IDE.
Yep, that's what I was trying to point out primarily.
(COM-interfaces don't really care about the "bitness" of the implementations behind them).
Originally Posted by wqweto
The performance of an early-bound out-of-process call is more influenced by it being out-of-process than early-bound.
That's right - as said, it'd be more about IDE-comfort (inside the VBA environment).
And thus one has to "play by the Out-Of-Process calling-rules", meaning:
- not too many "fine-granular" calls across process-boundaries (e.g. foreign Prop-access within tight loops from inside the caller, here: VBA)
- instead those loops should happen on the side of the callee (within the implementation, here: the VB6-Ax-Exe)
- within appropriately designed "heavier, more universal" methods on the interfaces in question, to keep the call-count low
Olaf
-
Nov 18th, 2020, 10:18 AM
#14
Re: [RESOLVED] VB6 command-line executable sending "result" code back to caller
Ok, it's now all up-and-running (with basically the code I posted in post #9, and #8).
Wqweto, thanks for the enhanced "ShellAndWait" procedure.
And yes, Olaf, you're correct about the "bitness" concerns. The shelling (ShellAndWait) program is MS-Access (within the VBA), and may very well be 64-bit. In fact, the MS-Access I've tested it with (on my machine) is, in fact, 64-bit. And, as we know, VB6 is only-and-always 32-bit. But all that seems to work fine with the way it's implemented.
When putting the code in post #9 into the VBA, I did have to tweak the API declarations, and I also had to change that lProcessHandle variable to LongPtr (rather than just Long). Here are the VBA API declarations for anyone who may want them:
Code:
Private Declare PtrSafe Function OpenProcess Lib "kernel32" (ByVal dwDesiredAccess As Long, ByVal bInheritHandle As Long, ByVal dwProcessId As Long) As LongPtr
Private Declare PtrSafe Function CloseHandle Lib "kernel32" (ByVal hObject As LongPtr) As Long
Private Declare PtrSafe Function WaitForSingleObject Lib "kernel32" (ByVal hHandle As LongPtr, ByVal dwMilliseconds As Long) As Long
Private Declare PtrSafe Function GetExitCodeProcess Lib "kernel32" (ByVal hProcess As LongPtr, lpExitCode As Long) As Long
I didn't convert the ExitProcess API, as it was only used on the VB6 side. However, looking at it, it shouldn't require any conversion beyond just adding the PtrSafe keyword to it.
Thanks To All,
Elroy
EDIT: Also, just to say a word about "alternative" way of doing this. I know that there are almost always multiple ways to "skin the cat". However, it does seem like this ExitProcess & GetExitCodeProcess combination was designed to rather precisely do what I'm after, so why not use them?
Last edited by Elroy; Nov 18th, 2020 at 10:31 AM.
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
-
Nov 18th, 2020, 12:02 PM
#15
Re: [RESOLVED] VB6 command-line executable sending "result" code back to caller
From a comment in OptionBase1's link:
Originally Posted by Microsoft Support
PRB: Call to ExitProcess() from Visual Basic Application Hinders Process Exit
SYMPTOMS
If a Visual Basic application makes a direct call to the ExitProcess() API, the process may not properly exit. In some instances, a call to ExitProcess() will even cause an access violation or cause the process to deadlock.
Calling ExitProcess() from a Visual Basic application is unsupported.
CAUSE
The ExitProcess() API should not be called by a thread when other threads still need to clean up their own resources. If there are other running threads that have not exited, the ExitProcess() routine will abruptly terminate them. This can cause data loss and other unpredictable behavior.
The Visual Basic run-time engine is responsible for the execution of a Visual Basic application. Not only does this engine interpret and execute the code within the application, but it also initializes and cleans up the process. Because the Visual Basic run-time engine allocates resources, only it can know when the resources are released. Thus, only the run-time engine can safely call ExitProcess().
RESOLUTION
The proper way to exit a Visual Basic application is to naturally exit Sub Main or unload the form specified as the "Startup Object" within the application's Project Properties.
MORE INFORMATION
One particular instance in which a call to ExitProcess() is known to cause a problem involves COM objects. If a Visual Basic application has an outstanding reference to an out-of-process COM object when it calls ExitProcess(), the calling process is likely to "hang" or cause an access violation. A direct call to CoUninitialize() immediately before the call to ExitProcess() will usually prevent this problem. Although this approach allows you to work around the problem, it is not recommended or supported by Microsoft.
The only advantage that would arise out of calling ExitProcess() from a Visual Basic application is the ability to set an exit code for the process. But because of the unpredictable nature of calling ExitProcess() from Visual Basic, it is better to communicate the success or failure of the process through some other means, such as writing an exit code to a file or sending a windows message to another process.
I don't know if VBA supports this, but you might want to consider using DDE, which VB6 has built-in support for.
-
Nov 18th, 2020, 12:03 PM
#16
Re: [RESOLVED] VB6 command-line executable sending "result" code back to caller
Originally Posted by Elroy
However, it does seem like this ExitProcess & GetExitCodeProcess combination was designed to rather precisely do what I'm after, so why not use them?
These are perfectly fine and I don't think there is anything else you can use in this regard. It's the OpenProcess that is superfluous. This is like starting a process and then attaching a debugger. Also there is the possibility the process has already exited before your OpenProcess call or that the current user does not have permissions to "open" it's own processes (doubt it but still).
The way to start a process and get it's handle atomicly is by using *CreateProcess* API directly. But ShellExecute wraps CreateProcess and builds on it -- namely allows elevation which is not available with bare *CreateProcess* and it has intresting "verbs" which are completely shell/explorer abstration that NT kernel knows nothing about.
cheers,
</wqw>
-
Nov 18th, 2020, 12:15 PM
#17
Re: [RESOLVED] VB6 command-line executable sending "result" code back to caller
Originally Posted by wqweto
These are perfectly fine and I don't think there is anything else you can use in this regard. It's the OpenProcess that is superfluous. This is like starting a process and then attaching a debugger. Also there is the possibility the process has already exited before your OpenProcess call or that the current user does not have permissions to "open" it's own processes (doubt it but still).
The way to start a process and get it's handle atomicly is by using *CreateProcess* API directly. But ShellExecute wraps CreateProcess and builds on it -- namely allows elevation which is not available with bare *CreateProcess* and it has intresting "verbs" which are completely shell/explorer abstration that NT kernel knows nothing about.
cheers,
</wqw>
Wqweto, thanks a great deal for those clarifications, especially about OpenProcess/CreateProcess. I'll definitely save a copy of your procedure.
However, I just hit the "Send" button on an email which basically "implements" the changes the way I had originally done it. If we have problems, I'll possibly slip in your approach, but I'm going to leave it alone for the time being.
These Covid times really make things rough for coordination. It seems like the days where I'd fly in and go through a series of changes with a team "on the ground" were some distant lifetime ago. I still haven't made up my mind about Zoom (or MS-Teams). I sort of have a love-hate relationship about it all.
But Again, Thanks,
Elroy
Any software I post in these forums written by me is provided "AS IS" without warranty of any kind, expressed or implied, and permission is hereby granted, free of charge and without restriction, to any person obtaining a copy. To all, peace and happiness.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|