Results 1 to 15 of 15

Thread: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

  1. #1

    Thread Starter
    New Member
    Join Date
    Sep 2022
    Posts
    9

    VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    I'm maintaining an old system written in VB6, which has two activex 'server' components which can be run on a different machine.

    Currently, it uses this call prior to attempting to create the main class of the activeX server:

    Dim lngHr As Long
    Dim lngAuthn As Long
    lngAuthn = RPC_C_AUTHN_DEFAULT
    blnSecurityCalled = True
    ' lngHr = CoInitializeSecurity(ByVal API_NULL, -1, _
    ' lngAuthn, ByVal API_NULL, _
    ' RPC_C_AUTHN_LEVEL_NONE, RPC_C_IMP_LEVEL_ANONYMOUS, _
    ' API_NULL, EOAC_NONE, ByVal API_NULL)


    The global DCOM settings on the server are typically Connect and Identity, the two exposed classes for the servers themselves are set to None.

    Does anyone have any thoughts on how to fix this, so that it will work with the requirement of using PACKET_INTEGRITY? I tried switching the RPC_C_AUTHN_LEVEL_NONE to RPC_C_AUTHN_PKT_INTEGRITY, but the activation chain breaks down .. IE, I get an "Object requred" error when VB6 runs the "CreateObject()" call.

    Any insights on this would be greatly appreciated, as I don't understand the depths of DCOM, and VB6 doesn't expose that level of integration very well.

    Thanks in advance.

  2. #2
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    I've fought with this stuff back in the late 90's and can "feel your frustration".

    There's a whole lot of things you have to consider (registry- and setting-wise) -
    on both the Client- and Serverside, to make things work even in a "nicely controllable, local test-setup".

    And then the nightmare starts again, when you want to deploy these solutions to a customers server-environment.

    That's the main-reason I've developed a completely regfree working RPC-implementation (for both, Client- and Server).
    Here's a link to a CodeBank-article: https://www.vbforums.com/showthread....ement-per-RC5)

    Not sure, whether your 2 serverside ActiveX-COMponents are ActiveX-Exes or ActiveX-Dlls (registered in COM+)
    (the approach in the above Link prefers Public Classes in normal VB6-compiled ActiveX-Dlls as its "worker-endpoints").

    Also not sure, whether your solution currently requires the 2 Server-Endpoints "to talk among each other as well" -
    (in addition to "answering requests from the clients").
    In case they don't - you should be able to move your "Server-Classes" into one (or two) ActiveX-Dlls,
    recompile and after that implement the Clientside "Private Stub-Delegator-Classes" as described in the article.

    Sorry for not being "directly helpful" regarding your problem with RPC_C_AUTHN_PKT_INTEGRITY -
    but I suspect that this topic will not get a huge amount of responses, so you have at least an alternative,
    which is "known to work".

    Another one would be, to use the IIS as the Application-Service -
    which then calls into the COM-Methods of your serverside ActiveX-Dlls.
    (the Client-Requests wrapped into http-requests with the help of the WinHttp 5.1. Client-COMponent).
    There's another article here: https://www.vbforums.com/showthread....g-of-http-RPCs

    Olaf
    Last edited by Schmidt; Sep 29th, 2022 at 03:53 PM.

  3. #3
    PowerPoster wqweto's Avatar
    Join Date
    May 2011
    Location
    Sofia, Bulgaria
    Posts
    5,120

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Last time I fiddled with CoInitializeSecurity API it failed constantly in VB6 as there is no way to call this after CoInitialize but before any interface is marshaled in the process.

    https://thrysoee.dk/InsideCOM+/ch18d.htm

    You could probably use static config i.e. registry or Components Manager to achieve the same.

  4. #4

    Thread Starter
    New Member
    Join Date
    Sep 2022
    Posts
    9

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Quote Originally Posted by Schmidt View Post
    I've fought with this stuff back in the late 90's and can "feel your frustration".

    There's a whole lot of things you have to consider (registry- and setting-wise) -
    on both the Client- and Serverside, to make things work even in a "nicely controllable, local test-setup".

    And then the nightmare starts again, when you want to deploy these solutions to a customers server-environment.

    That's the main-reason I've developed a completely regfree working RPC-implementation (for both, Client- and Server).
    Here's a link to a CodeBank-article: https://www.vbforums.com/showthread....ement-per-RC5)

    Not sure, whether your 2 serverside ActiveX-COMponents are ActiveX-Exes or ActiveX-Dlls (registered in COM+)
    (the approach in the above Link prefers Public Classes in normal VB6-compiled ActiveX-Dlls as its "worker-endpoints").

    Also not sure, whether your solution currently requires the 2 Server-Endpoints "to talk among each other as well" -
    (in addition to "answering requests from the clients").
    In case they don't - you should be able to move your "Server-Classes" into one (or two) ActiveX-Dlls,
    recompile and after that implement the Clientside "Private Stub-Delegator-Classes" as described in the article.

    Sorry for not being "directly helpful" regarding your problem with RPC_C_AUTHN_PKT_INTEGRITY -
    but I suspect that this topic will not get a huge amount of responses, so you have at least an alternative,
    which is "known to work".

    Another one would be, to use the IIS as the Application-Service -
    which then calls into the COM-Methods of your serverside ActiveX-Dlls.
    (the Client-Requests wrapped into http-requests with the help of the WinHttp 5.1. Client-COMponent).
    There's another article here: https://www.vbforums.com/showthread....g-of-http-RPCs

    Olaf
    Thanks for the quick response. There is a client exe which calls into two different ActiveX exes. The client relies on a couple of callbacks to complete the connection, which complicates things a little. The client does use Coinitialize, but based on my research, that can't be done in an activex exe due to the way marshalling is handled by VB in the background. The two activex exes have minimal interaction with each other, but there is a link between them in one case.

    Currently, DCOM is configured on the server-side to use Connect & Identify globally, and the two ActiveX exe's classes are set to authentication "none". They run as an Admin or admin-equivalent on the machine they're registered on.

    Anonymous is granted launch & activation permission on the server and client.

    Pretty standard setup, and has worked untouched for years.

    Just trying to figure out IF and how to get around the new requirement, when the tool used to maintain the application is getting in the way. Unfortunately, no way to re-write in a different language at this time, there are thousands of lines of code, and I'm the only person maintaining them.

  5. #5

    Thread Starter
    New Member
    Join Date
    Sep 2022
    Posts
    9

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Thanks for the info. It's messy, finicky and a bit unpredictable.. Which makes any sort of minor tweak/adjustment a royal pain to figure out. The fact that there's very little diagnostic data to look at when things fail, is also not helpful. It doesn't really tell you why the call failed, only that it did *sigh*. This product has been in the field since 2000, and I wasn't part of the initial design team, nor do I have formal training in DCOM from other languages (not that it would help much here).

  6. #6

    Thread Starter
    New Member
    Join Date
    Sep 2022
    Posts
    9

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Yeah, I know it's a huge long shot. MS has been trying to 'kill' off VB6 for a while, and I know it's likely not widely used right now, but we have a major customer having issues, so I'm trying to figure out some kind of remediation. That framework looks interesting. Does it handle out of process activex exes? I wasn't able to verify that at a quick glance. We have two, one which is primarily geared towards the client-side DB calls, and another which handles the posting of data to the database from connected hardware. The trickiest piece probably is the callback interface which is used to transfer data from the server -> client.

  7. #7
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    If you don't want to "move around" Source-Classes from the AX-Exes (into Dlls),
    then you could certainly leave the Ax-Exe-Binaries as they are -
    and would then just have to implement their interfaces
    in additional Classes - and re-delegate from the implementing Class with small "Proxy-Stubs".

    The point in doing that (at the serverside) would be, that the existing Ax-Exes could then be used "machine-local"
    (instantiated via "normal CreateObject"-calls, not passing any IPs in the second Param).

    It's the "Cross-Machine"-calls you want to replace with "something else" (though still following the interface of the Ax-Exe-Classes).

    Your callback-stuff is indeed tricky, because the RC5/RC6 RPC implementations are "one way"...
    There is a clear separation into a Client(Request) part, and a Server(listener/worker) part, which "only answers".

    Whats used in modern implementations for such a (callback-push) scenario,
    are WebServer-instances which handle normal requests via normal http-protocol -
    and the "complicated Server-push-callbacks" via WebSockets.
    (e.g. NodeJs can be set up this way quite nicely, with only a few short JSON-config- and JS-code-snippets).

    You'd have to describe your scenario (the flows, especially when they cross machine-boundaries) in more detail,
    for us to give a decent recommendation, how to (re)implement which part anew.

    Perhaps invest "a day or two" into studying the COM+ (GUI-)Tools
    (as posted by wqweto) - to setup the behaviour of your Ax-Exes, to match your needs).

    Maybe you have success in doing so, without having to re-implement anything.

    Olaf

  8. #8

    Thread Starter
    New Member
    Join Date
    Sep 2022
    Posts
    9

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Quote Originally Posted by Schmidt View Post
    I've fought with this stuff back in the late 90's and can "feel your frustration".

    There's a whole lot of things you have to consider (registry- and setting-wise) -
    on both the Client- and Serverside, to make things work even in a "nicely controllable, local test-setup".

    And then the nightmare starts again, when you want to deploy these solutions to a customers server-environment.

    That's the main-reason I've developed a completely regfree working RPC-implementation (for both, Client- and Server).
    Here's a link to a CodeBank-article: https://www.vbforums.com/showthread....ement-per-RC5)

    Not sure, whether your 2 serverside ActiveX-COMponents are ActiveX-Exes or ActiveX-Dlls (registered in COM+)
    (the approach in the above Link prefers Public Classes in normal VB6-compiled ActiveX-Dlls as its "worker-endpoints").

    Also not sure, whether your solution currently requires the 2 Server-Endpoints "to talk among each other as well" -
    (in addition to "answering requests from the clients").
    In case they don't - you should be able to move your "Server-Classes" into one (or two) ActiveX-Dlls,
    recompile and after that implement the Clientside "Private Stub-Delegator-Classes" as described in the article.

    Sorry for not being "directly helpful" regarding your problem with RPC_C_AUTHN_PKT_INTEGRITY -
    but I suspect that this topic will not get a huge amount of responses, so you have at least an alternative,
    which is "known to work".

    Another one would be, to use the IIS as the Application-Service -
    which then calls into the COM-Methods of your serverside ActiveX-Dlls.
    (the Client-Requests wrapped into http-requests with the help of the WinHttp 5.1. Client-COMponent).
    There's another article here: https://www.vbforums.com/showthread....g-of-http-RPCs

    Olaf
    I'm unfortunately not certain a conversion to activex dll will work in our case. There are currently cases where one activex.exe must call a method in the other activex.exe. Is it possible to use classes in activex exes with that framework?

    This is also unfortunately a fairly complex application. There's one main exe which runs & loads several activex DLLs that handle the major functional modules which can make calls to the activex servers. Curious whether the 'stubs' for calls to the RPC servers in your framework would be required in all the subordinate dlls. Also whether the way this is implemented.. One public class which has functions to create and expose others to the client, would be a good fit for that approach. Extensive use of ADO recordsets, as it uses a SQL Server database.

    I'd appreciate your thoughts, and thanks for your time.

  9. #9
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Let's assume you have a ServerHost, with the HostName "SH".

    Only on the HostMachine "SH" you have:
    - registered ActiveX-Exe A.exe (with a Public Class in it, named cA)
    - registered ActiveX-Exe B.exe (with a Public Class in it, named cB)

    Since A- and B-Exes are both registered on SH, they can talk with each other already "machine-locally"
    (via normal CreateObject- or New-instantiations).

    E.g. when cA (in A.exe) has a: Public Function GetRs(ByVal SQL As String) As Recordset
    You can call that method from inside B.exe (cross-process, but machine-local) via:
    Dim Rs As Recordset
    Set Rs = CreateObject("A.cA").GetRs("Select GetDate()")
    Debug.Print Rs(0).Value

    ---------------------------------------------
    Then you have a Client-Machine, named "CM" and a Client-App CM.exe.
    Only from that Client-Machine you need to make "cross-machine-RPCs".

    Variant1 (using a properly set up IIS-Service on SH):
    - add a Private proxy-class, cpA.cls into your CM-StdExe-Project
    - cpA will have a Method with the same Signature as your A.exes cA.GetRs
    Public Function GetRs(ByVal SQL As String) As Recordset
    Set GetRs = DoHttpRPC(ParamsToRs(SQL, TimeOutSeconds)) 'DoHttpRPC is using WinHttp 5.1 under the covers
    End Function

    The IIS (on SH) only need a few Lines of VBScript in an *.asp-File,
    to answer the HTTP-RPC from Client "CM" with a (serialized) ADO-Rs.

    Set ParamRs = ByteArrayToRs(Request.BinaryRead)
    SQL = ParamRs(0).Value

    Set ResultRs = CreateObject("A.cA").GetRs(SQL)
    Response.BinaryWrite RsToByteArray(ResultRs)

    And that's it for the IIS-scenario (regarding Lines of code in an *.asp VBScript)
    (the red-colored helper-functions are contained in the IIS-Demo-Link I've posted already).
    ------------------------------------------------------

    Variant2 (using RPC-HelperClasses of the RC5 or RC6-Framework) wil look similar to the IIS-Setup:
    - but running your own RPC_AppServer.exe (instead of the IIS-Service)
    - using ActiveX-Dlls (with Classes), then doing basically the same thing as the IIS- *.asp-Scripts at the serverside
    .. (not much code in these Classes, only delegations to your already registered ActiveX-Exes)

    The Clientside will use the same Private Class in CM.exe, only that the inside of the Proxy-Method will look like:
    Public Function GetRs(ByVal SQL As String) As Recordset
    Set GetRs = oRPCConn.RPC(ServerDllName, ServerClsName, "GetRs", TimeOutSeconds, SQL)
    End Function
    ------------------------------------------------------

    Since coding-efforts (for the wrapper- and proxy-classes) in both variants is nearly the same -
    I'd suggest Variant1 with the IIS-setup, because it is the more modern approach:
    - which operates on the firewall-friendly "no issues ports" 80 and 443 (which will "go through" anywhere)
    - can do more than just "playing AppServer for Desktop-Clients" (enables real Browser-Apps, if needed)
    - would allow you to "move this stuff" also into the Cloud later on
    - and saves headaches with proper "Service-configuration"...
    (the IIS is - after installation - already configured to run as a proper Windows-Service).

    HTH

    Olaf

  10. #10

    Thread Starter
    New Member
    Join Date
    Sep 2022
    Posts
    9

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Quote Originally Posted by Schmidt View Post
    Let's assume you have a ServerHost, with the HostName "SH".

    Only on the HostMachine "SH" you have:
    - registered ActiveX-Exe A.exe (with a Public Class in it, named cA)
    - registered ActiveX-Exe B.exe (with a Public Class in it, named cB)

    Since A- and B-Exes are both registered on SH, they can talk with each other already "machine-locally"
    (via normal CreateObject- or New-instantiations).

    E.g. when cA (in A.exe) has a: Public Function GetRs(ByVal SQL As String) As Recordset
    You can call that method from inside B.exe (cross-process, but machine-local) via:
    Dim Rs As Recordset
    Set Rs = CreateObject("A.cA").GetRs("Select GetDate()")
    Debug.Print Rs(0).Value

    ---------------------------------------------
    Then you have a Client-Machine, named "CM" and a Client-App CM.exe.
    Only from that Client-Machine you need to make "cross-machine-RPCs".

    Variant1 (using a properly set up IIS-Service on SH):
    - add a Private proxy-class, cpA.cls into your CM-StdExe-Project
    - cpA will have a Method with the same Signature as your A.exes cA.GetRs
    Public Function GetRs(ByVal SQL As String) As Recordset
    Set GetRs = DoHttpRPC(ParamsToRs(SQL, TimeOutSeconds)) 'DoHttpRPC is using WinHttp 5.1 under the covers
    End Function

    The IIS (on SH) only need a few Lines of VBScript in an *.asp-File,
    to answer the HTTP-RPC from Client "CM" with a (serialized) ADO-Rs.

    Set ParamRs = ByteArrayToRs(Request.BinaryRead)
    SQL = ParamRs(0).Value

    Set ResultRs = CreateObject("A.cA").GetRs(SQL)
    Response.BinaryWrite RsToByteArray(ResultRs)

    And that's it for the IIS-scenario (regarding Lines of code in an *.asp VBScript)
    (the red-colored helper-functions are contained in the IIS-Demo-Link I've posted already).
    ------------------------------------------------------

    Variant2 (using RPC-HelperClasses of the RC5 or RC6-Framework) wil look similar to the IIS-Setup:
    - but running your own RPC_AppServer.exe (instead of the IIS-Service)
    - using ActiveX-Dlls (with Classes), then doing basically the same thing as the IIS- *.asp-Scripts at the serverside
    .. (not much code in these Classes, only delegations to your already registered ActiveX-Exes)

    The Clientside will use the same Private Class in CM.exe, only that the inside of the Proxy-Method will look like:
    Public Function GetRs(ByVal SQL As String) As Recordset
    Set GetRs = oRPCConn.RPC(ServerDllName, ServerClsName, "GetRs", TimeOutSeconds, SQL)
    End Function
    ------------------------------------------------------

    Since coding-efforts (for the wrapper- and proxy-classes) in both variants is nearly the same -
    I'd suggest Variant1 with the IIS-setup, because it is the more modern approach:
    - which operates on the firewall-friendly "no issues ports" 80 and 443 (which will "go through" anywhere)
    - can do more than just "playing AppServer for Desktop-Clients" (enables real Browser-Apps, if needed)
    - would allow you to "move this stuff" also into the Cloud later on
    - and saves headaches with proper "Service-configuration"...
    (the IIS is - after installation - already configured to run as a proper Windows-Service).

    HTH

    Olaf
    Thanks again for your quick and informative responses. I'll need to think a bit before that sinks in. I've not had great luck with attempting to use IIS in the past. Maybe it's improved since, I don't know. The less factors involved during setup, the better in my past experience, as some of the customers that use our product barely know where the power switch is for their machine.

  11. #11
    PowerPoster
    Join Date
    Jul 2010
    Location
    NYC
    Posts
    5,651

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Quote Originally Posted by wqweto View Post
    Last time I fiddled with CoInitializeSecurity API it failed constantly in VB6 as there is no way to call this after CoInitialize but before any interface is marshaled in the process.
    If you did call CoInitialize/CoInitializeSecurity before anything else... would a 2nd call to CoInitialize mess things up? Just a random thought about overriding the default entry point, calling it, then calling the regular entry point. Off in driver land MS has a product lock in scheme going where they do a bunch of crap in a secret undocumented overridden entry point before calling the original one. Or if the 2nd call would ruin things, perhaps it could be patched to jump over it (or even just CoInitSec patched in, if you're doing that).

  12. #12

    Thread Starter
    New Member
    Join Date
    Sep 2022
    Posts
    9

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Quote Originally Posted by fafalone View Post
    If you did call CoInitialize/CoInitializeSecurity before anything else... would a 2nd call to CoInitialize mess things up? Just a random thought about overriding the default entry point, calling it, then calling the regular entry point. Off in driver land MS has a product lock in scheme going where they do a bunch of crap in a secret undocumented overridden entry point before calling the original one. Or if the 2nd call would ruin things, perhaps it could be patched to jump over it (or even just CoInitSec patched in, if you're doing that).
    The basic issue is that VB6 does some stuff in the runtime before the RPC call is made by the application. The main application which creates classes & executes calls to the RPC server uses CoInitialize. The problem, is that attempting to change that from "None" & "Anonymous" to "Packet Integrity" breaks it completely. I can see the calls to the remote server being made through WireShark, but the server is forcing security down to connect, even though I have set global permissions in DCOM to "Packet Integrity" and "Identify". The information I can find states you cannot execute Coinitialize in an Activex EXE, due to the marshalling in the vb6 runtime that happens prior to that call.

    The error(s) posted in Event Viewer simply state that it cannot create the object or RPC server doesn't exist. Nothing terribly useful from a trouble-shooting standpoint. CoInitialize cannot run in debug, so there's no way to track anything further in debug.

    It's tough finding any sort of information on VB6 these days to be honest. MS has basically abandoned support "It just runs".. But not when they pull stuff like this.

    I have no clue how to find nor override entry points. If you have any guidance there, it would be appreciated.

  13. #13
    PowerPoster
    Join Date
    Jul 2010
    Location
    NYC
    Posts
    5,651

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    I was just thinking based on wqweto's post that if calling CoInit/CoInitSec *before* VB did anything at all without the packet integrity flag/marshalling things would solve the problem, that could theoretically be done. But from what you've said it doesn't sound like that would help (and I definitely don't know enough to tell you if it would)... so I don't know if this would be a rabbit hole worth going down.

    If you could call those before absolutely anything else whatsoever occurred (and those APIs would have to be declared in a TLB because you won't even have the runtime to call APIs defined normally), and you think that would resolve the issue (again, zero clue here if it would), only then would it be worth considering.

  14. #14

    Thread Starter
    New Member
    Join Date
    Sep 2022
    Posts
    9

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Quote Originally Posted by TNoyce View Post
    Thanks again for your quick and informative responses. I'll need to think a bit before that sinks in. I've not had great luck with attempting to use IIS in the past. Maybe it's improved since, I don't know. The less factors involved during setup, the better in my past experience, as some of the customers that use our product barely know where the power switch is for their machine.
    Olaf,

    I'm not 100% certain what I'm doing incorrectly. I've created two proxy dlls which have class and function stubs. I created a quick & dirty exe which starts the RPC server. I've confirmed I have a listening port on the 'server' pc with the correct number. Everything runs fine if I have the project files in a group, and the RPC server by itself. However, when I attempt to deploy the files to a remote client, it fails. It's a bit tough to nail down, since I don't have VB installed on the client system, nor would I wish to in this case.

    Just to be certain I am not missing something:
    On the client system
    Main EXE for client application and support dlls (this app has a few..)
    the new ActiveX dlls which contain the stubs I previously mentioned
    The RPCServer.ini file with the correct port & ip address configured
    The three support dlls (RichClient5, DirectCOM, and cairo)
    Richclient5 is registered, as are the activex dlls.

    When the client on the other PC attempts to create an instance of a class in the remote DLL which is registered on the server, all I see is a "Cannot create object" error.

    I was also curious if there's a way to have a class persist, rather than be created & destroyed with each function or method call through your RPC framework.

    Thanks in advance.

    -Tim

  15. #15
    PowerPoster wqweto's Avatar
    Join Date
    May 2011
    Location
    Sofia, Bulgaria
    Posts
    5,120

    Re: VB6, Activex servers, Coinitialize and the upcoming PACKET_INTEGRITY enforcement

    Couple of toughts:

    1. Simplify your test projects to bare minimum both server and client (don't compile original prodcution ones)
    2. Test/exercise your deployment on VMs with exact Windows versions as failing target machine

    cheers,
    </wqw>

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width