Results 1 to 28 of 28

Thread: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

  1. #1

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    This demonstrates, how one can implement robust, asynchronous Workers in VB6 -
    by using a (cross-process) SharedMemory-approach (which is one of the common IPC-mechanisms).

    For convenient usage, the more complex stuff (the MemoryMapping- as well as the CreateProcess-APIs),
    are encapsulated in two generic "Drop-In"-Classes (cIPCMaster and cIPCWorker).

    Those Classes can be used either "split-up" (in separate VB6-Projects for Master and Worker) -
    but also "all-in-one" (when the same Project - or Process - is used to act as Master and Worker both).

    The latter case (using an "all-in-one"-Project), is the most convenient for developing/testing.
    Here the Worker-Process is "shelled" against the same ExeName as the Main- (or Master-) Project,
    using a simple "forking" in Sub Main() ... (as shown below from the content of Demo1 modMain.bas):
    Code:
    Option Explicit
    
    Sub Main() 'Process-Startup-Forking
      If Len(Trim$(Command$)) = 0 Then 'no Cmd-Arg was passed (it was started as the Master-Process)
        fMaster.Show '... so we simply startup the Main-Form
        
      Else 'it was started as a WorkerProcess (pass the CommandLine-Arg into the WorkerRoutine)
        EnterWorkerLoop New cIPCWorker, Trim$(Command$)
      End If
    End Sub
    Above, the blue-colored call represents the "master-fork-path" to your normal "GUI-Project-Code"
    (entered when no Commandline-Param was passed to your Executable).

    And as the magenta-colored routine-name (in the "worker-fork-path") suggests, the upstarting worker is not fired up
    like in an old "classic WebServer-CGI-call" (where the Process exits, after only a single Job was performed).

    Instead the current mechanism is implemented in a way, that the WorkerProcess is fired up once -
    and then enters an "IDLE-loop" (waiting for Jobs, provided by the Master-Process later).
    This way one of the disadvantages of MultiProcessing (the higher Startup-Costs, compared to MultiThreading) is avoided.

    The advantages of doing MultiProcessing instead of threading are:
    - no typelibs, no extra-Dlls are needed
    - in the IDE (after compiling the same Project), the asynchronous workers will behave the same way as in the compiled binary
    - a hard terminate of a worker is possible in a stable and "residue-free" manner (though graceful termination-support is of course built-in)
    - the communication between Master and Worker(s) happens in an absolute "non-blocking" way

    To explain the last point above a bit more... "non-blocking" means, that neither Post- or SendMessage-calls are involved
    (as in VB6-OleBased-Communications between "threaded Apartments", where Events, raised from the Workers will block the Main-Thread) -
    nor are there other mechanisms in play like Mutexes or CriticalSections, which are normally used in conjunction with shared memory...

    Instead the Demo shows, how "state-machine-based" communication (using the shared mem-area) can be implemented.

    The approach is extremely robust, completely IDE- and crash-safe, and "cleans up after itself" under any circumstances:
    - upstarted Worker-Processes will automatically close, when the Master-Class goes out of scope
    - you can even use the IDE-stop-button, whilst asynchronous workers are "deep within a Job" (worker-processes will autoclose nevertheless)

    There is also not a single thing, which is "forbidden to use" in the workers (like in many of the threading-approaches for VB6)...

    The Zip below comes with 3 Demo-Folders (a simple one to get up to speed - and two "medium-difficult" ones for MandelBrot-rendering).

    Ok, here is, what the MandelBrot-Demos will produce (using two Workers, independent from the Main-App):


    And here's the Demo-Zip: IPCSharedMem.zip

    Have fun...

    Olaf

  2. #2
    Frenzied Member
    Join Date
    Dec 2012
    Posts
    1,470

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Very interesting. I'm sure I can find a use for it. Thanks for sharing.

    J.A. Coutts

  3. #3
    Hyperactive Member
    Join Date
    Jan 2018
    Posts
    264

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    This looks really useful. I could never get named pipes working for IPC, and TCP is just so much overhead when you only need simple communication between processes.

  4. #4
    Fanatic Member
    Join Date
    Aug 2016
    Posts
    673

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by Schmidt View Post
    This demonstrates, how one can implement robust, asynchronous Workers in VB6 -
    by using a (cross-process) SharedMemory-approach (which is one of the common IPC-mechanisms).

    For convenient usage, the more complex stuff (the MemoryMapping- as well as the CreateProcess-APIs),
    are encapsulated in two generic "Drop-In"-Classes (cIPCMaster and cIPCWorker).

    Those Classes can be used either "split-up" (in separate VB6-Projects for Master and Worker) -
    but also "all-in-one" (when the same Project - or Process - is used to act as Master and Worker both).

    The latter case (using an "all-in-one"-Project), is the most convenient for developing/testing.
    Here the Worker-Process is "shelled" against the same ExeName as the Main- (or Master-) Project,
    using a simple "forking" in Sub Main() ... (as shown below from the content of Demo1 modMain.bas):
    Code:
    Option Explicit
    
    Sub Main() 'Process-Startup-Forking
      If Len(Trim$(Command$)) = 0 Then 'no Cmd-Arg was passed (it was started as the Master-Process)
        fMaster.Show '... so we simply startup the Main-Form
        
      Else 'it was started as a WorkerProcess (pass the CommandLine-Arg into the WorkerRoutine)
        EnterWorkerLoop New cIPCWorker, Trim$(Command$)
      End If
    End Sub
    Above, the blue-colored call represents the "master-fork-path" to your normal "GUI-Project-Code"
    (entered when no Commandline-Param was passed to your Executable).

    And as the magenta-colored routine-name (in the "worker-fork-path") suggests, the upstarting worker is not fired up
    like in an old "classic WebServer-CGI-call" (where the Process exits, after only a single Job was performed).

    Instead the current mechanism is implemented in a way, that the WorkerProcess is fired up once -
    and then enters an "IDLE-loop" (waiting for Jobs, provided by the Master-Process later).
    This way one of the disadvantages of MultiProcessing (the higher Startup-Costs, compared to MultiThreading) is avoided.

    The advantages of doing MultiProcessing instead of threading are:
    - no typelibs, no extra-Dlls are needed
    - in the IDE (after compiling the same Project), the asynchronous workers will behave the same way as in the compiled binary
    - a hard terminate of a worker is possible in a stable and "residue-free" manner (though graceful termination-support is of course built-in)
    - the communication between Master and Worker(s) happens in an absolute "non-blocking" way

    To explain the last point above a bit more... "non-blocking" means, that neither Post- or SendMessage-calls are involved
    (as in VB6-OleBased-Communications between "threaded Apartments", where Events, raised from the Workers will block the Main-Thread) -
    nor are there other mechanisms in play like Mutexes or CriticalSections, which are normally used in conjunction with shared memory...

    Instead the Demo shows, how "state-machine-based" communication (using the shared mem-area) can be implemented.

    The approach is extremely robust, completely IDE- and crash-safe, and "cleans up after itself" under any circumstances:
    - upstarted Worker-Processes will automatically close, when the Master-Class goes out of scope
    - you can even use the IDE-stop-button, whilst asynchronous workers are "deep within a Job" (worker-processes will autoclose nevertheless)

    There is also not a single thing, which is "forbidden to use" in the workers (like in many of the threading-approaches for VB6)...

    The Zip below comes with 3 Demo-Folders (a simple one to get up to speed - and two "medium-difficult" ones for MandelBrot-rendering).

    Ok, here is, what the MandelBrot-Demos will produce (using two Workers, independent from the Main-App):


    And here's the Demo-Zip: IPCSharedMem.zip

    Have fun...

    Olaf
    in demo

    jobPrepared 'set from the Master-Instance
    Private Sub cmdStart_Click()
    M.JobState = jobPrepared '' if JobState= jobPrepared ,in sub "M_JobStateChanged" JobState =processing why ≠ Prepared ? Is it ignored automatically?
    End Sub
    Private Sub M_JobStateChanged(ByVal JobState As eJobState)
    cmdCancel.Enabled = (JobState = jobProcessing)
    cmdStart.Enabled = Not cmdCancel.Enabled
    Caption = M.JobStateFriendlyName ----------------------
    End Sub
    Last edited by xxdoc123; Jun 1st, 2018 at 02:21 AM.

  5. #5

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Yes, apparently the StateSwitch (from jobPrepared to jobProcessing) happens so fast, that the JobStateChanged-Event skips it.

    Well, and besides - it is of no interest for the Master-Process (who initiated that State-Switch in the first place).

    Instead the Master-Process is interested in those State-Switches, which were triggered inside the Worker:
    - jobProcessing (which the Worker switches to, after the Master has signaled it "New Input-Parameters" via jobPrepared
    - jobFinished (which the Worker switches to from jobProcessing, after successful finishing a given Job)

    HTH

    Olaf

  6. #6
    Frenzied Member
    Join Date
    Dec 2012
    Posts
    1,470

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    When running the program (either the combined or separated programs), one copy appears in the Apps section of the Task Manager, and the other in the Background section. It seems to me that using a separate worker thread would be more efficient from a memory usage perspective. To that end, and to get a better understanding of the internal workings of the program, I converted the Simple Demo into 2 separate programs. I did take a few liberties with the code to make it easier to follow, so I hope you don't mind.

    J.A. Coutts
    Attached Files Attached Files
    Last edited by couttsj; Aug 20th, 2021 at 12:40 PM.

  7. #7
    Fanatic Member
    Join Date
    Aug 2016
    Posts
    673

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by couttsj View Post
    When running the program (either the combined or separated programs), one copy appears in the Apps section of the Task Manager, and the other in the Background section. It seems to me that using a separate worker thread would be more efficient from a memory usage perspective. To that end, and to get a better understanding of the internal workings of the program, I converted the Simple Demo into 2 separate programs. I did take a few liberties with the code to make it easier to follow, so I hope you don't mind.

    J.A. Coutts
    I have a need is a work process to read the server data used winhttp or your http mod, processed and passed to the control main process to show, do not know if the work process will die. I haven't done this yet.

  8. #8
    Fanatic Member
    Join Date
    Aug 2016
    Posts
    673

    Question Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by Schmidt View Post
    This demonstrates, how one can implement robust, asynchronous Workers in VB6 -
    by using a (cross-process) SharedMemory-approach (which is one of the common IPC-mechanisms).

    For convenient usage, the more complex stuff (the MemoryMapping- as well as the CreateProcess-APIs),
    are encapsulated in two generic "Drop-In"-Classes (cIPCMaster and cIPCWorker).

    Those Classes can be used either "split-up" (in separate VB6-Projects for Master and Worker) -
    but also "all-in-one" (when the same Project - or Process - is used to act as Master and Worker both).

    The latter case (using an "all-in-one"-Project), is the most convenient for developing/testing.
    Here the Worker-Process is "shelled" against the same ExeName as the Main- (or Master-) Project,
    using a simple "forking" in Sub Main() ... (as shown below from the content of Demo1 modMain.bas):
    Code:
    Option Explicit
    
    Sub Main() 'Process-Startup-Forking
      If Len(Trim$(Command$)) = 0 Then 'no Cmd-Arg was passed (it was started as the Master-Process)
        fMaster.Show '... so we simply startup the Main-Form
        
      Else 'it was started as a WorkerProcess (pass the CommandLine-Arg into the WorkerRoutine)
        EnterWorkerLoop New cIPCWorker, Trim$(Command$)
      End If
    End Sub
    Above, the blue-colored call represents the "master-fork-path" to your normal "GUI-Project-Code"
    (entered when no Commandline-Param was passed to your Executable).

    And as the magenta-colored routine-name (in the "worker-fork-path") suggests, the upstarting worker is not fired up
    like in an old "classic WebServer-CGI-call" (where the Process exits, after only a single Job was performed).

    Instead the current mechanism is implemented in a way, that the WorkerProcess is fired up once -
    and then enters an "IDLE-loop" (waiting for Jobs, provided by the Master-Process later).
    This way one of the disadvantages of MultiProcessing (the higher Startup-Costs, compared to MultiThreading) is avoided.

    The advantages of doing MultiProcessing instead of threading are:
    - no typelibs, no extra-Dlls are needed
    - in the IDE (after compiling the same Project), the asynchronous workers will behave the same way as in the compiled binary
    - a hard terminate of a worker is possible in a stable and "residue-free" manner (though graceful termination-support is of course built-in)
    - the communication between Master and Worker(s) happens in an absolute "non-blocking" way

    To explain the last point above a bit more... "non-blocking" means, that neither Post- or SendMessage-calls are involved
    (as in VB6-OleBased-Communications between "threaded Apartments", where Events, raised from the Workers will block the Main-Thread) -
    nor are there other mechanisms in play like Mutexes or CriticalSections, which are normally used in conjunction with shared memory...

    Instead the Demo shows, how "state-machine-based" communication (using the shared mem-area) can be implemented.

    The approach is extremely robust, completely IDE- and crash-safe, and "cleans up after itself" under any circumstances:
    - upstarted Worker-Processes will automatically close, when the Master-Class goes out of scope
    - you can even use the IDE-stop-button, whilst asynchronous workers are "deep within a Job" (worker-processes will autoclose nevertheless)

    There is also not a single thing, which is "forbidden to use" in the workers (like in many of the threading-approaches for VB6)...

    The Zip below comes with 3 Demo-Folders (a simple one to get up to speed - and two "medium-difficult" ones for MandelBrot-rendering).

    Ok, here is, what the MandelBrot-Demos will produce (using two Workers, independent from the Main-App):


    And here's the Demo-Zip: IPCSharedMem.zip

    Have fun...

    Olaf
    If I want to pass string data to the shared memory from the main process, how does the working process know the length of the shared string when reading data?


    Public Type RecordType

    str As String * 100 ------Must use a fixed-length string

    End Type

    Now I think of this method to solve this problem

    '====add
    Private Declare Function lstrcpyn _
    Lib "kernel32" _
    Alias "lstrcpynA" (ByVal lpString1 As Any, _
    ByVal lpString2 As Any, _
    ByVal iMaxLength As Long) As Long

    Private Declare Function lstrlen _
    Lib "kernel32" _
    Alias "lstrlenA" (ByVal lpString As Any) As Long
    Private str As String

    Private Sub cmdStart_Click()
    Label2.Caption = ""

    str = "hello:Schmidt" '
    'M.UserDataWrite StrPtr(str), lstrlen(str)
    lstrcpyn M.UserDataPtr, ByVal str, lstrlen(str) + 1
    'MsgBox lstrlen(M.UserDataPtr)
    M.JobState = jobPrepared


    Private Sub DoJob(W As cIPCWorker)

    Dim i As Long

    Dim tmpString As String

    tmpString = Space(lstrlen(W.UserDataPtr))
    lstrcpyn ByVal tmpString, W.UserDataPtr, lstrlen(W.UserDataPtr) + 1

    MsgBox tmpString

    For i = 1 To 100
    W.JobProgress = i / 100
    W.Wait 50 '<- simulate some workload (instead of doing some real stuff)

    If W.MasterWantsToClose Or W.JobState <> wjobProcessing Then Exit Sub 'early exit
    Next
    str = Replace(tmpString, "hello", "thanks")

    lstrcpyn W.UserDataPtr, ByVal str, lstrlen(str) + 1
    W.JobState = wjobFinished 'if the worker-routine reached this point, switch to the next state

    End Sub
    Demo1 Send string.zip

    Do you have other methods?
    Last edited by xxdoc123; Jun 4th, 2018 at 12:38 AM.

  9. #9
    Frenzied Member
    Join Date
    Dec 2012
    Posts
    1,470

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by xxdoc123 View Post
    I have a need is a work process to read the server data used winhttp or your http mod, processed and passed to the control main process to show, do not know if the work process will die. I haven't done this yet.
    As far as I can tell, the worker thread remains in memory as long as the main thread.

    J.A. Coutts

  10. #10

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by couttsj View Post
    As far as I can tell, the worker thread remains in memory as long as the main thread.
    To be precise, the WorkerProcess remains alive (assuming your own Worker-Functions don't cause an unhandled VB6-error-condition) until:
    - the cIPCMaster-Object of the Main-Process calls its .CloseWorkerGracefully(TimeOutSeconds) method
    - or is set to Nothing "roughly" (in which case it performs a Hard-Terminate of the Worker-Process)

    There's also a third condition under which a WokerProcess automatically closes itself:
    - if the Form.hWnd one has passed via InitAndBindWorkerTo(...) dies, the Worker will recognize this also and will automatically close
    .. (this is BTW the mechanism which ensures automatic Worker-closing, when you press the Stop-Button in the IDE)

    Ok, so the only other condition (the fourth one) under which a WorkerProcess can "go out" is the one I mentioned at the top of this posting:
    - you run into an unhandled error-condition in your own Worker-Code
    A cIPCMaster-Instance in your Main-App-Thread can always ask, whether a Worker ran into such a condition (is still alive) via:
    oMaster.IsWorkerActive

    If you poll that IsWorkerActive-Property against a given cIPCMaster-Instance, (e.g. when you manage a few of them in
    a dedicated "Worker-Pool-Manager-Class") - you could easily react to "unexpected closing" of a worker, by removing
    said Master-instance (which only refers to a "dead worker") from your "Worker-Pool-Collection" - readding a new one instead.
    This allows for an easy way to set up a real stable-working "WorkerPool-Manager-Process" (e.g. for a robust Server-Scenario).

    As to your other question:
    It seems to me that using a separate worker thread would be more efficient from a memory usage perspective.
    An upstarted VB-Process (with no GUI) consumes about 1.5MB of memory...
    So, yes - that's a bit more overhead than an upstarting "normal thread" would consume (memory-wise).
    But these days, I'd consider that amount of memory-overhead not really "an issue" (even if you run a pool of 16 Workers or so).

    HTH

    Olaf

  11. #11

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by xxdoc123 View Post
    If I want to pass string data to the shared memory from the main process, how does the working process know the length of the shared string when reading data?

    Now I think of this method to solve this problem...

    Do you have other methods?
    There is no need to apply a (non-unicode-capable) API for that.

    I'd simply:
    - define all Strings of a Shared-Memory-Area as fixed-length-string-members of an UDT
    - then later always read (or write) the entire UDT-content from (or to) the Shared-Area via: UserDataWrite VarPtr(MyUDT), LenB(MyUDT)
    - and for "InBetween" happening changes of the UDTs FixedLength-StringMembers I'd use a simple "Property-Helper" like shown below:

    Code:
    Option Explicit
    
    Private Type tMyUDT
      S As String * 100
    End Type
    
    Private Sub Form_Load()
      Dim MyUDT As tMyUDT
      Debug.Print Len(FLS(MyUDT.S)), FLS(MyUDT.S) 'print the S-member content from a fresh initialized UDT
      
      FLS(MyUDT.S) = "123": Debug.Print Len(FLS(MyUDT.S)), FLS(MyUDT.S) 'print it after assigning "123"
      FLS(MyUDT.S) = "":    Debug.Print Len(FLS(MyUDT.S)), FLS(MyUDT.S) 'print it again, after assigning ""
    End Sub
    
    Private Property Let FLS(FLSmember As String, RHS As String)
      If Len(FLSmember) <= Len(RHS) Then Err.Raise 7
      FLSmember = RHS & vbNullChar
    End Property
    Private Property Get FLS(FLSmember As String) As String
      Dim Pos As Long: Pos = InStr(FLSmember, vbNullChar)
      If Pos > 0 Then FLS = Left$(FLSmember, Pos - 1)
    End Property
    Olaf

  12. #12

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by couttsj View Post
    ... to get a better understanding of the internal workings of the program, I converted the Simple Demo into 2 separate programs.
    I did take a few liberties with the code to make it easier to follow, so I hope you don't mind.
    Just tried to run your Demo1B.zip above... -
    though it seems you forgot to include a Sub Main() triggering of EnterWorkerLoop within your modWorker.bas:
    Code:
    Sub Main()
      If Len(Trim$(Command$)) Then EnterWorkerLoop New cIPCWorker, Trim$(Command$)
    End Sub
    Olaf

  13. #13
    Fanatic Member
    Join Date
    Aug 2016
    Posts
    673

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by Schmidt View Post
    There is no need to apply a (non-unicode-capable) API for that.

    I'd simply:
    - define all Strings of a Shared-Memory-Area as fixed-length-string-members of an UDT
    - then later always read (or write) the entire UDT-content from (or to) the Shared-Area via: UserDataWrite VarPtr(MyUDT), LenB(MyUDT)
    - and for "InBetween" happening changes of the UDTs FixedLength-StringMembers I'd use a simple "Property-Helper" like shown below:

    Code:
    Option Explicit
    
    Private Type tMyUDT
      S As String * 100
    End Type
    
    Private Sub Form_Load()
      Dim MyUDT As tMyUDT
      Debug.Print Len(FLS(MyUDT.S)), FLS(MyUDT.S) 'print the S-member content from a fresh initialized UDT
      
      FLS(MyUDT.S) = "123": Debug.Print Len(FLS(MyUDT.S)), FLS(MyUDT.S) 'print it after assigning "123"
      FLS(MyUDT.S) = "":    Debug.Print Len(FLS(MyUDT.S)), FLS(MyUDT.S) 'print it again, after assigning ""
    End Sub
    
    Private Property Let FLS(FLSmember As String, RHS As String)
      If Len(FLSmember) <= Len(RHS) Then Err.Raise 7
      FLSmember = RHS & vbNullChar
    End Property
    Private Property Get FLS(FLSmember As String) As String
      Dim Pos As Long: Pos = InStr(FLSmember, vbNullChar)
      If Pos > 0 Then FLS = Left$(FLSmember, Pos - 1)
    End Property
    Olaf
    thanks .

    I have a suggestion to add a length shared variable in the two class module. You can directly return the length

    Private Type tShared 'a convenience-UDT, sitting at the top of the shared mem-area
    UserDataLen As Long
    MasterHwnd As Long
    JobProgress As Single
    JobPreparedHPSec As Double
    JobStartedHPSec As Double
    JobFinishedHPSec As Double
    JobState As eJobState
    UserWriteDataLen As Long '
    End Type
    Public Sub UserDataWrite(ByVal pSrc As Long, _
    ByVal ByteLen As Long, _
    Optional ByVal Offs As Long)

    If BaseAddr Then
    RtlMoveMemory ByVal UserDataPtr + Offs, ByVal pSrc, ByteLen
    T.UserWriteDataLen = ByteLen
    WriteInternalType T
    End If
    End Sub
    I can know the length of the currently written data, and if it needs to be read, I can initialize the length of the variable.

  14. #14
    Frenzied Member
    Join Date
    Dec 2012
    Posts
    1,470

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by Schmidt View Post
    Just tried to run your Demo1B.zip above... -
    though it seems you forgot to include a Sub Main() triggering of EnterWorkerLoop within your modWorker.bas:
    Olaf
    You are right! Don't know how I managed that. Probably added it, created the executable, and created the zip without saving the file. Anyway, I added a message to tell me I can't run it in the IDE.
    Code:
    Sub Main()
        If Len(Trim$(Command$)) Then
            EnterWorkerLoop New cIPCWorker, Trim$(Command$)
        Else
            MsgBox "To be able to start a Worker, this Project needs to be compiled first"
        End If
    End Sub
    Still looking for an appropriate application to use this.

    J.A. Coutts

  15. #15

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by couttsj View Post
    Still looking for an appropriate application to use this.
    We use it (at the place where I currently work) for one type of special report (which takes quite long to generate, interacting with MS-Word-XML) -
    and as well for validation of larger amounts of Email-addresses (which involve DNS-queries with potentially longer blocking whilst asking for MX-record-entries)...

    Basically it has the same async use-cases as threading, but the advantage of being more robust -
    due to allowing for "proper-cleanup of single Workers from a WorkerPool" without affecting the "Pool-Hosting-App".

    Also device-communication for time-critical (e.g. streaming-) devices comes to mind (one Worker being responsible for exactly one device).
    Over a decade ago, I've used a quite similar approach to manage the incoming DataStreams from up to 12 Gig-E cameras, which pumped
    their incoming frames with 200Hz each into a larger RingBuffer in each WorkerProcess (on a quite powerful Server-Host, which ran 24/7).

    Olaf

  16. #16
    Fanatic Member
    Join Date
    Sep 2010
    Location
    Italy
    Posts
    678

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Hello
    I thought I could implement MultiProcess (Or multithread) in my PhotoModularFX program.
    Initially I thought about creating a thread for each "Node" / effect, but this way is not good because some very slow effects can create bottlenecks in the processing flow.
    (Before executing the operations of a "node" you have to wait until all its inputs are ready)
    So it is better to create more threads or processes for each node. (although not all nodes / effects are "spatially separable")

    I tried using this proposed way (StdExe-IPC-via-SharedMemory)
    But without success.
    I tried to understand how it worked.
    I can not understand:

    Code:
    Private Function CreateWorkerProcess (MemSizeToReserve) As cIPCMaster
      Set CreateWorkerProcess = New cIPCMaster
      If CreateWorkerProcess.InitAndBindWorkerTo (Me, MemSizeToReserve, ProcessPath) = 0 Then
         MsgBox "To be able to start a Worker", this Project needs to be compiled first "
      End If
    End Function
    - How much must be the memory to reserve?
    - How to pass a 2D Array as input ?

    Basically in my program I have various functions that have as input and output 2D arrays of singles.

    So I have a list of "DoJob" functions which, for simplification, could be represented as follows:

    Code:
    Sub DoJob_Effect1 (sngInput () As Single, sngOutput () As Single)
        Dim X As Long, Y As Long, W As Long, H As Long
        W = UBound (sngInput, 1)
        H = UBound (sngInput, 2)
        ReDim sngOutput (W, H)
        For X = 0 To W
            For Y = 0 To H
                sngOutput (X, Y) = sngInput (X, Y) ^ 2 '' '' some result from sngInput ()
            Next
            'setProgress 100 * X / W
        Next
    End Sub
    The number of input and output 2D arrays can be different:
    For example, 3 inputs and 1 output. (depending on the "effect")

    Code:
    Sub DoJob_Effect3 (sngInput1 () As Single, sngInput2 () As Single, sngInput3 () As Single, _
                      sngOutput () As Single)
        Dim X As Long, Y As Long, W As Long, H As Long
        W = UBound (sngInput1, 1)
        H = UBound (sngInput1, 2)
        ReDim sngOutput (W, H)
        For X = 0 To W
            For Y = 0 To H
                sngOutput (X, Y) = (sngInput1 (X, Y) + sngInput2 (X, Y) + sngInput3 (X, Y)) / 3
            Next
            'setProgress 100 * X / W
        Next
    End Sub
    The flow of operations performed by the "nodes" is managed in such a way that as soon as all the inputs of a node are ready, it begins processing.
    So, a very important point to take into consideration is how to handle the end of single function processing (Node / Effect).
    As this determines whether the inputs of the next Node (s) are ready or not.

    I hope for some help.

    (If it is preferable, let's continue this discussion on PhotoModularFX Thread)

  17. #17
    Fanatic Member
    Join Date
    Sep 2010
    Location
    Italy
    Posts
    678

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    I almost succeed.
    I'm not very sure it's the best way.
    Everything seems a little cumbersome. And above all perhaps it is not correct to have 2 outputs to manage. (instead it should be one)
    Attached Files Attached Files

  18. #18
    Fanatic Member
    Join Date
    Sep 2010
    Location
    Italy
    Posts
    678

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Question:
    If I have a 2D input array which size is not constant.
    How can I change the memory size to reserve? (Since it can be different (per call))
    Do I have to Destroy and re-create WorkerProcesses ? or is there another way ?

    Related:
    Still unclear how much must be the memory size to reserve "MemSizeToReserve"
    Impossible to understand where + 8192 value comes from.

  19. #19
    Fanatic Member
    Join Date
    Sep 2010
    Location
    Italy
    Posts
    678

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    As for the implementation of multiprocess in my project, I made some progress.
    I think it's a very powerful tool.
    And I'm starting to handle it with a little more familiarity, (Better than #17)

    Now another question arises:
    Is it possible to have a "worker" who is able to perform different types of "Jobs"?
    Practically I have many types of jobs, and I wonder if, to perform them, it is necessary to have as many types of workers, or, if in some way it is possible to have only one type of worker who performs more types of jobs. (one at a time)


    .... I wonder if I'm asking you stupid questions, seeing the number of answers I've received.

  20. #20

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by reexre View Post
    As for the implementation of multiprocess in my project, I made some progress.
    I think it's a very powerful tool.
    And I'm starting to handle it with a little more familiarity, (Better than #17)
    Glad you got your head wrapped around it some more...
    (Shared-Memory-stuff with "concurrent access" always needs a bit of time - in any language).

    Quote Originally Posted by reexre View Post
    Now another question arises:
    Is it possible to have a "worker" who is able to perform different types of "Jobs"?
    Practically I have many types of jobs, and I wonder if, to perform them, it is necessary to have as many types of workers, or, if in some way it is possible to have only one type of worker who performs more types of jobs. (one at a time)
    Sure, the "MemSizeToReserve" is your own reserved Memory-area (in Bytes) -
    and you can split it up in any way you like:
    - e.g. the first 2KByte for example, to be reserved for your own UDT-like "JobDescriptions" and Input-Params.
    - and then maybe another 2MegaBytes as a large enough Buffer for your raw input-pixels
    - and maybe another 2MB for your output-pixels
    so MemSizeToReserve would be 4MBytes + 2KBytes "total size"

    And yes, the MultiProcesses are thought for exactly that (sitting there idling, waiting for the next Job) -
    they are not really thought for a sequence of:
    - Process-Startup
    - Do something specific
    - Process-Terminate
    for every single Job - no.

    You can use them to handle different kind of Jobs (if you pass the right params - and have large enough Buffers reserved, to allow for some flexibility also memory-allocation-wise).

    Quote Originally Posted by reexre View Post
    .... I wonder if I'm asking you stupid questions, seeing the number of answers I've received.
    No - it's just that I don't have much time these days - and whilst I'm answering some simpler questions now and then in this forum -
    those usually don't require me to "clean out everything else, to dive in deeply" - which this stuff here demands...

    Maybe I find time at the coming weekend to go over - and comment on - your first try with a "multijob-capable example"
    (if you find time to upload one for me or other readers of this thread).

    Olaf

  21. #21
    Fanatic Member
    Join Date
    Sep 2010
    Location
    Italy
    Posts
    678

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Thank you!
    At the moment I am able to have "Worker" that perform multiple types of "jobs". (Just by usging JobType )
    I will post it when I have perfected it a little. (it will be a little customized according to my needs)

    This is the structure of Shared Type
    Code:
    Public Type tParams
         P (1 To 10) As String * 25
    End Type
    
    Public Type tWorkDATA
         W As Long
         H As Long
         StartY As Long
         EndY As Long
         CurrProcess As Long  '
         NProcesses As Long
         JobType As Long   'To have many Types of Jobs
         JobParams As tParams 'Job Parameters
    End Type

  22. #22
    Fanatic Member
    Join Date
    Sep 2010
    Location
    Italy
    Posts
    678

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    I managed to "destroy" Master and re-initialize it with a larger Memory to reserve.

    I did this:
    - Initialize the Masters with the memory I need.
    - If I need less memory, I do nothing. (I leave the first initialization)
    - If I need more memory, I do this:

    1) If Not (fMain.Master1 Is Nothing) Then fMain.Master1.CloseWorkerGracefully: Set fMain.Master1 = Nothing

    2) Set fMain.Master1 = CreateWorkerProcess (MemoryToReserve) 'Bigger memory to reserve

    3) To avoid error on tmrJobState creation (Runtime Error 727 There is already a control named: ...) I have changed this line of code in InitAndBindWorkerTo
    from
    Code:
    If tmrJobState Is Nothing Then Exit Function Else tmrJobState.Interval = 7
    to
    Code:
         If tmrJobState Is Nothing Then
             MasterForm.Controls.Remove "tmrJobState" & ObjPtr (Me)
             Set tmrJobState = MasterForm.Controls.Add ("VB.Timer", "tmrJobState" & ObjPtr (Me))
         End If
         tmrJobState.Interval = 7
    It seems to work !!! (however, I do not know if this way implies other "hidden" consequences)

  23. #23

    Thread Starter
    PowerPoster
    Join Date
    Jun 2013
    Posts
    7,219

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by reexre View Post
    It seems to work !!! (however, I do not know if this way implies other "hidden" consequences)
    If this refers to the timer-handling - you can always replace the VB.Timer (and thus the necessary VB-Form)
    whit an RC5-Timer-Class (which won't force you to Add/Remove a Control, just to get a "tick-event").

    Code:
      If tmrJobState Is Nothing then Set tmrJobState = New_c.Timer(7, True) '<- Optional True, if you want to enable it already
    You'd have to change the Type of tmrJobState from VB.Timer to cTimer in the Declaration-Section.

    Olaf

  24. #24
    Fanatic Member
    Join Date
    Sep 2010
    Location
    Italy
    Posts
    678

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    This is my example of use for image processing.
    The advantage is that in this way all the CPUs are used so that, the same "Effect" algorithm, can be performed 4 times faster (compared to the use of a single process) if you have a 4Cores.

    Raw description:

    - STARTUPProcesses
    We Initialize Master workers. The Number of them is equal to Environ("NUMBER_OF_PROCESSORS")
    -SETUPProcessesMemory
    Memory to reserve for each worker process depends by Picture size , number of Input and Output Channels and Number of the Workers.
    If we need less memory than before we do not re-initialize workers.
    Each time we start an Effect-job, before we must run this sub. (at least to check if we have reserved enought memory)

    -StartJOBS_11 StartJOBS_33
    Public Sub StartJOBS_11(parameters As tParams, Inp() As Single, Out() As Single, JobType As Long, Optional NIterations As Long = 1)
    We use this sub to start processing an Effect.
    JobType indicates which Effect we want to Run.
    11 stands for 1 input 1 outoput
    33 stands for 3 inputs 3 outoputs
    Inputs are get by GetRGBPixelValues that transforms 0-255 RGB bytes to 0-1 Singles.

    -StartStripeJob1Input(M As cIPCMaster, sngINP() As Single, Proc As Long, ofProc As Long, JobType As Long, Params As tParams)
    The previous sub calls this one.
    Were we (pass and) write Inputs channels Data plus other info (Parameters and typeOfJob [tWorkDATA]) to Workers.
    There are many kind of this sub , depending on the Number of Input channels.
    We set M.JobState = jobPrepared to start Worker to do jobs.

    Then we wait for all Jobs to be completed. (Maybe there's a better way)
    Do
    Wait 100
    Loop While Not (ProcessDone1 And ProcessDone2)



    Once they are completed we Read Workers Output and put them to Out() arrays.

    fMain.Master1.UserDataRead VarPtr(WD), LenB(WD) 'to get WD.StartY
    fMain.Master1.UserDataRead VarPtr(out1(0, WD.StartY)), ProcOutSize, Offset
    fMain.Master1.UserDataRead VarPtr(out2(0, WD.StartY)), ProcOutSize, Offset + ProcOutSize
    fMain.Master1.UserDataRead VarPtr(out3(0, WD.StartY)), ProcOutSize, Offset + ProcOutSize * 2


    Now Out() arrays are ready to be shown on Picture
    So we:
    Output3ToPIC PIC.Image.Handle, out1, out2, out3
    PIC.Refresh


    -Output3ToPIC
    Converts 0-1 single values to 0-255 RGB and show them on PIC.
    BTW:
    I use 0-1 singles because in my software there is a cascade of effects.
    0-255 values are used only in 1st image input and last Image result. While between intermediate effects are used float 0-1 Values. In many case this leads to higher final output precision.
    (Sort of Continuous vs. Discrete)



    At the moment I am satisfied with what I have achieved.
    (If there are changes, I will update the attachment of this post or delete it to post a new one)

    Name:  Immagine.jpg
Views: 1483
Size:  28.9 KB
    Attached Files Attached Files

  25. #25
    Fanatic Member
    Join Date
    Sep 2010
    Location
    Italy
    Posts
    678

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    I don't like to have timers with small interval enabled continuously firing useless events.
    So in the "cIPCmaster" class I added these two enable/disable timer lines:

    Code:
    Private Sub tmrJobState_Timer()
    
        Static JobState As eJobState, JobPSec#, JobFSec#, Progress!
        T = ReadInternalType
        If JobState <> T.JobState Or JobPSec <> T.JobPreparedHPSec Or JobFSec <> T.JobFinishedHPSec Then
            JobState = T.JobState: JobPSec = T.JobPreparedHPSec: JobFSec = T.JobFinishedHPSec
    
            RaiseEvent JobStateChanged(JobState)
    
            If JobState = jobFinished Then tmrJobState.Enabled = False
            
        End If
        If Progress <> T.JobProgress Then
            Progress = T.JobProgress
            RaiseEvent ProgressChanged(Progress)
        End If
    
    End Sub
    Code:
    Public Property Let JobState(ByVal RHS As eJobState)
        If BaseAddr = 0 Then Exit Property
        T = ReadInternalType
        T.JobState = RHS
        If RHS = jobPrepared Then T.JobPreparedHPSec = HPTimer: T.JobProgress = 0
        WriteInternalType T
        
        If RHS = jobPrepared Then tmrJobState.Enabled = True
    
    End Property

    could this lead to bad consequences ?

  26. #26
    Lively Member
    Join Date
    Jun 2016
    Posts
    106

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    Quote Originally Posted by couttsj View Post
    You are right! Don't know how I managed that. Probably added it, created the executable, and created the zip without saving the file. Anyway, I added a message to tell me I can't run it in the IDE.
    Code:
    Sub Main()
        If Len(Trim$(Command$)) Then
            EnterWorkerLoop New cIPCWorker, Trim$(Command$)
        Else
            MsgBox "To be able to start a Worker, this Project needs to be compiled first"
        End If
    End Sub
    Still looking for an appropriate application to use this.

    J.A. Coutts

    all sample files here have same bug for me:
    MsgBox "To be able to start a Worker, this Project needs to be compiled first"

    i try include the code above into sample projects but not know the place to put the code, or if i need create some class or module.
    i compiled the exe of the last sample file but same problem
    exists a working version of the last sample file or the place i can include the code above?

    thanks!

  27. #27
    Frenzied Member
    Join Date
    Dec 2012
    Posts
    1,470

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    I forgot to include the Sub Main (see post #14), and I never updated the download file. It has now been updated.

    J.A. Coutts

  28. #28
    Lively Member
    Join Date
    Jun 2016
    Posts
    106

    Re: VB6 MultiProcessing (StdExe-IPC via SharedMemory)

    yes, runs ok now, i include the code Sub main of post#14 into worker project, worker module, and copiled this worker project first, after compiled the Multiprocess project and then runs the Multiprocess exe fine.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width