ThreadPool and large processing of WorkItem

[Techno]

Been trying to implementing ThreadPool over a large and variable amount of data like the following:

public sub ProcessWorkItem(byval state as object)

   Dim originalState as CustomStateInfo = CType(state, CustomStateInfo)

   'Do stuff

   originalState.ManualResetEventInstance.Set()

   
end sub

'
'

public sub DoWork()

   Dim myData as List(Of Something) = someSource.GetData() 'this can be 100, 4000 or more objects in the collection

   for each currentItem as Something in myData

      ThreadPool.QueueUserWorkItem(......)
      'somewhere here we do a WaitHandle.WaitAll(.....)
   next


end sub

This of course will bomb out if we have more than 64 items/threads being placed on the ThreadPool.

So, how would one process say x amount of items until there are no more items left to process? What is a better way of doing this?

[jmcilhinney]

You don't place a thread on the ThreadPool. The ThreadPool already contains all the threads its going to contain. What you're doing is creating a delegate and placing it in a queue to then be invoked on the first available ThreadPool thread.

It's not an issue queuing more than 64 items on the ThreadPool. It's an issue waiting on all or any of more than 64 WaitHandles. Keep in mind that I've never actually used these classes before in an app. I've only read about them, so this is a learning experience for me too. I didn't know about that limit but now that I've done some reading it seems that large numbers of WaitHandles are expensive. With a bit of experimentation I came up with this example:

Imports System.Threading
 
Public Class Form1
 
    Private syncRoot As New Object
 
    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim rng As New Random
 
        'Create a variable number of work items to process.
        Dim data As Integer() = Enumerable.Range(0, rng.Next(100, 201)).ToArray()
 
        'This will be signalled by a worker thread when it has processed a work item.
        Dim callHandle As New AutoResetEvent(False)
 
        'This will be signalled by the control thread to acknowledge completion of a work item.
        Dim responseHandle As New AutoResetEvent(False)
 
        'Queue each work item.
        For Each item As Integer In data
            ThreadPool.QueueUserWorkItem(AddressOf ProcessWorkItem,
                                         New WorkItemInfo With {.CallHandle = callHandle,
                                                                .ResponseHandle = responseHandle})
        Next
 
        'Get the number of work items that must be processed.
        Dim workItemCount As Integer = data.Length
 
        Do
            'Wait for a work item to complete.
            callHandle.WaitOne()
 
            'Decrement the number of work items to complete.
            workItemCount -= 1
 
            'Acknowledge the completed work item.
            responseHandle.Set()
        Loop Until workItemCount = 0
 
        'All work items are complete.
        MessageBox.Show(data.Length & " work items completed.")
    End Sub
 
    Private Sub ProcessWorkItem(ByVal info As Object)
        Dim wii = DirectCast(info, WorkItemInfo)
        Dim callHandle As AutoResetEvent = wii.CallHandle
        Dim responseHandle As AutoResetEvent = wii.ResponseHandle
 
        'Process the data.
        Thread.Sleep(1000)
 
        SyncLock Me.syncRoot
            'Signal the control thread and wait for an acknowledgement before completing.
            WaitHandle.SignalAndWait(callHandle, responseHandle)
        End SyncLock
    End Sub
End Class
 
 
Friend Class WorkItemInfo
    Public CallHandle As AutoResetEvent
    Public ResponseHandle As AutoResetEvent
End Class

[Techno]

interesting. will try this, thank-you very much!

[Techno]

ok just tried it but I don't think it's quite doing threading. I see the disk activity thrashing in my sample, which is to pull back 2 fields from a DB given an ID (all sitting locally).

looking at the output also in the ProcessWorkItem, it seems to process records in order whenever its calling as I would have thought that it would do it in any order the ThreadPool would see fit (as we are placing a delegate callback on the thread, so no guarnatee on which one will execute first).

re-looking at the code at this time of day, this is what it is doing. but is it REALLY doing threading (in this case making the processing of workitems faster rather than in a sync format)? Seems to me it is working in a syncronise thread pattern

last night I almost had a solution working but this was me creating 50 ResetEvent handles at a time, which probably isn't the best way, then resetting it on the next way round in the loop. so when 50 has been reached, it would do a waitall(), then process the next set. this seems to work faster than the above but of course my little experiment isn't the best or efficient way

[jmcilhinney]

Think about it. You're calling QueueUserWorkItem. What's a queue? It's a first in, first out list. Of course the items are going to be processed in order. When multiple items are being processed simultaneously, those multiple items will be interleaved in a non-deterministic way, but that doesn't mean that they're going to start, or even finish, out of order. Every now and again you may get a thread that started later finishing earlier, but it's not that likely. If it does happen then that's an indication that the system isn't managing the threads in such a way that they get even processor time.

Besides that I just changed my original code to incorporate a Stopwatch. I did this:

Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
    Dim timer = Stopwatch.StartNew()

and then this:

'All work items are complete.
MessageBox.Show(data.Length & " work items completed in " & timer.Elapsed.ToString())

The first time I ran the project the message displayed was:

112 work items completed in 00:00:12.0131713

So, each work item must take at least 1 second, because I'm calling Thread.Sleep(1000) in the ProcessWorkItem method. If there was no multithreading going on then the whole process should have taken more than 112 seconds. Instead it took just over 12, which is a 10-fold reduction in execution time. Where did that time go if there's no multi-threading?

There may be a more efficient way to do this that I haven't thought of too. The SyncLock and the SignalAndWait at the end of each task does eat up a little bit of time, but I found that the control thread missed some of the signals without it. The control thread does very little work for each call and response cycle though, so the amount of time added shouldn't be significant.

I just received the latest Microsoft Flash newsletter today and the headline story was 7 Free .NET Development and Architecture E-books. Although it's aimed at C#, the one on Threading might be worth a look for you right now.

[jmcilhinney]

To make things a bit clearer, I've updated my code a bit:

Imports System.Threading
 
Public Class Form1
 
    Private syncRoot As New Object
 
    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim timer = Stopwatch.StartNew()
        Dim rng As New Random
 
        'Create a variable number of work items to process.
        Dim data As Integer() = Enumerable.Range(0, rng.Next(100, 201)).ToArray()
 
        'This will be signalled by a worker thread when it has processed a work item.
        Dim callHandle As New AutoResetEvent(False)
 
        'This will be signalled by the control thread to acknowledge completion of a work item.
        Dim responseHandle As New AutoResetEvent(False)
 
        'Queue each work item.
        For Each item As Integer In data
            ThreadPool.QueueUserWorkItem(AddressOf ProcessWorkItem,
                                         New WorkItemInfo With {.CallHandle = callHandle,
                                                                .ResponseHandle = responseHandle,
                                                                .Value = item})
        Next
 
        'Get the number of work items that must be processed.
        Dim workItemCount As Integer = data.Length
 
        Do
            'Wait for a work item to complete.
            callHandle.WaitOne()
 
            'Decrement the number of work items to complete.
            workItemCount -= 1
 
            'Acknowledge the completed work item.
            responseHandle.Set()
        Loop Until workItemCount = 0
 
        'All work items are complete.
        MessageBox.Show(data.Length & " work items completed in " & timer.Elapsed.ToString())
    End Sub
 
    Private Sub ProcessWorkItem(ByVal info As Object)
        Dim wii = DirectCast(info, WorkItemInfo)
        Dim callHandle As AutoResetEvent = wii.CallHandle
        Dim responseHandle As AutoResetEvent = wii.ResponseHandle
        Dim value As Integer = wii.Value
 
        'Process the data.
        Thread.Sleep(1000)
 
        SyncLock Me.syncRoot
            Console.WriteLine("Work item {0} completed at {1:HH:mm:ss.fffffff}", value, Date.Now)
 
            'Signal the control thread and wait for an acknowledgement before completing.
            WaitHandle.SignalAndWait(callHandle, responseHandle)
        End SyncLock
    End Sub
End Class
 
 
Friend Class WorkItemInfo
    Public CallHandle As AutoResetEvent
    Public ResponseHandle As AutoResetEvent
    Public Value As Integer
End Class

Create a new project with just a Button on the form and then add that code. Run the project and click the Button. After 10 - 20 seconds you'll see the message telling you how many work items were processed and how long it took. Without closing the app, go back to the IDE and check out the Output window. It will contain a list of all the work items and the time they finished. There are three things to note there:

1. The numbers of the work items should be fairly close to ascending but you should see some of them slightly out of order. That is evidence that multi-threading is occurring and that the threads are being scheduled in a non-deterministic manner.

2. The times for the threads should appear in blocks. In each block the times should be almost the same, varying by up to a few tenths of a second. Between blocks, the times should vary by about one second. Here's the output from one of my runs:

Work item 0 completed at 17:47:56.3095126
Work item 2 completed at 17:47:56.3205133
Work item 1 completed at 17:47:56.3205133
Work item 3 completed at 17:47:56.3205133
Work item 4 completed at 17:47:57.2905688
Work item 6 completed at 17:47:57.3205705
Work item 8 completed at 17:47:57.3205705
Work item 5 completed at 17:47:57.3205705
Work item 7 completed at 17:47:57.3205705
Work item 9 completed at 17:47:58.2886258
Work item 10 completed at 17:47:58.2906260
Work item 11 completed at 17:47:58.3206277
Work item 13 completed at 17:47:58.3206277
Work item 12 completed at 17:47:58.3206277
Work item 14 completed at 17:47:58.3206277
Work item 16 completed at 17:47:59.2886830
Work item 15 completed at 17:47:59.2886830
Work item 17 completed at 17:47:59.2906832
Work item 21 completed at 17:47:59.3206849
Work item 20 completed at 17:47:59.3206849
Work item 18 completed at 17:47:59.3206849
Work item 19 completed at 17:47:59.3206849
Work item 22 completed at 17:48:00.2917404
Work item 23 completed at 17:48:00.2927405
Work item 24 completed at 17:48:00.2927405
Work item 25 completed at 17:48:00.2927405
Work item 26 completed at 17:48:00.3207421
Work item 28 completed at 17:48:00.3207421
Work item 29 completed at 17:48:00.3207421
Work item 27 completed at 17:48:00.3207421
Work item 34 completed at 17:48:01.2927977
Work item 30 completed at 17:48:01.2927977
Work item 32 completed at 17:48:01.2927977
Work item 31 completed at 17:48:01.2927977
Work item 33 completed at 17:48:01.2927977
Work item 35 completed at 17:48:01.3207993
Work item 38 completed at 17:48:01.3207993
Work item 36 completed at 17:48:01.3207993
Work item 37 completed at 17:48:01.3207993
Work item 39 completed at 17:48:02.2888546
Work item 42 completed at 17:48:02.2928549
Work item 44 completed at 17:48:02.2928549
Work item 41 completed at 17:48:02.2928549
Work item 40 completed at 17:48:02.2928549
Work item 43 completed at 17:48:02.2928549
Work item 45 completed at 17:48:02.3208565
Work item 46 completed at 17:48:02.3208565
Work item 48 completed at 17:48:02.3208565
Work item 47 completed at 17:48:02.3208565
Work item 50 completed at 17:48:03.2889118
Work item 49 completed at 17:48:03.2889118
Work item 52 completed at 17:48:03.2929121
Work item 53 completed at 17:48:03.2929121
Work item 51 completed at 17:48:03.2929121
Work item 55 completed at 17:48:03.2929121
Work item 54 completed at 17:48:03.2929121
Work item 56 completed at 17:48:03.3209137
Work item 59 completed at 17:48:03.3209137
Work item 57 completed at 17:48:03.3209137
Work item 58 completed at 17:48:03.3209137
Work item 60 completed at 17:48:04.2889690
Work item 62 completed at 17:48:04.2889690
Work item 61 completed at 17:48:04.2889690
Work item 63 completed at 17:48:04.2929693
Work item 67 completed at 17:48:04.2929693
Work item 65 completed at 17:48:04.2929693
Work item 64 completed at 17:48:04.2929693
Work item 66 completed at 17:48:04.2929693
Work item 68 completed at 17:48:04.3209709
Work item 70 completed at 17:48:04.3209709
Work item 69 completed at 17:48:04.3209709
Work item 71 completed at 17:48:04.3209709
Work item 74 completed at 17:48:05.2890262
Work item 73 completed at 17:48:05.2890262
Work item 75 completed at 17:48:05.2890262
Work item 72 completed at 17:48:05.2890262
Work item 76 completed at 17:48:05.2930265
Work item 79 completed at 17:48:05.2930265
Work item 78 completed at 17:48:05.2930265
Work item 77 completed at 17:48:05.2930265
Work item 80 completed at 17:48:05.2930265
Work item 81 completed at 17:48:05.3210281
Work item 83 completed at 17:48:05.3210281
Work item 82 completed at 17:48:05.3210281
Work item 84 completed at 17:48:05.3210281
Work item 88 completed at 17:48:06.2890834
Work item 85 completed at 17:48:06.2890834
Work item 89 completed at 17:48:06.2890834
Work item 87 completed at 17:48:06.2890834
Work item 86 completed at 17:48:06.2900835
Work item 91 completed at 17:48:06.2930837
Work item 94 completed at 17:48:06.2930837
Work item 93 completed at 17:48:06.2930837
Work item 92 completed at 17:48:06.2930837
Work item 90 completed at 17:48:06.2930837
Work item 95 completed at 17:48:06.3210853
Work item 97 completed at 17:48:06.3210853
Work item 96 completed at 17:48:06.3210853
Work item 98 completed at 17:48:06.3210853
Work item 99 completed at 17:48:07.2891406
Work item 101 completed at 17:48:07.2891406
Work item 100 completed at 17:48:07.2891406
Work item 102 completed at 17:48:07.2891406
Work item 103 completed at 17:48:07.2901407
Work item 104 completed at 17:48:07.2901407
Work item 107 completed at 17:48:07.2931409
Work item 105 completed at 17:48:07.2931409
Work item 109 completed at 17:48:07.2931409
Work item 106 completed at 17:48:07.2931409
Work item 108 completed at 17:48:07.2931409
Work item 110 completed at 17:48:07.3211425
Work item 111 completed at 17:48:07.3211425

I've distinguished the bloacks and highlighted one of them in blue. Note that the first time in the block is about one second later than the time before it and the last time in the block is about one second earlier than the one after it, while the times within the block vary by less than 0.3 of a second. This is an indication that six threads became available and the ThreadPool dequeued the next six work items in a small space of time. Those threads then all slept for one second and then all completed in rapid succession. As those threads finished processing those work items they became available again, so the ThreadPool used them to process the next block of work items.

3. Each block is one larger than the block before. This indicates that the ThreadPool was progressively allocating more threads to these queued work items. I wasn't aware that that was the case and I'm not sure of the specifics. Maybe the ThreadPool creates more threads as they are required, within the bounds of what the system can reasonably support, and what the system can support is determined iteratively.

[jmcilhinney]

Every now and again you may get a thread that started later finishing earlier, but it's not that likely.

I just thought I'd revisit this comment after having reviewed the output posted in the previous post. Obviously it is more likely than I thought. There are quite a few work items completed out of order, although the margins are always very small. Notice also that work items are only ever out of order within blocks. This indicates that the items are started in order, even if they may finish out of order.

[Techno]

correct. thanks for this, i will try. I was already aware that there is no guarentee that the processing of items will happen in the order they are given to be processed but with the previous example you had given, implementing it showed me that it was being processed in order and didn't think this would be correct.

I also know that the ThreadPool will create more threads as required for processing items but will reuse the thread if processing is "free"

interesting learnings!

[Techno]

implementing it on the real code is having some issues but this is as they are using LINQ to SQL entities.
for each workitem, its performing a query based on the parameters (passed in as the state object) which is required.

of course, I get an exception as it failed to open a connection as its going through multiple threads :-/

update: I placed a synclock around the area it was doing the LINQ query in the worker thread method but not sure if this clashes in terms of the perf for threading as synclock gains exclusive access to that bit of code/operation and once released, other threads can access it again.

[jmcilhinney]

I decided to change things a little further and did this in the SyncLock block:

Console.WriteLine("Work item {0} completed on thread {1} at {2:HH:mm:ss.fffffff}",
                  value,
                  Thread.CurrentThread.ManagedThreadId,
                  Date.Now)

It showed what you might expect, i.e. that each block of completed work items reported using the same threads as the preceding block with addition of one new thread. Clicking the Button a second time in quick succession showed the pattern continuing from where it left of with the same threads. Clicking the Button a third time after a reasonable pause showed the pattern start again at the beginning with a new set of threads. As a result, the second run was significantly faster than the other two (less than half the time) because it had more threads available to process work items. I set the number of work items to a constant 200 and the first and third runs took between 17 and 18 seconds, while the second run took just over 8.6 seconds.

Another option to consider is parallel programming in .NET 4.0. I'm not sure what Framework version you're targeting but here's a reimplementation of my previous code using the Parallel class:

Imports System.Threading
Imports System.Threading.Tasks
 
Public Class Form1
 
    Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
        Dim timer = Stopwatch.StartNew()
        Dim rng As New Random
 
        'Create a variable number of work items to process.
        Dim data As Integer() = Enumerable.Range(0, rng.Next(100, 201)).ToArray()
 
        Parallel.ForEach(data, AddressOf ProcessWorkItem)
 
        'Get the number of work items that must be processed.
        Dim workItemCount As Integer = data.Length
 
        'All work items are complete.
        MessageBox.Show(data.Length & " work items completed in " & timer.Elapsed.ToString())
    End Sub
 
    Private Sub ProcessWorkItem(ByVal data As Integer)
        'Process the data.
        Thread.Sleep(1000)
 
        Console.WriteLine("Work item {0:000} completed on thread {1:000} at {2:HH:mm:ss.fffffff}",
                          data,
                          Thread.CurrentThread.ManagedThreadId,
                          Date.Now)
    End Sub
 
End Class

In my tests this method was slightly slower than the previous method but the code is certainly easier to follow. The results show basically the same pattern as before except that this time, the work items do indeed seem to be processed in a random order:

Work item 000 completed on thread 010 at 19:42:21.9162029
Work item 032 completed on thread 006 at 19:42:21.9162029
Work item 096 completed on thread 012 at 19:42:21.9222032
Work item 064 completed on thread 011 at 19:42:21.9222032
Work item 128 completed on thread 013 at 19:42:21.9242033
Work item 001 completed on thread 014 at 19:42:22.9242605
Work item 002 completed on thread 010 at 19:42:22.9392614
Work item 004 completed on thread 013 at 19:42:22.9402615
Work item 065 completed on thread 011 at 19:42:22.9402615
Work item 097 completed on thread 012 at 19:42:22.9402615
Work item 033 completed on thread 006 at 19:42:22.9402615
Work item 006 completed on thread 014 at 19:42:23.9263179
Work item 035 completed on thread 015 at 19:42:23.9263179
Work item 003 completed on thread 010 at 19:42:23.9393186
Work item 005 completed on thread 013 at 19:42:23.9403187
Work item 098 completed on thread 012 at 19:42:23.9403187
Work item 066 completed on thread 011 at 19:42:23.9403187
Work item 034 completed on thread 006 at 19:42:23.9403187
Work item 067 completed on thread 016 at 19:42:24.9253750
Work item 007 completed on thread 014 at 19:42:24.9263751
Work item 036 completed on thread 015 at 19:42:24.9263751
Work item 008 completed on thread 010 at 19:42:24.9393758
Work item 012 completed on thread 013 at 19:42:24.9403759
Work item 068 completed on thread 011 at 19:42:24.9403759
Work item 099 completed on thread 012 at 19:42:24.9403759
Work item 038 completed on thread 006 at 19:42:24.9403759
Work item 072 completed on thread 016 at 19:42:25.9254322
Work item 103 completed on thread 017 at 19:42:25.9254322
Work item 037 completed on thread 015 at 19:42:25.9264323
Work item 016 completed on thread 014 at 19:42:25.9264323
Work item 009 completed on thread 010 at 19:42:25.9394330
Work item 013 completed on thread 013 at 19:42:25.9404331
Work item 100 completed on thread 012 at 19:42:25.9404331
Work item 069 completed on thread 011 at 19:42:25.9404331
Work item 039 completed on thread 006 at 19:42:25.9404331
Work item 104 completed on thread 017 at 19:42:26.9254894
Work item 073 completed on thread 016 at 19:42:26.9254894
Work item 020 completed on thread 018 at 19:42:26.9254894
Work item 017 completed on thread 014 at 19:42:26.9264895
Work item 042 completed on thread 015 at 19:42:26.9264895
Work item 010 completed on thread 010 at 19:42:26.9394902
Work item 014 completed on thread 013 at 19:42:26.9404903
Work item 040 completed on thread 006 at 19:42:26.9404903
Work item 101 completed on thread 012 at 19:42:26.9404903
Work item 070 completed on thread 011 at 19:42:26.9404903
Work item 074 completed on thread 016 at 19:42:27.9255466
Work item 021 completed on thread 019 at 19:42:27.9255466
Work item 105 completed on thread 017 at 19:42:27.9255466
Work item 022 completed on thread 018 at 19:42:27.9255466
Work item 043 completed on thread 015 at 19:42:27.9265467
Work item 018 completed on thread 014 at 19:42:27.9265467
Work item 011 completed on thread 010 at 19:42:27.9395474
Work item 015 completed on thread 013 at 19:42:27.9405475
Work item 041 completed on thread 006 at 19:42:27.9405475
Work item 071 completed on thread 011 at 19:42:27.9405475
Work item 102 completed on thread 012 at 19:42:27.9405475
Work item 106 completed on thread 017 at 19:42:28.9256038
Work item 075 completed on thread 016 at 19:42:28.9256038
Work item 046 completed on thread 020 at 19:42:28.9256038
Work item 023 completed on thread 018 at 19:42:28.9256038
Work item 024 completed on thread 019 at 19:42:28.9256038
Work item 019 completed on thread 014 at 19:42:28.9266039
Work item 044 completed on thread 015 at 19:42:28.9266039
Work item 026 completed on thread 010 at 19:42:28.9396046
Work item 047 completed on thread 013 at 19:42:28.9406047
Work item 078 completed on thread 011 at 19:42:28.9406047
Work item 055 completed on thread 006 at 19:42:28.9406047
Work item 110 completed on thread 012 at 19:42:28.9406047
Work item 107 completed on thread 017 at 19:42:29.9256610
Work item 076 completed on thread 016 at 19:42:29.9256610
Work item 087 completed on thread 018 at 19:42:29.9256610
Work item 086 completed on thread 021 at 19:42:29.9256610
Work item 063 completed on thread 020 at 19:42:29.9256610
Work item 025 completed on thread 019 at 19:42:29.9256610
Work item 091 completed on thread 014 at 19:42:29.9266611
Work item 045 completed on thread 015 at 19:42:29.9266611
Work item 027 completed on thread 010 at 19:42:29.9396618
Work item 048 completed on thread 013 at 19:42:29.9406619
Work item 056 completed on thread 006 at 19:42:29.9406619
Work item 111 completed on thread 012 at 19:42:29.9406619
Work item 079 completed on thread 011 at 19:42:29.9406619
Work item 108 completed on thread 017 at 19:42:30.9257182
Work item 088 completed on thread 018 at 19:42:30.9257182
Work item 124 completed on thread 022 at 19:42:30.9257182
Work item 118 completed on thread 021 at 19:42:30.9257182
Work item 120 completed on thread 020 at 19:42:30.9257182
Work item 077 completed on thread 016 at 19:42:30.9257182
Work item 092 completed on thread 014 at 19:42:30.9267183
Work item 028 completed on thread 010 at 19:42:30.9397190
Work item 049 completed on thread 013 at 19:42:30.9407191
Work item 080 completed on thread 011 at 19:42:30.9407191
Work item 112 completed on thread 012 at 19:42:30.9407191
Work item 057 completed on thread 006 at 19:42:30.9407191
Work item 109 completed on thread 017 at 19:42:31.9257754
Work item 121 completed on thread 020 at 19:42:31.9257754
Work item 119 completed on thread 021 at 19:42:31.9257754
Work item 125 completed on thread 022 at 19:42:31.9257754
Work item 089 completed on thread 018 at 19:42:31.9257754
Work item 093 completed on thread 014 at 19:42:31.9267755
Work item 029 completed on thread 010 at 19:42:31.9397762
Work item 113 completed on thread 012 at 19:42:31.9407763
Work item 050 completed on thread 013 at 19:42:31.9407763
Work item 081 completed on thread 011 at 19:42:31.9407763
Work item 058 completed on thread 006 at 19:42:31.9407763
Work item 126 completed on thread 022 at 19:42:32.9258326
Work item 090 completed on thread 018 at 19:42:32.9258326
Work item 122 completed on thread 020 at 19:42:32.9258326
Work item 094 completed on thread 014 at 19:42:32.9268327
Work item 030 completed on thread 010 at 19:42:32.9398334
Work item 051 completed on thread 013 at 19:42:32.9408335
Work item 082 completed on thread 011 at 19:42:32.9408335
Work item 114 completed on thread 012 at 19:42:32.9408335
Work item 059 completed on thread 006 at 19:42:32.9408335
Work item 127 completed on thread 022 at 19:42:33.9258898
Work item 123 completed on thread 020 at 19:42:33.9258898
Work item 095 completed on thread 014 at 19:42:33.9268899
Work item 031 completed on thread 010 at 19:42:33.9398906
Work item 052 completed on thread 013 at 19:42:33.9408907
Work item 060 completed on thread 006 at 19:42:33.9408907
Work item 083 completed on thread 011 at 19:42:33.9408907
Work item 115 completed on thread 012 at 19:42:33.9408907
Work item 053 completed on thread 013 at 19:42:34.9409479
Work item 084 completed on thread 011 at 19:42:34.9409479
Work item 116 completed on thread 012 at 19:42:34.9409479
Work item 061 completed on thread 006 at 19:42:34.9409479

I've truncated the output a bit because my post was too long but you get the idea.

[jmcilhinney]

implementing it on the real code is having some issues but this is as they are using LINQ to SQL entities.
for each workitem, its performing a query based on the parameters (passed in as the state object) which is required.

of course, I get an exception as it failed to open a connection as its going through multiple threads :-/

update: I placed a synclock around the area it was doing the LINQ query in the worker thread method but not sure if this clashes in terms of the perf for threading as synclock gains exclusive access to that bit of code/operation and once released, other threads can access it again.

If the query is the time-consuming part of the operation then locking that section does kind of defeat the purpose. If there is more time-consuming processing to be done in each task then you would still gain an advantage.

You might also check that MARS is enabled in your connection string. Maybe it currently isn't and enabling it would allow your data context to execute multiple queries simultaneously. I'm not sure but it's certainly worth checking out.

[Techno]

thanks for that. MARS is not in the connection string so.... here we go!

[Techno]

well digging deeper, there are many projects within the solution all having some DB conn string - but of course only around 3/4 of the projects are being used in conjunction with 1 winforms app.
as they are using the Entity Framework, I believe by default, and it shows, the MARS is being set to true in the connectionstring.

So the exception I get, when multiplethreading is that it cannot read data when the connection is closed. (i know what it means).

but when synclock is placed, its fine and I still see the performance increased than before.

running STA (single thread) was taking, for the same criteria, around 1m 45 where as when MTA (Multithreading) it took 43s (44s ish with synclock)

still investigating..

[Techno]

Thanks again for the kind help. certainly is faster, just wondering if there is an even faster way of doing this. like I said, I almost had it twice as fast as it is now yesterday but that was poor practice and something i tried to scrap together! (Creating a large number of Resets and processing them within the 1 for loop)

even though MARS is set in the EntityFramework connection details, and because SQL queries is not multi thread safe, you cannot query the same context on different threads which means unfortunately....synclock....

[Techno]

well, I guess I obtained a reference locally scoped in the ProcessItem method which has the reference to the Context and used that reference for the LINQ query. one thing I just noticed is that writing out some debug lines in the ProcessItem worker method seems to display it twice - method being executed twice?! shouldnt be. unfortunately can't use .NET 4.0, which is what I would have loved to use and use the TPL. using .NET 3.5 here.

I've also noticed that when I place a timer in the processworkitem method, it seems to take longer and longer for each call being made to the method via the threadpool. something doesnt look/sound right.

stripped code:

public sub ProcessWorkItem(byval state as object)

   Dim myState as MyType = CType(state, MyType)

   'do processing here with some synclocks in a couple of areas

   WaitHandle.SignalAndWait(myState.CallHandle, myState.ResponseHandle)
   

end sub


public sub DoStuff()

   ' some logic here, then declare vars below

   dim myCollection as List(Of Person) = someSource.GetPersons()
   dim callHandler as new AutoResetEvent(false)
   dim responseHandler as new AutoResetEvent(false)
   dim counter as integer = 0
   dim totalItems as Integer = myCollection.Count

   for each currentItem in someCollection

      counter += 1
      Dim myState as MyType = BuildMyState(callHandler, responseHandler, currentItem)
      ThreadPool.QueueUserWorkItem(AddressOf ProcessWorkItem, myState)

   next

   do
      callHandler.WaitOne()

      totalItems -= 1

      responseHandler.Set()

   loop until totalItems = 0

end sub

update: most likely its the synclock slowing down things (well, that is what it does to sync access) which is wrapped around a LINQ to SQL query. But can't do it in MTA mode really as it throws an exception as multiple threads are accessing the Context (Entity context) and sometimes the connection is closed and the next thread thinks its open at that time or the connection state is closing etc....

:-/ d'oh.

[Techno]

I may have found a better way but this is not tested with LINQ to SQL but only a simulation without that logic.

InterLocked.Decrement, and a single ManualResetEvent.
in the worker thread, check to see if a counter is set to 0 and if so, Set() the ResetEvent. Example:

private tempCounter as Integer = 0
private single as new ManualResetEvent(False)

public sub BeginSomeWork()

Me.tempCounter = myItems.Count

for each currentItem in myItems
   'Create some StateObject
   Dim state as StateObject = Me.BuildStateObject(.....)
   ThreadPool.QueueUserWorkItem(new WaitCallBack(AddressOf ProcessWorkItem), new object() { state } )
   
next

WaitHandle.WaitAll(new ManualResetEvent() { single } )

end sub 

public sub ProcessWorkItem(byval state as object)

   Dim myState as StateObject = CType(state, StateObject)
   'do some lengthy work
   
   Interlocked.Decrement(Me.tempCounter)
   if Me.tempCounter = 0 then
      Me.single.Set()
   end if

end sub

[jmcilhinney]

Looks good. Just note that there's no point calling WaitHandle.WaitAll and passing an array containing one ManualResetEvent. You would just call WaitOne on the ManualResetEvent itself.

It's worth noting that without a SyncLock block, this code:

   Interlocked.Decrement(Me.tempCounter)
   if Me.tempCounter = 0 then
      Me.single.Set()
   end if

could conceivably call Set more than once. That's not an issue though, so it's not really worth worrying about in this case. It is worth keeping in mind for other situations though.

[Techno]

Thanks kindly. yes you are correct about the WaitAll() suggestion, WaitOne is better in this case.

But in regards to LINQ to SQL Query in the ProcessWorkItem (It is using a DataContext using the EF), is there a way to make it threading safe? Currently if I do not do this, it will throw an exception at some point stating that it tried to read data when the connection state is not open (understandably)

I guess I need to keep that synclock (lock in C#) in place around that query?

[jmcilhinney]

But in regards to LINQ to SQL Query in the ProcessWorkItem (It is using a DataContext using the EF)

That doesn't seem to make sense. LINQ to SQL and the Entity Framework are two different ways to do the same thing. I don't see how or why you would be using them both in the same project.

Currently if I do not do this, it will throw an exception at some point stating that it tried to read data when the connection state is not open (understandably)

I guess I need to keep that synclock (lock in C#) in place around that query?

I guess that the data context has on one connection and if multiple queries are in progress then the first one may close the connection before the other(s) has finished. The fact that a LINQ to SQL connection string doesn't enable MARS while an EF connection string does suggests to me that maybe the EF can handle that scenario gracefully while LINQ to SQL can't. If so then I guess that a SyncLock around the query would be your only option when using LINQ to SQL. Maybe you could add some partial class definition(s) that could work around the issue but I've never used LINQ to SQL so I really wouldn't know.

[Techno]

sure. I haven't used LINQ to SQL myself either or much of the Entity Framework.

Basically they have an edmx which defines the tables/relationships etc...

They use LINQ to query the entities (this is what happens in the ProcessWorkItem method) by using some joins on a couple of Context().EntityName's with a where clause.

This is calling SQL directly to return back the results into a collection.

MARS is enabled on the connectionstring but seems to still error out after a few calls, and as you have said and what I'm aware of is that the first call may finish the connection before the other has finished - this is the problem right here. How to maintain the connection in someway that it can successfully be left open or something more elegant so it can keep using that connection (whilst open) and execute multiple queries at any given time

[jmcilhinney]

If you have an EDMX file then that's Entity Framework. LINQ to SQL creates a DBML file. LINQ to SQL is a specific technology, not just a general term for using LINQ and SQL Server together. Whenever you are using the EF the flavour of LINQ you are using is LINQ to Entities.

[Techno]

gotcha, thanks for the clarification

[Techno]

The issue now is that it will not really report the progress back to the UI with the new threading implementation (many many calls being made at once I guess). With your solution, it does but it may not be "clean code" than the solution I had posted.
*sigh*
we all can't win!
:-/

[Techno]

ok so after a few runs i get the odd time when all the items have been processed, the app hangs and when pressing "pause" in the IDE, it points to the callHandle.WaitOne() method even though all the items are processed (as it seems from the counter).

not sure why!

running the exact same criteria/parameters etc... the same dataset, it works 95% of the time but the other times, it just "hangs" (so one one of the threads, it doesnt seem to be calling the Set() method)

[jmcilhinney]

That sounds like what was happening to me originally, which is why I ended up suggesting SignalAndWait. Multiple workers were signalling the WaitHandle before the control thread could react, so some of the signals ended up being missed. Without seeing your code it's not really possible to know what the issue is because behaviour is all very specific. Also, is this not the same topic as your other thread where I suggested a progressHandle and a completedHandle rather than a callHandle and a responseHandle?

[Techno]

ah yes, interesting.

the reason for not yet implementing the progressHandle/completedHandle was because the real work was being done in a different layer and not within the UI. I guess I can pass via parameters the completedHandle object to the other layer so it would signal when its done what it needs to do perhaps.

let me get pen/paper and try to replan it out. to place the worker items on the threadpool - again this all has to be done on the BLL, not the UI.

the UI just needs to wait for a signal I guess so it would update the progress.

[jmcilhinney]

You should implement your BLL much like a BackgroundWorker and, internally, actually use a BackgroundWorker, e.g.

Imports System.Threading
Imports System.ComponentModel
 
Public Class BusinessLogicLayer
 
    'The number of work items to process.
    Private workItemCount As Long
 
    'Signalled when the number of work items processed changes.
    Private progressHandle As New AutoResetEvent(False)
 
    'Signalled when all work items have been processed.
    Private completedHandle As New ManualResetEvent(False)
 
    Private WithEvents asyncWorker As BackgroundWorker
 
 
    Public Event ProgressChanged As EventHandler
    Public Event DoSomethingCompleted As EventHandler
 
 
    Public Function DoSomethingAsync() As Boolean
        Dim result As Boolean = False
 
        If Me.asyncWorker Is Nothing Then
            Me.asyncWorker = New BackgroundWorker
        End If
 
        If Not Me.asyncWorker.IsBusy Then
            Me.asyncWorker.RunWorkerAsync()
            result = True
        End If
 
        Return result
    End Function
 
    Private Sub ProcessWorkItem(ByVal data As Object)
        'Process the work item.
        Thread.Sleep(1000)
 
        If Interlocked.Decrement(Me.workItemCount) = 0 Then
            'All work items have been processed.
            Me.completedHandle.Set()
        Else
            'The number of work items processed has changed but there are more to process.
            Me.progressHandle.Set()
        End If
    End Sub
 
 
    Private Sub asyncWorker_DoWork(ByVal sender As Object, ByVal e As System.ComponentModel.DoWorkEventArgs) Handles asyncWorker.DoWork
        'Generate a variable number of work items.
        Dim rng As New Random
        Dim workItems As Integer() = Enumerable.Range(0, rng.Next(50, 151)).ToArray()
 
        'Create an array of handles to wait on.
        Dim waitHandles As WaitHandle() = {Me.progressHandle, Me.completedHandle}
 
        'Initialise the data for the background threads.
        Me.workItemCount = workItems.LongLength
        Me.completedHandle.Reset()
 
        'Queue all the work items.
        For Each workItem As Integer In workItems
            ThreadPool.QueueUserWorkItem(AddressOf ProcessWorkItem, workItem)
        Next
 
        'Keep waiting as long as it is the first wait handle, i.e. the progress handle, that gets signalled.
        'Break out of the loop as soon as it is the second wait handle, i.e. the completed handle, gets signalled.
        Do While WaitHandle.WaitAny(waitHandles) = 0
            'Update the progress based on the number of work items processed.
            Me.asyncWorker.ReportProgress(CInt(Interlocked.Read(Me.workItemCount)))
        Loop
    End Sub
 
    Private Sub asyncWorker_ProgressChanged(ByVal sender As Object, ByVal e As System.ComponentModel.ProgressChangedEventArgs) Handles asyncWorker.ProgressChanged
        RaiseEvent ProgressChanged(Me, EventArgs.Empty)
    End Sub
 
    Private Sub asyncWorker_RunWorkerCompleted(ByVal sender As Object, ByVal e As System.ComponentModel.RunWorkerCompletedEventArgs) Handles asyncWorker.RunWorkerCompleted
        RaiseEvent DoSomethingCompleted(Me, EventArgs.Empty)
    End Sub
 
End Class

Now your presentation layer can just call DoSomethingAsync and handle the ProgressChanged and DoSomethingCompleted events of your BLL object. You'll probably want to change the type of the ProgressChanged and DoSomethingCompleted events so that you can pass data to the event handlers, much as the BackgroundWorker itself does.

[Techno]

Thanks. The problem here is that I need to wait until everything has been completed then return a collection back to the caller (the UI) for it to bind to the screen.

or should it be said that this collection should be returned by raising the DoSomethingCompleted event when the worker has finished processing, then the event in the UI will "hear" this and then do what it needs to do?

[jmcilhinney]

That's exactly right. Like I said, you are mimicking the pattern implemented by the BackgroundWorker here. How do you get data passed to you on the UI thread by a BackgroundWorker? By the e.Result property in the RunWorkerCompleted event handler, which is executed on the UI thread. That's why I said:

You'll probably want to change the type of the ProgressChanged and DoSomethingCompleted events so that you can pass data to the event handlers, much as the BackgroundWorker itself does.

Your ProgressChanged event is going to want to pass some data to its event handler that indicates progress, just as the BackgroundWorker.ProgressChanged event does, and your DoSomethingCompleted event is going to want to pass the end result of the process to its event handler, just as the BackgroundWorker.RunWorkerCompleted event does.

[Techno]

thanks again for this. ive implemented it, seems to be working and got an even more performance improvement by around 30%.

[Techno]

i seem to be getting problems from time to time where on the last hurdle or the last percentage/workitem - it hangs - doesnt send a signal back to complete no idea why. sometimes it works, sometimes it doesnt like today.