I realize this has been covered a million times before but with threading sometimes a problem is app specific. I have a multithreaded app that assigns a url to a thread which it then fetches and crawls. Now the function where I collect each threads items I have added synclocks assuming this would avoid alterations. But I get the "collection was modified, enumeration operation may not execute" error.
Please look at my code:
Also I forgot to mention that this entire process that includes the creation of threads for the url fetching is being run by a background thread also. I needed an individual thread to run this processes since the main thread needs to handle something else.Code:For Each landingPage As String In landingPages 'logger.constructLog("Working on landingPage link: " & landingPage & " at " & DateTime.Now.ToString, True) Dim thread = New Thread(AddressOf processQuery) count = count + 1 thread.Name = "Worm" & count thread.Start(landingPage) ThreadList.Add(thread) If numThread >= 10 Then Exit For Else numThread = numThread + 1 End If 'processQuery(landingPage) logger.constructLog("Complete with link search: " & landingPage & " at " & DateTime.Now.ToString, True) Next ...process function If Not String.IsNullOrEmpty(html) Then If searchChoice = "something1" Or fromUrl.Contains("something1") Then SyncLock something1lock links = parsingUtilities.Getsomething1LinksFromHtml(fromUrl, html, searchItem) posts = parsingUtilities.GetPsomething1FromHtml(links) End SyncLock ElseIf searchChoice = "something2" Or fromUrl.Contains("something2") Then SyncLock something2lock links = parsingUtilities.Getsomething2LinksFromHtml(fromUrl, html, searchItem) posts = parsingUtilities.GetPsomething2FromHtml(links) End SyncLock ElseIf searchChoice = "something3" Or fromUrl.Contains("something3") Then SyncLock something3lock links = parsingUtilities.Getsomething3LinksFromHtml(fromUrl, html, searchItem) posts = parsingUtilities.GetPsomething3FromHtml(links) End SyncLock ElseIf searchChoice = "something4" Or fromUrl.Contains("something4") Then SyncLock something4lock links = parsingUtilities.GetsomethingLinksFromHtml(fromUrl, html, searchItem) posts = parsingUtilities.GetPsomething4FromHtml(links) End SyncLock End If Dim ExecutionError As String Try SyncLock dlock For Each l As String In links finallinks.Add(l) Next For Each p As String In posts finalposts.Add(p) Next End SyncLock Catch ex As ApplicationException messagebox.show(ex.message.tostring) End Try Try SyncLock dlock Dim filepath As String = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments) Dim outfile As New StreamWriter(filepath & "\Visual Studio 2008\Projects\output.txt", True) Dim i As Integer = 0 For Each link As String In finallinks While i < posts.Count() postlink = finalposts.Item(i) i = i + 1 ' ...the rest is code to enter data into a database
I thought I would post the part where I create the threads and assign them since I think there is an issue here as well, I
I know the code is poorly done...so please offer suggestions.
Thanks




Reply With Quote