-
Jan 30th, 2018, 08:08 AM
#1
Thread Starter
Junior Member
Multithreading with a Do Loop
So i want a sub() that is multithreaded with like 100 threads and do a Do until loop. i want to console.writeline("Do loop is done.") after that whole sub is done. does anybody know how to do this? i used to have background threads but if i do a async sub with task.waitall() the background threads **** with it. anybody can give me a real simple example code?
-
Jan 30th, 2018, 08:59 AM
#2
Re: Multithreading with a Do Loop
You're going to have to provide more information. A Do Until loop exists to perform an action until a particular condition is True but that becomes incoherent if you're executing multiple iterations simultaneously. Please explain what you're trying to achieve, rather than how you're trying to achieve it.
-
Jan 30th, 2018, 10:19 AM
#3
Re: Multithreading with a Do Loop
You won't get 100 threads anyways, unless each thread spends most of its time waiting for something. You can't get more processes running efficiently than you have processor cores to work on. If the threads are waiting, then others will get spun up, but if the threads are busy, you'd get one per core, tops. You can create 100 threads, they just won't run simultaneously.
My usual boring signature: Nothing
-
Jan 30th, 2018, 10:48 AM
#4
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
Originally Posted by Shaggy Hiker
You won't get 100 threads anyways, unless each thread spends most of its time waiting for something. You can't get more processes running efficiently than you have processor cores to work on. If the threads are waiting, then others will get spun up, but if the threads are busy, you'd get one per core, tops. You can create 100 threads, they just won't run simultaneously.
I will need to do some multithreading Do until loadedfile.count = 0 so i need to do a process with every line in a file.
-
Jan 30th, 2018, 10:49 AM
#5
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
Originally Posted by jmcilhinney
You're going to have to provide more information. A Do Until loop exists to perform an action until a particular condition is True but that becomes incoherent if you're executing multiple iterations simultaneously. Please explain what you're trying to achieve, rather than how you're trying to achieve it.
Sorry but the replies are with different people but, Shaggy how would you do it then? the stage per item will be like 10-20 seconds probably so i will need to do it multithreaded.
-
Jan 30th, 2018, 11:41 AM
#6
Re: Multithreading with a Do Loop
Oh, that's a fine approach. Start as many Tasks as you want. The thread scheduler will decide how many threads will actually be spun up. If the existing threads end up waiting a long time, it will spin up a new thread such that the processor is fully occupied. That is handled by the OS, and it does to pretty efficiently, so you don't need to worry about it. My point was just that you likely won't have 100 threads, nor should you try to force that many threads. Just fire off the tasks and let the scheduler schedule them as processor time becomes available. What you've done sounds reasonable, and a suitable use of threads, perhaps even an ideal use. In the original post, it wasn't clear whether you were testing threading itself, or using threading to do some real work. Since it's clearly the latter, it sounds good.
I think you'd have to explain the problem you are seeing a bit more....without the asterisks, as profanity is against the AUP.
My usual boring signature: Nothing
-
Jan 30th, 2018, 02:30 PM
#7
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
Woops sorry, well the problem i encounter is that i want after that process is done open up a savefiledialog and open up a notify when that sub() is done, but when i do that after the Do loop all the threads will show up a notify and i will receive an STA error because i cant open up a savefiledialog with a background thread. what i have right now is:
Code:
thrednum = "100"
For i = 0 To thrednum
Dim trd = New Thread(Sub()
Do Until loadedfile.Count = 0
Loop
End Sub) With {
.IsBackground = True
}
threads.Enqueue(trd)
trd.Start()
Next
'
when i put it after the "Next" i get the message after the program generated all the threads, but the threads are still going you know. so my question is, where should i put the notify and the savefiledialog? because im really stuck right now...
-
Jan 30th, 2018, 05:11 PM
#8
Re: Multithreading with a Do Loop
Originally Posted by SoldierCrimes
I will need to do some multithreading Do until loadedfile.count = 0 so i need to do a process with every line in a file.
And that's exactly why you need to provide a proper explanation. A Do loop is the wrong choice in that case, even without multithreading. Logically, you want to perform an action for each line in the file. It should be obvious that a For Each loop is the appropriate choice there. If you want to parallelise that then you would simply call Parallel.ForEach.
-
Jan 30th, 2018, 05:40 PM
#9
Re: Multithreading with a Do Loop
So let's assume you have a long running task that reads a file, does *something* for each line, and then finishes by launches a SaveFileDialog. If that is the case, then a simple Async method would work, then piggy-backing off of JMcIlhinney's suggestion you could also implement Parallel.ForEach:
Code:
Private Async Sub DoSomething(ByVal path As String)
Await Task.Run(Sub()
'Get each line from the file
Dim lines() As String = IO.File.ReadAllLines(path)
'Iterate through each line
Parallel.ForEach(lines, Sub(line As String)
'do "something"
Threading.Thread.Sleep(500) 'Zzz... (remove from development)
End Sub)
End Sub)
Using sfd As SaveFileDialog = New SaveFileDialog
With sfd
'Properties
.Filter = "*.txt|*.txt"
.InitialDirectory = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments)
'etc.
'Display
If .ShowDialog = DialogResult.OK Then
'Do something
End If
End With
End Using
End Sub
-
Jan 31st, 2018, 01:50 AM
#10
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
Thanks for that man! The thing is working right now except the savefiledialog, that still gives me an error when i try to open it. (again an STA error.)
Last edited by SoldierCrimes; Jan 31st, 2018 at 02:10 AM.
Reason: Error added
-
Jan 31st, 2018, 10:27 AM
#11
Re: Multithreading with a Do Loop
The error indicates you're either trying to launch the dialog from a worker thread, or you aren't in a Windows Forms application and maybe your project template's configured weird. I'm betting more on "wrong thread", because Async/Await can only get you back to the UI thread if you call the method from the UI thread.
We simply can't offer advice if we can't see the bulk of your code.
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
-
Jan 31st, 2018, 11:12 AM
#12
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
The code that dday9 gave me, its command line but with imported windows forms.
Last edited by SoldierCrimes; Jan 31st, 2018 at 12:09 PM.
Reason: Wrong person mentioned.
-
Jan 31st, 2018, 12:03 PM
#13
Re: Multithreading with a Do Loop
I can reproduce it, but I don't think the solution is going to be easy. Async/Await doesn't work quite the same in a console application, and a lot of things you need guarantees for in Windows Forms are not easy to maintain in a console application.
Here's the summary:
Windows Forms needs a few things to be true. It needs a message loop to be established, and it needs that message loop established on a thread that is marked STA like the exception complains yours isn't. I don't fully understand what that means, but I know it's true. All of this matters because a ton of assumptions Windows Forms makes get broken if you ever try to interact with the message loop from a different thread. So we say controls have "thread affinity" and a myriad of symptoms arise if you break the rules. To help you with this, Windows Forms has a concept of "marshalling", a process by which you can tell a control to execute some code on the correct thread. But if you don't have a control, you can't get to that thread. (WPF has a better mechanism!)
Console applications do not establish a message loop. That may or may not trigger the exception that mentions needing an STA thread; it could be a generalized handler for a lot of things goign wrong. Console applications also don't have thread affinity. That means they have no mechanism like Windows Forms for guaranteeing anything happens on a particular thread. We call this a "free threading" model.
Async/Await relies on a concept called the "synchronization context" to make sure it comes back to the original thread. In Windows Forms, this works because if you are on the "UI thread", there is infrastructure for the Task API to get back onto it when needed. In a Console application, none of that infrastructure is there, so when you come back from Await you can be on a different thread from the original.
So even if your code sets up a message loop in the console app's main thread, Async/Await is unstable because there's not a synchronization context to use for that thread. Maybe we can set one up, maybe not. I think it's just as likely the only safe way to use Windows Forms from a console application is to establish a Thread that runs the message loop and give it a mechanism by which you can marshal code to it. I'm not sure how to do that, which is why I'm not committing to a solution just yet.
I sort of hope someone comes up with a different approach before I get too deep, but I think in general "console applications shouldn't use Windows Forms" is good advice.
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
-
Jan 31st, 2018, 12:08 PM
#14
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
Totally true, and again thanks for the great explanations! But the savefiledialog is just so sexy if you want to keep it clean and dont ask for: Console.writeline("Please enter the file path: ") or something like that. i hope this can be done in an easy way. anyways thanks for helping sir!
-
Feb 1st, 2018, 10:49 AM
#15
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
Does anybody else can help me? im still struggling with the Savefiledialog...
-
Feb 1st, 2018, 12:15 PM
#16
Re: Multithreading with a Do Loop
You're trying to force a square peg into a round hole; console applications are not designed to use UI components. Let's take a different approach, why don't you tell us what the overall scope of your project is so that we can help you determine what type of application you should be focusing on.
-
Feb 1st, 2018, 02:29 PM
#17
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
I'm trying to develop a proxy checker, so i let the user scrape proxies after that save them in a filedialog
so what the program does is just go to a couple of sites with a specific regex to get the proxies and save them in a list. but all those request take some time thats why i need it multithreaded. so after that process is done i want to open a savefiledialog where the user can save the proxies in a text file. and thats where the error happens.
-
Feb 1st, 2018, 02:44 PM
#18
Re: Multithreading with a Do Loop
Does the user specify the websites and RegEx?
-
Feb 1st, 2018, 02:48 PM
#19
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
Nope, the regex is standard and i have all the sites added in a list in my program.
-
Feb 2nd, 2018, 05:12 PM
#20
Re: Multithreading with a Do Loop
I still don't think that you should try to implement a UI control in a console application. For example, take a look at this makeshift directory explorer I made for a console application integrated with your need:
Code:
Public Module Module1
Public Sub Main()
'Get each line from the file
Dim lines() As String = IO.File.ReadAllLines(IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments), "example.txt"))
'Iterate through each line
Parallel.ForEach(lines, Sub(line As String)
'do "something"
Threading.Thread.Sleep(500) 'Zzz... (remove from development)
End Sub)
'Save results
Dim directory As String = DirectoryExplorer(New IO.DirectoryInfo("C:/"))
Console.ResetColor()
Console.WriteLine("Please enter your file name")
Console.Write(directory & "\")
Dim file As String = Console.ReadLine
'Do something with the file
Dim path As String = IO.Path.Combine(directory, file)
End Sub
Private Function DirectoryExplorer(ByVal root As IO.DirectoryInfo, Optional ByVal skip As Integer = 0) As String
Console.Clear()
'Print the title in green
Console.ForegroundColor = ConsoleColor.Green
Console.WriteLine("File Explorer: {0}", root.FullName)
'Keep track of the F code and the paths
Dim command As String = "F"
Dim counter As Integer = 1
Dim paths As List(Of String) = New List(Of String)
'Add the directory-up command if applicable
If root.Parent IsNot Nothing Then
paths.Add("../")
End If
'Get each directory first
paths.AddRange(root.EnumerateDirectories.Select(Function(dir) dir.Name))
'Change the color to white
Console.ForegroundColor = ConsoleColor.White
'Iterate through the displayed paths
For Each path As String In paths.Skip(skip).Take(24)
'Print the command followed by the name
Console.WriteLine(" {0}{1}: {2}", command, counter, path)
'Increment the counter
counter += 1
If counter = 13 Then
'Change the shift code and input the counter
command = "Shift+F"
counter = 1
End If
Next
'Print if there will be a next page
If skip + 24 < paths.Count - 1 Then
Console.WriteLine(" --Use the key combination SHIFT + ENTER key to display the next page of results")
End If
'Print if there will be a previous page
If skip > 0 Then
Console.WriteLine(" --use the key combination SHIFT + ESC key to display the previous page of results")
End If
'Display the options
Console.WriteLine()
Console.ForegroundColor = ConsoleColor.Magenta
Console.WriteLine("Hit the ENTER key to return the current directory.")
Console.WriteLine("Hit the ESC key to return nothing.")
'Prompt for the input
Dim input As ConsoleKeyInfo = Console.ReadKey
Dim fkeys() As ConsoleKey = {ConsoleKey.F1, ConsoleKey.F2, ConsoleKey.F3, ConsoleKey.F4, ConsoleKey.F5, ConsoleKey.F6, ConsoleKey.F7, ConsoleKey.F8, ConsoleKey.F9, ConsoleKey.F10, ConsoleKey.F11, ConsoleKey.F12, ConsoleKey.F13, ConsoleKey.F14, ConsoleKey.F15, ConsoleKey.F16, ConsoleKey.F17, ConsoleKey.F18, ConsoleKey.F19, ConsoleKey.F20, ConsoleKey.F21, ConsoleKey.F22, ConsoleKey.F23, ConsoleKey.F24}
Dim index As Integer = Array.IndexOf(fkeys, input.Key)
If input.Key = ConsoleKey.Enter AndAlso input.Modifiers = ConsoleModifiers.Shift AndAlso skip + 24 < paths.Count - 1 Then
Return DirectoryExplorer(root, skip + 24)
ElseIf input.Key = ConsoleKey.Enter Then
Return root.FullName
ElseIf input.Key = ConsoleKey.Escape AndAlso input.Modifiers = ConsoleModifiers.Shift Then
Return DirectoryExplorer(root, skip - 24)
ElseIf input.Key = ConsoleKey.Escape Then
Console.Clear()
Console.ResetColor()
Return String.Empty
ElseIf index = 0 AndAlso paths.Item(skip) = "../" Then
Return DirectoryExplorer(root.Parent)
ElseIf index > -1 Then
Console.Clear()
Dim item_index As Integer = skip + index + If(input.Modifiers = ConsoleModifiers.Shift, 12, 0)
Return DirectoryExplorer(New IO.DirectoryInfo(IO.Path.Combine(root.FullName, paths.Item(item_index))))
Else
Return DirectoryExplorerInvalidInput(root, skip, paths.ToArray)
End If
End Function
Private Function DirectoryExplorerInvalidInput(ByVal root As IO.DirectoryInfo, ByVal skip As Integer, ByVal paths() As String) As String
Console.Clear()
'Print the title in green
Console.ForegroundColor = ConsoleColor.Green
Console.WriteLine("File Explorer: {0}", root.FullName)
'Keep track of the F code and the paths
Dim command As String = "F"
Dim counter As Integer = 1
'Change the color to white
Console.ForegroundColor = ConsoleColor.White
'Iterate through the displayed paths
For Each path As String In paths.Skip(skip).Take(24)
'Print the command followed by the name
Console.WriteLine(" {0}{1}: {2}", command, counter, path)
'Increment the counter
counter += 1
If counter = 13 Then
'Change the shift code and input the counter
command = "Shift+F"
counter = 1
End If
Next
'Print if there will be a blank page
If skip + 24 < paths.Count - 1 Then
Console.WriteLine(" --Hit the ENTER key to display the next page of results")
End If
'Display the escape option
Console.WriteLine()
Console.ForegroundColor = ConsoleColor.Magenta
Console.WriteLine("Hit the ESC key to return nothing.")
'Prompt for the input
Dim input As ConsoleKeyInfo = Console.ReadKey
Dim fkeys() As ConsoleKey = {ConsoleKey.F1, ConsoleKey.F2, ConsoleKey.F3, ConsoleKey.F4, ConsoleKey.F5, ConsoleKey.F6, ConsoleKey.F7, ConsoleKey.F8, ConsoleKey.F9, ConsoleKey.F10, ConsoleKey.F11, ConsoleKey.F12, ConsoleKey.F13, ConsoleKey.F14, ConsoleKey.F15, ConsoleKey.F16, ConsoleKey.F17, ConsoleKey.F18, ConsoleKey.F19, ConsoleKey.F20, ConsoleKey.F21, ConsoleKey.F22, ConsoleKey.F23, ConsoleKey.F24}
Dim index As Integer = Array.IndexOf(fkeys, input.Key)
If input.Key = ConsoleKey.Enter AndAlso input.Modifiers = ConsoleModifiers.Shift AndAlso skip + 24 < paths.Count - 1 Then
Return DirectoryExplorer(root, skip + 24)
ElseIf input.Key = ConsoleKey.Enter Then
Return root.FullName
ElseIf input.Key = ConsoleKey.Escape Then
Console.Clear()
Return String.Empty
ElseIf index = 0 AndAlso paths(skip) = "../" Then
Return DirectoryExplorer(root.Parent)
ElseIf index > -1 Then
Dim item_index As Integer = skip + index + If(input.Modifiers = ConsoleModifiers.Shift, 12, 0)
Console.Clear()
Return DirectoryExplorer(New IO.DirectoryInfo(IO.Path.Combine(root.FullName, paths(item_index))))
Else
Return DirectoryExplorerInvalidInput(root, skip, paths.ToArray)
End If
End Function
End Module
-
Feb 3rd, 2018, 09:40 AM
#21
Re: Multithreading with a Do Loop
I just don't think it's going to work. I've tried two or three different approaches and the closest I think I can come involves bootstrapping a hidden Form so there's something for VB to point the message loop at. Part of the problem is how WinForms thread marshalling requires a Control to establish the context. The only ideas I have at this point are to explore API and see about hosting a console in a WinForms app, but it'd be a lot easier to just write a WinForms app in the first place. You'd already be done if you took that path.
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
-
Feb 3rd, 2018, 11:25 AM
#22
Thread Starter
Junior Member
Re: Multithreading with a Do Loop
The only problem with this is that, its a bit to slow because i need to make like 1 requests for each site with proxies. so isnt it possible to use background threads but DO await on them so i can continue with my code and show a notify? Thanks.
-
Feb 3rd, 2018, 12:25 PM
#23
Re: Multithreading with a Do Loop
Yes. It's easier than ever in more modern VB versions (2015 onwards for sure, maybe 2013.)
The problem with this in the past has been it's illegal to mess with the UI from worker threads, so you have to write code that acts like a secret tunnel between the two threads. The worker thread does work, gets a result, puts the result somewhere, then says "Hey, UI, you need to do something with that result!"
The Task API was built to coordinate with the Async and Await keywords to make that easy. To use them, the Sub that contains the code has to use the keyword Async as part of its definition. (We'll see that later.) Then, when you call something that returns a Task with a result, you use the Await keyword to call it. That does some magic. Let's talk about it via code.
Let's pretend I have a function to tell if a server is alive and returns a True/False result. You'll see it look like this in my documentation:
Code:
Function IsAliveAsync(ByVal url As String) As Task(Of Boolean)
Because it returns Task(Of Boolean), this is how you're supposed to read this sentence:
I am a function that returns a promise of a future Boolean. When you call me, I will hand you that promise in the form of a Task. I will start a worker thread that determines if a server is alive. When I finish, the Task will be able to give you my result.
So let's say you want to check just one server in a button click handler. You could write this:
Code:
Private Async Sub Button1_Click(...) Handles ...
Dim url As String = "..."
Try
Dim isAlive = Await IsAliveAsync(url)
Dim message = $"Server {url} is alive? {isAlive}"
txtResults.Text = message
Catch ex As Exception
txtResults.Text = $"Error accessing {url}."
End Catch
End Sub
Let's talk about the magic that happens. When you compile this code, here's what the compiler turns it into:
Code:
Private Sub Button1_Click(...) Handles ...
Dim url As String = "..."
Dim future As Task(Of Boolean) = IsAliveAsync(url)
future.ContinueWith(AddressOf WhenFinished, <on the UI thread>)
End Sub
SuperPrivate Sub WhenFinished(ByVal isAliveTask As Task(Of Boolean))
Dim message As String = ""
Try
Dim isAlive = isAliveTask.Result
message = $"Server {url} is alive? {isAlive}"
Catch ex As Exception
message = $"Error accessing {url}."
End Catch
txtResults.Text = message
End Sub
It "cuts" the method into two parts: the part before the Await and the part after. The ContinueWith() method says, "When this Task is finished, please execute this method." You can tell ContinueWith() that it should do that on the UI thread via some configuration that doesn't matter here. So, what this does is:
- Start a worker thread that checks the server status.
- When that task finishes, it will produce a Boolean or an Exception.
- Send that Boolean/Exception back to the UI thread.
- Update the UI with the result.
We used to always have to write it the longer way, but as you can see Async/Await makes it faster.
The compiler's really smart about how it makes its cuts. It's OK to make the "cut" inside a loop, it will do the right thing. So if you want to check many servers, you could:
Code:
Private Async Sub Button1_Click(...) Handles ...
Dim urls() As String = { ... }
For Each url In urls
Dim message As String = ""
Try
Dim isAlive = Await isAliveTask.Result
message = $"Server {url} is alive? {isAlive}"
Catch ex As Exception
message = $"Error accessing {url}."
End Catch
txtResults.Text = message
Next
End Sub
I have some ideas for how you could check multiple servers at once, but I'm not at an actual Windows machine and I'm worried about writing that code without testing it. It'll be Monday before I can make an attempt, maybe someone else will get there first.
This answer is wrong. You should be using TableAdapter and Dictionaries instead.
Posting Permissions
- You may not post new threads
- You may not post replies
- You may not post attachments
- You may not edit your posts
-
Forum Rules
|
Click Here to Expand Forum to Full Width
|