Results 1 to 23 of 23

Thread: Multithreading with a Do Loop

  1. #1

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Multithreading with a Do Loop

    So i want a sub() that is multithreaded with like 100 threads and do a Do until loop. i want to console.writeline("Do loop is done.") after that whole sub is done. does anybody know how to do this? i used to have background threads but if i do a async sub with task.waitall() the background threads **** with it. anybody can give me a real simple example code?

  2. #2
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,297

    Re: Multithreading with a Do Loop

    You're going to have to provide more information. A Do Until loop exists to perform an action until a particular condition is True but that becomes incoherent if you're executing multiple iterations simultaneously. Please explain what you're trying to achieve, rather than how you're trying to achieve it.

  3. #3
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Multithreading with a Do Loop

    You won't get 100 threads anyways, unless each thread spends most of its time waiting for something. You can't get more processes running efficiently than you have processor cores to work on. If the threads are waiting, then others will get spun up, but if the threads are busy, you'd get one per core, tops. You can create 100 threads, they just won't run simultaneously.
    My usual boring signature: Nothing

  4. #4

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    Quote Originally Posted by Shaggy Hiker View Post
    You won't get 100 threads anyways, unless each thread spends most of its time waiting for something. You can't get more processes running efficiently than you have processor cores to work on. If the threads are waiting, then others will get spun up, but if the threads are busy, you'd get one per core, tops. You can create 100 threads, they just won't run simultaneously.
    I will need to do some multithreading Do until loadedfile.count = 0 so i need to do a process with every line in a file.

  5. #5

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    Quote Originally Posted by jmcilhinney View Post
    You're going to have to provide more information. A Do Until loop exists to perform an action until a particular condition is True but that becomes incoherent if you're executing multiple iterations simultaneously. Please explain what you're trying to achieve, rather than how you're trying to achieve it.
    Sorry but the replies are with different people but, Shaggy how would you do it then? the stage per item will be like 10-20 seconds probably so i will need to do it multithreaded.

  6. #6
    Super Moderator Shaggy Hiker's Avatar
    Join Date
    Aug 2002
    Location
    Idaho
    Posts
    38,988

    Re: Multithreading with a Do Loop

    Oh, that's a fine approach. Start as many Tasks as you want. The thread scheduler will decide how many threads will actually be spun up. If the existing threads end up waiting a long time, it will spin up a new thread such that the processor is fully occupied. That is handled by the OS, and it does to pretty efficiently, so you don't need to worry about it. My point was just that you likely won't have 100 threads, nor should you try to force that many threads. Just fire off the tasks and let the scheduler schedule them as processor time becomes available. What you've done sounds reasonable, and a suitable use of threads, perhaps even an ideal use. In the original post, it wasn't clear whether you were testing threading itself, or using threading to do some real work. Since it's clearly the latter, it sounds good.

    I think you'd have to explain the problem you are seeing a bit more....without the asterisks, as profanity is against the AUP.
    My usual boring signature: Nothing

  7. #7

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    Woops sorry, well the problem i encounter is that i want after that process is done open up a savefiledialog and open up a notify when that sub() is done, but when i do that after the Do loop all the threads will show up a notify and i will receive an STA error because i cant open up a savefiledialog with a background thread. what i have right now is:
    Code:
    thrednum = "100"
    For i = 0 To thrednum
                    Dim trd = New Thread(Sub()
                                             Do Until loadedfile.Count = 0
    
                                             Loop
    
                                         End Sub) With {
                        .IsBackground = True
                                         }
                    threads.Enqueue(trd)
                    trd.Start()
    
                Next
    '

    when i put it after the "Next" i get the message after the program generated all the threads, but the threads are still going you know. so my question is, where should i put the notify and the savefiledialog? because im really stuck right now...

  8. #8
    Super Moderator jmcilhinney's Avatar
    Join Date
    May 2005
    Location
    Sydney, Australia
    Posts
    110,297

    Re: Multithreading with a Do Loop

    Quote Originally Posted by SoldierCrimes View Post
    I will need to do some multithreading Do until loadedfile.count = 0 so i need to do a process with every line in a file.
    And that's exactly why you need to provide a proper explanation. A Do loop is the wrong choice in that case, even without multithreading. Logically, you want to perform an action for each line in the file. It should be obvious that a For Each loop is the appropriate choice there. If you want to parallelise that then you would simply call Parallel.ForEach.

  9. #9
    Super Moderator dday9's Avatar
    Join Date
    Mar 2011
    Location
    South Louisiana
    Posts
    11,711

    Re: Multithreading with a Do Loop

    So let's assume you have a long running task that reads a file, does *something* for each line, and then finishes by launches a SaveFileDialog. If that is the case, then a simple Async method would work, then piggy-backing off of JMcIlhinney's suggestion you could also implement Parallel.ForEach:
    Code:
    Private Async Sub DoSomething(ByVal path As String)
        Await Task.Run(Sub()
                           'Get each line from the file
                           Dim lines() As String = IO.File.ReadAllLines(path)
    
                           'Iterate through each line
                           Parallel.ForEach(lines, Sub(line As String)
                                                       'do "something"
                                                       Threading.Thread.Sleep(500) 'Zzz... (remove from development)
                                                   End Sub)
                       End Sub)
    
        Using sfd As SaveFileDialog = New SaveFileDialog
            With sfd
                'Properties
                .Filter = "*.txt|*.txt"
                .InitialDirectory = Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments)
                'etc.
    
                'Display
                If .ShowDialog = DialogResult.OK Then
                    'Do something
                End If
            End With
        End Using
    End Sub
    "Code is like humor. When you have to explain it, it is bad." - Cory House
    VbLessons | Code Tags | Sword of Fury - Jameram

  10. #10

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    Thanks for that man! The thing is working right now except the savefiledialog, that still gives me an error when i try to open it. (again an STA error.)
    Name:  857da8285296aabb918d986a8a8c5748.jpg
Views: 273
Size:  10.2 KB
    Last edited by SoldierCrimes; Jan 31st, 2018 at 02:10 AM. Reason: Error added

  11. #11
    You don't want to know.
    Join Date
    Aug 2010
    Posts
    4,578

    Re: Multithreading with a Do Loop

    The error indicates you're either trying to launch the dialog from a worker thread, or you aren't in a Windows Forms application and maybe your project template's configured weird. I'm betting more on "wrong thread", because Async/Await can only get you back to the UI thread if you call the method from the UI thread.

    We simply can't offer advice if we can't see the bulk of your code.
    This answer is wrong. You should be using TableAdapter and Dictionaries instead.

  12. #12

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    The code that dday9 gave me, its command line but with imported windows forms.
    Last edited by SoldierCrimes; Jan 31st, 2018 at 12:09 PM. Reason: Wrong person mentioned.

  13. #13
    You don't want to know.
    Join Date
    Aug 2010
    Posts
    4,578

    Re: Multithreading with a Do Loop

    I can reproduce it, but I don't think the solution is going to be easy. Async/Await doesn't work quite the same in a console application, and a lot of things you need guarantees for in Windows Forms are not easy to maintain in a console application.

    Here's the summary:

    Windows Forms needs a few things to be true. It needs a message loop to be established, and it needs that message loop established on a thread that is marked STA like the exception complains yours isn't. I don't fully understand what that means, but I know it's true. All of this matters because a ton of assumptions Windows Forms makes get broken if you ever try to interact with the message loop from a different thread. So we say controls have "thread affinity" and a myriad of symptoms arise if you break the rules. To help you with this, Windows Forms has a concept of "marshalling", a process by which you can tell a control to execute some code on the correct thread. But if you don't have a control, you can't get to that thread. (WPF has a better mechanism!)

    Console applications do not establish a message loop. That may or may not trigger the exception that mentions needing an STA thread; it could be a generalized handler for a lot of things goign wrong. Console applications also don't have thread affinity. That means they have no mechanism like Windows Forms for guaranteeing anything happens on a particular thread. We call this a "free threading" model.

    Async/Await relies on a concept called the "synchronization context" to make sure it comes back to the original thread. In Windows Forms, this works because if you are on the "UI thread", there is infrastructure for the Task API to get back onto it when needed. In a Console application, none of that infrastructure is there, so when you come back from Await you can be on a different thread from the original.

    So even if your code sets up a message loop in the console app's main thread, Async/Await is unstable because there's not a synchronization context to use for that thread. Maybe we can set one up, maybe not. I think it's just as likely the only safe way to use Windows Forms from a console application is to establish a Thread that runs the message loop and give it a mechanism by which you can marshal code to it. I'm not sure how to do that, which is why I'm not committing to a solution just yet.

    I sort of hope someone comes up with a different approach before I get too deep, but I think in general "console applications shouldn't use Windows Forms" is good advice.
    This answer is wrong. You should be using TableAdapter and Dictionaries instead.

  14. #14

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    Totally true, and again thanks for the great explanations! But the savefiledialog is just so sexy if you want to keep it clean and dont ask for: Console.writeline("Please enter the file path: ") or something like that. i hope this can be done in an easy way. anyways thanks for helping sir!

  15. #15

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    Does anybody else can help me? im still struggling with the Savefiledialog...

  16. #16
    Super Moderator dday9's Avatar
    Join Date
    Mar 2011
    Location
    South Louisiana
    Posts
    11,711

    Re: Multithreading with a Do Loop

    You're trying to force a square peg into a round hole; console applications are not designed to use UI components. Let's take a different approach, why don't you tell us what the overall scope of your project is so that we can help you determine what type of application you should be focusing on.
    "Code is like humor. When you have to explain it, it is bad." - Cory House
    VbLessons | Code Tags | Sword of Fury - Jameram

  17. #17

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    I'm trying to develop a proxy checker, so i let the user scrape proxies after that save them in a filedialog
    so what the program does is just go to a couple of sites with a specific regex to get the proxies and save them in a list. but all those request take some time thats why i need it multithreaded. so after that process is done i want to open a savefiledialog where the user can save the proxies in a text file. and thats where the error happens.

  18. #18
    Super Moderator dday9's Avatar
    Join Date
    Mar 2011
    Location
    South Louisiana
    Posts
    11,711

    Re: Multithreading with a Do Loop

    Does the user specify the websites and RegEx?
    "Code is like humor. When you have to explain it, it is bad." - Cory House
    VbLessons | Code Tags | Sword of Fury - Jameram

  19. #19

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    Nope, the regex is standard and i have all the sites added in a list in my program.

  20. #20
    Super Moderator dday9's Avatar
    Join Date
    Mar 2011
    Location
    South Louisiana
    Posts
    11,711

    Re: Multithreading with a Do Loop

    I still don't think that you should try to implement a UI control in a console application. For example, take a look at this makeshift directory explorer I made for a console application integrated with your need:
    Code:
    Public Module Module1
    
        Public Sub Main()
            'Get each line from the file
            Dim lines() As String = IO.File.ReadAllLines(IO.Path.Combine(Environment.GetFolderPath(Environment.SpecialFolder.MyDocuments), "example.txt"))
    
            'Iterate through each line
            Parallel.ForEach(lines, Sub(line As String)
                                        'do "something"
                                        Threading.Thread.Sleep(500) 'Zzz... (remove from development)
                                    End Sub)
    
            'Save results
            Dim directory As String = DirectoryExplorer(New IO.DirectoryInfo("C:/"))
            Console.ResetColor()
            Console.WriteLine("Please enter your file name")
            Console.Write(directory & "\")
            Dim file As String = Console.ReadLine
    
            'Do something with the file
            Dim path As String = IO.Path.Combine(directory, file)
    
        End Sub
    
        Private Function DirectoryExplorer(ByVal root As IO.DirectoryInfo, Optional ByVal skip As Integer = 0) As String
            Console.Clear()
    
            'Print the title in green
            Console.ForegroundColor = ConsoleColor.Green
            Console.WriteLine("File Explorer: {0}", root.FullName)
    
            'Keep track of the F code and the paths
            Dim command As String = "F"
            Dim counter As Integer = 1
            Dim paths As List(Of String) = New List(Of String)
    
            'Add the directory-up command if applicable
            If root.Parent IsNot Nothing Then
                paths.Add("../")
            End If
    
            'Get each directory first
            paths.AddRange(root.EnumerateDirectories.Select(Function(dir) dir.Name))
    
            'Change the color to white
            Console.ForegroundColor = ConsoleColor.White
    
            'Iterate through the displayed paths
            For Each path As String In paths.Skip(skip).Take(24)
                'Print the command followed by the name
                Console.WriteLine("  {0}{1}: {2}", command, counter, path)
    
                'Increment the counter
                counter += 1
    
                If counter = 13 Then
                    'Change the shift code and input the counter
                    command = "Shift+F"
                    counter = 1
                End If
            Next
    
            'Print if there will be a next page
            If skip + 24 < paths.Count - 1 Then
                Console.WriteLine("  --Use the key combination SHIFT + ENTER key to display the next page of results")
            End If
    
            'Print if there will be a previous page
            If skip > 0 Then
                Console.WriteLine("  --use the key combination SHIFT + ESC key to display the previous page of results")
            End If
    
            'Display the options
            Console.WriteLine()
            Console.ForegroundColor = ConsoleColor.Magenta
            Console.WriteLine("Hit the ENTER key to return the current directory.")
            Console.WriteLine("Hit the ESC key to return nothing.")
    
            'Prompt for the input
            Dim input As ConsoleKeyInfo = Console.ReadKey
            Dim fkeys() As ConsoleKey = {ConsoleKey.F1, ConsoleKey.F2, ConsoleKey.F3, ConsoleKey.F4, ConsoleKey.F5, ConsoleKey.F6, ConsoleKey.F7, ConsoleKey.F8, ConsoleKey.F9, ConsoleKey.F10, ConsoleKey.F11, ConsoleKey.F12, ConsoleKey.F13, ConsoleKey.F14, ConsoleKey.F15, ConsoleKey.F16, ConsoleKey.F17, ConsoleKey.F18, ConsoleKey.F19, ConsoleKey.F20, ConsoleKey.F21, ConsoleKey.F22, ConsoleKey.F23, ConsoleKey.F24}
            Dim index As Integer = Array.IndexOf(fkeys, input.Key)
    
            If input.Key = ConsoleKey.Enter AndAlso input.Modifiers = ConsoleModifiers.Shift AndAlso skip + 24 < paths.Count - 1 Then
                Return DirectoryExplorer(root, skip + 24)
            ElseIf input.Key = ConsoleKey.Enter Then
                Return root.FullName
            ElseIf input.Key = ConsoleKey.Escape AndAlso input.Modifiers = ConsoleModifiers.Shift Then
                Return DirectoryExplorer(root, skip - 24)
            ElseIf input.Key = ConsoleKey.Escape Then
                Console.Clear()
                Console.ResetColor()
                Return String.Empty
            ElseIf index = 0 AndAlso paths.Item(skip) = "../" Then
                Return DirectoryExplorer(root.Parent)
            ElseIf index > -1 Then
                Console.Clear()
                Dim item_index As Integer = skip + index + If(input.Modifiers = ConsoleModifiers.Shift, 12, 0)
                Return DirectoryExplorer(New IO.DirectoryInfo(IO.Path.Combine(root.FullName, paths.Item(item_index))))
            Else
                Return DirectoryExplorerInvalidInput(root, skip, paths.ToArray)
            End If
        End Function
    
        Private Function DirectoryExplorerInvalidInput(ByVal root As IO.DirectoryInfo, ByVal skip As Integer, ByVal paths() As String) As String
            Console.Clear()
    
            'Print the title in green
            Console.ForegroundColor = ConsoleColor.Green
            Console.WriteLine("File Explorer: {0}", root.FullName)
    
            'Keep track of the F code and the paths
            Dim command As String = "F"
            Dim counter As Integer = 1
    
            'Change the color to white
            Console.ForegroundColor = ConsoleColor.White
    
            'Iterate through the displayed paths
            For Each path As String In paths.Skip(skip).Take(24)
                'Print the command followed by the name
                Console.WriteLine("  {0}{1}: {2}", command, counter, path)
    
                'Increment the counter
                counter += 1
    
                If counter = 13 Then
                    'Change the shift code and input the counter
                    command = "Shift+F"
                    counter = 1
                End If
            Next
    
            'Print if there will be a blank page
            If skip + 24 < paths.Count - 1 Then
                Console.WriteLine("  --Hit the ENTER key to display the next page of results")
            End If
    
            'Display the escape option
            Console.WriteLine()
            Console.ForegroundColor = ConsoleColor.Magenta
            Console.WriteLine("Hit the ESC key to return nothing.")
    
            'Prompt for the input
            Dim input As ConsoleKeyInfo = Console.ReadKey
            Dim fkeys() As ConsoleKey = {ConsoleKey.F1, ConsoleKey.F2, ConsoleKey.F3, ConsoleKey.F4, ConsoleKey.F5, ConsoleKey.F6, ConsoleKey.F7, ConsoleKey.F8, ConsoleKey.F9, ConsoleKey.F10, ConsoleKey.F11, ConsoleKey.F12, ConsoleKey.F13, ConsoleKey.F14, ConsoleKey.F15, ConsoleKey.F16, ConsoleKey.F17, ConsoleKey.F18, ConsoleKey.F19, ConsoleKey.F20, ConsoleKey.F21, ConsoleKey.F22, ConsoleKey.F23, ConsoleKey.F24}
            Dim index As Integer = Array.IndexOf(fkeys, input.Key)
    
    
            If input.Key = ConsoleKey.Enter AndAlso input.Modifiers = ConsoleModifiers.Shift AndAlso skip + 24 < paths.Count - 1 Then
                Return DirectoryExplorer(root, skip + 24)
            ElseIf input.Key = ConsoleKey.Enter Then
                Return root.FullName
            ElseIf input.Key = ConsoleKey.Escape Then
                Console.Clear()
                Return String.Empty
            ElseIf index = 0 AndAlso paths(skip) = "../" Then
                Return DirectoryExplorer(root.Parent)
            ElseIf index > -1 Then
                Dim item_index As Integer = skip + index + If(input.Modifiers = ConsoleModifiers.Shift, 12, 0)
                Console.Clear()
                Return DirectoryExplorer(New IO.DirectoryInfo(IO.Path.Combine(root.FullName, paths(item_index))))
            Else
                Return DirectoryExplorerInvalidInput(root, skip, paths.ToArray)
            End If
        End Function
    
    End Module
    "Code is like humor. When you have to explain it, it is bad." - Cory House
    VbLessons | Code Tags | Sword of Fury - Jameram

  21. #21
    You don't want to know.
    Join Date
    Aug 2010
    Posts
    4,578

    Re: Multithreading with a Do Loop

    I just don't think it's going to work. I've tried two or three different approaches and the closest I think I can come involves bootstrapping a hidden Form so there's something for VB to point the message loop at. Part of the problem is how WinForms thread marshalling requires a Control to establish the context. The only ideas I have at this point are to explore API and see about hosting a console in a WinForms app, but it'd be a lot easier to just write a WinForms app in the first place. You'd already be done if you took that path.
    This answer is wrong. You should be using TableAdapter and Dictionaries instead.

  22. #22

    Thread Starter
    Junior Member
    Join Date
    Jan 2018
    Posts
    18

    Re: Multithreading with a Do Loop

    The only problem with this is that, its a bit to slow because i need to make like 1 requests for each site with proxies. so isnt it possible to use background threads but DO await on them so i can continue with my code and show a notify? Thanks.

  23. #23
    You don't want to know.
    Join Date
    Aug 2010
    Posts
    4,578

    Re: Multithreading with a Do Loop

    Yes. It's easier than ever in more modern VB versions (2015 onwards for sure, maybe 2013.)

    The problem with this in the past has been it's illegal to mess with the UI from worker threads, so you have to write code that acts like a secret tunnel between the two threads. The worker thread does work, gets a result, puts the result somewhere, then says "Hey, UI, you need to do something with that result!"

    The Task API was built to coordinate with the Async and Await keywords to make that easy. To use them, the Sub that contains the code has to use the keyword Async as part of its definition. (We'll see that later.) Then, when you call something that returns a Task with a result, you use the Await keyword to call it. That does some magic. Let's talk about it via code.

    Let's pretend I have a function to tell if a server is alive and returns a True/False result. You'll see it look like this in my documentation:
    Code:
    Function IsAliveAsync(ByVal url As String) As Task(Of Boolean)
    Because it returns Task(Of Boolean), this is how you're supposed to read this sentence:
    I am a function that returns a promise of a future Boolean. When you call me, I will hand you that promise in the form of a Task. I will start a worker thread that determines if a server is alive. When I finish, the Task will be able to give you my result.
    So let's say you want to check just one server in a button click handler. You could write this:
    Code:
    Private Async Sub Button1_Click(...) Handles ...
        Dim url As String = "..."
        Try
            Dim isAlive = Await IsAliveAsync(url)
            Dim message = $"Server {url} is alive? {isAlive}"
            txtResults.Text = message
        Catch ex As Exception
            txtResults.Text = $"Error accessing {url}."
        End Catch
    End Sub
    Let's talk about the magic that happens. When you compile this code, here's what the compiler turns it into:
    Code:
    Private Sub Button1_Click(...) Handles ...
        Dim url As String = "..."
        Dim future As Task(Of Boolean) = IsAliveAsync(url)
        future.ContinueWith(AddressOf WhenFinished, <on the UI thread>)
    End Sub
    
    SuperPrivate Sub WhenFinished(ByVal isAliveTask As Task(Of Boolean))
        Dim message As String = ""
        Try
            Dim isAlive = isAliveTask.Result
            message = $"Server {url} is alive? {isAlive}"
        Catch ex As Exception
            message = $"Error accessing {url}."
        End Catch
    
        txtResults.Text = message
    End Sub
    It "cuts" the method into two parts: the part before the Await and the part after. The ContinueWith() method says, "When this Task is finished, please execute this method." You can tell ContinueWith() that it should do that on the UI thread via some configuration that doesn't matter here. So, what this does is:
    1. Start a worker thread that checks the server status.
    2. When that task finishes, it will produce a Boolean or an Exception.
    3. Send that Boolean/Exception back to the UI thread.
    4. Update the UI with the result.

    We used to always have to write it the longer way, but as you can see Async/Await makes it faster.

    The compiler's really smart about how it makes its cuts. It's OK to make the "cut" inside a loop, it will do the right thing. So if you want to check many servers, you could:
    Code:
    Private Async Sub Button1_Click(...) Handles ...
        Dim urls() As String = { ... }
        For Each url In urls
            Dim message As String = ""
            Try
                Dim isAlive = Await isAliveTask.Result
                message = $"Server {url} is alive? {isAlive}"
            Catch ex As Exception
                message = $"Error accessing {url}."
            End Catch
    
            txtResults.Text = message
        Next
    End Sub
    I have some ideas for how you could check multiple servers at once, but I'm not at an actual Windows machine and I'm worried about writing that code without testing it. It'll be Monday before I can make an attempt, maybe someone else will get there first.
    This answer is wrong. You should be using TableAdapter and Dictionaries instead.

Posting Permissions

  • You may not post new threads
  • You may not post replies
  • You may not post attachments
  • You may not edit your posts
  •  



Click Here to Expand Forum to Full Width