Printable View
I've always found the topic of Multi threading somewhat too involved in VB.
I have a BIG loop of items that process a batch of files. Normally takes 30+ mins to complete.
Would be nice to be able to figure out how to enable MultiThreading in VB so I could speed up this loop.
Any templates to get me started?
Before you dive into multithreading, I'd try to optimize "the loop" in single-threaded mode first.Quote:
Originally Posted by some1uk03
And since you talk about files, a few more infos would be good as e.g.:
- the total amount of files your current 30min-loop is touching
- the average size of a single file (in case you "go through a files content")
- and the type of file - or operation you perform (on said file - as e.g. "word-parsing on text-files")
This would help, to give better advice (whether threading is helpful at all, or whether single-threaded optimization can help).
Olaf
For something that heavy you may as well use a separate process anyway.
To expand on this a little further, I have a list of 10k+ items.
So a custom class/byteArray of 10k Items with a file size averaging 100kb to 1mb per item.
So the loop goes on one by one to "process" these items.
By process, I mean it goes through a custom Function to do some processing..... let's assume it's something similar to ZIPPing/Encoding the data.
I've already tried to optimise the processing as much as I could and loop takes an average of 1.5+ secs per file.
It sounds like the bulk of that 1.5 seconds might be spent in the CPU. If that is the case, then threading will not gain you as much as you might expect. Threading really shines brightest when each thread will spend a fair amount of time waiting for something. Reading and writing the file may or may not count, but zipping/encoding is more of a processor dependent action. There won't be any waiting to speak of on any thread. That would mean that, if the number of threads gets higher than the number of available execution cores (which may well be less than the number of cores in the CPU), then the threads will slow down the overall time as the cost of switching contexts begins to add up.
I don't know how well VB6 handles thread pooling, but if you have to do that yourself, you need to be careful how many threads you spin up, or you'll see total performance go down rather than up. A language with more tools for threading would allow you to fire off threads (or something like them, such as Tasks in .NET) for all the files all at once, but it won't launch all those threads. Instead, it would stage them all, but keep the number of actual threads down to whatever number is optimal for performance (or as best as can be guessed at by the language and/or OS). As each thread finished, it would get the next task. So, the number of actual threads could be one or more, and the benefit of threading would differ on different hardware.
You are probably wanting your single app to truly multithread multiple tasks specified within it however if you have a bunch of tasks specified in multiple batch files or other programs you can run them concurrently by just letting the (multitasking) Windows OS do it for you. e.g;
Personally I use something like this to compile a bunch of about 20 .vbp files to exes. If I use a simple batch file and compile the projects in serial fashion the process takes about 25 seconds, doing it with code like the above takes about 7 seconds.Code:Option Explicit
Public Declare Function OpenProcess Lib "kernel32" (ByVal dwDesiredAccess As Long, ByVal bInheritHandle As Long, ByVal dwProcessId As Long) As Long
Public Declare Function WaitForInputIdle Lib "user32" (ByVal hProcess As Long, ByVal dwMilliseconds As Long) As Long
Public Declare Function WaitForSingleObject Lib "kernel32" (ByVal hHandle As Long, ByVal dwMilliseconds As Long) As Long
Public Declare Function CloseHandle Lib "kernel32" (ByVal hObject As Long) As Long
Public Const INFINITE = &HFFFF, SYNCHRONIZE = &H100000
Sub Main()
'no Form is required
Dim Pid(0 To 3) As Long, hProc As Long
Dim i As Integer, t As Single
t! = Timer
'Run the batch files concurrently, each in its own hidden Cmd window and return the Process id for each
'Full paths to the Batch files are required of course
Pid(1) = Shell(Environ$("COMSPEC") & " /C " & "BatchFile1.bat", vbHide)
Pid(2) = Shell(Environ$("COMSPEC") & " /C " & "BatchFile2.bat", vbHide)
Pid(3) = Shell(Environ$("COMSPEC") & " /C " & "BatchFile3.bat", vbHide)
'Prevent THIS app from closing until the last process has completed
For i = 1 To UBound(Pid)
hProc = OpenProcess(SYNCHRONIZE, 0&, Pid(i))
If hProc <> 0& Then
WaitForInputIdle hProc, INFINITE
WaitForSingleObject hProc, INFINITE
CloseHandle hProc
End If
Next
MsgBox "That took " & Timer - t & " seconds"
End Sub
Shaggy Hiker: Essentially it is heavy CPU based. But essentially these numbers are quick with my PC and I have a pretty decent fast PC. So in comparison with an average PC, multiply my numbers by 2 or 3.
Oh and there is no file read/write. It's all in Memory.
Magic Ink: These are not external files that I'm dealing with. All these items are loaded within the APP/MEMORY.
Essentially, think of it as an Asynchronous Download Manager. Where you can download multiple files with separate progress bars etc...
It's the same methodology. I have a list of 10k Items and loop that iterates through them, processing each item through a PROCESSING Function.
So the Idea is to be able to simultaneously pass a number of these via the Processing Function, rather than one by one.
I have posted in the CodeBank a multithreading example: http://www.vbforums.com/showthread.p...Multithreading
Also, you can find another simple version in post #9 in that thread
Ax-DLL for easy to use Multithreading:
http://www.vbforums.com/showthread.p...ultithreading)
With SxS manifest can be used so no registration is needed.
As dilettante already mentioned, what about shelling several instances of the program (with some command) or of a helper program?
It could be much easier than threading.
best VB6 CreateThread by Chinese(support ocx,Support UserControl)-VBForums
http://www.vbforums.com/showthread.p...t-UserControl)
best VB6 CreateThread by Chinese(support ocx,Support UserControl)-VBForums
http://www.vbforums.com/showthread.p...t-UserControl)
Attachment 173919
if your cpu like i5 8600k(6 cpu core，Maybe you can you 6 thread or 12 thread)
you have 10000 plan,put in arr planlist(10000) as string
so 6 thread use 6 plan,when one thread finished,use plan 7.
so your cpu will be used for best.
yor also can view taskmgr.exe(if your cpu only use 30%),you can use 18 threads
you can reply in my thread :http://www.vbforums.com/showthread.p...t-UserControl)
Double posting is a problem on this forum. If you get a message saying that you have to wait N seconds before posting again, don't do it. Your post has already gone through, and if you do wait a few seconds, you'll end up with two posts.
One alternative is to hit Go Advanced. Posting from that form is not quite as convenient as the Quick Reply, but it does not double post.
Hi,Quote:
Originally Posted by xiaoyao
I was just going through your Code.
It seems to be creating a NEW Thread on EACH loop!
Shouldn't there be a limit as to how many threads you want running depending on your CPU?Code:
For i = 1 To 300
c.CreateThread "additem", Now
Next
Like set MAX Threads = getCPUThreadcount ?
you can see taskmgr.exe ,how much cpu used x%。。Quote:
Originally Posted by some1uk03
in fact，i like createthread(*,*,addressof sub1),without class
Code:Private Sub Command3_Click()
CreateNewThread AddressOf ThreadTest, 33&, 0&
End Sub
Public Sub ThreadTest(ByVal lpParam As Long)
InitFun
msgbox "put your code here in a *.bas file"
ExitFun
End Sub
Quote:
Originally Posted by The trick
The Trick: I've been exploring your JULIA example. Looks great. 1 Module does all that I need :)
Here's a question though. Once compiled, i can see it all working fine and sped up quite fast running a 4 thread cycle.
However, in task manager, you don't get to see the individual Threads ?
Any reasonable reason to this?
https://i.ibb.co/CBFF793/julia.pngQuote:
The Trick: I've been exploring your JULIA example. Looks great. 1 Module does all that I need
Here's a question though. Once compiled, i can see it all working fine and sped up quite fast running a 4 thread cycle.
However, in task manager, you don't get to see the individual Threads ?
Any reasonable reason to this?
Just my two-cents.
It's a bit of a different situation, but I had to develop a color smoothing algorithm about a year ago. I needed it to run rapidly, because the user might tweak, and run it over and over. When it was inline in my code, it took 5 or 6 seconds, which was annoying.
I shoved it all into an ActiveX DLL, and ticked all the Advanced optimizations, and I was amazed at how much faster it ran. It decreased the time to 1/3rd, at least.
So, there's that.
---
Regarding multi-threading, I've gotten "test" projects to successfully run on multiple occasions. However, for VB6, I've never gotten comfortable enough with it to trust it for distribution to users. Also, I realized that there's little downside to just spawning an entirely new process (i.e., a separate program). This (sort of) solves all the marshaling issues. However, you've still got to work out a fast mode of inter-process communication. I typically find (or create) some window (form) to subclass to receive the messages. Sending messages to another form doesn't require subclassing, just the use of SendMessageTimeout. I can post code I use to do that, but it's not difficult.
Then, all you've got to work out is the handshaking protocols between the two processes, and that's entirely up to you.
And, as a check on someone running this secondary process (program) from the command line, you might put some quasi-secret command line argument in it, and not load the inter-process-communications-form if it's not there.
---
Anyway, that's my two-cents.
more times,you can use only one standard exe for simple createthread,no need activex dll,no need activex.exe.
You just can create an instance of a COM class and call the methods directly between processes. Moreover you can call the methods between different machines or cross-bitness applications, scripts like vbs etc. The COM does all the job.Quote:
Regarding multi-threading, I've gotten "test" projects to successfully run on multiple occasions. However, for VB6, I've never gotten comfortable enough with it to trust it for distribution to users. Also, I realized that there's little downside to just spawning an entirely new process (i.e., a separate program). This (sort of) solves all the marshaling issues. However, you've still got to work out a fast mode of inter-process communication. I typically find (or create) some window (form) to subclass to receive the messages. Sending messages to another form doesn't require subclassing, just the use of SendMessageTimeout. I can post code I use to do that, but it's not difficult.
Then, all you've got to work out is the handshaking protocols between the two processes, and that's entirely up to you.
And, as a check on someone running this secondary process (program) from the command line, you might put some quasi-secret command line argument in it, and not load the inter-process-communications-form if it's not there.