Printable View
Niya, I really have to say, by far this is the-best breakdown I have ever seen of multithreading for vb.net. I was having so much issues trying to figure this out for my self. You put this in plain,easy to follow language. I am forever great-full! As you can see it is 5:30AM and I have not been to sleep and I felt the need to register for the forums just to tell you this, thank you very much.:D
Thank you friend....I'm glad it helped you :)
I thought it might be helpful to try and explain it in the way I would have liked someone to explain it to me when I didn't know how to do it. I seemed to have hit the mark :) Thx
Niya, Thank you for providing these examples. I don't know if your still following this thread (as the last post was almost 4 months ago). However in case you are, would you be so kind as to provide an identical example of your count program without utilizing your ThreadExtensions routine. I'm using VB2005 and it will not resolve some of the 'With' instructions you used in the routine. And please don't tell me to upgrade, there are reasons why I'm using VB2005 and .net framework 2.0, which I won't discuss as it is off topic. Threading is a something I haven't had a need to code until now - so I'm new at it. Any help would be appreciated.
What I'm trying to do is to create a second thread that runs continuously in an endless loop. The code in the loop will read about a dozen Boolean values set from the UI thread, and send them out through the serial port to another device (PLC), then turn around and read some other data back from the serial port. Once the data is read back, it is unpacked and then checked to see if any values have changed since the last read. If so, I want to raise a 'status change' event on the UI thread and pass to the event what changed (which class instance) and the new property value (which can be set from either thread). After raising the event (if necessary), the thread will loop back to the beginning and start the communications again (a monitor program). The UI thread will respond to the event and update the screen accordingly. I have the communications part working. I'm trying to work out the threading - your count program pretty much has the functions I need - only I'm unable to use parts of the ThreadExtensions routine as they will not resolve in VB2005. How else can I do this?
Thanks.
Use this one:-
vbnet Code:
Thanks a bunch Niya. That works perfectly in VB2005.
Dale
Hi Niya.
i have a small question! Can help me.
example: i have a url: (http://mangafox.me/manga/kingdom_hea...04/c030/1.html). And i wana get comic pictures in this page. Specifically, Downloading 3-4 or more pages in same time to speed up load.
only way I solved my problem.
In the section where I show how to do multi-threaded classes, I made a class that executed the count on a non-UI thread. In your case you would have a your download method instead of a count. You simply instantiate multiple classes to perform multiple downloads.
thanks Niya.
Hello Niya. Thanks for writing the article. I'm a beginner in VB so I could pickup first 2-3 posts and messed up afterwards.
Actually I have a list of files in my program and need to download 2 files simultaneously and on the completion of a file, it start downloading next in the list. and I'm just confused what to do and how to alter the code..
Will you help me please..
Does your problem have to do with multi-threading it or downloading the files ? Have you already got the downloading part working ?
downloading all files in a listbox one by one is not an issue, I used for next loop and it did the job. And also I like to mention that I used:
My.Computer.Network.Download()
I may use Webclient Download Method but thats not an issue at the moment. Multithreading is the issue..
Show me the code you're using to download the files.
here it is:
Code:Dim Dir As String = InputBox("Directory", "Dir")
For i As Integer = 0 To lstLinks.Items.Count - 1
Dim targetFile As String = lstLinks.Items(i).ToString
Dim destFile As String
destFile = "E:\Downloads\" & Dir & "\" & i & ".jpg"
Try
My.Computer.Network.DownloadFile(targetFile, destFile, "", "", True, 10000, True, FileIO.UICancelOption.DoNothing)
Catch ex As Exception
tbStatus.Text += vbNewLine & "ERROR: File: " & i & " ERR Number: " & Err.Number & ". Description: " & Err.Description & "File Name: " & targetFile
End Try
Next
I'm going to assume you're using VS2010 and later.
The simplest way to achieve what you want would be to put the downloading part in its own sub like this:-
vbnet Code:
And call it like:-
vbnet Code:
Now this is really a simple scenario. It doesn't account for progress or knowing when the downloads are finished but its the most straightforward way if you don't require any of the complexities of a more robust file download system.
Hello Niya, First of all Thanks for this delicious article. After reading your articles and reply here, I couldn't stop to ask.
I want to ask How can we pass multiple arguments on a thread.
Now As you have mentioned before that Thread.Start can accept only one parameter.. But why i am still asking because,
I have a Procedure which loads a .Txt file on a Listview.. in this function I pass .Txt file path and a Listview name.
Here I am not sure about how many times this procedure will be called. (It depends on number of .Txt files of a directory)
In other words, the procedure loads all the .Txt files of a directory on listviews.
Now my Query is, How to use ThreadPool to load all the .Txt files together. I mean instead of calling procedure one by one , how can i call this procedure to load the next .Txt file while the previous .Txt file is being loaded on the listview?
Here are the sample of codes i am talking about, Please have a look:
And Normally I am calling it like :Code:Public Sub LoadDB(ByVal bFile As String, ByVal LvName As ListView)
Try
Dim inputstream As New IO.StreamReader(bFile)
Dim newstr(5) As String
Dim ttt As String
Do While inputstream.Peek <> -1
Application.DoEvents()
ttt = inputstream.ReadLine().Trim
newstr = Split(ttt, vbtab)
With LvName.Items.Add(newstr(0).ToUpper)
.SubItems.Add(newstr(1).ToUpper)
.SubItems.Add(newstr(2).ToUpper)
.SubItems.Add(newstr(3).ToUpper)
.SubItems.Add(newstr(4).ToUpper)
End With
Loop
inputstream.Close()
Catch
End Try
End Sub
Even I am calling it like..Code:For i = 1 to numberOftxtFiles
LoadDB(fileDirpath & i.ToString, LvPhH1)
LoadDB(fileDirpath & i.ToString, LvPhH2)
Next i
Code:For i = 1 to numberOftxtFiles
Threading.ThreadPool.QueueUserWorkItem(Sub() LoadDB(fileDirpath & i.ToString, LvPhH1))
Threading.ThreadPool.QueueUserWorkItem(Sub() LoadDB(fileDirpath & i.ToString, LvPhH2))
Next i
But it's not working...
Thanks
Regards, :)
Sorry for dual posting.... Above code was not working due to Cross thread Problem..
Now after
CheckForIllegalCrossThreadCalls = False
it is working... Is it good to Disable CheckForIllegalCrossThreadCalls?
NO NO NO NO NO! Don't do that at all. You're begging for disasters in the future. I dedicated an entire paragraph towards explaining why you cannot call change control properties from a non-UI thread. I showed at least two ways to properly interact with controls from worker threads. One way by use the Invoke property of controls or by use of a SynchronizationContext object. I strongly suggest you go back and read those particulars again. You should NEVER EVER set CheckForIllegalCrossThreadCalls to False.Quote:
Originally Posted by green.pitch
Ok thanks Niya, I tried to invoke a control on another project... But i am getting error. Please have a look to the following code.
In above code I'm trying to load a form multiple times. Getting error on 'AddressOf LoadFrm'.Code:Private Sub LoadFrm(ByVal pIDD As String, ByVal pMsg As String)
memFrmNum = memFrmNum + 1
FrmObj(memFrmNum) = New Form2
If FrmObj(memFrmNum).InvokeRequired Then
FrmObj(memFrmNum).Invoke(New Action(Of String)(AddressOf LoadFrm), pIDD)
FrmObj(memFrmNum).Invoke(New Action(Of String)(AddressOf LoadFrm), pMsg)
Else
FrmObj(memFrmNum).TextBox1.Text = pIDD
FrmObj(memFrmNum).Label1.Text = pMsg
End If
FrmObj(memFrmNum).Show()
End Sub
Please check the attachment.
Attachment 102589
Regards,
The problem there is that you're are trying to invoke LoadFrm with Action(Of String) when you should be invoking with Action(Of String, String) because LoadFrm has two string parameters. However, you are targeting .Net 2.0 which doesn't seem to offer Action(Of T1, T2) so here is another way to correct it if you're using VS2010:-Quote:
Originally Posted by green.pitch
vbnet Code:
Oh ok thanks again Niya :) . I am very new in vb.net. Now the codes are not showing any error, but Form2 is still hanging after loading.. How can I solve this with Multi threading? Please check the attachment file of my last post.
After clicking button on the form1, Open notepad.exe... and you will found Form2 will be hanging up.
How can we do it with Control properties on framework above 2.0?
If you can make any correction on it, please kindly do it.
Thanks again,
regards
What is that app supposed to do ?
Its about a process gateway.. where if any new process will execute, app will prompt that new process name. .. It's for my personal use. And to clear my doubt, i thought it should be a good example to ask... :)Quote:
Originally Posted by Niya
It's incomplete yet, Please suggest if you have any better idea to do its task.
Thank you Niya
Regards,
Well you have me at a disadvantage here. I haven't written any apps yet that deal with processes in complex way so I know little on this subject. I'm trying to understand how threading fits in here and what it is that you're trying to do with those Forms
Well Ok, I'm not asking to work on processes. What about the Multi threading on Form2 ? How to make free that form which is hanging.. It can be done by multi threading.. :)Quote:
Originally Posted by Niya
What multi-threading on Form2 ? There is no code written inside Form2 at all.Quote:
Originally Posted by green.pitch
Niya, after reading this thread and forcefeeding :confused: myself on it, I registered--Just to thank you!:wave:
I don't mean to slight any other posters, just that this topic is currently important to me.
I made the decision to finally ween off VB6 after writing an app to read 50,000 INT16 values from an industrial controller.
Using VB6/Winsocks, it is taking about 90 seconds.
With vb.net/multithreading/ (and whatever TCP client methodology I choose) I hope to trim down to less than 30 seconds.
If the task itself is parallelizable then you would definitely see an improvement in performance especially on systems with multi-core processors. I don't really know anything about reading from controllers but if you can read these integers by starting at an index then you can break the list up into several buckets and assign a thread to read each. However, if the controller itself can only be read at the hardware level as a serial device then that will be a bottleneck as there it wouldn't actually be reading in parallel. There would be no improvement.
I commend your effort to move away from VB6 and into VB.Net as its a much more powerful development tool by leaps and bounds but I cannot say for certain that it would help in this particular case.
I already have a vb6 app reading the data. The controller registers are indexed.
The parallel bucket reference is exactly my approach in the VB6 app , but sadly, VB6 isn't suited to multithreading.
I make multiple run-time instances of the WinSock control, issuing multiple queries on the controller, but then process the return data of each WinSock 'sequentially.'
That appears to be the bottleneck.
The controller access is via ethernet connection. My initial concept was: multiple Operator Screens can make simultaneous connections to the controller,
so a pc app should be able to make multiple connections to the controller. Well, that concept has been proven - except for the VB6 limitations.
I have come here to read and learn First. Then ask for help later. So, thanks for the opportunity to 'read and learn!' Much appreciated.
Well I don't think you will get a speed improvement by using multiple connections. There is only one physical connection so the data being transmitted over these multiple TCP/IP connections will be multiplexed into a single stream to be transmitted across the wire, so it would only appear to be working in parallel in code. The sum total transfer rate of say five TCP connections will be the same transfer rate as a single connection.Quote:
Originally Posted by OnErrorGoAway
The only way to this could be truly parallel is if the controller itself allowed you to make multiple physical connections via multiple ethernet ports and cables. Then the device itself can truly work in parallel.
You're welcome :)Quote:
Originally Posted by OnErrorGoAway
Niya,
In a preceding post you said that you were not detailing your ThreadExtensions class. I got interested!
I have spent a couple days poring over those 14 lines, F8'ing through and also reading Nick's thread Here :check:, among others.
I haven't figured out all of the class and HOW it works, but I think that I have made progress in extending the parameter list.
I would appreciate if you could tell me if I am heading in the right direction, and if not, tell me so.
The changes do work, but if I have made a dumb mistake, the IDE didn't tell me. 0 Errors, 0 Warnings, 0 Messages
[VB2010 Express]
My changes are highlighted in green, explanations are bold italic
I used the DateTime because I got bored with simple Integers
Code:'Form1 code
Private Sub Button1_Click(ByVal sender As System.Object, ByVal e As System.EventArgs) Handles Button1.Click
Dim DT As DateTime = #8/13/2002 12:14 PM# ' DateTime from MSDN example - just to notice a change when returned
m_cn.CountAsync(600, 500, 50, DT) 'Changed Max to 600 and Added three new params (2 integer and one DateTime) - I did change Min/Max of ProgressBar1
End Sub
Private Sub m_cn_CountChanged(ByVal sender As Object, ByVal e As CountChangedEventArgs) Handles m_cn.CountChanged
ProgressBar1.Value = CStr(e.CurrentCount)
Label2.Text = CStr(e.CurrentCount)
Label3.Text = CStr(e.Mode) 'Show the new param as passed back
Label4.Text = CStr(e.DT) 'Show the new param as passed back
Label5.Text = CStr(e.Min) 'Show the new param as passed back
End Sub
Code:'Counter Class code
Public Sub CountAsync(ByVal Max As Integer, ByVal Min As Integer, ByVal Mode As Integer, ByVal DT As DateTime)
'>>------------------------------------ Change New Func Signature and Count Signature
ThreadExtensions.QueueUserWorkItem(New Func(Of Integer, Integer, Integer, DateTime, String)(AddressOf Count), Max, Min, Mode, DT)
End Sub
'Add new variables declarations to the Count signature
Public Function Count(ByVal Max As Integer, ByVal Min As Integer, ByVal Mode As Integer, ByVal DT As DateTime) As String
Dim startTime As DateTime = DateTime.Now
Dim e As CountChangedEventArgs
Dim msg As String
For i = 1 To Max
e = New CountChangedEventArgs(i, Max, Min, Mode, DT) 'Add return variables here
If context Is Nothing Then
OnCountChanged(e)
Else
ThreadExtensions.ScSend(context, New Action(Of CountChangedEventArgs)(AddressOf OnCountChanged), e)
End If
Threading.Thread.Sleep(200)
Next
msg = "Count took : " + (DateTime.Now - startTime).ToString
If context Is Nothing Then
OnCountCompleted(New CountCompletedEventArgs(msg))
Else
ThreadExtensions.ScSend(context, New Action(Of CountCompletedEventArgs)(AddressOf OnCountCompleted), New CountCompletedEventArgs(msg))
End If
Return msg
End Function
Thank you for your time and effort!Code:'CounterChangedEventArgs Class code
Private _Max As Integer
Private _Min As Integer ' Support first ADDED variable
Private _Mode As Integer ' Support second ADDED variable
Private _DT As DateTime ' Support third ADDED variable
Public Sub New(ByVal cc As Integer, ByVal max As Integer, ByVal min As Integer, ByVal Mode As Integer, ByVal DT As DateTime)
_CurrentCount = cc
_Max = max
_Min = min 'Get the new value to return to UI
_Mode = Mode 'Get the new value to return to UI
_DT = DateTime.Now 'Get the new value to return to UI
End Sub
'Finally, add support for the public properties
Public ReadOnly Property Min() As Integer
Get
Return _Min
End Get
End Property
Public ReadOnly Property Mode() As Integer
Get
Return _Mode
End Get
End Property
Public ReadOnly Property DT() As DateTime
Get
Return _DT
End Get
End Property
You did everything 100% correctly. If this were a test, you'd get an A++ :DQuote:
Originally Posted by OnErrorGoAway
Note that if you're using VS2010 that the ThreadExtensions class is not necessary because the VB2010 compiler gives you a Sub keyword for lambdas that don't return a value so you can use QueueUserWorkItem of the ThreadPool class directly and specify a Sub lambda to call your functions. Its far more terse that using ThreadExtensions.Quote:
Originally Posted by OnErrorGoAway
Eg: This:-
vbnet Code:
Can become:-
vbnet Code:
ThreadExtensions is necessary if you're using VS2008 which doesn't have a Sub keyword.
Thank you. You are too kind.Quote:
You did everything 100% correctly. If this were a test, you'd get an A++
Works as specified. Thank you, again.Quote:
the VB2010 compiler gives you a Sub keyword for lambdas that don't return a value so you can use QueueUserWorkItem of the ThreadPool class directly and specify a Sub lambda to call your functions.
Niya,
I have Button1.Enabled = False in Button1_Click and then Button1.Enabled = True in m_cn_CountCompleted(..)
I read that threadpool threads do NOT re-init any data from the last usage of that thread. Here
Now, since I am interlocking Button1 click, the same thread must be used the next time that I click Button1.
My reason/evidence for this is that the Count function does NOT restart at my new min value -- it is at max immediately!
Before interlocking the button, 'new' threads started at min.
In the original function, i is not declared, so I declared Dim i As Integer = 0, and the count starts at min again.
Does this sound reasonable?
source MSDN
Quote:
When the thread pool reuses a thread, it does not clear the data in thread local storage or in fields that are marked with the ThreadStaticAttribute attribute.
Therefore, when a method examines thread local storage or fields that are marked with the ThreadStaticAttribute attribute, the values it finds might be left over from an earlier use of the thread pool thread.
i is not using thread local storage. It is stored on the stack which means the values no longer exist when the function exits. Thread local storage has to explicitly be used. It is never implicit. So that observation has nothing to do with thread local storage. Please explain what you mean by "interlocking" the button.
Interlocking may have been a poor choice of terminology for .Net.
I meant that I disable the the button (Button1) when clicking it and then re-enabling it in m_cn_CountCompleted(..) sub.
I was trying to provide a cause for what I observed. With your assessment I think that I found my problem. -- it was NOT a thread local storage issue...
I was trying to 'cancel' (or in this case HURRY-UP) the thread by passing a bool value to it.
I never reset the bool value in the code in Form1.
I do not know if it is correct to pass a 'Cancel' bool to the running thread, but what I have done is:
overloaded the Count function with
and in the Count function, check for CancelRequest, to let the thread handle the Cancel request. ( I feel that it is better than attempting to abort a thread)Code:Public Overloads Sub Count(ByVal Cancel As Boolean)
CancelRequest = Cancel
End Sub
If a better/correct way exists, I haven't read about it or found it yet. I will keep plodding on.Code:
Public Overloads Function Count(ByVal Max As Integer, ByVal Min As Integer, ByVal Mode As Integer, ByVal DT As DateTime) As String
.
.
.
For i = Min To Max
If Not CancelRequest Then
e = New CountChangedEventArgs(i, Max, Min, Mode, DT)
If context Is Nothing Then
OnCountChanged(e)
Else
ThreadExtensions.ScSend(context, New Action(Of CountChangedEventArgs)(AddressOf OnCountChanged), e)
End If
Threading.Thread.Sleep(50)
Else
i = Max
End If
Next
.
.
.
End Function
Thank you for your valued input!
You have a knack for this. Yes what you're doing is the correct way. You use a cancel field and let the thread cancel itself. You never ever want to abort a running thread. Only the thread itself knows about its internal state so it should be responsible to stopping itself. From the outside you cannot know where the thread is currently at so aborting from the outside can cause some nasty side effects.Quote:
Originally Posted by OnErrorGoAway
For example, imagine a thread that opens files in a list one by one. You want when you stop the thread, that it closes the current file it has open before aborting. Its easy to do that from inside the thread, if a cancel variable was set you close and exit whatever loop is iterating the list. But if you try to abort the thread from the outside, you could have called it smack in the middle of reading a file and it would just stop abruptly, leaving the file opened and possibly locked.
Niya,
Am I missing something?
You mentioned earlier that
Does this hold true for your ThreadExtensions.ScSend(..., or was that statement directed at starting a thread?Quote:
ThreadExtensions class is not necessary because the VB2010 compiler gives you a Sub keyword for lambdas
Yes, it applies to ScSend as well as it also takes a delegate argument.
Thank you. Another puzzle. :bigyello:
RE: my project (post #86)
I am StopWatching it at 10.9 seconds in IDE and 10.2 seconds in .EXE.
I have two 'iterations' of 25 threads, each thread making a connection and reading 10 blocks of data, shutting down and closing the connection, then reporting that data back to UI.
Each thread updates its UI ProgressBar after each of the 10 reads and then at completion dumps its data to a TextBox.
This is a learning demo so I am not ready to ask for any help. I know that my project has kludges, but they will be removed as my understanding increases.:confused:
I have merged multiple examples from this forum and other sources such as MSDN. Thanks to all.
Now, I must understand how it does what it does.
[EDIT:Replace string functions with stringbuilder]
I am StopWatching it at 9.7 seconds in IDE and 9.6 seconds in .EXE.
Edit: just received books
-Programming VB.Net (Cornell & Morrison)
-VB 2010 Prog Ref (Stephens)
So I may be absent for a while :thumb:
Ok well, good luck to you my friend. I'll always be around if you have any questions. :)
RE: ThreadExtensions [Niya VB2008] -vs- SendOrPostCallback [VB2010]
I think that I may be on to something! I have been F8'ing some more.
If I have done something really wrong, please do not hesitate to correct me!
Replace:
with this:Code:
ThreadExtensions.ScSend(context, New Action(Of CountChangedEventArgs)(AddressOf OnCountChanged), e)
CountChanged worked.Code:context.Send(New SendOrPostCallback(Sub(state As Object) RaiseEvent CountChanged(Me, e)), Nothing)
Then replace this:
with this:Code:ThreadExtensions.ScSend(context, New Action(Of CountCompletedEventArgs)(AddressOf OnCountCompleted), New CountCompletedEventArgs(msg))
CountComplete worked.Code:
Dim em As CountCompletedEventArgs
.
.
.
em = New CountCompletedEventArgs(msg)
.
.
.
context.Send(New SendOrPostCallback(Sub(state As Object) RaiseEvent CountCompleted(Me, em)), Nothing)
Notes:
VB default project settings
Option Explicit ON
Option Strict ON
Option Compare BINARY
Option Infer ON
--If I had .Net Reflector, I would need this chair removed surgically!--
Yes, those are correct but a little too verbose for my tastes. You could just do this:-
vbnet Code:
Niya it will be great to see if you start some thread on LINQ (vb.net) please
Aite ;)Quote:
Originally Posted by make me rain
Great post on Multithreading!
Have you by any chance created a sample code in C#?
If not, I'll just stick with the VB.NET example and play around
with it.
KG
No I haven't. It would be exactly the same in C#. Only the syntax would differ. As a matter of fact, you should have little to no problems using an online converter to convert the samples to C#.Quote:
Originally Posted by KGComputers
Great!
Thanks.. :)
Thanks for source code
You're welcomed.
Hi Niya
Sitting here reading your very well described article about Multi-Threading - great reading!! Enjoyed it very much.
I'm working on a Windows service that amoung other things will scan a number of servers on the Network and due to the length of a server scan I'm thinking of building a Multi-Threading code to handle this. That way I am able to start let's say 50 scans at the same time. There could be a lot more servers - might be up 4.000....
Now it is just the question on how.
My Sub called ScanServer(ServerName as string) will need to be called but I want to be able to set a max on how many scans that can run at the same time - 50 is a starting point but not a static number.
I have the server names in a list
Private ServerList As New List(Of ServerClass)
ServerClass defined as:
Public Class ServerClass
Public ServerName As String
Public IPAddress As String
Sub ServerClass()
ServerName = String.Empty
IPAddress = String.Empty
End Sub
End Class
How do I put together a piece of code that can start the scans (up to X number at a time) and keep them filling the queue until the last server is scanned?
The Sub (ScanServer) will not return any data - the scan will be written into a XML file on the disk so I have no demand for keeping track of the thread while running.
Thanks in advance!!
Best Regards
/Mogge
I'm curious, why do you have this:-
instead of:-vbnet Code:
vbnet Code:
Sure - I'm currently in the building phase and all things are not settle yet.
I find your suggestion very good.
/Mogge
Booo!
sorry if this has already been answered, lots of activity :)....
Why do you use AddressOf instead of Sub()? Sub() allows you specify multiple parameters. Does AddressOf offer better performance or error handling?
Could specify exactly where you're talking about? Been a while since I looked over the code. I can't recall off hand.Quote:
Originally Posted by demanaz
Hi Niya,
thank you for this thread. I'm relatively new to programming and VB.net. I use VS'13 and SQL server Express.
I'm in the middle of programming an app that does the following:
through a 3rd party ActiveX API I read in records to a SQL table at high rate; several million records in 8hours a day.
While the streaming of the records occurs, I need to go over the records, do some calculations and apply filters, and represent the results in realtime. (filters it down to around a 100 records/day)
I guess you see already where I'm going with this;
Currently I do the calculations/filters on the fly when records are streaming in; this causes the streaming to delay and I loose the connection, So I need to multi-thread the two tasks.
the streaming into SQL table is done as follows:
On Form1 :
clicking on the button, invokes the AxTDAAPIComm1_OnL1Quote sub, which streams back records.Code:Private Sub SubscrBTN_Click(sender As Object, e As EventArgs) Handles SubscrBTN.Click
AxTDAAPIComm1.Subscribe(StreamOL, tdaactx.TxTDASubTypes.TDAPI_SUB_L1)
End Sub
Private Sub AxTDAAPIComm1_OnL1Quote(sender As Object, e As Axtdaactx.ITDAAPICommEvents_OnL1QuoteEvent) Handles AxTDAAPIComm1.OnL1Quote
SQL.AddStream(e.quote.Symbol, e.quote.Bid, e.quote.Ask, e.quote.Last, e.quote.PrevClose, e.quote.Volume, e.quote.TradeTime, e.quote.QuoteTime, e.quote.TradeDate, e.quote.QuoteDate, e.quote.Volatility, e.quote.OpenInterest, e.quote.UnderlyingSymbol, e.quote.CallPut, e.quote.LastSize, e.quote.MH_LastSize, e.quote.MH_IsQuote, e.quote.MH_IsTrade)
End Sub
End Class
Then in my SQL Class I send it off to my SQL table:
perfect works great!Code:Public Sub AddStream(Symbol As String, Bid As Single, Ask As Single, Last As Single, PrevClose As Single, Volume As Integer, TradeTime As Integer, QuoteTime As Integer, TradeDate As Integer, Quotedate As Integer, Volatility As Single, OpenInterest As Integer, UnderlyingSymbol As String, CallPut As String, LastSize As Integer, MH_LastSize As Integer, MH_IsQuote As Boolean, MH_IsTrade As Boolean)
Try
Dim TradeAmount As Single = 0
Dim Trade As String = ""
TradeAmount = LastSize * Last * 100
Select Case Last
Case Is = Ask
Trade = "Ask"
Case Is > Ask
Trade = "Above Ask"
Case Is = Bid
Trade = "Bid"
Case Is < Bid
Trade = "Below Bid"
Case Else
Trade = "Mid"
End Select
Dim strStream As String = "INSERT INTO OptionStream (Symbol,Bid,Ask,Last,PrevClose,Volume,TradeTime,QuoteTime,TradeDate,QuoteDate,Volatility,OpenInterest,UnderlyingSymbol,CallPut,TradeAmount,Trade,LastSize,MH_LastSize,MH_IsQuote,MH_IsTrade) " & _
"VALUES (" & _
"'" & Symbol & "'," & _
"'" & Bid & "'," & _
"'" & Ask & "'," & _
"'" & Last & "'," & _
"'" & PrevClose & "'," & _
"'" & Volume & "'," & _
"'" & TradeTime & "'," & _
"'" & QuoteTime & "'," & _
"'" & TradeDate & "'," & _
"'" & Quotedate & "'," & _
"'" & Volatility & "'," & _
"'" & OpenInterest & "'," & _
"'" & UnderlyingSymbol & "'," & _
"'" & CallPut & "'," & _
"'" & TradeAmount & "'," & _
"'" & Trade & "'," & _
"'" & LastSize & "'," & _
"'" & MH_LastSize & "'," & _
"'" & MH_IsQuote & "'," & _
"'" & MH_IsTrade & "') "
SQLCon.Open()
SQLCmd = New SqlCommand(strStream, SQLCon)
SQLCmd.ExecuteNonQuery()
SQLCon.Close()
Catch ex As Exception
MsgBox(ex.Message)
End Try
End Sub
On my Form I have created a text box, which I use to enter a SQL query statement, which I capture as follows:
As you can see, it now goes of to the SQL class and runs the Query:Code:Private Sub cmdQuery_Click(sender As Object, e As EventArgs) Handles cmdQuery.Click
If txtQuery.Text <> "" Then
If SQL.HasConnection = True Then
SQL.RunQueryWL(txtQuery.Text)
If SQL.SQLDatasetWL.Tables.Count > 0 Then
DGVData.DataSource = SQL.SQLDatasetWL.Tables(0)
End If
End If
End If
End Sub
and the result is presented in a datagrid view on my form.Code:Public Function HasConnection() As Boolean
Try
SQLCon.Open()
SQLCon.Close()
Return True
Catch ex As Exception
MsgBox(ex.Message)
Return False
End Try
End Function
Public Sub RunQueryWL(Query As String)
Try
SQLCon.Open()
SQLCmd = New SqlCommand(Query, SQLCon)
SQLDA = New SqlDataAdapter(SQLCmd)
SQLDatasetWL = New DataSet
SQLDA.Fill(SQLDatasetWL)
SQLCon.Close()
Catch ex As Exception
MsgBox(ex.Message)
If SQLCon.State = ConnectionState.Open Then
SQLCon.Close()
End If
End Try
End Sub
As you can imagine, querying against a couple million records takes a few seconds, therefore I need to move the whole query routines onto a different thread, so that the streaming won't get interrupted.
Any ideas on how I can do this?
I'm still trying to grasp the whole concept, so any detailed info you can give me will be greatly appreciated.
Niya,
At this point I don't suppose you ~need~ to hear again what an outstanding job you've done here but I think you should here it!!! And like several others, your posts and in particular your responses to the replies of others inspired me to register here just to tell you how much I appreciate how cogent your instructions are, how patient you are in your responses to nuubs (myself included hopefully), how generous you are of your time to continue supporting this thread now for over almost two and a half years and (he bows his head in embarrassment) to ask a few questions of my own.
Notwithstanding my questions, my inspiration to register came from my need to tell you that you are truly the "Angel of Code," for this is, by far, the best thread on multi-threading (sorry for the pun) that I've found and I am certain not to find one better. What has made this thread even better are some of the queries to it and your thoughtful, helpful and instructive responses to them. I hope that (even as a nuub) my questions can make a contribution in this realm.
Question 1) Please confirm. (my situation) If you have a third party component instantiated in the main thread; when you handle a callback from that component, the callback is directly run in the main UI thread, if the callback delegate and method are in the code for the main form, yes? I am pretty sure the answer is 'Yes,' but I'm not 100% sure that the code in the component (clearly a separate thread) extend itself into the callback method...
Question 2) Please confirm. Some of your responses to queries were (understandably) VB.Net Framework version dependent . With the final version of your post (i.e., using the "QueueUserWorkItem" of ThreadPool, which works under Framework 3.5, my target), is it necessary to "EndInvoke" this thread? Again I'm pretty sure, based on reading descriptions written elsewhere about ThreadPool, but ask so I can sleep at night with no lingering doubts...
Question 3) I have an app that receives a callback from an embedded component. The callback is fed into the main UI thread (assuming your answer to question 1 is 'Yes'). The issue is that the callback can occur anywhere from 0 to 100 times (or more) per second and each time requires the app to do a number of things including updating a database and the UI. This is an obvious opportunity to use multi-threading (otherwise, the responsiveness of the UI would be compromised). My assumption and my question is that instantiating a static thread for this callback to use is the way to go (yes?) and.... how would I best manage it?
Question 4) Finally, is there a pattern that is similar to the one in your posts, but which uses Begin/End Invoke and why might it better or worse than "QueueUserWorkItem" in your opinion? (this seems ~similar~ to the ConnectionThread discussion you had with psoftware, but not enough so to make it clear to me how to proceed).
Thanks Ever So Much :-)
While waiting for Niya, I'll give my opinion on some aspects of Questions 1 and 3.
I wouldn't assume the callback ran on the main thread. The opposite is usually the problem that people run into.
For instance, using a serialport object and trying to update a GUI element in the DataArrival event.
The component is running in another thread, and posts DataArrival events using yet another thread, so that it doesn't hang itself up from reading the serialport.
When the user in the DataArrival event tries to update a GUI element, they get a crossthread access violaton.
My assumption would be that the "callback" is not run on the GUI thread, and that can be tested by seeing if Invoking is necessary.
Sometimes all that a young child needs (though I am most decidedly not young!), to hear is "No" and they "get it." passel, your reply was that for me, thanks! Though its been quite some time since, earlier I got intermittent crossthread access violation along with the UI slowdown as the callbacks were being processed. These two things brought me to the conclusion that I must add a separate thread generated within the callback to handle it and avoid both problems. I did so and "solved" the problem(s), ostensibly. This only generated a problem that hasn't exposed itself until now, a memory leak (a poor implementation of the BeginInvoke/EndInvoke pair). The memory leak has caused me to reconsider all and which has allowed me to find Niya's wonderful discussion here.
I feel I'm getting closer to a well constructed fix, if only cognitively at this point, with all the help I've found in this thread.
But curiously, to the point of your reply, while the callback did generate crossthread violations, it also caused the UI to slow to a crawl. How is this possible, i.e., a callback, with clear "ties" to a thread separate from the main UI thread (your point), also slow the main UI thread? It would seem that it would be either a crossthread violation or a UI slow down, but no both, yes? Is this even possible? Or, is the slow down the result of some other side effect? Note: the callbacks UI updates were not a continuous flow of updates rather, updates to text boxes and graphical objects based on summary results of the callback.
PS thanks for your reply passel!
so I'm trying the following:
Code:Private Sub cmdQuery_Click(sender As Object, e As EventArgs) Handles cmdQuery.Click
sw.Restart()
sw.Start()
If txtQuery.Text <> "" Then
If SQL.HasConnection = True Then
Threading.ThreadPool.QueueUserWorkItem(AddressOf DoQry)
End If
' End If
End If
End Sub
Private Sub DoQry()
SQL.RunQuery(txtQuery.Text)
UpdateGrid()
End Sub
Private Sub UpdateGrid()
If DGVData.InvokeRequired Then
DGVData.Invoke(New action(AddressOf UpdateGrid))
Else
DGVData.DataSource = SQL.SQLDataset.Tables(0)
ETlbl.Text = sw.ElapsedMilliseconds
End If
End Sub
While records are streaming into my SQL Table, at >200 records/second, I invoke a SQL query against it : a simple "select * from Optionstream" , which takes about 7 seconds.
However now I get an error: " The connection was not closed, The connection's current state is open".
which is the Exception error from the Try-Catch in my Addstream sub in the SQL Class. see previous post.
Before I added the multi-threading this was not the case.
What am I doing wrong?