Speed up received message converted to string

[max_carpenter]

Hi all.

I have my Async sockets working perfectly thanks to some help from some kind people here however I have now discovered that it misses some communications if they are quick. This is connecting to a chat server I have no control over the speed of sent messages but upon testing I can get it to miss a substantial amount of incoming messages and it all seems to be linked to the conversion of the message from a byte array to a string.

Here is the onreceive routine:

Private Sub OnReceive(ByVal ar As IAsyncResult)
 
            _Client.EndReceive(ar)
 
            Dim msg As String = System.Text.ASCIIEncoding.ASCII.GetString(_Bytes)
            Array.Clear(_Bytes, 0, _Bytes.Length)
 
            _Client.BeginReceive(_Bytes, 0, _Bytes.Length, SocketFlags.None, New AsyncCallback(AddressOf OnReceive), _Client)
 
            RaiseEvent ReceiveMessage(msg)
 
        End Sub

If another message is sent to the bot between the start of this rountine and the 4th line it misses it. On hammering the chat 20 lines the bot only sees 13. Is there anyway I can speed up the above code or do it a different way to get it ready to receive the next message quicker?

Thanks,
Max

[.paul.]

this is the fastest solution I could think of:

Imports System.Threading

Public Class Form1

    Dim _Bytes() As Byte

    Private Sub OnReceive(ByVal ar As IAsyncResult)
        _Client.EndReceive(ar)
        ThreadPool.QueueUserWorkItem(AddressOf processMsg, _Bytes.Clone)
        Array.Clear(_Bytes, 0, _Bytes.Length)
        _Client.BeginReceive(_Bytes, 0, _Bytes.Length, SocketFlags.None, New AsyncCallback(AddressOf OnReceive), _Client)
    End Sub

    Private Sub processMsg(ByVal newMsg As Object)
        Dim msg As String = System.Text.ASCIIEncoding.ASCII.GetString(DirectCast(newMsg, Byte()))
        RaiseEvent ReceiveMessage(msg)
    End Sub

End Class

[Shaggy Hiker]

I was going to suggest that you use Array.Copy to copy the bytes to a new array and add them to a Queue, then dequeue them on a different thread such that one thread is listening, dumping the data, and getting back to listening, but that's pretty much what .Paul. did, only without the Queue. I do kind of wonder about the RaiseEvent statement, though. In fact, that may have been the original problem. After all, it seems like the event should end up being raised on the thread that is doing the receiving, so whatever handling you are doing on that event is added to the cost of processing one message. That's still true for .Paul.'s code, except that it will be the threadpool thread rather than the listener thread that will do the processing. Still, I would think that you might want to have the event raised on the UI thread, rather than either of the other threads. It all depends on what you are doing with the message.

[max_carpenter]

Thanks for the responses Guys, Shaggy There is no UI thread as this is a class library I am trying to make a generic library for sockets so all of my applications that need it can just use this library to make life easier. Different applications will be doing different things with the strings but will always receive as a string. The one I am primarily making this for at the moment will be an IRC bot which will run as a windows service however I am now deciding to make another class library for IRC with sockets in due to limitations of using a socket library and then all irc stuff inside the application.

[formlesstree4]

Thanks for the responses Guys, Shaggy There is no UI thread as this is a class library I am trying to make a generic library for sockets so all of my applications that need it can just use this library to make life easier. Different applications will be doing different things with the strings but will always receive as a string. The one I am primarily making this for at the moment will be an IRC bot which will run as a windows service however I am now deciding to make another class library for IRC with sockets in due to limitations of using a socket library and then all irc stuff inside the application.

The thing I've always done when dealing with sockets/networking is that get the response processed as fast as possible and prepare for another response before even shipping the data out to be processed.

.paul. has the right idea; I never really kept passing around a single byte array, but kept multiple byte arrays for each request instead. However, cloning it like that should be just fine.

[max_carpenter]

Ok thanks for the advice I will make those alterations when I get back there. Making a dedicated library for IRC is much more complicated :-)

[Shaggy Hiker]

There IS a UI thread, it just isn't in the dll. This actually makes my point a bit more significant. Somebody is going to work with the objects in this dll, if the object raises an event, where would you like it raised?

[Evil_Giraffe]

Seems to me that you're assuming that one "send" from the server will cause one "receive" event on the client. This is not correct. I suspect all your "missing" data is being mushed up into other events. You need to account for two things when receiving messages over the network (if you're using TCP which provides certain guarantees): 1) More than one message in a single set of received bytes, 2) One message being split over several sets of received bytes (and of course a combination of the two, where you have two partial messages in one set of bytes (the end of one followed by the beginning of another) with possibly many complete messages between them). I don't see any code that will deal with that in what you've posted.

[max_carpenter]

Ok interesting thanks. No I didn't know about that. That could very well explain the problem. So let me see if I can get this right:

Sockets can and will lump multiple messages together or split messages across multiple receives with no indication of separate messages its up to the server to know what individual messages are. For instance with irc a Line feed is sent between each line of data.

So if I was to have a secondary buffer all received data gets added to this buffer I could then have a routine that starts at the beginning of of the buffer and looks for a complete line (ending in a line feed) if it sees one it removes those bytes from the buffer and sends them on to the application then continues with the next set of bytes in the buffer.

Is this correct?

[max_carpenter]

Hi Shaggy, The event is to be raised in the calling application. In this instance I am working on a Windows Service but yes it could well be a forms app.

[Evil_Giraffe]

So if I was to have a secondary buffer all received data gets added to this buffer I could then have a routine that starts at the beginning of of the buffer and looks for a complete line (ending in a line feed) if it sees one it removes those bytes from the buffer and sends them on to the application then continues with the next set of bytes in the buffer.

Okay, but what happens when the message gets split over three sets of bytes?
Also, you're looking for separators at the text level. What happens when the bytes that make up a single character get split between read boundaries? Better to be able to work out message boundaries from the bytes, and then convert the relevant bytes from the message into the text.

[max_carpenter]

Okay, but what happens when the message gets split over three sets of bytes?

Well as I described all incoming data could be put into this secondary buffer until a secondary routine can make a full line up then remove it from the buffer and send to the application.

Also, you're looking for separators at the text level. What happens when the bytes that make up a single character get split between read boundaries? Better to be able to work out message boundaries from the bytes, and then convert the relevant bytes from the message into the text.

I am not sure what you mean by this. Are you meaning that the character which splits the lines up when converted to bytes get splits into two transfers? In which case then my secondary buffer would still work as its building all the bytes back up together?

[Evil_Giraffe]

Well as I described all incoming data could be put into this secondary buffer until a secondary routine can make a full line up then remove it from the buffer and send to the application.

If your secondary buffer is expanding in size to hold as much data as it needs to, then that will be fine, but that isn't what I think when I think "buffer". I had in mind a fixed-size structure.

I am not sure what you mean by this. Are you meaning that the character which splits the lines up when converted to bytes get splits into two transfers? In which case then my secondary buffer would still work as its building all the bytes back up together?

As long as the bytes you convert to text start from a place that is definitely the beginning of a message, you should be okay. I'd be careful though.

[Shaggy Hiker]

Hi Shaggy, The event is to be raised in the calling application. In this instance I am working on a Windows Service but yes it could well be a forms app.

Right. As it stands, the event will be raised in the calling application, but in a secondary thread rather than the main thread. For a Windows Service, it is quite likely that this won't matter. For a forms app, it could cause you some interesting exceptions, such as Cross-Threaded Reference exceptions. You might want to search on SynchronizationContext to look at ways to get the event raised on the right thread, but that could prove to be interesting. The way I generally work with it is that I store the SynchronizationContext to a variable prior to launching the thread. By doing this, the stored context is the UI context. You'd have to identify a point where you initiated the call to store the context for later use.

[max_carpenter]

Ahh Yes I know what you mean. I have run into that issue during testing and used variables to get round it such as an output text box for all communications I have a variable in the UI part which I add everything to and the UI refreshes the text box with what the variable contains but yes I see what you mean.

Evil_Giraffe, Thank you for all your information I will do some deep thinking over the next few days to decide the best way for me to tackle this. As a rule of thumb I should always know what the start and end of data is but things go wrong. Things can be lost in transportation or the client/server could send wrong data missing something. This is going to take a bit of time to implement I think... and it was going so well this time around lol.

[max_carpenter]

Ok I have had to take a bit of a break from coding for a few days but I have come back to this with a fresh head and this is what I have done:

Private Sub OnReceive(ByVal ar As IAsyncResult)
 
            _Client.EndReceive(ar)
            ThreadPool.QueueUserWorkItem(AddressOf MessageHandler, _Bytes.Clone)
 
            Array.Clear(_Bytes, 0, _Bytes.Length)
 
            _Client.BeginReceive(_Bytes, 0, _Bytes.Length, SocketFlags.None, New AsyncCallback(AddressOf OnReceive), _Client)
 
End Sub

Private Sub MessageHandler(ByVal InBytes As Object)
 
            Dim msg As String = System.Text.ASCIIEncoding.ASCII.GetString(DirectCast(InBytes, Byte()))
 
            For Each chars In msg
                _MessageBuffer.Enqueue(chars)
            Next
 
 End Sub

Private Sub MessageProcessor()
            Dim Message As String
            Do
                If _MessageBuffer.Count > 0 Then
                    Message = Message & _MessageBuffer.Dequeue
                    If Message.EndsWith(_MessageSeperator) Then
                        RaiseEvent ReceiveMessage(Message)
                        Message = Nothing
                    End If
                End If
            Loop
End Sub

The MessageProcessor is started on a background thread automatically by the Connect routine once it connects to the server. The thread is also aborted upon the disconnect routine.

The _MessageSeperator is defaulted to vbCr but can be changed via a property.

I need to build a handler encase the separator doesn't get sent to stop the message getting too large but how does that look for now?

[formlesstree4]

Just cap it at a certain byte count?

[max_carpenter]

Yeh that is roughly what I was thinking just haven't done it yet. Also how to handle it ie as in if it reaches its limit and no end of data character detected should it dump the buffer etc, should it start deleting the front of the buffer to add more stuff to the end raise an error etc. What do you think is the best option?

[dbasnett]

What if the code that is adding to _MessageBuffer and the code that is removing from _MessageBuffer execute at the same time?

[dbasnett]

I didn't test this but some food for thought

    Dim _MessageBuffer As New System.Text.StringBuilder
    Dim _mbLock As New Object
    Dim newMsg As New Threading.AutoResetEvent(False)
    Dim _MessageSeperator As String = ControlChars.CrLf

    Private Sub MessageHandler(ByVal InBytes As Object)
        Threading.Monitor.Enter(_mbLock)
        _MessageBuffer.Append(System.Text.ASCIIEncoding.ASCII.GetString(DirectCast(InBytes, Byte())))
        Threading.Monitor.Exit(_mbLock)
        newMsg.Set()
    End Sub

    Private Sub MessageProcessor()
        Dim Message As String
        Do
            newMsg.WaitOne()
            Threading.Monitor.Enter(_mbLock)
            Dim inBuf As String = _MessageBuffer.ToString
            _MessageBuffer.Length = 0
            Threading.Monitor.Exit(_mbLock)
            For x As Integer = 0 To inBuf.Length - 1
                Message &= inBuf(x)
                If Message.EndsWith(_MessageSeperator) Then
                    RaiseEvent ReceiveMessage(Message)
                    Message = ""
                End If
            Next
        Loop
    End Sub

[max_carpenter]

It's a queue so it can handle adding and removing at the same time? at least I thought it can works in my tests.

[dbasnett]

I am under the impression that collection based objects are not safe from separate reader / writer threads. This little experiment eventually faulted with some weird exception

    Private Sub Button1_Click(sender As Object, e As EventArgs) Handles Button1.Click
        Debug.WriteLine("go")
        Dim t1 As New Threading.Thread(AddressOf outQ)
        t1.IsBackground = True
        t1.Start()
        Dim t2 As New Threading.Thread(AddressOf inQ)
        t2.IsBackground = True
        t2.Start()
    End Sub

    Dim foo As New Queue(Of String)
    Dim arst As New Threading.AutoResetEvent(False)

    Private Sub inQ()
        Do
            For x As Integer = 1 To 100
                Try
                    foo.Enqueue(x.ToString)
                Catch ex As Exception
                    Debug.WriteLine(foo.Count & " " & ex.Message)
                    Stop
                End Try
                If x = 1 Then arst.Set()
            Next
            Threading.Thread.Sleep(10)
        Loop
    End Sub

    Private Sub outQ()
        Do
            Do While foo.Count > 0
                Try
                    Dim z As String = foo.Dequeue
                Catch ex As Exception
                    Debug.WriteLine(foo.Count)
                    Stop
                End Try
            Loop
            arst.WaitOne()
        Loop
    End Sub

[Evil_Giraffe]

If you're multi-threading then you need to use the ConcurrentQueue and its TryDequeue method in place of Dequeue. (Enqueue is the same as on the standard Queue)

[max_carpenter]

Hi thanks for your response.

I will look into your proposed message but this has intrigued me to do further testing. You are quite right with your example you do get some odd errors occuring but I think its because its adding to the queue quicker then it can remove and eventually bombs out. This I appear to have proven with the following:

Sub Main()
 
        Dim loaderthread As New Threading.Thread(AddressOf Queueloader)
        loaderthread.Start()
 
        Do
            If myqueue.Count > 0 Then
                Console.WriteLine(myqueue.Dequeue)
            End If
        Loop
 
    End Sub
 
    Sub queueloader()
        Do
            For i = 1 To 999999999
                myqueue.Enqueue(i)
            Next
        Loop
    End Sub

This bombs out after awhile like yours. If I add:

Threading.Thread.Sleep(5)

after me .dequeue line it bombs out much quicker and the bigger the value in the sleep the quicker it bombs out with the same wierd errors. No if I then remove that sleep and put it after then .enqueue like so:

Sub Main()
 
        Dim loaderthread As New Threading.Thread(AddressOf Queueloader)
        loaderthread.Start()
 
        Do
            If myqueue.Count > 0 Then
                Console.WriteLine(myqueue.Dequeue)
            End If
        Loop
 
    End Sub
 
    Sub queueloader()
        Do
            For i = 1 To 999999999
                myqueue.Enqueue(i)
                Threading.Thread.Sleep(10)
            Next
        Loop
    End Sub

This appears like its going on forever. So far this code has been running for about 4 hours, I will leave this going all day to see what happens and report back but I am reading and writing on different threads but the inbound cannot get too high which going back to my actual use not only will there be a limit on how much I will receive at one given time but I will also limit it so the queue can only get to a certain size.

[max_carpenter]

Also as another interesting test I have made a forms app with a big counter and a timer set to 1 then two threads one that adds to the queue and one that removes from the queue with no delays. On the timer I update the counter with queue.count. Most of the time it jumps between 5000 and 0 very very quickly but occasioanly its getting to extream high numbers. Also from testing I have filled a queue up until it cannot handle any more, I done 5 tests which came back with the maximum counts of:

64013780
64013482
64405440
64013781
64278146

Pretty consistent. Running the forms app that adds and removes from the queue on multi threads instantly has not got that high yet but also has not bombed out yet.... At the moment I am only filling up the queue with 1 character per queue item (exactly how I will be using it) I will test later with multiple characters per queue item.

[dbasnett]

Did you take a look at the code I provided that used stringbuilder? I didn't understand the point of queuing the chars one at a time. Also, MessageProcessor was in a very tight loop that I bet causes your CPU usage to be 100%. I avoided that with the autoresetevent.

[max_carpenter]

Ye I do appreciate the code you provided and I will implement it tonight and get it working But this whole thing is intriguing me with the queue and I just want to test further to see what does and does not work. This will be my last test unless any of you have any other interesting variants to add but the code below again queues and dequeues from different threads and a timer displays current queue count to the screen but this time I have have added random delays on both the read and write threads and it is yet to crash again and I think this is still because despite it being very random the queue count never gets high. I havent seen this code generate a queue much more than 50 items:

Public Class Queue
 
    Dim myqueue As New Queue(Of String)
    Dim strbuild As String
 
    Private Sub Form1_Load(sender As System.Object, e As System.EventArgs) Handles MyBase.Load
 
        Timer1.Start()
 
        Dim t As New Threading.Thread(AddressOf counter)
        t.Start()
 
        Dim s As New Threading.Thread(AddressOf uncounter)
        s.Start()
 
    End Sub
 
    Private Sub counter()
        Threading.Thread.Sleep(1000)
        Do
            For i = 10000000000000 To 99000000000000
                myqueue.Enqueue(i)
                Threading.Thread.Sleep(GetRandomNumber(5, 100))
            Next
        Loop
    End Sub
 
    Private Sub uncounter()
        Threading.Thread.Sleep(1250)
        Do
            While myqueue.Count > 0
                strbuild = strbuild & myqueue.Dequeue()
                Threading.Thread.Sleep(GetRandomNumber(1, 50))
            End While
            Threading.Thread.Sleep(GetRandomNumber(1000, 5000))
        Loop
    End Sub
 
    Private Sub Timer1_Tick(sender As System.Object, e As System.EventArgs) Handles Timer1.Tick
        tbocounter.Text = myqueue.Count.ToString
    End Sub
 
    Dim objRandom As New System.Random(CType(System.DateTime.Now.Ticks Mod System.Int32.MaxValue, Integer))
 
    Public Function GetRandomNumber(Optional ByVal Low As Integer = 1, Optional ByVal High As Integer = 100) As Integer
        ' Returns a random number,
        ' between the optional Low and High parameters
        Return objRandom.Next(Low, High + 1)
    End Function
 
End Class

[dbasnett]

As evilg pointed out you need to use the ConcurrentQueue version if you have separate readers and writers, or you can implement your own locking. If you use multiple threads you will learn that if multiple threads are manipulating an object, safe is better than sorry, and tracking down these bugs ain't (sic) fun, I know. Look at what you are doing just to prove that you do indeed need some mechanism to control access to the queue.

[max_carpenter]

Ok I have made the change to the concurrent queue as suggested but just as and example I know adding them all into one string is not a clever idea but I want to point out I get exactly the same results as before it can add to the queue quicker then it can remove and eventually bombs out because there is too much in memory for it to keep going. If however I add a micro delay to the adding of items to the queue then it works perfectly. This gets exactly the same results (ok its safer) but just as an example works the same. In the end it is very important to restrict the inbound flow of data is what this lesson shows.

Public Class Queue
 
    Dim myqueue As New Concurrent.ConcurrentQueue(Of String)
    Dim strbuild As String
 
    Private Sub Form1_Load(sender As System.Object, e As System.EventArgs) Handles MyBase.Load
 
        Timer1.Start()
 
        Dim t As New Threading.Thread(AddressOf counter)
        t.Start()
 
        Dim s As New Threading.Thread(AddressOf uncounter)
        s.Start()
 
    End Sub
 
    Private Sub counter()
        Threading.Thread.Sleep(1000)
        Do
            For i = 10000000000000 To 99000000000000
                myqueue.Enqueue(i)
            Next
        Loop
    End Sub
 
    Private Sub uncounter()
        Threading.Thread.Sleep(100)
        Dim str As String = Nothing
        Do
            While myqueue.Count > 0
                myqueue.TryDequeue(str)
                strbuild = strbuild & str
            End While
        Loop
    End Sub
 
    Private Sub Timer1_Tick(sender As System.Object, e As System.EventArgs) Handles Timer1.Tick
        tbocounter.Text = myqueue.Count.ToString
    End Sub
 
    Dim objRandom As New System.Random(CType(System.DateTime.Now.Ticks Mod System.Int32.MaxValue, Integer))
 
    Public Function GetRandomNumber(Optional ByVal Low As Integer = 1, Optional ByVal High As Integer = 100) As Integer
        ' Returns a random number,
        ' between the optional Low and High parameters
        Return objRandom.Next(Low, High + 1)
    End Function
 
End Class

[max_carpenter]

Also just want to add the code I put in post #24 has been running for about 7 hours now and still running smoothly lol.

[.paul.]

Also just want to add the code I put in post #24 has been running for about 7 hours now and still running smoothly lol.

don't want to suggest you cut short your testing, but i'd say that probably works

[max_carpenter]

Well for thorough testing and peace of mind I will leave it running until the end of the week (providing my pc doesn't crash or auto restart) just to see if it can maintain stable running for all that time (I mean the eventual app will in theory run indefinitely) When I get home tonight I will implement whats been said in this thread by others but I am just interested how it seems to work perfect yet im told its not and will have problems.

[dbasnett]

Also just want to add the code I put in post #24 has been running for about 7 hours now and still running smoothly lol.

Is the queue a concurrent queue? If not it might run for weeks, but it will fail. When MSDN says, "To allow the collection to be accessed by multiple threads for reading and writing, you must implement your own synchronization." believe them, regardless of your testing. The way it sounds you are going to test yourself into a mistake.

[Evil_Giraffe]

Um. With a single thread calling Enqueue() and a single thread calling Dequeue() concurrently, you might be alright with a standard queue, but then again you might not.
If you use a ConcurrentQueue you will definitely be alright.
If you ever have multiple threads calling Enqueue or Dequeue then you definitely won't be alright with a standard Queue.

Given that a ConcurrentQueue is no harder to use that a Queue I fail to see why you'd even try a standard Queue. In fact, it's probably nicer to use, if you use it properly. Instead of this:

Private Sub uncounter()
    Threading.Thread.Sleep(100)
    Dim str As String = Nothing
    Do
        While myqueue.Count > 0
            myqueue.TryDequeue(str)
            strbuild = strbuild & str
        End While
    Loop
End Sub

You should be doing this:

Private Sub uncounter()
    Threading.Thread.Sleep(100)
    Dim str As String = Nothing
    Do
        While myqueue.TryDequeue(str)
            strbuild = strbuild & str
        End While
        ' Also, shouldn't you do something like pause for a little while here?
        ' Maybe have an exit condition that is checked to gracefully end the thread?
    Loop
End Sub

[dbasnett]

Um. With a single thread calling Enqueue() and a single thread calling Dequeue() concurrently, you might be alright with a standard queue, but then again you might not.
If you use a ConcurrentQueue you will definitely be alright.
If you ever have multiple threads calling Enqueue or Dequeue then you definitely won't be alright with a standard Queue....

In case this, "With a single thread calling Enqueue() and a single thread calling Dequeue() concurrently, you might be alright with a standard queue, but then again you might not." is not clear. EG means one thread with two methods, not one thread doing enqueue and another thread doing the dequeue.

[dbasnett]

Using a list instead of a queue. Also moves the conversion to the message processor.

    Dim _MessageBuffer As New List(Of Byte)
    Dim _mbLock As New Object
    Dim newMsg As New Threading.AutoResetEvent(False)
    Dim _MessageSeperator As String = ControlChars.CrLf

    Private Sub MessageHandler(ByVal InBytes As Object)
        'add the bytes to the buffer
        Threading.Monitor.Enter(_mbLock)
        _MessageBuffer.AddRange(DirectCast(InBytes, Byte()))
        Threading.Monitor.Exit(_mbLock)

        newMsg.Set() 'so the message processor runs
    End Sub

    Private Sub MessageProcessor()
        Dim Message As String
        Do
            newMsg.WaitOne() 'wait here for new message arrival
            If _MessageBuffer.Count > 0 Then
                'get all of the bytes currently in the buffer
                Threading.Monitor.Enter(_mbLock)
                Dim inBuf As String = System.Text.ASCIIEncoding.ASCII.GetString(_MessageBuffer.ToArray)
                _MessageBuffer.Clear() 'clear the buffer
                Threading.Monitor.Exit(_mbLock)

                For x As Integer = 0 To inBuf.Length - 1 'process the characters
                    Message &= inBuf(x)
                    If Message.EndsWith(_MessageSeperator) Then
                        RaiseEvent ReceiveMessage(Message)
                        Message = ""
                    End If
                Next
            End If
        Loop
    End Sub

[max_carpenter]

Ok I am now completely lost. I have implemented what you guys have said (I believe correctly) and first thing this morning it seemed to be working ok, However now something seems to be going wrong in the mix and I am not sure what.

I am using it to connect to "irc.rizon.net" on port 6667 for now. Now when you connect to that server (you can verify via telnet) you receive this from the server:

:irc.sxci.net 439 * :Please wait while we process your connection.
:irc.sxci.net NOTICE AUTH :*** Looking up your hostname...
:irc.sxci.net NOTICE AUTH :*** Checking Ident
:irc.sxci.net NOTICE AUTH :*** No Ident response
:irc.sxci.net NOTICE AUTH :*** Found your hostname

Nothing has changed between this morning and now on that response and it shouldn't (apart from the server replying) At the end of each message should be a linefeed which as shown previously I am using vbLf to determine the end of each message.

On my MessageProcessor Routine right before it starts processing character for charcter it returns whats currently in the buffer to the program with raw: in front of it and then after its gone through the letters each time it receives the end of line character it sends it back to the program with msg: in front of it. In theory I should get 1 Raw message and 1 Msg message for each line the server sends to me. Here is how my application outputs:

raw: :irc.rizon.io 439 * :Please wait while we process your connection.
msg: :irc.rizon.io 439 * :Please wait while we process your connection.
raw: :irc.rizon.io NOTICE AUTH :*** Found your hostname (cached)
:irc.rizon.io NOTICE AUTH :*** Checking Ident
msg: msg: :irc.rizon.io NOTICE AUTH :*** Checking Ident
raw: :irc.rizon.io NOTICE AUTH :*** No Ident response
msg:

As you can see the first message is twice fine, the second one appears to receive two messages in one packet so we get a two lined raw output which is ok and it should generate two msg outputs however you can see the first msg: is blank and the second one shows the second line of the raw output.
We then receive a 3rd line from the server and again the Msg: output is blank... This is now where my application gets stuck. I have a received byte counter which adds from the onreceive routine and I can see it receiving more data after this but I get no more on the output.

Here is my MessageProcessor Routine:

Private Sub MessageProcessor()
 
            Dim Message As String = ""
 
            Do
                newMsg.WaitOne() 'wait here for new message arrival
 
                If _MessageBuffer.Count > 0 Then
                    'get all of the bytes currently in the buffer
                    Threading.Monitor.Enter(_mbLock)
                    Dim inBuf As String = System.Text.ASCIIEncoding.ASCII.GetString(_MessageBuffer.ToArray)
                    _MessageBuffer.Clear() 'clear the buffer
                    Threading.Monitor.Exit(_mbLock)
 
                    RaiseEvent MessageReceived("raw: " & inBuf)
 
                    For x As Integer = 0 To inBuf.Length - 1 'process the characters
                        Message &= inBuf(x)
 
                        If Message.EndsWith(_MessageSeperator) Then
 
                            RaiseEvent MessageReceived("msg: " & Message)
 
                            Message = ""
 
                        End If
                    Next
                End If
            Loop
 
        End Sub

And just for verification here is my OnReceive and MessageHandler routines:

Private Sub OnReceive(ByVal ar As IAsyncResult)
 
                _Client.EndReceive(ar)
           
            ThreadPool.QueueUserWorkItem(AddressOf MessageHandler, _Bytes.Clone)
 
            'Indicator of how many bytes have been received
            _BytesReceived += _Bytes.Count
 
            'Clear receive buffer for next message
            Array.Clear(_Bytes, 0, _Bytes.Length)
 
                _Client.BeginReceive(_Bytes, 0, _Bytes.Length, SocketFlags.None, New AsyncCallback(AddressOf OnReceive), _Client)
 
        End Sub

Private Sub MessageHandler(ByVal InBytes As Object)
            'add the bytes to the buffer
            Threading.Monitor.Enter(_mbLock)
            _MessageBuffer.AddRange(DirectCast(InBytes, Byte()))
            Threading.Monitor.Exit(_mbLock)
 
            newMsg.Set() 'so the message processor runs
        End Sub

Can anyone see the bug that's happening?