buffering an incoming data

[pannam]

hie all,
suppose i receive a tcp packet with the begining with "02 00 0C..........................."
all the incoming tcp packet start with "02" so, how do i buffer a tcp packet till i receive the complete packet??

[drag0n_45]

I'd say to create a byte array and put the data into element 0 of the byte array, then redim the array with each byte until you reach another 02, at which point you'd start filling the next array while processing that one.

Alternatively, if your packets are of a standard size (which they should be) create a byte array of that size and simply fill it with the incoming data until it's full.

However, there are plenty of TCP/IP controls out there, I'd look into those instead of trying to make your own. Maybe even a few API calls?

[dilettante]

Very few protocols frame messages (not packets) by signaling the start of a new message. If you try to do this the receiver will always have to wait for the start of a new message before knowing the previous one is finished.

It is much more likely that there is either some length field within the header (first few bytes) or else some delimiting character or character sequence at the end of each message.

[DigiRev]

The length of the data should be in the header like dilettante said. If you are coding a mig33 client. I remember glancing at the protocol documentation that you posted and the header should contain the length.

These kind of protocols are a little harder to work with than ones that just delimit packets with a special sequence like vbCrLf or something.

[pannam]

thank you for your suggestion guyz ...i better code with the header instead of buffering all the packet and waiting for the other packet to arrive..

[DigiRev]

Declare a form-scope variable for the buffer, like:

Option Explicit
 
Private strBuffer As String

On DataArrival, get the data into a string, and append it into the buffer, like:

strBuffer = strBuffer & ReceivedData

Then process the buffer one packet at a time. Once a packet from the buffer has been processed, remove it from the buffer, doing something like:

'Packet processed.
strBuffer = Mid$(strBuffer, LengthOfPacket + 1)

If you need help with this, then post a URL to the protocol documentation and your current code.

[Doogle]

The mig33 protocol (http://www.devinsmith.net/articles/m...#Communication) is a bit tricky, but I'd start off by defining a UDT to represent the fixed length portion of the Header. Once you've received that, the Command1 and Command2 codes will determine the format of the rest of the data.

Private Type migHeader
    Magic As Byte
    Command1(1) As Byte
    Command2(1) As Byte
    Pack_Len(3) As Byte
End Type

Private bytBuffer() As Byte

Private Sub Form_Load()
'
' Allocate some initial space for the Buffer
' (100 bytes)
'
ReDim bytBuffer(99)
End Sub

Private Sub ws_DataArrival(ByVal bytesTotal As Long)
Dim udtHead As migHeader
Dim bytRx() As Byte
Static lngLast As Long          ' This points to the next free element in the buffer
Dim lngBufLen As Long
Dim intI As Integer
'
' Read the data
' Calculate whether it will fit into the buffer
' and extend the buffer if required
'
ws.GetData bytRx
lngBufLen = lngLast + UBound(bytRx)
If lngBufLen > UBound(bytBuffer) Then ReDim Preserve bytBuffer(UBound(bytBuffer) + 100)
'
' Add the data received into the buffer
'
For intI = lngLast To lngLast + UBound(bytRx)
    bytBuffer(intI) = bytRx(intI - lngLast)
Next intI
lngLast = intI
'
' If we have received enough data then populate
' the header
'
If UBound(bytBuffer) >= Len(udtHead) Then
    Copymemory udtHead, bytBuffer(0), Len(udtHead)

You now have a design decision to make in order to interpret the Command1 and Command2 values, (actually you should have made that decision before writing any code ) within the UDT they are stored as two byte arrays. You could inspect each one in turn to determine the Command or you could cast them to two Integer values and use those or you could cast all 4 bytes to a Long variable and use that. If you cast to Integer or Long you'll have to remember that the bytes will be stored as Little Endian (ie reversed) and you may end up with some 'funny' values.

For example, the Protocol defines Command1 = 00 CB and Commad2 = 00 02 for the LOGIN_OK / MOTD message.

Casting those two values from the byte arrays to integers would result in intCommand1 = CB 00 (-13568) and intCommand2 = 02 00 (512) which is not necessarily intuitive when compared to the original Protocol values.

Casting the 4 bytes to a Long would result in 02 00 CB 00 (33606400).

Bearing in ,mind that you will have to fiddle around with the length values in order to cast them properly, I'd be tempted to store them in reverse order when 'casting' to Integer or Long types in which case they would retain their values as defined in the protocol. (Using the ByteToInteger and ByteToLong functions I mentioned in your other thread: http://www.vbforums.com/showthread.php?t=524353)

Option Explicit

Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" _
            (Destination As Any, _
             Source As Any, _
             ByVal Length As Long)

Private Const LOGIN_OK1 As Integer = 203
Private Const LOGIN_OK2 As Integer = 2
'
' Insert other Protocol Constants here
'
Private Type migHeader
    Magic As Byte
    Command1(1) As Byte
    Command2(1) As Byte
    Pack_Len(3) As Byte
End Type

Private bytBuffer() As Byte

Private Sub Form_Load()
'
' Allocate some initial space for the Buffer
' (100 bytes)
'
ReDim bytBuffer(99)
End Sub

Private Sub ws_DataArrival(ByVal bytesTotal As Long)
Dim udtHead As migHeader
Dim bytRx() As Byte
Static lngLast As Long          ' This points to the next free element in the buffer
Dim lngBufLen As Long
Dim boComplete As Boolean
Dim intI As Integer
Dim intCommand1 As Integer
Dim intCommand2 As Integer
Dim lngPackLen As Long
'
' Read the data
' Calculate whether it will fit into the buffer
' and extend the buffer if required
'
ws.GetData bytRx
lngBufLen = lngLast + UBound(bytRx)
If lngBufLen > UBound(bytBuffer) Then ReDim Preserve bytBuffer(UBound(bytBuffer) + 100)
'
' Add the data received into the buffer
'
For intI = lngLast To lngLast + UBound(bytRx)
    bytBuffer(intI) = bytRx(intI - lngLast)
Next intI
lngLast = intI
Do
    '
    ' If we have received enough data then populate
    ' the header
    '
    If UBound(bytBuffer) >= Len(udtHead) Then
        CopyMemory udtHead, bytBuffer(0), Len(udtHead)
        intCommand1 = ByteToInteger(udtHead.Command1, 0)
        intCommand2 = ByteToInteger(udtHead.Command2, 0)
        lngPackLen = ByteToLong(udtHead.Pack_Len, 0)
        '
        ' lngPackLen is the total length of the Pay Load
        ' less 9. This can be used to determine if the
        ' complete message has been received
        '
        If lngPackLen + 9 <= lngLast - 1 Then
            '
            ' We have at least one complete message
            ' Now we can determine what sort of message it is and unblock it
            '
            Select Case intCommand1
                Case LOGIN_OK1
                    Select Case intCommand2
                        Case LOGIN_OK2
                            '
                            ' Code to process the LOGIN_OK message
                            '
                        '
                        ' Extra Command2 selections go here
                        '
                    End Select
                    '
                    ' Extra Command1 and their Command2 selections go here
                    '
            End Select
            '
            ' Having processed the complete message
            ' we have to inspect the buffer to see the there's
            ' another message or part of a message left in it
            ' if there is, then we have to jiggle around with it and
            ' process it.
            ' Otherwise flush the buffer and exit
            '
            If lngLast > lngPackLen + 9 Then
                '
                ' There's more data
                ' shuffle it to the front of the buffer
                '
                For intI = lngPackLen + 9 To lngLast - 1
                    bytBuffer(intI - lngPackLen) = bytBuffer(intI)
                Next intI
                lngLast = intI
            Else
                ReDim bytBuffer(99)
                lngLast = 0
            End If
        Else
            boComplete = True
        End If
    Else
        boComplete = True
    End If
Loop Until boComplete = True
End Sub

Private Function ByteToLong(byt() As Byte, lng As Long) As Long
Dim intI As Integer
Dim lngVal As Long
For intI = 0 To 3
    CopyMemory ByVal VarPtr(lngVal) + 3 - intI, byt(intI + lng), 1
Next intI
ByteToLong = lngVal
End Function

Private Function ByteToInteger(byt() As Byte, lng As Long) As Integer
Dim intI As Integer
Dim intVal As Integer
For intI = 0 To 1
    CopyMemory ByVal VarPtr(intVal) + 1 - intI, byt(intI + lng), 1
Next intI
ByteToInteger = intVal
End Function

Note that none of the code above is tested in any way shape or form (except the ByteTo.. functions) and is probably littered with errors. However the Logic is sound (and is only one of many possible ways you can approach it). Others with more experience and knowledge will have other ideas and most probably better implementations. eg Using a string buffer as opposed to a byte buffer will simplify buffer management, but I think that's actually the least of the problems. This could be a 'starter-for-10'.

[pannam]

oh Doogle you have put in such an effort ..i honestly thought this thread would be left as it is...thank you very much ..although i have reached quite far with my project i will still practice what you have taught and let you know..thank you very much once again..

[pannam]

hey guyz ..thanx for replying ..@Doogle i had taken your #6 post into consideration and wrote my code...i don't know wether this way works or not ..if not i will move on to your latter posts..this is the way i am trying to buffer ..my module

Public Function cSizeHex(buff As String) As String
Dim pcklen As Integer
pcklen = Len(buff)
Dim FstNum As String
FstNum = 0
Do While pcklen > 255
FstNum = FstNum + 1
pcklen = pcklen - 256
Loop
cSizeHex = " " & Hex(FstNum) & " " & Hex(pcklen) & " "
End Function

lets say a packet arrives in my winsock_data arrival event

 02 01 F4 00 00 00 00 00 5B 00 02 00 00 00 0A 
62 65 72 6C 69 6E 77 61 6C 6C 00 04 00 00 00 0A 62 
65 72 6C 69 6E 77 61 6C 6C 00 08 00 00 00 1F 43 75 
72 72 65 6E 74 6C 79 20 69 6E 20 74 68 65 20 72 6F 
6F 6D 3A 20 77 69 72 65 77 6F 72 6D 00 06 00 00 00 
02 00 01 00 01 00 00 00 01 01 00 03 00 00 00 01 03

5B is the length of the packet from there till the end i.e. 03 = 91

and in my winsockdata arrival i have this code

 Private Sub sock1_DataArrival(ByVal bytesTotal As Long)

Dim strHeader As String
sock1.GetData dat, vbString
strHeader = Left(dat, 3)
Select Case strHeader
case is = HextoAscii("02 01 F4")
'''''''''here i want to write a code that can buffer the whole packet'''
'''code to be processed''
end select
end sub

[pannam]

as each of the tcp data ends with 03 i added this buffer and it works

 If Right$(strBuffer, 1) <> HextoAscii("03") Then
      bolTrunc = True
        lonTruncStart = InStrRev(strBuffer, HextoAscii("03"))
        If lonTruncStart > 0 Then
          ' MsgBox "true"
            strTrunc = Mid$(strBuffer, lonTruncStart + 1)
        End If
    End If
    
    If InStr(1, strBuffer, HextoAscii("03")) > 0 Then
        strPackets() = Split(strBuffer, HextoAscii("03"))
        lonUB = UBound(strPackets)
        
        If bolTrunc Then lonUB = lonUB - 1
        
        For lonLoop = 0 To lonUB
            If Len(strPackets(lonLoop)) > 3 Then
Select Case Left$(strPackets(lonLoop), 3)
'code to process
end select
End If
        Next lonLoop
    
    End If
    
    Erase strPackets
    
    strBuffer = vbNullString
    
    If bolTrunc Then
        strBuffer = strTrunc
    End If
    
    strTrunc = vbNullString

althought the above code works ,i don't think its complete ..as sometimes a msg that has only 3 letters in it too has 03 infront of it which marks the length of the message ..the first byte is always 02 ,the length of the tcp data is in the 9th byte if i were to buffer it from there what else do i have to add in my code ?? i am bringing this up as you guyz have mentioned about getting the length of the data from the header..

[pannam]

hey Doogle i get compile error as expected array in this part

 ws.GetData bytRx
lngBufLen = lngLast + UBound(bytRx)
If lngBufLen > UBound(bytBuffer) Then ReDim Preserve bytBuffer(UBound(bytBuffer) + 100)

[Doogle]

Hi, If you look atPost #7 you'll see that bytBuffer is defined thus:

Private bytBuffer() As Byte

in the Declarations Section of the Form as a Dynamic Byte Array. I suspect you have missed of the '()' or defined bytBuffer in the wrong place.

[pannam]

hey, doogle i got the code working i re-wrote all my code according to what you provided me.. but as i need to use ascii string as out put in my text box i used this code

dat = StrConv(bytRx, vbUnicode)

initially i ran into some trouble because i couldn't get the required header to process the data and it didn show up in select case ..so, i changed your code

If lngPackLen + 9 <= lngLast - 1 Then

to

If lngPackLen + 8 <= lngLast - 1 Then

doing this i am able to process the data ,i get the hash string from the server (38 byte) ..and i send the login response , but i am getting errors when i receive the login ok packet.. subscription out of range in

  bytBuffer(intI) = bytRx(intI - lngLast)

i didn make any other changes and i am waiting for your reply.