[RESOLVED] how to handle winsock_DataArrival

[pannam]

this is my current code for data arrival the simple one

 Private Sub sock1_DataArrival(ByVal bytesTotal As Long)
Dim hash As String
Dim strHeader As String

sock1.GetData dat, vbString  'writes the new data in our string dat ( string format )
Debug.Print dat
TxtAscii = TxtAscii & "server : " & StringToHex(dat) & vbCrLf

now, i get lots of incomming data and i handle them by command buttons for now i know its lame, so when a certain data arrives i trigger a command manually and it fires a tcp packet ...and this is how i have been interacting with the server all this time..however, the transaction have been successful so far, i would like to handle incoming data automatically..(from my code) in order to do so , what all things i need to know ?

[leinad31]

There should be something in the stream received to serve as basis of what to do next, e.g. process code, keyword, etc parsed from text received.

[Doogle]

Well, briefly, there's some important things you have to remember when using TCP.

When you use the Winsock .SendData method the payload (ie your data) is packaged up by TCP and IP and transmitted over the network. During its transition over the network, a packet may be re-packaged and either split into one or more packets or combined with other packets. When it is received, IP and TCP un-package the data and present the payload to the Winsock_DataArrival event. So you never (normally) see a packet.

There's nothing you can do to affect this - that's the way the protocols work.

The only thing you can assume is, that when using TCP (as opposed to UDP) all the data will arrive, or an error will be generated, and the data will arrive in the same order as it was sent.

This can present a challenge, since you have to determine when all the expected data has actually arrived, and buffer it until it does. This is where the Application Protocol comes into play.

This is just the set of 'Rules' that the Sender and Receiver abide by in order to understand each other. It can be as simple as appending a particular character to the end of each message to mark the end of the message.

Alternatively you may decide to send fixed length data so the receiver can count the bytes received, and when it reaches the pre-defined data length it knows that a complete message has been received. Also, you coud define your payload such that, say, the first 4 bytes represent the expected message length,followed by the data, receive that and then count the bytes until the bytes received equal the expected message length. There are also many other methods, and which one you choose will be dependent upon the type of application and data that you are sending and receiving.

In your case it looks as if you're sending and receiving non-printable data and a valid data byte can take any value between Hex 00 and Hex FF, so using a particular character to determine the end of message is not an option. (Unless the application has reserved a character for this purpose)

So you need to fully understand the data stream being sent. If you are controlling it then it's up to you to define and implement a suitable Application Protocol that fits the needs of your application. If you are not controlling it then you have to understand the existing Application Protocol in use by the sender and implement that in your receiving code.

Creating a solid, reliable, and performing TCP application is a complex process, there are no 'short cuts'.

[DigiRev]

Couldn't have said it better than the Doogle...

How to handle the data depends on the protocol. If it's an HTTP server, it's pretty simple (although it can get pretty involved depending on what you're doing/how much of the HTTP protocol you want to implement).

If it's something that is not well-documented (logging into a chat server, or something like that), then you are on your own. I've had to figure out several protocols for games, and other things that were not documented at all and it is not an easy task.

If you want any specific advice, you should tell us what server you are communicating with (not just what type, but the name of the software), and probably a "packet dump" of the packets sent/received by the program (most packet sniffers have a Save/Export feature). Attaching that along with the info and your current code would be a good start.

Almost all protocols contain status/command codes (like HTTP has status codes for 404 not found, 200 ok) which can help you determine what to send next.

[pannam]

the server is "mig33" its a mobile application .the communication takes place either by tcp or http connection. i am building a pc executable and communicating via tcp.it is well documented and the protocols are explained in

 http://www.devinsmith.net/articles/mig33/mig33.html

how ever not all protocols are explained. ....
this is the login_packet sent by the client to login

 02 20 C8 20 01 20 20 20 4A 20 09 20 20 20 04 20 20 20 
01 20 08 20 20 20 04 6A 32 6D 65 20 07 20 20 20 09 4A 32 4D 45
76 33 2E 30 32 20 05 20 20 20 0A 6A 65 73 75 73 73 6D 69 6C 65
20 03 20 20 20 02 01 2E 20 02 20 20 20 01 02 20 01 20 20 20 02
20 01

in this packet is my username & the platform i am using the client application..this i can do...
in respone to this the server sends me a login_challenge packet for example

02 00 C9 00 01 00 00 00 1D 00 02 00 00 00 08 72 54 51 6F 
52 34 45 66 00 01 00 00 00 09 78 50 43 4A 6E 38 54 75 35

this packet has a hash_string which i have to concatenate with my password and send it to the server ..
for 3 letter username the packet size is 35 byte, for 4 letter username the packet size is 36 and for 5 letter username its 37 and from 6 -20 its 38.
in the login_challenge packet the first 15 byte are always the same

 02 00 C9 00 01 00 00 00 1D 00 02 00 00 00 08

now when i get this i send them my login_response packet for example

02 00 CA 00 02 00 00 00 0A 00 01 00 00 00 04 1B EE AF F2

which is of 19 byte...i can get this too

isn't this information useful enough to handle it in winsock ??i would like to write a code for "" if a packet arrives with 35 or 36 or 37 or 38 byte then send the challenge packet ""

[Doogle]

That Protocol looks to be well defined. Personally I wouldn't approach the problem from looking at the data lengths, rather I'd create some UDTs that define the various Commands and Responses you wish to process and dynamically build them / decode them from the protocol definitions. That way you wont 'code yourself into a corner' and have to do a lot re-coding when you want to expand the functionality of your client.

It might seem a bit of a drudge to start with (as I said earlier, there are no short cuts) but it will pay off large dividends as you progress. It will also help you understand better the way in which the protocol and the application work.

As they say, if it's worth doing then it's worth doing well.

[pannam]

Doogle thanx for your advice ..i have kept command buttons to handle all the incoming bytes ...
BUT.......
i want to atleast login automatically..for now i have to click on 2 commands to login ..which is not practical ..who clicks 2 buttons to login ??? ..isn't is poosible to get the header of the tcp packet and fire a command or code? for example ...or get the byte length of the incoming packet and then use a command ...?

[Doogle]

Yes, it's very possible. The protocol decsribes the 'Challenge - Response' mechanism quite nicely. I'm a bit busy at the moment but I'll take alook and see if I can come up with something for you as an example.

[pannam]

hey doogle ..i got it right this time..i got the header of the tcp packet and used used the case fucntion to identify the packets..now, i don't need command buttons any more ... hehe..

[Hack]

Post the code you wound up with.

Others might find it useful.

Thanks.

[pannam]

oh yes hack ..pardon me for not providing the full detail..
here's what i did

 Private Sub sock1_DataArrival(ByVal bytesTotal As Long)
Dim hash As String, bash As String
Dim strHeader As String

sock1.GetData dat, vbString        
strHeader = Left(dat, 3) ' the first 3 bytes is the header of the packet for my server ..for other serve it may differ!

Select Case strHeader

      Case Is = HextoAscii("02 00 C9 ") 'i converted the first 3 byte to hex for better understanding ;) 
        now i code under this section for this packet,or invoke a command
    

'similarly for different header i code differently
 
case is =hextoascii("02 00 C0 ")
  'write a script
end select
end sub

[Doogle]

Before you get too far, I suggest you re-read Post #3, especially about Buffering the data in the DataArrival event. You are currently assuming that the complete record will arrive in one event which will not always be the case. In the code you have so far you're only examining the first 3 bytes so there's a good chance of them being there but you cannot assume that will always be true. When you get onto processing messages you wil run into problems that will manifst themselves as corrupt messages, missing messages and / or incomplete messages - all due to no buffering.

The Protocol uses variable length messages with the appropriate lengths and markers so you will have to wait for a particular number of bytes to be received, unblock the Command1 and Command2 words and from those values determine the format of the record. Once you've performed that you will again have to determine the number of bytes you need in order to pick up the lengths of the various components of the message, wait for them to arrive and process them.

One slight complication I think you may need to overcome, and others will correct me if I'm wrong, is that the various lengths (some 2 byte, some 4 byte) are not sent as Little Endian (well, I couldn't see any indication in the Protocol Documentation and the example data streams you gave show that they're not) so you will need to convert prior to using them.

For example, you may receive a length 00 00 01 A0 (416), in order to cast that from your Byte Array to a Long variable so you can use it, you will need to store it as A0 01 00 00 into the Long Variable. It's a fairly straightforward Function, using the CopyMemory API. Here's an example of a couple of Fuctions you could use for converting to Long and Integer types from a Byte Array:

Option Explicit

Private Declare Sub CopyMemory Lib "kernel32" Alias "RtlMoveMemory" _
            (Destination As Any, _
             Source As Any, _
             ByVal Length As Long)

Private Function ByteToLong(byt() As Byte, lng As Long) As Long
Dim intI As Integer
Dim lngVal As Long
For intI = 0 To 3
    CopyMemory ByVal VarPtr(lngVal) + 3 - intI, byt(intI + lng), 1
Next intI
ByteToLong = lngVal
End Function

Private Function ByteToInteger(byt() As Byte, lng As Long) As Integer
Dim intI As Integer
Dim intVal As Integer
For intI = 0 To 1
    CopyMemory ByVal VarPtr(intVal) + 1 - intI, byt(intI + lng), 1
Next intI
ByteToInteger = intVal
End Function

Private Sub Command1_Click()
Dim bytArray(10) As Byte
Dim lngVal As Long
Dim intVal As Integer
bytArray(6) = &H0
bytArray(7) = &H0
bytArray(8) = &H1
bytArray(9) = &HA0
'
' Convert the 4 bytes starting at element 6
' of the Byte Array to a Long
'
lngVal = ByteToLong(bytArray(), 6)
Debug.Print lngVal
'
' Convert the 2 bytes starting at element 8
' of the Byte Array to an Integer
'
intVal = ByteToInteger(bytArray(), 8)
Debug.Print intVal
End Sub

Of course, you'll have to perform the reverse if / when you construct a message to the server which requires lengths. Also you'll have to convert any Strings to Bytes using, for example, the strConv function with the vbFromUnicode option.

[pannam]

thanx a lot doogle for digging deep... i knew i had to buffer the incoming packet but honestly didn have a single clue on how to do it..more over the incoming packets were always always changing according to the username ,password or msg ...it grew too confusing for me to stick to the whole packet so , i decided to stick to the first 3 byte as it was different for most of the packet....as you said now, i am facing error with incoming msg in a chat room as the first 3 byte are same for all the user's ....i am still struggling with it..i shall keep in mind what you told me and do accordingly ..lets see if i can code it better that way..