[HELP NEEDED] Socket + UDT + CopyMemory...

[Neverbirth]

I'm having some serious problems using CopyMemory to receive data from a socket... and a true solution is becoming more and more urgent... I have to receive from a socket a series of message types, each one with different variable types (done in C).

Currently I use a static byte array (static because some of the messages are rather large and arrive in several packages), and once I know I've received the whole message I use copymemory with a UDT, and treat it...

The thing is the net where it's working seems to have some serious problems, and the messages end overlaping and sometimes CopyMemory makes my program to wholy crash...

While now what I do is using another static variable so if I'm already receiving some data the next package waits till the end, but then if some DoEvents enters in action the wait is skipped and the program may crash anyway... Also, the messages don't have some start chain per se...

Please, I'd love to receive some help. Thanks in advance.

[Merri]

You're probably doing buffer overflows, but it is impossible to say exactly how things are wrong without seeing your current code. You could have misunderstood how UTDs work or how data is storaged in them, or you could just overwrite the byte arrays, or etc.

[Neverbirth]

Well, my code is rather large since there are a lot of message types, plus I have to take into account several other things, but the code in the DataArrival giving the problem could be reduced to:

Static btArr() as Byte
Dim msgBody as tMsgGeneral
 
Winsock1.GetData btBuff, vbArray + vbByte, bytesTotal
 
CopyMemory msgBody, btBuff(0), Len(msgTop)
 
parseMsg msgBody
 
Erase btBuff

Just as simple as that... but as I've said the net is giving a lot of problems... in fact, the timeout between connections if no message was received had to be forced to 19 seconds.

[Merri]

Can you post tMsgGeneral?

The bit of the above code doesn't really tell much anything as I don't know what msgTop is, I don't see btArr anywhere despite it is declared, I see btBuff that hasn't been declared and I don't see redimming (so that I could check if the buffer is of correct size or not).

[Neverbirth]

Can you post tMsgGeneral?

The bit of the above code doesn't really tell much anything as I don't know what msgTop is, I don't see btArr anywhere despite it is declared, I see btBuff that hasn't been declared and I don't see redimming (so that I could check if the buffer is of correct size or not).

Well, my bad, btBuff == btArr (in the actual code it's correct). Also, think of tmsgGeneral as something simple like:

Type tMsgGeneral
    type As Long
    length as Long
    number as Long
    CRC as String * 16
End Type

[Milk]

(I've never used Winsock) is the string returned by Winsock Ansi or Unicode? Also I don't think the string data of CRC is held consecutively in the UDT memory, rather a pointer to the string. With this in mind try something like...

'to get the 12 byte message header...
CopyMemory msgBody, btBuff(0), 12 
 
'to get the message...
'Assuming the string data to be unicode
CopyMemory ByVal StrPtr(msgBody.CRC), btBuff(12), 32
'or if ansi
Dim StrBuf(31) As Byte
CopyMemory StrBuf(0), btBuff(12), 32
msgBody.CRC = StrConv(StrBuf, vbUnicode)

*Untested off the top of my head warning*

Edit: better still... Use copymemory for the header and this for the string

msgBody.CRC = StrConv(Mid$(btbuf, 7, 16), vbUnicode)

[Neverbirth]

(I've never used Winsock) is the string returned by Winsock Ansi or Unicode? Also I don't think the string data of CRC is held consecutively in the UDT memory, rather a pointer to the string. With this in mind try something like...

'to get the 12 byte message header...
CopyMemory msgBody, btBuff(0), 12 
 
'to get the message...
'Assuming the string data to be unicode
CopyMemory ByVal StrPtr(msgBody.CRC), btBuff(12), 32
'or if ansi
Dim StrBuf(31) As Byte
CopyMemory StrBuf(0), btBuff(12), 32
msgBody.CRC = StrConv(StrBuf, vbUnicode)

*Untested off the top of my head warning*

Edit: better still... Use copymemory for the header and this for the string

msgBody.CRC = StrConv(Mid$(btbuf, 7, 16), vbUnicode)

The CRC is a normal string of just 16 bytes, so I could even change it to something like

CRCArr(15) As Byte.

Dunno, I have to make some more tests. Changing the byte array to something like a String and so trying to make:

CopyMemory ByVal VarPtr(msgBody), ByVal StrPtr(strBuff), LenB(msgBody)

But even so I'm not fully sure if it won't crash...

[ccoder]

I have a lot of experience sending/receiving UDTs to/from both C and Java apps. I posted some code here that demonstrates receiving large UDTs and assembling the 'chunks' for processing.

One quick question about the longs and integers in your UDT; are they big-endian (aka network byte order)? If so, you will need to use the API functions to convert to little-endian. You can see this in the DataArrival code in the above link. The C app that the code in the link connects to runs on another platform that is big-endian. Java, by default, converts values to big-endian. We always send values in big-endian and leave it to the receiving end to convert accordingly.

Milk, fixed length strings in a UDT are stored in the UDT. If a string is variable length, then the BSTR is stored in the UDT and that address, which is on the sending machine, won't be of any use on the receiving machine.

[Neverbirth]

Your code is rather similar to what I already do, with just some slight differences . For example instead of global variables I use static ones as already pointed out, and I don't use two different byte arrays for the data treatment.

One quick question about the longs and integers in your UDT; are they big-endian (aka network byte order)? If so, you will need to use the API functions to convert to little-endian. You can see this in the DataArrival code in the above link. The C app that the code in the link connects to runs on another platform that is big-endian. Java, by default, converts values to big-endian. We always send values in big-endian and leave it to the receiving end to convert accordingly.

Yes, I have to, and already, make big<->little conversions.

[ccoder]

Your code is rather similar to what I already do, with just some slight differences . For example instead of global variables I use static ones as already pointed out, and I don't use two different byte arrays for the data treatment.

In your original post you stated

Currently I use a static byte array (static because some of the messages are rather large and arrive in several packages), and once I know I've received the whole message I use copymemory with a UDT, and treat it...

but, from the few lines that you posted, it appears that you always receive the incoming data into the static byte array, copy it to the UDT and then clear it. It isn't clear that you are preserving the data until all of it has arrived.

I used a global byte array because the entire UDT does not arrive in one piece. The other byte array was used for the .GetData and then I used copyPtr to move the receiving byte array to the appropriate position in the final byte array. I don't recall why I used a global for the initial receive.

[Neverbirth]

In your original post you stated
but, from the few lines that you posted, it appears that you always receive the incoming data into the static byte array, copy it to the UDT and then clear it. It isn't clear that you are preserving the data until all of it has arrived.

I used a global byte array because the entire UDT does not arrive in one piece. The other byte array was used for the .GetData and then I used copyPtr to move the receiving byte array to the appropriate position in the final byte array. I don't recall why I used a global for the initial receive.

? That's why I use static variables, because that way they keep their values when the data arrives in several packages. I check if the data is the continuation from a previous one, and if so I keep the data until it finishes coming, and then erase the static array for new data.

[Billy Conner]

While now what I do is using another static variable so if I'm already receiving some data the next package waits till the end, but then if some DoEvents enters in action the wait is skipped and the program may crash anyway... Also, the messages don't have some start chain per se...

(i may be reading this wrong) you werent using a loop in the _DataArrival Sub while waiting for data i hope. i think you should save the existing data and exit the sub until the length requirement is met by the UDT header you pass. that would help some of the errors, if thats the case.

[Milk]

Okay so I'm a bit out of my depth here ( @ CCoder) but...

<snip> The CRC is a normal string of just 16 bytes, so I could even change it to something like

CRCArr(15) As Byte.

<snip> Also, think of tmsgGeneral as something simple like:

Type tMsgGeneral
          type As Long
          length as Long
          number as Long
          CRC as String * 16
      End Type

Is it a string of 16 bytes or 32 bytes? (CRC as String * 16 is 32 bytes)
Just what is Len(msgTop)?

[Merri]

Strings are UTF-16/BSTR and hold 2 bytes per character, but when using API calls, such as CopyMemory, VB automatically converts the contents of the string (and in this case the whole UDT) into ANSI, so String * 16 actually is 16 bytes. Then the returning String / UDT is then converted back into the format VB actually uses, so you have 32 bytes, but 16 characters.

This also happens when saving/loading text files with Strings.

This doesn't happen if a pointer is passed instead of the string or UDT, however, there are other things that must be noticed: because these are out of the actual topic in this thread, I leave them out.