[Winsock]Spliting sticked data

[BlackCatSLO]

Hy. I have problem im working on some app with winsock, but some times data came glued together. I did make function for spliting and data format does look like this
L0004T0001dataHF

[ L0004 <--- lenght of data(value L0000 to L9999) T0001 <--- data type(values T1 to T9999) ] - header[ 10 Byts]
[ data ] - data[ lenght 0B - 9999B ]
[ HF <-- checksum not very good ] - footer [ 2 Byts ]

and 12Bytes is alot for data for spliting...do anyone knows how to split data withow single chars witch can messup data information
(example)
1. packet data[454353453vddffs324]
spliter [v]
2. packet data[dgfh45egh54645645]

recived data [454353453vddffs324vdgfh45egh54645645]
result 1:[454353453]
result 2:[ddffs324]
result 3:[dgfh45egh54645645]


But i will get three packets from two becouse the function did think there are two spliters.
Can anybody tell me what to use that method im using right now(L0004T0001dataHF ) or....?

[opus]

-1- For using a splitter you need to use a character that is not used inside your messages! It seems you are using straight numbers and characters in your messages, so use something like "|", "{" ......
-2- in your message headers you have the infromation about the length of a single message, you could parse the header and read only the needed length of the message recieved, delete the read part of the message recieved and restart the parsing. This way you don't even need a splitter!

[BlackCatSLO]

I only tried to show you what problems do i have with spliters i do have all chars in the data from 0...255 with wide.

[the182guy]

The only way to do it is to use delimeters and sub delimaters like..


key
PACKET_DEL = "|"
SUB_DEL = "$"

PACKET_DEL header SUB_DEL data SUB_DEL footer PACKET_DEL header SUB_DEL data SUB_DEL footer PACKET_DEL


In the above example each packet is delimted, but then the sections of a single packet are delimited by a seperate char

[Al42]

I only tried to show you what problems do i have with spliters i do have all chars in the data from 0...255 with wide.Assuming you have no control over the data format:

If all characters are valid data (including possibly escaped characters), and the elements aren't fixed length, the data wasn't designed to be parsed. So evidently you're missing something in your analysis of the data. If you're correct, the data is useless.

[alkatran]

It looks like you're trying to do what I used to do: put the length of each packet at the start. This isn't the best way to do it for two reasons: it's complicated and easy to screw up.

What you want to do is separate packets using a delimiter (any character will do). The only thing you need to worry about then is making sure the delimiter doesn't end up in the data!


'In a module
public function makePacket(byval data as string) as string
data = replace(data, "/", "/a") '/ is our escape character
data = replace(data, "|", "/b") '| is our packet delimiter
makepacket = data & "|"
end function
public function breakPacket(byref refReceivedData as string, byref refPacketData as string) as boolean
refPacketData = left(refReceivedData, instr(refReceivedData, "|")) 'grab the first packet
if refPacketData <> "" then '(it will be "" if there were no complete packets)
refReceivedData = mid(refReceivedData , len(refReceivedData)) 'cut packet off received data
refPacketData = replace(refPacketData, "/b", "|") 'get '|'s in data back
refPacketData = replace(refPacketData, "/a", "/") 'get '\'s in data back
breakPacket = true
end if
end function

'In your received event
static data as string
dim s as string
winsock.getdata s,,bytestotal
data = data & s 'build up data over time, in case packets are split in half
while breakPacket(data, s) 'take packets out of data
process(s)
wend

[TokersBall_CDXX]

[VB6]

You can split using a delim of more than one characters (a string)


Dim sStringParse As String
Dim sArray() As String
Dim iString As Variant

sStringParse = "This is a test *** test is * a test *** test 123"
sArray = Split(sStringParse, "***")
For Each iString In sArray
MsgBox iString
Next

[BlackCatSLO]

Hmm i did come to another problem. If is more than 4k data sticket together then i it will be dataarrived triggered multiple times so if is this 4kB of data then ***Data*Data*Dat
and then i will recive next part a*Data***Da and next
ta*Data***Data* How do i know when is the end of the packet, and then trigger data spliting. I think this problem can be solved with data lenght in the header.
Every time i start thinking more and more logical problems came to my mind.

[alkatran]

The code I provided keeps them together, as long as you always send data that's been run through 'makePacket'

[CVMichael]

If you send the length of the "packet" then the receiving application will know how much to read.
When using this method, you don't need to send a packet delimiter like makepacket (or something like that).

See the code here as an example:
http://www.vbforums.com/showpost.php?p=2884817&postcount=8

[alkatran]

Problems with preceding the packet with their length:
- just one incorrect packet will take out a lot of other packets
- limited size packets (going over the size limit creates an incorrect packet)
- more complicated to write and read

[CVMichael]

Problems with preceding the packet with their length:
........

- just one incorrect packet will take out a lot of other packets
as long as you send as much as you told the client you will send, there is no problem... you could also write code to avoid that if you really think it happens, but in all my years of programming with winsock, this never hapened
- limited size packets (going over the size limit creates an incorrect packet)
i don't get this one... there is no limit anywhere...
- more complicated to write and read
it's pretty simple to me... and as long as I know i'm doing the right thing, i don't care how long or complicated the code is... and even though I did not test for speed, i'm pretty sure my code is faster also

But anyways... everyone with it's own style... you code is good also, but I would not use it for bigger applications...

[alkatran]

For the incorrect packet part, I was thinking more in term of a transfer involving UDP, where part of the message may be lost.

For the speed part, it is probably true that my vb code is slower than yours because I use replace a bunch of times. This mainly has to do with how VB handles strings. I could do all the replacing in one pass in C.

The size limitation has to do with needing to only use a fixed number of digits for your size. You can get around this, but it adds a few more bytes overhead to all your packets.

In the end, both will perform almost exactly the same if you use them right.

[Al42]

For the speed part, it is probably true that my vb code is slower than yours because I use replace a bunch of times. This mainly has to do with how VB handles strings. I could do all the replacing in one pass in C.You can do all the replacing in one pass (in the source code) in VB too, but the actual work is done in multiple loops regardless of the language refPacketData = Replace(Replace(refPacketData, "/b", "|"), "/a", "/")

[BlackCatSLO]

i never had problems with length(L0000) spliter
I only had problems with data length when the data was splited becouse of DataArrived event.
the problem is the program will never kown when is end of data

[CVMichael]

All you have to do is to buffer all data received until you get the total amount that you sent in the header.

So at data arrival, read the header, get the total length, then don't process anymore data until you get all of it...

You find examples in the previous posts