Can I use the API funcition "setsockopt" in conjunction with the Winsock OCX? Or will I have to go all-API for my Winsock implementation, if I choose to use the "setsockopt" API call?
I don't think you can obtain the scoket descriptor from Winsock which would be required in order to use setsocket. You'll probably end up with having to 'roll your own' using the APIs. What particular 'feature' of setsocket are you wishing to implement ?
I want to disable Winsock's combining of TCPIP packets. When I send data from my client with 2 successive send operations I want it to send 2 distinct packets, such that on the receiving end, it will perfrom the DataArrival event twice, once for each packet. Below is the sending code I have for testing.
Code:
Winsock1.SendData "abc"
Winsock1.SendData "123"
I read elsewhere on the net, that to get it to send 2 descrete packets one after the next in ultra rapidfire way, without it combining the packets internally, I would need to disable the Nagle algorithm.
And I read that to disable the Nagle algorithm, I would need to invoke the setsockopt API call.
But I'm not sure how to use this.
Also note that it does seem possible to get the socket number from winsock for the purpose of passing it to the API, with the winsock control's SocketHandle property.
When I send data from my client with 2 successive send operations I want it to send 2 distinct packets, such that on the receiving end, it will perfrom the DataArrival event twice, once for each packet.
This really begs the question; Why ?
If you really need to do this then the API route is the only one available to you (I believe). The handle is not the same as the socket descriptor.
If you really need to do this then the API route is the only one available to you (I believe). The handle is not the same as the socket descriptor.
Not sure if it's just a "hack", or if it's the official way to separate SendData calls, but I've heard that if you use DoEvents between any two successive SendData calls, they will be sent separately.
It dependson what you mean by 'sent'. When you issue a 'SendData' the TCP stack will package up the data to be sent (the Payload) into one or more packets and pass these to the IP stack. Once TCP has finished it will raise the 'SendComplete' event which basically tells the Application that it's 'safe' (i.e. the internal Winsock Buffers have been processed and flushed) to send something else. At some point in time, probably after the 'SendComplete' event has triggered, the actual IP packet will be put on the network.
Using DoEvents just causes the Application to 'pause' and service any outstanding interrupts; it's not usually required in Winsock applications. (in fact, unless the Application is written very carefully, DoEvents can cause havoc)
TCP is a streaming protocol and packets can be merged and split during their life on the network. I would suggest that rather than try to 'force' the protocol to do things it wasn't really designed to do, you look at buffering data until you have a complete payload received. That way you don't have to worry about the underlying protocols. Note that your application never sees a packet, only the payload.
It dependson what you mean by 'sent'. When you issue a 'SendData' the TCP stack will package up the data to be sent (the Payload) into one or more packets and pass these to the IP stack. Once TCP has finished it will raise the 'SendComplete' event which basically tells the Application that it's 'safe' (i.e. the internal Winsock Buffers have been processed and flushed) to send something else. At some point in time, probably after the 'SendComplete' event has triggered, the actual IP packet will be put on the network.
Using DoEvents just causes the Application to 'pause' and service any outstanding interrupts; it's not usually required in Winsock applications. (in fact, unless the Application is written very carefully, DoEvents can cause havoc)
TCP is a streaming protocol and packets can be merged and split during their life on the network. I would suggest that rather than try to 'force' the protocol to do things it wasn't really designed to do, you look at buffering data until you have a complete payload received. That way you don't have to worry about the underlying protocols. Note that your application never sees a packet, only the payload.
I'm considering making my own streaming video protocol for experimental purposes. And I want each video frame to be guarentied to be sent as a separate packet, so that the recieving application can have a separate DataArrival event for each frame, so that the code in the DataArrival event won't have to include code to parse the stream to find the start of each packet (and hope that it can find it and keep it all in sync), but rather the code in the DataArrival event will just be a simple winsock1.getdata and a SetDibits API call to display the received image. And the faster I can send the data without 2 adjacent packets becoming merged, the better (because the moment 2 packets become merged, my scheme breaks down, be cause all of a sudden I start to have to add code to the DataArrival event to separate the data back into discrete packets).
Furthermore it becomes complicated with the 8kb max per packet, so I can't send a complete image (of any decent size) in a single packet, and I need to add YET MORE CODE to reassemble images.
So the complete code needs to do 2 things:
1) combine packets for data that exceeds 8kb
2) re-split packets along the ORIGINALLY intended packet boarders so that all the data remains in sync
What I would LIKE to have it do to avoid this unnecesarilly complicate programming is:
1) regardless of how close 2 packets are sent to each other, they are treated as 2 separate packets (not combined)
2) have the winsock transmit buffer auto-size itself to the size of the data in a given packet
In the end, an ideal implementation would send 1MB of data as a single 1MB packet (not 128 packets of 8kb each), and yet would would also send 10 4byte packets sent VERY FAST one right after the next as indeed 10 separate packets of 4bytes each (and not a single 40byte packet). Otherwise the data alignment/sync code just becomes HUGE and VERY COMPLICATED (so complicated in fact that I've been trying for like a year on coming up with code to accomplish the task, without any success).
If it's a Protocol you're attempting to create, then VB6 is not the vehicle to use. You need to be below the Application Layer. i.e. at the Transport Layer.
It dependson what you mean by 'sent'. When you issue a 'SendData' the TCP stack will package up the data to be sent (the Payload) into one or more packets and pass these to the IP stack. Once TCP has finished it will raise the 'SendComplete' event which basically tells the Application that it's 'safe' (i.e. the internal Winsock Buffers have been processed and flushed) to send something else. At some point in time, probably after the 'SendComplete' event has triggered, the actual IP packet will be put on the network.
The Winsock control doesn't really work in quite this way.
A call to SendData copies the data to its own intermediate buffer and starts an async write on the underlying socket if it is idle. Each such write sends a maximum of the amount of free space in the socket's output buffer.
The SendComplete event is raised whenever there is "enough room" to accept another send. What this means isn't clear however a subsequent SendData (once SendComplete has been raised) should never block. But it doesn't actually mean that sending is complete.
Data is taken from the intermediate output buffer by a worker thread inside of the Winsock control and sent via Winsock (the underlying Winsock layer, not the Winsock control itself) in chunks no larger than the outgoing buffer (normally 8KB). As these sends complete the Winsock control eventually "decides" when to raise SendComplete. If the intermediate buffer has room it may raise SendComplete immediately.
It is the Winsock layer that then talks to the TCP layer of the TCP/IP stack. All sorts of magic goes on here from the TCP layer to the IP layer and eventually on to the MAC layer. Only the MAC layer actually deals with "packets."
What actually ends up going over the wire is entirely independent of the socket buffer size, the SendData size, or much of anything else. The more important size involved here is the Maximum Segment Size which your program has little influence on.
For inbound data the Winsock control's worker thread looks for completion of the async read operation it starts on the underlying socket upon connection. When this read completes it returns at most 8KB, again due to the socket buffer size.
The worker thread raises the DataArrival event, and once you call GetData it starts a new async read.
Last edited by dilettante; Mar 22nd, 2013 at 04:15 PM.
@Dilettante: Thanks for the correction and explanation. Hopefully it explodes the myth that the 'SendComplete' means that the data is 'on the network'.
This 'packet' word is quite confusing. I always thought of TCP packaging the payload into a TCP 'packet', IP then packaging the TCP 'packet' into an IP 'packet' and MAC packaging the IP 'packet' and slapping it on the Network.
I suppose it doesn't really matter as long as the message, that a 'normal' Winsock Application never sees a 'packet', gets through
Last edited by Doogle; Mar 23rd, 2013 at 02:41 AM.
And I want each video frame to be guarentied to be sent as a separate packet, so that the recieving application can have a separate DataArrival event for each frame, so that the code in the DataArrival event won't have to include code to parse the stream to find the start of each packet (and hope that it can find it and keep it all in sync), but rather the code in the DataArrival event will just be a simple winsock1.getdata and a SetDibits API call to display the received image. And the faster I can send the data without 2 adjacent packets becoming merged, the better (because the moment 2 packets become merged, my scheme breaks down, be cause all of a sudden I start to have to add code to the DataArrival event to separate the data back into discrete packets).
Well, if you're not intending to write anything at the Transport Layer you're going to have to live with the constraints that the underlying protocols place upon you. I'm not too sure what the issues are. Assembling 1Mb worth of data from 128 X 8K 'chunks' is not a complex task. Don't you just buffer the data up until 1Mb has been received and then do whatever you need to do?
Given that you're developing the Application Protocol wouldn't it be a fairly simple matter to send the length followed by the data; at the receiving end pick up the length and then keep buffering until that number or greater bytes have been received. Process the data and if there was more than expected, pick up the new length (if it's been received yet) and start buffering again. In VB it's a bit 'messy' having to copy Byte Arrays about but it's doable and I suspect others here will have some ideas about the methodology / efficient implementation - ADO Stream perhaps - I know that Dilettante has posted an/some example/examples on this Forum somewhere using that technology- it may be worth you searching.
While it may supposedly trigger a "sendcomplete" event even before the send is complete, I've never had this happen. In fact in various software I've made before, my experimental protocols depend on the previous send being complete before sending a new packet, and it depends on the sendcomplete event to indicate such (by setting a globally declared boolean var to TRUE, from the sendcomplete event), and I've never had it fail before.
Sending a length header and then the body works pretty well for binary data. There is a little juggling to do but it's a fairly routine thing. Who hasn't made use of the RtlMoveMemory API call?
An ADO Stream works quite well when performance isn't paramount. You can also gin up your own stream class and add bells and whistles to fit your purpose. Sort of a fancy binary version of the common "string builder" classes floating around.
We don't have a complete picture of the performance requirements. Webcams and 1MB bitmaps were mentioned but nothing at all about the frame rate needed.
It appears that questions about this same project have been asked elsewhere. What about the suggestion that was made to use JPEG compression? It is fairly fast and results in a lot of overhead reduction elsewhere, in particular in terms of network bandwidth use.
Here is a quick and dirty pair of projects based on code I already had on hand. There is a WebCam server project and a matching simple WebCam client project in the attached archive.
The server project requires Edanmo's OLELIB.TLB but most VB6 programmers probably already have that installed. You can still get it from his Visual Basic 6 page though.
I have reposted the attachment with a small optimization to the client. There was a silly thing being done that made it slower than it could be, though it wasn't a fatal or data corrupting bug. Some minor re-arrangement was done along with a few more comments. The DataArrival handler now looks like:
Code:
Private Sub Winsock_DataArrival(ByVal bytesTotal As Long)
Dim Bytes() As Byte
Dim FrameSize As Long
Winsock.GetData Bytes
With Buffer
'Append to end of buffer.
.Position = .Size
.WriteBytes Bytes
'Process any complete frame messages bufferred.
.Position = 0
Do While .HaveAtLeast(Len(FrameSize))
FrameSize = .ReadLong()
If .HaveAtLeast(FrameSize) Then
Bytes = .ReadBytes(FrameSize)
'Back up and delete the entire message.
.Position = 0
.DeleteBytes Len(FrameSize) + FrameSize
imgCamera.Picture = BytesToStdPicture(Bytes)
lblFrameBytes.Caption = Format$(FrameSize, "#,##0")
lblFramesPerSec.Caption = Format$(FPS(), "0.0")
Else
Exit Do
End If
Loop
End With
End Sub
Last edited by dilettante; Mar 24th, 2013 at 12:13 PM.
Reason: reposted attachment
Here is a quick and dirty pair of projects based on code I already had on hand. There is a WebCam server project and a matching simple WebCam client project in the attached archive.
The server project requires Edanmo's OLELIB.TLB but most VB6 programmers probably already have that installed. You can still get it from his Visual Basic 6 page though.
What's OLELIB.TLB, and where can I download a copy?
Yeah, the stylesheets this site is using result in pastel shades of gray and blue for everything, including link colors. These guys must just love the washed out look of the Windows 7 desktop.
I went back and manually underlined it to make it stand out just a little bit more.
Here is a slightly modified version that makes some changes to the server (including cleaning up a few comments and changing some Public items to Private that did not need to be Public).
Prior Version
In the prior version posted above, the server uses a UserControl called CineGrab (along with some helper modules) to perform the capture operation.
The capture is via Video for Windows (AviCap) and captured at a given frame-rate. However since the capture callback runs on a non-VB thread it can't use very many facilities provided by VB6 except pretty much straight, simple code. Even Declare'd API calls will fail because COM has not been initialized.
One of the things Edanmo's TLB is used for in the server is just that. The callback needs to use RtlMoveMemory ("CopyMemory") and since I was already using olelib.tlb which has it defined as MoveMemory I'm using just that.
But since the callback can't just raise a VB event or call something that raises an event, CineGrab was using a Timer to poll a global Boolean (ImageBytesReady) to know when an image frame is available.
This results in the sort of nasty situation that the Timer interval cannot be synced to the actual frame capture.
The two interval values used in the earlier version (frame capture and Timer) work well enough for most purposes, but could easily result in unnecessarily dropped frames or even frames sent twice!
This Version
To address this, the new version provided here gets down and dirty about this.
The Timer is still used now, but just as a way to kick off the frame fetching.
The capture callback routine still uses ImageBytesReady in the same way (since it is really used for 2-way communication between the callback and CineGrab) but adds a Win32 Event (handle hImageBytesReady here).
When a frame is ready the callback sets this Event. This runs afoul of the same issue as with RtlMoveMemory, so I am using a custom TLB (Kernel32SetEvent.tlb, provided) to make SetEvent In Kernel32 available to the callback routine.
Within CineGrab there is now a subroutine (PollFrame) that loops calling MsgWaitForMultipleObjects until hImageBytesReady has become "set" or something else that needs handling has occurred. If "something else" then DoEvents is called to handle it. The Timer is now used as a one-shot to kick off a call to PollFrame.
This means that while capturing, the entire rest of the program is being handled from inside the PollFrame subroutine.
Overkill
While this should seriously minimize (if not eliminate) frame sync issues very few programs ever need to take such radical steps. There is a small gain in performance but hardly enough to make such tricky programming worth the effort to write and support.
It was interesting to play with though.
Last edited by dilettante; Mar 24th, 2013 at 04:18 PM.
I looked back here and realized the original question was never answered.
Version 3 does this. It is Version 2 with Nagle suppression added.
Note that this gains us nothing in CPU performance and probably ends up costing a little network performance. The Nagle Algorithm is meant to enhance performance by keeping us from sending small TCP segments over the wire. Disabling it merely undoes that bit of assistance, shooting ourselves in the foot.
Not recommended for a streaming media application.
Late note to anyone coming upon this thread later:
The "clever" alternative to polling with a Timer that was used in versions 2 and 3 of the server in the examples above shouldn't be used. I find that it can be quite stable when using some cameras but quite unstable using others.
I'm finding that when it doesn't work there isn't any capture and streaming taking place and "ending" the server program leaves the process hung and you have to kill it.
Reply With Quote
Originally Posted by Ben321
