During the process of upgrading my JACMail program to use SimpleSock, I ran into an issue when transferring large attachments. The program became noticeably faster using SimpleSock, but that speed caused some other issues. So I modified the SimpleSock demonstration program to test file transfers. Where you run into this issue will depend on the system you are using, as well as the other end and the network in between.
Outbound data is placed in a binary buffer, and dished out to Operating System in blocks that it can handle. On my Windows Vista, that block is 8,192 bytes, and on my Win 8.1, that block is 65,536 bytes. From there, the operating system will further break it down into the network packet size (about 1,500 bytes) before sending it out. Binary file transfer is actually simpler than formatted data transfer. Since disk I/O is buffered, as the binary data is received, it is added directly to the file.
To keep it simple, I sent out the file name in the first block, and the file data in the subsequent blocks. That required a 100 ms delay between blocks to prevent them from being added together. 10 ms worked in one direction, but not the other. Upon completion, there is also a 1 second (1000 ms) delay before issuing the Close Socket command to give the last packet time to reach it's destination.
The receiver end needs a "Downloads" sub-directory off the application directory to store the received files. The port number is entered and the "Listen for File" is clicked to put the receiver in the listen mode. On the sender end, the user selects a file to transfer using the "Get File" button. At this point, the user can optionally Base64 encode the file. If so, a ".B64" extension is added to the file name. Clicking on send establishes a connection with the destination/port, sends the file name, waits 100 ms, and then sends the file data. The receiver receives the file name and creates a file by that name in the "Downloads" directory. The subsequent file data is added to the file as it is received. When the sender closes the socket, the receiver closes the file. If the file is encoded, the received file can be decoded using the "Decode File" button. The result is stored under the original file name.
While building this demonstration program, I discovered a bug in SimpleSock in the binary "DeleteByte" function. How this escaped detection is beyond me, but I suspect that I was only dealing with single block transfers.
This demonstration program has been tested with various binary files up to 1,439,974 bytes. There is no reason that encrypted files could not be transferred as well, but the issue of key transfer has to be solved.
J.A. Coutts
Last edited by couttsj; Jul 20th, 2017 at 10:53 AM.
FileTest was never tested with Unicode, nor was it tested using the loopback address. Because of the speed with which loopback transactions are made, there could be timing issues. Having said that, the first image you sent does not look right. There should be 2 separate blocks sent, one with the file name, and one with the the actual data, as in the first image below.
The second image makes less sense. For sending and receiving files, the top row of controls are not used. By using the "Listen for File" button, the FileFlg is set and the the "EncrDataArrival" event is utilized. With the FileFlg set, the message you see in your bottom image should never appear.
To avoid confusion, I have modified the File Transfer program to permit Unicode file names, and to remove traces of the previous program. To accomplish this I am using the MS InkEdit Control.
After a little experimentation, I was able to speed things up a bit.
Code:
Option Explicit
Dim SendBusy As Boolean
Private Sub mSocket_Connect()
Dim bTmp() As Byte
Dim Msg As String
lblStatus.Caption = "Connected!"
bTmp = UniToByte(FileName)
mSocket.bOutBuffer = bTmp
SendBusy = True
mSocket.TCPSend
Do Until SendBusy = False
DoEvents
Loop
Sleep 10
mSocket.bOutBuffer = ByteBuffer
mSocket.TCPSend
End Sub
Private Sub mSocket_SendComplete()
SendBusy = False
FileFlg = FileFlg + 1
If FileFlg > 2 Then
Sleep 10
Call mSocket_CloseSck
End If
End Sub
Before sending data, the "SendBusy" flag is set. The flag is reset by the "SendComplete" event. The "Do" loop checks the flag, and must use a "DoEvents" function in order to allow the "SendComplete" event message to occur. The 10 ms Sleep is still required, and the Sleep was reduced to 10 ms before closing the socket.
After a little experimentation, I was able to speed things up a bit.
Code:
Option Explicit
Dim SendBusy As Boolean
Private Sub mSocket_Connect()
Dim bTmp() As Byte
Dim Msg As String
lblStatus.Caption = "Connected!"
bTmp = UniToByte(FileName)
mSocket.bOutBuffer = bTmp
SendBusy = True
mSocket.TCPSend
Do Until SendBusy = False
DoEvents
Loop
Sleep 10
mSocket.bOutBuffer = ByteBuffer
mSocket.TCPSend
End Sub
Private Sub mSocket_SendComplete()
SendBusy = False
FileFlg = FileFlg + 1
If FileFlg > 2 Then
Sleep 10
Call mSocket_CloseSck
End If
End Sub
Before sending data, the "SendBusy" flag is set. The flag is reset by the "SendComplete" event. The "Do" loop checks the flag, and must use a "DoEvents" function in order to allow the "SendComplete" event message to occur. The 10 ms Sleep is still required, and the Sleep was reduced to 10 ms before closing the socket.
J.A. Coutts
i think doevents sleep is not very well
may be used timer control. or timer cls
i think doevents sleep is not very well
may be used timer control. or timer cls
The problem with using a timer is "What do you set the timer interval to?". The sending of data could last 1/100 of a second, or a minute or more for a long file. The problem with the DoEvents loop that I have shown is that if something goes wrong, it could get stuck in an endless loop. That is where a timer or the Timer function might be useful. Personally, I find the Timer function quicker and easier to use.
Attached is the same test program using a different methodology. It uses a header with each outgoing record. That eliminates the need for a time separation between records to differentiate the File Name from the file data itself. The receiver is able to recover as much data as it can, but only process the data that it needs determined by the record length contained in the header. It is more complex, and it introduces a different problem. On long file transfers, the closing of the socket on the sender end causes the receiver to stop processing data before it has completed the transfer. That issue was addressed by adding a 5 second timer delay before closing the socket.
Alternative #3. This version lets the data accumulate in the class buffer, whereas the previous versions were store as you go. As each chunk of data is received, the "DataArrival" is fired and updates the amount of data available. When all the data is available, the calling program recovers and saves the data to file. There may be a limit to the amount of data that it can handle, but that has not been established.