The attached server program is my first attempt at using IOCP {Input Output Completion Port). It is based on one submitted by xiaoyao, but I am not sure who deserves the credit as it was a repost.
To be effective at fully serving the needs of a high usage server, IOCP needs to be used in Overlapped mode as well as Multi Threaded mode. Overlapped I/O in non-blocking or asynchronous mode using multiple threads based on the number of CPU cores available, delivers the highest throughput according to Microsoft. My own experience so far supports that. Unfortunately there is very little information out there on utilizing this function with VB6. What I could find was information about C++ and .NET implementations with echo servers, and this information was not very helpful. Stepping through the program to identify problems proved to be next to impossible without the IDE crashing, so I was left with a lot of trial and error solutions to fix bugs.
Although this program is currently designed to use non-encrypted data, the intention is to eventually add encryption. To this end the first transmission after connecting to the server is a simple Client Hello ("Hello") using a 5 byte header. Overlapped I/O apparently has the ability to split the data into separate buffers (eg. Header/Body), but I have not yet attempted to implement that.
Overlapped I/O uses a fixed length buffer along with a length variable. Therefore it is not necessary to clear the buffer on each pass. The size of that buffer is configurable, and should be adjusted to suit the application. An application that hosts many connections with relatively short messages should have a smaller buffer than one that exchanges larger messages. This has to be balanced against operations such as encryption. Longer buffers take more processor power than shorter buffers. For this particular application, I have chosen 8,192 bytes as a starting point.
When a request is passed to a thread with a WSARecv or WSASend, the request is directed to the appropriate thread defined by ComKey, which is the Completion Key being used as the index to the connection array.
Sending encrypted files over a network requires messages to be sent as data defined by a header that can be decrypted as a complete block of data as defined in: https://www.vbforums.com/showthread....13-VB6-GetFile
This worked well until I started experimenting with high speed server traffic over a Local Area network using IOCP in the IDE. File transfers must store the file blocks as they are received and decrypted to avoid having to store the entire file in memory before storage. Normally, network speed is slower than program speed, but in this scenario, the opening of the file for download was slow enough to only save the first record. Subsequently I have implemented an option to open the file in the client prior to starting the download.
Because much of the work is done by the API, the code is relatively short in comparison to my previous attempts at server networking. The downside is that it is also less flexible. The same buffer is used for sending as receiving. We signal the buffer to receive data by applying a "WSARecv". An endless loop in the worker thread detects incoming traffic with GetQueuedCompletionStatus (GQCS), the index for the connection is recovered, and the recovered bytes are made available. Whether or not the buffer is returned before it is complete is unknown at this time. To change from receiving to sending, we apply a "WSASend" with the bytes to be sent in "PerData(nPerIndex).Buf" and the number of bytes in "PerData(nPerIndex).Len". This program currently does not support IPv6.
That's the gist of it, with a few extra guidelines to control file streaming. There are 3 different scenarios I had to contend with. The first one is when the file is less than or equal in size to the buffer. The second is when the file size is greater than the buffer size, necessitating multiple records. The third is when the file size is evenly divisible by the buffer size, and this special case necessitates sending one more record with just the header. In both the second and third cases, the sending program detects that the last record was sent and changes the connection back to "Recv". Failure to do that will cause the IDE to crash.
The client end also detects that the last record was received and closes the file. A word of caution here. If the current file in the download directory is of a greater length than the new file of the same name, the file length of the original file will be retained. You must either kill or rename the present file before downloading.
There will probably be some bugs to fix. Feedback is welcome.
J.A. Coutts
Note: Original IOCP_Svr.zip & getfile.zip have temporarily been replaced. See post #12 for details
Last edited by couttsj; Jan 25th, 2024 at 03:00 PM.
A problem has already surfaced. When I compile the server program and attempt to run it, it crashes after initialization. I could really use some help with this one, because my experience with Multi Threading is very limited.
I commend you for the work you have done on multi threading in VB6, but in this particular instance that module is far more than what I need. The threads do not have to communicate with each other or any other controls, and do not need to be marshalled. I simply need to make them work fast and efficiently when compiled.
I commend you for the work you have done on multi threading in VB6, but in this particular instance that module is far more than what I need. The threads do not have to communicate with each other or any other controls, and do not need to be marshalled. I simply need to make them work fast and efficiently when compiled.
J.A. Coutts
Okay, then don't be surprised when you get crashes The module provides "flat" threads as well when you use vbCreateThread.
Okay, then don't be surprised when you get crashes The module provides "flat" threads as well when you use vbCreateThread.
Well, you are right. After more research, the standard EXE that VB6 produces apparently marshals all the threads into one. The ActiveX EXE does not. Also, I checked the IDE version to see if all the threads were being used, and they are. But the IDE crashes when I attempt to transfer multiple records over anything but the first connection.
How does one tell if more than one CPU core is being used?
It's better to make an AxDll (apartment threading) and create an object for each IOCP thread. I wouldn't recommend to use Std-EXE threading because you lose the debugging ability (each IOCP thread sleeps in most time). Of course you could use a small asm thunk (the cycle with GetQueuedCompletionStatus call) to transmit the data from an IOCP thread to the main thread using my module as well. Regarding to AxDll approach, i had the experience with such approach when i needed to run several instances of ActiveScripting engines simultaneously:
Code:
Public Function ThreadProc( _
ByRef tData As tThreadData) As Long
Dim hWnd As Long
Dim tMsg As MSG
Dim lRet As Long
Dim pObj As Long
If tData.hr < 0 Then Exit Function
hWnd = tData.hWnd
pObj = tData.pObjectRaw
Do
lRet = GetMessage(tMsg, 0, 0, 0)
If lRet = -1 Then
' // Error
tData.hr = E_FAIL
Exit Do
ElseIf lRet = 0 Then
' // Exit message queue
Exit Do
Else
TranslateMessage tMsg
DispatchMessage tMsg
End If
Loop While True
If tData.pObjectRaw Then
vbaObjSetAddref pObj, ByVal 0&
End If
End Function
Public Function ThreadInitProc( _
ByRef tData As tThreadData) As Long
Dim hr As Long
Dim tClsId As UUID
' // Create communication window
tData.hWnd = CreateWindowEx(0, THREAD_WND_CLASS, vbNullString, 0, 0, 0, 0, 0, HWND_MESSAGE, 0, tData.hInstance, ByVal 0&)
If tData.hWnd = 0 Then
hr = E_FAIL
GoTo exit_proc
End If
tData.lThreadID = GetCurrentThreadId()
tData.hThread = OpenThread(SYNCHRONIZE, 0, tData.lThreadID)
hr = CLSIDFromProgID(tData.sProgID, tClsId)
If hr < 0 Then GoTo exit_proc
' // Create object
hr = CoCreateInstance(tClsId, Nothing, CLSCTX_INPROC_SERVER, IID_IUnknown, tData.pObjectRaw)
If hr < 0 Then GoTo exit_proc
' // Marshal
hr = CoMarshalInterThreadInterfaceInStream(tData.tIID, ByVal tData.pObjectRaw, tData.pStream)
If hr < 0 Then GoTo exit_proc
tData.bInitialized = True
exit_proc:
If hr < 0 Then
If tData.pStream Then
vbaObjSetAddref tData.pStream, ByVal 0&
End If
If tData.hThread Then
CloseHandle tData.hThread
tData.hThread = 0
End If
If tData.lThreadID Then
tData.lThreadID = 0
End If
If tData.pObjectRaw Then
vbaObjSetAddref tData.pObjectRaw, ByVal 0&
End If
tData.bInitialized = False
If tData.hWnd Then
DestroyWindow tData.hWnd
tData.hWnd = 0
End If
End If
tData.hr = hr
End Function
...
Public Sub CreateObjectInThread( _
ByRef sProgID As String)
Dim hr As Long
Dim cObj As Object
If m_tThreadData.hThread Then
Err.Raise 5, "CThreadItem::CreateObjectInThread", "There is an active object in the thread"
End If
With m_tThreadData
.sProgID = sProgID
.tIID = IID_IDispatch
.hInstance = App.hInstance
.hr = 0
.hThread = 0
.hWnd = 0
.lThreadID = 0
.pObjectRaw = 0
.pStream = 0
End With
hr = SHCreateThread(AddressOf ThreadProc, m_tThreadData, CTF_COINIT_STA, AddressOf ThreadInitProc)
If hr < 0 Then Err.Raise hr
If m_tThreadData.hr Then Err.Raise m_tThreadData.hr
hr = CoGetInterfaceAndReleaseStream(ByVal m_tThreadData.pStream, IID_IDispatch, cObj)
If hr < 0 Then Err.Raise hr
Set m_cObjInThread = cObj
End Sub
I have made a little progress with this application. I discovered that when a thread is created, it is executed immediately, and that presented timing issues. Setting up the listening socket does not require that code to be part of the listening thread, so it was moved outside of the thread code.
In the IDE, it was discovered that only the first working thread was created. Using a brief delay (Sleep 10) in the working thread initialization loop solved that problem, and coincidently it solved the same issue in the executable. The executable would crash after initializing the first working thread, and now it crashes after initializing all the working threads.
Experimenting with the executable, it would remain running if I did not initialize the working threads. So I enabled the working thread initialization routine, but commented out the bulk of the code within the routine. Low and behold, the executable stayed running. So the problem in the executable seems to be related to the size of the code. I am finding it difficult to merge IOCP with any existing VB thread code, as IOCP already handles much of it.
It's better to make an AxDll (apartment threading) and create an object for each IOCP thread. I wouldn't recommend to use Std-EXE threading because you lose the debugging ability (each IOCP thread sleeps in most time). Of course you could use a small asm thunk (the cycle with GetQueuedCompletionStatus call) to transmit the data from an IOCP thread to the main thread using my module as well. Regarding to AxDll approach, i had the experience with such approach when i needed to run several instances of ActiveScripting engines simultaneously:
Code:
Public Function ThreadProc( _
ByRef tData As tThreadData) As Long
Dim hWnd As Long
Dim tMsg As MSG
Dim lRet As Long
Dim pObj As Long
If tData.hr < 0 Then Exit Function
hWnd = tData.hWnd
pObj = tData.pObjectRaw
Do
lRet = GetMessage(tMsg, 0, 0, 0)
If lRet = -1 Then
' // Error
tData.hr = E_FAIL
Exit Do
ElseIf lRet = 0 Then
' // Exit message queue
Exit Do
Else
TranslateMessage tMsg
DispatchMessage tMsg
End If
Loop While True
If tData.pObjectRaw Then
vbaObjSetAddref pObj, ByVal 0&
End If
End Function
Public Function ThreadInitProc( _
ByRef tData As tThreadData) As Long
Dim hr As Long
Dim tClsId As UUID
' // Create communication window
tData.hWnd = CreateWindowEx(0, THREAD_WND_CLASS, vbNullString, 0, 0, 0, 0, 0, HWND_MESSAGE, 0, tData.hInstance, ByVal 0&)
If tData.hWnd = 0 Then
hr = E_FAIL
GoTo exit_proc
End If
tData.lThreadID = GetCurrentThreadId()
tData.hThread = OpenThread(SYNCHRONIZE, 0, tData.lThreadID)
hr = CLSIDFromProgID(tData.sProgID, tClsId)
If hr < 0 Then GoTo exit_proc
' // Create object
hr = CoCreateInstance(tClsId, Nothing, CLSCTX_INPROC_SERVER, IID_IUnknown, tData.pObjectRaw)
If hr < 0 Then GoTo exit_proc
' // Marshal
hr = CoMarshalInterThreadInterfaceInStream(tData.tIID, ByVal tData.pObjectRaw, tData.pStream)
If hr < 0 Then GoTo exit_proc
tData.bInitialized = True
exit_proc:
If hr < 0 Then
If tData.pStream Then
vbaObjSetAddref tData.pStream, ByVal 0&
End If
If tData.hThread Then
CloseHandle tData.hThread
tData.hThread = 0
End If
If tData.lThreadID Then
tData.lThreadID = 0
End If
If tData.pObjectRaw Then
vbaObjSetAddref tData.pObjectRaw, ByVal 0&
End If
tData.bInitialized = False
If tData.hWnd Then
DestroyWindow tData.hWnd
tData.hWnd = 0
End If
End If
tData.hr = hr
End Function
...
Public Sub CreateObjectInThread( _
ByRef sProgID As String)
Dim hr As Long
Dim cObj As Object
If m_tThreadData.hThread Then
Err.Raise 5, "CThreadItem::CreateObjectInThread", "There is an active object in the thread"
End If
With m_tThreadData
.sProgID = sProgID
.tIID = IID_IDispatch
.hInstance = App.hInstance
.hr = 0
.hThread = 0
.hWnd = 0
.lThreadID = 0
.pObjectRaw = 0
.pStream = 0
End With
hr = SHCreateThread(AddressOf ThreadProc, m_tThreadData, CTF_COINIT_STA, AddressOf ThreadInitProc)
If hr < 0 Then Err.Raise hr
If m_tThreadData.hr Then Err.Raise m_tThreadData.hr
hr = CoGetInterfaceAndReleaseStream(ByVal m_tThreadData.pStream, IID_IDispatch, cObj)
If hr < 0 Then Err.Raise hr
Set m_cObjInThread = cObj
End Sub
when i needed to run several instances of ActiveScripting engines simultaneously:
Is there a test project for this? Thank you
SHCreateThread
vbaObjSetAddref
' // Marshal
hr = CoMarshalInterThreadInterfaceInStream(tData.tIID,
CoGetInterfaceAndReleaseStream(ByVal
Can you talk about why you use these APIs? I've never used these things before.
The reason for developing this program is that on larger file transfers, the Client program I was using (built with SimpleSock), was not fast enough to keep up with the IOCP server program I was working on, when both programs were operating in the IDE. I wanted to NOT use multithreading on the Client program, but I could not find a way to leave the endless loop in a signaled state without using the API function "CreateThread". So I ended up with a single threaded Client program instead.
Each record is preceded by a 5 byte header, containing the record Type, the 2 byte version, and the 2 byte record length.
To summarize the program flow:
- Initialize working thread.
- Connect to server port (5150).
- Send Client Hello (Type 1).
- Server responds with a list of available files and length (Type 8).
- Client adds these to the listBox.
- Client clicks on desired file, sends it to the server (Type 8), and opens the file for receipt.
- Server opens that file and starts sending in blocks of 8192 bytes (Type 9).
- Client receives each block, adds it to the file and decrements the file length.
- When the file length is less than or equal to the "BlockLen", the file is closed and the operation is complete.
All this worked quite nicely while operating in the IDE. But when I compiled the program and ran it, it crashed as soon as it attempted to save the file. A zero length file would be left open. I suspected that the problem might be a conflict between IOCP and the VB6 file system. That is why I experimented with the Windows file API (See: https://www.vbforums.com/showthread....ndows-File-API ). That appears to be a good guess, because when I changed the program to use the API, the executable crash problem went away.
For testing purposes, I have provided a server program that uses SimpleSock. It was easy to throw together, and SimpleSock is quite stable, but also quite slow. If you are going to use this program in the IDE for files more than a few records, be sure to disable some of the Debug.Print statements, as the Immediate window has very limited capacity.
This is a work in progress, as is the IOCP server program. To implement IOCP in this Client program, I used a single structure to define the server, but I retained the structure array for the Client in case I wanted to allow it to support more than one download at a time. I have temporarily removed the IOCP server program as I try to resolve some of the issues in that particular program.
J.A. Coutts
Last edited by couttsj; Jan 26th, 2024 at 12:56 AM.
Some Chinese have achieved iocp server for webserver,websocket
the goal of supporting 100,000 concurrency without crashing.
You definitely need to use multithreading.
Some Chinese have achieved iocp server for webserver,websocket
the goal of supporting 100,000 concurrency without crashing.
You definitely need to use multithreading.
The need for multi threading on the Client end is highly questionable considering the amount of resources it requires. On the Server end however, it is very desirable.
Reply With Quote
Originally Posted by The trick
The module provides "flat" threads as well when you use vbCreateThread.
