eSeGeCe
software
Indy Servers - IOCP Windows (2 / 3)
From sgcWebSockets 2022.9.0 the Indy Server IOCP IOHandler has been rewritten from scratch and performance improved. The IOCP IOHandler is only available on sgcWebSockets Enterprise Package.
Using IOCP you can avoid the "one-thread-per-client" problem where the performance decrease a lot as more connections are handled by the server. IOCP provides a few threads that handle multiple clients. The threads are suspended and don't use CPU cycles until there is something to process.
Configuration
To enable IOCP for Indy Servers, Go to IOHandlerOptions property and select iohIOCP as IOHandler Type.
Server.IOHandlerOptions.IOHandlerType := iohIOCP; Server.IOHandlerOptions.IOCP.IOCPThreads := 0; // the number of IOCP threads will be calculated automatically using the number of processors. Server.IOHandlerOptions.IOCP.WorkOpThreads := 0;
1. IOCPThreads are the threads used for IOCP asynchronous requests (overlapped operations), by default the value is zero which means the number of threads are calculated using the number of processors (except for Delphi 7 and 2007 where the number of threads is set to 32 because the function cpucount is not supported).
2. WorkOpThreads only must be enabled if you want that connections are processed always in the same thread. When using IOCP, the requests are processed by a pool of threads, and every request (for the same connection) can be processed in different threads. If you want to handle every connection in the same thread set in WorkOpThreads the number of threads used to handle these requests. This impacts in the performance of the server and it's only recommended to set a value greater of zero only if you require this feature.
Enabling IOCP for windows servers is recommended when you need handle thousands of connections, if your server is only handling 100 concurrent connections at maximum you can stay with default Indy Thread model.
Performance Test
A simple test will show the differences between the Indy Thread model and IOCP. The test will connect to the server using websocket protocol and the cpu usage and memory consumption will be shown in the following table. The % usage of cpu is while the server is IDLE.
| Num. Connections | Indy Default IOHandler | Indy IOCP IOHandler |
| 100 | 0% (4.1MB) | 0% (3.9MB) |
| 500 | 0% (17.1MB) | 0% (7.7MB) |
| 1000 | 0% (32.2MB) | 0% (12.4MB) |
| 1500 | 7.3% (46.8MB) | 0% (17.1MB) |
| 2000 | 15.4% (61.6MB) | 0% (21.9MB) |
| 2500 | 51.9% (76.5MB) | 0% (26.5MB) |
| 3000 | 68.8% (91.7MB) | 0% (31.2MB) |
| 3500 | 72.3% (106MB) | 0% (35.9MB) |
The Indy server started to experience slowdowns with more than 3000 concurrent connections, most probably due to the thread context switch and the cpu usage was very high while the server was idle.
The IOCP server was using no CPU while it was in idle state and the memory consumption was lower too.
The next test measures how much time it takes to connect 3000 clients, each sends a message when connected to the server and the server echoes this message.
| Indy Default IOHandler | Indy IOCP IOHandler | |
| Connect 3000 clients and echo message | 15.32 seconds | 6.89 seconds |
The Performance Tests have been done in a Windows Server 2022 with 16 cores and RAM 32GB.
When you subscribe to the blog, we will send you an e-mail when there are new updates on the site so you wouldn't miss them.
