Homemade Internet Service Relay (V)

To achieve high performance, my implementation do two things. Asynchronous I/O and no redundant copies.

First, it never block any threads on I/O requests. This can reduces resource consumption. When multiple requests arrives when all threads are in use (blocked or not), requests need to be served by new threads. Thread creation takes significant amount of time. If we eliminate the possibility that a thread is blocked on I/O, we significantly decrease the need for thread creation.

Practical experience is, if you can make sure the server don’t block on I/O for all operations within a request, then we can save significant amount of time. But if some operation block on I/O anyway, then we are essentially in the situation of having one thread per request, and doing asynchronous I/O will be meaningless.

Another observation is that copying of the data take significant amount of time as most of the relay is handshaking the messages and very little other processing. The design attempts to reduce to amount of copy to minimum, except from the stream abstraction we are required to copy the data to the user specified buffer, which we cannot avoid if we wanted to reuse the same abstraction.