On Sat, Apr 25, 2009 at 09:12:44AM -0400, Aaron Wiebe wrote:
At this point I'm focusing on two operating systems only - linux and freebsd. I'd like to support Solaris as well, but the vast majority of our base is in the first two categories.
That's good. We can be probably be sure that modern Linux and FreeBSD systems have decent threading support.
This way the primary poll/read/parse thread is free to continue without actually having to manage write queues. In your originally suggested method, the context switch is extremely likely anyway, since there is a good chance that a target socket is "owned" by a different read thread. This way at least the read/parse thread can focus on reading and parsing, rather than trying to do everything.
I'm thinking this is what could happen to eliminate this problem: 4. If the write queue was empty before the new messages are added to a write queue, and the socket for the queue is marked as writable, then the thread tries to do a non-blocking write of the queue once. On success, we are done. On failure, it marks the socket as not writable, and moves on. The owning thread will then wake up when the socket becomes writable again, and will do the job of trying to flush the write queue until it is empty again. I'm currently playing with my own code that follows some of these threading ideas that we have discussed. Hopefully I'll have something interesting to show soon. -- Ned T. Crigler