Greetings All, As promised, here is a basic rundown on the library that I've been working on that I see as the basis of Bahamut 2.0. I'm currently calling it libd - lib daemon, because I'm so original and creative. At the bottom of this email is the code of a simple echo server based on the library, which might be useful as a reference. I'm working on a slightly object oriented model, while still using straight C. The general idea is simple - the library implements all lower level functions regarding nonblocking socket interaction. It (will) support ipv6, compression, SSL, and threading. At current, it supports ipv6 only of those features. The API is designed based on an object model, for example: To create a listener, call the routine create_listener within the socket engine, get an "object", and set up some basic things you want associated with the listener. The minimum would be to give the library a function to identify what constitutes a message, then a function that would parse and act on that message. You could also define functions that would be called when we accept a connection and when a connection is dropped. There are four core objects - the socket engine handler itself, SockEng, the listener object, Listener, the client object, Client, and the grouping object, Group. SockEng is used to manage the "global" definitions of the library - such as defining error handling routines. Listeners are used to manage listeners, and what default settings a client has when they connect. Clients are used to interact with clients. Groups are used to group together different sets of clients, and send to sets of clients using some optimizations accordingly. This gives a very high-level overview - but I'd like to talk about the threading model I have in mind. This hasn't been implemented yet, but I've been designing a lot of the system with this in mind for the future. I've decided that threading is important to support simply because the recent crop of CPU designers hate programmers enough to effectively force it on us. Threading has always been a point of heated argument in the past - how do we do it? I've come up with the following design: 1. Poll thread - basically hosts the polling engine, whatever it happens to be. 2. Read/Write thread set(s). Watch queues and polling states, reads and writes from and to sockets, and interacts with queues. 3. Event trigger thread(s). Schedules and triggers events on timers, for anything that isn't edge-triggered from a socket read. 4. Parsing/worker thread pool. Executes actual work from event or read threads, and places data into write thread queues. I'm trying to design the library API in such a fashion that it actually handles most of the threading logic. However, getting this to work as we want will take a lot of time, but I have a feeling that IRCd will thread quite well in this method. My hope is to build a library that will be flexible enough, without being too complex, that it could be applied to other projects. So what are people thinking so far? Does this look like a good direction? Does anyone have comments on the current design? I'll try to clean up the code and get it available sometime in the next week for people to start actually poking. Talk! Discuss! Criticize! Insult! Lets get talking. -epiphani ====== test.c ======= #include "sockeng.h" int client_packet_delim(Client *c, char *buf, int len) { int i = 0; /* find a \n */ while(len > i) { if(buf[i] == '\n') return ++i; i++; } return 0; } int client_echo_parser(Client *c, char *start, int len) { char buf[BUFSIZE]; memset(buf, 0, BUFSIZE); snprintf(buf, len+1, "%s", start); c->send(c, buf, len); return 0; } void client_disconnecty(Client *s, int err) { printf("Client disconnected: %s\n", strerror(err)); } int main(int argc, char *argv[]) { SockEng *s; Listener *l; s = init_sockeng(); l = s->create_listener(s, 1111, NULL); if(!l) { printf("no listener create\n"); return -1; } else { l->set_packeter(l, client_packet_delim); l->set_parser(l, client_echo_parser); l->set_onclose(l, client_disconnecty); } while(1) { if(s->poll(s, 1)) { printf("poll error\n"); return -1; } } return 0; }
<snip>
s = init_sockeng(); l = s->create_listener(s, 1111, NULL); </snip>
IP? How would it detect IP protocol? Is it limited to just tcp, or is udp an option? How about low level socket operations, such as buffers, reuse and timeouts? <snip>
l->set_packeter(l, client_packet_delim); l->set_parser(l, client_echo_parser); l->set_onclose(l, client_disconnecty); </snip>
Perhaps these should be set in a fashion similar to setsockopt, so as to decrease the overall footprint of the system? 'Tis a lot easier for a compiler and strip to optimize a single function with a switch/case than 3+ functions. I'm also interested in how you'd go about passing messages between threads without corruption and without blocking. (That's one of many things I suck at when it comes to threads.)
On Fri, Apr 24, 2009 at 12:17 PM, Michael Reynolds <michael.reynolds@gmail.com> wrote:
<snip>
s = init_sockeng(); l = s->create_listener(s, 1111, NULL); </snip>
IP? How would it detect IP protocol? Is it limited to just tcp, or is udp an option? How about low level socket operations, such as buffers, reuse and timeouts?
The API is somewhat incomplete in the example, but at present there is no UDP support. I would like to add that sometime in the future. In terms of setsockopts type options, I do have stubs for functions to do those presently. They're not tied in yet.. but the plan is to support them. The idea, however, is to give defaults that will work for 90% of what we'd want to do.
<snip>
l->set_packeter(l, client_packet_delim); l->set_parser(l, client_echo_parser); l->set_onclose(l, client_disconnecty); </snip>
Perhaps these should be set in a fashion similar to setsockopt, so as to decrease the overall footprint of the system? 'Tis a lot easier for a compiler and strip to optimize a single function with a switch/case than 3+ functions.
I like this, good idea. I'll add that to my TODO list (or maybe you could do it after I release the code! :D)
I'm also interested in how you'd go about passing messages between threads without corruption and without blocking. (That's one of many things I suck at when it comes to threads.)
That's actually one thing that I've come up against. We can avoid actual blocking, but the methods of passing data between threads is still a bit fuzzy in my mind. If someone has a good general-purpose lockless queuing algorithm kicking around, I'd love to see it. -epi
On Fri, Apr 24, 2009 at 11:09:56AM -0400, epiphani wrote:
I've been designing a lot of the system with this in mind for the future. I've decided that threading is important to support simply because the recent crop of CPU designers hate programmers enough to effectively force it on us.
You may have trouble with operating systems that emulate threads in userspace.
Threading has always been a point of heated argument in the past - how do we do it? I've come up with the following design:
1. Poll thread - basically hosts the polling engine, whatever it happens to be. 2. Read/Write thread set(s). Watch queues and polling states, reads and writes from and to sockets, and interacts with queues. 3. Event trigger thread(s). Schedules and triggers events on timers, for anything that isn't edge-triggered from a socket read. 4. Parsing/worker thread pool. Executes actual work from event or read threads, and places data into write thread queues.
This design uses too many threads that need to talk to each other. For example, when there is new data to be read on a socket, this is what will happen: 1. The kernel will tell the poll thread that there is activity on a socket. The poll thread will then wake up and then have to inform the read thread that there is activity to read (for example, by adding the socket to a queue, or saving the new poll state somewhere) 2. The read thread will then wake up and realize that there is new activity to be read on the socket, and read it. Whatever is read from the socket will be queued up for the parsing/worker thread to process. 3. The parsing/worker thread will wake up and realize that there is new incoming activity to parse and process. It will then do that work, and then presumably have a response to write out. The outgoing response is then queued up for the write thread to write out to the socket. 4. The write thread will then wake up and realize that there is new activity to be written on the socket, and write it out. Each step will require a context switch to transfer control to a different thread. This will be especially bad if not all threads are on the same processor/core in a SMP machine. In order to finish processing activity on a socket, we will be bouncing back and forth between threads before we are done. On a non-SMP machine, this just multiplies the work the processor has to do, since the processor can only do one thing at a time anyway. On a SMP machine, this limits our ability to use multiple processors in parallel, as we have extra context switches to handle the threads talking to each other. A better approach is to minimize the number of threads that need to be woken up to completely process some activity. We could have an accept thread which does nothing else other than wait for activity on all the listening sockets and accept connections. When a new connection arrives, it can assign a connection to a worker thread. Each worker thread then waits for activity on only the connections that it has been assigned, and handles all the work itself. We can then limit the number of worker threads to some configurable number that will allow each processor to share the load of processing activity. -- Ned T. Crigler
Good points - response inline... On Fri, Apr 24, 2009 at 9:49 PM, Ned T. Crigler <crigler@gmail.com> wrote:
You may have trouble with operating systems that emulate threads in userspace.
At this point I'm focusing on two operating systems only - linux and freebsd. I'd like to support Solaris as well, but the vast majority of our base is in the first two categories.
This design uses too many threads that need to talk to each other. For example, when there is new data to be read on a socket, this is what will happen: <snip> Each step will require a context switch to transfer control to a different thread. This will be especially bad if not all threads are on the same processor/core in a SMP machine. In order to finish processing activity on a socket, we will be bouncing back and forth between threads before we are done.
We could have an accept thread which does nothing else other than wait for activity on all the listening sockets and accept connections. When a new connection arrives, it can assign a connection to a worker thread.
Each worker thread then waits for activity on only the connections that it has been assigned, and handles all the work itself.
I like this idea - however, it does introduce a slightly complicated behavior - if you have "ownership" of a client by one thread, then messages triggered by other clients (which is 90% of things) gets a bit more tricky. What do you think about combining the two methods - ownership of clients for reading and polling purposes, but still having a separate writing thread (set)? This way you're not actually waking up the somewhat busy read/poll thread for that client on every write queue insert. As follows: 1. Client is accepted by the accept thread, and assigned to a reading/polling/parsing thread. 2. A message is sent by the client to a channel, for which there are 2 clients involved. 3. A shared buffer is created by the parsing thread, and inserted into the other two client queues. 4. The write thread is then woken up, checks the current write state for the clients, and sends if possible. This way the primary poll/read/parse thread is free to continue without actually having to manage write queues. In your originally suggested method, the context switch is extremely likely anyway, since there is a good chance that a target socket is "owned" by a different read thread. This way at least the read/parse thread can focus on reading and parsing, rather than trying to do everything. Thoughts? -epi
On Sat, Apr 25, 2009 at 09:12:44AM -0400, Aaron Wiebe wrote:
At this point I'm focusing on two operating systems only - linux and freebsd. I'd like to support Solaris as well, but the vast majority of our base is in the first two categories.
That's good. We can be probably be sure that modern Linux and FreeBSD systems have decent threading support.
This way the primary poll/read/parse thread is free to continue without actually having to manage write queues. In your originally suggested method, the context switch is extremely likely anyway, since there is a good chance that a target socket is "owned" by a different read thread. This way at least the read/parse thread can focus on reading and parsing, rather than trying to do everything.
I'm thinking this is what could happen to eliminate this problem: 4. If the write queue was empty before the new messages are added to a write queue, and the socket for the queue is marked as writable, then the thread tries to do a non-blocking write of the queue once. On success, we are done. On failure, it marks the socket as not writable, and moves on. The owning thread will then wake up when the socket becomes writable again, and will do the job of trying to flush the write queue until it is empty again. I'm currently playing with my own code that follows some of these threading ideas that we have discussed. Hopefully I'll have something interesting to show soon. -- Ned T. Crigler
participants (4)
-
Aaron Wiebe -
epiphani -
Michael Reynolds -
Ned T. Crigler