[DALnet-src] Libd Design (Bahamut 2.0)

24 Apr 2009

      Greetings All,

As promised, here is a basic rundown on the library that I've been
working on that I see as the basis of Bahamut 2.0.  I'm currently
calling it libd - lib daemon, because I'm so original and creative.
At the bottom of this email is the code of a simple echo server based
on the library, which might be useful as a reference.

I'm working on a slightly object oriented model, while still using
straight C.  The general idea is simple - the library implements all
lower level functions regarding nonblocking socket interaction.  It
(will) support ipv6, compression, SSL, and threading.  At current, it
supports ipv6 only of those features.  The API is designed based on an
object model, for example:

To create a listener, call the routine create_listener within the
socket engine, get an "object", and set up some basic things you want
associated with the listener.  The minimum would be to give the
library a function to identify what constitutes a message, then a
function that would parse and act on that message.  You could also
define functions that would be called when we accept a connection and
when a connection is dropped.

There are four core objects - the socket engine handler itself,
SockEng, the listener object, Listener, the client object, Client, and
the grouping object, Group.

SockEng is used to manage the "global" definitions of the library -
such as defining error handling routines.
Listeners are used to manage listeners, and what default settings a
client has when they connect.
Clients are used to interact with clients.
Groups are used to group together different sets of clients, and send
to sets of clients using some optimizations accordingly.

This gives a very high-level overview - but I'd like to talk about the
threading model I have in mind.  This hasn't been implemented yet, but
I've been designing a lot of the system with this in mind for the
future.  I've decided that threading is important to support simply
because the recent crop of CPU designers hate programmers enough to
effectively force it on us.  Threading has always been a point of
heated argument in the past - how do we do it?  I've come up with the
following design:

1.  Poll thread - basically hosts the polling engine, whatever it happens to be.
2.  Read/Write thread set(s).  Watch queues and polling states, reads
and writes from and to sockets, and interacts with queues.
3.  Event trigger thread(s).  Schedules and triggers events on timers,
for anything that isn't edge-triggered from a socket read.
4.  Parsing/worker thread pool.  Executes actual work from event or
read threads, and places data into write thread queues.

I'm trying to design the library API in such a fashion that it
actually handles most of the threading logic.  However, getting this
to work as we want will take a lot of time, but I have a feeling that
IRCd will thread quite well in this method.  My hope is to build a
library that will be flexible enough, without being too complex, that
it could be applied to other projects.

So what are people thinking so far?  Does this look like a good
direction?  Does anyone have comments on the current design?  I'll try
to clean up the code and get it available sometime in the next week
for people to start actually poking.

Talk!  Discuss!  Criticize!  Insult!  Lets get talking.
-epiphani

====== test.c =======
#include "sockeng.h"

int client_packet_delim(Client *c, char *buf, int len)
{
        int i = 0;
        /* find a \n */
        while(len > i) {
                if(buf[i] == '\n')
                        return ++i;
                i++;
        }
        return 0;
}

int client_echo_parser(Client *c, char *start, int len)
{
        char buf[BUFSIZE];
        memset(buf, 0, BUFSIZE);
        snprintf(buf, len+1, "%s", start);
        c->send(c, buf, len);
        return 0;
}

void client_disconnecty(Client *s, int err)
{
        printf("Client disconnected:  %s\n", strerror(err));
}

int main(int argc, char *argv[])
{
        SockEng *s;
        Listener *l;

        s = init_sockeng();
        l = s->create_listener(s, 1111, NULL);
        if(!l) {
                printf("no listener create\n");
                return -1;
        } else {
                l->set_packeter(l, client_packet_delim);
                l->set_parser(l, client_echo_parser);
                l->set_onclose(l, client_disconnecty);
        }
        while(1) {
                if(s->poll(s, 1)) {
                        printf("poll error\n");
                        return -1;
                }
        }
        return 0;
}