Sockets and blocking...

[Stuka]

In coding a server app, if I open multiple sockets on the same port, will activity on one socket block all activity on that port, or only on that socket? I have a major project to work on involving server apps, and so I need all the theory I can get, and fast!

[LloydM]

Only on that socket. You also have to make sure though, that you don't block so you can service all the other sockets (eg with select, or non blocking io, or using separate threads/processes for the other sockets).

[Strike]

For handling multiple requests, here's the order in which I'd suggest trying to do it:

1. Using select()
2. Using multiple threads
3. Using non-blocking I/O (with fcntl(), I think it is)

With non-blocking I/O you become a CPU hog because you constantly poll the socket. With multiple threads, there's a bit of overhead there (and it's not as easy as select()).

[Stuka]

Ok, so if I'm using select(), from what I understand, I'm waiting for a given socket to be ready for I/O operations, then doing those I/Os, right? How would that be handled with multiple threads? Each thread doing its own bit of I/O? And how would that affect performance? One of the servers my group will have to do is an echo server, which shouldn't cause many problems, but the other is a chat server, which I can see being a resource hog if not done carefully.
<edit>PS, Strike, just read your profile - are you in school at UHCL? I'm just startin' down there....</edit>

[This message has been edited by Stuka (edited 10 November 2000).]

[jemfinch]

Originally posted by Strike:
With non-blocking I/O you become a CPU hog because you constantly poll the socket.


I don't think this can be true. Certainly not the part about hogging the CPU. The fastest web server in the world, Zeus, uses a non-blocking sockets + select() model to serve its content. Non-blocking IO and select will always be faster than a comparable forking/threading model due to the lower overhead.

Jeremy

[pdc]

You don't need to non-block on a server. You bind, listen and accept. You typically set non-blocking with ioctl when you don't want to wait for the timeout on a connect for a client.

The quick and dirty and not bad method, if your code is small, is simply to fork() a new piece of yourself on each accept.When you accept you get a copy of the socket. Close the one the client connected on and use the copy during the fork. You then process the request while the rest of the code goes back to a listen loop.
Paul






[This message has been edited by pdc (edited 11 November 2000).]

[Strike]

Originally posted by jemfinch:
I don't think this can be true. Certainly not the part about hogging the CPU. The fastest web server in the world, Zeus, uses a non-blocking sockets + select() model to serve its content. Non-blocking IO and select will always be faster than a comparable forking/threading model due to the lower overhead.

Jeremy
That is true, but what I think LloydM meant was to not use select(), just strictly non-blocking sockets which are polled constantly. That is certainly one way to do it. Just a really bad way to do it, because you will become a CPU hog with all the polling that has to go on for any decent interactivity. select() with non-blocking sockets is probably even faster than select() with blocking sockets, but I was talking about non-blocking sockets without select()

[Strike]

Originally posted by Stuka:
Ok, so if I'm using select(), from what I understand, I'm waiting for a given socket to be ready for I/O operations, then doing those I/Os, right? How would that be handled with multiple threads? Each thread doing its own bit of I/O? And how would that affect performance? One of the servers my group will have to do is an echo server, which shouldn't cause many problems, but the other is a chat server, which I can see being a resource hog if not done carefully.
<edit>PS, Strike, just read your profile - are you in school at UHCL? I'm just startin' down there....</edit>
First of all, no not at UH-Clear Lake, I'm in San Antonio for school (Trinity University) http://www.linuxnewbie.org/ubb/smile.gif Good to know a fellow Houstonian though (I actually live nowhere near Clear Lake, the drive to Johnson Space Center for work this past summer was about 45 minutes).

Anyway... with multiple threads, each thread handles a connection, yes.

I just wrote a fledgling little "chat" program myself for class, but it was only required to work for two people (but the twist was that the initial server had to be a client as well, in the same binary). Next we have to write a BBS-style server that can handle file requests and transfers among other things. Yahoo http://www.linuxnewbie.org/ubb/wink.gif

[jemfinch]

Originally posted by pdc:
You don't need to non-block on a server. You bind, listen and accept. You typically set non-blocking with ioctl when you don't want to wait for the timeout on a connect for a client.


You do have to set non blocking on the server. Select tells you if a socket is readable or writable, but it doesn't tell you how much you can read or write. You have to set non-blocking or you'll block on the writes where you try to write too much. Similarly, if you try to read more than is available to be read (and the other side didn't close the connection) you'll block in the read() call.


The quick and dirty and not bad method, if your code is small, is simply to fork() a new piece of yourself on each accept.When you accept you get a copy of the socket. Close the one the client connected on and use the copy during the fork. You then process the request while the rest of the code goes back to a listen loop.


A forking method is inherently slower than a non-blocking IO method since it involves forking a process, which is an expensive process (in linux, a task_struct, the struct of information regarding a process, is 980 or so bytes that need to be copied for each fork, in addition to many other things, like setting up addressable memory and so on)

Jeremy





[This message has been edited by pdc (edited 11 November 2000).][/B][/QUOTE]

[jemfinch]

Originally posted by Strike:
That is true, but what I think LloydM meant was to not use select(), just strictly non-blocking sockets which are polled constantly. That is certainly one way to do it. Just a really bad way to do it, because you will become a CPU hog with all the polling that has to go on for any decent interactivity. select() with non-blocking sockets is probably even faster than select() with blocking sockets, but I was talking about non-blocking sockets without select()

Ah, well that explains it. http://www.linuxnewbie.org/ubb/smile.gif

Check out http://www.acme.com/software/thttpd/benchmarks.html for a look at a few benchmarks run by thttpd (lies, damn lies, and benchmarks!) showing the performance of several different http server models (select+non-blocking IO, preforking, and threaded, mostly). Also see http://www.nightmare.com/medusa/medusa.html for some nice advocacy information about select+non-blocking IO.

Jeremy

[Sterling]

Best way to read or write from a socket (I've found this out programming an I/O stream like class for telnet sessions) is to use select() to determine if its ok to read or write. (although for telnet, you can always write, its just not garunteed how the server will handle the write) Then you use non-blocking I/O to read until you don't read anything. In that case, read() will return -1 and set errno to E_AGAIN, IIRC.

------------------
-Sterling
-This post made with the Lizard! (http://www.mozilla.org)

[jemfinch]

Originally posted by Sterling:
although for telnet, you can always write, its just not garunteed how the server will handle the write.


Unless I'm wrong (I could be, since telnet uses OOB data [the PSH flag]) you shouldn't be able to write more than the size of the tcp window.

Am I wrong? I'm curious (having never worked really closely with the telnet protocol)

Jeremy

[pdc]

Jeremy,
Don't worry about the window size, yet. TCP handles those details. Your window and buffer size control how the stack handles your data. Your socket buffer sizes don't directly correlate. TCP buffers fit within windows. Socket writes and reads fill TCP buffers, which when full are sent (unless TCPNODELAY is set). A window slides along keeping track of however many TCP buffers fit within it (or someting along those lines).

The client and server, if both are coded such, can set the window and buffer size. There a defaults and to use LARGE sizes, the tcp stack has to have rfc 1323 implmented to go beyond 64k. With the big bufs and tcpnodelay set, you can get close the max throughput of a given media. Caveat: no one else will get much done if one app is hogging the max thoughput of the network.

Paul
p.s. Telnet and FTP ( a telnet derivative ) are not the best client/server examples to work from. Look at the apache code and the original NCSA code for a good stateless reference.

[This message has been edited by pdc (edited 12 November 2000).]

[LloydM]

write *will* block, if your send buffer is full, or there is not enough room to write your data in the buffer.

You can set the send buffer size through SO_SNDBUF, but it's usually not necessary to do this, unless for some good reason. There are several cases in which the sendbuf is still full, eg the other end's recv buffer (usually the advertised window size) is full (user has not read the data yet), packets are being lost and we are still waiting for acks, etc.


Socket writes and reads fill TCP buffers, which when full are sent (unless TCPNODELAY is set)


Most TCP implementations do not wait for the buffers to fill, instead they implement the nagle algorithm, which usually waits for a full buffer or a time out whichever comes first. The TCP_NODELAY option basically turns this algorithm off, for applications that need better interaction like TELNET.


p.s. Telnet and FTP ( a telnet derivative ) are not the best client/server examples to work from. Look at the apache code and the original NCSA code for a good stateless reference.


Yes, if you really want performance, a stateless threadpool seems to be your best option.

Not having seen Apache's source code, I have heard that it is not exactly stateless, though, but pre-created threads/process with a mutex/critical section on accept.

This is not a bad technique either, but it doesn't scale well though, so I don't know whether what I've heard is true or not.

[pdc]

I've been nagled before....shudder. Super fast switched network with huge buffers...data would just set there. The vendor (Oracle) finally introduced code that would set tcpnodelay.

Apache does preload x number of threads. This is defined by the admin in the httpd.conf file. This is an attempt to handle connect flurries (or something along those lines).

Thinking about your comments further, whether using threads or forking, using non-blocking via ioctl might provide a better method of handling and cleaning up in a env where connection quality is not always guaranteed. Having a bunch of forks or threads setting out there in write/read limbo may consume resources better spent dealing with connections that are working.

Paul

[LloydM]

usually waits for a full buffer or a time out whichever comes first


I've been nagled before....shudder. Super fast switched network with huge buffers...data would just set there


Okay, sorry I lied. There is actually a low watermark for the send buffer (actually the receive has one too but defaults to 1, so it returns right away as soon as data arrives).

There's no need really for setting this, unless you know the behavior of your actual data, and can tweak it accordingly.

For example, if you are writing to the socket in small chunks, but frequently, you'll probably save some bandwidth if you just set the low water mark smaller, rather than turning off the nagle algorithm altogether.