Skip Menu |

This queue is for tickets about the Net-Server CPAN distribution.

Report information
The Basics
Id: 89586
Status: open
Priority: 0/
Queue: Net-Server

People
Owner: Nobody in particular
Requestors: FGA [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Wishlist
Broken in: 2.007
Fixed in: (no value)



Subject: Explicitely rejecting incoming connections when all workers are busy
Hi, We're using Net::Server::PreFork for an LMTP mail server. So far so good, everything works. Great work on a very useful module. Our problem is the project specs say when no workers are available and email arrives, we should return "432 Too many sessions" instead of waiting until a worker finishes and is able to call accept() on the socket. We have set the *_spare_servers variables to 0 since we don't actually want to spawn more children. (We're only using PreFork instead of PreForkSimple to have a chance to get more accurate tallies of waiting/processing workers, really.) There doesn't seem to be a way to do this in Net::Server::PreFork (or Simple). I've tried various things with e.g. the idle_loop_hook, but that doesn't get run when I need it. The accept_deny stuff is ideally placed (even though that's not what it's for), but it only runs in the workers, who have no idea what the other workers are doing anyway. Basically I'm looking for: (a) if and only if all workers are busy (b) and at least one incoming connection is pending (c) the master process itself (not necessarily, but it's the only one who has a reasonable expection of knowing if (a) is true) takes responsability for servicing the pending connections, if only to tell the client that they have to come back later Is there such a thing in Net::Server, or a workaround for it? (Also on the wishlist: exposing the various worker tallies as methods available in the master process, with documentation on when they are available/correct/out of date.)
Subject: Re: [rt.cpan.org #89586] Explicitely rejecting incoming connections when all workers are busy
Date: Thu, 17 Oct 2013 09:40:44 -0600
To: bug-Net-Server [...] rt.cpan.org
From: Paul Seamons <paul [...] seamons.com>
So, the issue you have below is somewhat fun - I have not previously been familiar with LMTP. The protocol and the 432 response message are rather odd beasts (actually mysql connections will often return a similar message if you exceed the maximum). Perhaps something can be woven into Net::Server directly, but this sort of checking in a prefork (vs fork or threaded) server such requires the use of some sort of central balance checking with all of the race condition preventions necessary to keep it safe. In a Net::Server::Fork server since it is the parent that is doing the listening - it could be possible to modify the loop that occurs when $n_children > $prop->{'max_servers'} to attempt to also see if there are incoming connections as well, and optionally throw them to some other handler (including all of the possible SSL negotiations needed). Threaded (which isn't yet in Net::Server but likely will eventually) could use a local shared variable to keep track of the total number of connections. But on PreFork, you have somehow coordinate this work between all of the child processes and the parent. In a PreFork server, the parent does know how many children are working and controls how many children there are lying around, but the parent is not directly selecting/accepting on the open server socket ports and has no easy built in way to communicate back to the children (well - it sort of does). But here is something I could imagine working (pseudo code only) Turn on child_communication 1 during server start up. Add a sub child_is_talking_hook {} that listens for requests from children in the parent. When a request is received, it can look at the current total number of children vs what the limits are. If the number of active children are at the max_servers, or max_servers - 2, or whatever threshold you set, then you could send a signal back to the child to return a 432. Otherwise if the parent thinks there is capacity left it could just signal to go ahead and process the request. Perhaps something like: sub child_is_talking_hook { my ($self, $child_sock) = @_; my $req = <$child_sock>; my $prop = $self->{'server'}; if (($prop->{'tally'}->{'processing'} || 0) >= $prop->{'max_servers'}) { # potentially stale print $child_sock "432\n"; } else { print $child_sock "proceed\n"; } } In the child you would have something like this at the top of your process_request hook: sub process_request { my $self = shift; my $parent_sock = $self->{'server'}->{'parent_sock'}; print $parent_sock "New Request\n"; my $line = <$parent_sock>; if ($line =~ /^432/) { print "432 Too many sessions\r\n"; return; } # do the normal processing here } There seem to be two options - both of them require a bit of magic. Use PreForkSimple with a predetermined number of children. Use some sort of shared p On 10/17/2013 06:21 AM, Fabrice Gabolde via RT wrote: Show quoted text
> Thu Oct 17 08:21:23 2013: Request 89586 was acted upon. > Transaction: Ticket created by FGA > Queue: Net-Server > Subject: Explicitely rejecting incoming connections when all workers are busy > Broken in: 2.007 > Severity: (no value) > Owner: Nobody > Requestors: FGA@cpan.org > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=89586 > > > > Hi, > > We're using Net::Server::PreFork for an LMTP mail server. So far so > good, everything works. Great work on a very useful module. > > Our problem is the project specs say when no workers are available and > email arrives, we should return "432 Too many sessions" instead of > waiting until a worker finishes and is able to call accept() on the > socket. We have set the *_spare_servers variables to 0 since we don't > actually want to spawn more children. (We're only using PreFork > instead of PreForkSimple to have a chance to get more accurate tallies > of waiting/processing workers, really.) > > There doesn't seem to be a way to do this in Net::Server::PreFork (or > Simple). I've tried various things with e.g. the idle_loop_hook, but > that doesn't get run when I need it. The accept_deny stuff is ideally > placed (even though that's not what it's for), but it only runs in the > workers, who have no idea what the other workers are doing anyway. > > Basically I'm looking for: > > (a) if and only if all workers are busy > > (b) and at least one incoming connection is pending > > (c) the master process itself (not necessarily, but it's the only one > who has a reasonable expection of knowing if (a) is true) takes > responsability for servicing the pending connections, if only to > tell the client that they have to come back later > > Is there such a thing in Net::Server, or a workaround for it? > > (Also on the wishlist: exposing the various worker tallies as methods > available in the master process, with documentation on when they are > available/correct/out of date.) >
On Fri, 18 Oct 2013 17:22:02 -0400, RHANDOM wrote: Show quoted text
> So, the issue you have below is somewhat fun - I have not previously > been familiar with LMTP. The protocol and the 432 response message > are rather odd beasts (actually mysql connections will often return > a similar message if you exceed the maximum).
Well, I don't think "432 Too many sessions" is an actual message specified by any RFC. Many MTAs define their own custom error codes, though. This appears to be purely an invention of our own, but it does fit a 4xx temporary error. Show quoted text
> [elided] > > [On] PreFork, you have somehow coordinate this work between all of > the child processes and the parent. In a PreFork server, the parent > does know how many children are working and controls how many > children there are lying around, but the parent is not directly > selecting/accepting on the open server socket ports and has no easy > built in way to communicate back to the children (well - it sort of > does). But here is something I could imagine working (pseudo code > only) > > [solution elided]
This seems like it could work. It has the disadvantages of requiring a couple extra workers, and having every worker block on the master process so they can check if they can relay messages normally, but that's probably manageable. Show quoted text
> There seem to be two options - both of them require a bit of magic. > > Use PreForkSimple with a predetermined number of children. Use some > sort of shared p
... It would appear RT cut your reply short :/ It turns out this part of the specs is not necessary after all. After we got access to the client's previous system, which we would replace with the LMTP server I'm developing, we realized they only have a couple dozen emails per day that need relaying through this custom gateway. So our initial 10 workers are probably already overkill, and in any case even if somehow all 25 emails arrived at the same time we're still managing sub-second delays with the current configuration. I'm still academically interested in your second solution, though! Thanks for the pointers -- Fabrice Gabolde