| Subject: | More efficient close()-on-fork |
As discussed...
The close()-on-fork behaviour slows down a bit when max FD is high, for example running under
ulimit -n 1000000
Making this OS-specific and using the /proc/$$/fd interface on Linux helps performance there noticeably, other platforms might have similar facilities (or at least the ability to report max actual used FD).
Patch attached for moving this to a method on IO::Async::OS and a basic ::linux implementation, unfortunately it includes the /proc/$$/fd dir handle since I couldn't see a way to exclude this fileno from the list (fileno($dir) returns undef).
Does this need a platform-specific test? Maybe override CORE::close and check that it's not being called too often?
cheers,
Tom
| Subject: | 2014-08-11-linux-fd-close.patch |
=== modified file 'lib/IO/Async/ChildManager.pm'
--- lib/IO/Async/ChildManager.pm 2014-07-11 14:16:53 +0000
+++ lib/IO/Async/ChildManager.pm 2014-08-11 19:22:19 +0000
@@ -604,7 +604,7 @@
}
}
- foreach ( 0 .. OPEN_MAX_FD ) {
+ foreach ( IO::Async::OS->potentially_open_fds ) {
next if $fds_refcount{$_};
next if $_ == fileno $writepipe;
POSIX::close( $_ );
=== modified file 'lib/IO/Async/OS.pm'
--- lib/IO/Async/OS.pm 2014-07-11 14:16:53 +0000
+++ lib/IO/Async/OS.pm 2014-08-11 18:57:26 +0000
@@ -30,6 +30,9 @@
use IO::Socket (); # empty import
+use POSIX qw( sysconf _SC_OPEN_MAX );
+use constant OPEN_MAX_FD => eval { sysconf(_SC_OPEN_MAX) } || 1024;
+
# Some constants that define features of the OS
use constant HAVE_SOCKADDR_IN6 => defined eval { pack_sockaddr_in6 0, inet_pton( AF_INET6, "2001::1" ) };
@@ -565,6 +568,19 @@
undef $SIG{$signal};
}
+=head2 my @fds = IO::Async::OS->potentially_open_fds
+
+Returns a list of filedescriptors which might need
+closing. By default this will return _SC_OPEN_MAX,
+OS-specific subclasses may have a better guess.
+
+=cut
+
+sub potentially_open_fds
+{
+ 0 .. OPEN_MAX_FD;
+}
+
=head1 AUTHOR
Paul Evans <leonerd@leonerd.org.uk>
=== added file 'lib/IO/Async/OS/linux.pm'
--- lib/IO/Async/OS/linux.pm 1970-01-01 00:00:00 +0000
+++ lib/IO/Async/OS/linux.pm 2014-08-11 19:23:54 +0000
@@ -0,0 +1,55 @@
+# You may distribute under the terms of either the GNU General Public License
+# or the Artistic License (the same terms as Perl itself)
+#
+# (C) Paul Evans, 2014 -- leonerd@leonerd.org.uk
+
+package IO::Async::OS::linux;
+
+use strict;
+use warnings;
+
+our $VERSION = '0.63';
+
+our @ISA = qw( IO::Async::OS::_Base );
+
+=head1 NAME
+
+C<IO::Async::OS::linux> - operating system abstractions on C<Linux> for C<IO::Async>
+
+=head1 DESCRIPTION
+
+This module contains OS support code for C<Linux>.
+
+See instead L<IO::Async::OS>.
+
+=cut
+
+# Try to use /proc/pid/fd to get the list of
+# actually-open file descriptors for our
+# process. Saves a bit of time when running with
+# high ulimit -n / fileno counts.
+sub potentially_open_fds
+{
+ my $class = shift;
+ opendir my $fd_path, "/proc/$$/fd" or do {
+ warn "Cannot open /proc/$$/fd, falling back to generic method - $!";
+ return $class->SUPER::potentially_open_fds
+ };
+
+ # Skip ., .., our directory handle itself and any other cruft
+ # except fileno() isn't available for the handle so we'll
+ # end up with that in the output anyway. As long as we're
+ # called just before the relevant close() loop, this
+ # should be harmless enough.
+ my @fd = map /^([0-9]+)$/ ? $1 : (), readdir $fd_path;
+ closedir $fd_path;
+ return @fd;
+}
+
+=head1 AUTHOR
+
+Paul Evans <leonerd@leonerd.org.uk>
+
+=cut
+
+0x55AA;