Skip Menu |

This queue is for tickets about the Net-DNS CPAN distribution.

Report information
The Basics
Id: 16630
Status: resolved
Priority: 0/
Queue: Net-DNS

People
Owner: OLAF [...] cpan.org
Requestors: 210726 [...] rt.noris.net
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 0.55
Fixed in: (no value)



Subject: Net::DNS::Resolver::Recurse issues lots of IMHO unnecessary DNS requests.
When trying to find out why our monitoring system occasionally gets timeouts when trying to resolve NS records using Net::DNS::Resolver::Recurse version 515 from Net::DNS 0.55 (that is, the resolving process sometimes seems to take about a minute), I noticed that Net::DNS::Resolver::Recurse issues a lot of IMHO unneccessary queries, e.g. it needs 18 queries for resolving (the A RRset for) www.noris.de where 6 should be sufficient, cmp. the debug output attached, which I created using the following script: | #!/usr/bin/perl -w | | use strict; | | use Net::DNS::Resolver::Recurse; | | my $res = Net::DNS::Resolver::Recurse->new or die; | $res->hints('198.41.0.4'); | $res->debug(1); | $res->query_dorecursion('www.noris.de'); The problem is as follows: In its answer to request #2, c.de.net tells the module that dns{1,2,3}.noris.net are authoritative for noris.net, but does not provide IP addresses for this servers in the additional section. So Net::DNS::Resolver::Recurse tries to find out these IP addresses itself, which it does in a very unefficient manner: * It first tries to look up an AAAA RRset for dns1.noris.net, even though our machine does not have IPv6 connectivity anyway. * When it learns from answer #5 that there is no AAAA record for dns1.noris.net, it starts to look up the A RRset for that server. Although it should already know from answer #4 that dns.noris.{de,net,ch} are authoritative for noris.net, it starts over at d.root-servers.net to find that out again (requests #6 and #7). * After finding out the (A)ddress for dns1.noris.net it does the same (first resolving the AAAA RRset, then the A RRset) for dns{2,3}.noris.net, issuing ten further DNS queries (#8 to #17). Why does it do that anyway? IMHO the knowledge of the address of dns1.noris.net should be sufficient to send query #18 and thus determine the requested answer. Please let me know whether you intend to optimize this in the near future yourself, or if you'd appreciate a patch for it, because if neither nor, I'd probably have to switch back to my own resolver library. Regards, fany
Download test-rr.debug
application/octet-stream 61.8k

Message body not shown because it is not plain text.

[FANY - Tue Dec 20 07:45:31 2005]: Show quoted text
> Please let me know whether you intend to optimize this in the > near future yourself, or if you'd appreciate a patch for it, > because if neither nor, I'd probably have to switch back to > my own resolver library. >
I'll have a crack at this soon. I am not planning on doing a major rewrite of this code since but I'll see how we can gain withouth needing to maintain a "cache". I hope to have something by end of year. If you have patches available allready I'd like to have a look, --Olaf
Hi Olaf, thanks for your quick reply! [OLAF - Tue Dec 20 08:01:08 2005]: Show quoted text
> If you have patches available allready I'd like to have a look,
Sorry, I haven't written one yet. Regards, fany
[FANY - Tue Dec 20 07:45:31 2005]: Show quoted text
> * It first tries to look up an AAAA RRset for dns1.noris.net, > even though our machine does not have IPv6 connectivity anyway.
Intermediary result: This patch below is trivial and will help you with the above. The code will not do AAAA queries if there is no V6 support. Where V6 support is defined as having Socket6 and IO::Socket::INET6 installed. If you have those two modules and you do not have V6 transport then you are toast. The only option you have then is to set $Net::DNS::Resolver::Base::has_inet6=0; --Olaf ========================================================= ========== --- lib/Net/DNS/Resolver/Recurse.pm (revision 545) +++ lib/Net/DNS/Resolver/Recurse.pm (working copy) @@ -165,16 +165,19 @@ foreach my $ns (keys %{ $known_authorities }) { if (!@{ $known_authorities->{$ns} }) { print ";; _dorecursion() Manual lookup for authority [$ns]\n" if $self->{'debug'}; - my $auth_packet = - $self->_dorecursion - ($self->make_query_packet($ns,"AAAA"), # packet - ".", # known_zone - $self->{'hints'}, # known_authorities - $depth+1); # depth + my $auth_packet; my @ans; - @ans = $auth_packet->answer if $auth_packet; + + if ($Net::DNS::Resolver::Base::has_inet6){ + $auth_packet = + $self->_dorecursion + ($self->make_query_packet($ns,"AAAA"), # packet + ".", # known_zone + $self->{'hints'}, # known_authorities + $depth+1); # depth + @ans = $auth_packet->answer if $auth_packet; + } - $auth_packet = $self->_dorecursion ($self->make_query_packet($ns,"A"), # packet
From: 210726 [...] rt.noris.net
[OLAF - Thu Dec 22 06:44:17 2005]: Show quoted text
> This patch below is trivial and will help you with the above. > The code will not do AAAA queries if there is no V6 support.
Thanks a lot so far! That is already half the battle for us and should significantly lower the probability of getting timeouts again. BTW, another fact I haven't mentioned yet: These timeouts usually occur when at least one of the relevant nameservers is not reachable, because in this case several retries are sent to that server in a row before falling back to another server at the same level, e.g.: ;; _dorecursion() depth=[3] known_zone=[.] ;; _dorecursion() cutting deck of (13) authorities... ;; _dorecursion() Trying nameserver [198.32.64.12] ;; send_udp(198.32.64.12:53) ;; send_udp(198.32.64.12:53) ;; send_udp(198.32.64.12:53) ;; send_udp(198.32.64.12:53) ;; _dorecursion() Trying nameserver [192.203.230.10] ;; send_udp(192.203.230.10:53) ;; answer from 192.203.230.10:53 : 501 bytes That, of course, can cost many seconds. I think it'd be better to send retries alternately to each available server. Regards, fany
From: "Olaf M. Kolkman" <olaf [...] dacht.net>
Subject: Re: [cpan #16630] Net::DNS::Resolver::Recurse issues lots of IMHO unnecessary DNS requests.
Date: Thu, 22 Dec 2005 14:35:12 +0100
To: bug-Net-DNS [...] rt.cpan.org
RT-Send-Cc:
On Dec 22, 2005, at 13:31 , via RT wrote: Show quoted text
> > > Thanks a lot so far! That is already half the battle for us and > should significantly lower the probability of getting timeouts > again. >
In fact.. it turns out that it can be done a little more elegant. But let me first continue with the other parts. Show quoted text
> BTW, another fact I haven't mentioned yet: These timeouts > usually occur when at least one of the relevant nameservers is > not reachable, because in this case several retries are sent to > that server in a row before falling back to another server at > the same level, e.g.:
As an aside... Don;t you guys have a caching forwarder available somewhere in your infrastructure that does the work for you? (the one from /etc/ resolv.conf ?) Net::DNS::Resolver will automagically fire queries to those servers. I like hacking the Recursive resolver in perl but you will never get the speed that you would get if you would let your local resolver figure stuff out. Besides you will have the benefit of caching, which sort of saves the Internet. --Olaf ------------------------------------------------------ Ik dacht net... heel even maar.
Download PGP.sig
application/pgp-signature 227b

Message body not shown because it is not plain text.

From: fany [...] noris.net
[olaf@dacht.net - Thu Dec 22 08:36:10 2005]: Show quoted text
> Don;t you guys have a caching forwarder available somewhere in your > infrastructure that does the work for you?
Of course we have. For our domain monitoring, however, I cannot use caching (at least not up to the normal TTLs), because its main purpose is to notice quickly if there's any problem regarding the delegation of one of the domains being monitored. Regards, fany
From: "Olaf M. Kolkman" <olaf [...] dacht.net>
Subject: Re: [cpan #16630] Net::DNS::Resolver::Recurse issues lots of IMHO unnecessary DNS requests.
Date: Thu, 22 Dec 2005 15:19:31 +0100
To: bug-Net-DNS [...] rt.cpan.org
RT-Send-Cc:
Show quoted text
> > Of course we have. For our domain monitoring, however, I cannot use > caching (at least not up to the normal TTLs), because its main purpose > is to notice quickly if there's any problem regarding the > delegation of > one of the domains being monitored. >
But in that case you have a list of authoritative nameservers that provide the delegation and you do not need the whole recursion do you? Anyway, I am working on it... I may have to build a somewhat more persistent and intelligent cache for the Recurse method itself. Still investigating. --Olaf ------------------------------------------------------ Ik dacht net... heel even maar.
Download PGP.sig
application/pgp-signature 227b

Message body not shown because it is not plain text.

From: fany [...] noris.net
[olaf@dacht.net - Thu Dec 22 09:20:17 2005]: Show quoted text
> > For our domain monitoring, however, I cannot use > > caching (at least not up to the normal TTLs), because its main > > purpose is to notice quickly if there's any problem regarding the > > delegation of one of the domains being monitored.
Show quoted text
> But in that case you have a list of authoritative nameservers that > provide the delegation and you do not need the whole recursion do you?
I'd like to be able to detect problems at any level of the delegation hierarchy, e.g. the nameservers for the TLD may fail or the NIC might let expire one of the domains. Regards, fany
A status update I have not found any time to work on this issue yet. It involves some code overhaul and I am somewhat conservative with that. --Olaf
From: rwfranks [...] acm.org
0.79 implements a glue cache which greatly reduces the total number of queries.