Skip Menu |

This queue is for tickets about the Net-DNS CPAN distribution.

Report information
The Basics
Id: 15515
Status: resolved
Priority: 0/
Queue: Net-DNS

People
Owner: Nobody in particular
Requestors: christopher [...] pobox.com
pzdeb [...] softservecom.com
Cc: RCAPUTO [...] cpan.org
AdminCc:

Bug Information
Severity: Wishlist
Broken in: 0.71
Fixed in: (no value)



Subject: bgsend ignores usevc and communicates via UDP only always
my $res = Net::DNS::Resolver->new('usevc'=>1,nameservers => [qw(1.2.3.4)]); $res->usevc(1); my $answer = $res->search('tcp.example.com'); # This works (TCP query arrives at my name server) my $bgsock = $res->bgsend("host7.example.com"); # This generates a UDP query at my name server, even though I wanted TCP... my $sel = IO::Select->new($bgsock); If fixing this - be warned that some perl versions ship with an INET.pm that has nonblocking TCP socket bugs (ie: they ignore the nonblocking request, and block anyhow). Let me know if you hit this problem and I'll send the fix.
Is there any urgency in fixing this. The fix is not trivial and given your warning about the version dependend behavior blocking I am hesitant to dive into this directly. A pointer to the blocking behavior for the different Inet.pm versions would be appreciated. --Olaf
[christopher@pobox.com - Wed Nov 9 19:08:26 2005]: Show quoted text
> Meanwhile - here's the INET.pm nonblocking correction. This is > definitely the Linux fix; I also use Mac and Win32 for programming, > but I can't remember if this fixes those platforms too or not (I guess > it does - 0x8004667E looks like a win32 thing to me). I've got a > sample test script somewhere which validates an email address by > simultaneously connecting to all the MX servers for it, which > highlights the bug (or verifies my fix) very effectively. If you need > this, let me know. >
As long as there is no INET.pm that does nonblocking correctly and I am not able to set a prerequisite version so that we can be sure that the code is cross-platform portable I think we have to live with bgsend not using TCP tranport. I am not very happy with this but its probably the best thing to do now. Stalling the ticket.
Subject: Exception: incomplete data
For some DNS queries Net-DNS returns error messages to the console: Exception: incomplete data at /usr/local/lib/perl5/site_perl/5.8.5/mach/Net/DNS/RR.pm line 513, <IN> line 58. Query to reproduce : PTR 73.88.194.66.in-addr.arpa. I use Net-DNS in background mode ("bgsend", "bgread" methods). Error message shows when I'm trying to read "answer" property. Code example: my $res = Net::DNS::Resolver->new; my $socket = $res->bgsend("PTR 73.88.194.66.in-addr.arpa."); if ($res->bgisready($socket)) { $nsq = $res->bgread($socket); if ($nsq) { foreach $rr ($nsq->answer) { .. } } } Data index received from Net::DNS::Packet::dn_expand() is wrong. Therefore resolver can't parse the answer. Please note, that on Net-DNS pre 0.62 version the same queries have been resolved with empty result, and as we could expect - no any exceptions to the console. But Unix "dig PTR 73.88.194.66.in-addr.arpa." command returns good result: ; <<>> DiG 9.3.2 <<>> PTR 73.88.194.66.in-addr.arpa. ;; global options: printcmd ;; Got answer: ;; ->>HEADER<<- opcode: QUERY, status: NOERROR, id: 52148 ;; flags: qr rd ra; QUERY: 1, ANSWER: 32, AUTHORITY: 0, ADDITIONAL: 0 ;; QUESTION SECTION: ;73.88.194.66.in-addr.arpa. IN PTR ;; ANSWER SECTION: 73.88.194.66.in-addr.arpa. 3305 IN PTR spasweats.com. 73.88.194.66.in-addr.arpa. 3305 IN PTR emtrend.net. ... Perl version - v5.8.5; OS - FreeBSD 6.1-RELEASE/amd64 Ports installed: p5-Digest-SHA1-2.11 p5-Net-DNS-0.63
From: dzhus [...] i.ua
Втр. Июл. 15 11:14:08 2008, zdebp писал: Show quoted text
> For some DNS queries Net-DNS returns error messages to the console: > Exception: incomplete data > at /usr/local/lib/perl5/site_perl/5.8.5/mach/Net/DNS/RR.pm > line 513, <IN> line 58. > > Query to reproduce : > PTR 73.88.194.66.in-addr.arpa.
The Net::DNS crashes when receives answer over UDP that has truncate bit set and has malformed answer section.
From: dzhus [...] i.ua
Втр. Июл. 22 08:28:13 2008, mdzhus писал: Show quoted text
> Втр. Июл. 15 11:14:08 2008, zdebp писал:
> > For some DNS queries Net-DNS returns error messages to the console: > > Exception: incomplete data > > at /usr/local/lib/perl5/site_perl/5.8.5/mach/Net/DNS/RR.pm > > line 513, <IN> line 58. > > > > Query to reproduce : > > PTR 73.88.194.66.in-addr.arpa.
> > The Net::DNS crashes when receives answer over UDP that has truncate bit > set and has malformed answer section.
The same malformed answer returns djbdns DNS-servers (dnscache, tinydns servers).
From: rwfranks [...] acm.org
Perl 5.010000 Linux 2.6.25.14-108.fc9.i686 i686 Net::DNS 0.63 Your code fragment (with minor changes) 1 use Net::DNS; 2 my $res = Net::DNS::Resolver->new(debug=>1); 3 my $socket = $res->bgsend("PTR 73.88.194.66.in-addr.arpa."); 4 if ($res->bgisready($socket)) { # test for socket ready; does NOT wait 5 $nsq = $res->bgread($socket); 6 if ($nsq) { 7 foreach $rr ($nsq->answer) { 8 $rr->print; 9 } 10 } 11 12 } else { 13 print "socket not ready\n"; 14 } produces ;; bgsend(192.168.1.1 : 53) socket not ready == The example code provides no evidence of a bug in Net::DNS. The program exception almost certainly resulted from attempting to read a socket before it is ready. Please read comment at line 4 and related documentation.
From: pzdeb [...] softservecom.com
Чтв. Авг. 28 13:32:53 2008, rwfranks@acm.org писал: Show quoted text
> 3 my $socket = $res->bgsend("PTR 73.88.194.66.in-addr.arpa."); > 4 if ($res->bgisready($socket)) { # test for socket
ready; does NOT wait Show quoted text
> 5 $nsq = $res->bgread($socket); > > produces > > ;; bgsend(192.168.1.1 : 53) > socket not ready > > == > > The example code provides no evidence of a bug in Net::DNS. > > The program exception almost certainly resulted from attempting to
read Show quoted text
> a socket before it is ready. Please read comment at line 4 and related > documentation. >
Provided example code was only the test. Real working code does wait before socket reading. Problem is in another: Exception appears when DNS-resolver receives DNS-response that can't be transmitted in one UDP datagram. Its Header section contains truncated bit = 1 and the number of expected RR records in the Answer section. For some reason such DNS-responses from our DNS-servers are corrupted: they completely miss the Answer section (while the Header section still contains the number of expected RR records). The last version of Net::DNS (version 0.63) performs an additional check when parsing RR records: if there is no expected data in the Answer section it dies (previous versions just returned the empty data). We tested our code with another (correct) DNS server - all is working correctly (there is no error and we get all expected records). To resolve the problem we made the patch for Net-DNS-0.63 package by our side. Main idea is to add TCP protocol usage functionality for bgsend() and bgread() functions when TC bit is enabled. TC bit we control by our end product and set use the TCP protocol by usevc flag (like for send()). Diffs of our patch will be attached to the post.
+++ Net/DNS/Resolver.pm Wed Jul 30 09:51:04 2008 @@ -729,8 +729,6 @@ The current implementation supports TSIG only on outgoing packets. No validation of server replies is performed. -bgsend does not honor the usevc flag and only uses UDP for transport. - =head1 COPYRIGHT Copyright (c) 1997-2002 Michael Fuhr. ============================================================================== +++ Net/DNS/Resolver/Base.pm @@ -921,6 +921,10 @@ my $packet = $self->make_query_packet(@_); my $packet_data = $packet->data; + if ($self->{'usevc'}) { + return $self->bgsend_tcp($packet, $packet_data); + } + my $srcaddr = $self->{'srcaddr'}; my $srcport = $self->{'srcport'}; @@ -1010,38 +1014,127 @@ } +sub bgsend_tcp { + my ($self, $packet, $packet_data) = @_; + my $lastanswer; + + my $srcport = $self->{'srcport'}; + my $srcaddr = $self->{'srcaddr'}; + my $dstport = $self->{'port'}; + + unless ( $self->nameservers()) { + $self->errorstring('no nameservers'); + print ";; ERROR: bgsend_tcp: no nameservers\n" if $self->{'debug'}; + return; + } + + $self->_reset_errorstring; + + foreach my $ns ($self->nameservers()) { + print ";; attempt to bgsend_tcp($ns:$dstport) (src port = $srcport)\n" + if $self->{'debug'}; + + my $sock; + my $sock_key = "$ns:$dstport"; + my ($host,$port); + if ($self->persistent_tcp && $self->{'sockets'}[AF_UNSPEC]{$sock_key}) { + $sock = $self->{'sockets'}[AF_UNSPEC]{$sock_key}; + print ";; using persistent socket\n" + if $self->{'debug'}; + unless ($sock->connected){ + print ";; persistent socket disconnected (trying to reconnect)" + if $self->{'debug'}; + undef($sock); + $sock = $self->_create_tcp_socket($ns); + next unless $sock; + $self->{'sockets'}[AF_UNSPEC]{$sock_key} = $sock; + } + } else { + $sock = $self->_create_tcp_socket($ns); + next unless $sock; + + $self->{'sockets'}[AF_UNSPEC]{$sock_key} = $sock if $self->persistent_tcp; + } + + my $lenmsg = pack('n', length($packet_data)); + print ';; sending ', length($packet_data), " bytes\n" + if $self->{'debug'}; + + # note that we send the length and packet data in a single call + # as this produces a single TCP packet rather than two. This + # is more efficient and also makes things much nicer for sniffers. + # (ethereal doesn't seem to reassemble DNS over TCP correctly) + + unless ($sock->send( $lenmsg . $packet_data)) { + $self->errorstring($!); + print ";; ERROR: send_tcp: data send failed: $!\n" + if $self->{'debug'}; + next; + } + + return $sock; + } + return; +} + sub bgread { my ($self, $sock) = @_; my $buf = ''; - my $peeraddr = $sock->recv($buf, $self->_packetsz); - - if ($peeraddr) { + my $peeraddr; + unless ($self->{'usevc'}) { + $peeraddr = $sock->recv($buf, $self->_packetsz); + } + + if ($self->{'usevc'}) { + $buf = read_tcp($sock, Net::DNS::INT16SZ(), $self->{'debug'}); + my ($len) = unpack('n', $buf); + print ";; can't determine length: $buf\n" unless $len; + my $sel = IO::Select->new($sock); + unless ($sel->can_read($self->{'tcp_timeout'})) { + $self->errorstring('timeout'); + print ";; TIMEOUT\n" if $self->{'debug'}; + next; + } + $buf = read_tcp($sock, $len, $self->{'debug'}); + $self->answerfrom($sock->peerhost); + + if ($buf) { + print ';; answer from ', $sock->peerhost, ':', + $sock->peerport, ' : ', length($buf), " bytes\n" + if $self->{'debug'}; + } + } elsif ($peeraddr) { print ';; answer from ', $sock->peerhost, ':', $sock->peerport, ' : ', length($buf), " bytes\n" if $self->{'debug'}; - - my ($ans, $err) = Net::DNS::Packet->new(\$buf, $self->{'debug'}); - - if (defined $ans) { - $self->errorstring($ans->header->rcode); - $ans->answerfrom($sock->peerhost); - } elsif (defined $err) { - $self->errorstring($err); - } - - return $ans; } else { $self->errorstring($!); return; } + + my ($ans, $err) = Net::DNS::Packet->new(\$buf, $self->{'debug'}); + + if (defined $ans) { + $self->errorstring($ans->header->rcode); + $ans->answerfrom($sock->peerhost); + } elsif (defined $err) { + $self->errorstring($err); + } + + return $ans; } sub bgisready { my $self = shift; my $sel = IO::Select->new(@_); - my @ready = $sel->can_read(0.0); + my @ready; + if ($self->{'usevc'}) { + @ready = $sel->can_read($self->{'tcp_timeout'}); + } else { + @ready = $sel->can_read(0.0); + } return @ready > 0; }
Subject: Re: [rt.cpan.org #37697] Exception: incomplete data
Date: Fri, 29 Aug 2008 14:45:17 +0200
To: bug-Net-DNS [...] rt.cpan.org
From: Olaf Kolkman <olaf [...] dacht.net>
Show quoted text
> > Provided example code was only the test. Real working code does wait > before socket reading. > > Problem is in another: > Exception appears when DNS-resolver receives DNS-response that can't > be > transmitted in one UDP datagram. Its Header section contains > truncated > bit = 1 and the number of expected RR records in the Answer section. > > For some reason such DNS-responses from our DNS-servers are > corrupted: they completely miss the Answer section (while the Header > section still contains the number of expected RR records).
When setting the TC bit the RRsets that do not fit in the answer or authority section need to be dropped so I think that the bug your software is encountering is that it does not reset the count in the header, otherwise the behavior might be OK. Are you willing to share the details of the misbehaving server? Show quoted text
> > The last version of Net::DNS (version 0.63) performs an additional > check when parsing RR records: if there is no expected data in the > Answer section it dies (previous versions just returned the empty > data). > We tested our code with another (correct) DNS server - all is working > correctly (there is no error and we get all expected records). > > To resolve the problem we made the patch for Net-DNS-0.63 package by > our side. Main idea is to add TCP protocol usage functionality for > bgsend() and bgread() functions when TC bit is enabled. TC bit we > control by our end product and set use the TCP protocol by usevc flag > (like for send()).
An approach that passes the laugh-test. Show quoted text
> > > Diffs of our patch will be attached to the post. > +++ Net/DNS/Resolver.pm Wed Jul 30 09:51:04 2008 > @@ -729,8 +729,6 @@
Thanks, I will review this patch later, and write test cases for it. Not having read one line of the code yet the first question is, will it work with IPv6 --Olaf
Download PGP.sig
application/pgp-signature 235b

Message body not shown because it is not plain text.

From: rwfranks [...] acm.org
Show quoted text
> Provided example code was only the test. Real working code does wait > before socket reading.
As I said, your test code fails to demonstrate a bug in Net-DNS. Show quoted text
> Problem is in another: > Exception appears when DNS-resolver receives DNS-response that can't be > transmitted in one UDP datagram. Its Header section contains truncated > bit = 1 and the number of expected RR records in the Answer section.
None of which is stated in your initial problem report. Problem covered by bug#15515. This is considered 'Unimportant' and stalled because of cross-platform portability issues. Show quoted text
> For some reason such DNS-responses from our DNS-servers are > corrupted: they completely miss the Answer section (while the Header > section still contains the number of expected RR records). > The last version of Net::DNS (version 0.63) performs an additional > check when parsing RR records: if there is no expected data in the > Answer section it dies (previous versions just returned the empty data).
Net::DNS raises an exception when it encounters a corrupted packet. This is a 0.63 feature, not a bug. Unhandled exceptions cause the program to terminate. Perl's use of 'die' is historical, and is not as final as the usual meaning might suggest. Show quoted text
> We tested our code with another (correct) DNS server - all is working > correctly (there is no error and we get all expected records).
:-) Show quoted text
> To resolve the problem we made the patch for Net-DNS-0.63 package by > our side. Main idea is to add TCP protocol usage functionality for > bgsend() and bgread() functions when TC bit is enabled. TC bit we > control by our end product and set use the TCP protocol by usevc flag > (like for send()). > > Diffs of our patch will be attached to the post.
Thanks. This will be reviewed in the context of RT#15515.
Fairly involved code change for a low priority bug. Leaving this for a future release, not for 0.64
Fairly involved code change for a low priority bug. Leaving this for a future release, not for 0.64
This would be awesome for asynchronous DNS clientry like POE::Component::Client::DNS, which has been using Net::DNS' bgsend() for years. And in fact has an open bug because it doesn't support TCP: https://rt.cpan.org/Ticket/Display.html?id=27644
Hi Christopher and Rocco, Just to let you know; I recognize the desirability for this and will work on this, but, because of the complexity and amount of work needed, not for the 0.67 release. -- Willem
From: g.sisson [...] gmail.com
On Wed Oct 19 15:57:03 2011, WILLEM wrote: Show quoted text
> Just to let you know; I recognize the desirability for this and will > work on this, but, because of the complexity and amount of work needed, > not for the 0.67 release.
I just got burned by this one. I appreciate it's not trivial to add TCP support to bgsend. However, it would be helpful if the feature was documented, and the code didn't _silently_ ignore the usevc option. I suggest the following for the 0.67 release: 1) update the documentation to reflect that bgsend does not support usevc. The documentation already says: "bgsend does not support persistent sockets." Suggest you add: "bgsend does not support the usevc option (TCP)." 2) add something like the following to sub bgsend: if ($self->{'usevc'}) { Carp::carp "usevc option unsupported for bgsend"; } Since this is a suggestion to fix the reporting and documentation (rather than the functionality itself), shall I open a new ticket?
Hi Geoff, I have applied the suggested documentation change. I am not yet sure what to do with the extra warning though. I will discuss it with the other developers. Regards, -- Willem
Note, I raised this issue was raised in another ticket already, but it wouldn't be obvious from the title since there were a few issues raised there. Might want to mark the old one as a duplicate so the stalled wish list can be decremented by one. :-) See #47050 John
On Tue 18 Dec 2012 20:40:32, JTK wrote: Show quoted text
> Might want to mark the old one as a duplicate so the stalled > wish list can be decremented by one. :-) See #47050
I noticed, but I keep that ticket (47050) for the persistent sockets for the bg* functions feature request. Adapted the subject to reflect that. Regards, Willem
From: rwfranks [...] acm.org
On Wed Jun 15 13:00:13 2011, RCAPUTO wrote: Show quoted text
> This would be awesome for asynchronous DNS clientry like > POE::Component::Client::DNS, which has been using Net::DNS' bgsend() for > years. And in fact has an open bug because it doesn't support TCP: > https://rt.cpan.org/Ticket/Display.html?id=27644
bgsend etc. will support TCP in Net::DNS 1.03
Resolved in upcoming 1.03