Bug #18819 for Net-DNS: EBCDIC vs ISO coded character sets

Fri Apr 21 11:45:59 2006 Guest - Ticket created

Subject:

EBCDIC vs ISO coded character sets

Hello, querying MX records from an EBCDIC system against ISO/Unicode based DNS does not work as expected. The name of the questioned domain is transferred in EBCDIC coded characters. This can be checked e.g. with the ethereal program. I think, the DNS server interprets these EBCDIC coded characters through an ISO code page. Result we see is that DNS delivers no MX records because there are none for the questioned domain (NOERR result code). Changing the method presentation2wire of DNS.pm so that a character set conversion from EBCDIC coded characters to ISO is done prior to sending the question to the DNS delivers the expected behaviour. The DNS server answers with the expected MX records. However, a similar problem is seen viceversa, the answer from the DNS server needs to be retranslated from ISO coded characters into EBCDIC for further processing. I also changed the method wire2presentation accordingly. But am not quite sure, that it is called to derive entries for the MX records. I think there is a similar problem for querying other DNS resources like looking up host name addresses. Sincerely yours Michael Kläschen - Perl Version is 5.8.7, - Version of Net::DNS is 0.57 - Operating System Type and Version is z/OS V1R7 - Version of nameserver is unknown, probably some Microsoft Windows DNS - exact text of error message or description of problem : please see above - the shortest possible program that exhibits the problem : examples of the Net::DNS manual page - the specific queries you're making, if the data is available to Internet nameservers : no internet DNS

Fri Apr 21 12:14:34 2006 olaf [...] dacht.net - Correspondence added

Subject:	Re: [rt.cpan.org #18819] EBCDIC vs ISO coded character sets
Date:	Fri, 21 Apr 2006 18:13:27 +0200
To:	bug-Net-DNS [...] rt.cpan.org
From:	"Olaf M. Kolkman" <olaf [...] dacht.net>

On Apr 21, 2006, at 5:46 PM, Guest via RT wrote: Show quoted text

> > querying MX records from an EBCDIC system against ISO/Unicode based > DNS does not work as expected.

Net::DNS is not IDN aware. You will have to do nameprep yourself before sending the query (RFC3490 and RFC3491). In other words you will have to take your unicode string and convert it to xn--blafoo and then hand it to Net::DNS. See e.g. http://search.cpan.org/~cfaerber/Net-IDN-Encode-0.02/Encode.pm --Olaf ------------------------------------------------------ Ik dacht net... heel even maar.

Download PGP.sig
application/pgp-signature 227b

Message body not shown because it is not plain text.

Fri Apr 21 12:14:36 2006 The RT System itself - Status changed from 'new' to 'open'

Fri Apr 21 13:07:53 2006 Guest - Correspondence added

From:

michael.klaeschen [...] provinzial.de

On Fr. 21. Apr. 2006, 12:14:34, olaf@dacht.net wrote: Show quoted text

> > On Apr 21, 2006, at 5:46 PM, Guest via RT wrote:

> > > > querying MX records from an EBCDIC system against ISO/Unicode based > > DNS does not work as expected.

> > Net::DNS is not IDN aware. > > You will have to do nameprep yourself before sending the query > (RFC3490 and RFC3491). In other words you will have to take your > unicode string and convert it to xn--blafoo and then hand it to > Net::DNS. > > See e.g. http://search.cpan.org/~cfaerber/Net-IDN-Encode-

0.02/Encode.pm Show quoted text

> > > --Olaf > > ------------------------------------------------------ > Ik dacht net... heel even maar. > > >

Hello Olaf, thank you for the answer. I am not much familiar with these RFCs. But the questioned domain names are correct by means of just using "default" characters like a to z. The domain for which MX records are queried in this example is "provinzial.de". So, I do not think that IDN is a matter in this case. The Net::DNS perl module forms a UDP packet to question the DNS server. Within this packet, the domain name is coded as "provinzialde" as expected. My problem is, that the characters "provinzialde" are coded in EBCDIC because the perl program runs in an EBCDIC based system. The characters "provinzialde" are not translated into ISO character set (like ISO8859-1 which I think is a superset of the ASCII coded character set) before forming or sending the UDP packet. The DNS server obviously expects the questioned domain name coded in ISO or unicode character set (I am not sure which of it). Since the meaning of codes differs in EBCDIC and ISO (e.g. the charcter 'p' is coded as 0x97 in EBCDIC IBM-1047 but 0x97 in ISO 8859-1 shows "ø"), the DNS server cannot deliver the corect answer. I cannot translate the questioned domain name "provinzial.de" before calling the mx method because then, the dot-sign is also changed from EBCDIC to ISO and therefore not accessible for Net:DNS methods forming the UPD packet any more. Sincerely yours Michael

Fri Apr 21 13:28:18 2006 olaf [...] dacht.net - Correspondence added

Subject:	Re: [rt.cpan.org #18819] EBCDIC vs ISO coded character sets
Date:	Fri, 21 Apr 2006 19:26:50 +0200
To:	bug-Net-DNS [...] rt.cpan.org
From:	"Olaf M. Kolkman" <olaf [...] dacht.net>

Guest via RT wrote: Show quoted text

> > > The DNS server obviously expects the questioned domain name coded in > ISO or unicode character set (I am not sure which of it). Since the > meaning of codes differs in EBCDIC and ISO (e.g. the charcter 'p' is > coded as 0x97 in EBCDIC IBM-1047 but 0x97 in ISO 8859-1 shows "ø"), > the DNS server cannot deliver the corect answer. > > I cannot translate the questioned domain name "provinzial.de" before > calling the mx method because then, the dot-sign is also changed from > EBCDIC to ISO and therefore not accessible for Net:DNS methods forming > the UPD packet any more. >

Aha... I misread the problem. This is not trival and not directly part of my area of expertise. Besides, I do not have access to systems that speak EBCDIC. Could you send me the patches you made so I can start of with trying to understand what you did to tackle this? I am afraid that I may not be able to solve this in a reasonable time. --Olaf

Download signature.asc
application/pgp-signature 252b

Message body not shown because it is not plain text.

Sat Apr 22 01:48:16 2006 Guest - Correspondence added

From:

michael.klaeschen [...] provinzial.de

On Fr. 21. Apr. 2006, 13:28:18, olaf@dacht.net wrote: Show quoted text

> Guest via RT wrote:

> > > > > > The DNS server obviously expects the questioned domain name coded

in Show quoted text

> > ISO or unicode character set (I am not sure which of it). Since

the Show quoted text

> > meaning of codes differs in EBCDIC and ISO (e.g. the charcter 'p'

is Show quoted text

> > coded as 0x97 in EBCDIC IBM-1047 but 0x97 in ISO 8859-1

shows "ø"), Show quoted text

> > the DNS server cannot deliver the corect answer. > > > > I cannot translate the questioned domain name "provinzial.de"

before Show quoted text

> > calling the mx method because then, the dot-sign is also changed

from Show quoted text

> > EBCDIC to ISO and therefore not accessible for Net:DNS methods

forming Show quoted text

> > the UPD packet any more. > >

> > > Aha... I misread the problem. > > This is not trival and not directly part of my area of expertise. > Besides, I do not have access to systems that speak EBCDIC. Could you > send me the patches you made so I can start of with trying to

understand Show quoted text

> what you did to tackle this? > > I am afraid that I may not be able to solve this in a reasonable

time. Show quoted text

> > --Olaf > > >

Hello Olaf, the only perl program I changed is DNS.pm. The file was stored by the make install procedure in "$PERL5LIB/site_perl/5.8.7/os390-thread- multi/Net/DNS.pm" . I marked my changes by adding comments in the form "#MKL". They should only appear within methods presentation2wire and wire2presentation. The sample getmx.pl program may give an impression on what I try to do. This MX testcase in particular works. But I did not succeed e.g. querying IP-addresses. Thank you, best regards Michael

Message body is not shown because it is too large.

use Net::DNS; my $res = Net::DNS::Resolver->new( nameservers => [qw(172.17.144.144 172.17.170.175 172.19.231.103)], #nameservers => [qw(10.11.114.247)], recurse => 0, debug => 0, ); $name = "provinzial.de"; @mx = mx($res, $name); if (@mx) { foreach $rr (@mx) { print $rr->preference, " ", Net::DNS::wire2presentation($rr->exchange), "\n"; } } else { print "can't find MX records for $name: ", $res->errorstring, "\n"; } exit 0;

Fri May 25 08:34:06 2007 OLAF [...] cpan.org - Severity Wishlist added

Fri May 25 08:34:18 2007 OLAF [...] cpan.org - Status changed from 'open' to 'stalled'

Fri May 25 08:34:50 2007 OLAF [...] cpan.org - Status changed from 'stalled' to 'open'

Fri May 25 08:35:03 2007 OLAF [...] cpan.org - Taken

Fri May 25 09:00:54 2007 OLAF [...] cpan.org - Status changed from 'open' to 'stalled'

Sat Oct 27 11:52:14 2012 rwfranks [...] acm.org - Correspondence added

Subject:	Unicode support: was EBCDIC vs ISO coded character sets
From:	rwfranks [...] acm.org

I guess nobody cares about EBCDIC any more. Perl's internal character encoding and Unicode support has also changed. Unfortunately, Net::DNS also has to work on Perl 5.6.1 which uses the native character encoding (ASCII/EBCDIC) and does not know what to do with the spare bits. All these unpleasant details have now been locked away inside the Domain.pm and Text.pm packages and RR.pm and its derived types are implemented in terms of these primitive objects. Please read Perl's Unicode tutorial and other documentation. If you get messages warning about "wide characters", this indicates a lack of understanding on your part, and/or bugs in your program. It is very unlikely to be a bug in Net::DNS which rigidly segregates binary and character data.

Sat Oct 27 11:52:15 2012 The RT System itself - Status changed from 'stalled' to 'open'

Mon Oct 29 11:13:08 2012 NLNETLABS [...] cpan.org - Status changed from 'open' to 'resolved'