Skip Menu |

This queue is for tickets about the perl-ldap CPAN distribution.

Report information
The Basics
Id: 21904
Status: resolved
Priority: 0/
Queue: perl-ldap

People
Owner: Nobody in particular
Requestors: jjn1056 [...] yahoo.com
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 0.33
Fixed in: (no value)



Subject: Trouble reading entrys from *.LDIF files with /n/r line endings
I'm not sure if this is within the specification or not but when I export my addressbook from Mozilla Thunderbird under WindowsXP the LDIF file it makes contains DOS style line terminators. Several other LDIF addressbook exporters do this as well, included the one from Yahoo Email. If I try to read that file using Net::LDAP::LDIF it sees everything as one big entry, however if I convert the file to Unix style line terminiators it goes back to working correctly. Sorry, I'm not smart enough to offer a patch, but I believe something is up in the _read_lines subroutine for Net::LDAP::LDIF This is an example of how I'm using the code (It's running under a recent Linux and Perl 5.8.7) my $ldif = Net::LDAP::LDIF->new( $fhandle ); my @contacts; while( not $ldif->eof ) { my $entry = $ldif->read_entry; push @contacts, { email => [($entry->get_value('mail'))], name => [($entry->get_value('cn'))], }; } The variable $fhandle is a IO::File object. In this case if the file has DOS style line terminators then @contacts has a length of 1 no matter how many entries in the LDIF file. I attached a test case for this. There are three files in the test case, two LDIF files that have 7 entries and are identical except one uses DOS style linefeeds. Also there is a .t test that runs against these two files. Right now the test fails when reading the DOS style feeds. Sorry I tried to write a test case that would integrate with your testing system but I'm not that great at making tests. So this is a stand alone test. Please let me know how I can improve this for you. I know I can work around this by writing the file out to disk, filtering it and then making a new IO::File handle (what I'm doing as a work around now) but that solution scales poorly. Thank you. John Napiorkowski jjn1056@yahoo.com
Subject: tests.tar.gz
Download tests.tar.gz
application/x-gzip 1k

Message body not shown because it is not plain text.

From: jjn1056 [...] yahoo.com
BTW, I found a better workaround. If you are on a modern Perl there is the excellent module "PerlIO::eol" which is a IO layer to normalize line terminators. You can do something like: use PerlIO::eol; my $fh = IO::File->new(); fh->open("< dos_lf.ldif") binmode $fh, ":raw:eol(LF)"; my $ldif = Net::LDAP::LDIF->new( $fh ); And everything just works. I think this would work with Mac style line terms as well. I created a new test against the preceeding attached files that shows this in action. It doesn't screw things up if the LF are already unix style but fixes up any trouble with DOS style. Not sure if this will help updating this module since PerlIO::eol only works on modern perls and I think you might want Net::LDAP::LDIF to work on older perls as well. But maybe could help others running into this problem. Thank you for making my job easier :) --John
#!perl use strict; use warnings; use Test::More tests => 12; use_ok('Net::LDAP::LDIF'); use_ok('IO::File'); use_ok('PerlIO::eol'); CHECK_UNIX: { my $fh = IO::File->new(); ok( $fh->open("< unix_lf.ldif"), "Openning the unix test file."); my $ldif = Net::LDAP::LDIF->new( $fh ); my @contacts; while( not $ldif->eof ) { my $entry = $ldif->read_entry; push @contacts, { email => [($entry->get_value('mail'))], name => [($entry->get_value('cn'))], }; } is( $#contacts, 7, "Checking to make sure we get the correct number of entries."); $fh->close; } CHECK_UNIX_WITH_CONVERT: { my $fh = IO::File->new(); ok( $fh->open("< unix_lf.ldif"), "Openning the unix test file."); binmode $fh, ":raw:eol(LF)"; my $ldif = Net::LDAP::LDIF->new( $fh ); my @contacts; while( not $ldif->eof ) { my $entry = $ldif->read_entry; push @contacts, { email => [($entry->get_value('mail'))], name => [($entry->get_value('cn'))], }; } is( $#contacts, 7, "Checking to make sure we get the correct number of entries."); $fh->close; } CHECK_DOS_WITH_CONVERT: { my $fh = IO::File->new(); ok( $fh->open("< dos_lf.ldif"), "Openning the unix test file with the eol(LF) converted."); binmode $fh, ":raw:eol(LF)"; my $ldif = Net::LDAP::LDIF->new( $fh ); my @contacts; while( not $ldif->eof ) { my $entry = $ldif->read_entry; push @contacts, { email => [($entry->get_value('mail'))], name => [($entry->get_value('cn'))], }; } is( $#contacts, 7, "Checking to make sure we get the correct number of entries"); } CHECK_DOS: { my $fh = IO::File->new(); ok( $fh->open("< dos_lf.ldif"), "Openning the dos test file, no preconversion attempted."); my $ldif = Net::LDAP::LDIF->new( $fh ); my @contacts; while( not $ldif->eof ) { my $entry = $ldif->read_entry; push @contacts, { email => [($entry->get_value('mail'))], name => [($entry->get_value('cn'))], }; } is( $#contacts, 7, "Checking to make sure we get the correct number of entries"); $fh->close; }
Subject: Re: [rt.cpan.org #21904] Trouble reading entrys from *.LDIF files with /n/r line endings
Date: Wed, 4 Oct 2006 22:56:43 -0500
To: bug-perl-ldap [...] rt.cpan.org
From: Graham Barr <gbarr [...] pobox.com>
Thanks for the workaround. I do not want to put this dependancy in the module, but it is good to know. The LDIF parser needs to be re-written to not be dependent on local line endings but until it is done, we now have a workaround to tell others, Thanks, Graham. On Oct 4, 2006, at 5:40 PM, jjn1056@yahoo.com via RT wrote: Show quoted text
> I found a better workaround. If you are on a modern Perl there is the > excellent module "PerlIO::eol" which is a IO layer to normalize line > terminators. > > You can do something like: > > use PerlIO::eol; > > my $fh = IO::File->new(); > fh->open("< dos_lf.ldif") > binmode $fh, ":raw:eol(LF)"; > > my $ldif = Net::LDAP::LDIF->new( $fh ); > > And everything just works. I think this would work with Mac style > line > terms as well. > > I created a new test against the preceeding attached files that shows > this in action. It doesn't screw things up if the LF are already unix > style but fixes up any trouble with DOS style. Not sure if this will > help updating this module since PerlIO::eol only works on modern perls > and I think you might want Net::LDAP::LDIF to work on older perls as > well. But maybe could help others running into this problem. > > Thank you for making my job easier :) > > --John > > > <dos_line_terms.t>