Am So 19. Mär 2017, 14:31:42, h.m.brand@xs4all.nl schrieb:
Show quoted text> On Sun, 19 Mar 2017 09:55:26 -0400, "Felix Antonius Wilhelm Ostmann via
> RT" <bug-Text-CSV_XS@rt.cpan.org> wrote:
>
> > here is a small script, which produce a strange behavior using
> > length (). The problem only kicks in when using:
> > * bind_columns
> > * a empty field
> > * a unicode character in the previous row for the empty field
>
> Your problem most likely lies in the strongly discouraged use of ":utf8"
>
> See:
> --8<---
> use strict;
> use warnings;
>
> use File::Temp qw(tempfile);
> use Text::CSV_XS;
>
> my $temp_fh = IO::File->new_tmpfile;
> $temp_fh->print (<<"CSV");
> field1,field2
> pröblem,ignore
> ,ignore
> CSV
> $temp_fh->seek (0, 0);
> $temp_fh->binmode (":utf8");
>
> my $order_csv = Text::CSV_XS->new ({ binary => 1, auto_diag => 1 });
>
> my $row;
> { my $row_header = $order_csv->getline ($temp_fh);
> $order_csv->bind_columns (\@{$row}{@$row_header});
> }
> while ($order_csv->getline ($temp_fh)) {
> printf "STRING: >%s< ; LENGTH: %d ; HOTFIX LENGTH: %d\n",
> $row->{field1},
> length $row->{field1},
> length "".$row->{field1};
> }
> -->8---
>
> =>
>
> utf8 "\xF6" does not map to Unicode at xx.pl line 25, <_GEN_0> line 2.
> Wide character in printf at xx.pl line 29, <_GEN_0> line 2.
> STRING: >pr�blem< ; LENGTH: 4 ; HOTFIX LENGTH: 4
> STRING: >< ; LENGTH: 4 ; HOTFIX LENGTH: 0
>
> but with the correct use of encoding:
>
> $temp_fh->binmode (":encoding(utf-8)");
>
> =>
>
> utf8 "\xF6" does not map to Unicode at xx.pl line 22.
> STRING: >pr\xF6blem< ; LENGTH: 10 ; HOTFIX LENGTH: 10
> STRING: >< ; LENGTH: 0 ; HOTFIX LENGTH: 0
>
> I'd suggest you stop using ":utf8" per direct and start using the safe
> way with ":encoding(utf-8)".
>
> Anyway, this doesn't look like a CSV_XS problem
>
>
Sorry, my script was ofcourse misleading without the charset information for the script.
----
I can use 'binary => 1' from Text::CSV_XS or 'binmode(":encoding(utf-8)")' from IO::File, both result in the error:
If this is not a bug for Text::CSV_XS, please guide me to the correct point.
----
#!/usr/bin/env perl
use Text::CSV_XS;
my $temp_fh = IO::File->new_tmpfile;
$temp_fh->print(<<CSV);
field1
pr\x{C3}\x{96}blem
CSV
$temp_fh->seek(0, 0);
my $csv = Text::CSV_XS->new({binary => 1});
my $row;
{
my $row_header = $csv->getline($temp_fh);
$csv->bind_columns(\@{$row}{@$row_header});
}
while ($csv->getline($temp_fh)) {
printf(
"STRING: >%s< ; LENGTH: %d ; HOTFIX LENGTH: %d\n",
$row->{field1},
length($row->{field1}),
length("".$row->{field1}),
);
}
----
#!/usr/bin/env perl
use Text::CSV_XS;
my $temp_fh = IO::File->new_tmpfile;
$temp_fh->print(<<CSV);
field1
pr\x{C3}\x{96}blem
CSV
$temp_fh->seek(0, 0);
$temp_fh->binmode('encoding(utf8)');
my $csv = Text::CSV_XS->new();
my $row;
{
my $row_header = $csv->getline($temp_fh);
$csv->bind_columns(\@{$row}{@$row_header});
}
while ($csv->getline($temp_fh)) {
printf(
"STRING: >%s< ; LENGTH: %d ; HOTFIX LENGTH: %d\n",
$row->{field1},
length($row->{field1}),
length("".$row->{field1}),
);
}
