Skip Menu |

This queue is for tickets about the Encode CPAN distribution.

Report information
The Basics
Id: 7831
Status: resolved
Priority: 0/
Queue: Encode

People
Owner: DANKOGAI [...] cpan.org
Requestors: bjoern [...] hoehrmann.de
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: 1.98
Fixed in: (no value)



Subject: piconv with ascii-incompatible output breaks on Win32
Using `piconv -f utf-8 -t UTF-16LE in > out` does not work on Win32 as Win32 stdio will translate line endings without considering the encoding of the stream, you get CR LF rather than e.g. LF 00 or CR 00 LF 00 which results in broken output. This can generally be solved by using binmode(STDOUT) except that this would create LF only line endings rather than proper CR LF pairs, e.g. perl -MEncode -e "local$/;$_=<>;binmode(STDOUT);Encode::from_to($_,'utf-8','UTF-16LE');print" in > out would work except for the shortcoming mentioned above.
Thank you for your report. [guest - Thu Sep 30 13:03:10 2004]: Show quoted text
> Using `piconv -f utf-8 -t UTF-16LE in > out` does not work on Win32 as > Win32 stdio will translate line endings without considering the > encoding of the stream [snip]
I reckon it is easy to fix but the problem is I do not have Win32 environment handy for the time being. Please give me some time. Or a patch is welcome. Dan the Maintainer Thereof
Show quoted text
> perl -MEncode -e > "local$/;$_=<>;binmode(STDOUT);Encode::from_to($_,'utf-8','UTF- > 16LE');print" in > out
This should be resolved by applying binmode to the input filehandle. The patch below should fix that. Would you try and tell me what happens? Dan the Maintainer Thereof --- bin/piconv 2004/05/16 20:55:16 2.0 +++ bin/piconv 2004/09/30 18:40:17 @@ -1,5 +1,5 @@ #!./perl -# $Id: piconv,v 2.0 2004/05/16 20:55:16 dankogai Exp dankogai $ +# $Id: piconv,v 2.0 2004/05/16 20:55:16 dankogai Exp $ # use 5.8.0; use strict; @@ -52,25 +52,39 @@ EOT } -# default -if ($scheme eq 'from_to'){ - while(<>){ - Encode::from_to($_, $from, $to, $Opt{check}); print; - }; -# step-by-step -}elsif ($scheme eq 'decode_encode'){ - while(<>){ - my $decoded = decode($from, $_, $Opt{check}); - my $encoded = encode($to, $decoded); - print $encoded; - }; -# NI-S favorite -}elsif ($scheme eq 'perlio'){ - binmode(STDIN, ":encoding($from)"); - binmode(STDOUT, ":encoding($to)"); - while(<>){ print; } -} else { # won't reach - die "$name: unknown scheme: $scheme"; +# we do not use <> (or ARGV) for the sake of binmode() +@ARGV or push @ARGV, \*STDIN; + +unless ($scheme eq 'perlio'){ + binmode STDOUT; + for my $argv (@ARGV){ + my $ifh = ref $argv ? $argv : undef; + $ifh or open $ifh, "<", $argv or next; + binmode $ifh; + if ($scheme eq 'from_to'){ # default + while(<$ifh>){ + Encode::from_to($_, $from, $to, $Opt{check}); + print; + } + }elsif ($scheme eq 'decode_encode'){ # step-by-step + while(<$ifh>){ + my $decoded = decode($from, $_, $Opt{check}); + my $encoded = encode($to, $decoded); + print $encoded; + } + } else { # won't reach + die "$name: unknown scheme: $scheme"; + } + } +}else{ + # NI-S favorite + binmode STDOUT => "raw:encoding($to)"; + for my $argv (@ARGV){ + my $ifh = ref $argv ? $argv : undef; + $ifh or open $ifh, "<", $argv or next; + binmode $ifh => "raw:encoding($from)"; + print while(<$ifh>); + } } sub list_encodings{