Bug #122521 for podlators: stray warning if output has no layers

Mon Jul 17 16:17:32 2017 zefram [...] fysh.org - Ticket created

Subject:	stray warning if output has no layers
Date:	Mon, 17 Jul 2017 21:17:22 +0100
To:	bug-podlators [...] rt.cpan.org
From:	Zefram <zefram [...] fysh.org>

With Pod::Man 4.09, and some combination of related modules that I haven't pinned down (possibly just a sufficiently old Pod::Perldoc), perldoc(1) is producing a warning on stderr: Use of uninitialized value $flag in bitwise and (&) at /home/zefram/usr/perl/perl_install/perl-5.16.0-i64-f52/lib/site_perl/5.16.0/Pod/Man.pm line 809. The code around there, in ->start_document, is trying to determine whether the output handle takes bytes or characters. It calls PerlIO::get_layers() with the detail option set, takes the last item from the returned list, and checks for a bit being set in it. It's expecting to get the flags for the topmost layer. But in the cases that produce the above warning, PerlIO::get_layers() is returning an empty list, so undef is obtained instead of the expected flag set. Everything else works; correct output is produced. By inserting test statements into the code in question, I've established that the file handle is an IO::File object, and is actually open and usable at that point: extra text printed to it goes into the formatted output. I'm not sure how a file handle gets to be usable while having no PerlIO layers. I suspect that the code should just accept the no-layers situation as meaning that the handle doesn't do UTF-8. The fix would be the insertion of "|| 0" on the end of the expression assigned to $flag. But because I don't understand the no-layers situation I'm not sure about this: it might be that there's a UTF-8 flag somewhere that this code fails to read. -zefram

Thu Aug 10 14:20:50 2017 eagle [...] eyrie.org - Correspondence added

Subject:	Re: [rt.cpan.org #122521] stray warning if output has no layers
Date:	Thu, 10 Aug 2017 11:14:29 -0700
To:	"Zefram via RT" <bug-podlators [...] rt.cpan.org>
From:	Russ Allbery <eagle [...] eyrie.org>

"Zefram via RT" <bug-podlators@rt.cpan.org> writes: Show quoted text

> With Pod::Man 4.09, and some combination of related modules that I haven't > pinned down (possibly just a sufficiently old Pod::Perldoc), perldoc(1) > is producing a warning on stderr:

Show quoted text

> Use of uninitialized value $flag in bitwise and (&) at /home/zefram/usr/perl/perl_install/perl-5.16.0-i64-f52/lib/site_perl/5.16.0/Pod/Man.pm line 809.

Show quoted text

> The code around there, in ->start_document, is trying to determine > whether the output handle takes bytes or characters. It calls > PerlIO::get_layers() with the detail option set, takes the last item from > the returned list, and checks for a bit being set in it. It's expecting > to get the flags for the topmost layer. But in the cases that produce > the above warning, PerlIO::get_layers() is returning an empty list, > so undef is obtained instead of the expected flag set.

Show quoted text

> Everything else works; correct output is produced. By inserting test > statements into the code in question, I've established that the file > handle is an IO::File object, and is actually open and usable at that > point: extra text printed to it goes into the formatted output. I'm not > sure how a file handle gets to be usable while having no PerlIO layers.

Show quoted text

> I suspect that the code should just accept the no-layers situation as > meaning that the handle doesn't do UTF-8. The fix would be the insertion > of "|| 0" on the end of the expression assigned to $flag. But because > I don't understand the no-layers situation I'm not sure about this: it > might be that there's a UTF-8 flag somewhere that this code fails to read.

Thank you for the bug report and the analysis! I'm going to go with your fix for the next release, since I don't see a good opportunity to do something better and the PerlIO layers stuff is remarkably opaque. At the least, this edge case seems to be very rare, and in your case meant there was not a UTF-8 translation layer in place (which is not the default anyway), so I think this maximizes the chances that we do the right thing without noisy warnings. -- Russ Allbery (eagle@eyrie.org) <http://www.eyrie.org/~eagle/>

Thu Aug 10 14:20:51 2017 The RT System itself - Status changed from 'new' to 'open'

Fri Aug 18 22:15:34 2017 zefram [...] fysh.org - Correspondence added

Subject:	Re: [rt.cpan.org #122521] stray warning if output has no layers
Date:	Sat, 19 Aug 2017 03:14:54 +0100
To:	Russ Allbery via RT <bug-podlators [...] rt.cpan.org>
From:	Zefram <zefram [...] fysh.org>

I've figured out what's going on with the no-layers result. Perl's IO objects can be referenced in multiple ways: it's common to have a reference (RV) directly to the IO-type object, but it's also common to have the IO object referenced by a glob, and to work with a reference to the glob or with the glob itself as a scalar. For historical reasons, even when dealing with an anonymous IO object it's common to have it wrapped up in a glob: for example, "open($fh, ...)" sets $fh to a reference to a glob containing the IO object. Most things that want IO objects as inputs can accept them in any of the three forms. For example, stat() accepts any of them. But it wasn't always so: prior to Perl 5.9.4 stat() wouldn't accept a direct reference to an IO object, but would only accept it wrapped up in a glob. PerlIO::get_layers() had such a requirement too, and the weird thing is that it *still* requires IO objects to be wrapped up in globs. What's going on in Pod::Man is that there's an IO object being handled in the form of a direct reference to the IO object. That reference gets passed to PerlIO::get_layers(), which doesn't know what to do with it and so returns an empty list. It's failing to look at the list of layers. In the tests I've run, the IO object actually has one layer, a "scalar" layer, with a well-defined set of flags, with the UTF-8 flag clear. The fix is to wrap the IO object in a glob by using the *{} operator. That has special-case logic to coerce an IO object into a glob. So the line in question becomes my $flag = (PerlIO::get_layers (*{$$self{output_fh}}, @options))[-1]; -zefram

Sun Sep 17 14:47:54 2017 RRA [...] cpan.org - Correspondence added

Subject:	Re: [rt.cpan.org #122521] stray warning if output has no layers
Date:	Sun, 17 Sep 2017 11:39:42 -0700
To:	"Zefram via RT" <bug-podlators [...] rt.cpan.org>
From:	Russ Allbery <rra [...] cpan.org>

"Zefram via RT" <bug-podlators@rt.cpan.org> writes: Show quoted text

> Most things that want IO objects as inputs can accept them in any of the > three forms. For example, stat() accepts any of them. But it wasn't > always so: prior to Perl 5.9.4 stat() wouldn't accept a direct reference > to an IO object, but would only accept it wrapped up in a glob. > PerlIO::get_layers() had such a requirement too, and the weird thing is > that it *still* requires IO objects to be wrapped up in globs.

Show quoted text

> What's going on in Pod::Man is that there's an IO object being handled > in the form of a direct reference to the IO object. That reference gets > passed to PerlIO::get_layers(), which doesn't know what to do with it > and so returns an empty list. It's failing to look at the list of > layers. In the tests I've run, the IO object actually has one layer, a > "scalar" layer, with a well-defined set of flags, with the UTF-8 flag > clear.

Show quoted text

> The fix is to wrap the IO object in a glob by using the *{} operator. > That has special-case logic to coerce an IO object into a glob. So the > line in question becomes

Show quoted text

> my $flag = (PerlIO::get_layers (*{$$self{output_fh}}, @options))[-1];

Oh, wow, thank you for the analysis! That's great digging, and it's so much better to know exactly what was going on. I've switched to this fix for the next release, which should be forthcoming shortly. -- #!/usr/bin/perl -- Russ Allbery, Just Another Perl Hacker $^=q;@!>~|{>krw>yn{u<$$<[~||<Juukn{=,<S~|}<Jwx}qn{<Yn{u<Qjltn{ > 0gFzD gD, 00Fz, 0,,( 0hF 0g)F/=, 0> "L$/GEIFewe{,$/ 0C$~> "@=,m,|,(e 0.), 01,pnn,y{ rw} >;,$0=q,$,,($_=$^)=~y,$/ C-~><@=\n\r,-~$:-u/ #y,d,s,(\$.),$1,gee,print

Mon May 07 20:39:59 2018 RRA [...] cpan.org - Broken in 2.4.1 added

Mon May 07 20:39:59 2018 RRA [...] cpan.org - Fixed in 4.10 added

Mon May 07 20:39:59 2018 RRA [...] cpan.org - Correspondence added

Belatedly closing as fixed in 4.10.

Mon May 07 20:40:00 2018 RRA [...] cpan.org - Status changed from 'open' to 'resolved'

Mon May 07 20:40:01 2018 RRA [...] cpan.org - Taken