Skip Menu |

This queue is for tickets about the XSConfig CPAN distribution.

Report information
The Basics
Id: 110573
Status: open
Priority: 0/
Queue: XSConfig

People
Owner: Nobody in particular
Requestors: SREZIC [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Critical
Broken in: 6.02
Fixed in: (no value)



Subject: XSConfig.pm breaks forks.pm
It seems that XSConfig 6.02 breaks the test suite of forks.pm: t/forks01.t ............... Not a HASH reference at /tmpfs/.cpan-build/2015122321/forks-0.36-0ihVC3/blib/lib/forks.pm line 121. BEGIN failed--compilation aborted at /tmpfs/.cpan-build/2015122321/forks-0.36-0ihVC3/blib/lib/forks.pm line 122. Compilation failed in require at t/forks01.t line 14. BEGIN failed--compilation aborted at t/forks01.t line 14. Undefined subroutine &threads::_END called at (eval 2) line 1. END failed--call queue aborted at t/forks01.t line 14. t/forks01.t ............... Dubious, test returned 22 (wstat 5632, 0x1600) No subtests run (etc.)
On Wed Dec 23 18:01:02 2015, SREZIC wrote: Show quoted text
> It seems that XSConfig 6.02 breaks the test suite of forks.pm: > > t/forks01.t ............... Not a HASH reference at /tmpfs/.cpan- > build/2015122321/forks-0.36-0ihVC3/blib/lib/forks.pm line 121. > BEGIN failed--compilation aborted at /tmpfs/.cpan- > build/2015122321/forks-0.36-0ihVC3/blib/lib/forks.pm line 122. > Compilation failed in require at t/forks01.t line 14. > BEGIN failed--compilation aborted at t/forks01.t line 14. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at t/forks01.t line 14. > t/forks01.t ............... Dubious, test returned 22 (wstat 5632, > 0x1600) > No subtests run > (etc.)
https://metacpan.org/source/RYBSKEJ/forks-0.36/lib/forks.pm#L121 forks.pm looks inside the PP tied %Config hash into private API. The public API of PP Config says it is a read only hash. Going inside %Config is not documented. Currently the inside of XS Config tied hash is an integer which is the each() iterator, not a perl hash. Implementing XS Config using Perl API (IE Perl hashes) seems pointless to me. The memory of perl hashes can't be shared between different perl processes without require()+fork(), and perl hashes can't be stored as memory mapped files, but the "serialized" form must be stored on disk as perl5 lang or some other serialization format, no memory mapping is possible. The only I solution I see is for forks.pm to make a copy of %Config into a new temp non-tied hash, then untie() global %Config (detach the XS subs/Config package methods from the tied hash), then copy the keys and vals of the non-tied temp has to the now untied global %Config. Then continue execution as before. Some benchmarks of XS Config vs PP Config, at 6.03+unpublished work. I need to collect memory/malloc usage numbers too. I am afraid any solution to support the inside of a XS Config tied hash, to be another hash, the way PP Config does, will degrade performance/increase memory usage. XS Config C:\sources\p5-Config>timeit -f t.dat perl -MConfig -e"my %h; %h = %Config for 0. .1000" Version Number: Windows NT 6.1 (Build 7601) Exit Time: 1:06 am, Thursday, December 24 2015 Elapsed Time: 0:00:10.534 Process Time: 0:00:10.264 System Calls: 192072 Context Switches: 53818 Page Faults: 5341 Bytes Read: 314999 Bytes Written: 180122 Bytes Other: 59780 C:\sources\p5-Config> C:\sources\p5-Config>timeit -f t.dat perl -MConfig -e"my $v; $v = $Config{'ccfla gs'} for 0..1000000;print $v" -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_S ECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_I MPLICIT_CONTEXT -DPERL_IMPLICIT_SYS Version Number: Windows NT 6.1 (Build 7601) Exit Time: 1:07 am, Thursday, December 24 2015 Elapsed Time: 0:00:05.187 Process Time: 0:00:05.116 System Calls: 90774 Context Switches: 25435 Page Faults: 3064 Bytes Read: 101769 Bytes Written: 25476 Bytes Other: 27796 C:\sources\p5-Config> With PP Config C:\sources\p5-Config>timeit -f t.dat perl -MConfig -e"my %h; %h = %Config for 0. .1000" Version Number: Windows NT 6.1 (Build 7601) Exit Time: 1:08 am, Thursday, December 24 2015 Elapsed Time: 0:00:16.229 Process Time: 0:00:15.756 System Calls: 280979 Context Switches: 82893 Page Faults: 8801 Bytes Read: 190076 Bytes Written: 815460 Bytes Other: 80142 C:\sources\p5-Config> C:\sources\p5-Config>timeit -f t.dat perl -MConfig -e"my $v; $v = $Config{'ccfla gs'} for 0..1000000;print $v" -nologo -GF -W3 -O1 -MD -Zi -DNDEBUG -GL -DWIN32 -D_CONSOLE -DNO_STRICT -D_CRT_S ECURE_NO_DEPRECATE -D_CRT_NONSTDC_NO_DEPRECATE -DPERL_TEXTMODE_SCRIPTS -DPERL_I MPLICIT_CONTEXT -DPERL_IMPLICIT_SYS Version Number: Windows NT 6.1 (Build 7601) Exit Time: 1:08 am, Thursday, December 24 2015 Elapsed Time: 0:00:06.117 Process Time: 0:00:05.959 System Calls: 116674 Context Switches: 31107 Page Faults: 3419 Bytes Read: 151872 Bytes Written: 74744 Bytes Other: 32720
On 2015-12-23 18:01:02, SREZIC wrote: Show quoted text
> It seems that XSConfig 6.02 breaks the test suite of forks.pm: > > t/forks01.t ............... Not a HASH reference at /tmpfs/.cpan- > build/2015122321/forks-0.36-0ihVC3/blib/lib/forks.pm line 121. > BEGIN failed--compilation aborted at /tmpfs/.cpan- > build/2015122321/forks-0.36-0ihVC3/blib/lib/forks.pm line 122. > Compilation failed in require at t/forks01.t line 14. > BEGIN failed--compilation aborted at t/forks01.t line 14. > Undefined subroutine &threads::_END called at (eval 2) line 1. > END failed--call queue aborted at t/forks01.t line 14. > t/forks01.t ............... Dubious, test returned 22 (wstat 5632, > 0x1600) > No subtests run > (etc.)
Another possible "victim": Test-Harness-3.36. See http://www.cpantesters.org/cpan/report/7c2be408-af70-11e5-a368-2a153e64ce68 (t/lib/NoFork.pm also fiddles with %Config, using the same trick as forks.pm to get over readonlyness)
On 2015-12-24 01:22:11, BULKDD wrote: Show quoted text
> On Wed Dec 23 18:01:02 2015, SREZIC wrote:
> > It seems that XSConfig 6.02 breaks the test suite of forks.pm: > > > > t/forks01.t ............... Not a HASH reference at /tmpfs/.cpan- > > build/2015122321/forks-0.36-0ihVC3/blib/lib/forks.pm line 121. > > BEGIN failed--compilation aborted at /tmpfs/.cpan- > > build/2015122321/forks-0.36-0ihVC3/blib/lib/forks.pm line 122. > > Compilation failed in require at t/forks01.t line 14. > > BEGIN failed--compilation aborted at t/forks01.t line 14. > > Undefined subroutine &threads::_END called at (eval 2) line 1. > > END failed--call queue aborted at t/forks01.t line 14. > > t/forks01.t ............... Dubious, test returned 22 (wstat 5632, > > 0x1600) > > No subtests run > > (etc.)
> > https://metacpan.org/source/RYBSKEJ/forks-0.36/lib/forks.pm#L121 > > forks.pm looks inside the PP tied %Config hash into private API. The > public API of PP Config says it is a read only hash. Going inside > %Config is not documented. Currently the inside of XS Config tied hash > is an integer which is the each() iterator, not a perl hash. > Implementing XS Config using Perl API (IE Perl hashes) seems pointless > to me. The memory of perl hashes can't be shared between different > perl processes without require()+fork(), and perl hashes can't be > stored as memory mapped files, but the "serialized" form must be > stored on disk as perl5 lang or some other serialization format, no > memory mapping is possible. > > The only I solution I see is for forks.pm to make a copy of %Config > into a new temp non-tied hash, then untie() global %Config (detach the > XS subs/Config package methods from the tied hash), then copy the keys > and vals of the non-tied temp has to the now untied global %Config. > Then continue execution as before.
Given that there are more problems with XSConfig (the UNINST issue, other modules fiddling with %Config::Config) --- have you considered to simply install XSConfig.pm as XSConfig.pm? It could set $INC{"Config.pm"}, so that a subsequent "use Config" is be a no-op. Users interested in the performance benefits could either manually symlink/copy XSConfig.pm to Config.pm (if they know that there are not affected by possible problems), or better, write "use XSConfig" in the principal script/webserver/.... Or use something like PERL5OPT=-MXSConfig.
On Sun Jan 03 14:59:46 2016, SREZIC wrote: Show quoted text
> Given that there are more problems with XSConfig (the UNINST issue, > other modules fiddling with %Config::Config) --- have you considered > to simply install XSConfig.pm as XSConfig.pm? It could set > $INC{"Config.pm"}, so that a subsequent "use Config" is be a no-op. > Users interested in the performance benefits could either manually > symlink/copy XSConfig.pm to Config.pm (if they know that there are not > affected by possible problems), or better, write "use XSConfig" in the > principal script/webserver/.... Or use something like PERL5OPT=- > MXSConfig.
I thought about it, XSConfig is useless if it doesn't replace PP Config.pm 100% of the time. In Cperl, XS Config is the only Config.pm, so this bug ticket is indirectly filed against Cperl. Fixing a bug on CPAN in XS Config will fix the same bug in Cperl core. Changing Config.pm to XSConfig.pm on CPAN won't fix the same bug in Cperl. Rewriting your own code to say "use XSConfig;" is too much work. There is also the issue of, if PP Config was already loaded in the process through some way (perl debugger, env vars, whatever), then some module does "use XSConfig;", then what? clean out the Config:: package globs at runtime to avoid redefined errors? CPU+memory churn. Plus I'm not sure I can free the big long PP Config DB string with all the settings. I think there are 2 ways to solve the %Config manipulating modules problem. 1st solution a module makes a copy of %Config to another hash, then does untie(%Config), then copies the other hash back to %Config. %Config is now a regular RW hash (memory usage regardless). What class (XS/PP) used to implement %Config is irrelevant. 2nd solution, XSConfig offers a Config::toPP() or Config::downgradetoPP() sub (a better sub name anymore?). This is in forks.pm BEGIN { require Config; my $h = tied %Config::Config; $h->{useithreads} = 1; } and would become BEGIN { require Config; Config::toPP() if defined &Config::toPP; my $h = tied %Config::Config; $h->{useithreads} = 1; } XSConfig during its self testing loads PP Config into a process that previously loaded XS Config process, and %Config becomes the PP implementation at runtime so runtime downgrading is possible, it just needs a documented API to do it. https://metacpan.org/source/BULKDD/XSConfig-6.06/t/XSConfig.t#L31 So if forks.pm calls Config::toPP() before called tied, it will be the PP implementation. I think this is a reasonable compromise. Since XS Config is the *ONLY* Config in cperl, looking down the road, it is not impossible that P5P perl one day would have XS Config in core, so either inflating and untie %Config solution or Config::toPP() must be implemented today. Nobody can deny forks.pm is using undefined behavior in manipulating %Config so a fix that requires patching forks.pm is reasonable. 3rd solution, a Config::setkey sub as a "YES I really want to write to RO data" sub, but P5P/PP Config doesn't have such an API and wont until 5.26 at the minimum, which doesn't solve the problem of %Config manipulating code needing to be back compat to PP Config.pm from as old as 5.8, so solution 3 wont work. There doesnt seem to be more than a couple modules that manipulate %Config http://grep.cpan.me/?q=tied%28\%28|\s*%29%25Config The UINST bug is partially fixed, it can't damage without human intervention smoker perl installations the way it was before, it just fails to automatically install now if it detects an unexpected UNINST=1.
On Thu Dec 31 03:28:46 2015, SREZIC wrote: Show quoted text
> On 2015-12-23 18:01:02, SREZIC wrote:
> > It seems that XSConfig 6.02 breaks the test suite of forks.pm
It's rather the other way round: fork.pm and some other modules, such as Test-Harness break Config. Config values are designed to be readonly. I fixed it this way: https://github.com/perl11/cperl/commit/42cac300080de7d3a9df8eed040750ec9583de20 Show quoted text
> Another possible "victim": Test-Harness-3.36. See > http://www.cpantesters.org/cpan/report/7c2be408-af70-11e5-a368- > 2a153e64ce68 > (t/lib/NoFork.pm also fiddles with %Config, using the same trick as > forks.pm to get over readonlyness)
-- Reini Urban