Bug #123681 for DBIx-Class-Schema-Loader: Tests started to fail

Sun Nov 19 15:19:06 2017 SREZIC [...] cpan.org - Ticket created

CC:	HERMES [...] cpan.org
Subject:	Tests started to fail

On my smokers (various OS & perls) the test suite started to fail: ... t/10_01sqlite_common.t .................. All 313 subtests passed (less 2 skipped subtests: 311 okay) (1 TODO test unexpectedly succeeded) ... DBIx::Class::ResultSource::schema(): Unable to perform storage-dependent operations with a detached result source (source 'Bar' is not associated with a schema). at t/20invocations.t line 18 # Looks like your test exited with 255 just after 7. t/20invocations.t ....................... Failed 55/62 subtests ... Arguments for _merge_hashes must be hash references at /usr/home/cpansand/.cpan/build/2017111918/DBIx-Class-Schema-Loader-0.07047-1/blib/lib/DBIx/Class/Schema/Loader.pm line 238. Arguments for _merge_hashes must be hash references at /usr/home/cpansand/.cpan/build/2017111918/DBIx-Class-Schema-Loader-0.07047-1/blib/lib/DBIx/Class/Schema/Loader.pm line 238. t/22dump.t .............................. All 11 subtests passed t/23dumpmore.t .......................... All 336 subtests passed ... (etc) ... Some test scripts exit with 138, which is SIGBUS on freebsd. Statistical analysis is not quite clear about the reasons, but it could be caused by newer Hash-Merge versions. One possible reason could be the change of the cloning mechanism in Hash-Merge. Regards, Slaven

Mon Nov 20 12:24:52 2017 ilmari+cpan [...] ilmari.org - Correspondence added

Hi Slaven, Thanks for the report and investigation. I had been seeing some test failures on Travis-CI, but hadn't had time to investigate it further. The culprit was indeed Hash::Merge, and it's been fixed in the latest release: https://metacpan.org/release/HERMES/Hash-Merge-0.299 Resolving this ticket without further action as the offending version was only on CPAN for 5 days. Regards, Ilmari

Mon Nov 20 12:24:54 2017 The RT System itself - Status changed from 'new' to 'open'

Mon Nov 20 12:24:54 2017 ilmari+cpan [...] ilmari.org - Status changed from 'open' to 'resolved'

Tue Nov 21 17:14:53 2017 SREZIC [...] cpan.org - Correspondence added

RT-Send-CC:

HERMES [...] cpan.org

On 2017-11-20 12:24:52, ilmari wrote: Show quoted text

> Hi Slaven, > > Thanks for the report and investigation. I had been seeing some test > failures on Travis-CI, but hadn't had time to investigate it further. > The culprit was indeed Hash::Merge, and it's been fixed in the latest > release: > > https://metacpan.org/release/HERMES/Hash-Merge-0.299 > > Resolving this ticket without further action as the offending version > was only on CPAN for 5 days.

I have to reopen, as I still see test failures, mostly segmentation faults, even with Hash-Merge-0.299. And it still seems to be caused by Hash-Merge: for example, with perl 5.26.1 and Hash::Merge 0.200 the test suite passes, but after upgrading to 0.299 it started to fail. A sample excerpt of the test log (wstat 11 = sigsegv): ... Test Summary Report ------------------- t/10_01sqlite_common.t (Wstat: 11 Tests: 313 Failed: 0) TODO passed: 313 Non-zero wait status: 11 t/20invocations.t (Wstat: 65280 Tests: 7 Failed: 0) Non-zero exit status: 255 Parse errors: Bad plan. You planned 62 tests but ran 7. t/22dump.t (Wstat: 11 Tests: 11 Failed: 0) Non-zero wait status: 11 t/23dumpmore.t (Wstat: 11 Tests: 336 Failed: 0) Non-zero wait status: 11 t/24loader_subclass.t (Wstat: 11 Tests: 9 Failed: 0) Non-zero wait status: 11 t/27filter_generated.t (Wstat: 11 Tests: 19 Failed: 0) Non-zero wait status: 11 t/30_01comments.t (Wstat: 11 Tests: 4 Failed: 0) Non-zero wait status: 11 t/40overwrite_modifications.t (Wstat: 11 Tests: 10 Failed: 0) Non-zero wait status: 11 t/50rt59849.t (Wstat: 11 Tests: 35 Failed: 0) Non-zero wait status: 11 t/70schema_base_dispatched.t (Wstat: 11 Tests: 10 Failed: 0) Non-zero wait status: 11 Files=41, Tests=1314, 27 wallclock secs ( 0.20 usr 0.01 sys + 23.85 cusr 1.75 csys = 25.81 CPU) ... Regards, Slaven

Tue Nov 21 17:14:55 2017 SREZIC [...] cpan.org - Status changed from 'resolved' to 'open'

Wed Nov 22 19:21:18 2017 ESAYM [...] cpan.org - Correspondence added

Same issues here as well. Attached build log.

Subject:

build.log

Download build.log
application/octet-stream 16.9k

Message body not shown because it is not plain text.

Thu Nov 23 02:22:39 2017 REHSACK [...] cpan.org - Correspondence added

Maybe having a look on fixed bugs in Hash::Merge helps digging to the root cause. RT#123715 reports an issue which is caused by bug-using (RT#69872). Bug-using is not supported. Players who earn extra coins by bug-using ... you know :)

Thu Nov 23 05:26:18 2017 ilmari [...] ilmari.org - Correspondence added

Subject:	Re: [rt.cpan.org #123681] Tests started to fail
Date:	Thu, 23 Nov 2017 10:25:55 +0000
To:	bug-DBIx-Class-Schema-Loader [...] rt.cpan.org
From:	ilmari [...] ilmari.org (Dagfinn Ilmari Mannsåker)

"Jens Rehsack via RT" <bug-DBIx-Class-Schema-Loader@rt.cpan.org> writes: Show quoted text

> Queue: DBIx-Class-Schema-Loader > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=123681 > > > Maybe having a look on fixed bugs in Hash::Merge helps digging to the > root cause. RT#123715 reports an issue which is caused by bug-using > (RT#69872). Bug-using is not supported. Players who earn extra coins > by bug-using ... you know :)

Closer investigation seems to indicate that the cloning (which only used to happen for plain ARRAY and HASH refs) corrupts the DBI handle somehow, thus causing the segfaults when they get DESTROY called. Calling ->set_clone_behaviour(0) seems to fix it, but I need to consinder whether that's actually what we want for the whole hash, or if Hash::Merge needs some more fine-grained cloning control (e.g. only clone unblessed hashes). - ilmari -- "A disappointingly low fraction of the human race is, at any given time, on fire." - Stig Sandbeck Mathisen

Thu Nov 23 08:27:15 2017 REHSACK [...] cpan.org - Correspondence added

On Thu Nov 23 05:26:18 2017, ilmari@ilmari.org wrote: Show quoted text

> "Jens Rehsack via RT" <bug-DBIx-Class-Schema-Loader@rt.cpan.org> writes: >

> > Queue: DBIx-Class-Schema-Loader > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=123681 > > > > > Maybe having a look on fixed bugs in Hash::Merge helps digging to the > > root cause. RT#123715 reports an issue which is caused by bug-using > > (RT#69872). Bug-using is not supported. Players who earn extra coins > > by bug-using ... you know :)

> > Closer investigation seems to indicate that the cloning (which only used > to happen for plain ARRAY and HASH refs) corrupts the DBI handle > somehow, thus causing the segfaults when they get DESTROY called.

So you relied on misusing the misbehavior cloning only 1 level in depth when Clone was used. The cloning is now consistent. When you need cloning, then clone - when you don't need it, instantiate yourself a Hash::Merge object set clone behavior to false and go ahead. Show quoted text

> Calling ->set_clone_behaviour(0) seems to fix it, but I need to > consinder whether that's actually what we want for the whole hash, or if > Hash::Merge needs some more fine-grained cloning control (e.g. only > clone unblessed hashes).

Impressive idea. Please provide an appropriate Clone::Context with context sensitive fine grained clone control list (maybe steal from log4j). Tell us when we can include the Clone::Context backend into Clone::Choose. Cheers, Jens

Fri Nov 24 16:53:15 2017 gregoa [...] cpan.org - Cc GREGOA added

Mon Dec 11 07:22:22 2017 DDUMONT [...] cpan.org - Cc DDUMONT added

Sat Jan 06 06:01:12 2018 SREZIC [...] cpan.org - Reference by ticket #123859 added

Sat Jan 06 06:01:42 2018 SREZIC [...] cpan.org - Severity Important added

Sat Jan 06 16:08:55 2018 SREZIC [...] cpan.org - Correspondence added

RT-Send-CC:

VTI [...] cpan.org, FREW [...] cpan.org

On 2017-11-23 08:27:15, REHSACK wrote: Show quoted text

> On Thu Nov 23 05:26:18 2017, ilmari@ilmari.org wrote:

> > "Jens Rehsack via RT" <bug-DBIx-Class-Schema-Loader@rt.cpan.org> writes: > >

> > > Queue: DBIx-Class-Schema-Loader > > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=123681 > > > > > > > Maybe having a look on fixed bugs in Hash::Merge helps digging to the > > > root cause. RT#123715 reports an issue which is caused by bug-using > > > (RT#69872). Bug-using is not supported. Players who earn extra coins > > > by bug-using ... you know :)

> > > > Closer investigation seems to indicate that the cloning (which only used > > to happen for plain ARRAY and HASH refs) corrupts the DBI handle > > somehow, thus causing the segfaults when they get DESTROY called.

> > So you relied on misusing the misbehavior cloning only 1 level in depth > when Clone was used. The cloning is now consistent. > When you need cloning, then clone - when you don't need it, instantiate > yourself a Hash::Merge object set clone behavior to false and go ahead. >

> > Calling ->set_clone_behaviour(0) seems to fix it, but I need to > > consinder whether that's actually what we want for the whole hash, or if > > Hash::Merge needs some more fine-grained cloning control (e.g. only > > clone unblessed hashes).

> > Impressive idea. Please provide an appropriate Clone::Context with context > sensitive fine grained clone control list (maybe steal from log4j). > Tell us when we can include the Clone::Context backend into Clone::Choose. >

This problem is also causing test failures (segmentation faults or bus errors) in other CPAN distributions: * VTI/MojoX-Session-0.33.tar.gz * FREW/DBIx-Class-DeploymentHandler-0.002222.tar.gz Probably DBIx::Class::Schema::Loader was already installed before the Hash::Merge upgrade, then Hash::Merge was upgraded and this is now causing the failures.

Sat Jan 20 08:16:22 2018 SREZIC [...] cpan.org - Correspondence added

On 2018-01-06 16:08:55, SREZIC wrote: Show quoted text

> On 2017-11-23 08:27:15, REHSACK wrote:

> > On Thu Nov 23 05:26:18 2017, ilmari@ilmari.org wrote:

> > > "Jens Rehsack via RT" <bug-DBIx-Class-Schema-Loader@rt.cpan.org> > > > writes: > > >

> > > > Queue: DBIx-Class-Schema-Loader > > > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=123681 > > > > > > > > > Maybe having a look on fixed bugs in Hash::Merge helps digging to > > > > the > > > > root cause. RT#123715 reports an issue which is caused by bug- > > > > using > > > > (RT#69872). Bug-using is not supported. Players who earn extra > > > > coins > > > > by bug-using ... you know :)

> > > > > > Closer investigation seems to indicate that the cloning (which only > > > used > > > to happen for plain ARRAY and HASH refs) corrupts the DBI handle > > > somehow, thus causing the segfaults when they get DESTROY called.

> > > > So you relied on misusing the misbehavior cloning only 1 level in > > depth > > when Clone was used. The cloning is now consistent. > > When you need cloning, then clone - when you don't need it, > > instantiate > > yourself a Hash::Merge object set clone behavior to false and go > > ahead. > >

> > > Calling ->set_clone_behaviour(0) seems to fix it, but I need to > > > consinder whether that's actually what we want for the whole hash, > > > or if > > > Hash::Merge needs some more fine-grained cloning control (e.g. only > > > clone unblessed hashes).

> > > > Impressive idea. Please provide an appropriate Clone::Context with > > context > > sensitive fine grained clone control list (maybe steal from log4j). > > Tell us when we can include the Clone::Context backend into > > Clone::Choose. > >

> > This problem is also causing test failures (segmentation faults or bus > errors) in other CPAN distributions: > * VTI/MojoX-Session-0.33.tar.gz > * FREW/DBIx-Class-DeploymentHandler-0.002222.tar.gz > > Probably DBIx::Class::Schema::Loader was already installed before the > Hash::Merge upgrade, then Hash::Merge was upgraded and this is now > causing the failures.

A possible workaround: use Hash::Merge::Simple instead of Hash::Merge. At least the test suite passes. Here's the patch: http://cpan.cpantesters.org/authors/id/S/SR/SREZIC/patches/DBIx-Class-Schema-Loader-0.07047-RT123681.patch CPAN.pm users may use the following distroprefs file for automatic patching: https://github.com/eserte/srezic-cpan-distroprefs/blob/master/DBIx-Class-Schema-Loader.yml

Sat Jan 20 08:35:16 2018 SREZIC [...] cpan.org - Correspondence added

On 2018-01-20 08:16:22, SREZIC wrote: Show quoted text

> On 2018-01-06 16:08:55, SREZIC wrote:

> > On 2017-11-23 08:27:15, REHSACK wrote:

> > > On Thu Nov 23 05:26:18 2017, ilmari@ilmari.org wrote:

> > > > "Jens Rehsack via RT" <bug-DBIx-Class-Schema-Loader@rt.cpan.org> > > > > writes: > > > >

> > > > > Queue: DBIx-Class-Schema-Loader > > > > > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=123681

> > > > > >

> > > > > > > > > > Maybe having a look on fixed bugs in Hash::Merge helps digging > > > > > to > > > > > the > > > > > root cause. RT#123715 reports an issue which is caused by bug- > > > > > using > > > > > (RT#69872). Bug-using is not supported. Players who earn extra > > > > > coins > > > > > by bug-using ... you know :)

> > > > > > > > Closer investigation seems to indicate that the cloning (which > > > > only > > > > used > > > > to happen for plain ARRAY and HASH refs) corrupts the DBI handle > > > > somehow, thus causing the segfaults when they get DESTROY called.

> > > > > > So you relied on misusing the misbehavior cloning only 1 level in > > > depth > > > when Clone was used. The cloning is now consistent. > > > When you need cloning, then clone - when you don't need it, > > > instantiate > > > yourself a Hash::Merge object set clone behavior to false and go > > > ahead. > > >

> > > > Calling ->set_clone_behaviour(0) seems to fix it, but I need to > > > > consinder whether that's actually what we want for the whole > > > > hash, > > > > or if > > > > Hash::Merge needs some more fine-grained cloning control (e.g. > > > > only > > > > clone unblessed hashes).

> > > > > > Impressive idea. Please provide an appropriate Clone::Context with > > > context > > > sensitive fine grained clone control list (maybe steal from log4j). > > > Tell us when we can include the Clone::Context backend into > > > Clone::Choose. > > >

> > > > This problem is also causing test failures (segmentation faults or > > bus > > errors) in other CPAN distributions: > > * VTI/MojoX-Session-0.33.tar.gz > > * FREW/DBIx-Class-DeploymentHandler-0.002222.tar.gz > > > > Probably DBIx::Class::Schema::Loader was already installed before the > > Hash::Merge upgrade, then Hash::Merge was upgraded and this is now > > causing the failures.

> > A possible workaround: use Hash::Merge::Simple instead of Hash::Merge. > At least the test suite passes. > > Here's the patch: > http://cpan.cpantesters.org/authors/id/S/SR/SREZIC/patches/DBIx-Class- > Schema-Loader-0.07047-RT123681.patch > > CPAN.pm users may use the following distroprefs file for automatic > patching: > https://github.com/eserte/srezic-cpan-distroprefs/blob/master/DBIx- > Class-Schema-Loader.yml

I overlooked that Hash::Merge and Hash::Merge::Simple have different precedence rules (left vs. right precedence). Here's a new patch where the merge arguments are also swapped: http://cpan.cpantesters.org/authors/id/S/SR/SREZIC/patches/DBIx-Class-Schema-Loader-0.07047-RT123681-02.patch Interestingly the test suite did not fail, so maybe here's a missing test case...

Sun Jan 21 18:21:28 2018 ilmari+cpan [...] ilmari.org - Correspondence added

The change in behaviour in Hash::Merge has been mitigated by https://github.com/dbsrgits/dbix-class-schema-loader/pull/16 and released to CPAN in version 0.07048. - ilmari

Sun Jan 21 18:21:29 2018 ilmari+cpan [...] ilmari.org - Status changed from 'open' to 'resolved'