Bug #105376 for DBIx-Class: Actions on txn commit ( was: Conflict with autodie < 2.26 )

On 06/19/2015 02:34 PM, Matthew Astley via RT wrote: Show quoted text
It wouldn't for the simple reason that DBIx::Class itself does not use autodie for anything. All you see is a "Public Service Announcement" of sorts (there are several of those) - DBIC keeps on trucking after it encounters (and recovers) from such a condition. Therefor a "conflicts" would be a lie, as everything continues working. Show quoted text
Can you expand on this? What was the exact failure mode and why did it end up costing you hours? The very goal of this PSA was to save people time... Cheers!

On Fri, Jun 19, 2015 at 08:42:08AM -0400, Peter Rabbitson via RT wrote: Show quoted text
Show quoted text
(This PSA text was from a cloned dbix-class on current/blead) Show quoted text
OK. It does seem that, in every case, the transaction was rolled back. The fallout was afterwards. Show quoted text
In short, an old DBIx::Class gave a strange and less helpful error message and then interacted badly with DESTROY of other objects. When I first met it, I was using DBIx::Class 0.08250 autodie 2.22 and my code, which did this... mca@cgpfoo:~/gitwk-cgp/cgpDataOut$ perl script/cgpout --db canpg init -d /foo (in cleanup) (in cleanup) Operation "ne": no method found, left argument in overloaded package autodie::exception, right argument has no overloaded magic at /software/perl-5.16.2/lib/site_perl/5.16.2/DBIx/Class/Storage/TxnScopeGuard.pm line 60. Not an ARRAY reference at /nfs/users/nfs_m/mca/gitwk-cgp/cgpDataOut/lib/Sanger/CGP/DataOut/TxnMinder.pm line 96. 13@cgpfoo:~/gitwk-cgp/cgpDataOut$ (the 13 is $? being non-zero, and I'm not sure where it came from) ...for the (execution-ordered) pseudocode use warnings FATAL=>'all'; # not relevant? use autodie ':all'; # a local convention try { # Try::Tiny my $g = $schema->txn_minder; # my wrapper object # in $g constructor $g->{_dbtxn} = $schema->txn_scope_guard; open my $fh, '<', '/does/not/exist'; # boom # in $g->DESTROY my ($self) = @_; # that $g again my $R = delete $self->{_rollbacks}; # ARRAY of CODE delete $self->{_dbtxn}; # database ROLLBACK, if necessary # I deduce that it went boom again here, in $_dbtxn->DESTROY if ($R && @$R) { # the error that stops the program? # then $g->DESTROY happens again later, because we made a new # reference to it and then dropped it } catch { # didn't happen }; # not reached Actually it looks like the entire try{} block is happening again, because when I put a print at the top it happens twice..?! Presumably this is due to some deep failure with DESTROYing things while propagating exceptions. Then, more weirdly, that @$R changes to the printed text (?!) when I'm printing at the top of the try. try { my $g = $schema->txn_minder; print "init = $self<\n"; ... and code similar to above, with more prints ... produces mca@cgpfoo:~/gitwk-cgp/cgpDataOut$ perl script/cgpout --db canpg init -d /foo init = Sanger::CGP::DataOut::Command::init=HASH(0x1e9f268)< DESTROY! deleted _rollbacks: r=ARRAY(0x22c4d30)=r deleted _dbtxn g=DBIx::Class::Storage::TxnScopeGuard=HASH(0x1fdd608)=g (in cleanup) (in cleanup) Operation "ne": no method found, left argument in overloaded package autodie::exception, right argument has no overloaded magic at /software/perl-5.16.2/lib/site_perl/5.16.2/DBIx/Class/Storage/TxnScopeGuard.pm line 60. init = Sanger::CGP::DataOut::Command::init=HASH(0x1e9f268)< Can't use string ("init = Sanger::CGP::DataOut::Com"...) as an ARRAY ref while "strict refs" in use at /nfs/users/nfs_m/mca/gitwk-cgp/cgpDataOut/lib/Sanger/CGP/DataOut/TxnMinder.pm line 101. 20160DESTROY! 13@cgpfoo:~/gitwk-cgp/cgpDataOut$ The only place 20160 appears is as 14 days' worth of minutes, which will be the value of a field in the row INSERTed & rolled back. I don't know why it was printed. "DESTROY!\n" is my $g going away. I haven't done anything fancy with STDOUT. We weren't planning on publishing this because it depends strongly on internal systems, but I can put it up somewhere if you want to look..? Above was with branch weirdness commitid 86e4455, with a set of dependencies I can trace from commitid 909e9053 in ./CPAN/ on top of a potentially variable background install of perl-5.16.2 plus site-wide libraries. The TxnMinder itself might interest you. Its purpose is to run a LIFO list of CODE blocks, after the database rollbacks are completed, for things like removing files which were being built off the uncommitted rows. However, I'm now suspecting that doing this much work in DESTROY is inadvisable. Show quoted text
It was obvious that autodie's exception class was disagreeing with TxnScopeGuard, but I had not yet seen the PSA and started digging. I concluded fairly quickly that there was a bug, so I set about writing a test: clone dbix-class build enough of it to `prove -bv t/storage/txn_scope_guard.t` start trying to provoke the problem, and see how the outcome shows up. Possibly this was premature, since I didn't know how the failure in my $g->DESTROY or $_dbtxn->DESTROY was managing to change $R to something true-but-not-ARRAYref in time for the @$R. At this point I saw the PSA... Why am I using an old Perl with old libraries installed? It is our production environment. You could say we chose stability and old bugs over instability, fixes and new bugs. Thanks for reading. Does this answer your question? -- Matthew -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.

On Wed, Mar 30, 2016 at 11:18:46AM -0400, Peter Rabbitson via RT wrote: Show quoted text
No worries. It does mean I have semi-garbage-collected most of the context, so I don't remember what was important for me. I think my original request, that others be warned of autodie<2.26, is addressed by your explanation of the PSA messages. And maybe also the fact that search engines can find this thread. Show quoted text
[ execution-event-annotated pseudocode was here ] Show quoted text
OK. I guess a 'cluck' might show that up, but I didn't think of that at the time. I had previously concluded that DESTROY blocks may happen more than once per object, even without the re-blessing mentioned in L<perlobj/Destructors>, so I try to remember to keep them idempotent. Show quoted text
Thanks for the tip. I made a note on my cgpDataOut project to try this, (internal : https://gitlab.internal.sanger.ac.uk/CancerIT/cgpDataOut/issues/1 ) but it may be a while before I'm back on the DBIC side of this. Show quoted text
(I'm not sure I understood your question.) I was building a command line tool and underlying API to supply temporary copies of large data files to a compute cluster, and track those files in the database via DBIC in order to clean them up later. What I wanted was a way to keep the disk and database in sync, for commit or rollback, and to report on the progress of this. Assuming that the Perl process doesn't suddenly disappear. It seemed that a D:C:S:TxnScopeGuard-like object was the answer, because then I can ask it to (e.g.) mkdir and it will correctly rmdir if the database transaction is rolled back. Therefore I create my object (to BEGIN transaction), add to it some CODE to undo each non-db piece of work as I do the work, and then tell it when to COMMIT. The last line of POD says | This "has a" DBIx::Class::Storage::TxnScopeGuard rather than | inheriting, to avoid needing to mess with the privates of the | superclass. Possibly I would have had less trouble if subclassing TxnScopeGuard was explicitly OK? In this case I preferred to avoid poking undocumented magic. Show quoted text
(But I haven't found a more convenient way. Scope::Guard-like behaviour really simplifies code and that removes places to hide bugs.) Show quoted text
If the description above isn't enough, I can probably just publish the project. It would probably be GPL if it could possibly be interesting to anyone outside the organisation; but I should do the licensing thing if I do. Show quoted text
Thanks. 8-) -- Matthew -- The Wellcome Trust Sanger Institute is operated by Genome Research Limited, a charity registered in England with number 1021457 and a company registered in England with number 2742969, whose registered office is 215 Euston Road, London, NW1 2BE.