Bug #37954 for Catalyst-Plugin-Session-Store-DBIC: Catalyst::Plugin::Session::Store::DBIC fails with forking dev-server

Sat Jul 26 12:11:18 2008 dluke [...] geeklair.net - Ticket created

Subject:	Catalyst::Plugin::Session::Store::DBIC fails with forking dev-server
Date:	Sat, 26 Jul 2008 12:10:59 -0400
To:	bug-Catalyst-Plugin-Session-Store-DBIC [...] rt.cpan.org
From:	"Daniel J. Luke" <dluke [...] geeklair.net>

When the development server is run with -fork, I see these errors: [error] Caught exception in engine "DBIx::Class::Row::insert(): DBI Exception: DBD::Pg::st execute failed: ERROR: duplicate key value violates unique constraint "sessions_pkey" [for Statement "INSERT INTO sessions (id) VALUES (?)" with ParamValues: 1='flash: 5cacf0ace5d0ba4f66c7c6364525ff40a5db643a'] Everything works fine without -fork -- Daniel J. Luke +========================================================+ | *---------------- dluke@geeklair.net ----------------* | | *-------------- http://www.geeklair.net -------------* | +========================================================+ | Opinions expressed are mine and do not necessarily | | reflect the opinions of my employer. | +========================================================+

Wed Sep 24 17:50:01 2008 danieltwc [...] cpan.org - Correspondence added

I've updated Catalyst::Plugin::Session::Store::DBIC to use transactions, which should prevent these errors: 0.07 Wed Sep 24 17:08:34 EDT 2008 - Code was silently truncating storage to MySQL, rendering the session unreadable. Patched to check DBIx::Class size from column_info (if available) - Wrap find_or_create calls in a transaction to (hopefully) avoid issues with duplicate flash rows Please test the new version and let me know if you continue to have problems. Thanks!

Wed Sep 24 17:50:06 2008 The RT System itself - Status changed from 'new' to 'open'

Mon Sep 29 10:31:25 2008 danieltwc [...] cpan.org - Correspondence added

Marking as resolved for now. Feel free to reopen if you continue to see this issue with 0.07.

Mon Sep 29 10:31:27 2008 danieltwc [...] cpan.org - Status changed from 'open' to 'resolved'

Tue Jul 07 05:50:16 2009 steve [...] purkis.ca - Severity Important added

Tue Jul 07 05:50:16 2009 steve [...] purkis.ca - Broken in 0.08 added

Tue Jul 07 06:04:00 2009 steve [...] purkis.ca - Correspondence added

I've just come across this using Catalyst::Plugin::Session::Store::DBIC v0.08. Had a quick look at 0.09 and it doesn't look like this is fixed there. I've included the the error report I've been seeing at the end of this - we've been getting this maybe 30 times a day, with ~200 users. The problem is that there's still a race condition in DBIC's find_or_create. The method *IS NOT* atomic. It does a find first, then a create if it found nothing. In between it failing to find something, another process may create it. I think the easiest way around this is to wrap the txn_do in an eval, and retry it $n times: # in Catalyst::Plugin::Session::Store::DBIC::Delegate sub _load_row { ... for (1..$n) { eval { $row = $self->model->result_source->schema->txn_do($load_sub); }; if (my $e = $@) { next if $e =~ /duplicate key value violates unique constraint "sessions_pkey"/; die $e; # pass on } } ... } Note I've not tested this. Will try out that patch locally and see if it cuts down the errors we're getting. ----------------------------------------------------------------------- Exception caught: Error: Couldn't render template "undef error - DBIx::Class::ResultSet::find_or_create(): DBI Exception: DBD::Pg::st execute failed: ERROR: duplicate key value violates unique constraint "sessions_pkey" [for Statement "INSERT INTO public.sessions ( id) VALUES ( ? )" with ParamValues: 1='flash:d1ca1884fb9b94475a5b593b1ec9ca9d10ecb867'] at /opt/xt/xt- perl/lib/site_perl/5.8.8/Catalyst/Plugin/Session/Store/DBIC/Delegate.pm line 73 "

Tue Jul 07 06:04:01 2009 The RT System itself - Status changed from 'resolved' to 'open'

Tue Jul 07 09:16:51 2009 chisel [...] chizography.net - Cc CHISEL added

Tue Jul 07 11:36:15 2009 danieltwc [...] cpan.org - Correspondence added

On Tue Jul 07 06:04:00 2009, SPURKIS wrote: Show quoted text

> I've just come across this using > Catalyst::Plugin::Session::Store::DBIC v0.08. Had a quick look > at 0.09 and it doesn't look like this is fixed there. I've included > the the error report I've been > seeing at the end of this - we've been getting this maybe 30 times a > day, with ~200 users. > > The problem is that there's still a race condition in DBIC's > find_or_create. The method *IS > NOT* atomic. It does a find first, then a create if it found nothing. > In between it failing to > find something, another process may create it.

Agreed, however, the txn_do wrapper should help. I guess the queries are not happening at an appropriate isolation level, but is there a DBD-agnostic way of setting this? Show quoted text

> I think the easiest way around this is to wrap the txn_do in an eval, > and retry it $n times: > > # in Catalyst::Plugin::Session::Store::DBIC::Delegate > sub _load_row { > ... > for (1..$n) { > eval { > $row = $self->model->result_source->schema->txn_do($load_sub); > }; > if (my $e = $@) { > next if $e =~ /duplicate key value violates unique constraint > "sessions_pkey"/; > die $e; # pass on > } > } > ... > }

That would likely reduce the likelihood of the duplicate key error, but you're also adding to the number of queries significantly. What version of Catalyst::Plugin::Session are you using? Recent versions should store flash data along with the session itself, avoiding the need for separate flash rows. There was some talk of a race condition if you used $c->flash before $c->session before this change, but I don't recall the details.

Tue Jul 07 12:36:05 2009 steve [...] purkis.ca - Correspondence added

On Tue Jul 07 11:36:15 2009, DANIELTWC wrote: Show quoted text

> On Tue Jul 07 06:04:00 2009, SPURKIS wrote:

> > I've just come across this using > > Catalyst::Plugin::Session::Store::DBIC v0.08. Had a quick look > > at 0.09 and it doesn't look like this is fixed there. I've included > > the the error report I've been > > seeing at the end of this - we've been getting this maybe 30 times a > > day, with ~200 users. > > > > The problem is that there's still a race condition in DBIC's > > find_or_create. The method *IS > > NOT* atomic. It does a find first, then a create if it found nothing. > > In between it failing to > > find something, another process may create it.

> > Agreed, however, the txn_do wrapper should help. I guess the queries > are not happening at an appropriate isolation level, but is there a > DBD-agnostic way of setting this?

Hmm... not that I know of. Show quoted text

> > I think the easiest way around this is to wrap the txn_do in an eval, > > and retry it $n times:

[snip] Show quoted text

> > That would likely reduce the likelihood of the duplicate key error, but > you're also adding to the number of queries significantly.

True, but only in cases where the extra queries are executed. I was thinking of limiting it to 2. Show quoted text

> What version of Catalyst::Plugin::Session are you using?

0.19 Show quoted text

> Recent > versions should store flash data along with the session itself, avoiding > the need for separate flash rows. There was some talk of a race > condition if you used $c->flash before $c->session before this change, > but I don't recall the details.

Ah, thanks for the pointer - will try upgrading that first. -Steve

Fri Jul 10 18:37:38 2009 danieltwc [...] cpan.org - Correspondence added

I'm closing this assuming that upgrading Catalyst::Plugin::Session fixed it. Feel free to reopen it if not.

Fri Jul 10 18:37:40 2009 danieltwc [...] cpan.org - Status changed from 'open' to 'resolved'