Skip Menu |

This queue is for tickets about the DBIx-Class CPAN distribution.

Report information
The Basics
Id: 64679
Status: open
Priority: 0/
Queue: DBIx-Class

People
Owner: Nobody in particular
Requestors: ether [...] cpan.org
TJC [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in:
  • 0.08124
  • 0.08125
  • 0.08126
Fixed in: (no value)



Subject: find_or_create converts subselects to NULLs
I think I have found a bug in find_or_create(), although maybe it's more a case of "it should throw an error if you try this", instead of silenting doing the Wrong Thing. Viz: Assume you have these tables: table1, with columns "luser" and "role". The primary key is on "luser, role". "role" is a foreign key into table2.. table2 has columns "id" and "name". my $rs = $schema->resultset('table1'); $rs->find_or_create( { luser => 'john', role => { name => 'Admin' }, } ); In this case, the SELECT that dbix class does to try and find the user will be WHERE luser='john' AND role=NULL. When it then creates the row, it will correctly fill it in with the ID from the role table. Eg. like: INSERT INTO mytable (luser,role) VALUES ('john', (SELECT id FROM role WHERE name='Admin')); See attached for a failing test demonstrating the issue. (Provided as both patch, and raw t file) -Toby
Subject: update_or_create_single.t
use strict; use warnings; use Test::More; use lib qw(t/lib); use DBICTest; my $schema = DBICTest->init_schema(); plan tests => 20; my $artist = $schema->resultset ('Artist')->first; my $genre = $schema->resultset ('Genre') ->create ({ name => 'par excellence' }); is ($genre->search_related( 'model_cd' )->count, 0, 'No cds yet'); # expect a create $genre->update_or_create_related ('model_cd', { artist => $artist, year => 2009, title => 'the best thing since sliced bread', }); # verify cd was inserted ok is ($genre->search_related( 'model_cd' )->count, 1, 'One cd'); my $cd = $genre->find_related ('model_cd', {}); is_deeply ( { map { $_, $cd->get_column ($_) } qw/artist year title/ }, { artist => $artist->id, year => 2009, title => 'the best thing since sliced bread', }, 'CD created correctly', ); # expect a year update on the only related row # (non-qunique column + unique column as disambiguator) $genre->update_or_create_related ('model_cd', { year => 2010, title => 'the best thing since sliced bread', }); # re-fetch the cd, verify update is ($genre->search_related( 'model_cd' )->count, 1, 'Still one cd'); $cd = $genre->find_related ('model_cd', {}); is_deeply ( { map { $_, $cd->get_column ($_) } qw/artist year title/ }, { artist => $artist->id, year => 2010, title => 'the best thing since sliced bread', }, 'CD year column updated correctly', ); # expect an update of the only related row # (update a unique column) $genre->update_or_create_related ('model_cd', { title => 'the best thing since vertical toasters', }); # re-fetch the cd, verify update is ($genre->search_related( 'model_cd' )->count, 1, 'Still one cd'); $cd = $genre->find_related ('model_cd', {}); is_deeply ( { map { $_, $cd->get_column ($_) } qw/artist year title/ }, { artist => $artist->id, year => 2010, title => 'the best thing since vertical toasters', }, 'CD title column updated correctly', ); # expect a year update on the only related row # (non-unique column only) $genre->update_or_create_related ('model_cd', { year => 2011, }); # re-fetch the cd, verify update is ($genre->search_related( 'model_cd' )->count, 1, 'Still one cd'); $cd = $genre->find_related ('model_cd', {}); is_deeply ( { map { $_, $cd->get_column ($_) } qw/artist year title/ }, { artist => $artist->id, year => 2011, title => 'the best thing since vertical toasters', }, 'CD year column updated correctly without a disambiguator', ); # Test multi-level find-or-create functionality. # We should be able to find-or-create this twice, with the second time # returning the same item and genre as the first.. # This first test has the sub-level query on a non-unique key, ie. it # isn't checked as part of the "find" half of the method. my $genre_name = 'Highlander'; my %cd_details = ( year => '2010', title => 'Tasty Treats', genre => { name => $genre_name } ); my $genre2 = $schema->resultset ('Genre') ->create ({ name => $genre_name }); my $found1 = $artist->find_or_create_related('cds', { %cd_details }); ok($found1->id, "Found (actually created) album in first iteration"); is($found1->genre->name, $genre_name, ".. with correct genre"); my $found2 = $artist->find_or_create_related('cds', { %cd_details }); ok($found2->id, "Found album in second iteration"); is($found2->id, $found1->id, "..and the IDs are the same."); is($found2->genre->name, $genre_name, ".. with correct genre"); # Now we repeat the tests, using a sub-level query on one of the critical # keys that IS used in the "find" part. # Unfortunately DBIC's generated SQL looks for "artist=NULL" here. my $artist_name = 'Peanut and Cashew Mix'; my %new_cd = ( year => '2011', title => 'Various Failures', artist => { name => $artist_name }, ); my $found3 = $genre2->find_or_create_related('cds', { %new_cd }); ok($found3->id, "Found (actually created) album in first iteration"); is($found3->artist->name, $artist_name, "..with correct artist name"); my $found4 = $genre2->find_or_create_related('cds', { %new_cd }); ok($found4->id, "Found album in second iteration"); is($found4->id, $found3->id, "..and the IDs are the same."); is($found4->artist->name, $artist_name, ".. with correct artist name"); is($found4->artist->id, $found3->artist->id, "..matching artist ids");
Subject: 0001-Add-test-for-sub-levels-in-find_or_create.patch
From 7649e2e7726fe4a39aec9b2eb1883451c8e9a9ec Mon Sep 17 00:00:00 2001 From: Toby Corkindale <toby@dryft.net> Date: Mon, 20 Dec 2010 13:27:09 +1100 Subject: [PATCH] Add test for sub-levels in find_or_create. Currently failing. --- t/relationship/update_or_create_single.t | 48 +++++++++++++++++++++++++++++- 1 files changed, 47 insertions(+), 1 deletions(-) diff --git a/t/relationship/update_or_create_single.t b/t/relationship/update_or_create_single.t index a0e31fb..416a130 100644 --- a/t/relationship/update_or_create_single.t +++ b/t/relationship/update_or_create_single.t @@ -7,7 +7,7 @@ use DBICTest; my $schema = DBICTest->init_schema(); -plan tests => 9; +plan tests => 20; my $artist = $schema->resultset ('Artist')->first; @@ -95,3 +95,49 @@ is_deeply ( }, 'CD year column updated correctly without a disambiguator', ); + + +# Test multi-level find-or-create functionality. +# We should be able to find-or-create this twice, with the second time +# returning the same item and genre as the first.. +# This first test has the sub-level query on a non-unique key, ie. it +# isn't checked as part of the "find" half of the method. + +my $genre_name = 'Highlander'; +my %cd_details = ( + year => '2010', + title => 'Tasty Treats', + genre => { name => $genre_name } +); +my $genre2 = $schema->resultset ('Genre') + ->create ({ name => $genre_name }); + +my $found1 = $artist->find_or_create_related('cds', { %cd_details }); +ok($found1->id, "Found (actually created) album in first iteration"); +is($found1->genre->name, $genre_name, ".. with correct genre"); + +my $found2 = $artist->find_or_create_related('cds', { %cd_details }); +ok($found2->id, "Found album in second iteration"); +is($found2->id, $found1->id, "..and the IDs are the same."); +is($found2->genre->name, $genre_name, ".. with correct genre"); + + +# Now we repeat the tests, using a sub-level query on one of the critical +# keys that IS used in the "find" part. +# Unfortunately DBIC's generated SQL looks for "artist=NULL" here. +my $artist_name = 'Peanut and Cashew Mix'; +my %new_cd = ( + year => '2011', + title => 'Various Failures', + artist => { name => $artist_name }, +); +my $found3 = $genre2->find_or_create_related('cds', { %new_cd }); +ok($found3->id, "Found (actually created) album in first iteration"); +is($found3->artist->name, $artist_name, "..with correct artist name"); + +my $found4 = $genre2->find_or_create_related('cds', { %new_cd }); +ok($found4->id, "Found album in second iteration"); +is($found4->id, $found3->id, "..and the IDs are the same."); +is($found4->artist->name, $artist_name, ".. with correct artist name"); +is($found4->artist->id, $found3->artist->id, "..matching artist ids"); + -- 1.7.3.4
On Tue Jan 11 00:49:37 2011, TJC wrote: Show quoted text
> I think I have found a bug in find_or_create(), although maybe it's > more a case of "it should throw an error if you try this", instead of > silenting doing the Wrong Thing.
Unfortunately the find_or_create interface is generally not thought out, as it requires a user to pass incompatible args (the find and the create one) in the same method call. The fix to make your tests pass is trivial and you can see it here: http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=dbsrgits/DBIx-Class.git;a=commitdiff;h=dac9bfca09553f052ff6e37a680d9760646a8b66#patch1 Whether this is the right thing to do - I am not entirely sure, The fact that resolve_condition was there in the first place implies that someone thought it would be a good idea at the time, completely disregarding your particular use case. Let me know what you think about the situation in general, maybe you can come up with some solution that I am not seeing.
On Thu Jan 13 07:53:33 2011, RIBASUSHI wrote: Show quoted text
> The fix to make your tests pass is trivial > and you can see it here: > > http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=dbsrgits/DBIx- > Class.git;a=commitdiff;h=dac9bfca09553f052ff6e37a680d9760646a8b66#patch1 > > Whether this is the right thing to do - I am not entirely sure, The > fact > that resolve_condition was there in the first place implies that > someone > thought it would be a good idea at the time, completely disregarding > your particular use case.
Hmm, I can already see a case where the suggested fix wouldn't be valid, I'm afraid. Consider a table like this, which is linking three foreign keys: CREATE TABLE foo ( user_id INTEGER NOT NULL REFERENCES users(id), group_id INTEGER NOT NULL REFERENCES groups(id), role_id INTEGER NOT NULL REFERENCES roles(id), PRIMARY KEY (user_id, group_id, role_id) ); Now say you tried to do: $schema->resultset('Foo')->find_or_create( { user_id => 1, group_id => { name => "DBIC Developers" }, role_id => { name => "Moderator" } } ); $schema->resultset('Foo')->find_or_create( { user_id => 1, group_id => { name => "DBIC Users" }, role_id => { name => "Moderator" } } ); This would fail using the suggested fix, as the find.. portion would just look for the user_id component and ignore the other two. Show quoted text
> Let me know what you think about the situation in general, maybe you > can come up with some solution that I am not seeing.
It sounds like we need the sub-queries to be resolved properly during the find() portion, but I guess that's hard, if they are only supported during a create()? Otherwise, I'd prefer to just have an error thrown, although that might surprise anyone relying upon the bug's existing behaviour, if any?
On Fri Jan 14 00:21:36 2011, TJC wrote: Show quoted text
> On Thu Jan 13 07:53:33 2011, RIBASUSHI wrote:
> > The fix to make your tests pass is trivial > > and you can see it here: > > > > http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=dbsrgits/DBIx- > >
> Class.git;a=commitdiff;h=dac9bfca09553f052ff6e37a680d9760646a8b66#patch1
> > > > Whether this is the right thing to do - I am not entirely sure, The > > fact > > that resolve_condition was there in the first place implies that > > someone > > thought it would be a good idea at the time, completely disregarding > > your particular use case.
> > Hmm, I can already see a case where the suggested fix wouldn't be > valid, I'm afraid.
I mistook an irc 'tjc' to be you and braindumped the following: <ribasushi> tjc: on RT#64679 - I am generally not opposed to having *something* done on this front, as long as someone else does it and does it right (i.e. with regard to backwards comp, doing the necessary research why things are currently the way they are etc) <ribasushi> tjc: so don't take my skepticism expressed in the ticket as "this is to remain broken" <ribasushi> tjc: I just don't want to be the one to unfuck it, got enough other things :) <ribasushi> tjc: fwiw the original code is old and have not changed much (and judging by the commit message wasn't without controversy either :) <ribasushi> http://git.shadowcat.co.uk/gitweb/gitweb.cgi?p=dbsrgits/DBIx-Class.git;a=blobdiff;f=lib/DBIx/Class/ResultSet.pm;h=64bcd4ea53230b3549f94e275605bc097f47cada;hp=6ab3ec08344aef2970cafd8c9a9cae1810a6af0f;hb=096f421241;hpb=03f24ee3c4fd551a0de43a1cc2821184f8864cb8 So if you can help me with this issue - awesome! Cheers
Stalling ticket as "needs sqla2" unless better ideas pop up in the meantime.
Subject: find_or_create on nested values does not work as expected
here is a repro case: use strict; use warnings; use Test::More; use Test::Exception; use lib qw(t/lib); use DBICTest; my $schema = DBICTest->init_schema(); my $cd_rs = $schema->resultset('CD'); my $cd = $cd_rs->create({ title=>"Music for Silly Walks", year=>2000, artist => { name=>"Silly Musician", } }); my $artist = $cd->artist; is($cd->title, "Music for Silly Walks", 'got cd row'); is($artist->name, "Silly Musician", 'got artist row'); my $cd2 = $cd_rs->find_or_create({ title=>"More Music for Silly Walks", year=>2001, artist => { name=>"Silly Musician", } }); is($cd2->title, "More Music for Silly Walks", 'got second cd row'); my $artist2 = $cd2->artist; is($artist2->name, "Silly Musician", 'got artist row'); is($artist2->artistid, $artist->artistid, 'artist row is the same as the first'); my $cd3 = $cd_rs->find_or_create({ title=>"More Music for Silly Walks", # same as cd2! year=>2001, artist => { name=>"Silly Musician", } }); is($cd3->title, "More Music for Silly Walks", 'got "third" cd row'); is($cd3->cdid, $cd2->cdid, 'cd row is the same as the second'); done_testing; __END__ with DBIC_TRACE, gives: SELECT me.artistid, me.name, me.rank, me.charfield FROM artist me WHERE ( me.name = ? ): 'Silly Musician' INSERT INTO artist ( name) VALUES ( ? ): 'Silly Musician' INSERT INTO cd ( artist, title, year) VALUES ( ?, ?, ? ): '4', 'Music for Silly Walks', '2000' COMMIT SELECT me.artistid, me.name, me.rank, me.charfield FROM artist me WHERE ( me.artistid = ? ): '4' ok 1 - got cd row ok 2 - got artist row SELECT me.cdid, me.artist, me.title, me.year, me.genreid, me.single_track FROM cd me WHERE ( ( me.artist IS NULL AND me.title = ? AND me.year = ? ) ): 'More Music for Silly Walks', '2001' SELECT me.artistid, me.name, me.rank, me.charfield FROM artist me WHERE ( me.name = ? ): 'Silly Musician' INSERT INTO cd ( artist, title, year) VALUES ( ?, ?, ? ): '4', 'More Music for Silly Walks', '2001' ok 3 - got second cd row SELECT me.artistid, me.name, me.rank, me.charfield FROM artist me WHERE ( me.artistid = ? ): '4' ok 4 - got artist row ok 5 - artist row is the same as the first SELECT me.cdid, me.artist, me.title, me.year, me.genreid, me.single_track FROM cd me WHERE ( ( me.artist IS NULL AND me.title = ? AND me.year = ? ) ): 'More Music for Silly Walks', '2001' SELECT me.artistid, me.name, me.rank, me.charfield FROM artist me WHERE ( me.name = ? ): 'Silly Musician' INSERT INTO cd ( artist, title, year) VALUES ( ?, ?, ? ): '4', 'More Music for Silly Walks', '2001' DBI Exception: DBD::SQLite::st execute failed: UNIQUE constraint failed: cd.artist, cd.title [for Statement "INSERT INTO cd ( artist, title, year) VALUES ( ?, ?, ? )"] at lib/DBIx/Class/Schema.pm line 1081, <> line 1. DBIx::Class::Schema::throw_exception(DBICTest::Schema=HASH(0x7ff28af27cf8), "DBI Exception: DBD::SQLite::st execute failed: UNIQUE constra"...) called at lib/DBIx/Class/Storage.pm line 113 DBIx::Class::Storage::throw_exception(DBIx::Class::Storage::DBI::SQLite=HASH(0x7ff28af2d6f8), "DBI Exception: DBD::SQLite::st execute failed: UNIQUE constra"...) called at lib/DBIx/Class/Storage/DBI.pm line 1473 DBIx::Class::Storage::DBI::__ANON__("DBD::SQLite::st execute failed: UNIQUE constraint failed: cd."..., DBI::st=HASH(0x7ff28bdb2a60), undef) called at lib/DBIx/Class/Storage/DBI.pm line 1840 DBIx::Class::Storage::DBI::_dbh_execute(DBIx::Class::Storage::DBI::SQLite=HASH(0x7ff28af2d6f8), DBI::db=HASH(0x7ff28bbae998), "INSERT INTO cd ( artist, title, year) VALUES ( ?, ?, ? )", ARRAY(0x7ff28bdb2b68), ARRAY(0x7ff28bdbca90)) called at lib/DBIx/Class/Storage/DBI/SQLite.pm line 289 DBIx::Class::Storage::DBI::SQLite::_dbh_execute(DBIx::Class::Storage::DBI::SQLite=HASH(0x7ff28af2d6f8), DBI::db=HASH(0x7ff28bbae998), "INSERT INTO cd ( artist, title, year) VALUES ( ?, ?, ? )", ARRAY(0x7ff28bdb2b68), ARRAY(0x7ff28bdbca90)) called at lib/DBIx/Class/Storage/DBI.pm line 855 DBIx::Class::Storage::DBI::__ANON__() called at lib/DBIx/Class/Storage/BlockRunner.pm line 136 DBIx::Class::Storage::BlockRunner::try {...} () called at /Users/ether/.perlbrew/libs/20.0@std/lib/perl5/Try/Tiny.pm line 77 eval {...} called at /Users/ether/.perlbrew/libs/20.0@std/lib/perl5/Try/Tiny.pm line 72 Try::Tiny::try(CODE(0x7ff28bdbd1f8), Try::Tiny::Catch=REF(0x7ff28bdb2ac0)) called at lib/DBIx/Class/Storage/BlockRunner.pm line 140 DBIx::Class::Storage::BlockRunner::__ANON__() called at /Users/ether/.perlbrew/libs/20.0@std/lib/perl5/Context/Preserve.pm line 32 Context::Preserve::preserve_context(CODE(0x7ff28bdbd588), "replace", CODE(0x7ff28bdbd6d8)) called at lib/DBIx/Class/Storage/BlockRunner.pm line 219 DBIx::Class::Storage::BlockRunner::_run(DBIx::Class::Storage::BlockRunner=HASH(0x7ff28bdbd1e0), CODE(0x7ff28ae4dae0)) called at lib/DBIx/Class/Storage/BlockRunner.pm line 111 DBIx::Class::Storage::BlockRunner::run(DBIx::Class::Storage::BlockRunner=HASH(0x7ff28bdbd1e0), CODE(0x7ff28ae4dae0)) called at lib/DBIx/Class/Storage/DBI.pm line 856 DBIx::Class::Storage::DBI::dbh_do(undef, undef, "INSERT INTO cd ( artist, title, year) VALUES ( ?, ?, ? )", ARRAY(0x7ff28bdb2b68), ARRAY(0x7ff28bdbca90)) called at lib/DBIx/Class/Storage/DBI.pm line 1821 DBIx::Class::Storage::DBI::_execute(DBIx::Class::Storage::DBI::SQLite=HASH(0x7ff28af2d6f8), "insert", DBIx::Class::ResultSource::Table=HASH(0x7ff28af3d328), HASH(0x7ff28bcbf1c8), undef) called at lib/DBIx/Class/Storage/DBI.pm line 1983 DBIx::Class::Storage::DBI::insert(DBIx::Class::Storage::DBI::SQLite=HASH(0x7ff28af2d6f8), DBIx::Class::ResultSource::Table=HASH(0x7ff28af3d328), HASH(0x7ff28ba52a68)) called at lib/DBIx/Class/Row.pm line 405 DBIx::Class::Row::insert(DBICTest::CD=HASH(0x7ff28bdbd498)) called at lib/DBIx/Class/ResultSet.pm line 2800 DBIx::Class::ResultSet::create(DBICTest::BaseResultSet=HASH(0x7ff28bc349f8), HASH(0x7ff28bdb2478)) called at lib/DBIx/Class/ResultSet.pm line 2882 DBIx::Class::ResultSet::find_or_create(DBICTest::BaseResultSet=HASH(0x7ff28bc349f8), HASH(0x7ff28bdb2478)) called at t/find_or_create_multi.t line 42 # Auto checked 5 references for leaks - none detected # Tests were run but no plan was declared and done_testing() was not seen. # Looks like your test exited with 255 just after 5.
When I add { key => 'cd_artist_title' } as the second argument to all $cd_rs->find_or_create calls, this warning message appears for all calls, and helps decipher the problem: DBIx::Class::ResultSet::_build_unique_cond(): NULL/undef values supplied for requested unique constraint 'cd_artist_title' (NULL values in column(s): 'artist'). This is almost certainly not what you wanted, though you can set DBIC_NULLABLE_KEY_NOWARN to disable this warning. at t/find_or_create_multi.t line 28 So, even though we provide enough information to find a unique artist, the artist PK is not being fed into the cd->find query.
On Fri Sep 05 22:22:22 2014, ETHER wrote: Show quoted text
> ... > So, even though we provide enough information to find a unique artist, > the artist PK is not being fed into the cd->find query.
As you can tell from the merge-action on this ticket - this is a known issue (originally RT#64679). The work done so far by Toby is sitting in https://github.com/dbsrgits/dbix-class/commit/6572c1f4c2 (this is the same commit as the dac9bfca09 mentioned in https://rt.cpan.org/Ticket/Display.html?id=64679#txn-881074, it simply drifted since, due to a rebase). It never went forward due to lack of clarity whether this is actually the right approach (see resume in https://rt.cpan.org/Ticket/Display.html?id=64679#txn-882420, and particularly the original commit introducing the inconsistency) Now almost 4 years later the landscape has changed a bit. In particular: - there is a very good warning-emitting framework in place allowing for unobtrusive deprecations (carp_unique) - the code of find() has been refactored multiple times, and provides a clear delineation of where/how it does its stuff - we have a very sophisticated relationship condition resolver, able to reliably pinpoint which cross-table columns are mapped together. Putting all of the above together I am confident an interested party can produce a workable plan going forward. As I said earlier - the current situation sucks, and likely is worked around in many situations. The sticking point is that while it is possible to break these workarounds, the *reason* for breaking them should be clear and unambiguous. Therefore - let's spec this up and see if we can poke any holes in it. If we can't - it will get implemented (and likely be made available for immediate use with the proper $rs-level attr). I am unstalling this ticket in the hope that we can go somewhere new :) But just as 3 years ago - volunteers to do the *spec* legwork are sorely needed (the code itself is likely to be trivial). Cheers
On Fri Jan 14 06:21:36 2011, TJC wrote: Show quoted text
> > It sounds like we need the sub-queries to be resolved properly during > the find() > portion, but I guess that's hard, if they are only supported during a > create()?
An extra note on this particular point: there is now a working (albeit a bit limited) implementation of a "related creation with ids retrieved via a subquery" as part of https://github.com/dbsrgits/dbix-class/commit/d0cefd99a9 (read the commit message, especially paragraphs 4~6). The actual implementation of this can be found here: https://github.com/dbsrgits/dbix-class/blob/e089c417ce/lib/DBIx/Class/ResultSet.pm#L2450-L2486 , and even more importantly the subset https://github.com/dbsrgits/dbix-class/blob/e089c417ce/lib/DBIx/Class/ResultSet.pm#L2473-L2481 , which basically can be translated as: "Create a resultset which has each of our PKs defined via subqueries into the parent resultset, and then call create on that". Nothing prevents you from firing an actual find, it just has to be implemented with care, and a ton of extra tests. For completeness - this is what a populate query looks like based on this code: INSERT INTO fourkeys_to_twokeys ( autopilot, f_bar, f_foo, f_goodbye, f_hello, t_artist, t_cd ) VALUES ( ?, ?, ?, ?, ?, ( SELECT me.artist FROM twokeys me WHERE artist = ? AND cd = ? ), ( SELECT me.cd FROM twokeys me WHERE artist = ? AND cd = ? ) ) Hope this is helpful.