Skip Menu |

This queue is for tickets about the DBD-File CPAN distribution.

Report information
The Basics
Id: 33015
Status: resolved
Priority: 0/
Queue: DBD-File

People
Owner: Nobody in particular
Requestors: MARKOV [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Important
Broken in: 0.35
Fixed in: (no value)



Subject: fetchrow-arrayref complains at end
When running through a table with fetchrow_arrayref, we get an error at the end of the table: $sth->set_err($DBI::stderr, "Attempt to fetch row from a Non-SELECT statement"); This happens because there is no data left, so always. Removing the error makes the problem go away.
Subject: Re: [rt.cpan.org #33015] AutoReply: fetchrow-arrayref complains at end
Date: Fri, 8 Feb 2008 16:48:16 +0100
To: Bugs in DBD-File via RT <bug-DBD-File [...] rt.cpan.org>
From: Mark Overmeer <mark [...] overmeer.net>
Show quoted text
> "fetchrow-arrayref complains at end",
I must correct a mistake, the problem is with fetchALL_arrayref. Sorry for the confusion. And I must add that I do access DBD::File via DBD::CSV. The error is produced by DBD::File, and I think the source of the problem is there as well. ---- 8< --- The script #!/usr/bin/perl use warnings; use strict; use DBI; my $dbh = DBI->connect("DBI:CSV:csv_sep_char=#;csv_eol=\n") or die "unable to connect to test CSV database: $DBI::errstr"; $dbh->{csv_tables}{mytable} = {file => 'a.csv'}; my $sth = $dbh->prepare('SELECT * FROM mytable') or die "NS query failed: ".$dbh->errstr; $sth->execute or die "NS execute failed: ".$sth->errstr; while(1) { my $rows = $sth->fetchall_arrayref; $rows && @$rows or last; use Data::Dumper; warn Dumper $rows; } ---- 8< --- Example data (file a.csv) A#B 2#4 -- MarkOv
You're calling fetchall_arrayref in a loop that doesn't include an execute(), so the second time round the loop there's nothing to fetch and you get the error. I've changed the error text to read: "Attempt to fetch row without a preceeding execute() call or from a non-SELECT statement"
Subject: Re: [rt.cpan.org #33015] fetchrow-arrayref complains at end
Date: Fri, 8 Feb 2008 23:55:33 +0100
To: Tim_Bunce via RT <bug-DBD-File [...] rt.cpan.org>
From: Mark Overmeer <mark [...] overmeer.net>
* Tim_Bunce via RT (bug-DBD-File@rt.cpan.org) [080208 22:30]: Show quoted text
> > <URL: http://rt.cpan.org/Ticket/Display.html?id=33015 > > > You're calling fetchall_arrayref in a loop that doesn't include an > execute(), so the second time > round the loop there's nothing to fetch and you get the error. > > I've changed the error text to read: > > "Attempt to fetch row without a preceeding execute() call or from a non-SELECT statement"
I don't think that is correct, and the Oracle driver behaves as documented where CSV does not. In the docs, I read If there are no rows to return, "fetchall_arrayref" returns a reference to an empty array. If an error occurs, "fetchall_arrayref" returns the data fetched thus far, which may be none. You should check "$sth->err" afterwards (or use the "RaiseError" attribute) to discover if the data is complete or was truncated due to an error. So: if I call it for the second time, it should return an empty array. The problem also exists when I call $dbh->fetchall_arrayref(undef, 10) When am I through all rows? Or am I reading the man-pages wrongly? -- Regards, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Subject: Re: [rt.cpan.org #33015] fetchrow-arrayref complains at end
Date: Fri, 8 Feb 2008 23:34:59 +0000
To: Mark Overmeer via RT <bug-DBD-File [...] rt.cpan.org>
From: Tim Bunce <Tim.Bunce [...] pobox.com>
On Fri, Feb 08, 2008 at 05:56:07PM -0500, Mark Overmeer via RT wrote: Show quoted text
> > > > You're calling fetchall_arrayref in a loop that doesn't include an > > execute(), so the second time > > round the loop there's nothing to fetch and you get the error. > > > > I've changed the error text to read: > > > > "Attempt to fetch row without a preceeding execute() call or from a non-SELECT statement"
> > I don't think that is correct, and the Oracle driver behaves as documented > where CSV does not. In the docs, I read > > If there are no rows to return, "fetchall_arrayref" returns a reference > to an empty array. If an error occurs, "fetchall_arrayref" returns the > data fetched thus far, which may be none. You should check "$sth->err" > afterwards (or use the "RaiseError" attribute) to discover if the data > is complete or was truncated due to an error. > > So: if I call it for the second time, it should return an empty array.
The doc you quote doesn't address the issue of calling fetchall_arrayref a second time *without an intervening call to execute*. Which is what the example you gave does. Show quoted text
> The problem also exists when I call $dbh->fetchall_arrayref(undef, 10)
(I presume you mean $sth not $dbh.) That's arguably a bug, though it's not documented other than an old note in the Changes file: =head2 Changes in DBI 1.31, 29th November 2002 The fetchall_arrayref method, when called with a $maxrows parameter, no longer gives an error if called again after all rows have been fetched. This simplifies application logic when fetching in batches. Also added batch-fetch while() loop example to the docs. Show quoted text
> When am I through all rows? > Or am I reading the man-pages wrongly?
I think the docs are not explicit about this case. I'd expect a second consecutive $sth->fetchall_arrayref() to cause an error, and a $sth->fetchall_arrayref(undef, 10) to not. Though both should return undef. Patches welcome. Tim.
Subject: Re: [rt.cpan.org #33015] fetchrow-arrayref complains at end
Date: Sat, 9 Feb 2008 09:55:11 +0100
To: Tim Bunce via RT <bug-DBD-File [...] rt.cpan.org>
From: Mark Overmeer <mark [...] overmeer.net>
* Tim Bunce via RT (bug-DBD-File@rt.cpan.org) [080208 23:35]: Show quoted text
> I'd expect a second consecutive $sth->fetchall_arrayref() to > cause an error, and a $sth->fetchall_arrayref(undef, 10) to not. > Though both should return undef.
I expect $sth->fetchall_arrayref to behave as $sth->fetchall_arrayref(undef, undef) and an <undef> number of rows as <all rows> More like splice, or any queue. From empty you get nothing. The fix is simple: remove the error message. -- Regards, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Subject: Re: [rt.cpan.org #33015] fetchrow-arrayref complains at end
Date: Sat, 9 Feb 2008 21:35:47 +0000
To: Mark Overmeer via RT <bug-DBD-File [...] rt.cpan.org>
From: Tim Bunce <Tim.Bunce [...] pobox.com>
On Sat, Feb 09, 2008 at 03:55:35AM -0500, Mark Overmeer via RT wrote: Show quoted text
> > Queue: DBD-File > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=33015 > > > * Tim Bunce via RT (bug-DBD-File@rt.cpan.org) [080208 23:35]:
> > I'd expect a second consecutive $sth->fetchall_arrayref() to > > cause an error, and a $sth->fetchall_arrayref(undef, 10) to not. > > Though both should return undef.
> > I expect $sth->fetchall_arrayref to behave as > $sth->fetchall_arrayref(undef, undef) > and an <undef> number of rows as <all rows> > More like splice, or any queue. From empty you get nothing.
You're changing the subject slightly. The original issue is whether this code should case a warning: while (1) { my $rows = $sth->fetchall_arrayref; $rows && @$rows or last; warn Dumper $rows; } I believe it should. There's simply no point in putting a fetch*all* rows call into a loop like that. The DBI issues a warning because in 99.97364% of cases the warning reflects a problem with the logic of the application. To address your point above, it's an error to try to fetch from a statement handle after you're already fetched all the rows. Always has been, always will. An exception was made for fetchall_arrayref(..., $max_rows) because without it this code would almost always trigger an error: my $rows = []; # cache for batches of rows while( my $row = ( shift(@$rows) || # get row from cache, or reload cache: shift(@{$rows=$sth->fetchall_arrayref(undef,10_000)||[]}) ) ) { ... } Show quoted text
> The fix is simple: remove the error message.
Not that simple. That would also remove the error message that should be given for any single row fetch beyond the end of the result set. Tim.
Subject: Re: [rt.cpan.org #33015] fetchrow-arrayref complains at end
Date: Sun, 10 Feb 2008 00:17:12 +0100
To: Tim Bunce via RT <bug-DBD-File [...] rt.cpan.org>
From: Mark Overmeer <mark [...] overmeer.net>
* Tim Bunce via RT (bug-DBD-File@rt.cpan.org) [080209 21:37]: Show quoted text
> > I expect $sth->fetchall_arrayref to behave as > > $sth->fetchall_arrayref(undef, undef) > > and an <undef> number of rows as <all rows> > > More like splice, or any queue. From empty you get nothing.
> > You're changing the subject slightly. The original issue is whether this > code should case a warning: > > while (1) { > my $rows = $sth->fetchall_arrayref; > $rows && @$rows or last; > warn Dumper $rows; > }
True. First, I detected the bug with row-count argument, then simplified the situation into without arguments. Usually, it is better to file a bug as simple as possible. In this case, it changed your opinion. But apparently, this situation is designed to be non-orthogonal: the behavior of fetchall_arrayref differs on the number of parameters given, not on the content of the parameters. Use less parameters means use more defaults. Show quoted text
> I believe it should. There's simply no point in putting a fetch*all* rows > call into a loop like that. The DBI issues a warning because in 99.97364% > of cases the warning reflects a problem with the logic of the application.
Might be true. But that is not a problem on its own: without error message, people will see that they loop endlessly or get an empty array on the second loop which results in a false. Show quoted text
> To address your point above, it's an error to try to fetch from a > statement handle after you're already fetched all the rows. Always has > been, always will.
Perl does not do this (I know I am not telling you something you don't know). If I shift on an empty array, I do not get an error. my @a = <>; # execute while(1) { splice @a, 0, @a } So, IMO your restriction is against Perl's common philosophy and non-orthogonal... all to help an accidental abuse. Show quoted text
> An exception was made for fetchall_arrayref(..., $max_rows) because > without it this code would almost always trigger an error: > > my $rows = []; # cache for batches of rows > while( my $row = ( shift(@$rows) || # get row from cache, or reload cache: > shift(@{$rows=$sth->fetchall_arrayref(undef,10_000)||[]}) ) > ) { > ... > }
I know this line from the man-page. It does not contain an error-check on the fetch. I prefer: while(1) { my $rows = $sth->fetchall_arrayref(undef, ROW_CACHE_SIZE); die "not all records received: ".$sth->errstr if $sth->err; defined $rows && @$rows or last; foreach (@$rows) { ... } } 4 understandable lines of code + check, instead of three very compact lines. which also dies with DBD::CSV::st fetchall_arrayref failed: Attempt to fetch row from a Non-SELECT statement [for Statement "SELECT * FROM mytable"] at ./a line 26. Show quoted text
> Not that simple. That would also remove the error message that should be > given for any single row fetch beyond the end of the result set.
my @a = 1..100; $a[3000] returns undef In my view, Perl modules should imitate behavior of perl core functionality, if they can. Perl doesn't complain much. -- Regards, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net
Subject: Re: [rt.cpan.org #33015] fetchrow-arrayref complains at end
Date: Mon, 11 Feb 2008 12:49:30 +0000
To: Mark Overmeer via RT <bug-DBD-File [...] rt.cpan.org>
From: Tim Bunce <Tim.Bunce [...] pobox.com>
On Sat, Feb 09, 2008 at 06:17:36PM -0500, Mark Overmeer via RT wrote: Show quoted text
> > Queue: DBD-File > Ticket <URL: http://rt.cpan.org/Ticket/Display.html?id=33015 > > > * Tim Bunce via RT (bug-DBD-File@rt.cpan.org) [080209 21:37]:
> > > I expect $sth->fetchall_arrayref to behave as > > > $sth->fetchall_arrayref(undef, undef) > > > and an <undef> number of rows as <all rows> > > > More like splice, or any queue. From empty you get nothing.
> > > > You're changing the subject slightly. The original issue is whether this > > code should case a warning: > > > > while (1) { > > my $rows = $sth->fetchall_arrayref; > > $rows && @$rows or last; > > warn Dumper $rows; > > }
> > True. First, I detected the bug with row-count argument, then simplified > the situation into without arguments. Usually, it is better to file a bug > as simple as possible. In this case, it changed your opinion. > > But apparently, this situation is designed to be non-orthogonal: > the behavior of fetchall_arrayref differs on the number of parameters > given, not on the content of the parameters. Use less parameters means > use more defaults.
It's not the number of parameters. The behaviour depends on whether the $max_rows parameter is true. The bug here is that the "don't warn about fetch-without-execute" logic that was added in 1.31 only covered the "driver implemented in C" case. I've added this line to the top of sub fetchall_arrayref in DBI.pm: return undef if $max_rows and not $sth->{Active}; which fixes your test case (when a $max_rows argument is used). Show quoted text
> > I believe it should. There's simply no point in putting a fetch*all* rows > > call into a loop like that. The DBI issues a warning because in 99.97364% > > of cases the warning reflects a problem with the logic of the application.
> > Might be true. But that is not a problem on its own: without error > message, people will see that they loop endlessly or get an empty > array on the second loop which results in a false.
That's just one kind of case. I've seen plenty of examples where the error has helped detect problems in more complex logic (where execution and fetching are well separated) such as ORMs. Show quoted text
> > To address your point above, it's an error to try to fetch from a > > statement handle after you're already fetched all the rows. Always has > > been, always will.
> > Perl does not do this (I know I am not telling you something you don't > know). If I shift on an empty array, I do not get an error. > > my @a = <>; # execute > while(1) { splice @a, 0, @a } > > So, IMO your restriction is against Perl's common philosophy and > non-orthogonal... all to help an accidental abuse.
I'll see your "non-orthogonal" argument and raise you a "has been this way for the entire ~14 year life of the DBI" :) It's not going to change now. Show quoted text
> > An exception was made for fetchall_arrayref(..., $max_rows) because > > without it this code would almost always trigger an error: > > > > my $rows = []; # cache for batches of rows > > while( my $row = ( shift(@$rows) || # get row from cache, or reload cache: > > shift(@{$rows=$sth->fetchall_arrayref(undef,10_000)||[]}) ) > > ) { > > ... > > }
> > I know this line from the man-page. It does not contain an error-check on > the fetch. I prefer:
I've added a note to the docs that the example assumes RaiseError is enabled. Even if it wasn't, a check after the loop would be fast and effective. I'm sorry the fix just missed the DBI 1.602 release. It'll be in 1.603. Thanks for your help Mark. Tim.
Subject: Re: [rt.cpan.org #33015] fetchrow-arrayref complains at end
Date: Mon, 11 Feb 2008 20:07:20 +0100
To: Tim Bunce via RT <bug-DBD-File [...] rt.cpan.org>
From: Mark Overmeer <solutions [...] overmeer.net>
* Tim Bunce via RT (bug-DBD-File@rt.cpan.org) [080211 13:20]: Show quoted text
> I've added this line to the top of sub fetchall_arrayref in DBI.pm: > return undef if $max_rows and not $sth->{Active};
Thanks a lot for your effort. Show quoted text
> I'll see your "non-orthogonal" argument and raise you a "has been this > way for the entire ~14 year life of the DBI" :)
I maintain some 14 years old code as well :( [MailTools] You never get a chance to redo interface decissions. Show quoted text
> I've added a note to the docs that the example assumes RaiseError is enabled. > Even if it wasn't, a check after the loop would be fast and effective.
In my experience, people tend not to read the text, but simple copy examples. Therefore, I try to include the error handling in my examples... Show quoted text
> I'm sorry the fix just missed the DBI 1.602 release. It'll be in 1.603.
No problem, I'm no hurry. -- Thanks Tim, MarkOv ------------------------------------------------------------------------ Mark Overmeer MSc MARKOV Solutions Mark@Overmeer.net solutions@overmeer.net http://Mark.Overmeer.net http://solutions.overmeer.net