Bug #28890 for CPAN-SQLite: 05meta.t fails "no such table: auths(1)" on OS X

Sat Aug 18 17:33:51 2007 mschwern [...] cpan.org - Ticket created

Subject:

05meta.t fails "no such table: auths(1)" on OS X

t/05meta.t fails with: DBD::SQLite::db prepare failed: no such table: auths(1) at dbdimp.c line 271 at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/DBI/Search.pm line 86. # Looks like you planned 2515 tests but only ran 1. Here's a stack trace from the debugger. main::(t/05meta.t:62): my $auth = CPAN::Shell->expand("Author", $cpanid); DB<2> CPAN: CPAN::SQLite loaded ok (v0.18) Creating database file ... at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/META.pm line 308 CPAN::SQLite::META::reload('CPAN::SQLite::META=HASH(0x1c2bf9c)', 'time', 1187472335, 'force', 'undef') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 4584 CPAN::Index::reload('CPAN::Index') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 1124 CPAN::exists('CPAN=HASH(0x1b409fc)', 'CPAN::Author', 'LDS') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 2795 CPAN::Shell::expand_by_method('CPAN::Shell', 'CPAN::Author', 'ARRAY(0x1b2d3e8)', 'LDS') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 2715 CPAN::Shell::expand('CPAN::Shell', 'Author', 'LDS') called at t/05meta.t line 62 Removing existing /var/local/cpan_shell/cpandb.sql Gathering information from index files ... Populating database tables ... Done! at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/META.pm line 316 CPAN::SQLite::META::reload('CPAN::SQLite::META=HASH(0x1c2bf9c)', 'time', 1187472335, 'force', 'undef') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 4584 CPAN::Index::reload('CPAN::Index') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 1124 CPAN::exists('CPAN=HASH(0x1b409fc)', 'CPAN::Author', 'LDS') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 2795 CPAN::Shell::expand_by_method('CPAN::Shell', 'CPAN::Author', 'ARRAY(0x1b2d3e8)', 'LDS') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 2715 CPAN::Shell::expand('CPAN::Shell', 'Author', 'LDS') called at t/05meta.t line 62 DBD::SQLite::db prepare failed: no such table: auths(1) at dbdimp.c line 271 at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/DBI/Search.pm line 86. at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/DBI/Search.pm line 86 CPAN::SQLite::DBI::Search::fetch_and_set('CPAN::SQLite::DBI::Search::auths=HASH(0x1c8923c)', 'search', 'HASH(0x1c89650)', 'fields', 'ARRAY(0x1c896ec)', 'order_by', 'cpanid', 'table', 'auths', ...) called at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/Search.pm line 112 CPAN::SQLite::Search::author::search('CPAN::SQLite::Search::author=HASH(0x1c87c50)', 'search', 'HASH(0x1c89650)', 'meta_obj', 'CPAN::SQLite::META::Author=HASH(0x1acf430)') called at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/Search.pm line 78 CPAN::SQLite::Search::query('CPAN::SQLite::Search=HASH(0x1c88004)', 'query', 'undef', 'mode', 'undef', 'name', 'undef', 'id', 'undef', ...) called at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite.pm line 71 CPAN::SQLite::query('CPAN::SQLite=HASH(0x1c2b168)', 'mode', 'author', 'name', 'LDS', 'meta_obj', 'CPAN::SQLite::META::Author=HASH(0x1acf430)') called at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/META.pm line 53 CPAN::SQLite::META::Author::set_one('CPAN::SQLite::META::Author=HASH(0x1acf430)') called at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/META.pm line 20 CPAN::SQLite::META::set('CPAN::SQLite::META=HASH(0x1c2bf9c)', 'CPAN::Author', 'LDS') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 1130 CPAN::exists('CPAN=HASH(0x1b409fc)', 'CPAN::Author', 'LDS') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 2795 CPAN::Shell::expand_by_method('CPAN::Shell', 'CPAN::Author', 'ARRAY(0x1b2d3e8)', 'LDS') called at /usr/local/perl/5.8.8/lib/CPAN.pm line 2715 CPAN::Shell::expand('CPAN::Shell', 'Author', 'LDS') called at t/05meta.t line 62 DBD::SQLite::db prepare failed: no such table: auths(1) at dbdimp.c line 271 at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/DBI/Search.pm line 86. # Looks like you planned 2515 tests but only ran 1. Debugged program terminated. Use q to quit or R to restart, use o inhibit_exit to avoid stopping after program termination, h q, h R or h o to get additional info. DB<2> Here's some more info: DB<2> x $CPAN::Config->{cpan_home} 0 '/Users/schwern/devel/CPAN-SQLite/t/dot-cpan' Looking at /Users/schwern/devel/CPAN-SQLite/t/dot-cpan/cpandb.sql via sqlite confirms there are no tables... 0 windhund ~/devel/CPAN-SQLite/t/dot-cpan$ sqlite3 cpandb.sql SQLite version 3.1.3 Enter ".help" for instructions Show quoted text

sqlite> .tables sqlite>

Sat Aug 18 17:47:32 2007 mschwern [...] cpan.org - Correspondence added

The problem appears to be in CPAN::SQLite::META::reload(). It figures out where the database file is, if it exists and whether it needs updating or setup. It then completely ignores all this information and goes and runs a shell command to setup or update the new database *without telling that command where the database file is!* Thus, the new database winds up in the current working directory, which is the top level of the source tree and not t/dot-cpan/ as expected by the shell. The db_dir and db_name should be passed through to setup/update. Ideally by not shelling out to call setup/update() and instead just using the current object or a copy of the current object.

Sat Aug 18 17:47:34 2007 mschwern [...] cpan.org - Status changed from 'new' to 'open'

Sun Aug 19 17:49:42 2007 RKOBES [...] cpan.org - Correspondence added

Thanks very much for the analysis. Having just got access to an OS X (through my daughter), I had thought this problem was fixed in version 0.18 (which passed on her system), but apparently not. The reason that CPAN::SQLite::META::reload() calls setup/update() through a shell command is to save on memory. setup/update() requires memory to hold the distribution/module/author info of all the CPAN packages, which if kept would make the memory footprint of CPAN::SQLite within CPAN.pm as large as without it. By calling setup/update() via a shell command, this large memory requirement is only needed while the setup/update to the database is being done; when the shell process is over, this memory is returned to the system, thus resulting in a lower memory footprint afterwards of CPAN.pm. CPAN::SQLite::META::reload() does figure out where the database file is, using $CPAN::Config->{cpan_home} (which can be assumed to exist at this point). It does this just to determine if a journal file exists, which if present, indicates that another process is accessing the database, and so updates to the database shouldn't take place. This was done to accommodate multiple simultaneous CPAN.pm shells being possible in newer versions of CPAN.pm. setup/update() calls CPAN::SQLite::new(), which should find also the same database file; I realize this is partial duplication, but CPAN::SQLite::new() is more general, as it can handle usage outside of CPAN.pm as well. I think the problem lies in the fact that, within the stack trace you provided, there's CPAN::Shell::expand('CPAN::Shell', 'Author', 'LDS') called at t/05meta.t line 62 Removing existing /var/local/cpan_shell/cpandb.sql Gathering information from index files ... Populating database tables ... Done! at /Users/schwern/devel/CPAN-SQLite/lib/CPAN/SQLite/META.pm line 316 which indicates the database file /var/local/cpan_shell/cpandb.sql is being used. However, this should have been under DB<2> x $CPAN::Config->{cpan_home} 0 '/Users/schwern/devel/CPAN-SQLite/t/dot-cpan' One thing that I can think of that's causing this problem is that the setting of the PERL5LIB environment variable in t/05meta.t isn't somehow being recognized when it comes time to create the database file in this test. Can you print out the value of PERL5LIB at this point? Thanks very much again.

Sat Jan 12 21:24:32 2008 kwilliams [...] cpan.org - Correspondence added

I'm seeing the same problem, also on OS X. Is this likely to be a testing-only bug, or would it affect performance after installation too? -Ken

Sun Jan 13 00:35:43 2008 RKOBES [...] cpan.org - Correspondence added

From:

RKOBES [...] cpan.org

On Sat Jan 12 21:24:32 2008, KWILLIAMS wrote: Show quoted text

> I'm seeing the same problem, also on OS X. Is this likely to be a > testing-only bug, or would it > affect performance after installation too? > > -Ken >

I'm not sure if this is just the testing environment, or if it'll also occur after installation. A couple weeks ago I looked at this on my daughter's OS X; it had some issues separate from this, but after resolving those, all tests passed. I have some ideas what might be causing this; I'll try to get access to a machine that reproduces this problem to try them out.

Sun Jan 13 12:30:58 2008 schwern [...] pobox.com - Correspondence added

Subject:	Re: [rt.cpan.org #28890] 05meta.t fails "no such table: auths(1)" on OS X
Date:	Sun, 13 Jan 2008 09:30:16 -0800
To:	bug-CPAN-SQLite [...] rt.cpan.org
From:	Michael G Schwern <schwern [...] pobox.com>

RKOBES via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=28890 > > > On Sat Jan 12 21:24:32 2008, KWILLIAMS wrote:

>> I'm seeing the same problem, also on OS X. Is this likely to be a >> testing-only bug, or would it >> affect performance after installation too? >> >> -Ken >>

> > I'm not sure if this is just the testing environment, or if it'll also > occur after installation. A couple weeks ago I looked at this on my > daughter's OS X; it had some issues separate from this, but after > resolving those, all tests passed. > > I have some ideas what might be causing this; I'll try to get access to > a machine that reproduces this problem to try them out.

You still have a login on windhund.schwern.org... just as soon as me and my router have a polite conversation about how port forwarding is supposed to work. /me gets the "debugging stick" -- 31. Not allowed to let sock puppets take responsibility for any of my actions. -- The 213 Things Skippy Is No Longer Allowed To Do In The U.S. Army http://skippyslist.com/?page_id=3

Thu Jan 17 21:52:31 2008 RKOBES [...] cpan.org - Correspondence added

On Sun Jan 13 12:30:58 2008, schwern@pobox.com wrote: Show quoted text

> You still have a login on windhund.schwern.org... just as > soon as me and my > router have a polite conversation about how port forwarding > is supposed to > work. /me gets the "debugging stick"

Thanks very much for that. Unfortunately, I still get a "host unreachable" error when trying to ssh in.

Sun Feb 03 19:39:59 2008 RKOBES [...] cpan.org - Correspondence added

Thanks to these comments, and also David Wheeler, I think this has been resolved - the included CPAN::MyConfig file needed for the tests wasn't getting picked up on OS X through PERL5LIB. This (hopefully) has been fixed in version 0.19, which has just been uploaded to CPAN.

Sun Feb 03 19:40:11 2008 RKOBES [...] cpan.org - Status changed from 'open' to 'resolved'