Skip Menu |

This queue is for tickets about the Test-Database CPAN distribution.

Report information
The Basics
Id: 40106
Status: resolved
Priority: 0/
Queue: Test-Database

People
Owner: Nobody in particular
Requestors: MLAWREN [...] cpan.org
Cc:
AdminCc:

Bug Information
Severity: Normal
Broken in: (no value)
Fixed in: (no value)



Subject: Temporary or cleaned databases should be the normal mode
My database tests typically require a fresh database, at a minimum for the very first test, not necessarily for the rest. However Test::Database by default always uses the same database which may be 'dirty' from the last run, or dirty from a previous Test::Database user within another modules tests. Test::Database (or perhaps the individual Drivers) should have a 'reset' or 'clean' method to make sure that the database is empty. Alternatively, the _default_ case could always return a fresh database. I think for SQLite this is easily done (using File::Temp from my earlier bug report). Or, the default case could return a database with a name based on the distribution/module being tested. For other database types I guess it is just a question of the right permissions, but also how to clean up possibly many new databases. Cheers, Mark. -- Mark Lawrence
Just thinking this through a bit more, I think the documentation kind of already specifies what I want, but doesn't actually do that in practice. How about the following suggestion for behaviour: * For a single test that creates a fresh fresh database, which is automatically cleaned up when the single test process exits (perhaps via an Test::Database::END block?): Test::Database->dbh('SQLite') For SQLite the cleanup is easy. For things like MySQL it becomes something like $dbh->do('DROP TABLE * CASCADE') or whatever the syntax is. * For a multiple tests, using named databases, make the cleanup a manual process: t/01-first-test.t Test::Database->reset('SQLite' => 'mydb') Test::Database->dbh('SQLite' => 'mydb') t/0?-middle-test.t Test::Database->dbh('SQLite' => 'mydb') t/09-last-test.t Test::Database->dbh('SQLite' => 'mydb') # the tests, and then END { Test::Database->reset('SQLite' => 'mydb') } For SQLite the once again reset() is easy. For things like MySQL it becomes something like $dbh->do('DROP DATABASE mydb') or whatever the syntax is. Cheers, Mark. -- Mark Lawrence
On Thu Oct 16 14:08:27 2008, MLAWREN wrote: Show quoted text
> My database tests typically require a fresh database, at a minimum for > the very first test, not necessarily for the rest. However > Test::Database by default always uses the same database which may be > 'dirty' from the last run, or dirty from a previous Test::Database user > within another modules tests. > > Test::Database (or perhaps the individual Drivers) should have a 'reset' > or 'clean' method to make sure that the database is empty.
The method is called cleanup(), and be called like this: Test::Database->cleanup(); # clean the whole directory Test::Database::Driver->cleanup; # same as above Test::Database::Driver::Foo->cleanup; # clean only the driver's directory Show quoted text
> Alternatively, the _default_ case could always return a fresh database. > I think for SQLite this is easily done (using File::Temp from my earlier > bug report). Or, the default case could return a database with a name > based on the distribution/module being tested.
In a test suite, if one needs a clean database at the beginning, they'll only need to ask for it in the first script, then all the others will use the one that was setup first. I think it's more practical that using the default in the first script, and then explicitely request to *not* clean the database in the following ones. Show quoted text
> For other database types I guess it is just a question of the right > permissions, but also how to clean up possibly many new databases. > > Cheers, > Mark.
On Thu Oct 16 14:32:53 2008, MLAWREN wrote: Show quoted text
> Just thinking this through a bit more, I think the documentation kind of > already specifies what I want, but doesn't actually do that in practice. > > How about the following suggestion for behaviour: > > * For a single test that creates a fresh fresh database, which is > automatically cleaned up when the single test process exits (perhaps via > an Test::Database::END block?): > > Test::Database->dbh('SQLite') > > For SQLite the cleanup is easy. For things like MySQL it becomes > something like $dbh->do('DROP TABLE * CASCADE') or whatever the syntax is. >
In my opinion, the easiest way to cleanup the database that were fully created by Test::Database is to just drop the directory... Oh well, you're right: with SQLite, I'll drop every database. So maybe we need a way to drop a single database, by name. Show quoted text
> * For a multiple tests, using named databases, make the cleanup a manual > process: > > t/01-first-test.t > Test::Database->reset('SQLite' => 'mydb') > Test::Database->dbh('SQLite' => 'mydb') > > t/0?-middle-test.t > Test::Database->dbh('SQLite' => 'mydb') > > t/09-last-test.t > Test::Database->dbh('SQLite' => 'mydb') > # the tests, and then > END { > Test::Database->reset('SQLite' => 'mydb') > } > > For SQLite the once again reset() is easy. For things like MySQL it > becomes something like $dbh->do('DROP DATABASE mydb') or whatever the > syntax is.
Actually, I think I'm going to discourage using named databases. Here's the rationale: - some databases are much harder to setup than SQLite or MySQL, and the only way to reasonably provide one to modules for their test suite is by having the machine owner (CPAN tester?) configure the available databases - therefore I plan to add very soon the support for preconfigured databases: either via an environment variable (TEST_DSN, TEST_DSN\d+, DBI_DSN, etc), or via explicit configuration (Test::Database::Config, probably) In this cases, one cannot know the name of the databases in advance, and dropping them would not be very nice (of course, dropping all the tables is accepted). It also makes the original goal of the module (provide a standardized way for test script to obtain a dbh to run their test suites in) more difficult to achieve. (But see my reply to #40104 for a more elaborate way to handle those cases.) Thank you very much for your input, I appreciate it a lot. I'll write down the use cases, and we'll continue via email. -- BooK
Subject: Re: [rt.cpan.org #40106] Temporary or cleaned databases should be the normal mode
Date: Fri, 17 Oct 2008 23:57:29 +0200
To: Philippe 'BooK' Bruhat via RT <bug-Test-Database [...] rt.cpan.org>
From: Mark Lawrence <nomad [...] null.net>
Show quoted text
> > Test::Database (or perhaps the individual Drivers) should have a 'reset' > > or 'clean' method to make sure that the database is empty.
> > The method is called cleanup(), and be called like this: > > Test::Database->cleanup(); # clean the whole directory > Test::Database::Driver->cleanup; # same as above > Test::Database::Driver::Foo->cleanup; # clean only the driver's > directory
Woops. Sorry, not reading the entire documentation before posting. But I think we both came to the same conclusion about it not making sense to have a global ->cleanup method. Show quoted text
> In a test suite, if one needs a clean database at the beginning, they'll > only need to ask for it in the first script, then all the others will > use the one that was setup first. > I think it's more practical that using the default in the first script, > and then explicitely request to *not* clean the database in the > following ones.
You are right, I got this backwards. Show quoted text
> > For SQLite the cleanup is easy. For things like MySQL it becomes > > something like $dbh->do('DROP TABLE * CASCADE') or whatever the syntax is. > >
> > In my opinion, the easiest way to cleanup the database that were fully > created by Test::Database is to just drop the directory... Oh well, > you're right: with SQLite, I'll drop every database. So maybe we need > a way to drop a single database, by name. > > ... > > Actually, I think I'm going to discourage using named databases. > > Here's the rationale: > - some databases are much harder to setup than SQLite or MySQL, and the > only way to reasonably provide one to modules for their test suite is by > having the machine owner (CPAN tester?) configure the available databases
I agree. But the same rationale says to me that start_server and stop_server in Test::Database::Driver also shouldn't be part of this distribution. The starting and stopping of database processes is very operating system dependent, and not something that can easily be defined in the (to be determined Test::Database::Config) configuration file. I know you've said you'll write down the use cases and reply, but I'm in a different time zone so I'll raise the question now :-) What should the interface between the various components (Author, Test::Database, Tester) be? I think it goes like this: Portable SQL Test Writer ------------------------- If the SQL is very simple and likely to be extrememly portable, and the author wants to check against everything available then she doesn't need to specify any specific drivers: # Case 1 foreach my $driver (Test::Database->drivers) { # tests } Non-Portable SQL Test Writer ---------------------------- Author knows (or wants) that their code will only work against a specific subset of drivers, so they are specified: # Case 2 foreach my $driver (Test::Database->drivers('SQLite', 'mysql') { # tests } Perhaps they want to include drivers specific to their local setup: # Case 3 foreach my $driver (Test::Database->drivers('mysql', 'mysql4') { # tests } Perhaps they could specify negation instead: # Case 4 foreach my $driver (Test::Database->drivers('!DBM') { # tests } Test::Database -------------- At "use Test::Database" an internal drivers configuration is built from: 1. File-system based installed DBD::* over-written with: 2. Intersection of $HOME/.Test-Database and installed DBD::* Default Case ------------ The result is something like this for the default tester case: %drivers = ( 'DBM' => { driver => 'DBM', dsn => 'dbi::DBM:t/Test-Database/DBM/DBM', user => '', pass => '', }, 'SQLite' => { driver => 'SQLite', dsn => 'dbi::SQLite:t/Test-Database/SQLite/SQLite.db', user => '', pass => '', }, ); Power Tester ------------ The result is something like this for tester who was good enough to decide that they'll run MySQL as well. $ cat $HOME/.Test-Database name = mysql dsn = dbi::mysql:databse=test user = myuser pass = mypass %drivers = ( 'DBM' => { driver => 'DBM', dsn => 'dbi::DBM:t/Test-Database/DBM/DBM', user => '', pass => '', }, 'SQLite' => { driver => 'SQLite', dsn => 'dbi::SQLite:t/Test-Database/SQLite/SQLite.db', user => '', pass => '', }, 'mysql' => { driver => 'mysql', dsn => 'dbi::mysql:databse=test', user => 'myuser', pass => 'mypass', }, ); Module Author ------------- The result is something like this for module author who runs MySQL version 4 on port 3307 in addition to the latest MySQL on the default port: $ cat $HOME/.Test-Database name = mysql dsn = dbi::mysql:databse=test user = myuser pass = mypass name = mysql4 dsn = dbi::mysql:databse=test;port=3307 user = myolduser pass = myoldpass %drivers = ( 'DBM' => { driver => 'DBM', dsn => 'dbi::DBM:t/Test-Database/DBM/DBM', user => '', pass => '', }, 'SQLite' => { driver => 'SQLite', dsn => 'dbi::SQLite:t/Test-Database/SQLite/SQLite.db', user => '', pass => '', }, 'mysql' => { driver => 'mysql', dsn => 'dbi::mysql:databse=test', user => 'myuser', pass => 'mypass', }, 'mysql4' => { driver => 'mysql', dsn => 'dbi::mysql:databse=test;port=3307', user => 'myolduser', pass => 'myoldpass', }, ); Testing ------- The result of Test::Database->drivers(@list) is clearly then the intersection of "@list" with "keys %drivers" when @list is defined, and "keys %drivers" otherwise. Design Thoughts --------------- There are two models: file-system based databases and non-file-system based. If the interface between Test::Database and databases was strictly limited to DBI/DBD functionality, and we ignore the whole named database issue, then the entire Driver/* tree could be collapsed into two, maybe even one, module. package Test::Database; # static list of predefined file-based DSNs %file_drivers = ( DBM => { file_based => 1, driver => 'DBM', dsn => 'dbi:DBM:f_dir=t/Test-Database/DBM', ... }, SQLite => { file_based => 1, driver => 'SQLite', dsn => 'dbi:SQLite:t/Test-Database/DBM', ... } ); # Dynamically generated at 'use' %drivers = ( %file_drivers + $HOME/.Test-Database definitions, ) sub drivers { my @return = (); my @candidates = map {$drivers{$_}} @_; foreach my $c (@candidates) { if (my $driver = Test::Database::Driver->new($c)) { push(@return, $driver); } } return @return; } And the above would work with: package Test::Database::Driver; sub new { my $proto = shift; my $class = ref($proto) || $proto; my $self = shift; # Doing this here saves Test::Database startup time? Maybe not. if (!eval "require DBD::$self->{driver}; 1;") { return; } if ($self->{file_based}) { # mkdir t/Test-Database/$self->{driver} } # croak now if this fails? $self->{dbh} = DBI->connect( $self->{dsn}, $self->{username}, $self->{password} ); return bless($self,$class); } sub cleanup { my $self = shift; if ($self->{file_based}) { $self->{dbh}->disconnect(); # rm -rf t/Test-Database/$self->{driver} # mkdir t/Test-Database/$self->{driver} exists $self->{dbh} = DBI->connect( $self->{dsn}, $self->{username}, $self->{password} ); } else { # DROP TABLE * # or maybe we do need Driver::<driver>. How do we find the # list of tables in the database? Does DBI provide such # functions? Otherwise $self->_cleanup(); } } sub connection_info { my $self = shift; return ($self->{dsn}, $self->{username}, $self->{password}); } sub dbh { my $self = shift; return $self->{dbh}; } Which then enables the following, which is more intuitive to me: foreach my $driver (Test::Database->drivers('SQLite', 'Pg') { SQL::DB->connect($driver->connection_info); # or SQL::DB->set_dbh($driver->dbh); ok(...., 'test name') } Show quoted text
> Thank you very much for your input, I appreciate it a lot.
I hope I haven't gone too far with this mail. However regardless of the outcome, the effort is gladly given, because I will definately appreciate Test::Database when it's finished! Cheers, Mark. -- Mark Lawrence
Subject: Re: [rt.cpan.org #40106] Temporary or cleaned databases should be the normal mode
Date: Sat, 18 Oct 2008 00:05:58 +0200
To: Philippe 'BooK' Bruhat via RT <bug-Test-Database [...] rt.cpan.org>
From: Mark Lawrence <nomad [...] null.net>
On Fri Oct 17, 2008 at 11:57:29PM +0200, Mark Lawrence wrote: Show quoted text
> Which then enables the following, which is more intuitive to me: > > foreach my $driver (Test::Database->drivers('SQLite', 'Pg') { > SQL::DB->connect($driver->connection_info); > # or > SQL::DB->set_dbh($driver->dbh); > > ok(...., 'test name') > }
I just thought of something else. The SYNOPSIS for Test::Database should actually be something like this, in the interests of making a good test plan: use Test::More; use Test::Database; my @drivers = Test::Database->drivers('SQLite', 'Pg'); plan tests => @drivers * 3; foreach my $driver (@drivers) { ok(....); # test 1 ok(....); # test 2 ok(....); # test 3 } Cheers, Mark. -- Mark Lawrence
Subject: Re: [rt.cpan.org #40106] Temporary or cleaned databases should be the normal mode
Date: Sat, 18 Oct 2008 00:49:53 +0200
To: Philippe 'BooK' Bruhat via RT <bug-Test-Database [...] rt.cpan.org>
From: Mark Lawrence <nomad [...] null.net>
Show quoted text
> use Test::More; > use Test::Database; > > my @drivers = Test::Database->drivers('SQLite', 'Pg'); > plan tests => @drivers * 3; > > foreach my $driver (@drivers) { > ok(....); # test 1 > ok(....); # test 2 > ok(....); # test 3 > } >
Sorry, one better would be this, to allow for database independent tests: use Test::More; use Test::Database; my @drivers = Test::Database->drivers('SQLite', 'Pg'); plan tests => 2 + (@drivers * 3); ok(...); # test a; ok(...); # test b; foreach my $driver (@drivers) { ok(....); # test 1 ok(....); # test 2 ok(....); # test 3 } Cheers, Mark. -- Mark Lawrence

Message body is not shown because it is too large.