Bug #18788 for DBD-SQLite: Fork a variant for distributable SQLite dbs

Wed Apr 19 12:38:36 2006 adamk [...] cpan.org - Ticket created

Subject:

Fork a variant for distributable SQLite dbs

The main DBD::SQLite dist keeps upgrading the bundled SQLite database. This is fine, except when you want to create a SQLite database for wide distribution as a container for some published data. I'm not sure how hard this would be, but it would be nice if there was a way to have a DBD::SQLiteV3 (or something) fork, that was the same as the main one, but bundled the oldest version of SQLite that the current version is capable of reading safely. So, I imagine it would be 3.0.0 or something? In any case, using this alternative version would mean you could use seperate drivers for 1. Generating distributable database files 2. Doing your normal stuff.

Mon May 08 09:09:45 2006 Guest - Correspondence added

From:

bart.lateur [...] pandora.be

It appears to me that using a driver of the generation of 3.3.0, should be fine: Version 3.3.0 can read and write database files created by prior versions of SQLite. But prior versions of SQLite will not be able to read or write databases created by Version 3.3.0 http://www.sqlite.org/formatchng.html

Mon May 08 09:09:46 2006 The RT System itself - Status changed from 'new' to 'open'

Mon May 08 10:06:14 2006 adam [...] phase-n.com - Correspondence added

Subject:	Re: [rt.cpan.org #18788] Fork a variant for distributable SQLite dbs
Date:	Tue, 09 May 2006 00:03:09 +1000
To:	bug-DBD-SQLite [...] rt.cpan.org
From:	Adam Kennedy <adam [...] phase-n.com>

Well, using the current 3.3.5 for example to generate published data, this means that someone with DBD::SQLite as recent as 1.09 may not be able to read the file. By using 3.0.0 (or nearish) it means all versions 1.00 or never can read it, as well as slightly older things in other languages. Adam K Guest via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=18788 > > > It appears to me that using a driver of the generation of 3.3.0, should > be fine: > > Version 3.3.0 can read and write database files created by prior > versions of SQLite. But prior versions of SQLite will not be able > to read or write databases created by Version 3.3.0 > > http://www.sqlite.org/formatchng.html

Mon May 08 16:07:35 2006 Guest - Correspondence added

From:

bart.lateur [...] pandora.be

On Mon May 08 10:06:14 2006, adam@phase-n.com wrote: Show quoted text

> Well, using the current 3.3.5 for example to generate published data, > this means that someone with DBD::SQLite as recent as 1.09 may not be > able to read the file.

Oh, right. We were talking about different things. I was thinking about what driver is safe to distribute with an app, so it can read people's databases. You're thinking what database version is safe to distribute to use with any module version. That is a serious problem with SQLite, IMO: the file format changes too often, and a newer version of the program isn't able to produce a database in a previous version. I'd rather thing of a data conversion tool, so all you have to do is convert the database to an older version before distributing it. But anyway, following the instructions for converting from SQLite2 to SQLite3, the text format of the dump should be usable for exchanging data.

Mon May 08 19:49:07 2006 adam [...] phase-n.com - Correspondence added

Subject:	Re: [rt.cpan.org #18788] Fork a variant for distributable SQLite dbs
Date:	Tue, 09 May 2006 09:44:57 +1000
To:	bug-DBD-SQLite [...] rt.cpan.org
From:	Adam Kennedy <adam [...] phase-n.com>

Hmm... using a text dump might just be good enough. Leave this bug open for now, and once the new DBIx::Class::Loader API turns up and we get a chance to experiment with doing SQLite published data I'll let you know how it goes. Only problem might be that some of the data sets being planned are huge, in the order of several hundred meg. I'm wondering if the load time for large data sets is going to be a concern, if there's a million or so records in a data set (and a few planned data sets are that big), at a notional 1,000 inserts a second that means a good 20 minutes of importing. There's also a smaller issue regarding bloat, but it's minor in comparison. I'll get back to you on the export file solution. Adam K Guest via RT wrote: Show quoted text

> <URL: http://rt.cpan.org/Ticket/Display.html?id=18788 > > > On Mon May 08 10:06:14 2006, adam@phase-n.com wrote:

>> Well, using the current 3.3.5 for example to generate published data, >> this means that someone with DBD::SQLite as recent as 1.09 may not be >> able to read the file.

> > Oh, right. We were talking about different things. I was thinking about > what driver is safe to distribute with an app, so it can read people's > databases. You're thinking what database version is safe to distribute > to use with any module version. > > That is a serious problem with SQLite, IMO: the file format changes too > often, and a newer version of the program isn't able to produce a > database in a previous version. > > I'd rather thing of a data conversion tool, so all you have to do is > convert the database to an older version before distributing it. But > anyway, following the instructions for converting from SQLite2 to > SQLite3, the text format of the dump should be usable for exchanging data.

Thu Apr 23 06:49:46 2009 adamk [...] cpan.org - Correspondence added

Closing my own bug, to reduce the bug count :)

Thu Apr 23 06:49:47 2009 adamk [...] cpan.org - Status changed from 'open' to 'resolved'