Bug #20244 for Python-Serialise-Pickle: Cannot Handle Mailman Data File config.pck

Sun Jul 02 14:23:54 2006 imacat [...] cpan.org - Ticket created

Subject:

Cannot Handle Mailman Data File config.pck

=================== Hi. This is imacat from Taiwan. I tried to read Mailman database with Python::Serialise::Pickle tonight, and found an error. This is what I got: imacat@rinse /tmp % cat /tmp/test.pl #! /usr/bin/perl -w use Python::Serialise::Pickle; use Data::Dumper; my ($pr, $data); $pr = Python::Serialise::Pickle->new("/var/lib/mailman/lists/mailman/config.pck"); while ($data = $pr->load) { print Data::Dumper->Dump([$data], [qw($data)]); } $pr->close; __END__ imacat@rinse /tmp % /tmp/test.pl LOAD C=} We have no handler to deal with '}' imacat@rinse /tmp % It seems that Python::Serialise::Pickle cannot handle a certain data type. Could this be fixed? Thank you.

Mon Jul 03 06:49:22 2006 simon [...] thegestalt.org - Correspondence added

CC:	undisclosed-recipients: ;
Subject:	Re: [rt.cpan.org #20244] Cannot Handle Mailman Data File config.pck
Date:	Mon, 3 Jul 2006 11:48:58 +0100
To:	via RT <bug-Python-Serialise-Pickle [...] rt.cpan.org>
From:	Simon Wistow <simon [...] thegestalt.org>

On Sun, Jul 02, 2006 at 02:23:54PM -0400, via RT said: Show quoted text

> Hi. This is imacat from Taiwan. I tried to read Mailman database > with Python::Serialise::Pickle tonight, and found an error. This is > what I got:

The Python pickle format has changed and I haven't had time to sit down and parse the new format. A work around might be to use Python to dump out a YAML file and then use Perl to read that back in. Apologies, Simon

Mon Jul 03 06:49:25 2006 The RT System itself - Status changed from 'new' to 'open'

Mon Jul 03 07:04:56 2006 imacat [...] cpan.org - Correspondence added

From:

imacat [...] mail.imacat.idv.tw

On 2006-07-03 06:49:22 星期一, simon@thegestalt.org wrote: Show quoted text

> On Sun, Jul 02, 2006 at 02:23:54PM -0400, via RT said:

> > Hi. This is imacat from Taiwan. I tried to read Mailman database > > with Python::Serialise::Pickle tonight, and found an error. This is > > what I got:

> The Python pickle format has changed and I haven't had time to sit down > and parse the new format. > A work around might be to use Python to dump out a YAML file and then > use Perl to read that back in.

Well... If I can do that, I would code my application in pure Python and need no Python::Serialise::Pickle at all. ^^; Thank you for this anyway. Currently I'm wrapping the output of mailman scripts. It's quite stupid and inefficient. I'm waiting for your new Python::Serialise::Pickle. Please tell me if there is anything I can help.

Sat Jan 05 13:36:08 2008 hjp [...] hjp.at - Correspondence added

From:

hjp [...] hjp.at

On Mon Jul 03 06:49:22 2006, simon@thegestalt.org wrote: Show quoted text

> On Sun, Jul 02, 2006 at 02:23:54PM -0400, via RT said:

> > Hi. This is imacat from Taiwan. I tried to read Mailman database > > with Python::Serialise::Pickle tonight, and found an error. This is > > what I got:

> > The Python pickle format has changed and I haven't had time to sit down > and parse the new format.

I ran into the same problem this week and implemented enough of protocol version 1 to be able to parse the Mailman config files I have. It's still not even a complete implementation of version 1 (not to mention version 2), it's quite buggy, undocumented and missing test cases. But if you are interested I'll clean it up and send you the patches. hp

Sun Jan 06 22:49:05 2008 hjp [...] hjp.at - Correspondence added

From:

hjp [...] hjp.at

On Sat Jan 05 13:36:08 2008, hjp@hjp.at wrote: Show quoted text

> I ran into the same problem this week and implemented enough of protocol > version 1 to be able to parse the Mailman config files I have. It's > still not even a complete implementation of version 1 (not to mention > version 2), it's quite buggy, undocumented and missing test cases. But > if you are interested I'll clean it up and send you the patches.

Well, I added tests for all the new opcodes, and fixed some of the old ones. There may still be some bugs, but the tests all pass now and they cover all source lines, so I guess it's clean enough now. I am attaching the complete distribution because the patch would be larger.

Download Python-Serialise-Pickle-0.02.tar.gz
application/x-gzip 11.3k

Message body not shown because it is not plain text.

Tue Feb 26 01:33:54 2008 imacat [...] cpan.org - Correspondence added

From:

imacat [...] mail.imacat.idv.tw

On Sun Jan 06 22:49:05 2008, hjp@hjp.at wrote: Show quoted text

> Well, I added tests for all the new opcodes, and fixed some of the old > ones. There may still be some bugs, but the tests all pass now and they > cover all source lines, so I guess it's clean enough now.

Dear Peter, Hi. This is imacat from Taiwan. Sorry for the reply late. Well, your test suites fails. 1. That t/tmp is read-only after untarring the source. I did a "chmod +w t/tmp" and things work. Module::Build changes every source file to read-only before "./Build dist". You can either use ExtUtils::MakeMaker instead, or use another file name for your purpose. I would suggest you to untar your "./Build dist" source and test again before release. Several unexpected things happens during "./Build dist", in my experience. 2. That float comparison fails at t/09binnumbers.t. The failure is attached below: imacat@cotton tmp/Python-Serialise-Pickle-0.02 % perl -Iblib/lib t/09binnumbers.t 1..7 ok 1 - use Python::Serialise::Pickle; ok 2 LOAD ok 3 - binint1 LOAD ok 4 - binint2 LOAD ok 5 - binint (positive) LOAD ok 6 - binint (negative) LOAD not ok 7 - binfloat # Failed test 'binfloat' # at t/09binnumbers.t line 15. # got: '3.14000000000000012' # expected: '3.14' # Looks like you failed 1 test of 7. imacat@cotton tmp/Python-Serialise-Pickle-0.02 % I suppose this fails on x86_64 and "i386 compiled with uselongdouble". I do not remember how others solve this, but I suppose you can use: is(sprintf("%f", $ps->load) + 0, 3.14, "binfloat"); instead of is($ps->load, 3.14, "binfloat"); I suppose there must be wiser solution than this somewhere else. Please tell me if you need any more information, or if I could be of any help. Thank you.

Sun Mar 30 05:31:59 2008 hjp [...] hjp.at - Correspondence added

From:

hjp [...] hjp.at

On Tue Feb 26 01:33:54 2008, IMACAT wrote: Show quoted text

> On Sun Jan 06 22:49:05 2008, hjp@hjp.at wrote:

> > Well, I added tests for all the new opcodes, and fixed some of the old > > ones. There may still be some bugs, but the tests all pass now and they > > cover all source lines, so I guess it's clean enough now.

> > Hi. This is imacat from Taiwan. Sorry for the reply late.

No problem. I'm late, too (somehow I expected rt to send me mail if the ticket changes, but apparently it doesn't) and the Simon is late, too ;-). Show quoted text

> Well, your test suites fails. > > 1. That t/tmp is read-only after untarring the source.

It shouldn't be in source. Apparently I added it to the MANIFEST by mistake. (And the tests which create it should probably remove it when they are finished). Show quoted text

> I would suggest you to untar your "./Build dist" source and test > again before release.

Good advice. I'll keep it in mind. Show quoted text

> 2. That float comparison fails at t/09binnumbers.t. The failure is > attached below: > > imacat@cotton tmp/Python-Serialise-Pickle-0.02 % perl -Iblib/lib > t/09binnumbers.t > 1..7 > ok 1 - use Python::Serialise::Pickle; > ok 2 > LOAD > ok 3 - binint1 > LOAD > ok 4 - binint2 > LOAD > ok 5 - binint (positive) > LOAD > ok 6 - binint (negative) > LOAD > not ok 7 - binfloat > # Failed test 'binfloat' > # at t/09binnumbers.t line 15. > # got: '3.14000000000000012' > # expected: '3.14' > # Looks like you failed 1 test of 7. > imacat@cotton tmp/Python-Serialise-Pickle-0.02 % > > I suppose this fails on x86_64 and "i386 compiled with

uselongdouble". Yes, that's expected. The test was designed to check if the result is exact with 64 bit doubles. It cannot work with longer (or shorter) doubles. It could be skipped for other sizes, but there is a better way ... Show quoted text

> I do not remember how others solve this, but I suppose you can use: > > is(sprintf("%f", $ps->load) + 0, 3.14, "binfloat");

Nope, that doesn't check if the result is exactly what it should be. A better test would be is($ps->load, 3.140000000000000124344978758017532527446746826171875, "binfloat"); that should work with all possible floating point representations. hp

Fri Feb 13 01:50:17 2009 jpierce [...] cpan.org - Correspondence added

I would very much like to see 0.02 (or later) officially released. Reading works for me, but I'm not too keen on playing with writing until it's official.