Skip Menu |

Preferred bug tracker

Please visit the preferred bug tracker to report your issue.

This queue is for tickets about the Sereal-Encoder CPAN distribution.

Report information
The Basics
Id: 88178
Status: resolved
Priority: 0/
Queue: Sereal-Encoder
People
Owner: Nobody in particular
Requestors: zefram [...] fysh.org
Cc:
AdminCc:
Bug Information
Severity: (no value)
Broken in: (no value)
Fixed in: (no value)

History Show all quoted text
  Tue Aug 27 10:03:01 2013 zefram [...] fysh.org - Ticket created
Subject: test relies on shuffling
Date: Tue, 27 Aug 2013 15:02:47 +0100
To: bug-Sereal-Encoder [...] rt.cpan.org
From: Zefram <zefram [...] fysh.org>
Sereal-Encoder randomly fails its tests one time out of N, because the sort_keys test checks that operation without the sort_keys option actually generates different results for each of four runs on equivalent data structures. I haven't determined N either in theory or in practice, looks like N > 20. I've actually seen the test fail twice. It looks like the test tries to check whether it's actually got hashes using different orders; maybe the new hash randomisation is causing reordering that the test doesn't know about. Anyway, it's testing behaviour that isn't guaranteed. For my use I've disabled that test, patch attached. If you want to be cleverer you could compare several encoding runs and look for some proportion of outputs being distinct, rather than all being distinct, and you can tune the number of trials and the threshold to bring the probability of false failure below some target. Bigger picture: I question the value of supporting non-sort_keys operation. Sorting isn't that expensive. -zefram

Message body is not shown because sender requested not to inline it.

  Mon Sep 02 01:19:53 2013 smueller [...] cpan.org - Correspondence added
On Tue Aug 27 10:03:01 2013, zefram@fysh.org wrote: Show quoted text
> Sereal-Encoder randomly fails its tests one time out of N, because > the sort_keys test checks that operation without the sort_keys option > actually generates different results for each of four runs on equivalent > data structures.
[...] Indeed, I confirmed this on 5.19.1. I've applied your patch to skip the test and will try to make a release some time soon. Alas, we're also in the run-up for a Sereal protocol version 2, so that collides a bit. :( I'll do my best. Show quoted text
> Bigger picture: I question the value of supporting non-sort_keys > operation. Sorting isn't that expensive.
I don't agree. But the way it is now, you can have it your way and I can have mine. :) Thanks a lot for figuring this out and submitting the bug! --Steffen
  Mon Sep 02 01:19:53 2013 The RT System itself - Status changed from 'new' to 'open'
  Mon Sep 02 01:19:54 2013 smueller [...] cpan.org - Status changed from 'open' to 'patched'
  Mon Sep 02 01:50:37 2013 smueller [...] cpan.org - Correspondence added
Sereal::Encoder/Decoder version 0.37 just uploaded to PAUSE including your patch. Proper fix will have to wait for Yves to have time.
  Mon Sep 02 01:50:38 2013 smueller [...] cpan.org - Status changed from 'patched' to 'resolved'
  Mon Sep 02 07:06:30 2013 demerphq [...] gmail.com - Correspondence added
Subject: Re: [rt.cpan.org #88178] test relies on shuffling
Date: Mon, 2 Sep 2013 13:06:12 +0200
To: bug-Sereal-Encoder [...] rt.cpan.org
From: demerphq <demerphq [...] gmail.com>
Hi Zefram, just wanted to say any patches or feedback you have to Sereal would be much appreciated. Someone with your knowledge contributing is very useful. Yves On 27 August 2013 16:03, Zefram via RT <bug-Sereal-Encoder@rt.cpan.org> wrote: Show quoted text
> Tue Aug 27 10:03:01 2013: Request 88178 was acted upon. > Transaction: Ticket created by zefram@fysh.org > Queue: Sereal-Encoder > Subject: test relies on shuffling > Broken in: (no value) > Severity: (no value) > Owner: Nobody > Requestors: zefram@fysh.org > Status: new > Ticket <URL: https://rt.cpan.org/Ticket/Display.html?id=88178 > > > > Sereal-Encoder randomly fails its tests one time out of N, because > the sort_keys test checks that operation without the sort_keys option > actually generates different results for each of four runs on equivalent > data structures. I haven't determined N either in theory or in practice, > looks like N > 20. I've actually seen the test fail twice. > > It looks like the test tries to check whether it's actually got hashes > using different orders; maybe the new hash randomisation is causing > reordering that the test doesn't know about. Anyway, it's testing > behaviour that isn't guaranteed. For my use I've disabled that test, > patch attached. If you want to be cleverer you could compare several > encoding runs and look for some proportion of outputs being distinct, > rather than all being distinct, and you can tune the number of trials and > the threshold to bring the probability of false failure below some target. > > Bigger picture: I question the value of supporting non-sort_keys > operation. Sorting isn't that expensive. > > -zefram >
-- perl -Mre=debug -e "/just|another|perl|hacker/"
  Mon Sep 02 07:12:41 2013 zefram [...] fysh.org - Correspondence added
Subject: Re: [rt.cpan.org #88178] test relies on shuffling
Date: Mon, 2 Sep 2013 12:12:24 +0100
To: demerphq via RT <bug-Sereal-Encoder [...] rt.cpan.org>
From: Zefram <zefram [...] fysh.org>
demerphq via RT wrote: Show quoted text
>Hi Zefram, just wanted to say any patches or feedback you have to >Sereal would be much appreciated.
It's an interesting project, but I'm afraid I'm not really using Sereal. I only ran into the test failure because it's an indirect dependency of Dancer::Session::Cookie, which is used by an app at work that I don't work on but for which I manage the Perl build. So don't expect much from me. -zefram