Bug #76044 for Digest-combined: implement Digest::combined

Mon Mar 26 05:04:30 2012 oneingray [...] gmail.com - Ticket created

CC:	Ivan Shmakov <oneingray [...] gmail.com>
Subject:	implement Digest::SHA::Combined
Date:	Mon, 26 Mar 2012 16:04:15 +0700
To:	bug-Digest-SHA [...] rt.cpan.org
From:	Ivan Shmakov <oneingray [...] gmail.com>

For my application, I need to compute both SHA-256 and SHA-1 digests at the same time. I've ended up writing a Digest::SHA::combined (which currently lives embedded into [1]) wrapper package, which I feel may be useful to the community, and now considering releasing it via CPAN (as a public domain library.) However, I wonder whether it could be considered for inclusion into the Digest-SHA distribution instead? TIA. Background Certain catalogues and protocols out there allow (or require) one to list several digests for a single file (or, rather, an octet sequence), provided that these are of different “kinds.” Consider, e. g., the magnet: URI schema [2], or Debian's “Package” database: Package: beep ... Filename: pool/main/b/beep/beep_1.2.2-22_amd64.deb Size: 24036 MD5sum: dec6eb5a0eb38f4ac85e24d653c01916 SHA1: 15df36acc29d696c91cf432986e3bbd99761eada SHA256: 869fc8d7d8e3d0cba191ea430e8a32426cc29efeb54e0b55af49c3fea90cddf0 Having a shortcut for such computations could simplify the code dealing with those catalogues. [1] http://gray.am-1.org/~ivan/archives/git/gitweb.cgi?p=fc-2012.git;a=blob;f=src/fc-scan.perl;h=2da2cb6845afdad3b696a2492a2b2a9d8fe3c8ac;hb=ebbe7f9576da3d8a2745f60001c80f93e02b0047 [2] http://en.wikipedia.org/wiki/Magnet_URI_scheme -- FSF associate member #7257

Mon Mar 26 05:20:39 2012 ONEGRAY [...] cpan.org - Severity Wishlist added

Mon Mar 26 06:23:30 2012 mshelor [...] cpan.org - Correspondence added 30 min

RT-Send-CC:

gisle [...] ActiveState.com

Hello Ivan, I've studied both your proposal and the fc-scan code, and understand what you seek: namely, the added convenience of calculating and displaying multiple digest values from a single input stream. However, I oppose the inclusion of such a feature into Digest::SHA for two reasons: 1. It further complicates an already-large and complex module, and doesn't introduce any efficiencies that can't be obtained at the application layer. Moreover, displaying combined digest values is really a matter of presentation, and doesn't have anything to do with digest computation, which is the main business of Digest::SHA. 2. If the displaying of multiple digest values becomes increasingly common or standard in the future, then there's no reason to restrict this capability to Digest::SHA. Rather, it would be most sensibly placed in the CPAN Digest module so that the MD family and other algorithms would be included as well. Accordingly, I've copied this reply to Gisle Aas, the author of the Digest module. Regards, Mark

Mon Mar 26 06:23:34 2012 The RT System itself - Status changed from 'new' to 'open'

Mon Mar 26 06:23:35 2012 mshelor [...] cpan.org - Status changed from 'open' to 'rejected'

Mon Mar 26 07:33:13 2012 ONEGRAY [...] cpan.org - Cc ONEGRAY added

Mon Mar 26 07:34:48 2012 ONEGRAY [...] cpan.org - Cc ONEGRAY deleted

Mon Mar 26 07:34:49 2012 ONEGRAY [...] cpan.org - Requestor ONEGRAY added

Mon Mar 26 07:35:10 2012 ONEGRAY [...] cpan.org - Cc gisle@ActiveState.com added

Mon Mar 26 07:48:25 2012 oneingray [...] gmail.com - Correspondence added

CC:	Ivan Shmakov <oneingray [...] gmail.com>
Subject:	Re: [rt.cpan.org #76044] implement Digest::combined
Date:	Mon, 26 Mar 2012 18:48:12 +0700
To:	bug-Digest-SHA [...] rt.cpan.org
From:	Ivan Shmakov <oneingray [...] gmail.com>

Show quoted text

>>>>> Mark Shelor via RT <bug-Digest-SHA@rt.cpan.org> writes:

Show quoted text

> I've studied both your proposal and the fc-scan code, and understand > what you seek: namely, the added convenience of calculating and > displaying multiple digest values from a single input stream.

(More like “recording” than “displaying”, actually.) Show quoted text

> However, I oppose the inclusion of such a feature into Digest::SHA > for two reasons:

Show quoted text

> 1. It further complicates an already-large and complex module, and > doesn't introduce any efficiencies that can't be obtained at the > application layer. Moreover, displaying combined digest values is > really a matter of presentation, and doesn't have anything to do with > digest computation, which is the main business of Digest::SHA.

Agreed. Thanks. Show quoted text

> 2. If the displaying of multiple digest values becomes increasingly > common or standard in the future, then there's no reason to restrict > this capability to Digest::SHA. Rather, it would be most sensibly > placed in the CPAN Digest module so that the MD family and other > algorithms would be included as well. Accordingly, I've copied this > reply to Gisle Aas, the author of the Digest module.

I've added him to the Cc: list for the ticket, and have no objections to transferring this ticket to the Digest queue. (And re-opening it there.) I'd also note that the module being discussed should provide interfaces to obtain the computed digests as both one concatenated digest, /and/ the list of individual digests. TIA. -- FSF associate member #7257

Mon Mar 26 07:48:26 2012 The RT System itself - Status changed from 'rejected' to 'open'

Mon Mar 26 08:14:01 2012 mshelor [...] cpan.org - Correspondence added 10 min

RT-Send-CC:

oneingray [...] gmail.com, gisle [...] ActiveState.com

I certainly support your suggestion of transferring the ticket to the Digest queue rather than continuing to reactivate it under Digest::SHA. If Gisle is amenable to the idea of expanding or modifying the Digest interface to accommodate multiple digests, I'll be happy to contribute to any further discussions along those lines. Mark

Mon Mar 26 08:14:02 2012 mshelor [...] cpan.org - Status changed from 'open' to 'rejected'

Mon Mar 26 08:38:18 2012 ONEGRAY [...] cpan.org - Subject changed from 'implement Digest::SHA::Combined ' to 'implement Digest::combined'

Mon Mar 26 08:38:18 2012 ONEGRAY [...] cpan.org - Status changed from 'rejected' to 'open'

Mon Mar 26 08:38:18 2012 ONEGRAY [...] cpan.org - Queue changed from Digest-SHA to Digest

Mon Mar 26 08:39:38 2012 ONEGRAY [...] cpan.org - Cc MSHELOR added

Mon Mar 26 15:35:18 2012 GAAS [...] cpan.org - Correspondence added

My suggestion would be to make this it's own module and just upload it to CPAN. This does not appear to be a good fit for the Digest:: base dist. Here is my implementation:

package Digest::combined;

use strict;

use base 'Digest::base';

use Digest ();

sub new {

my $class = shift;

my $self = bless [ map Digest->new(ref($_) ? @$_ : $_), @_ ], $class;

return $self;

}

sub reset {

my $self = shift;

for (@$self) {

$_->reset;

}

sub clone {

my $self = shift;

bless [ map $_->clone, @$self ], ref($self);

}

sub add {

my $self = shift;

for (@$self) {

$_->add(@_);

}

sub digests {

my $self = shift;

die unless wantarray;

map $_->digest, @$self;

}

sub digest {

my $self = shift;

my $separator = shift || "";

return join($separator, $self->digests);

}

1;

Mon Mar 26 19:36:44 2012 mshelor [...] cpan.org - Correspondence added

RT-Send-CC:

oneingray [...] gmail.com, gisle [...] ActiveState.com

On Mon Mar 26 15:35:18 2012, GAAS wrote: Show quoted text

> My suggestion would be to make this it's own module and just upload it > to CPAN. > This does not appear to be a good fit for the Digest:: base dist. Here > is my > implementation:

FWIW that looks like the most effective solution. The implementation you give is clean and compact. I can't see any obvious way to generalize it. Initially I balked at the idea of an empty string as the default digest separator, but on reflection that's the sensible choice for binary fixed-length data. Mark

Tue Mar 27 06:43:01 2012 oneingray [...] gmail.com - Correspondence added

CC:	306914 [...] bugs.debian.org, Ivan Shmakov <oneingray [...] gmail.com>
Subject:	[rt.cpan.org #76044] IPv6 support? (patch)
Date:	Tue, 27 Mar 2012 17:42:47 +0700
To:	bug-libwww-perl [...] rt.cpan.org
From:	Ivan Shmakov <oneingray [...] gmail.com>

The way I've dealt with the problem [1] is by requiring IO::Socket::INET6 first, while still allowing the code to fall back to IO::Socket::INET. (Just like Net::HTTP already falls back to IO::Socket if IO::Socket::INET is itself unavailable.) The patch I'm using is MIME'd. [1] http://permalink.gmane.org/gmane.comp.lang.perl.modules.lwp/2556 -- FSF associate member #7257

--- HTTP.pm.~1~ 2012-02-17 03:17:26.000000000 +0700 +++ HTTP.pm 2012-03-27 17:05:37.000000000 +0700 @@ -5,8 +5,13 @@ $VERSION = "6.03"; unless ($SOCKET_CLASS) { - eval { require IO::Socket::INET } || require IO::Socket; - $SOCKET_CLASS = "IO::Socket::INET"; + if (eval { require IO::Socket::INET6 }) { + $SOCKET_CLASS = "IO::Socket::INET6"; + } else { + eval { require IO::Socket::INET } + || require IO::Socket; + $SOCKET_CLASS = "IO::Socket::INET"; + } } require Net::HTTP::Methods; require Carp;

Thu Mar 29 11:59:54 2012 oneingray [...] gmail.com - Correspondence added

CC:	Ivan Shmakov <oneingray [...] gmail.com>
Subject:	Re: [rt.cpan.org #76044] IPv6 support? (patch)
Date:	Thu, 29 Mar 2012 22:59:44 +0700
To:	bug-libwww-perl [...] rt.cpan.org
From:	Ivan Shmakov <oneingray [...] gmail.com>

Show quoted text

>>>>> Ivan Shmakov <oneingray@gmail.com> writes:

Show quoted text

> The way I've dealt with the problem [1] is by requiring > IO::Socket::INET6 first,

[…] Of course, this one was intended for #29468, not #76044. Resent it there. My apologies for the inconvenience. PS. Will get back to Digest::combined in a few days. -- FSF associate member #7257

Tue Apr 03 02:00:36 2012 oneingray [...] gmail.com - Correspondence added

CC:	Ivan Shmakov <oneingray [...] gmail.com>
Subject:	Re: [rt.cpan.org #76044] implement Digest::combined
Date:	Tue, 03 Apr 2012 13:00:19 +0700
To:	bug-Digest [...] rt.cpan.org
From:	Ivan Shmakov <oneingray [...] gmail.com>

Show quoted text

>>>>> Gisle Aas via RT <bug-Digest@rt.cpan.org> writes:

Show quoted text

> My suggestion would be to make this it's own module and just upload > it to CPAN. This does not appear to be a good fit for the Digest:: > base dist.

ACK, thanks! Do I understand it correctly that the convention for the packages that don't supply any actual digest algorithms is to be named in all-lowercase (after the Digest:: prefix)? Show quoted text

> Here is my implementation:

[…] Show quoted text

> sub new { > my $class = shift; > my $self > = bless [ map Digest->new (ref ($_) ? @$_ : $_), @_ ], $class;

There, the code relies on that either Digest->new () or Digest->IMPLEMENTATION->new () will die () should there be a problem. However, while Digest->new () will do just that should no implementation be found, it isn't actually specified in the documentation (and neither it will die () should the implementation's own ->new () return undef.) Therefore, I'd prefer to have an explicit “or die” both here and in ->clone (). OTOH, I guess that it's reasonably safe to assume that the ->reset (), ->add (), and ->digest () methods will never fail. Show quoted text

> return $self; > }

[…] Show quoted text

> sub digest { > my $self = shift; > my $separator = shift || "";

Shouldn't it rather read “shift // ""”? For the unlikely case that one may want to separate the digests with 0's. OTOH, there doesn't seem to be much use for any $separator other than an empty string here. (As opposed to ->hexdigest () and ->b64digest (). Should these be implemented, BTW?) Show quoted text

> return join ($separator, $self->digests); > }

[…] -- FSF associate member #7257

Mon Apr 09 03:41:44 2012 ONEGRAY [...] cpan.org - Queue changed from Digest to Digest-combined

Mon Apr 09 03:41:58 2012 ONEGRAY [...] cpan.org - Taken

Mon Apr 09 05:05:14 2012 oneingray [...] gmail.com - Correspondence added

CC:	Ivan Shmakov <oneingray [...] gmail.com>
Subject:	Re: [rt.cpan.org #76044] implement Digest::combined
Date:	Mon, 09 Apr 2012 16:04:59 +0700
To:	bug-Digest [...] rt.cpan.org
From:	Ivan Shmakov <oneingray [...] gmail.com>

Show quoted text

>>>>> Gisle Aas via RT <bug-Digest@rt.cpan.org> writes:

[Moved the ticket to Digest-combined and taken.] Show quoted text

> My suggestion would be to make this it's own module and just upload > it to CPAN. This does not appear to be a good fit for the Digest:: > base dist. Here is my implementation:

[…] I was a bit eager to release 0.1 (e. g., [1]), and I've used this implementation as the basis. May it be considered as being in the public domain? (which is the “license” I've chosen for the whole package.) My point is that this code is hardly worth being protected by copyright laws, and may be uncopyrightable (at least in certain jurisdictions) at all. TIA. I've briefly tested the code (it installs via cpan(1) and hashes the test strings correctly), and intend to close this ticket shortly, unless there'd be any objections. [1] http://search.cpan.org/perldoc?Digest::combined -- FSF associate member #7257

Mon Apr 09 14:46:15 2012 GAAS [...] cpan.org - Correspondence added

I have no problem with the license or the way you published it. You are herby granted the permission to release the code I provided above as public domain or to republish it under the Perl license (the license I usually prefer).

Mon Apr 09 14:57:29 2012 oneingray [...] gmail.com - Correspondence added

CC:	Ivan Shmakov <oneingray [...] gmail.com>
Subject:	Re: [rt.cpan.org #76044] implement Digest::combined
Date:	Tue, 10 Apr 2012 01:57:19 +0700
To:	bug-Digest-combined [...] rt.cpan.org
From:	Ivan Shmakov <oneingray [...] gmail.com>

Show quoted text

>>>>> Gisle Aas via RT <bug-Digest-combined@rt.cpan.org> writes:

Show quoted text

> I have no problem with the license or the way you published it. You > are herby granted the permission to release the code I provided above > as public domain or to republish it under the Perl license (the > license I usually prefer).

ACK. Thanks! While the package still has a few minor issues, I believe that the now released package fulfills the requirements, and thus closing the ticket. -- FSF associate member #7257

Mon Apr 09 14:59:30 2012 ONEGRAY [...] cpan.org - Status changed from 'open' to 'resolved'

Mon Apr 09 14:59:30 2012 The RT System itself - System error

Sending the previous mail has failed. Please contact your admin, they can find more details in the logs.