Skip Menu |

This queue is for tickets about the IO-Compress CPAN distribution.

Report information
The Basics
Id: 126005
Status: resolved
Priority: 0/
Queue: IO-Compress
People
Owner: Nobody in particular
Requestors:
Cc:
AdminCc:
Bug Information
Severity: Wishlist
Broken in: (no value)
Fixed in: (no value)
Attachments
IO-Compress-Zstd-master.zip

History Show all quoted text
  Sun Aug 05 08:58:17 2018 https://me.yahoo.com/howdidwegetherereally#f714d - Ticket created
Subject: Please add IO-Compress-Zstandard
Please add IO-Compress-Zstandard so that Any(Un)Compress can work on such files. If there's a description of what needs to be done to create such a module let me know and I see if I can help to do that.
  Sun Aug 05 16:44:44 2018 pmqs [...] cpan.org - Correspondence added
On Sun Aug 05 08:58:17 2018, https://me.yahoo.com/howdidwegetherereally#f714d wrote: Show quoted text
> Please add IO-Compress-Zstandard so that Any(Un)Compress can work on > such files. If there's a description of what needs to be done to > create such a module let me know and I see if I can help to do that.
Zstandard is on my list to keep an eye on. Haven't seen it used in the wild, or had anyone expressing interest in it (until now). Are you aware of anytihing that makes use of this format? Paul
  Sun Aug 05 16:44:44 2018 The RT System itself - Status changed from 'new' to 'open'
  Mon Aug 06 15:13:31 2018 https://me.yahoo.com/howdidwegetherereally#f714d - Correspondence added
On Sun Aug 05 16:44:44 2018, PMQS wrote: Show quoted text
> Zstandard is on my list to keep an eye on. Haven't seen it used in the > wild, or had anyone expressing interest in it (until now). > > Are you aware of anything that makes use of this format?
Btrfs uses it for on-disk compression (they even changed the licence to get it into the kernel) and it's moving quite fast into similar application areas (good compression at high speed and _really_ fast uncompression): http://facebook.github.io/zstd/#references I am mostly interested in getting something that compresses almost as good as LZMA (if a tad slower), but decompresses faster into memory than you could read it from external filers. That way, you essentially get the compression ratio as a speedup on read. The existing Perl binding is here: https://metacpan.org/release/Compress-Zstd It uses the streaming API and looks OK to me in general, but I really want to have it available via AnyUncompress in order to keep supporting all other formats without adding extra code.
  Tue Feb 19 08:56:27 2019 pmqs [...] cpan.org - Correspondence added
On Mon Aug 06 15:13:31 2018, https://me.yahoo.com/howdidwegetherereally#f714d wrote: ... Show quoted text
> > The existing Perl binding is here: > > https://metacpan.org/release/Compress-Zstd > > It uses the streaming API and looks OK to me in general, but I really > want to have it available via AnyUncompress in order to keep > supporting all other formats without adding extra code.
Unfortunately the streaming support in Compress-Zstd is incomplete. It cannot integrate with AnyUncompress in its current form. Created a proof-of-concept pull request at https://github.com/spiritloose/Compress-Zstd/pull/18 to add the missing features. Have a tentative implementation of IO-Compress-Zstandard created that uses the pull request.
  Wed May 01 11:25:27 2019 https://me.yahoo.com/howdidwegetherereally#f714d - Correspondence added
On Tue Feb 19 08:56:27 2019, PMQS wrote: Show quoted text
> Unfortunately the streaming support in Compress-Zstd is incomplete. It > cannot integrate with AnyUncompress in its current form.
I see. Hopefully that can be rectified soon. Show quoted text
> Created a proof-of-concept pull request at > https://github.com/spiritloose/Compress-Zstd/pull/18 to add the > missing features. > > Have a tentative implementation of IO-Compress-Zstandard created that > uses the pull request.
Thanks for working on this. BTW, the CI tests only succeeded for 5.26 and fails for all other Perl versions. Is your POC iplementation available somewhere? I'd like to give it a good test (Cygwin's Perl is 5.26.3, so that should work).
  Thu May 02 11:06:34 2019 pmqs [...] cpan.org - Correspondence added
... Show quoted text
> > Thanks for working on this. BTW, the CI tests only succeeded for 5.26 > and fails for all other Perl versions.
My excuse is this is just a proof-of-concept. :-) Show quoted text
> Is your POC implementation available somewhere? I'd like to give it a > good test (Cygwin's Perl is 5.26.3, so that should work).
See enclosed. It (obviously) needs the version of Compress-ZStd at https://github.com/spiritloose/Compress-Zstd/pull/18 to work. That pull request is very much a proof-of-concept. I don't expect the change to be merged into Compress-ZStd mainline as is. Paul
Subject: IO-Compress-Zstd-master.zip
Download IO-Compress-Zstd-master.zip
application/x-zip-compressed 88.9k

Message body not shown because it is not plain text.

  Sun Sep 08 15:44:57 2019 https://me.yahoo.com/howdidwegetherereally#f714d - Correspondence added
Show quoted text
> See enclosed. It (obviously) needs the version of Compress-ZStd at > https://github.com/spiritloose/Compress-Zstd/pull/18 to work. That > pull request is very much a proof-of-concept. I don't expect the > change to be merged into Compress-ZStd mainline as is.
I finally got around to test this. The MANIFEST is missing and the private/MakeUtil.pm is a dangling symlink, but I replaced it with the one in the release of IO-Compress and that worked. I also patched Compress-Zstd to use the system zstd library instead of the bundled one and removed the legacy MT interface (which I didn't configure in). After all this, things actually work and all tests pass except one that is missing a file: t/25interop-io-string.t .. Can't locate interop-io-string.pl in @INC I haven't yet used it in any actual code, but I hope to get to that soon-ish.
  Sun Sep 08 16:02:13 2019 pmqs [...] cpan.org - Correspondence added
On Sun Sep 08 15:44:57 2019, https://me.yahoo.com/howdidwegetherereally#f714d wrote: Show quoted text
> > See enclosed. It (obviously) needs the version of Compress-ZStd at > > https://github.com/spiritloose/Compress-Zstd/pull/18 to work. That > > pull request is very much a proof-of-concept. I don't expect the > > change to be merged into Compress-ZStd mainline as is.
> > I finally got around to test this. The MANIFEST is missing and the > private/MakeUtil.pm is a dangling symlink, but I replaced it with the > one in the release of IO-Compress and that worked. I also patched > Compress-Zstd to use the system zstd library instead of the bundled > one and removed the legacy MT interface (which I didn't configure in). > After all this, things actually work and all tests pass except one > that is missing a file: > > t/25interop-io-string.t .. Can't locate interop-io-string.pl in @INC > > I haven't yet used it in any actual code, but I hope to get to that > soon-ish.
Good to hear that it works for you. Bear in mind that this is unreleased code that is likely to change (both in Compress-Zstd and my code). I can't move forward with this until Compress-Zstd is updated. My pull request for that is still pending - see https://github.com/spiritloose/Compress-Zstd/pull/18.
  Thu Jul 23 17:05:31 2020 pmqs [...] cpan.org - Correspondence added
IO-Cmpress-Zstd release to CPAN - see https://metacpan.org/release/IO-Compress-Zstd
  Thu Jul 23 17:05:32 2020 pmqs [...] cpan.org - Status changed from 'open' to 'resolved'