| Subject: | Consider transparent compression |
.pmat files contain raw memory contents quite a lot and (especially on 64bit systems) are both large, but easily compressed.
It could be useful if D:M:Dumper would transparently compress files if the name ends ".gz" or some other recognised suffix; and D:M:Dumpfile would transparently decompress them, perhaps by inspecting for some magic header.
It would be good to select a compression system that favours compression speed, to ensure it doesn't slow down the dumper time. Though given the corresponding reduction in disk write hopefully it will overall be a good tradeoff whatever is picked.
Some algorithms to consider are mentioned at
https://github.com/facebook/zstd
Given both dumper and loader are XS code, embedding some small C implementation should still be quite easy.
--
Paul Evans