| Subject: | Deterministic linking order |
While working on the "reproducible builds" effort [0], we have noticed that the linking order of object files in Module::Build::c_link() depends on readdir() order, which is nondeterministic. This affects the generated binary, rendering it non-reproducible.
The nondeterminism originates in rscan_dir(). The attached patch makes it return its file lists in sorted order. Some alternative fixes would be to call File::Find with the "preprocess" argument to sort the list, or sort the list of object files in process_support_files() or later in c_link().
It's not clear to me if the latter options are safe, or if a distribution might inject its own list of object files and expect their order to be preserved. In contrast, since there's no existing guarantee of the order of rscan_dir() results, it's clearly safe. The downside is a number of probably unnecessary sort() calls when rscan_dir() gets called in other contexts.
I'm happy to rework the patch to one of the other alternatives, and make a GitHub pull request out of that if you like. Please just let me know what kind of fix you would prefer.
Thanks for your work on Module-Build!
[0] https://wiki.debian.org/ReproducibleBuilds
| Subject: | 0001-Sort-file-lists-generated-by-rscan_dir.patch |
From 7bfcb26d8e314bce37aeeef4048f99b66fcdfbbc Mon Sep 17 00:00:00 2001
From: Niko Tyni <ntyni@debian.org>
Date: Tue, 1 Sep 2015 22:05:27 +0300
Subject: [PATCH] Sort file lists generated by rscan_dir()
The rscan_dir() function traverses a directory with File::Find,
which returns files in readdir() order. This order is nondeterministic
and depends on the file system.
The lists are used, among other things, to find C files to compile
(in process_support_files()) and later to link (in c_link()).
The linking order affects the generated binary, essentially rendering
it nondeterministic and breaking reproducibility.
---
lib/Module/Build/Base.pm | 2 +-
1 file changed, 1 insertion(+), 1 deletion(-)
diff --git a/lib/Module/Build/Base.pm b/lib/Module/Build/Base.pm
index 1cbc61e..d9ea82f 100644
--- a/lib/Module/Build/Base.pm
+++ b/lib/Module/Build/Base.pm
@@ -5255,7 +5255,7 @@ sub rscan_dir {
die "Unknown pattern type";
File::Find::find({wanted => $subr, no_chdir => 1}, $dir);
- return \@result;
+ return [ sort @result ];
}
sub delete_filetree {
--
2.1.4