[Nix-dev] Reproducibility testing in Hydra

Eelco Dolstra eelco.dolstra at logicblox.com
Tue Dec 13 14:09:58 CET 2016


On the occasion of the second Reproducible Builds workshop currently taking
place in Berlin [1], I'm happy to announce that Hydra is now capable of
repeating builds in order to test for certain types of binary reproducibility
problems. Hydra can now be configured to run every build step in a jobset
multiple times, comparing the result. If the result differs between rounds, the
build step will still succeed, but the step will be marked as "Succeeded with
non-deterministic result". (Thus reverse dependencies can still proceed.) For


Furthermore, if there is a difference between rounds, diffoscope (a tool for
comparing arbitrary directory trees [2]) is run on the the resulting store
paths. For example, here is the build log for Samba:


The diffoscope output is at the end (starting at "diff with previous round"),
showing that Samba stores a timestamp in lib/samba/libsmbd-base-samba4.so.

Similarly, here is the log for python-urllib3:


This shows that the pip build tool uses a random temporary directory, which ends
up in .pyc files. (This affects almost 1800 packages.)

Currently, out of 36550 build steps in a full NixOS evaluation, only 4699
(12.8%) were detected as non-deterministic. (Note that Hydra can only detect the
presence of non-determinism, not its absence). Here is the full list:


Currently only the nixos:reproducibility jobset runs build steps multiple times
(twice, to be precise).

[1] https://reproducible-builds.org/events/berlin2016/
[2] https://diffoscope.org/

Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/

More information about the nix-dev mailing list