R 13 eproducibilit Y: NixOS

Is NixOS Reproducible?

Tracking: nixos-unstable's iso_minimal job for x86_64-linux.

Build via:

git clone https://github.com/nixos/nixpkgs.git
cd nixpkgs
git checkout eeef6e1341273588ba724819be35e937b7c66165
nix-build ./nixos/release-combined.nix -A nixos.iso_minimal.x86_64-linux

4607 out of 4796 (96.06%) paths in the minimal installation image are reproducible!

13 unchecked

unreproduced paths

• /nix/store/411yb6z665ink6c6mhjc0fbwgv1wyy8s-hspec-discover-2.7.10.drv
• (diffoscope) out

unchecked paths

• /nix/store/0l0vks3kn4aqmhw1wcvkrnlhyxpqns5r-format-security.diff.drv
• /nix/store/bjnm579c42j6g1rxrdbnmj24bm5ijfyk-CVE-2011-0764.diff.drv
• /nix/store/kx9xn864lk5h7pnrlsl79z02qfv83vyz-source.drv
• /nix/store/wbhbvcdf800g7i6byfg30i12mlnd02ak-CVE-2011-1552_1553_1554.patch.drv
• /nix/store/b4xk0f27qspwv3pdd92nld73ffzbx4y6-lib-cleanup.diff.drv
• /nix/store/ik7vdhfk0zrfi36m9y7d2kwzbhyxs4km-source.drv
• /nix/store/669x5z4wa2hmrbk0ixnjzvp96a0i7qh4-attachment.cgi?id=359589.drv
• /nix/store/9jcjy6yygq2624c1msysi9kafr7pc8g9-arrow-1.2.0.tar.gz.drv
• /nix/store/hdjkm4byh91vassj17a45wzylc2sy15l-CVE-2010-2642.patch.drv
• /nix/store/w0pgfmh4aw0ffs9ylgnycki4cm1nda2p-tzdata-2021.1.tar.gz.drv
• /nix/store/akxbp6ax9vbcm7f0vamsrbaqpm4bgj0b-mariadb-connector-c-3.2.3-src.tar.gz.drv
• /nix/store/xqmjkiaayf5i1zxwd4pvwnnlyc1dplr0-factory_boy-3.2.1.tar.gz.drv
• /nix/store/z1qfpsgxn57x5i053cgbdqsx2gi57bgj-attachment.cgi?id=361056.drv

How are these tested?

Each build is run twice, at different times, on different hardware running different kernels.

How confident can we be in the results?

Fairly. We don't currently inject randomness at the filesystem layer, but many of the reproducibility issues are being exercised already. It isn't possible to guarantee a package is reproducible, just like it isn't possible to prove software is bug-free. It is possible there is nondeterminism in a package source, waiting for some specific circumstance.

This is why we run these tests: to track how we are doing over time, to submit bug fixes for nondeterminism when we find them.

How can I help?

• Look at issues and pull requests with the 'reproducible builds' label.
• Open new issues/PR's for reproducibility problems (like the ones reported above)
• Join the #reproducible-builds:nixos.org room on Matrix

How can we do better?

There are further steps we could take. For example, the next likely step is using disorderfs which injects additional nondeterminism by reordering directory entries.

How can I test my patches?

Nix has built-in support for checking a path is reproducible. There are two routes.

Pretending you are debugging a nondeterminism bug in hello. To check it, you build the package, and then build it again with --check --keep-failed. This will provide the differing output in a separate directory which you can use diffoscope on.

$ nix-build . -A hello
$ nix-build . -A hello --check --keep-failed
[...snip...]
error: derivation '/nix/store/...hello.drv' may not be deterministic:
output '/nix/store/...-hello' differs from '/nix/store/...hello.check'
$ diffoscope /nix/store/...hello /nix/store/...hello.check

Note: the .check output is not a valid store path, and will automatically be deleted on the next run of the Nix garbage collector.

There is support for an automatic diff-hook in Nix 2, but it is much more complicated to set up. If you would like to work on this, or need help setting it up, contact us on Matrix. We can work together to write docs on how to use it.