[Nix-dev] Binary cache substituter
eelco.dolstra at logicblox.com
Wed Sep 19 16:33:54 CEST 2012
The latest nixUnstable in Nixpkgs has an experimental new feature called the
"binary cache substituter" intended to eventually replace the current
The main problem with the nix-pull approach is that it requires the manifest you
nix-pulled to be in sync with your Nix expressions. If these are not in sync,
then Nix will fall back to building from source, even though binaries may be
The new binary cache substituter doesn't pre-download a manifest; rather, every
time Nix needs to build a new store path, it will ask a list of HTTP servers
(the "binary caches") if they have a binary of that store path available.
For instance, if you add the line
binary-caches = http://nixos.org/binary-cache
to your nix.conf, then if Nix needs a store path /nix/store/<hash>-<name>, then
it will probe http://nixos.org/binary-cache/<hash>.narinfo. That file, if it
exists, will contain a pointer to a NAR file containing the binary, and metadata
like the size and references of the store path. Example:
References: ... jfcs9xnfbmiwqs224sb0qqsybbfl3sab-linux-headers-220.127.116.11
In other words, it contains the information that used to be in nix-pull manifest
files, but only for one particular store path.
As an example of how useful this is, here is how you would get Firefox 18.104.22.168
from the ancient Nixpkgs 0.11 release:
$ cd /my/nixpkgs/checkout
$ git checkout 0.11
$ nix-build -A firefox --argstr system i686-linux
The nix-build will notice that binaries of the requested Firefox are available
at http://nixos.org/binary-cache and use them. You don't need to make sure you
have the right manifest installed or anything like that. So this makes the Nix
notion of binary deployment as a "transparent" optimisation of source deployment
much more reality.
Some random points:
* Instead of setting binary-caches in nix.conf, you can specify/override binary
caches on the command line, e.g. "nix-build -A firefox --option binary-caches
* When building through the Nix daemon, you cannot use --option binary-caches to
configure arbitrary caches because that would be a security hole. You can only
specify a subset of the binary-caches and trusted-binary-caches values
configured in nix.conf. "trusted-binary-caches" is a list of caches that are
not used by default but can be specified by untrusted users.
* Subscribing to a Nix channel will auto-configure the use of a corresponding
binary cache. For instance, the channel
http://nixos.org/releases/nixos/channels/nixos-unstable will automatically cause
http://nixos.org/binary-cache to be used.
* A binary cache can be created using nix-push, e.g. "nix-push --dest /tmp/cache
* The binary cache substituter supports xz compression. Xz compresses about 50%
better than bzip2 and decompresses about twice as fast (at the price of
significantly slower compression, but that only needs to be done once per path
on the server). See commit 4911a10a4e51102a21a5d123a852c75d2ec92dbc for some stats.
* The main technical challenge in implementing the binary cache substituter is
dealing with the latency involved in checking lots of remote .narinfo files.
Nix uses persistent HTTP connections and lots of parallelism to look up .narinfo
files efficiently. For instance, if you do "nix-env -qas \*" on Nixpkgs (which
involves checking the existence of thousands of .narinfo files), Nix will spam
the server with 150 parallel HTTP connections. This will probably require more
tweaking. Positive and negative .narinfo lookups are cached in
/nix/var/nix/binary-cache-v1.sqlite. Negative lookups are expired after a
* The main downside to the binary cache substituter is that it has no support
for binary patches (and it's not obvious how to support them). If you really
need them, you can add "force-manifest = true" to nix.conf to force nix-channel
to nix-pull the channel's manifest.
Eelco Dolstra | LogicBlox, Inc. | http://nixos.org/~eelco/
More information about the nix-dev