[Nix-dev] Sharing the Nix store between systems

[Rickard Nilsson]

Hi,

I've been talking about this before, but now I've done some more  
experimentation.

I want to share one Nix store between several systems via NFS or CIFS.  
Ideally, the store should be read-only on all but one system, and all  
builds should be delegated to that one system (and maybe to some build  
machine slaves too). Oh, and I want to run mixed CPU architectures.

I had it figured out quite well, with one "master" machine that could  
perform all writes to the store and to the Nix database, but then I  
discovered that I could not get this master machine to cross-build ARM for  
a complete NixOS system. So, then I tried to share both the store and the  
Nix database with write permissions to two different systems (one ARM and  
one x86_64). Unfortunately, the database got severly corrupted when both  
machines was building things at the same time. I guess this is a feature  
of sqlite+NFS+file locks.

So, now I'm looking for suggestions on what to try next. I guess the  
sqlite problem could be solved if a PostgreSQL or MySQL backend was  
implemented for Nix. Then the database could be shared between systems.  
However, I really like the idea that only one host in a complete  
infrastructure has write permissions to the Nix store, but users on other  
hosts can still do "nixos-rebuild" or "nix-env -i".

One idea could be if slave build machines didn't build directly into the  
store used by the host itself, but instead wrote to a temporary store, and  
also to a temporary database (or maybe it doesn't need to write to any db  
at all). The master host could then pick up the build results and put them  
into its database and into the shared store. I think that, in theory, this  
scheme could work. How about in practice?


Best regards,
   Rickard Nilsson