[Nix-dev] Multiple machines with nix store shared via NFS
Renato Alves
alves.rjc at gmail.com
Thu Jul 7 18:32:08 CEST 2016
Hi Erik,
This is a different question. As far as I understand it NixOps
replicates a machine configuration. It copies the store. I want the
exact opposite. One store shared across all using NFS.
Anyway just for the record in case someone goes down the same road.
At the time there is no official/supported way to achieve what I wanted.
nix-daemon relies on communication via a socket which is only available
on the machine running that daemon.
What I ended up going for is:
https://gist.github.com/Unode/2e41d8d2766580612d8996aa4f2887f7
Not exactly what I was hoping for but works for now until a better
solution is developed.
Renato
On 03/07/16 22:51, 4levels wrote:
> Hi Renato,
>
> I'm no expert whatsoever here, but isn't NixOps doing this in a way? At
> least on my nixos deployed servers all stores are kept in sync, only the
> server-specific packages are unique for each machine.
> Don't ask me how it works, I can only confirm that it does ;-)
>
> Just my 2 cents..
>
> Kind regards,
>
> Erik
>
>
> On Fri, Jul 1, 2016, 23:16 Renato Alves <alves.rjc at gmail.com
> <mailto:alves.rjc at gmail.com>> wrote:
>
> Had a look at the links. The 2011 thread mentions problems that have
> been addressed by either nix-daemon or remote-build nodes.
>
> There's also some references to socat but not much. Seems like
> forwarding could indeed be a solution but from what I read elsewhere
> it's a fragile setup and requires ssh-agent and passwordless keys.
>
> There's also some articles about using haproxy and nginx to proxy
> sockets but it's all very much ad-hoc. I'm also not sure how this would
> work given the security model of nix-daemon.
>
> The 2014 thread is also a good read but focused on using the cluster
> infrastructure to compile things. I don't want to go down that road yet.
> Perhaps in the future if necessary.
> At the moment I'd be happy with having nix-env and nix-shell on all
> machines.
>
> Is there currently any support for NIX_REMOTE specifying a connection to
> a daemon on a different host? (i.e. literally REMOTE :)
>
> R
>
>
>
> On 01/07/16 22:34, Layus wrote:
> > I never sait it was a good idea, just soemthing worth investigating.
> >
> > Now, this has been discussed on the ML already
> >
> > See for example
> > [Nix-dev] Sharing the Nix store between systems
> >
> <http://lists.science.uu.nl/pipermail/nix-dev/2011-December/007381.html>
> /Rickard
> > Nilsson
> > /[Nix-dev] nix on compute cluster?
> >
> <http://lists.science.uu.nl/pipermail/nix-dev/2014-October/014584.html>
> //Andreas
> > Herrmann
> >
> > //And more generally Google search on
> > site:http://lists.science.uu.nl/pipermail/nix-dev/ :-).
> >
> > -- Layus.
> >
> > On 01/07/16 21:55, Renato Alves wrote:
> >> I never tried it with sockets, only tcp ports.
> >>
> >> I need to move the socket to a different location. Currently it also
> >> sits on NFS. I can play with symlinks making it point to a local
> disk.
> >>
> >> The entire setup sounds terribly hackish though.
> >>
> >> If anyone has better alternatives I'd love to hear them.
> >>
> >> Thanks,
> >> R
> >>
> >> On 01/07/16 20:51, Layus wrote:
> >>> Just one idea off the top of my head: Would it be possible to
> forward
> >>> the unix socket of the daemon ?
> >>>
> >>> -- Layus ?
> >>>
> >>> On 01/07/16 15:10, Renato Alves wrote:
> >>>> Hi everyone,
> >>>>
> >>>>
> >>>> Recently I've been trying to setup nix on a multiple-machine shared
> >>>> environment. At the moment I have one machine setup with nix-daemon
> >>>> running from a store in a custom location (/shared/myuser/nix).
> This
> >>>> location is shared via NFS with several other machines. This is all
> >>>> still pretty new to me as I've only recently started using nix and
> >>>> haven't used nixos yet.
> >>>>
> >>>> Before setting up nix-daemon I ran into some problems with database
> >>>> corruption (sqlite) due to concurrent processes. With
> nix-daemon these
> >>>> have gone away.
> >>>>
> >>>> One limitation that I identified recently is that on any of the
> other
> >>>> machines that do not have nix-daemon running (but have read
> access to
> >>>> the store) I can run commands from the store without problems but
> >>>> cannot, for instance, use nix-shell.
> >>>> So I thought of spawning a nix-daemon instance on those
> machines but I'm
> >>>> not sure what is the best way to set it up without risking DB
> corruption
> >>>> again.
> >>>>
> >>>> These are the requirements I'd like to meet:
> >>>> * /shared/myuser/nix is shared across multiple machines with
> the same
> >>>> architecture (x86_64-linux).
> >>>> * All machines should be able to use nix-shell or any nix related
> >>>> command, including requesting installation of packages.
> >>>> * Only some machines are allowed to compile. Machines that are not
> >>>> allowed should rely on something like remote-systems.conf to
> distribute
> >>>> work.
> >>>>
> >>>> Is this kind of setup currently possible with nix? If so, can
> someone
> >>>> provide some guidance?
> >>>>
> >>>>
> >>>> Thanks,
> >>>> Renato
> >>>>
> >>>>
> >>>>
> >>>> _______________________________________________
> >>>> nix-dev mailing list
> >>>> nix-dev at lists.science.uu.nl <mailto:nix-dev at lists.science.uu.nl>
> >>>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
> >>>
> >>>
> >>>
> >>> _______________________________________________
> >>> nix-dev mailing list
> >>> nix-dev at lists.science.uu.nl <mailto:nix-dev at lists.science.uu.nl>
> >>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
> >>>
> >>
> >>
> >> _______________________________________________
> >> nix-dev mailing list
> >> nix-dev at lists.science.uu.nl <mailto:nix-dev at lists.science.uu.nl>
> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
> >
> >
> >
> >
> > _______________________________________________
> > nix-dev mailing list
> > nix-dev at lists.science.uu.nl <mailto:nix-dev at lists.science.uu.nl>
> > http://lists.science.uu.nl/mailman/listinfo/nix-dev
> >
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl <mailto:nix-dev at lists.science.uu.nl>
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 819 bytes
Desc: OpenPGP digital signature
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20160707/cdbd2a2b/attachment-0001.sig>
More information about the nix-dev
mailing list