[Nix-dev] NixOps failes to deploy: bash: Argument list too long
4levels
4levels at gmail.com
Sat Apr 15 12:14:59 CEST 2017
Hi Profpatsch,
the name nixos-system-secure comes from one machine called "secure".
I'm using various nix files to separate / group some parts of the machine
configurations we're using.
We're deploying +10 servers for our application code (mancloud) and the
machine with name "secure" is supposed to have all keys from all other
machines on it.
I don't see where I should add the mentioned `set -x`
We're usually calling nixops as follows
nixops deploy --include secure --build-only -I nixpkgs=
https://github.com/NixOS/nixpkgs-channels/archive/nixos-16.09.tar.gz
When viewing the rest of the file mentioned before (/nix/store/
wz9mvb6x082h4mvww1gsf0x0l9fq8p7z-nixos-system-secure-16.09pre-git.drv), I
can see a lot of comments adding up to the total length of this file, eg.
["-e","/nix/store/9krlzvny65gdc8s7kpb6lkx8cd02c25b-default-builder.sh"],[("activationScript","#!
/nix/store/gabjbkwga2dhhp2wzyaxl83r8hjjfc37-bash-4.3-p48/bin/bash\n\nsystemConfig=@out@\n\nexport
PATH=/empty\nfor i in
/nix/store/lcwdbh37ha51z86c62mm65vbcfc990dd-coreutils-8.25
/nix/store/zwr4gj1xj67iifpa5mixwz5az7k0j1c4-gnugrep-2.25
/nix/store/95vfigaqdg8jg5bk961k1x06j86p5fh0-findutils-4.6.0
/nix/store/bm052v0zqk8w4gvfwqacszb6b9kijcs4-glibc-2.24-bin
/nix/store/2ij54lmg02915s43kdwlf5hf5qnm04xx-shadow-4.4
/nix/store/n46pca88k2avx96q9zp6927xq3apv2xd-net-tools-1.60_p20120127084908;
do\n PATH=$PATH:$i/bin:$i/sbin\ndone\n\n_status=0\ntrap \"_status=1\"
ERR\n\n# Ensure a consistent umask.\numask 0022\n\n#### Activation script
snippet stdio:\n# Needed by some programs.\nln -sfn /proc/self/fd
/dev/fd\nln -sfn /proc/self/fd/0 /dev/stdin\nln -sfn /proc/self/fd/1
/dev/stdout\nln -sfn /proc/self/fd/2 /dev/stderr\n\n\n#### Activation
script snippet binsh:\n# Create the required /bin/sh symlink; otherwise
lots of things\n# (notably the system() function) won't work.\nmkdir -m
0755 -p /bin\nln -sfn
\"/nix/store/7y6azp3wqbbf1wldyyam9zj49s8v09rp-bash-4.3-p48/bin/sh\"
/bin/.sh.tmp\nmv /bin/.sh.tmp /bin/sh # atomically replace
/bin/sh\n\n\n#### Activation script snippet domain:\n\n\n#### Activation
script snippet etc...
But I'm not even sure if this file has anything to do with the error itself
:-(
Hope someone can help! Deploying 1600 key files to a server shouldn't be a
problem right? This number is only going to increase over time as well..
I've added some more info on our deploy setup below..
Output of nixops info
nixops info
vultr.nix defaults.nix servers.nix keys.nix
*vultr.nix* contains some Vultr specifics, like bootloader, filesystem and
swap config
*defaults.nix* contains some server defaults, like user accounts, sshd
setup, etc etc
*servers.nix* contains specifications for the various machines we deploy,
"secure" being one of them
with import <nixpkgs/lib>;
let
serverKeys = keys:
genAttrs keys (n:
{
text = fileContents (./keys + "/${builtins.replaceStrings
["@"] ["-"] n}");
group = "keys";
permissions = "0640";
}
)
;
serverKeysHost = host: keys:
genAttrs keys (n:
{
text = fileContents (./keys + "/${host}" +
"/${builtins.replaceStrings ["@"] ["-"] n}");
group = "keys";
permissions = "0640";
}
)
;
in
{
secure =
{ config, pkgs, lib, nodes, ... }:
{
deployment = {
targetHost = "45.76.38.157";
keys = (serverKeys [
"phpmyadmin.password"
"phpmyadmin.secret"
"oauth-private.key"
"oauth-public.key"
])
// serverKeysHost "v-ams01" nodes.v-ams01.config.services.mancloud.keys
// serverKeysHost "v-ams02" nodes.v-ams02.config.services.mancloud.keys
// serverKeysHost "v-ams03" nodes.v-ams03.config.services.mancloud.keys
// serverKeysHost "v-ams04" nodes.v-ams04.config.services.mancloud.keys
// serverKeysHost "v-ams05" nodes.v-ams05.config.services.mancloud.keys
// serverKeysHost "v-ams06" nodes.v-ams06.config.services.mancloud.keys
// serverKeysHost "v-fra01" nodes.v-fra01.config.services.mancloud.keys
// serverKeysHost "v-par01" nodes.v-par01.config.services.mancloud.keys
// serverKeysHost "v-lon01" nodes.v-lon01.config.services.mancloud.keys
// serverKeysHost "v-syd01" nodes.v-syd01.config.services.mancloud.keys
// serverKeysHost "v-mia01" nodes.v-mia01.config.services.mancloud.keys
;
};
*keys.nix* contains the list of key-files, being stored in ./keys/
subfolder and are read by the serverKeys(Host) calls above, eg.
{
v-ams01 =
{ config, pkgs, lib, ... }:
{
services.mancloud.keys = [
"mancloud.client1.api.clients.mancloud.secret"
"mancloud.client1.api.key"
"mancloud.client1.database.password"
"mancloud.client1.encryption.cipher"
"mancloud.client1.encryption.key"
...
On Sat, Apr 15, 2017 at 12:22 AM Profpatsch <mail at profpatsch.de> wrote:
> On 17-04-14 09:44pm, 4levels wrote:
> > Hi Profpatch,
> >
> > Thanks for your reply, but I don't know how to even try implementing your
> > suggestions. This file is generated by NixOps and is run just before the
> > actual deployment starts. This is at the end of the build phase before
> the
> > actual deploy. It starts with
> >
> >
> Derive([("out","/nix/store/j9p5zwqj03nq4f4id5a0phig5gk0snqd-nixos-system-secure-16.09pre-git","","")],
> > ...
>
> This is a derivation.
> They are normally generated by nix expressions,
> mostly out of the nixpkgs repo.
>
> nixos-system-secure looks like a system derivation
> created somewhere in nixpkgs/nixos.
> You might try searching for nixos-system-secure.
> In this case nixos-system leads you to the correct file.
> `buildCommand` is the build description. It is aliased
> above as `systemBuilder`. Try adding `set -x` into it
> and start nixops with your nixpkgs (man nixops, search for -I).
>
> `-I` is kind of stateful with nixops,
> so you might want to build that machine with nix-build first.
> nixpkgs/nixos/default.nix, concrete:
>
> nix-build -A vm \
> -E 'import /path/to/nixpkgs/nixos { configuration =
> /path/to/machine/configuration; };'
>
> Hope that helps finding the bug.
>
> >
> > I have literally no idea where to look or what to change in NixOps itself
> > to use the suggestions you made, as this goes way beyond my
> nix-knowledge.
> > For all I know, this issue could come from something totally different as
> > well, I'm just gessing here since this is the last output of the nixops
> > deploy call before the error message. Below the full output of the call
> >
> > [erik at no01:~]$ nixops deploy --build-only --show-trace
> > building all machine configurations...
> > these derivations will be built:
> >
> >
> /nix/store/wz9mvb6x082h4mvww1gsf0x0l9fq8p7z-nixos-system-secure-16.09pre-git.drv
> > /nix/store/bz3ajdlxalr0c8i1zik40zwvnsb2g8a7-nixops-machines.drv
> > building path(s)
> >
> ‘/nix/store/j9p5zwqj03nq4f4id5a0phig5gk0snqd-nixos-system-secure-16.09pre-git’
> > while setting up the build environment: executing
> > ‘/nix/store/gabjbkwga2dhhp2wzyaxl83r8hjjfc37-bash-4.3-p48/bin/bash’:
> > Argument list too long
> > builder for
> >
> ‘/nix/store/wz9mvb6x082h4mvww1gsf0x0l9fq8p7z-nixos-system-secure-16.09pre-git.drv’
> > failed with exit code 1
> > cannot build derivation
> > ‘/nix/store/bz3ajdlxalr0c8i1zik40zwvnsb2g8a7-nixops-machines.drv’: 1
> > dependencies couldn't be built
> > error: build of
> > ‘/nix/store/bz3ajdlxalr0c8i1zik40zwvnsb2g8a7-nixops-machines.drv’ failed
> > error: unable to build all machine configurations
> >
> >
> > Kind regards,
> >
> > Erik
> >
> >
> > On Fri, 14 Apr 2017, 22:00 Profpatsch, <mail at profpatsch.de> wrote:
> >
> > > On 17-04-14 02:26pm, 4levels wrote:
> > > > I think this is the generatd file causing the issue:
> > > >
> > >
> /nix/store/wz9mvb6x082h4mvww1gsf0x0l9fq8p7z-nixos-system-secure-16.09pre-git.drv
> > > > It looks like it's a single line of code, spanning 207073 characters,
> > > which
> > > > is pbbly more than the allowed argument length for bash.
> > > >
> > > > Are there any guidelines to avoid running into this argument list
> error?
> > >
> > > Depends on your bash use-case.
> > > Lots of programs support file-input.
> > > xargs may be a good candidate.
> > > If you don’t want it sequential, GNU parallel.
> > > Or just a plain old bash for; do; done loop.
> > >
> > > --
> > > Proudly written in Mutt with Vim on NixOS.
> > > Q: Why is this email five sentences or less?
> > > A: http://five.sentenc.es
> > > May take up to five days to read your message. If it’s urgent, call me.
> > >
>
> --
> Proudly written in Mutt with Vim on NixOS.
> Q: Why is this email five sentences or less?
> A: http://five.sentenc.es
> May take up to five days to read your message. If it’s urgent, call me.
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mailman.science.uu.nl/pipermail/nix-dev/attachments/20170415/444b8296/attachment-0001.html>
More information about the nix-dev
mailing list