[Nix-dev] garbage collection and nix-push

[Mathijs Kwik]

Michael Raskin <7c6f434c at mail.ru> writes:

>>Hi all,
>>
>>The hardware I use for building my channel is not very powerful. It
>>does a good job of building derivations, but packing nar files with xz
>>takes ages.
>>
>>My channel is nothing special, It just builds the configurations for a
>>few home machines and laptops and some packages that hydra doesn't
>>build with the optional dependencies I like.
>>
>>By registering the outputs as gc roots, I made sure no extensive
>>rebuilds are performed when I don't change much.
>>However, after a garbage collector run, all nar outputs get removed,
>>leading to long push times on the next run.
>>
>>So for now, I just lowered the garbage-collection cronjob to once a week.
>>
>>Does anyone have a suggestion on how to work around this?
>>For example registering the nar outputs as (temp/indirect) gc roots or
>>steering the garbage collector not to remove outputs that were built
>>less than 48 hours ago or something?
>
> I am not too familiar with precise nix-push inside. Does the idea of 
> building a directory with symlinks of NARs (wo it depends on them all),
> calling it something that includes current date, and removing 
> 100-hour-old link sets in cron look hard ot easy to implement?

That's the direction I was thinking in indeed.
However, apart from using "find" on the store to find the nar outputs, I
could not find an easy way.

nix-push outputs all nar files into 1 directory.
However, in the store, each file is in its own subdirectory and the name
of that dir is not easily guessable.

nix-push also outputs .narinfo files, but they don't contain info on
that dirname either.

So it will get scripty =)
First just find all .nar.xz files in /nix/store and store the listing in
a temporary file, then grep that file for every narfile in nix-push's
output dir and symlink to that.

It's not hard to do, but I was hoping for a somewhat easier way.