[Nix-dev] Multiple outputs and hierarchical Nix store paths

[Bjørn Forsman]

Hi Vladimir,

On 22 May 2016 at 09:36, Vladimír Čunát <vcunat at gmail.com> wrote:
> On 05/21/2016 03:53 PM, Bjørn Forsman wrote:
>> In Nix:
>> I'm actually unsure about the details here, but I *think* the standard
>> builder simply takes the first output of each package in buildInputs. [...]
>
> Yes, the first but that one typically propagates other outputs - by
> default the ones including headers, libraries and binaries. You can
> choose which by setting propagatedBuildOutputs = [ "foo" "bar" "out" ];

Aha, that makes it much easier (less work for humans doing packaging).

>> [...]
>> $ nix-build -A pkgconfig
>> /nix/store/wndacaca5dq5cpyfa41v1fv4sfhr0y77-pkg-config-0.29
>> $ nix-build -A pkgconfig.share
>> /nix/store/wndacaca5dq5cpyfa41v1fv4sfhr0y77-pkg-config-0.29/share
>> $ nix-build -A pkgconfig.share.doc
>> /nix/store/wndacaca5dq5cpyfa41v1fv4sfhr0y77-pkg-config-0.29/share/doc
>>
>> (Note that all outputs have the same store path prefix.)
>> [...]
>
> That is similar to what regular distros use... only they drop the hashed
> prefixes completely. The problem is with detecting runtime dependencies.
> First, we only scan for the hash part AFAIK. We might change that to
> scan for the whole prefix, but if someone gets a reference to
> pkgconfig.share.doc, one can do ../../lib/foo and get to other outputs
> (!) Sure it would actually solve some problems due to packages assuming
> everything is in the same output, but we would sacrifice part of purity,
> e.g. GC could break reachable stuff.

Hm, I didn't think of that. But I'm still very interested in seeing a
prototype of this. Maybe it'll work OK in practice? Maybe only a few
packages need to be "constrained" from too much garbage collection?
Maybe we can come up with a scanner that catches most of these
relative, and otherwise undetected, references?

IMHO, the current multiple-output architecture feels like having a
scalability issue. There is a limit to how much we can split due to
each output having to be in separate paths. (Or else lots of manual
work to make (sub)packages find each other.)

Now, how long will it take me to prototype this? 5+ years? :-D

- Bjørn