[Nix-dev] all-packages.nix and the unoverridable self.

[roconnor at theorem.ca]

Something seems wrong / bizzare with the recursion in all-packages.nix

I've been trying to override linux_4_4 in my configuration.nix with the 
following:

     nixpkgs.config.packageOverrides = super:
      { linux_4_4 = super.linux_4_4.override { extraConfig = "CHROME_PLATFORMS y";
                                               kernelPatches = [ { name = "f10_sysrq"; patch = ./f10_sysrq.patch; } ]; };
      };

but this no longer works.  I've spent a few hours studying the issue.
If I do the following chain of overrides upto the linuxPackages
attribute

     nixpkgs.config.packageOverrides = super: rec
      { linux_4_4 = super.linux_4_4.override { extraConfig = "CHROME_PLATFORMS y";
                                               kernelPatches = [ { name = "f10_sysrq"; patch = ./f10_sysrq.patch; } ]; };
        linuxPackages_4_4 = super.recurseIntoAttrs (super.linuxPackagesFor
        linux_4_4 linuxPackages_4_4);
        linuxPackages = linuxPackages_4_4;
        linux = linuxPackages.kernel;
      };

then it does work.

I couldn't for the life of me understand why copying what is essentially
the exact definitions of linuxPackages_4_4 and linuxPackages into my
packageOverrides caused it to work.  The whole point of the
packageOverride mechanism is to invoke late-binding so that I don't have
to override long chains.

I traced the issue to the following strange set of definitions:

all-packages.nix begins with something like this

{ ... }:
self: pkgs:

with pkgs;

{ ... }

It is a function of three arguments, (1) a set of parameters, (2) a
binding for self, (3) a binding for pkgs, and the with pkgs; bring all
the definitions from pkgs into scope.

This is called from top-level/default.nix with the following
expression:

        allPackages = self: super:
          let res = import ./all-packages.nix allPackagesArgs res self;
          in res;

So allPackageArgs contains the parameters, self gets bound to res, and
pkgs get bound to self.

The upshot of this is that within all-packages.nix self (which is bound
to res) is the result of only evaluating all-packanges *with no
overrides* while pkgs (which is bound to self) ends up late-bound and is
the set of packges *with all overrides*

So when linux and linuxPackages get bound using self in all-packages:

    # The current default kernel / kernel modules.
    linuxPackages = self.linuxPackages_4_4;
    linux = self.linuxPackages.kernel;

The use of self here (and throughout the linuxPackage definitions) means
that we are making reference to the *unoverridden package set*.  This is
why my packageOverrides of linux_4_4 did nothing, because the references
to linux_4_4 inn all-packages.nix are prefixed with "self." which means
it always gets the unoverriden packages.

Is this really the desired behaviour?  I think that all-packages.nix is
full of many questionable uses of the "self." prefix.  I suspect that
people don't really understand that "self." means "give me the
unoverriden version of packages".  I think renaming "self" in
all-packages.nix to "unoverridenPackages" would be a better name.

-- 
Russell O'Connor                                      <http://r6.ca/>
``All talk about `theft,''' the general counsel of the American Graphophone
Company wrote, ``is the merest claptrap, for there exists no property in
ideas musical, literary or artistic, except as defined by statute.''