[Nix-dev] A Journey into the Haskell NG infrastructure: Part I
Eric Seidel
eric at seidel.io
Sat Jan 10 00:07:38 CET 2015
Hi Peter,
this sounds very nice, thanks for all of the work you've put into
nix-haskell!
One immediate question I have is how to translate local haskell
packages (e.g. as described by [1]) into the haskellng regime. If I just
replace `haskellPackages` with `haskellngPackages` I get the following
error:
error: anonymous function at
"/Users/gridaphobe/Source/nixpkgs/pkgs/development/haskell-modules/default.nix":1:1
called with unexpected argument ‘extension’, at
"/Users/gridaphobe/Source/nixpkgs/lib/customisation.nix":58:12
Thanks!
Eric
[1]:
https://nixos.org/wiki/Haskell#Maintaining_your_own_set_of_additional_Haskell_packages
On Fri, Jan 9, 2015, at 07:46, Peter Simons wrote:
> Dear Haskellers `intersect` Nixers,
>
> this is the first installment of a series of postings to describe
> technical
> aspects of the re-factored Haskell infrastructure. When this is all done,
> I
> intend to use these articles to create some kind of *gasp* user
> documentation
> for the Nixpkgs manual. So I encourage everyone to point out of omissions
> and
> technical errors, or to ask for clarification if something is vague or
> misleading. I've chosen an FAQ-style approach because I felt that this is
> easiest for me to write. Hopefully this translates into the document
> being easy
> to read, too! Here we go ...
>
> Why should I care about this "new infrastructure"?
> --------------------------------------------------
>
> The new code will break evaluation of any Haskell-related configuration
> you
> may have in ~/.nixpkgs/config.nix or /etc/nixos/configuration.nix.
>
> Privately generated cabal2nix expressions will cease to compile.
>
> Installations that rely on ghc-wrapper finding GHC libraries
> automatically in
> your ~/.nix-profile are obsolete. If you use this approach, you won't
> be able
> to update your profile anymore.
>
>
> Duh, I don't want that!
> -----------------------
>
> You can stick to the established 'haskellPackages' set, if you prefer.
> The
> new code exists separately in 'haskellngPackages', and it's invisible
> to
> "nix-env -i <package-name>" and "nix-env -u". If you don't make a
> conscious
> effort to switch, you won't see any of it.
>
>
> So I'll never have to update if I don't want to?
> ------------------------------------------------
>
> My guess is that 'haskellPackages' and 'haskellngPackages' will
> co-exist for
> a while. Personally, I switched to Haskell NG, though, and I won't
> maintain
> any packages in the old hierarchy anymore. I suppose other contributors
> will
> do the same thing. Once you've converted your setup to
> 'haskellngPackages',
> there's no reason to look back, really.
>
> In other words, you don't have to convert your setup *right now*, but
> at some
> point in the future you will probably have to.
>
>
> Is it difficult to migrate to Haskell NG?
> -----------------------------------------
>
> Users of 'ghcWithPackages' can easily switch back and forth between
> both
> package sets. Your ~/.nixpkgs/config.nix file probably defines an
> attribute
> like this one:
>
> | {
> | packageOverrides = super: let self = super.pkgs; in
> | {
> | haskellEnv = self.haskellPackages.ghcWithPackages (p: with p; [
> | async attoparsec caseInsensitive fgl GLUT GLURaw haskellSrc
> | ]);
> | };
> | }
>
> Change this definition as follows to switch to Haskell NG:
>
> | {
> | provideOldHaskellAttributeNames = true;
> |
> | packageOverrides = super: let self = super.pkgs; in
> | {
> | haskellEnv = self.haskellngPackages.ghcWithPackages (p: with p; [
> | async attoparsec caseInsensitive fgl GLUT GLURaw haskellSrc
> | ]);
> | };
> | }
>
> There are only two differences: the "ng" bit in the name of the package
> set,
> and the new 'provideOldHaskellAttributeNames' config attribute. (Note
> that
> this lives at the top level of ~/.nixpkgs/config.nix -- not inside of
> 'packageOverrides'.)
>
> Users of the ghc-wrapper have to make a greater effort, I'm afraid,
> because
> ghc-wrapper no longer exists in the NG code. You'll have to switch to
> 'ghcWithPackages' first. You may find this article helpful for the
> process:
> <http://permalink.gmane.org/gmane.linux.distributions.nixos/15161>.
>
>
> What does 'provideOldHaskellAttributeNames' do?
> -----------------------------------------------
>
> The package 'cabal-install' has the attribute 'cabalInstall' in the old
> package set. In the new one, however, the attribute is now called
> 'cabal-install' -- just like the package.
> provideOldHaskellAttributeNames
> enables a compatibility layer that defins the old camel-case style
> names on
> top of the new ones. This allows you to use the same configuration for
> both
> the new and old package sets.
>
> Once you feel ready to migrate permanently to NG code, you can switch
> the
> compatibility layer off, to ensure that all your definition use the new
> names
> consistently.
>
>
> Why is the ghc-wrapper gone?
> ----------------------------
>
> ghc-wrapper's underlying promise is that you can install a random
> Haskell
> package into your Nix profile, and GHC will pick it up from there
> automatically. Now, this worked reasonably well for many packages, but
> for
> may other packages it didn't work, and those cases caused us a lot
> trouble,
> bug reports, and heated discussions in the past, and we ended up adding
> more
> and more layers of plumbing around xmonad, ghc-mod, and whatnot in an
> attempt
> to rescue the idea that GHC would somehow find packages automatically
> without
> being told about them.
>
> In the NG infrastructure, this whole notion has been abandoned. The
> proper
> way to install ghc is trough the 'ghcWithPackages' function. It knows
> about
> exactly those packages that you tell it to. It's slightly more effort
> to
> define your Haskell environment this way, but what you get in return
> for this
> extra effort is a totally deterministic compiler that doesn't depend on
> any
> wrapper scripts, it produces predictable results independent of what
> else you
> may or may not have installed, and it interacts just fine with
> cabal-install
> -- something the ghc-wrapper never managed to get right.
>
>
> When will the new code be available?
> ------------------------------------
>
> It already is. 'haskellngPackages' was introduced to 'master' by commit
> 54baa53d on 2015-01-07, and it has now been picked up by the "unstable"
> channel.
>
>
> Why would I want to migrate? What are the advantages?
> -----------------------------------------------------
>
> The NG package set contains the latest version of every package
> available
> from Hackage, plus some older versions that we need to make legacy
> stuff
> compile. In total, we deliver in excess of 7,500 packages. Just look at
>
> https://raw.githubusercontent.com/NixOS/nixpkgs/master/pkgs/development/haskell-modules/hackage-packages.nix
>
> if you don't believe it. The 'hackage-packages.nix' file is generated
> automatically by the hackage2nix utility -- which I'll describe in
> detail in
> a later installment --, and it allows us to easily track Hackage on a
> daily
> basis if we want to.
>
> Haskell NG supports GHC 7.10.x.
>
> Haskell NG allows you to configure the hell out of your installation.
> You can
> add new packages easily, you can build your own customized package
> sets, you
> can change Cabal flags, compiler arguments, or any other aspect of how
> a
> package is built by means of the 'overrideCabal' function without
> editing the
> source of Nixpgs. The file
>
> https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-common.nix
>
> illustrates this point nicely.
>
> It's possible to specify deep overrides with 'overrideScope', which
> I'll
> describe in detail in a later installment. To get a first impression,
> check
> out
>
> https://github.com/NixOS/nixpkgs/blob/master/pkgs/development/haskell-modules/configuration-ghc-7.8.x.nix#L61
>
> for a concrete example of a deep override that replaces the version of
> mtl
> 2.1.x with 2.2.x in the context of the 'amazonka' package family.
>
>
> Is there a binary cache available for the NG packages?
> ------------------------------------------------------
>
> Haskelll NG is built by
> http://hydra.nixos.org/jobset/nixpkgs/haskell-updates
> and http://hydra.cryp.to/jobset/nixpkgs/haskell-ng. You can get
> binaries from
> these servers by adding the command line flags
>
> --option extra-binary-caches http://hydra.nixos.org
> --option extra-binary-caches http://hydra.cryp.to
>
> to your invocations of nix-build, nix-env, etc.
>
> The normal binary cache that backs the "unstable" channel does not yet
> have
> binaries for NG.
>
>
> Does Haskell NG support compilers other than GHC 7.8.4?
> -------------------------------------------------------
>
> Yes, there is a family of package sets 'haskell-ng.packages.ghcXYZ'
> that
> define package sets for GHC versions 6.10.4, 6.12.3, 7.0.4, 7.2.2,
> 7.4.2,
> 7.6.3, 7.8.4, and HEAD a.k.a. 7.10.1-rc1. For example, to install a
> version
> of GHC 7.6.3, define
>
> ghc763Env = self.haskell-ng.packages.ghc763.ghcWithPackages (p: with
> p; [
> mtl network parallel parsec QuickCheck random regex-base
> regex-compat
> regex-posix stm syb text zlib alex cabal-install cpphs happy
> ghc-paths
> monad-par
> ]);
>
> in ~/.nixpkgs/config.nix. Note that the compiler-specific package sets
> do
> *not* have the compatibility layer. Inside of
> 'haskell-ng.packages.ghcXYZ',
> only the new attribute names exist!
>
> Furthermore, the package sets for GHC 7.6.3, 7.8.4, and 7.10.1-r1 are
> the
> only ones that can be expected to work. I have never tested any
> pre-7.6.3
> compiler. Getting those package sets to work well is almost certainly
> going
> to be a bit of an effort.
>
>
> What can I do to help develop Haskell NG?
> -----------------------------------------
>
> Use it!
>
> Switch your configuration to the 'haskellngPackages' package set and
> try
> whether you can update your profile to the new version. If you cannot,
> then
> please report the issue in this e-mail thread so that we can try to
> figure
> out what's going wrong.
>
> As of today, we can compile approximately 50% of Hackage successfully.
> See
> http://hydra.nixos.org/jobset/nixpkgs/haskell-updates for the current
> state
> of affairs. Obviously, we would like to get that percentage way up!
> Patches,
> bug reports, and feature requests will help us accomplish that.
>
>
> Have fun,
> Peter
>
> _______________________________________________
> nix-dev mailing list
> nix-dev at lists.science.uu.nl
> http://lists.science.uu.nl/mailman/listinfo/nix-dev
More information about the nix-dev
mailing list