[Nix-dev] versionedDeravation / php fpm / cups 1.7

[Marc Weber]

As Peti & eventually shlevy requested I'm moving this discussion to the
nix-dev list. Not everybody is watching the github repository.

To keep my own maintanance effort low I'd like to spend the time
making the cupsd update patch and the fpm patch acceptable - but I need
help understanding what to do and why. 

People who were participating (feeling more or less strong about it):
  shelvy: was closing requests for style issues - and probably agreeing that it should be discussed here
  peti  : was disliking this style and finally closing the issue https://github.com/NixOS/nixpkgs/issues/1957 to force discussion on nix-dev
  eelco : was disliking this style finally explained why in issues/1957
  (details below)

Two valuable patches which (still) use versioneDerivation and which solve
"real world problems" some nixos users might care about.

patch 1: cups update: https://github.com/NixOS/nixpkgs/pull/1120 
  fixes: printing with older canon printers MP980 and PIXMA BJC 4000 (by
  updating cups from 1.5.4 to 1.7 - which causes quite a lot of
  differences, see below)

patch 2: most complete php fpm implementation: https://github.com/NixOS/nixpkgs/pull/487
  Needs some refactoring - which?

Details:

* denotes info
+ denotes improvement
- denotes TODO - maybe with proposal how to fix
? denotes topic to be discussed (eg versioneDerivation usage)


cups changes details (patch 1):
===============================

  * based on PDF processing chain

  + makes my older canon printers work (MP980 and Pixma 4000).
    They previously only printed 30% of a page

  ? is using versioneDerivation, see [Examples] below
    https://github.com/MarcWeber/nixpkgs/tree/experimental/marc/pkgs/misc/cups/default.nix
    support 1.6 can be romved probably if 1.7 works for everyone.
    1.5.4 should be kept eventually to keep everything working as its now

    https://github.com/MarcWeber/nixpkgs/tree/experimental/marc/pkgs/pkgs/misc/ghostscript/default.nix
    9.06 does contain raster cups filter
    9.10 does no longer contain raster, AFAIK its contained in cupsFilters instead
    gnu ? no idea.

  - localhost:631 which is the cupsd configuration page cannot print
    testing pages

  - I should provide this interface (trivial to provide this)

      services.cupsd = // alias for 1_5, deprecating this

      services.cupsd_1_5 = {
        // still need old version because I cannot guarantee that 1.7 works
        // for everybody
        // ps processing chain
      }

      services.cupsd_1_7 = {
        // still need old version because I cannot guarantee that 1.7 works
        // for everybody
        // pdf processing chain

        gutenprintPackage = pkgs.gutentprint; # or gutenprintCVS, use this to enable gutenprint, this will build all ppds
        ghostscript = pkgs.ghostscript_Mainline_9_10; # or gnu version ..
      }

PHP fpm changes (patch 2)
========================
+ will figure out how many daemons/pools to create on its own based on
  ini files etc. Some options cannot be set in pool configuration - such
  as enabling xdebug, then you need multiple php services.
? uses one one php.nix file to run PHP versions - which is close to versioneDerivation style
  5.2 should be dropped
  5.3
  5.4
  5.5
  => It just happens that all can be run easily with only some small
  changes, see [Example 3] below

  and provide php.xdebug (tested, works for all php versions)
  php.apc and the like (untested, maybe no longer needed)

  and php.system_fpm_config (which should be moved into nixos/modules
  now - 5.2 will get dropped)

- 5.2 should be dropped: will fix this

versioneDerivation (use cases)
==============================
Usage example:

  [Example 1]:
  https://github.com/MarcWeber/nixpkgs/blob/experimental/marc/pkgs/misc/cups/default.nix
  installFlags are all the same (I was too lazy here)

  [Example 2]:
  https://github.com/MarcWeber/nixpkgs/blob/experimental/marc/pkgs/misc/ghostscript/default.nix

  Mind this comment:
    # This no longer contains raster for cups, should be contained in cups-filters now?

  should this be in "meta", too? (if so for which version) - this does
  make a big difference for cupsd module!

  Documenting such changes is easily spotted if you have one file.

  [Example 3] PHP
  https://github.com/MarcWeber/nixpkgs/blob/experimental/marc/pkgs/development/interpreters/php/default.nix
  Its not using versioneDerivation, but is very close - that close that
  it serves as "historically grown" example. The differences are still
  small, thus just search for "lessThan" using ctrl-f browser search to
  spot them all.

  all-packages.nix looks like this:
    php = callPackage ../development/interpreters/php { };
    php5_3 = php.override { version = "5.3.x"; };
    php5_3fpm = php5_3.override { sapi = "fpm"; };
    [.. same for 5.4 and 5.5 .. ]

  Is it really that bad as long as most code is shared (otherwise its
  the wrong tool)

discussing versionedDerivation - arguments
==========================================

In https://github.com/NixOS/nixpkgs/issues/1957 Eelco Dolstra described
what he "dislikes":

    == QUOTE 
      My main objection to versionedDerivation is the wackiness of
      having package functions that take the version as an argument,
      i.e.

      { stdenv, fetchurl, version ? "5.3" }:
      versionedDerivation "cups" version { ... }
      There is nothing about the function interface that tells you what
      the valid values of version are, and thus what versions are
      supported. What you should do is return an attribute set
      containing the supported versions:

      { stdenv, fetchurl }:
      {
        php_5_3 = ...;
        php_5_4 = ...;
      }
      Of course, you can factor out the commonality between versions any way you want, e.g.

      let
        makePHP = common: stdenv.mkDerivation ({ ... } // common);
      in {
        php_5_3 = makePHP {
          name = "php-5.3.50";
          src = fetchurl { ... };
        }
        ...
      }
      Alternatively, you can have separate files for each expression
      that include a file common.nix for the common stuff (like we do
      for the Linux kernel).

    == QUOTE END

  I agree that it does make sense to "docmuent" which versions are
  "supported".
  The easy answer is:
  The default version is documented in the argument list, the others
  just mean "can be build - or there has been a time where it did build"
  unless they get referenced somewhere (most likely in all-packages.nix,
  see php5_3fpm example above. Whether this should be documented this way
  is another story.
  => I've created a new page: https://nixos.org/w/index.php?title=Open_issues:maintenance_properties_of_a_package&action=submit

  Documenting wich packages are how well supported is an open issue
  IMHO. This just is "yet another way".

  The pattern Eelco Dolstra is discussing is used in different context
  and with some varation, eg in python-packages.nix:

  python-packages.nix takes as arguments:

    { pkgs, python, lowPrio }:

  Thus while its not taking a version, you still have no idea which
  python you may pass (or python versions ..) - thus IMHO its not that
  much differing :)

  A similar argument could be applied to "systems". (eg darwin vs
  x86_64 i686):
  php."5.3"."darwin" to indicate its fine to be used on darwin.

  From this point of view there is not that much wrong by
  versionedDerivation, the only change neccessary would be making it
  return all versions as attrs so that such usage would be valid:

    phps =    import ....php/default.nix;
    php_5_3 = import ....php/default.nix { }."5.3.x";
    php_5_4 = import ....php/default.nix { }."5.4.x";

  Thus does it differ that much from [Example 3] above which looks like
  this:

    php5_3fpm = php5_3.override { sapi = "fpm"; version = "5.3.x"; };

  How would the "perfect PHP nixpkgs implementation" look like?

    let commonConfigureFlagDescription = [ long list .. ];

    let phpDerivation = { commonConfigureFlagDescription, version,
      src_md5_hash, patches ? [], allowFastCGI }: {
        // the common code merging the options
      }

    mergexdebuglikestuff = php: php // {
      xdebug = ..
      acp = ..
      ... = ..;
    }

    phps = {
      php5_2 = mergexdebuglikestuff (phpDerivation {
        long list of options
      });
      php5_3 = mergexdebuglikestuff (phpDerivation {
      });
      php5_4 = mergexdebuglikestuff (phpDerivation {
      });
      php5_5 = mergexdebuglikestuff (phpDerivation {
      });
    }

   In the end is it that much more readable than what I already have?
   Link -> [Example 3] PHP above

   I totally agree that I should refactor if newer versions happen to be
   totally different - it just didn't happen that way (yet) could be
   cause I've been lucky though.

Can you reply if you have new arguments about why versionedDeravation
is nice/bad so that we get a comprehensive list and that I understand
which is the best way to rewrite those patches ?

Also ping me if you're interested in testing either patch.

Thanks for helping me gain awarness .. :)

Marc Weber