[Nix-dev] On npm2nix and the NPM package set in Nixpkgs
Graham Christensen
graham at grahamc.com
Tue Jul 5 18:17:53 CEST 2016
You all are great! Thank you so much!
Graham
On Tue, Jul 5, 2016 at 8:14 AM Kamil Chmielewski <kamil.chm at gmail.com>
wrote:
> +1.. I'll do this in go2nix.
>
> --
> Kamil
>
> 2016-07-05 15:10 GMT+02:00 Rok Garbas <rok at garbas.si>:
>
>> +1 ... i did just that recently for pypi2nix. but i'll also add a link
>> to the project home.
>>
>> [1]
>> https://github.com/garbas/pypi2nix/commit/339aee3b149909430ebe7e3e27b8cf158addaef1
>>
>> On Tue, Jul 5, 2016 at 2:47 PM, Graham Christensen <graham at grahamc.com>
>> wrote:
>> > I've found myself confused by multiple projects using the same lang2nix
>> > name, and big changes in format. One consistent complaint I have is the
>> top
>> > of the file usually says:
>> >
>> > // Generated by lang2nix
>> >
>> > but having more information like a version number and a URL to the
>> project
>> > would have saved hours of searching and trying different tools.
>> Something
>> > like:
>> >
>> > // Generated by lang2nix v0.1.0
>> > // See more at https://github.com/myuser/lang2nix
>> >
>> > would be a really nice usability adjustment.
>> >
>> > On Tue, Jul 5, 2016 at 7:36 AM Rok Garbas <rok at garbas.si> wrote:
>> >>
>> >> we can still keep and old version of npm2nix in nixpkgs for ppl who use
>> >> it.
>> >> and also a branch with old code could be created, for people that want
>> >> pudh bugfixes or develop further (very unlikely).
>> >>
>> >>
>> >> On Tue, Jul 5, 2016 at 11:16 AM, Tomasz Czyż <tomasz.czyz at gmail.com>
>> >> wrote:
>> >> > Rok,
>> >> >
>> >> > what about people who are already using previous solution? Why break
>> >> > their
>> >> > workflows?
>> >> >
>> >> > 2016-07-05 7:36 GMT+01:00 Rok Garbas <rok at garbas.si>:
>> >> >>
>> >> >> +1 for just keeping the name npm2nix and bumping up the version.
>> >> >>
>> >> >> i'm not using it on any active project, but i'm going to in the near
>> >> >> future.
>> >> >>
>> >> >>
>> >> >>
>> >> >> On Mon, Jul 4, 2016 at 10:11 PM, Tobias Pflug <tobias.pflug at gmx.net
>> >
>> >> >> wrote:
>> >> >> > Hi Sander,
>> >> >> >
>> >> >> > sorry for my very late response. I'll make this one brief as I am
>> >> >> > sadly
>> >> >> > on
>> >> >> > my phone.
>> >> >> >
>> >> >> > I belong to one of those who tried your new npm2nix and in fact
>> am
>> >> >> > already
>> >> >> > using it regularly. I am very much in favor of having your
>> >> >> > re-engineeering2
>> >> >> > branch replacing npm2nix as the de-facto node integration tool.
>> >> >> >
>> >> >> > I also definitely want to see the current set of auto-generated
>> node
>> >> >> > packages removed from nix. They are almost exclusively *totally*
>> >> >> > outdated.
>> >> >> >
>> >> >> > Thank you a lot for your continued efforts on this. Working with
>> >> >> > npm/node is
>> >> >> > annoying but we are better off with your contributions.
>> >> >> >
>> >> >> > cheers,
>> >> >> > Tobi
>> >> >> >
>> >> >> > On 22 Jun 2016, at 20:24, Sander van der Burg <
>> svanderburg at gmail.com>
>> >> >> > wrote:
>> >> >> >
>> >> >> > Hello Nix and Node.js users,
>> >> >> >
>> >> >> > I have been absent for a while in this discussion, but as far as I
>> >> >> > know
>> >> >> > the
>> >> >> > state of the NPM packages in Nixpkgs is still quite bad and
>> despite
>> >> >> > some
>> >> >> > discussions on the mailing list we have not really come to any
>> >> >> > consensus
>> >> >> > yet.
>> >> >> >
>> >> >> > As some of you may know, I have my own re-engineered version of
>> >> >> > npm2nix
>> >> >> > that
>> >> >> > lives in a specific branch in my own personal fork
>> >> >> > (https://github.com/svanderburg/npm2nix/tree/reengineering2). A
>> few
>> >> >> > months
>> >> >> > ago, I did some major efforts in getting npm 3.x's behaviour
>> >> >> > supported,
>> >> >> > which I have documented in this blog post:
>> >> >> >
>> >> >> >
>> >> >> >
>> http://sandervanderburg.blogspot.com/2016/02/managing-npm-flat-module-installations.html
>> >> >> >
>> >> >> > I have been using this reengineering2 branch for all my public and
>> >> >> > some
>> >> >> > of
>> >> >> > my private projects since the beginning of this year, and for me
>> it
>> >> >> > seems to
>> >> >> > work quite well, despite the fact that some of npm 3.x's flat
>> module
>> >> >> > installation oddities are still not accurately supported yet.
>> >> >> >
>> >> >> > I also received a couple of reports from other people claiming
>> that
>> >> >> > their
>> >> >> > projects work and even encountered some people saying that it
>> should
>> >> >> > replace
>> >> >> > the current npm2nix. :)
>> >> >> >
>> >> >> > Obviously, I do not want to claim that my implementation is the
>> >> >> > perfect
>> >> >> > solution as it (for example) is much slower than the vanilla
>> npm2nix,
>> >> >> > and it
>> >> >> > composes the entire set of dependencies in one derivation as
>> opposed
>> >> >> > to
>> >> >> > generating a Nix store path per NPM dependency. (I do this for a
>> very
>> >> >> > good
>> >> >> > reason. For more details, please read my blog post).
>> >> >> >
>> >> >> > Furthermore, I have also spoken to people that suggested
>> completely
>> >> >> > different kinds of approaches in getting NPM supported in a Nix
>> >> >> > environment.
>> >> >> >
>> >> >> > Something that I have not done yet is investigating whether this
>> >> >> > reengineered solution could be a potential replacement for the NPM
>> >> >> > packages
>> >> >> > set in Nixpkgs.
>> >> >> >
>> >> >> > Today, I have been working on an integration pattern, and the good
>> >> >> > news
>> >> >> > is:
>> >> >> > it seems that I was able to generate Nix expressions for almost
>> all
>> >> >> > packages
>> >> >> > that are in pkgs/top-level/node-packages.json. The only exceptions
>> >> >> > were
>> >> >> > the
>> >> >> > node-xmpp-* and bip-* packages, but some of them seem to have
>> broken
>> >> >> > dependencies, which is not npm2nix's fault.
>> >> >> >
>> >> >> > If we would proceed integrating, we have a number of practical
>> >> >> > implications:
>> >> >> >
>> >> >> > - I believe it is desired to have both Node.js 4.x and Node.js
>> 5.x,
>> >> >> > 6.x
>> >> >> > supported (I actually need all of them). To support all of these,
>> we
>> >> >> > need
>> >> >> > two different sets of generated Nix expressions. The former uses
>> npm
>> >> >> > 2.x
>> >> >> > with the classic dependency addressing approach and the latter
>> uses
>> >> >> > npm
>> >> >> > 3.x
>> >> >> > with flat module installations.
>> >> >> > - I think most library packages should be removed from
>> >> >> > node-packages.json:
>> >> >> > as explained in my blog post: how a package gets composed and to
>> >> >> > which
>> >> >> > version a range resolve depends on the state of the includer. When
>> >> >> > somebody
>> >> >> > wants their own NPM project to be deployed, he should use npm2nix
>> >> >> > directly
>> >> >> > on package.json, and not refer to any NPM libraries in Nixpkgs.
>> >> >> > - Some NPM packages must be overridden to provide native
>> >> >> > dependencies.
>> >> >> > The
>> >> >> > mechanisms that the reengineering2 branch use are different. It
>> would
>> >> >> > probably take a bit of effort to get these migrated.
>> >> >> >
>> >> >> > For example, this is how I override the webdrvr package to provide
>> >> >> > phantomjs
>> >> >> > and the Selenium webdriver:
>> >> >> >
>> >> >> > {pkgs, system}:
>> >> >> >
>> >> >> > let
>> >> >> > nodePackages = import ./composition-v4.nix {
>> >> >> > inherit pkgs system;
>> >> >> > };
>> >> >> > in
>> >> >> > nodePackages // {
>> >> >> > webdrvr = nodePackages.webdrvr.override (oldAttrs: {
>> >> >> > buildInputs = oldAttrs.buildInputs ++ [ pkgs.phantomjs ];
>> >> >> >
>> >> >> > preRebuild = ''
>> >> >> > mkdir $TMPDIR/webdrvr
>> >> >> >
>> >> >> > ln -s ${pkgs.fetchurl {
>> >> >> > url =
>> >> >> >
>> >> >> >
>> >> >> > "
>> https://selenium-release.storage.googleapis.com/2.43/selenium-server-standalone-2.43.1.jar
>> ";
>> >> >> > sha1 = "ef1b5f8ae9c99332f99ba8794988a1d5b974d27b";
>> >> >> > }} $TMPDIR/webdrvr/selenium-server-standalone-2.43.1.jar
>> >> >> > ln -s ${pkgs.fetchurl {
>> >> >> > url =
>> >> >> >
>> >> >> >
>> >> >> > "
>> http://chromedriver.storage.googleapis.com/2.10/chromedriver_linux64.zip
>> ";
>> >> >> > sha1 = "26220f7e43ee3c0d714860db61c4d0ecc9bb3d89";
>> >> >> > }} $TMPDIR/webdrvr/chromedriver_linux64.zip
>> >> >> >
>> >> >> > '';
>> >> >> > });
>> >> >> > }
>> >> >> >
>> >> >> >
>> >> >> > Although we have some practical issues, I think none of them would
>> >> >> > impose a
>> >> >> > serious problem.
>> >> >> >
>> >> >> > Then about npm2nix itself: Obviously, we could say that my version
>> >> >> > replaces
>> >> >> > the upstream npm2nix and gets "blessed" into the new "official"
>> >> >> > version,
>> >> >> > but
>> >> >> > I don't know whether everybody likes it.
>> >> >> >
>> >> >> > Alternatively, we could be a bit more pragmatic: I stop calling my
>> >> >> > reengineering2 version npm2nix, I give it a different name and I
>> >> >> > release
>> >> >> > it
>> >> >> > as a different package. This makes it possible for those who want
>> it,
>> >> >> > to
>> >> >> > still use the 'vanilla' npm2nix alongside my version.
>> >> >> >
>> >> >> > Then in Nixpkgs we can decide to:
>> >> >> >
>> >> >> > - to keep npm2nix the default and provide my tool as a package
>> >> >> > - or to make the reengineering2 version the default, and provide
>> >> >> > npm2nix
>> >> >> > as
>> >> >> > a package
>> >> >> > - in theory: support both package sets, but this might be a bit
>> >> >> > overkill
>> >> >> > :)
>> >> >> >
>> >> >> > For those who don't know: although my repository is a fork of
>> >> >> > npm2nix,
>> >> >> > the
>> >> >> > reengineering2 version is basically a rewrite of npm2nix and quite
>> >> >> > different
>> >> >> > than the upstream version. It is written in JavaScript (as
>> opposed to
>> >> >> > CoffeeScript), has a different modular structure and different
>> >> >> > command-line
>> >> >> > interface, so that's why I'm very careful in proposing to replace
>> the
>> >> >> > upstream npm2nix.
>> >> >> >
>> >> >> > Moreover, it also does not share any git revision history with the
>> >> >> > upstream
>> >> >> > npm2nix. :)
>> >> >> >
>> >> >> > As a final note: for those who do not know about this: the
>> >> >> > reengineering2
>> >> >> > tool can already be used outside Nixpkgs and this is what I have
>> been
>> >> >> > doing
>> >> >> > for all my projects. The expressions that it generates are based
>> on
>> >> >> > the
>> >> >> > principles I have described in this blog post:
>> >> >> >
>> >> >> >
>> >> >> >
>> http://sandervanderburg.blogspot.com/2014/07/managing-private-nix-packages-outside.html
>> >> >> >
>> >> >> > My apologies for this very long email, but I'd like to have your
>> >> >> > feedback
>> >> >> > and I don't want my preferences to disrupt other people's
>> workflows.
>> >> >> >
>> >> >> > What do you think?
>> >> >> >
>> >> >> > Best,
>> >> >> >
>> >> >> > Sander
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > nix-dev mailing list
>> >> >> > nix-dev at lists.science.uu.nl
>> >> >> > http://lists.science.uu.nl/mailman/listinfo/nix-dev
>> >> >> >
>> >> >> >
>> >> >> > _______________________________________________
>> >> >> > nix-dev mailing list
>> >> >> > nix-dev at lists.science.uu.nl
>> >> >> > http://lists.science.uu.nl/mailman/listinfo/nix-dev
>> >> >> >
>> >> >>
>> >> >>
>> >> >>
>> >> >> --
>> >> >> Rok Garbas
>> >> >> http://www.garbas.si
>> >> >> rok at garbas.si
>> >> >> _______________________________________________
>> >> >> nix-dev mailing list
>> >> >> nix-dev at lists.science.uu.nl
>> >> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>> >> >
>> >> >
>> >> >
>> >> >
>> >> > --
>> >> > Tomasz Czyż
>> >>
>> >>
>> >>
>> >> --
>> >> Rok Garbas
>> >> http://www.garbas.si
>> >> rok at garbas.si
>> >> _______________________________________________
>> >> nix-dev mailing list
>> >> nix-dev at lists.science.uu.nl
>> >> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>
>>
>>
>> --
>> Rok Garbas
>> http://www.garbas.si
>> rok at garbas.si
>> _______________________________________________
>> nix-dev mailing list
>> nix-dev at lists.science.uu.nl
>> http://lists.science.uu.nl/mailman/listinfo/nix-dev
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.science.uu.nl/pipermail/nix-dev/attachments/20160705/dd1e6472/attachment-0001.html>
More information about the nix-dev
mailing list