[Nix-dev] How to set python's module search path (was: nix store: hardlink vs symlink)

[Florian Friesdorf]

Hi Peter,

On Sun, 25 Sep 2011 16:08:35 +0200, Peter Simons <simons at cryp.to> wrote:
> Hi Florian,
> 
>  > The prefix determines where python searches for site-packages. [...]
> 
> my impression is that you are not concerned about the use of symlinks
> versus hardlinks, but you're really interested in ways to make Python
> find its extra libraries.

yes

> It looks on first sight as if those two issues are related, but they
> are really not. The proper way to make Python find its extra libraries
> is through use of environment variables, and there are two alternative
> solutions:
> 
>  > Python applications with an isolated environment
>  > ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>  > They call python as:
>  > /nix/store/vzpvrymynp4n93bznxha6hadj0ww68vx-python-2.7.1/bin/python
>  >
>  > the prefix is:
>  > /nix/store/vzpvrymynp4n93bznxha6hadj0ww68vx-python-2.7.1
>  >
>  > and python will load its empty site-packages in there. Modules needed
>  > by the application are made available via PYTHONPATH.
> 
> This approach has extremely reliable behavior. It is easy to predict how
> Python is going to have, because the set of available libraries does not
> depend on any external factors, such as the contents of the profile that
> Python happens to be installed in. Suppose I wanted Python to know about
> the Cheetah library, then would configure
> 
>   {
>       pythonCheetah = pkgs.pythonFull.override {
>         extraLibs = with pkgs.python27Packages; [ cheetah ];
>       };
>     };
>   }

I was not aware that this is possible. So far I was not able to find
documentation for config.nix.

> in ~/.nixpkgs/config.nix, install
> 
>   nix-env -p /tmp/foobar -iA pythonCheetah
> 
> ..., and I'd be able to run
> 
>   /tmp/foobar/bin/python -c 'import Cheetah'
> 
> just fine. That particular approach allows me to control which libraries
> Python knows, and which libraries Python *doesn't know*, and Python is
> going to behave that way no matter what I other things I do.

I will try this approach for my use cases and report back.
 
>  > Python development
>  > ~~~~~~~~~~~~~~~~~~
>  > You currently can install python modules into a profile, together with a
>  > python interpreter (of the same python version) they can build a python
>  > environment. Not meant for running applications, but for example
>  > development.
>  >
>  > For this environment to function properly python needs to treat the
>  > profile as its home. virtualenv for example currently breaks because of
>  > modules coming from different prefixes.
>  >
>  > If I install ipdb and ipython into a profile, I'd expect them to see
>  > all modules I added to the profile.
> 
> This approach doesn't work well within Nix, because Nix calls things
> directly from their store path (i.e. there is no "profile"), but it's a
> fine solution for end users who to install things into their home
> profile and just run them without re-configuring their expressions every
> time. The behavior is inherently unpredictable, though. When a certain
> library is installed in the profile, Python is going to know about it.
> When that library is missing, however, Python won't know about it. In
> other words, there are hidden dependencies.
> 
> One could argue that PYTHONHOME is not the proper way to accomplish this
> configuration, though. Personally, I'd rather set
> 
>   export PYTHONPATH=$HOME/.nix-profile/lib/python2.7/site-packages
>
> in my ~/.bashrc or something, but that's a matter of taste.
>
> Anyway, both solutions have their individual strengths and weaknesses,
> and I believe that we should support both.

+1

> The "pythonhomeWrapper" expression seems to be superfluous, though.
> There is no need for two different wrapper scripts, so I'd much rather
> see that functionality incorporated into the existing wrapper.

+1

thanks a lot!
-- 
Florian Friesdorf <flo at chaoflow.net>
  GPG FPR: 7A13 5EEE 1421 9FC2 108D  BAAF 38F8 99A3 0C45 F083
Jabber/XMPP: flo at chaoflow.net
IRC: chaoflow on freenode,ircnet,blafasel,OFTC
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 835 bytes
Desc: not available
Url : http://lists.science.uu.nl/pipermail/nix-dev/attachments/20110926/e2118917/attachment.bin