How to use custom python with existing packages - nix derivation? - nix

Question

I defined a custom nix cpython derivation with unmerged patches. If I specify it as a target nix-shell gives me the version I expect.
{ pkgs ? import <nixpkgs> {} }:
(pkgs.python3.overrideAttrs (old: {
src = pkgs.fetchFromGitHub {
owner = "bergkvist";
repo = "cpython";
rev = "01bcf2bef5f4ffffb454da35cb66b186a7a12598";
sha256 = "1713sx5izd745bgr6fdx6d1g7ivaqy6jrf9v5jgml31bd1nmfccy";
};
verions = "3.11.5";
})) })).override {
sourceVersion = {major = "3"; minor = "11" ; patch = "5"; suffix = ""; };
};
$ nix-shell
$ python
Python 3.11.0a0 (default, Aug 4 2021, 00:12:31) [Clang 7.1.0 (tags/RELEASE_710/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
But I don't see a way how to use new cpython for existing packages.
Straight forward overlay doesn't affect anything.
{ pkgs ? import <nixpkgs> {} }:
let myPatchedPython = ... ;
myPkgs = pkgs.extend(self: super: {
python3 = myPatchedPython;
python39 = myPatchedPython;
});
in myPkgs.python39Packages.send2trash
$ nix-shell
$ python3
Python 3.9.5 (default, Jul 18 2021, 14:31:15)
I noticed I can use override but it is only for runtime and it is not working for all packages.
let useMyPy = drv: drv.override {
python = myPatchedPython;
python3 = myPatchedPython;
python38 = myPatchedPython;
python39 = myPatchedPython;
};
pyobjc-core = useMyPy (pkgs.python39Packages.buildPythonPackage rec {
pname = "pyobjc-core";
version = "7.3";
adfasdf = true;
name = "${pname}-${version}";
src = pkgs.python39Packages.fetchPypi {
pname = "pyobjc-core";
inherit version;
sha256 = "0x3msrzvcszlmladdpl64s48l52fwk4xlnnri8daq2mliggsx0ah";
};
propagatedBuildInputs = [
myPatchedPython
pkgs.darwin.libobjc
pkgs.darwin.cctools
pkgs.darwin.apple_sdk.frameworks.Foundation
pkgs.darwin.apple_sdk.frameworks.AppKit
pkgs.darwin.apple_sdk.frameworks.CoreServices
pkgs.darwin.apple_sdk.frameworks.Cocoa
pkgs.python39Packages.setuptools
];
buildInputs = [ pkgs.libffi ];
});
in pyobjc-core
$ nix-shell
$ python3
Python 3.11.0a0 (default, Aug 4 2021, 00:12:31) [Clang 7.1.0 (tags/RELEASE_710/final)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import import AppKit
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "/nix/store/jcc0xf3zwxnzzh497nip7v57k68r4zb9-python3.9-pyobjc-7.3/lib/python3.9/site-packages/AppKit/__init__.py", line 10, in <module>
import Foundation
^^^^^^^^^^^^^^^^^
...

Answer 1

AFAIK, the official way to do this is to define myPatchedPython in an overlay for nixpkgs (in ~/.config/nixpkgs/overlays.nix), and invoke nix-shell like so:
nix-shell -p myPatchedPython
If this does not work, you can also try nix-shell -E '(import <nixpkgs> {}).myPatchedPython' and the like, which give you more of a debugging angle.
The more pedantic way (without overlays) that I have used in the past with success is to directly copy or include your patched Python version in the shell.nix file, e.g.:
{ pkgs ? import <nixpkgs> {} }:
pkgs.mkShell {
buildInputs = [
pkgs.python.overrideAttrs { ... }
];
}