From Section 12.5 of Nix Pills:
Finish the expression for graphviz with gd support (note the use of the with expression in buildInputs to avoid repeating pkgs):
let
pkgs = import <nixpkgs> {};
mkDerivation = import ./autotools.nix pkgs;
in mkDerivation {
name = "graphviz";
src = ./graphviz-2.38.0.tar.gz;
buildInputs = with pkgs; [ gd fontconfig libjpeg bzip2 ];
}
Question: Why is the the with pkgs needed at the end of the code snippet in buildInputs? I thought pkgs was already in scope through the let binding at the top?
Yes, pkgs is in scope, but that doesn't automatically put it's attributes into scope.
pkgs is a Nix set containing attributes such as gd, fontconfig... If you drop with pkgs then the list of buildInputs will not be able to resolve the aforementioned attributes. Using your example, the following two expressions are semantically identical:
buildInputs = with pkgs; [ gd fontconfig libjpeg bzip2 ];
buildInputs = [ pkgs.gd pkgs.fontconfig pkgs.libjpeg pkgs.bzip2 ];
Here's another example which is also semantically identical:
let
pkgs = import <nixpkgs> {};
mkDerivation = import ./autotools.nix pkgs;
gd = pkgs.gd;
fontconfig = pkgs.fontconfig;
libjpeg = pkgs.libjpeg;
bzip2 = pkgs.bzip2;
in mkDerivation {
name = "graphviz";
src = ./graphviz-2.38.0.tar.gz;
buildInputs = [ gd fontconfig libjpeg bzip2 ];
}
In the last example, gd, fontconfig... are brought into scope in the first part of the let expression, and then used in buildInputs without the with pkgs.
Hopefully that will help you intuit what with pkgs is doing.
Related
I can also run interpreter of Python 3.9 which installed in system using environment.systemPackages.
with import <nixpkgs>{};
let
in pkgs.mkShell {
buildInputs = with pkgs; [
ccls
clang-tools
cmake
gdb
gcc
fish
mono
python37Full
tcl
tk
];
inputsFrom = with pkgs; [
];
runScript = "fish";
}
I want to run a nix-shell with the following packages installed:
aspell
aspellDicts.en
hello
I can't simply do: nix-shell -p aspell aspellDicts.en hello --pure as this will not correctly install the aspell dictionaries. Nix provides a aspellWithDict function that can be used to build aspell with dictionaries:
nix-build -E 'with import <nixpkgs> {}; aspellWithDicts (d: [d.en])'
I want to use the result of this build as a dependency in another local package (foo). This is how I'm currently achieving this:
./pkgs/aspell-with-dicts/default.nix:
with import <nixpkgs> {};
aspellWithDicts (d: [d.en])
./pkgs/foo/default.nix:
{stdenv, aspellWithDicts, hello}:
stdenv.mkDerivation rec {
name = "foo";
buildInputs = [ aspellWithDicts hello ];
}
./custom-packages.nix:
{ system ? builtins.currentSystem }:
let
pkgs = import <nixpkgs> { inherit system; };
in
rec {
aspellWithDicts = import ./pkgs/aspell-with-dicts;
foo = import ./pkgs/foo {
aspellWithDicts = aspellWithDicts;
hello = pkgs.hello;
stdenv = pkgs.stdenv;
};
}
Running the shell works as expected: nix-shell ./custom-packages.nix -A foo --pure
So my solution works, but could this outcome be achieved in a more succinct idiomatic way?
Do you need to build foo? What in foo you will use?
Assume you only want to use the shell via nix-shell and not want to build/install anything using nix-build or nix-env -i, this should works.
The following shell.nix
with import <nixpkgs> {};
with pkgs;
let
myAspell = aspellWithDicts (d: [d.en]);
in
stdenv.mkDerivation {
name = "myShell";
buildInputs = [myAspell hello];
shellHooks = ''
echo Im in $name.
echo aspell is locate at ${myAspell}
echo hello is locate at ${hello}
'';
}
will give you a shell with aspell and hello
$ nix-shell
Im in myShell.
aspell is locate at /nix/store/zcclppbibcg4nfkis6zqml8cnrlnx00b-aspell-env
hello is locate at /nix/store/gas2p68jqbzgb7zr96y5nc8j7nk61kkk-hello-2.10
If it is the case that foo have some code to build and install.
mkDerivation in foo/default.nix must have src field which could be src = ./.; or something like fetchurl or fetchFromGithub (see document for examples).
Then you can use callPackages or import (depends on how the nix expression was written) with foo/default.nix as argument to bring what foo provided to use in this shell.
If you try to build this shell.nix (or foo/default.nix) it will failed with missing src
$ nix-build shell.nix
these derivations will be built:
/nix/store/20h8cva19irq8vn39i72j8iz40ivijhr-myShell.drv
building path(s) ‘/nix/store/r1f6qpxz91h5jkj7hzrmaymmzi9h1yml-myShell’
unpacking sources
variable $src or $srcs should point to the source
builder for ‘/nix/store/20h8cva19irq8vn39i72j8iz40ivijhr-myShell.drv’ failed with exit code 1
error: build of ‘/nix/store/20h8cva19irq8vn39i72j8iz40ivijhr-myShell.drv’ failed
To make this code more idiomatic, I have the following suggestions:
callPackage
Use the pkgs.callPackage function. It will take care of passing the arguments that your derivation needs. This is why many files in NixPkgs look like { dependency, ...}: something. The first argument is the function you want to inject dependencies into and the second argument is an attribute set that you can use to pass some dependencies manually.
By using callPackage you do not need to import <nixpkgs> {}, so your code will be easier to use in new contexts <nixpkgs> can't be used and it will evaluate a bit faster because it has to evaluate the NixPkgs fix-point only once.
(Of course you have to import <nixpkgs> once to get started, but after that, there should be no need.)
with
In pkgs/aspell-with-dicts/default.nix you use a with keyword, which is ok, but in this case it does not really add value. I prefer to refer to variables explicitly, so prefer to read pkgs.something when it's used once or twice, or inherit (pkgs) something if it's use more often. This way the reader can easily determine where a variable comes from.
I do use it when experimenting with unfamiliar packages or functions, because maintenance is not an issue then.
pkgs/aspell-with-dicts/default.nix
Unless you expect that your instantiation of aspell is something you want to reuse, it's probably easier to just construct it where you use it.
If you do expect to reuse a specific configuration of a package, you might want to make it a first class package by constructing it in an overlay.
That's it. I think the most important point is avoiding <nixpkgs> and apart from that it's already pretty idiomatic.
I don't know what your mysterious foo is, but if it's open source, please consider upstreaming it into NixPkgs. Nix has a very welcoming community in my experience.
I have the following shell.nix (to setup my development environment, no NixOS):
with import <nixpkgs> {};
stdenv.mkDerivation {
name = "my-shiny-project";
buildInputs = [
jq
nodejs-6_x
#postgis {version="2.3.1";}
#postgis ("2.3.1")
#postgis "2.3.1"
postgresql96
zsh
];
shellHook = ''
export SHELL=zsh
export PATH="$PWD/node_modules/.bin/:$PATH"
'';
}
PostGIS expects a version parameter. I'm not sure how to pass that parameter along. Whether I use postgis{version="2.3.1";}, postgis("2.3.1") or postgis "2.3.1", I receive the following error:
error: cannot coerce a set to a string, at /nix/store/0rj9y7gvzzahp93cvdmrwc2v2aznh61p-nixpkgs-18.03pre118061.69607d7662/nixpkgs/pkgs/stdenv/generic/make-derivation.nix:98:11
In the Nameless and single parameter section of functions and imports Nix pill, I see the syntax for calling a function is simply NAME PARAM.
What do I have to add to buildInputs to get PostGIS installed for that specific version of PostgreSQL?
Recently it became possible to use postgresql.withPackages:
with import <nixpkgs> {};
mkShell {
buildInputs = [
jq
nodejs
( postgresql11.withPackages (p: [ p.postgis ]) )
zsh
];
shellHook = ''
export SHELL=${zsh}/bin/zsh
export PATH="${builtins.toPath ./.}/node_modules/.bin/:$PATH"
export PGDATA=${builtins.toPath ./.}/pg
export PGHOST=$PGDATA
pg_ctl initdb
pg_ctl -o "-p 5555 -k $PGDATA" start
psql -p 5555 postgres -c 'create extension postgis' || true
'';
}
This is truly a stab in the dark given that postgresql is designed to be used as a service, but it's only available to you as a package since you're not on NixOS. But try this:
with import <nixpkgs> {};
stdenv.mkDerivation {
name = "my-shiny-project";
buildInputs = [
jq
nodejs-6_x
postgis.v_2_3_1
postgresql96
zsh
];
shellHook = ''
export SHELL=zsh
export PATH="$PWD/node_modules/.bin/:$PATH"
'';
}
Explanation
The postgis package doesn't produce a derivation, so it's output cannot be used directly. Instead it produces a set with attributes for two versions: 2.3.1 and 2.4.0.
Caveat
The above may not work at all. The issue is that postgis is normally provided to postgresql via the extraPlugins attribute like so:
services.postgresql.extraPlugins = [ (pkgs.postgis.override { postgresql = pkgs.postgresql95; }).v_2_3_1 ];
This causes postgresql to be installed in such a way that it can see the postgis library. However, this is done at the service level, not the package leve, and services are only available to NixOS. So if the above doesn't work, try this hunk of code:
with import <nixpkgs> {};
let
pg = postgresql96;
postgresqlWithPlugins =
buildEnv {
name = "postgresql-and-plugins-${(builtins.parseDrvName pg.name).version}";
paths = [ pg pg.lib (postgis.override { postgresql = pg; }).v_2_3_1) ];
buildInputs = [ makeWrapper ];
postBuild =
''
mkdir -p $out/bin
rm $out/bin/{pg_config,postgres,pg_ctl}
cp --target-directory=$out/bin ${pg}/bin/{postgres,pg_config,pg_ctl}
wrapProgram $out/bin/postgres --set NIX_PGLIBDIR $out/lib
'';
};
in
stdenv.mkDerivation {
name = "my-shiny-project";
buildInputs = [
jq
nodejs-6_x
postgresqlWithPlugins
zsh
];
shellHook = ''
export SHELL=zsh
export PATH="$PWD/node_modules/.bin/:$PATH"
'';
}
Basically, this is an untested port of the extraPlugins implementation.
I'm writing a .nix expression to be used primarily by nix-shell. I'm not sure how to do that. Note this is not on NixOS, but I don't think that is very relevant.
The particular example I'm looking at is that I want to get this version-dependent name that looks like:
idea-ultimate = buildIdea rec {
name = "idea-ultimate-${version}";
version = "2017.2.2"; /* updated by script */
description = "Integrated Development Environment (IDE) by Jetbrains, requires paid license";
license = stdenv.lib.licenses.unfree;
src = fetchurl {
url = "https://download.jetbrains.com/idea/ideaIU-${version}-no-jdk.tar.gz";
sha256 = "b8eb9d612800cc896eb6b6fbefbf9f49d92d2350ae1c3c4598e5e12bf93be401"; /* updated by script */
};
wmClass = "jetbrains-idea";
update-channel = "IDEA_Release";
};
My nix expression is the following:
let
pkgs = import <nixpkgs> {};
stdenv = pkgs.stdenv;
# idea_name = assert pkgs.jetbrains.idea-ultimate.name != ""; pkgs.jetbrains.idea-ultimate.name;
in rec {
scalaEnv = stdenv.mkDerivation rec {
name = "scala-env";
builder = "./scala-build.sh";
shellHook = ''
alias cls=clear
'';
CLANG_PATH = pkgs.clang + "/bin/clang";
CLANGPP_PATH = pkgs.clang + "/bin/clang++";
# A bug in the nixpkgs openjdk (#29151) makes us resort to Zulu OpenJDK for IDEA:
# IDEA_JDK = pkgs.openjdk + "/lib/openjdk";
# PATH = "${pkgs.jetbrains.idea-ultimate}/${idea_name}/bin:$PATH";
IDEA_JDK = /usr/lib/jvm/zulu-8-amd64;
# IDEA_JDK = /opt/zulu8.23.0.3-jdk8.0.144-linux_x64;
# IDEA_JDK = /usr/lib/jvm/java-8-openjdk-amd64;
buildInputs = with pkgs; [
ammonite
boehmgc
clang
emacs
jetbrains.idea-ultimate
less
libunwind
openjdk
re2
sbt
stdenv
unzip
zlib
];
};
}
I have commented out setting PATH as it depends on getting idea_name in the let-clause. As an interesting side note, as is, this does not fail if I leave it uncommented but causes a very bizarre error when executing nix-shell about not being able to execute bash. I've also tried the more simple case of let idea_name = pkgs.jetbrains.idea-ultimate.name; but this fails later on when idea_name is used in setting PATH since idea_name ends up being undefined.
Update:
I began exploring with nix-instantiate, but the derivation of interest seems empty:
[nix-shell:/nix/store]$ nix-instantiate --eval --xml -E "((import <nixpkgs> {}).callPackage ./3hk87pqgl2qdqmskxbhy23cyr24q8g6s-nixpkgs-18.03pre114739.d0d905668c/nixpkgs/pkgs/applications/editors/jetbrains { }).idea-ultimate";
<?xml version='1.0' encoding='utf-8'?>
<expr>
<derivation>
<repeated />
</derivation>
</expr>
If your intent is to get idea-ultimate into nix-shell environment, then just include that package to buildInputs. I see it's already included, so it should already be present in your PATH.
BTW, you can extend your shellHook and export PATH and other variables rather from there, where you have full bash. Why would you do it from bash? Less copying. When you specify
IDEA_JDK = /usr/lib/jvm/zulu-8-amd64;
in Nix, the file /usr/lib/jvm/zulu-8-amd64 get's copied to nix store and IDEA_JDK is set to point to file in /nix/store. Was that your intent?
Regarding nix-instantiate:
$ nix-instantiate --eval -E 'with import <nixpkgs>{}; idea.pycharm-community.outPath'
"/nix/store/71jk0spr30rm4wsihjwbb1hcwwvzqr4k-pycharm-community-2017.1"
but you still have to remove doublequotes (https://gist.github.com/danbst/a9fc068ff26e31d88de9709965daa2bd)
Also, nitpick, assert pkgs.jetbrains.idea-ultimate.name != ""; can be dropped as it's impossible to have empty derivation name in Nix.
And another nitpick. You'll soon find very incovenient to launch IDE from shell every time. It seems a good idea to specify, that some package is used for development, but nix-shell doesn't work well for non-cli applications. Not to mention occasional problems with Nix GC and nix-shell. You'd better install IDE globally or per-user, it is better long-term solution.
[ADDENDUM]
You are looking for this (dev-environment.nix):
with import <nixpkgs> { };
buildEnv {
name = "my-super-dev-env";
paths = [
#emacs
nano
idea.pycharm-community
];
buildInputs = [ makeWrapper ];
postBuild = ''
for f in $(ls -d $out/bin/*); do
wrapProgram $f \
--set IDEA_JDK "/path/to/zulu-jdk" \
--set CLANG_PATH ... \
--set CLANCPP_PATH ...
done
'';
}
which you install using nix-env -if ./dev-environment.nix. It will wrap your programs with those env vars, without polluting your workspace (you can pollute it further using nix-shell with shell hook, if you want).
In:
n = import <nixpkgs> {};
n contains an attribute n.pkgs, which also seems to contain all the available packages. What's the difference then between n and n.pkgs?
It seems it's related to the fixpoint semantics of Nix configuration and the availability to override some packages from nixpkgs, but I can't really wrap my head around it and find a clear distinction.
import <nixpkgs> {} gives you a pristine instance of Nixpkgs, i.e. without any user-configuration applied.
(import <nixpkgs> {}).pkgs gives you a version of Nixpkgs that has user-configured settings and overrides from ~/.nixpkgs/config.nix applied.
There is no difference between them. If you put this in your ~/.config/nixpkgs/config.nix:
{
packageOverrides = self: { newAttr = "testing testing"; };
}
... you'll see that these two commands have the same output:
$ nix-instantiate --eval -E 'with import <nixpkgs> {}; newAttr'
"testing testing"
$ nix-instantiate --eval -E 'with import <nixpkgs> {}; pkgs.newAttr'
"testing testing"
This is true for Nix 2.1.3 as well as for Nix 1.11.16.
The purpose of the pkgs alias inside nixpkgs is so that callPackage can fill in the pkgs parameter for a nix function that requires it.