I haven't found any info about handling temporary files in Nix derivations.
I found $TMP and $TMPDIR env vars, but they both point just to /tmp, which is system global.
{
pkgs ? import <nixpkgs> {}
}:
pkgs.stdenv.mkDerivation {
pname = "show-tmp"
version = "0.1.0";
src = ./.;
configurePhase = ''
echo "tmp = $tmp; TMP = $TMP; TMPDIR = $TMPDIR"
'';
buildPhase = '':'';
installPhase = '':'';
}
Variable $tmp is not defined inside mkDerivation. I would expect such thing, because other derivation scope vars follow low case style such as $out.
The problem with /tmp is obvious - it is global directory.
I need to worry about collisions and cleaning.
My derivation-hook archives a big folder tree.
If you're on Linux, don't worry. The Nix sandbox will give your build its own empty /tmp. It is removed when your derivation is done.
On macOS, $TMP and $TMPDIR are taken care of but /tmp is a potential problem.
How Nix creates a private /tmp on Linux
macOS Darwin, where Nix was installed in early 2020:
nix-build --expr 'with import <nixpkgs> {}; runCommand "hi" {} "echo a > /tmp/a; ls -al /tmp; sleep 1;"'
ls -al /private/tmp/
...
-rw-r--r-- 1 nixbld1 wheel 2 May 19 12:49 a
...
Related
This is the sequel of this question.
I've a bash list of command that generated a file nix directory when these commands are executed.
mkdir nix
rm -fr node_module
node2nix -16 --development --input package.json --lock package-lock.json --node-env ./nix/node-env.nix --composition ./nix/default.nix --output ./nix/node-package.nix
I have flake.nix file that use nix to create an envirnoment.
{
inputs = {
nixpkgs.url = "github:nixos/nixpkgs/nixos-unstable";
flake-utils.url = "github:numtide/flake-utils";
};
outputs = { self, nixpkgs, flake-utils, ... }:
flake-utils.lib.eachDefaultSystem (system:
let
pkgs = import nixpkgs { inherit system; };
#npm_pack = import (./nix );
npm_pack = import ./nix { inherit pkgs ;};
in with pkgs;{
#devShell = mkShell { buildInputs = [ npm_pack.package ];};
devShell = npm_pack.shell;
});
}
It is executed with this command:
nix develop --extra-experimental-features nix-command --extra-experimental-features flakes --ignore-environment
Is there a way to modify the flake.nix file to create the nix directory and then do the work it has to do with nix directory.
I know that I could ONE create bash file (see the in answer why I don't like it)
In order to create the flake.nix I thinking about using something like a sheelhook in the beginning.I need too to be sure that node and node2nix are installed. Therefore I need those line
node2nix.url ="github:svanderburg/node2nix"; # in the input
nodejs = pkgs.nodejs-16_x; #in the output
running this script works
#/nix/store/4xw8n979xpivdc46a9ndcvyhwgif00hz-bash-5.1-p16/bin/bash
#nix-shell -p node2nix nodejs stdenv --pure
npm init -y
npm install node-gyp-build
mkdir nix;
rm -fr node_modules ;
node2nix -16 --development --input package.json --lock package-lock.json --node-env ./nix/node-env.nix --composition ./nix/default.nix --output ./nix/node-package.nix
nix develop --extra-experimental-features nix-command --extra-experimental-features flakes --ignore-environment
but it doesn't work if somebody has not the bash in the directory than my bash.
Therefore I prefer a solution consisting of implementing a big flake.nix because I'm sure it works if nix is installed
and it need to files : the flake.nix and the script.sh
I'm trying to use nix-shell with a shell.nix file to get a clean development environment but I don't know how to change the location of the temporary build directories.
The buildInputs packages are built in /tmp but this path doesn't have enough space and I get an error: [Errno 28] No space left on device during the build.
I tried running nix-shell with a modified TMPDIR environment variable but it only affects the location of the nix-shell temporary files. The nix-build files are still put in /tmp.
I also tried to export a new value for TMPDIR in the shellHook but it doesn't work.
How can I change the TMPDIR of nix-build when it's started by nix-shell?
Here's my shell.nix:
let
pkgs = import <nixpkgs> {};
in
pkgs.mkShell {
name = "something";
buildInputs = with pkgs; [
python38
python38Packages.pytorchWithCuda
];
shellHook = ''
'';
}
I got the answer on the NixOS forum:
If this is a mutli-user install, you need to modify the Nix daemon’s TMPDIR.
To do that on my system I created a /etc/systemd/system/nix-daemon.service.d/override.conf with:
[Service]
Environment=TMPDIR=/var/tmp/nix-daemon
According to https://nixos.org/nixpkgs/manual/#ssec-unpack-phase I can use a zip file as a 'source'.
with import <nixpkgs> {};
stdenv.mkDerivation {
name = "test-environment";
nativeBuildInputs = [ unzip ];
buildInputs = [ unzip ];
src = "./x.zip";
}
nix-build
these derivations will be built:
/nix/store/431i1riasgh2hjs5rd9azfh0ssjlg7qj-test-environment.drv
building '/nix/store/431i1riasgh2hjs5rd9azfh0ssjlg7qj-test-environment.drv'...
unpacking sources
unpacking source archive ./x.zip
unzip: cannot find or open ./x.zip, ./x.zip.zip or ./x.zip.ZIP.
unzip: cannot find or open ./x.zip, ./x.zip.zip or ./x.zip.ZIP.
do not know how to unpack source archive ./x.zip
builder for '/nix/store/431i1riasgh2hjs5rd9azfh0ssjlg7qj-test-environment.drv' failed with exit code 1
error: build of '/nix/store/431i1riasgh2hjs5rd9azfh0ssjlg7qj-test-environment.drv' failed
There is a x.zip file in my working directory. Why is it failing?
Update:
If I set unpackCmd = "ls -altr"; I see the following output:
total 16
drwxr-x--- 9 nobody nixbld 4096 May 26 12:39 ..
drwx------ 2 nixbld nixbld 4096 May 26 12:39 .
-rw-r--r-- 1 nixbld nixbld 5806 May 26 12:39 env-vars
Why isn't my zip file appearing in that context?
You don't want your Nix derivation to reference local filesystem elements! If anything hasn't been hashed and put into the store, your build is impure, and won't work in sandboxed mode.
Change it to src = ./x.zip, and the file will be hashed and added to the store, and src will be set to a /nix/store/.....-x.zip path.
with import <nixpkgs> {};
stdenv.mkDerivation {
name = "test-environment";
nativeBuildInputs = [ unzip ];
buildInputs = [ unzip ];
src = ./x.zip; ## NO QUOTES!
}
I get the following collision error when attempting to build an environment that, as far as I can see, shouldn't have a collision (in this case, scala-env depends on ideaLocal, so it shouldn't conflict with it):
...
idea-IU-172.4155.36/bin/libyjpagent-linux.so
idea-IU-172.4155.36/bin/libyjpagent-linux64.so
idea-IU-172.4155.36/help/ideahelp.jar
idea-IU-172.4155.36/lib/libpty/linux/x86/libpty.so
idea-IU-172.4155.36/lib/libpty/linux/x86_64/libpty.so
idea-IU-172.4155.36/bin/format.sh
idea-IU-172.4155.36/bin/fsnotifier
idea-IU-172.4155.36/bin/fsnotifier-arm
idea-IU-172.4155.36/bin/fsnotifier64
idea-IU-172.4155.36/bin/idea.sh
idea-IU-172.4155.36/bin/inspect.sh
idea-IU-172.4155.36/bin/printenv.py
idea-IU-172.4155.36/bin/restart.py
building path(s) ‘/nix/store/29g92lnpi0kywy9x7vcgl9yivwa2blm6-scala-env’
created 696 symlinks in user environment
building path(s) ‘/nix/store/qrnbff8nhpmxlzkmv508aymz5razbhgf-user-environment’
Wide character in die at /nix/store/64jc9gd2rkbgdb4yjx3nrgc91bpjj5ky-buildenv.pl line 79.
collision between ‘/nix/store/75sz9nklqmrmzxvf0faxmf6zamgaznfv-idea-local/bin/idea’ and ‘/nix/store/29g92lnpi0kywy9x7vcgl9yivwa2blm6-scala-env/bin/idea’; use ‘nix-env --set-flag priority NUMBER PKGNAME’ to change the priority of one of the conflicting packages
builder for ‘/nix/store/8hp5kdicxy9i02fa07vx85p1gvh4i1bq-user-environment.drv’ failed with exit code 255
error: build of ‘/nix/store/8hp5kdicxy9i02fa07vx85p1gvh4i1bq-user-environment.drv’ failed
Here is the nix expression (most of which can be ignored, but it isn't too long so I'll paste the whole thing):
with import <nixpkgs> { };
let
ideaLocal = stdenv.mkDerivation {
name = "idea-local";
buildInputs = [ ];
builder = builtins.toFile "builder.sh" ''
source $stdenv/setup
mkdir -p $out/bin
tar zxvf $src -C $out/
ln -sf $out/idea-IU* $out/idea
ln -sf $out/idea/bin/idea.sh $out/bin/idea
'';
shellHook = ''
IDEA_JDK=/usr/lib/jvm/zulu-8-amd64
'';
src = fetchurl {
url = https://download.jetbrains.com/idea/ideaIU-2017.2.4-no-jdk.tar.gz;
sha256 = "15a4799ffde294d0f2fce0b735bbfe370e3d0327380a0efc45905241729898e3";
};
priority = 5;
};
in
buildEnv {
name = "scala-env";
paths = [
ammonite
boehmgc
clang
dbus # needed non-explicitly by vscode
emacs
git
# idea.idea-ultimate # disabled temporarily
ideaLocal
less
libunwind
openjdk
openssh
re2
rsync
sbt
stdenv
syncthing # for syncrhonizing data between containers
tmux
unzip
vscode
zlib
];
# builder = builtins.toFile "builder.sh" ''
# source $stdenv/setup
# mkdir -p $out
# echo "" > $out/Done
# echo "Done setting up Scala environment."
# '';
buildInputs = [ makeWrapper ];
# TODO: better filter, use ammonite script?:
postBuild = ''
for f in $(ls -d $out/bin/* | grep "idea"); do
sed -i '/IDEA_JDK/d' $f
wrapProgram $f \
--set IDEA_JDK "/usr/lib/jvm/zulu-8-amd64" \
--set CLANG_PATH "${clang}/bin/clang" \
--set CLANCPP_PATH "${clang}/bin/clang++"
done
'';
}
Edit:
(DevContainer)which idea
/home/brandon/.nix-profile/bin/idea
(DevContainer)ls -last /home/brandon/.nix-profile/bin/idea
4 lrwxrwxrwx 1 brandon brandon 63 Jan 1 1970 /home/brandon/.nix-profile/bin/idea -> /nix/store/75sz9nklqmrmzxvf0faxmf6zamgaznfv-idea-local/bin/idea
So it looks like ideaLocal is being imported as an environment - what's the right way to just have it installed as a package that is a dependency of scalaEnv?
Apparently the solution was to specify the profile, which I think of as the name of the environment, so that both environments aren't installed simultaneously:
nix-env -if scala-default.nix -p scala-env
Is there a way to override scripts and/or configs in an existing derivation without having to recompile the whole package?
I'd like to create a new version of gnome-session with modified $out/share/gnome-session/sessions/gnome.session modified. Using overridePackage I can change the preFixup phase, but this causes the whole gnome-session package to be recompiled.
A simple solution to problem is creating a new derivation without any sources. The trick is to create links to every file and directory in the base derivation, except for any file(s) in need of modification. These files are handled explicitly according to needs.
The snippet below shows how to create a new gnome-session with xmonad instead of gnome-shell.
{ nixpkgs ? import <nixpkgs> {} }:
let
inherit (nixpkgs) pkgs;
in
pkgs.stdenv.mkDerivation {
name = "gnome-session";
builder = pkgs.writeText "builder.sh" ''
# source standard environment
. $stdenv/setup
# shorthands
refpkg=${pkgs.gnome3.gnome_session}
file=share/gnome-session/sessions/gnome.session
# create output dirs for new derivation
mkdir -p $out/share
mkdir -p $out/share/gnome-session/sessions
# link unchanged files from the original gnome-session
ln -sf $refpkg/bin $out
ln -sf $refpkg/libexec $out
find $refpkg/share -maxdepth 1 \
-not -name gnome-session -exec ln -sf {} $out/share \;
# change gnome-shell to xmonad
sed 's/org.gnome.Shell/xmonad/' $refpkg/$file > $out/$file
'';
# make sure gnome-session is installed before deriving it!
buildInputs = [ pkgs.gnome3.gnome_session ];
}