massaging packages that install files outside of nix-store - nix

I'm using nix package-manager on macOS (Sierra).
My intention is to write a nix expression that will install the existing fish nix package along with the Bass fish plugin.
There are no existing expressions in nixpkgs for Bass, but the git repo contains a Makefile. This Makefile attempts to copy files to the $HOME dir. This is a problem as installing files outside of the nix-store is clearly not desirable and $HOME is not set when I build my package.
I can recognise why it's not desirable for nix packages to install files outside of the nix-store - in functional programming terms it's akin to a side-effect. But I'm also not clear on how to solve my problem:
By default Fish requires plugins such as Bass to be installed under $HOME/.config/fish/. Fish does provide a means to customise the config path by specifying the environment variable XDG_CONFIG_HOME. So I was thinking of doing something like this:
Create an expression for Bass patching the Makefile to install the files under $out.
Create an expression that installs fish and uses Bass as a build input. Use wrapProgram to set XDG_CONFIG_HOME pointing to the Bass install path in the nix-store.
Does this sound like the right approach? Are there alternative/better ways of solving this?
Thanks

This is the solution that I have gone with:
Expression for bass:
nix_local/pkgs/fish_plugins/bass/default.nix
{stdenv, fetchFromGitHub}:
let
version = "0.0.1";
in
stdenv.mkDerivation rec {
name = "bass-${version}";
src = fetchFromGitHub {
owner = "edc";
repo = "bass";
rev = "1fbf1b66f52026644818016015b8fa9e0f639364";
sha256 = "12bp8zipjbikasx20yz29ci3hikw0ksqlbxbvi2xgi4g6rmj7pxp";
};
patchPhase = ''
substituteInPlace Makefile --replace \
"~/.config/fish" \
$out/.config/fish
'';
}
Expression for fish_with_config:
nix_local/pkgs/fish_with_config/default.nix
{stdenv, fish, bass, makeWrapper}:
let
version = "0.0.1";
in
stdenv.mkDerivation rec {
name = "fish-with-config-${version}";
src = ./.;
buildInputs = [fish bass makeWrapper];
installPhase = ''
mkdir -p $out/.config/fish/functions
cp -r $src/.config/* $out/.config
cp -r ${bass}/.config/fish/functions/* \
$out/.config/fish/functions/
mkdir -p $out/bin
ln -s ${fish}/bin/fish $out/bin/fish
wrapProgram $out/bin/fish --set XDG_CONFIG_HOME "$out/.config"
'';
}
The Fish program is wrapped in order for it's config to be stored in the nix-store. This enables us to symlink the functions from Bass and also copy any additional config files from the local $src dir. Additional plugins could be symlinked in the same way.
The local src dir for the derivation contains the following files:
pkgs/fish_with_config
├── .config
│   └── fish
│   ├── fishd.8c8590486f8c
│   └── functions
└── default.nix
The .config/fish/fishd.8c8590486f8c file is a "universal variable file" which Fish requires in order to operate. In a standard Fish installation this file is stored under ~/config/fish/ and is created the first time you enter interactive mode. The contents of this file would typically change over time as users interact with Fish settings.
The fish_with_config derivation stores the Fish config in the nix-store, which means it can't be modified at a latter date (not writable). This means all the config settings need to be done upfront as any attempts by the user to modify the settings will result in permission errors - this is obviously a little inconvenient, but not a show stopper for me.
It's probably worth noting that the universal variable file may change with different releases of Fish and as such if I was to build fish_with_config with a newer version of Fish I would first determine it's default content by running fish in a nix-shell and inspecting the auto generated file under ~/config/fish/.
In summary the above works nicely, I have access to bass and any additional user defined functions I choose to "bake in" (pkgs/fish_with_config/.config/fish/functions).
If you see anything that could be improved or handled more idiomatically let me know.

Related

QNX 6.5 - Repack ifs root stage 2 for MIB2

I have been working on a custom patch for my car with MIB2 unit for some time and would like to try my modified MIBRoot file.
The command I used to repack everything is:
mkxfs -t ifs -nn -o ./ -r / ./mkifs_attributes.txt ./ifs_extracted ./patched_ifs.ifs
But the problem is that the repackaging is not working properly, because the mkifs_attribute.txt file used is probably not fine for my MIB2 or I have a version that is not correct.
In fact, when I do a:
diff -r ./original/ifs_extracted ./repack/ifs_extracted
all files in ./repack/ifs_extracted are different from the originals, when instead only the MIBRoot file should be different.
my mkifs_attribute contain this:
#Porsche PCM4/MIB2 ifs-root (stage 2) attributes file
#Before packing FS: EXPORT QNX_TARGET="/"
#Command to pack: mkxfs -t ifs -nn -o ./ -r / attributes.txt /unpacked_fs /packed_fs.ifs
[-followlink] #Do not resolve links
[compress=2] #LZO Compression
[-bigendian] #Little Endian (ARMLE)
[-autolink] #Do not auto link shared objects
[perms=777] #chmod 777
[uid=0 gid=0] #chown root:root
#You can add a script to be executed after mount here, one line only
#[+script] .script = {
#touch /dev/shmem/AudioFadedIn; touch /dev/shmem/production_mode}
#Symlinks. Add lines here to match your dumpifs symlinks
#[type=link] dest_file=source_file
[type=link] /lib/SPC_configuration.so=/lib/libSPC_configuration.so
[type=link] /lib/AudioConfig_ARM.so=/lib/libAudioConfig_ARM.so
[type=link] /lib/AudioConfig_DSP.so=/lib/libAudioConfig_DSP.so
[type=link] /usr/bin/flashunlock=/usr/bin/flashlock
[type=link] /usr/bin/libi2c-inic-master.so=/lib/libi2c-inic-master.so.1.0
[type=link] /lib/librdshbfpga.so.1.0.0=/lib/librdshbfpga.so
[type=link] /lib/libSysMoCCAFrameworkSharedSo.so=/lib/libSysMoCCAFrameworkSharedSo.so.6
[type=link] /lib/AudioConfig_DSP.so=/lib/libAudioConfig_DSP.so
[type=link] /lib/liba2itodspipc.so=/lib/liba2itodspipc.so.1
Or am i wrong something?
If anyone has a correct mkxfs and mkxfs_attribute.txt could you please provide it to me so I can give it a try?

How can I nix-env install a derivation from a nix expression file?

I've got a default.nix file that builds a derivation (at least my understanding of it).
{ nixpkgs ? import <nixpkgs> {}, compiler ? "ghc864" } :
nixpkgs.pkgs.haskell.packages.${compiler}.callCabal2nix "bhoogle" (./.) {}
I can successfully nix-build this. Is there a way I can install this into my user profile with nix-env directly? For example something like nix-env -i -f default.nix.
Otherwise I need to define a package in my system profile with something like:
example = pkgs.callPackage /home/chris/example/default.nix {};
Quite literally my initial guess (thanks #Robert):
nix-env -i -f default.nix
Documented in nix --help:
--file / -f path
Specifies the Nix expression (designated below as the active Nix expression) used by the --install, --upgrade, and --query
--available operations to obtain
derivations. The default is ~/.nix-defexpr.
If the argument starts with http:// or https://, it is interpreted as the URL of a tarball that will be downloaded and
unpacked to a temporary location. The tarball
must include a single top-level directory containing at least a file named default.nix.

How to record a reproducible profile in nix (especially from nix-env)?

So, finally starting to get a stable nix environment that I can basically do all of my development in. Hooray!
Now I want to make it reproducible, as in yarn.lock (for those familiar with npm/yarn in javascript land) or Pipfile.lock (very similar for Python).
Basically the idea is that I would have a way to generate a similar lock file whenever I run nix-env -if my-env.nix, or after running this command, if that is how it would work. From this lock file, I could then exactly restore my nix profile, down to the exact versions of dependencies and sub-dependencies of the installed profile. This could be checked into git or whatever after testing out new improvements, and so a record of the environment would be maintained.
It seems to me this would be one of the most obvious use cases for Nix, and one of the major advantages over just using Docker (though the two aren't mutually exclusive), so I apologize if I've missed some relevant documentation.
What you're probably looking for is a shell.nix file like this:
let
pkgs = import (fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/696c6bed4e8e2d9fd9b956dea7e5d49531e9d13f.tar.gz";
sha256 = "1v3yrpj542niyxp0h3kffsdjwlrkvj0mg4ljb85d142gyn3sdzd4";
}) {};
in pkgs.mkShell {
buildInputs = with pkgs; [
git
hello
];
}
Upon calling nix-shell (which by default uses the shell.nix file in the current directory), you'll be in an environment with git and hello from the specific given nixpkgs revision. This can be reproduced among all nix users (okay almost*). I can really recommend using shell.nix files for all development.
Alternatively, there's also the lesser known -r flag to nix-env, which states that
--remove-all, -r
Remove all previously installed packages first. This is equivalent to running nix-env -e '.*' first, except that everything happens in a single transaction.
You can effectively use it to replace the stateful ~/.nix-profile/manifest.nix. Create a file env.nix containing:
let
pkgs = import <nixpkgs> {};
in {
inherit (pkgs) git hello;
}
Now running nix-env -ir env.nix will install exactly git and hello and remove everything else, so you can reproduce your nix-env installations with this single file. To install additional things: Add it to the file and run the command again. You can also pin nixpkgs to a specific version as in the file above to not care about your nix-channel setup (which is also stateful).
Edit: It's also possible to get some packages from your channel and some of a specific nixpkgs revision:
let
pkgs = import <nixpkgs> {};
fixed = import (fetchTarball {
url = "https://github.com/NixOS/nixpkgs/archive/696c6bed4e8e2d9fd9b956dea7e5d49531e9d13f.tar.gz";
sha256 = "1v3yrpj542niyxp0h3kffsdjwlrkvj0mg4ljb85d142gyn3sdzd4";
}) {};
in {
inherit (pkgs) git;
inherit (fixed) hello;
}
*: The nix config, overlays and your system (Linux/Mac) can still influence this. It's a good idea to use import <nixpkgs> { config = {}; overlays = []; } for development to avoid this.
After many suggestions from folks on IRC, I was able to put together the following script, which takes a nix expression as its only argument, and copies the derivation file and records the nixpkgs versions locally while installing the specified environment:
#!/bin/bash
if [ -z "$1" ] || [ "${1: -4}" != ".nix" ] || [ ! -f "$1" ]
then
echo "No .nix file supplied"
exit -1
fi
ENV_DRV=$(nix-instantiate "$1")
cp "$ENV_DRV" ./env_backup.drv
chmod u+rw ./env_backup.drv
nix-env --set "$ENV_DRV"
NIXPKGS_VERSION=$(nix-instantiate --eval '<nixpkgs/lib>' -A version)
NIXOS_VERSION=$(nix-instantiate --eval '<nixos/lib>' -A version)
printf "nixpkgs: %s\\nnixos: %s" "$NIXPKGS_VERSION" "${NIXOS_VERSION}" > .nix_versions
Caveat: your nix expression should have the nix package in it, since we are using nix-env --set.
I haven't tried using a copied .drv file yet, but it somehow needs to be restored into the nix store; I've mainly included it as a last resort and for debugging. The output into .nix_versions should be more helpful, as these contain git commit hashes (after the last ".") that can be used to employ the correct revision of nixpkgs (thanks to infinisil on IRC):
pkgs = import "${(import <nixpkgs> {}).fetchFromGitHub { owner = "NixOS"; repo = "nixpkgs"; rev = "<your revision hash>"; sha256 = "<the hash of the output>"; }}" {}
To fill in the hash, either supply an incorrect hash to get the correct hash back, or just use the following: nix-prefetch-url --unpack github.com/nixos/nixpkgs/archive/<revision>.tar.gz.
Or, if manually checking out nixpkgs, you can just do e.g.:
with import ((builtins.getEnv "HOME") + "/workspace/nixpkgs") { }; # or:
with import "../nixpkgs" { }; # or similar
I haven't tested this sort of thing with nix-shell yet, but hope to do so soon.

Nix external software built/installed is not being found

I've just started using the Nix package manager on OSX and I'm attempting to create my first package for the pass binary (https://www.passwordstore.org) - which is not available in the Nixpkgs repository.
I'm attempting to specify a runtime dependency (getopt), however this doesn't appear to be available when the binary is used.
This is my packages's default.nix:
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
version = "1.7.1";
in {
pass = stdenv.mkDerivation rec {
name = "pass-${version}";
src = fetchurl {
url = "https://git.zx2c4.com/password-store/snapshot/password-store-1.7.1.tar.xz";
sha256 = "0scqkpll2q8jhzcgcsh9kqz0gwdpvynivqjmmbzax2irjfaiklpn";
};
buildInputs = [ stdenv makeWrapper];
installPhase = ''
make install PREFIX=$out/artifact
makeWrapper $out/artifact/bin/pass $out/bin/pass \
--set PATH ${stdenv.lib.makeBinPath [ getopt ]}
'';
meta = {
homepage = "https://www.passwordstore.org";
description = "The standard unix password manager";
license = stdenv.lib.licenses.gpl2Plus;
};
};
}
I can successfully build this package (nix-build --show-trace) and install it (nix-env -i ./result).
Listing the runtime dependencies for my package shows getopt listed:
nix-store -qR $(which pass)
...
/nix/store/c5swmygzc0kmvpq6cfkvwm2yz1k57kqy-getopt-1.1.4
However when I come to use the binary (pass init my-key) I get the following error:
/nix/store/...-pass-1.7.1/artifact/bin/pass: line 302:
/usr/local/bin/getopt: No such file or directory
Can anyone advise what I'm doing wrong?
Thanks
It looks like getopt gets a special treatment. The darwin.sh script looks for it using brew and port and falls back to /usr/local. That's why the (correct) wrapper has no effect.
So the solution seems to be, to make it look for getopt in PATH, which is provided by the wrapper script. You can probably make it as simple as GETOPT=getopt (which is similar to openbsd.sh)
For patching source code, see the NixPkgs documentation
After running nix-build, you should run cat result/bin/pass to look at your wrapper script and make sure it looks OK. It should be a shell script that sets the PATH to include getopt and then calls result/artifact/bin/pass.
Then try running the wrapper script. Note that the wrapper should be in result/bin, not result/artifact/bin.

Scons in Scratchbox: Can't find g++

I've been using sbox with a Make-based codebase with no problems. Now
I'm using a scons-based codebase, and am getting some odd problems.
It seems that within scratchbox, scons can't find g++. For example, it
attempts to execute things like:
o hello hello.c
When it should be doing:
g++ -o hello hello.c
So presumably its g++ string variable is empty. g++ is present and in
the PATH - "which g++" produces /scratchbox/compilers/bin/g++.
The same source builds fine outside of scratchbox, so it shouldn't be
a problem with scons or the codebase. There are no special environment
variables set outside of scratchbox when it works.
If I symbolically link /usr/bin/g++ to /scratchbox/compilers/bin/g++,
it gets a bit further (produces the correct-looking g++ commands) but
then upon executing them produces:
sb_gcc_wrapper (g++):
/scratchbox/compilers/arm-linux-cs2007q3-51sb3/bin/sbox-arm-none-linux-gnueabi-g++:
No such file or directory
The file listed is present.
PATH contains /scratchbox/compilers/bin, SBOX_REDIRECT_FROM_DIRS contains /usr/bin and SBOX_REDIRECT_TO_DIRS contains /scratchbox/compilers/bin, so I think it should be able to find it.
Any suggestions would be appreciated!
Thanks,
Ray
Edit: Perhaps related - it also can't find pkg-config unless I prepend the full path within the scons file
scons does not propagate the PATH environment variable, so testing e.g. 'which g++' doesn't help much.
Either set the compilers directly, e.g.
env['CXX'] = '/scratchbox/compilers/bin/g++'
Build your own explicit PATH
path = ['/scratchbox/compilers/bin/','/bin', '/usr/bin', '/sbin','/usr/sbin']
env = Environment(ENV = {'PATH' : path})
Or use the PATH env variable from your shell
import os
env = Environment(ENV = {'PATH' : os.environ['PATH']})

Resources