How to correctly use makeWrapper to change Yarn's --global-folder argument? - nix

I'm trying to write a derivation for a Go project that uses a Makefile to build its frontend with Yarn. As Yarn downloads files, I'm using a fixed derivation hash for networking.
However, Yarn attempts to write to its global package cache in /usr/local or /homeless-shelter, even though neither path exists.
To work around this issue, I tried to use makeWrapper to pass the --global-folder /tmp argument, but while the derivation runs it doesn't seem to be using it, as even --help is ignored.
How can I correctly wrap Yarn, or what other tool can I use to fix this?
cd frontend && yarn install --frozen-lockfile
yarn install v1.22.19
warning Skipping preferred cache folder "/homeless-shelter/.cache/yarn" because it is not writable.
warning Selected the next writable cache folder in the list, will be "/build/.yarn-cache-1000".
[1/4] Resolving packages...
[2/4] Fetching packages...
warning Cannot find a suitable global folder. Tried these: "/usr/local, /homeless-shelter/.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /homeless-shelter/.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /homeless-shelter/.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /homeless-shelter/.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /homeless-shelter/.yarn"
warning Cannot find a suitable global folder. Tried these: "/usr/local, /homeless-shelter/.yarn"
Done in 24.32s.
Error: ENOENT: no such file or directory, open '/homeless-shelter/.yarnrc'
make: *** [Makefile:218: frontend] Error 1
My config:
{ pkgs ? (
let
inherit (builtins) fetchTree fromJSON readFile;
inherit ((fromJSON (readFile ./flake.lock)).nodes) nixpkgs;
in
import (fetchTree nixpkgs.locked) {
overlays = [];
}
)
}:
with pkgs;
stdenv.mkDerivation rec {
pname = "someapp";
version = "v1.0";
src = fetchFromGitHub {
owner = "someowner";
repo = "someapp";
rev = "ffffffffffffffffffffffffffffffff";
sha256 = "sha256-fffffffffffffffffffffffffffffffff";
};
wrappedYarn = yarn.overrideAttrs (oldAttrs: {
buildInputs = oldAttrs.buildInputs or [] ++ [ makeBinaryWrapper ];
postInstall = oldAttrs.postInstall or "" + ''
wrapProgram $out/bin/yarn \
/*--append-flags "--global-folder /tmp/yarn --global-link /tmp/yarn2"*/
--append-flags "--help"
'';
});
buildInputs = [ go git wrappedYarn gcc gnumake ];
outputHashMode = "flat";
outputHashAlgo = "sha256";
outputHash = lib.fakeHash;
meta = with lib; {
description = "SomeApp";
homepage = "https://github.com/someowner/someapp";
platforms = platforms.linux;
};
}

Related

mkDerivation use cmake in buildPhase

I'm trying to create a derivation using nix files, and I'm a little stuck. A package I'm trying to install has a sh file in its repo to build it, and this sh file is running CMake with some arguments.
More specifically, this package is vcpkg.
Here's my vcpkg.nix file:
{ gcc11Stdenv, fetchFromGitHub, ninja, cmake, bash }:
gcc11Stdenv.mkDerivation {
name = "vcpkg-2021.05.12";
src = fetchFromGitHub {
owner = "microsoft";
repo = "vcpkg";
rev = "2021.05.12";
sha256 = "0290fp9nvmbqpgr33rnchn5ngsq4fdll2psvh0bqf0324w2qpsjw";
};
buildPhase = ''
./bootstrap-vcpkg.sh -useSystemBinaries
'';
}
When running it with nix-shell -p 'with (import <nixpkgs> {}); callPackage ./vcpkg.nix {}', I get this error:
configuring
no configure script, doing nothing
building
Could not find cmake. Please install it (and other dependencies) with:
sudo apt-get install cmake ninja-build
error: builder for '/nix/store/riq6vjdhv4z3xvzp8g597xjgwf2rvm03-vcpkg-2021.05.12.drv' failed with exit code 1;
last 9 log lines:
> unpacking sources
> unpacking source archive /nix/store/ycfd6vbgh3s1vy11hfb17b8x33rqj7aw-source
> source root is source
> patching sources
> configuring
> no configure script, doing nothing
> building
> Could not find cmake. Please install it (and other dependencies) with:
> sudo apt-get install cmake ninja-build
For full logs, run 'nix log /nix/store/riq6vjdhv4z3xvzp8g597xjgwf2rvm03-vcpkg-2021.05.12.drv'.
Then, I thought of making cmake and ninja available to buildPhase so the script can use those binaries by adding buildInputs = [cmake ninja];, but I then get this error:
configuring
fixing cmake files...
cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_SKIP_BUILD_RPATH=ON -DBUILD_TESTING=OFF -DCMAKE_INSTALL_LOCALEDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/share/locale -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/share/doc/vcpkg -DCMAKE_INSTALL_INFODIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/share/man -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/include -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/include -DCMAKE_INSTALL_SBINDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/bin -DCMAKE_INSTALL_NAME_DIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/854jyvxrvpdpbfn2zaba1v2qgqkxipyh-cctools-binutils-darwin-949.0.1/bin/strip -DCMAKE_RANLIB=/nix/store/854jyvxrvpdpbfn2zaba1v2qgqkxipyh-cctools-binutils-darwin-949.0.1/bin/ranlib -DCMAKE_AR=/nix/store/854jyvxrvpdpbfn2zaba1v2qgqkxipyh-cctools-binutils-darwin-949.0.1/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12
CMake Error: The source directory "/tmp/nix-build-vcpkg-2021.05.12.drv-0/source" does not appear to contain CMakeLists.txt.
Specify --help for usage, or press the help button on the CMake GUI.
error: builder for '/nix/store/76djky7f3xy6ym6v3qlmy941z0bjb8xw-vcpkg-2021.05.12.drv' failed with exit code 1;
last 9 log lines:
> unpacking sources
> unpacking source archive /nix/store/ycfd6vbgh3s1vy11hfb17b8x33rqj7aw-source
> source root is source
> patching sources
> configuring
> fixing cmake files...
> cmake flags: -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON -DCMAKE_BUILD_TYPE=Release -DCMAKE_SKIP_BUILD_RPATH=ON -DBUILD_TESTING=OFF -DCMAKE_INSTALL_LOCALEDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/share/locale -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/libexec -DCMAKE_INSTALL_LIBDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/lib -DCMAKE_INSTALL_DOCDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/share/doc/vcpkg -DCMAKE_INSTALL_INFODIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/share/info -DCMAKE_INSTALL_MANDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/share/man -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/include -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/include -DCMAKE_INSTALL_SBINDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/sbin -DCMAKE_INSTALL_BINDIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/bin -DCMAKE_INSTALL_NAME_DIR=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12/lib -DCMAKE_POLICY_DEFAULT_CMP0025=NEW -DCMAKE_OSX_SYSROOT= -DCMAKE_FIND_FRAMEWORK=LAST -DCMAKE_STRIP=/nix/store/854jyvxrvpdpbfn2zaba1v2qgqkxipyh-cctools-binutils-darwin-949.0.1/bin/strip -DCMAKE_RANLIB=/nix/store/854jyvxrvpdpbfn2zaba1v2qgqkxipyh-cctools-binutils-darwin-949.0.1/bin/ranlib -DCMAKE_AR=/nix/store/854jyvxrvpdpbfn2zaba1v2qgqkxipyh-cctools-binutils-darwin-949.0.1/bin/ar -DCMAKE_C_COMPILER=gcc -DCMAKE_CXX_COMPILER=g++ -DCMAKE_INSTALL_PREFIX=/nix/store/mp38jl4fkv0gqnqhz7a3agx4flwda59n-vcpkg-2021.05.12
> CMake Error: The source directory "/tmp/nix-build-vcpkg-2021.05.12.drv-0/source" does not appear to contain CMakeLists.txt.
> Specify --help for usage, or press the help button on the CMake GUI.
For full logs, run 'nix log /nix/store/76djky7f3xy6ym6v3qlmy941z0bjb8xw-vcpkg-2021.05.12.drv'.
It seems that adding cmake to buildInputs makes nix try to configure the project using cmake, but this is not what I'm trying to do since vcpkg don't simply have a CMakeLists.txt file in its repo.
Adding inherit cmake ninja; didn't seem to help.
You can prevent cmake from changing the configure phase by setting dontUseCmakeConfigure=true in your derivation.
From the nixpkgs manual:
6.7.29. cmake
Overrides the default configure phase to run the CMake command. By default, we use the Make generator of CMake. In addition, dependencies are added automatically to CMAKE_PREFIX_PATH so that packages are correctly detected by CMake. Some additional flags are passed in to give similar behavior to configure-based packages. You can disable this hook’s behavior by setting configurePhase to a custom value, or by setting dontUseCmakeConfigure. cmakeFlags controls flags passed only to CMake. By default, parallel building is enabled as CMake supports parallel building almost everywhere. When Ninja is also in use, CMake will detect that and use the ninja generator.

How to specify package/derivation runtime dependencies with Nix?

I'm making a haskell program and I'm setting buildInput like this to include pkgs.ffmpeg-full:
(myHaskellPackages.callCabal2nix "App" (./.) {}).overrideAttrs (oldAttrs: {
buildInputs = (oldAttrs.buildInputs or []) ++ [ pkgs.ffmpeg-full ];
})
However this seems to make the ffmpeg package accessible during build time only rather than runtime of the application.
What attribute do I need to set for ffmpeg-full to be available during runtime - being able to invoke the ffmpeg executable?
There is a section about runtime dependencies in nix pills but I don't understand that section, it doesn't make sense how it can always determine runtime dependencies by hashes alone? I mean if I reference an executable in a shell script - surely nix does not parse the shell script to determine the executable I reference. https://nixos.org/guides/nix-pills/automatic-runtime-dependencies.html#idm140737320205792
Something is different for runtime dependencies however. Build
dependencies are automatically recognized by Nix once they are used in
any derivation call, but we never specify what are the runtime
dependencies for a derivation.
There's really black magic involved. It's something that at first
glance makes you think "no, this can't work in the long term", but at
the same time it works so well that a whole operating system is built
on top of this magic.
In other words, Nix automatically computes all the runtime
dependencies of a derivation, and it's possible thanks to the hash of
the store paths.
default.nix:
{
ghc ? "ghc8106",
pkgs ? import <nixpkgs> {}
}:
with pkgs.haskell.lib;
let
haskellPkgs = pkgs.haskell.packages.${ghc};
inherit (pkgs) lib;
mySourceRegexes = [
"^app.*$"
"^.*\\.cabal$"
"package.yaml"
];
myApp = (haskellPkgs.callCabal2nix "my-hello"
(lib.sourceByRegex ./. mySourceRegexes) { });
in myApp
.overrideAttrs(
oa: {
nativeBuildInputs = oa.nativeBuildInputs ++ [pkgs.hello pkgs.makeWrapper];
installPhase = oa.installPhase + ''
ln -s ${pkgs.hello.out}/bin/hello $out/bin/hello
'';
postFixup = ''
wrapProgram $out/bin/x-exe --prefix PATH : ${pkgs.lib.makeBinPath [ pkgs.hello ]}
'';
})
src/Main.hs:
module Main where
import System.Process (callCommand)
main :: IO ()
main = do
putStrLn "HELLO"
callCommand "hello"
putStrLn "BYE"
Seems this is not directly supported with an explicitly stated list of dependencies. However we can indirectly achieve this with "wrapping".
I found more information about wrapping here: https://nixos.wiki/wiki/Nix_Cookbook#Wrapping_packages
So I can do a ls that references the package.
...
appPkg = (myHaskellPackages.callCabal2nix "HaskellNixCabalStarter" (./.) {}).overrideAttrs (oldAttrs: {
buildInputs = (oldAttrs.buildInputs or []) ++ [ pkgs.ffmpeg-full ];
});
in
(pkgs.writeScriptBin "finderapp" ''
#!${pkgs.stdenv.shell}
ls ${pkgs.ffmpeg-full}/bin/ffmpeg
exec ${appPkg}/bin/app
''
)
We can verify the built package(?) correctly depends on the appropriate with:
nix-store -q --references result
/nix/store/0cq84xic2absp75ciajv4lfx5ah1fb59-ffmpeg-full-4.2.2
/nix/store/rm1hz1lybxangc8sdl7xvzs5dcvigvf7-bash-4.4-p23
/nix/store/wlvnjx53xfangaa4m5rmabknjbgpvq3d-HaskellNixCabalStarter-0.1.0.0

How to disable unpack phase to prevent error "do not know how to unpack source archive"?

with derivation
let
pkgs = import <nixpkgs> {};
in
with pkgs;
stdenv.mkDerivation {
name = "asdfasdf";
version = "0.1";
src = /home/srghma/opt/foxitsoftware/foxitreader/FoxitReader; # this is executeable file
dontUnpack = true; # not fu**** working
installPhase = ''
echo "not even executed"
'';
}
I have an error
nix-build tmp.nix
these derivations will be built:
/nix/store/x75gi70i1i57y8d3k4hhx0r3z5kjn6h6-asdfasdf.drv
building '/nix/store/x75gi70i1i57y8d3k4hhx0r3z5kjn6h6-asdfasdf.drv'...
unpacking sources
unpacking source archive /nix/store/3hnf69pky6mqaxv4jxly9fyywqpq6iml-FoxitReader
do not know how to unpack source archive /nix/store/3hnf69pky6mqaxv4jxly9fyywqpq6iml-FoxitReader
builder for '/nix/store/x75gi70i1i57y8d3k4hhx0r3z5kjn6h6-asdfasdf.drv' failed with exit code 1
error: build of '/nix/store/x75gi70i1i57y8d3k4hhx0r3z5kjn6h6-asdfasdf.drv' failed
why dontUnpack not working?
Update: created bug issue at nixpkgs https://github.com/NixOS/nixpkgs/issues/65434
Try this:
let
pkgs = import <nixpkgs> {};
in
with pkgs;
stdenv.mkDerivation {
name = "asdfasdf";
version = "0.1";
# Renamed to imply that 'src' functionality is not being used.
executable = /home/srghma/opt/foxitsoftware/foxitreader/FoxitReader; # this is executeable file
phases = [ "installPhase" ]; # Removes all phases except installPhase
installPhase = ''
mkdir -p $out/bin
cp ${executable} $out/bin
'';
}
#FRidh wrote
Many of the dont attributes were not yet implemented in the
current and past stable releases. So, if you intend to use it with the
current stable or older stables, use instead unpackPhase = ":".

node2nix override; wrapProgram: command not found

I'm packaging a node script with an external dependency (GraphicsMagick), and when attempting to override the derivation generated from node2nix I get the error:
wrapProgram: command not found
The following text goes into detail of what I've tried to solve this error.
Reproducing the problem from scratch
I've created a minimal git repository that reproduces this problem if you'd just like to take a look there. Else, the steps to reproduce the problem are below.
Initial Shell Session:
In an empty directory, run:
npm init -y
npm install --save gm
curl https://i.imgur.com/addSfQi.jpg > image.png
(npm version: 5.6.0 & node version v8.9.4)
Create index.js
#!/usr/bin/env node
const path = require("path"); // node.js builtin
const gm = require("gm"); // GraphicsMagick module
const imagePath = path.join(__dirname, "image.png");
// Flip image horizontally and write to disk
gm(imagePath)
.flop()
.write(imagePath, error => {
console.log("error:", error);
});
Add a "bin" section to package.json:
"bin": "index.js"
Generate *.nix files with node2nix
node2nix -8 -l package-lock.json
Create override.nix
{ pkgs ? import <nixpkgs> {}
, system ? builtins.currentSystem
}:
let
nodePackages = import ./default.nix {
inherit pkgs system;
};
in
nodePackages // {
package = nodePackages.package.override (oldAttrs: {
postInstall = ''
wrapProgram "$out/bin/test-nodejs-gm-nixpkg" --prefix PATH : "${pkgs.graphicsmagick}/bin"
'';
});
}
Build nix package
nix-build override.nix -A package
The above fails with:
/nix/store/*/setup: line 95: wrapProgram: command not found
Helpful Resources
node2nix git repository - includes some basic examples.
example override in nixpkgs - example of how nixpkgs uses wrapProgram in postInstall with files generated by node2nix.
wrapProgram is contained within the makeWrapper package.
nativeBuildInputs = oldAttrs.nativeBuildInputs or [] ++ [ pkgs.makeWrapper ];
As mentioned by #ppb in the comments.

Nix external software built/installed is not being found

I've just started using the Nix package manager on OSX and I'm attempting to create my first package for the pass binary (https://www.passwordstore.org) - which is not available in the Nixpkgs repository.
I'm attempting to specify a runtime dependency (getopt), however this doesn't appear to be available when the binary is used.
This is my packages's default.nix:
{ pkgs ? import <nixpkgs> {} }:
with pkgs;
let
version = "1.7.1";
in {
pass = stdenv.mkDerivation rec {
name = "pass-${version}";
src = fetchurl {
url = "https://git.zx2c4.com/password-store/snapshot/password-store-1.7.1.tar.xz";
sha256 = "0scqkpll2q8jhzcgcsh9kqz0gwdpvynivqjmmbzax2irjfaiklpn";
};
buildInputs = [ stdenv makeWrapper];
installPhase = ''
make install PREFIX=$out/artifact
makeWrapper $out/artifact/bin/pass $out/bin/pass \
--set PATH ${stdenv.lib.makeBinPath [ getopt ]}
'';
meta = {
homepage = "https://www.passwordstore.org";
description = "The standard unix password manager";
license = stdenv.lib.licenses.gpl2Plus;
};
};
}
I can successfully build this package (nix-build --show-trace) and install it (nix-env -i ./result).
Listing the runtime dependencies for my package shows getopt listed:
nix-store -qR $(which pass)
...
/nix/store/c5swmygzc0kmvpq6cfkvwm2yz1k57kqy-getopt-1.1.4
However when I come to use the binary (pass init my-key) I get the following error:
/nix/store/...-pass-1.7.1/artifact/bin/pass: line 302:
/usr/local/bin/getopt: No such file or directory
Can anyone advise what I'm doing wrong?
Thanks
It looks like getopt gets a special treatment. The darwin.sh script looks for it using brew and port and falls back to /usr/local. That's why the (correct) wrapper has no effect.
So the solution seems to be, to make it look for getopt in PATH, which is provided by the wrapper script. You can probably make it as simple as GETOPT=getopt (which is similar to openbsd.sh)
For patching source code, see the NixPkgs documentation
After running nix-build, you should run cat result/bin/pass to look at your wrapper script and make sure it looks OK. It should be a shell script that sets the PATH to include getopt and then calls result/artifact/bin/pass.
Then try running the wrapper script. Note that the wrapper should be in result/bin, not result/artifact/bin.

Resources