I have a long string which I want to extract to a separate file.
prepare = lib.hm.dag.entryAfter ["writeBoundary"] '' very long script with ${...} '';
I can read it with builtins.readFile but it does not replace ${...} nix place holders.
How can I read string from a file and resolve nix variables inside it?
You could use the substitute family of bash functions (see Nixpkgs manual) or use the substituteAll Nix function which produces a derivation that performs the substitution.
substituteAll { src = ./sample.sh; inherit bash; hi = hello; }
sample.sh:
#!#bash#/bin/bash
#hi#
Result:
$ nix repl <nixpkgs>
nix-repl> substituteAll { src = ./sample.sh; inherit bash; hi = hello; }
«derivation /nix/store/7klv763a0ipgvwf3j84aazkzx2d5rljz-sample.sh.drv»
nix-repl> :b substituteAll { src = ./sample.sh; inherit bash; hi = hello; }
this derivation produced the following outputs:
out -> /nix/store/v03hpzw9ykrbyqpmalnijnyibq3waqhw-sample.sh
nix-repl>
/nix/store/v03hpzw9ykrbyqpmalnijnyibq3waqhw-sample.sh:
#!/nix/store/a4yw1svqqk4d8lhwinn9xp847zz9gfma-bash-4.4-p23/bin/bash
/nix/store/jmmw0d3nmklwafcwylvrjb9v69wrbcxf-hello-2.10
Related
I want to parse a file to a Nix list value inside flake.nix.
I have a shell script which does that
perl -007 -nE 'say for m{[(]use-package \s* ([a-z-0-9]+) \s* (?!:nodep)}xsgm' init.el
How can I execute external command while evaluating flake.nix?
programs.emacs = {
enable = true;
extraConfig = builtins.readFile ./init.el;
extraPackages = elpa: (shellCommandToParseFile ./init.el); # Runs shell script
};
You can run ./init.el by the same way you perform any other impure step in Nix: With a derivation.
This might look something vaguely like:
programs.emacs = {
enable = true;
extraConfig = ../init.el;
extraPackages = elpa:
let
packageListNix =
pkgs.runCommand "init-packages.nix" { input = ../init.el; } ''
${pkgs.perl}/bin/perl -007 -nE '
BEGIN {
say "{elpa, ...}: with elpa; [";
say "use-package";
};
END { say "]" };
while (m{[(]use-package \s* ([a-z-0-9]+) \s* (;\S+)?}xsgm) {
next if $2 eq ";builtin";
say $1;
}' "$input" >"$out"
'';
in (import "${packageListNix}" { inherit elpa; });
};
...assuming that, given the contents of your ./init.el, the contents of your resulting el-pkgs.nix is actually valid nix source code.
That said, note that like any other derivation (that isn't either fixed-output or explicitly impure), this happens inside a sandbox with no network access. If the goal of init.el is to connect to a network resource, you should be committing its output to your repository. A major design goal of flakes is to remove impurities; they're not suitable for impure derivations.
I want to write Google ZX scripts in an idiomatic way in Nix, but how can I do that? nixpkgs only offers writeShellScriptBin which is tied to bash as interpreter. How can I change the interpreter?
To solve it idiomatically, you could create a writeZxScriptBin attribute similar to writeShellScriptBin. Luckily, zx is already packaged in <nixpkgs>. Hence, we can write a zx-script-writer-bin.nix like this:
# This returns a writer that creates an executable Google ZX script in `<pkg>/bin/%name`
# with the provided content. Similar to `writeShellScriptBin`, it consumes the name of
# the script and the NodeJS/ZX-script content as parameters.
{ pkgs }:
name: text:
let
mjsBin = pkgs.writeTextFile {
name = "${name}-mjs";
executable = true;
# For Google ZX, the .mjs extension is mandatory.
destination = "/bin/${name}.mjs";
text = ''
#!${pkgs.zx}/bin/zx
${text}
'';
};
in
# Shebang script enables to call the zx-script without the .mjs version.
pkgs.writeTextFile {
name = "${name}";
executable = true;
destination = "/bin/${name}";
text = ''
#!${mjsBin}/bin/${name}.mjs
'';
}
We can invoke it like this:
$ nix-build -E "
let
pkgs = import <nixpkgs> {};
writeZxScriptBin = pkgs.callPackage ./write-zx-script-bin.nix {};
in
writeZxScriptBin \"foo\" ''
const a = 7
console.log(\`hello! sum=''\${3 + a}\`)
''" && result/bin/foo
Note that there is some escaping required. Alternatively, you can work with builtins.readFile to write the script code in pure JavaScript
I'm trying to understand how the vendorSha256 is calculated when using buildGoModule. In nixpkgs manual the only info I get is:
"vendorSha256: is the hash of the output of the intermediate fetcher derivation."
Is there a way I can calculate the vendorSha256 for a nix expression I'm writing? To take a specific example, how was the "sha256-Y4WM+o+5jiwj8/99UyNHLpBNbtJkKteIGW2P1Jd9L6M=" generated here:
{ lib, buildGoModule, fetchFromGitHub }:
buildGoModule rec {
pname = "oapi-codegen";
version = "1.6.0";
src = fetchFromGitHub {
owner = "deepmap";
repo = pname;
rev = "v${version}";
sha256 = "sha256-doJ1ceuJ/gL9vlGgV/hKIJeAErAseH0dtHKJX2z7pV0=";
};
vendorSha256 = "sha256-Y4WM+o+5jiwj8/99UyNHLpBNbtJkKteIGW2P1Jd9L6M=";
# Tests use network
doCheck = false;
meta = with lib; {
description = "Go client and server OpenAPI 3 generator";
homepage = "https://github.com/deepmap/oapi-codegen";
license = licenses.asl20;
maintainers = [ maintainers.j4m3s ];
};
}
From the manual:
The function buildGoModule builds Go programs managed with Go modules.
It builds a Go Modules through a two phase build:
An intermediate fetcher derivation. This derivation will be used to fetch all of the dependencies of the Go module.
A final derivation will use the output of the intermediate derivation to build the binaries and produce the final output.
You can see that, when you're trying to build the above expression in the ouput of nix-build. If you run:
nix-build -E 'with import <nixpkgs> { }; callPackage ./yourExpression.nix { }'
you see the first 2 lines of output:
these 2 derivations will be built:
/nix/store/j13s3dvlwz5w9xl5wbhkcs7lrkgksv3l-oapi-codegen-1.6.0-go-modules.drv
/nix/store/4wyj1d9f2m0521nlkjgr6al0wfz12yjn-oapi-codegen-1.6.0.drv
The first derivation will be used to fetch all dependencies for your Go module, and the second will be used to build your actual module. So vendorSha256 is the hash of the output of that first derivation.
When you write a Nix expression to build a Go module you don't know in advance that hash. You only know it that the first derivation has been realised(download dependencies and find the hash based on them). However you can use Nix validation to find out the value of vendorSha256.
Modify your Nix expression like so:
{ lib, buildGoModule, fetchFromGitHub }:
buildGoModule rec {
pname = "oapi-codegen";
version = "1.6.0";
src = fetchFromGitHub {
owner = "deepmap";
repo = pname;
rev = "v${version}";
sha256 = "sha256-doJ1ceuJ/gL9vlGgV/hKIJeAErAseH0dtHKJX2z7pV0=";
};
vendorSha256 = lib.fakeSha256;
# Tests use network
doCheck = false;
meta = with lib; {
description = "Go client and server OpenAPI 3 generator";
homepage = "https://github.com/deepmap/oapi-codegen";
license = licenses.asl20;
maintainers = [ maintainers.j4m3s ];
};
}
The only difference is vendorSha256 has now the value of lib.fakeSha256, which is just a fake/wrong sha256 hash. Nix will try to build the first derivation and will check the hash of the dependencies against this value. Since they will not match, an error will occur:
error: hash mismatch in fixed-output derivation '/nix/store/j13s3dvlwz5w9xl5wbhkcs7lrkgksv3l-oapi-codegen-1.6.0-go-modules.drv':
specified: sha256-AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA=
got: sha256-Y4WM+o+5jiwj8/99UyNHLpBNbtJkKteIGW2P1Jd9L6M=
error: 1 dependencies of derivation '/nix/store/4wyj1d9f2m0521nlkjgr6al0wfz12yjn-oapi-codegen-1.6.0.drv' failed to build
So this answer your question. The value of vendorSha256 you need is sha256-Y4WM+o+5jiwj8/99UyNHLpBNbtJkKteIGW2P1Jd9L6M=. Copy and add it your file and you're good to go!
Assumptions:
You have yq and nix installed on your OS running NixOS or some Linux distro.
Question:
Can nix maintain the original ordering of a set? i.e. If I create a sample.nix file:
{pkgs}:
let
dockerComposeConfig = {
version = "1.0";
services = {
srv1 = { name = "srv1"; };
srv2 = { name = "srv2"; };
};
};
in writeTextFile {
name = "docker-compose.json";
text = builtins.toJSON dockerComposeConfig;
}
When I build and convert the output to yaml below I notice is that the set has been alphabetized by Nix. Is there a workaround that keeps my JSON in the same ordering as intended by a Docker user such that the `dockerComposeConfig attrributes remain in the order they are created?
# Cmd1
nix-build -E "with import <nixpkgs> {}; callPackage ./sample.nix {}"
# Cmd2
cat /nix/store/SOMEHASH-docker-compose.json | yq r - --prettyPrint
Nix attribute sets don't have an ordering to their attributes and they are represented as a sorted array in memory. Canonicalizing values helps with reproducibility.
If it's really important you could write a function that turns a list of key value pairs into a JSON object as a Nix string. But that's not going to be easy to use like builtins.toJSON. I'd consider the JSON as "compiled build output" and not worry too much about aesthetics.
Side note: Semantically, they are not even created in any order. The Nix language is declarative: a Nix expression (excluding derivations) describes something that is, not how to create it, although it may be defined in terms of functions.
This is necessary for Nix's laziness to be effective.
Is there a simple way to add a bash function to the environment provided by stdenv? When developing with 'nix-shell', I can run commands like 'unpackPhase' or 'buildPhase' because mkDerivation puts them in scope--its super useful. My derivation attributes are added to the environment as well. But I'd also like to see a way to automatically add an attribute as a function in the build/shell environment.
To explain what I'm getting at, I can currently get functions into the environment by either using eval statements or toFile and source. For instance, with eval, something like:
{ stdenv, ... } : stdenv.mkDerivation
{ shellHook = ''eval "$myFunctions"'';
myFunctions = ''
myFunction1(){
echo "doing myFunction1"
}
myFunction2(){
echo "doing myFunction2"
}
''
}
will "source" myFunction1 and myFunction2 when I enter the nix-shell.
However, I was expecting something like a mkFunction utility to compliment mkWrapper, mkProgram, and other such basic utilities. I'd expect to use it like in the below, where the the attributes defined using mkFunction are automatically "sourced" as above.
{ stdenv, mkFunction, ... } : stdenv.mkDerivation
{ myFunction1 = mkFunction '' echo "doing myFunction1" '';
myFunction2 = mkFunction '' echo "doing myFunction2" '';
shellHook = '' echo "No need to source, myFunctions are already in scope." '';
}
I think such a utility would be useful. I thought setup hooks might cover this use, but I'm not really sure how to use them. And its not so bad doing it the first way (If I hadn't figured it out during the course of writing the question I wouldn't have bothered asking). But it seems like the kind of utility that nix would already have available, so I'm asking anyway.
Setup hooks can indeed do that, here's a simple example of one that defines a function foo:
with import <nixpkgs> {};
let
fooHook = stdenv.mkDerivation {
name = "foo-hook";
# Setting phases directly is usually discouraged, but in this case we really
# only need fixupPhase because that's what installs setup hooks
phases = [ "fixupPhase" ];
setupHook = writeText "my-setup-hook" ''
foo() { echo "Foo was called!"; }
'';
};
in mkShell {
buildInputs = [ fooHook ];
shellHook = "foo";
}
Running nix-shell on this yields the desired result:
$ nix-shell
Foo was called!
[nix-shell:~]$
Another simple possibility is to use the runHook function provided by stdenv.
with import <nixpkgs> {};
mkShell rec {
name = "runs-hook";
myFun = "echo The name is ${name}";
}
In the snippet above, a variable myFun is made available in the shell/build environment. You call it with:
> runHook myFun
The name is runs-hook
This is really similar to the eval method from the question.