One of the packages I'm trying to debug is hidden inside a few links of derivations. I've found a reference in one of the paths, but that's a string I can't pull inside of nix repl. Is there a way to go from nix store path (which doesn't exist yet, because that's the derivation that fails) to a nix derivation object?
nix-repl> de.dev.packages.hie-bios.pkgs
"[{\"paths\":[\"/nix/store/f04qyvqaj6s6y5f5a7svpfppsq5wx2p6-haskell-ide-engine-ghc864-7541d1ec71\"],\"priority\":-864}]"
Doesn't seem like you can. The Nix language does let you access the string context, but the string context is not intended to reproduce a derivation, let alone the original expression that the derivation came from.
nix-repl> :p builtins.getContext "example string ${(import <nixpkgs> {}).hello.outPath}"
{ "/nix/store/m2capxzda4ams4fi3awmriz7hfkdxyp9-hello-2.10.drv" = { outputs = [ "out" ]; }; }
Technically you can read the derivation contents and parse the ATerm inside etc etc, but that's really not supported. It doesn't handle string contexts, probably won't let you build anything and will kill your dog. And even then, you don't get the original expression back.
nix-repl> :p builtins.readFile (builtins.head (builtins.attrNames (builtins.getContext "example string ${(import <nixpkgs> {}).hello.outPath}")))
"Derive([(\"out\",\"/nix/[...]
It's probably best to solve your hie-bios problem directly.
Related
I'm trying to set up a Nix cache containing all the store paths needed to build a simple derivation. The goal is for this to work on an empty store, with no cache misses so I don't have to hit cache.nixos.org at all. I'm having trouble because Nix seems to download a bunch of extra bootstrapping stuff, apparently to help with fetching.
For example, consider this derivation:
# empty_store_test_simple.nix
with { inherit (import <nixpkgs> {}) fetchFromGitHub; };
(fetchFromGitHub {
owner = "codedownio";
repo = "templates";
rev = "ba68b83d25d2b74f5475521ac00de3bbb884c983";
sha256 = "sha256-LNTi1ZBEsThmGWK53U9Na1j5DKHljcS42/PRXj97p6s=";
})
If I build this on an empty store with --dry-run, I see the following:
λ nix build --impure --store ~/experimental-store --substituters https://cache.nixos.org/ --dry-run -f ./empty_store_test_simple.nix
this derivation will be built:
/nix/store/jr55kq3kk6va95rvjcdyn5jmh059007p-source.drv
these 48 paths will be fetched (24.52 MiB download, 121.41 MiB unpacked):
/nix/store/02bfycjg1607gpcnsg8l13lc45qa8qj3-libssh2-1.10.0
/nix/store/0fi0432kdh46x9kbngnmz2y7z0q68cdz-xz-5.2.5-bin
/nix/store/0rizskpri8d8qawx6qjqcnvlxcvzr1bm-keyutils-1.6.3-lib
/nix/store/1l4r0r4ab3v3a3ppir4jwiah3icalk9d-zlib-1.2.11
/nix/store/1xyz8jwyg9rya2f7gs549c7n2ah378v6-stdenv-linux
/nix/store/3h4a92kysiw3s3rvbsa6a2nys3lf8f8v-libkrb5-1.18
/nix/store/3ibnw61rlgj2lj5hycy2dn3ybpq7wapm-libev-4.33
/nix/store/5qbrz5fimkbywws73vaim8allyh8kjy5-nghttp2-1.43.0-bin
/nix/store/67x6kxbanrqafx3hg7pb3bc83i3d1v3f-gzip-1.11
/nix/store/6irxz4fbf1d1ac7wvdjf8cqb3sgmnvg8-zlib-1.2.11-dev
/nix/store/71pachqc22wlvf3xjhwjh2rqbl6l3ngg-diffutils-3.8
/nix/store/9mp06ni69a44dmrjhn28mn15brdry52w-gnused-4.8
/nix/store/9ppi191zsi7zvynkm8vy2bi22lci9iwg-bzip2-1.0.6.0.2
/nix/store/c9f15p1kwm0mw5p13wsnvd1ixrhbhb12-gcc-10.3.0-lib
/nix/store/d1n274a607fmqdgr7888nq19hdsj7av0-openssl-1.1.1l-bin
/nix/store/d8p27w4d21xs6svkaf3ij60lsw243rn2-openssl-1.1.1l-dev
/nix/store/fdbwa5jrijn0yzwl8l4xdxa0l5daf5j6-curl-7.79.1
/nix/store/fvprxgcxf4px865gdjd81fbwnxcjrg41-coreutils-9.0
/nix/store/gf6j3k1flnhayvpnwnhikkg0s5dxrn1i-openssl-1.1.1l
/nix/store/gmnh4jfjhx83aggwgwzcnrwmpmqr8fwf-gnutar-1.34
/nix/store/gmzhclix3kzhir5jmmwakwhpg6j5zwf1-acl-2.3.1
/nix/store/h0b8ajwz9lvw3a3vqrf41cxrhlx9dz7p-nghttp2-1.43.0-lib
/nix/store/h7srws2r1nalsih91lrm0hfhhar14jzm-libkrb5-1.18-dev
/nix/store/h97sr1q1rpv1ry83031q51jbkba7q0m4-bzip2-1.0.6.0.2-bin
/nix/store/ihscadskdrvwc9dvbirff51lr70cphjj-curl-7.79.1-bin
/nix/store/ikvp5db9hygc14da45lvxi1c9b4ylna9-pcre-8.44
/nix/store/ilszk5f0zcv8lifkixg47ja1f2lsgxkd-nghttp2-1.43.0-dev
/nix/store/k0qa3rjifblr2vrgx4g54a59zxlfhg90-xz-5.2.5
/nix/store/kd14wd2wfmb56zpv5y71yq2lqs11l06k-attr-2.5.1
/nix/store/ksqy6mszsld4z3w8ybxa2vkjf5cqxw3f-c-ares-1.17.2
/nix/store/l0wlqpbsvh1pgvhcdhw7qkka3d31si7k-bash-5.1-p8
/nix/store/lhambyc1v2c7qzzr5sq7p449xs1j6pg8-gnugrep-3.7
/nix/store/lypy3bif096j0qc1divwa87gdvv3r575-curl-7.79.1-dev
/nix/store/p12km8psjlmvbmi52wb9r6gfykqxcdnd-libssh2-1.10.0-dev
/nix/store/pkpynsyxm8c38z4m8ngv52c7v8vhkr2h-unzip-6.0
/nix/store/prq96vz3ywk955nnxlr7s892wf5qvbr0-mirrors-list
/nix/store/psqacrv7k5fxz6mdiawc28sxcdchb4c9-ed-1.17
/nix/store/qbdsd82q5fyr0v31cvfxda0n0h7jh03g-libunistring-0.9.10
/nix/store/qzr7r4w5gm5m20afn2wz4vlv7ah4sr89-gnumake-4.3
/nix/store/r5niwjr8r8qags2bzv9z583r9vajxag3-patchelf-0.13
/nix/store/rnx655nq2qs53yb5arv2gapa91r1wsbn-findutils-4.8.0
/nix/store/scz4zbxirykss3hh5iahgl39wk9wpaps-libidn2-2.3.2
/nix/store/sqn31ly001033hsz0dpxwcsay5qdbk2w-gawk-5.1.1
/nix/store/vslsa0l17xjcrdgm2knwj0z5hlvf73m7-perl-5.34.0
/nix/store/x6pz7c0ffcd6kxzc8m1rflvqmdbjiihh-nghttp2-1.43.0
/nix/store/yj11v0gdjqli4nzax4x48xjnh9y36b2q-curl-7.79.1-man
/nix/store/z56jcx3j1gfyk4sv7g8iaan0ssbdkhz1-glibc-2.33-56
/nix/store/zjm4xv4nr872mdhvv3j22bzb08rgf1hk-patch-2.7.6
However, I can't find all these paths by using the means I would expect:
λ nix repl empty_store_test_simple.nix
nix-repl> :b with import <nixpkgs> {}; closureInfo { rootPaths = [inputDerivation]; }
This derivation produced the following outputs:
out -> /nix/store/1iqm7sr2rr6i5njnfxlqqyzi567mb4cz-closure-info
λ cat /nix/store/1iqm7sr2rr6i5njnfxlqqyzi567mb4cz-closure-info/store-paths
/nix/store/icvdinlsgl6y2kxk9wzkj82a53jgpdlm-source
I would expect to see ~48 paths here, but I only see 1! How can I get all the build-time dependencies indicated by the dry run? I've seen this kind of issue in the past when IFD is present, could there be some going on in Nixpkgs?
If you build your derivation and store it in your own Nix cache that should work; Nix shouldn't need to get the build-time dependencies of that thing, it can just download the result (i.e. the source code you're fetching) from your cache.
If you want to get the build-time dependencies anyway, try:
nix-store -qR $(nix-instantiate test.nix)
I think this will get you one step closer but it's not a complete solution. You'd probably need to build all the derivations in this list or something.
So I'm browsing through a proprietary nix codebase and I'm trying to make sense of it. It is building rust packages among other things. There is this autogenerated nix code using crate2nix which among other things invokes nixpkgs.buildPackages. After googling this expression I get very random results about people using this for compiling many languages but I cannot find a single case of some doc mentioning this? Is this some builting nix expression? Where is it documented?
This looks like a package set to explicitly reference all packages built on the host during cross-compilation (see pkgs/top-level/stage.nix).
$ nix repl
Welcome to Nix version 2.3.10. Type :? for help.
nix-repl> :l <nixpkgs>
Added 13917 variables.
nix-repl> :t buildPackages
a set
nix-repl> builtins.length (builtins.attrNames buildPackages)
13918
nix-repl> lib.lists.take 20 (builtins.attrNames buildPackages)
[ "AAAAAASomeThingsFailToEvaluate" "AMB-plugins" "ArchiSteamFarm" "AusweisApp2" "CHOWTapeModel" "CoinMP" "DisnixWebService" "EBTKS" "EmptyEpsilon" "FIL-plugins" "Fabric" "LAStools" "LASzip" "LASzip2" "Literate" "MACS2" "MIDIVisualizer" "MMA" "NSPlist" "OSCAR" ]
I'm trying to add helpful messages for arbitrary builds. If the build fails the user can, for example, install the package with different arguments.
My interface idea is to provide a function, build-with-message, that would be called with something like this:
build-with-message
''Building ${pkg.name}. Alternative invocations are: ..''
pkg
My implementation is based on builtins.seq
build-with-message = msg : pkg :
seq
(self.runCommand "issue-message" {} ''mkdir $out; echo ${msg}'')
pkg;
When I build a package with build-with-message I never see the message. My hunch is that seq evaluates the runCommand far enough to see that a set is returned and moves on to building the package. I tried with deepSeq as well, but a deepSeq build fails on runCommand. I also tried calling out some attributes from the runCommand, e.g.
(self.runCommand "issue-message" {} ''mkdir $out; echo ${msg}'').drvPath
(self.runCommand "issue-message" {} ''mkdir $out; echo ${msg}'').out
My thought being that calling for one of these would prompt the rest of the build. Perhaps I'm not calling the right attribute, but in any case the ones I've tried don't work.
So:
Is there a way to force the runCommand to build in the above scenario?
Is there already some builtin that just lets me issue messages on top of arbitrary builds?
Here's me answering my own question again, consider this a warning.
Solution:
I've in-lined some numbered comments to help with the explanation.
build-with-message = msg : pkg :
let runMsg /*1*/ = self.runCommand "issue-message"
{ version = toString currentTime; /*2*/ } ''
cat <<EOF
${msg}
EOF
echo 0 > $out /*3*/
'';
in seq (import runMsg /*4*/) pkg; /*5*/
Explanation:
runMsg is the derivation that issues the message.
Adding a version based on the current time ensures that the build of runMsg will not be in /nix/store. Otherwise, each unique message will only be issued for the first build.
After the message is printed, a 0 is saved to file as the output of the derivation.
The import loads runMsg--a derivation, and therefore serialized as the path $out. Import expects a nix expression, which in this case is just the number 0 (a valid nix expression).
Now, since the runMsg output will not be available until after it has been built, the seq command will build it (issuing the message) and then build pkg.
Discussion:
I take note of Robert Hensing's comment to my question--this may not be something Nix was not intended for. I'm not arguing against that. Moving on.
Notice that issuing a message like so will add a file to your nix store for every message issued. I don't know if the message build will be garbage collected while pkg is still installed, so there's the possibility of polluting the nix store if such a pattern is overused.
I also think it's really interesting that the result of the runMsg build was to install a nix expression. I suppose this opens the door to doing useful things.
I try to parse a string containing multiple "_"s, but I get a CallFailed exception.
I have tried to create a small as possible example of the problem syntax.
layout Layout = WhitespaceAndComment* !>> [\ \t\n\r#];
lexical WhitespaceAndComment = [\ \t\n\r] | #category="Comment" "#" ![\n]* $;
syntax SourceList = sourceList: "$"? "{"? Id sourceFile "}"?;
lexical Id = ([a-zA-Z/.\-][a-zA-Z0-9_/.]* !>> [a-zA-Z0-9_/.]) \ Reserved;
keyword Reserved =
"$" | "{" | "}" ;
I am unable to parse this small example.
rascal>try { parse(#SourceList, "test"); } catch CallFailed(m, e): println("<m> : <e>");
|prompt:///|(25,9,<1,25>,<1,34>) : [type(sort("SourceList"),(sort("SourceList"):choice(sort("SourceList"),{prod(label("sourceList",sort("SourceList")),[opt(lit("$")),layouts("$default$"),opt(lit("{")),layouts("$default$"),label("sourceFile",lex("Id")),layouts("$default$"),opt(lit("}"))],{})}),layouts("$default$"):choice(layouts("$default$"),{prod(layouts("$default$"),[],{})}),empty():choice(empty(),{prod(empty(),[],{})}),lex("Id"):choice(lex("Id"),{prod(lex("Id"),[conditional(seq([\char-class([range(45,47),range(65,90),range(97,122)]),conditional(\iter-star(\char-class([range(46,57),range(65,90),range(95,95),range(97,122)])),{\not-follow(\char-class([range(46,57),range(65,90),range(95,95),range(97,122)]))})]),{delete(keywords("Reserved"))})],{})}),keywords("Reserved"):choice(keywords("Reserved"),{prod(keywords("Reserved"),[lit("$")],{}),prod(keywords("Reserved"),[lit("}")],{}),prod(keywords("Reserved"),[lit("{")],{})}))),"${test}"]
ok
A changed sourcefile from "test" to "${test}" gives exactly the same output.
The complete syntax in which SourceList is embedded has many more rules. But then I get the following results.
set(${TARGET_NAME}_DEPS
GenConfiguration_OBJ_TN_Common # accept
${COMMON_BB_PCMDEPS} # reject
COMMON_BB_PCMDEPS # accept
COMMON_BB_PCM_DEPS # reject
)
for which I want to have a solution.
What is wrong with the minimal example? Why is test or ${test} not accepted?
BTW: I am using the latest unstable. Does it make sense to install and try the stable release?
I've tried to reproduce your problem, but it seems to work here:
rascal>parse(#SourceList, "test")
SourceList: (SourceList) `test`
The unstable version is fine at the moment. In fact it's high time to release a stable version. So for now you're better off with the unstable version.
The CallFailed exception is confusing. It means that a function is called which can not be matched OR not be found. So maybe parse is not in scope by not importing ParseTree, or you have a different function called parse which does not have a type[Tree] and str as the expected parameters in scope. As long as the ParseTree module is imported, you're call to parse should be fine.
Please let me know if you have made progress. Perhaps a restart of Eclipse might clear up something as well.
How can I get a string constant automatically set to the datestamp as at compile time?
Something like:
const String COMPILE_DATESTAMP = eval_static(DateTime.now().toString());
...
String s = "This program was compiled $COMPILE_DATESTAMP";
where s would then be for e.g.
"This program was compiled 1971-02-03 04:05:06"
Thanks for the question!
There's no required compile step in Dart. (We do have an optional Dart-to-JavaScript compiler, or even a Dart-to-Dart processor that does tree shaking.) Dart's VM accepts input as text files. Similar to Ruby or Python, it runs text-based scripts.
As others have mentioned, this is a job for some sort of build step.
I'm new to Dart, but I haven't seen anything in the documentation to suggest that such a thing is possible. I strongly suspect that it isn't.
If you really need functionality like you describe, I think your best bet is to roll your own build script. Something simple like:
#!/bin/bash
sed -ri "s/INSERT_DATETIME_HERE/`date`/" $1
dart2js $1 -o$1.js
could be modified to suit your needs. (I'd want some sanity checks in there if it were me; I'm just suggesting a starting point.) Your code would become:
const String COMPILE_DATESTAMP = "INSERT_DATETIME_HERE";
...
String s = "This program was compiled $COMPILE_DATESTAMP";
You must write another dart program that could examine the actual compiled program. Then is simple as:
File compiledApp = new File('path/to/compiled/app.dart');
compiledApp.lastModified().then(
(modifiedDate)
{
print("This program was compiled $modifiedDate");
},
onError: (exp)
{
// File doesn't exist ?
}
);
This trick build on knowledge that compiler will modify the 'last modify date' of the file