What does the <nixpkgs> string / value mean in Nix? - nix

For example in the following (which I assume is a nix expression):
(import <nixpkgs> {}).haskellPackages.ghcWithPackages (hpkgs: with hpkgs; [
lens
aeson
turtle
])
What does <nixpkgs> reference? I also see it used in other contexts for example:
nix-shell '<nixpkgs>' -A linuxPackages.kernel

<nixpkgs> is a Nix expression that is evaluated by looking at the Nix search path in the NIX_PATH environment variable and/or -I option.
It is described in more detail in the Nix manual.
Note that the Nix search path is impractical in many situations. You can only pass it from the outside, and it easily creates impurity. In my experience, problems are better solved with explicit argument passing or the functions relating to fix-points like callPackage and the overlay system.
As an example, NixOS has only one extra search path parameter, and it is only read once in nixos/default.nix if no explicit configuration is given. This way, you have the flexibility to provide your own configuration, which is why you (nix-build) and hydra can confidently build the NixOS VM tests, bootable images, docker images, etc.

From the Nix manual, 5.2.1. Values, section "Simple Values":
Paths can also be specified between angle brackets, e.g. .
This means that the directories listed in the environment variable
NIX_PATH will be searched for the given file or directory name.
From the NixOS manual, 7.2. Common Environment Variables, section NIX_PATH:
A colon-separated list of directories used to look up Nix expressions
enclosed in angle brackets (i.e., ). For instance, the value
/home/eelco/Dev:/etc/nixos
will cause Nix to look for paths relative to /home/eelco/Dev and
/etc/nixos, in that order. It is also possible to match paths
against a prefix. For example, the value
nixpkgs=/home/eelco/Dev/nixpkgs-branch:/etc/nixos
will cause Nix to search for <nixpkgs/path> in
/home/eelco/Dev/nixpkgs-branch/path and /etc/nixos/nixpkgs/path.
If a path in the Nix search path starts with http:// or https://,
it is interpreted as the URL of a tarball that will be downloaded and
unpacked to a temporary location. The tarball must consist of a single
top-level directory. For example, setting NIX_PATH to
nixpkgs=https://github.com/NixOS/nixpkgs-channels/archive/nixos-15.09.tar.gz
tells Nix to download the latest revision in the Nixpkgs/NixOS 15.09
channel.
A following shorthand can be used to refer to the official channels:
nixpkgs=channel:nixos-15.09
The search path can be extended using the -I option, which takes
precedence over NIX_PATH.
Examples
1. with import <nixpkgs> {}; /* rest of the expression */
In my case, <nixpkgs> is /nix/var/nix/profiles/per-user/root/channels/nixos:
$ echo $NIX_PATH
# VVVVVVV
/home/a_user/.nix-defexpr/channels:nixpkgs=/nix/var/nix/profiles/per-user/root/channels/nixos:nixos-config=/etc/nixos/configuration.nix:/nix/var/nix/profiles/per-user/root/channels
# ^^^^^^^
Because <nixpkgs> evaluates to "a directory, the file default.nix in that directory is loaded" by import. (Nix manual, 15.4.1. Advanced Attributes, section import path, builtins.import path)
$ ll /nix/var/nix/profiles/per-user/root/channels/nixos
lrwxrwxrwx 1 root root 80 Dec 31 1969 /nix/var/nix/profiles/per-user/root/channels/nixos -> /nix/store/ywlfq2ns4
a3fzb2ap74lvahmrg1p0lmk-nixos-19.03.172231.7b36963e7a7/nixos/
$ ll $(readlink -f /nix/var/nix/profiles/per-user/root/channels/nixos)
total 3308
dr-xr-xr-x 8 root root 4096 Dec 31 1969 ./
dr-xr-xr-x 4 root root 4096 Dec 31 1969 ../
# (...)
dr-xr-xr-x 7 root root 4096 Dec 31 1969 nixos/
dr-xr-xr-x 17 root root 4096 Dec 31 1969 pkgs/
-r--r--r-- 1 root root 1097 Dec 31 1969 COPYING
-r--r--r-- 1 root root 968 Dec 31 ---> default.nix <---
# (...)
If my understanding is correct, after the import the provided Nix expression is evaluated with an empty attribute set ({}). The result is an attribute list, and the with expression will include all its containing attributes in the local lexical scope.
2. nix repl '<nixpkgs/nixos>'
Example taken from the NixOS manual, 5.3. Modularity, showing the active NixOS configuration settings in the repl.
# On NixOS 19.03
$ nix repl
Welcome to Nix version 2.2.2. Type :? for help.
nix-repl> <nixpkgs>
/nix/var/nix/profiles/per-user/root/channels/nixos
nix-repl> <nixpkgs/nixos>
/nix/var/nix/profiles/per-user/root/channels/nixos/nixos
Load the systems NixOS configuration on the repl:
nix-repl> :l <nixpkgs/nixos>
Added 6 variables.
Load all Nix expressions from Nixpkgs on the repl:
nix-repl> :l <nixpkgs>
Added 10089 variables.
Or loading them directly into the repl:
$ nix repl '<nixpkgs>'
Welcome to Nix version 2.2.2. Type :? for help.
Loading '<nixpkgs>'...
Added 10089 variables.
$ nix repl '<nixpkgs/nixos>'
Welcome to Nix version 2.2.2. Type :? for help.
Loading '<nixpkgs/nixos>'...
Added 6 variables.
Cheatsheet:
nix-repl> :help
The following commands are available:
<expr> Evaluate and print expression
<x> = <expr> Bind expression to variable
:a <expr> Add attributes from resulting set to scope
:b <expr> Build derivation
:i <expr> Build derivation, then install result into current profile
:l <path> Load Nix expression and add it to scope
:p <expr> Evaluate and print expression recursively
:q Exit nix-repl
:r Reload all files
:s <expr> Build dependencies of derivation, then start nix-shell
:t <expr> Describe result of evaluation
:u <expr> Build derivation, then start nix-shell
Because of the <nixpkgs/path> convention (where path equals nixos), the angle expression will evaluate to /nix/var/nix/profiles/per-user/root/channels/nixos/nixos. The above ll output also shows a nixos folder above default.nix, and inside there is indeed another default.nix that will get evaluated by nix repl:
$ ll $(readlink -f /nix/var/nix/profiles/per-user/root/channels/nixos/nixos)
total 72
dr-xr-xr-x 7 root root 4096 Dec 31 1969 ./
dr-xr-xr-x 8 root root 4096 Dec 31 1969 ../
-r--r--r-- 1 root root 886 Dec 31 ---> default.nix <---
-r--r--r-- 1 root root 197 Dec 31 1969 README
-r--r--r-- 1 root root 6074 Dec 31 1969 release-combined.nix
-r--r--r-- 1 root root 9251 Dec 31 1969 release.nix
-r--r--r-- 1 root root 2038 Dec 31 1969 release-small.nix
Miscellaneous
Issue #338 is still open to improve the Nix manual, documenting the angle syntax in their own section (UPDATE: Fixed and closed, 10/23/2019).
Nix Pills sections (e.g., 15.5. Conclusion and Chapter 16. Nixpkgs Parameters) consistently call the angle syntax "angular brackets" syntax.
In his answer, Robert Hensing warns against using <nixpkgs> everywhere, and an issue #1161 is still open on just that topic. (Excerpt: "We have an unofficial guideline that nobody should use angle brackets in nixpkgs, but why not make it into a hard requirement?")

One can evaluate the value using nix repl:
 nix repl
Welcome to Nix version 2.1.3. Type :? for help.
nix-repl> <nixpkgs>
/nix/var/nix/profiles/per-user/root/channels/nixos

Related

biber wants to load libcrypt.so.1 but it is missing

I am Arch GNU/Linux user who usually manages almost every package with pacman; I manage TeX and LaTeX-related things with tlmgr. I installed tlmgr from source.
I am writing paper. I would like to use bibliography.
When I tried latexmk -pdflua main.ltx:
Rc files read:
latexmkrc
Latexmk: This is Latexmk, John Collins, 20 November 2021, version: 4.76.
Latexmk: applying rule 'biber main'...
Rule 'biber main': The following rules & subrules became out-of-date:
'biber main'
------------
Run number 1 of rule 'biber main'
------------
------------
Running 'biber "main.bcf"'
------------
biber: error while loading shared libraries: libcrypt.so.1: cannot open shared object file: No such file or directory
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
biber main: Could not open biber log file for 'main'
Latexmk: Use the -f option to force complete processing,
unless error was exceeding maximum runs, or warnings treated as errors.
libcrypt.so* on my environment
$ pacman -Qo /usr/lib/libcrypt*
/usr/lib/libcrypto.so is owned by openssl 1.1.1.m-1
/usr/lib/libcrypto.so.1.1 is owned by openssl 1.1.1.m-1
/usr/lib/libcryptsetup.so is owned by cryptsetup 2.4.3-2
/usr/lib/libcryptsetup.so.12 is owned by cryptsetup 2.4.3-2
/usr/lib/libcryptsetup.so.12.7.0 is owned by cryptsetup 2.4.3-2
/usr/lib/libcrypt.so is owned by libxcrypt 4.4.28-1
/usr/lib/libcrypt.so.2 is owned by libxcrypt 4.4.28-1
/usr/lib/libcrypt.so.2.0.0 is owned by libxcrypt 4.4.28-1
What I tried else
I uninstalled and re-installed biber on tlmgr but did not work.
# ln -s /usr/lib/libcrypt.so /usr/lib/libcrypt.so.1
$ latexmkrc -pdflua main.ltx
Rc files read:
latexmkrc
Latexmk: This is Latexmk, John Collins, 20 November 2021, version: 4.76.
Latexmk: applying rule 'biber main'...
Rule 'biber main': The following rules & subrules became out-of-date:
'biber main'
------------
Run number 1 of rule 'biber main'
------------
------------
Running 'biber "main.bcf"'
------------
/tmp/par-716861/cache-0e6aa298f0c2e7a775de99938825b2d56bd2027f/biber: /usr/lib/libcrypt.so.1: version `GLIBC_2.2.5' not found (required by /tmp/par-716861/cache-0e6aa298f0c2e7a775de99938825b2d56bd2027f/biber)
Latexmk: Errors, so I did not complete making targets
Collected error summary (may duplicate other messages):
biber main: Could not open biber log file for 'main'
Latexmk: Use the -f option to force complete processing,
unless error was exceeding maximum runs, or warnings treated as errors.
File source
latexmkrc:
$latex='lualatex %O -synctex=1 -interaction=nonstopmode %S';
#$bibtex='upbibtex %O %B';
$bibtex='biber %O %B';
$makeindex='upmendex %O -o %D %S';
$pdf_mode=3;
Install libxcrypt-compat from the Core (1) repository, as suggest in this answer.
This made my biber from TeX Live 2020 work again. The interesting question is if newer TeX distributions will require this package, too.
(1) Thanks to #samueldy for the hint that the package was moved from the AUR to Core.
Adding on to #Christoph90's answer, as of 25 May 2022 the libxcrypt-compat package is still required in TeX Live 2022 in order for biber to function on Manjaro 21.2.6. However, the package has moved to core/libxcrypt-compat, so install by doing
sudo pacman -S libxcrypt-compat

How to count the occurence of a string in a file, for all files in a directory and output into a new file with shell

I have hundreds of files in a directory that I would like to count the occurrence of a string in each file.
I would like the output to be a summary file that contains the original file name plus the count (ideally on the same line)
for example
file1 6
file2 3
file3 4
etc
Thanks for your consideration
CAUTION: I am pretty much an enthusiastic amateur, so take everything with a grain of salt.
Several questions for you - depending on your answers, the solution below may need some adjustments.
Are all your files in the same directory, or do you also need to look through subdirectories and sub-subdirectories, etc.? Below I make the simplest assumption - that all your files are in a single directory.
Are all your files text files? In the example below, the directory will contain text files, executable files, symbolic links, and directories; the count will only be given for text files. (What linux believe to be text files, anyway.)
There may be files that do not contain the searched-for string at all. Those are not included in the output below. Do you need to show them too, with a count of 0?
I assume by "count occurrences" you mean all of them - even if the string appears more than once on the same line. (Which is why a simple grep -c won't cut it, as that only counts lines that contain the substring, no matter how many times each.)
Do you need to include hidden files (whose name begins with a period)? In my code below I assumed you don't.
Do you care that the count appears first, and then the file name?
OK, so here goes.
[oracle#localhost test]$ ls -al
total 20
drwxr-xr-x. 3 oracle oinstall 81 Apr 3 18:42 .
drwx------. 39 oracle oinstall 4096 Apr 3 18:42 ..
-rw-r--r--. 1 oracle oinstall 40 Apr 3 17:44 aa
lrwxrwxrwx. 1 oracle oinstall 2 Apr 3 18:04 bb -> aa
drwxr-xr-x. 2 oracle oinstall 6 Apr 3 17:40 d1
-rw-r--r--. 1 oracle oinstall 38 Apr 3 17:56 f1
-rw-r--r--. 1 oracle oinstall 0 Apr 3 17:56 f2
-rwxr-xr-x. 1 oracle oinstall 123 Apr 3 18:15 zfgrep
-rw-r--r--. 1 oracle oinstall 15 Apr 3 18:42 .zz
Here's the command to count 'waca' in the text files in this directory (not recursive). I define a variable substr to hold the desired string. (Note that it could also be a regular expression, more generally - but I didn't test that so you will have to, if that's your use case.)
[oracle#localhost test]$ substr=waca
[oracle#localhost test]$ find . -maxdepth 1 -type f \
> -exec grep -osHI "$substr" {} \; | sed "s/^\.\/\(.*\):$substr$/\1/" | uniq -c
8 aa
2 f1
1 .zz
Explanation: I use find to find just the files in the current directory (excluding directories, links, and whatever other trash I may have in the directory). This will include the hidden files, and it will include binary files, not just text. In this example I find in the current directory, but you can use any path instead of . I limit the depth to 1, so the command only applies to files in the current directory - the search is not recursive. Then I pass the results to grep. -o means find all matches (even if multiple matches per line of text) and show each match on a separate line. -s is for silent mode (just in case grep thinks of printing messages), -H is to include file names (even when there is only one file matching the substring), and -I is to ignore binary files.
Then I pass this to sed so that from each row output by grep I keep just the file name, without the leading ./ and without the trailing :waca. This step may not be necessary - if you don't mind the output like this:
8 ./aa:waca
2 ./f1:waca
1 ./.zz:waca
Then I pass the output to uniq -c to get the counts.
You can then redirect the output to a file, if that's what you need. (Left as a trivial exercise - since I forgot that was part of the requirement, sorry.)
Thanks for the detailed answer it provides me with ideas for future projects.
In my case the files were all the same format (output from another script) and the only files in the directory.
I found the answer in another thread
grep -c -R 'xxx'

How messages from printf are routed to the terminal?

Lets say I have opened two tabs in the konsole (Tab1 and Tab2).
When I run tty in both of them I have:
Tab1:
~$ tty
/dev/pts/23
Tab2:
~$ tty
/dev/pts/24
If I run a simple program hello.c with a printf("Hello") in Tab1, how the system goes from writing to the stdout (file id 1) to writing to /dev/pts/23, being read by the konsole and then appearing in Tab1?
How the system know it has to give the "Hello" string to /dev/pts/23 and not to /dev/pts/24? And how it does that?
Is there a parameter given by the bash to the program so it knows which psudoterminal to send the "Hello"? Or the program sends the string back to the bash (how?) who knows to which pseudoterminal to send the data?
Thank you for your help
If you look at your process open files, you can see that the STDOUT,STDERR, etc points to the specific psuedo terminal that you already figured out using tty in your question
root#hello:~# ls -l /proc/self/fd
total 0
lrwx------ 1 root root 64 May 21 02:18 0 -> /dev/pts/3
lrwx------ 1 root root 64 May 21 02:18 1 -> /dev/pts/3
lrwx------ 1 root root 64 May 21 02:18 2 -> /dev/pts/3
As you might know, a process is created by a fork system call that actually duplicates the open file descriptors from the parent. so basically, your process gets the file descriptors from its parent.
How did the parent hot these associated with him ? well, konsole already dealt with that.

Cannot execute gradle from the command line even though it's on my path

I have the following in ./bash_profile
export SRCROOT=/users/benjamin.beasley/work/svn/ccdev
export JAVA_HOME=/Library/Java/JavaVirtualMachines/jdk_dev/Contents/Home
export PATH=$PATH$:~/tools/tools-versions/gradle-2.2.1/bin
export PATH=$PATH$:~/tools/activator
In ~/tools/tools-versions/gradle-1.12/bin, I do
drwxr-xr-x# 4 xxx.xxx WORKDAYINTERNAL\Domain Users 136 Nov 12 11:47 .
drwxr-xr-x# 13 xxx.xxx WORKDAYINTERNAL\Domain Users 442 Apr 29 2014 ..
-rwxr-xr-x# 1 xxx.xxx WORKDAYINTERNAL\Domain Users 5071 Apr 29 2014 gradle
-rwxr-xr-x# 1 xxx.xxx WORKDAYINTERNAL\Domain Users 2395 Apr 29 2014 gradle.bat
echo $PATH$:
/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin$:/Users/xxx.xxx/tools/tools-versions/gradle-2.2.1/bin$:/Users/xxx.xxx/tools/activator90566
so I get the gradle executable which is executable. I can execute it from this directory. But if I start a new shell, and type "gradle" it says command not found. But I "echo $PATH$" and I see that the full canonical path to the ~/tools/tools-versions/gradle-2.2.1/bin folder is there.
However I can execute activator which is an executable in the ~/tools/activator directory. I have no clue why bash knows about activator and not gradle.
In summary:
gradle is executable by this user
gradle can be run from the command line.
gradle is in the $PATH$ environment variable
other programs such as activator, which are also in $Path$ are executable anywhere in terminal regardless of directory which is what I want to be true of gradle.
Unix environment variables are $PATH not $PATH$ (they aren't like Windows env vars).
This is causing your problem.
This path is busted: /usr/bin:/bin:/usr/sbin:/sbin:/usr/local/bin:/opt/X11/bin$:/Users/xxx.xxx/tools/tools-versions/gradle-2.2.1/bin$:/Users/xxx.xxx/tools/activator90566
Notice the 90566 at the end? That's from $$ having been expanded to the current process id when you set the variable.
None of these are paths that actually exist or work:
/opt/X11/bin$
/Users/xxx.xxx/tools/tools-versions/gradle-2.2.1/bin$
/Users/xxx.xxx/tools/activator90566

Lua change current/working dir, Linux (without LFS or any non-std modules)

I need to change current working dir in lua script for execute specific actions, but i have a trouble with this simple task.
I write test script test.lua :
os.execute("cd /usr")
os.execute("ls")
But lua test.lua output is:
test.lua
Current dir doesn't change. What's is wrong? And i can't use LFS or any non-std modules.
Thanks to all for explaining it situation. I choose another way : change work dir before run lua script, but i have a lot of troubles with paths which use in scripts and with scripts.
On Unix, os.execute() spawns a child process and its children. In
jpjacobs's answer, the first process would execute the shell.
The directory change operation only affects the child process, not the
process in which the Lua interpreter is executing your program.
It's simply not possible without external libraries. You can use stuff like
os.execute("cd /usr/ && ls")
The Lua standard library is intended to be both small and portable. Therefore, it is based on the capabilities of the C-standard library for all but a select few functions. It has no function to change directories; that's why libraries like LFS exist.
Have you considered that Lua may not be the appropriate language for your needs? If you're doing shell-style scripting work in an environment where you're not allowed to load non-standard modules, perhaps Python or Perl would be better for you. Both of them have extensive standard libraries with a host of features, all without the need to download non-standard modules.
If you want to do any real shell-style scripting in Lua, you need modules. It's that simple. So you should either use external modules or use a different language.
This is a bit of hack, but have you tried creating symbolic links to the folders you need in the current folder, so you don't have to change the current folder at all?
os.execute("ln -s /usr/foo") -- link called "foo" in current dir
os.execute("ln -s /usr/bar") -- link called "bar" in current dir
baz = require('foo.baz') -- loading file /usr/foo/baz.lua
Here is the program written using Zerobranie Lua 5.2:
local sP = "E:/Documents/Lua-Projs/SVN/ScriptsWireE2"
local a, b, c = os.execute("cd "..sP.."&& dir")
print(a,b,c)
And the outpuit:
Програма 'lua.exe' пусната в 'E:\Documents\Lua-Projs\ZeroBraineIDE\ZeroBraineProjects' (pid: 8832).
Volume in drive E is Data
Volume Serial Number is DE34-ED00
Directory of E:\Documents\Lua-Projs\SVN\ScriptsWireE2
08-03-2016 18:25 <DIR> .
08-03-2016 18:25 <DIR> ..
17-03-2016 18:15 <DIR> Bridges
07-03-2016 18:21 11 558 license
07-03-2016 18:21 87 readme.md
2 File(s) 11 645 bytes
3 Dir(s) 181 729 161 216 bytes free
true exit 0
Програмата завършена за 0.06 секунди (pid: 8832).

Resources