I'm writing a script, and I need to look up a command on the user's $PATH and get the full path to the command. The problem is that I don't know what the user's login shell is, or what strange stuff might be in their do files. I'm using the bourne shell for my simple little script because it needs to run on some older Solaris platforms that might not have bash.
Some implementations of "which" and "whence" will source the user's dot files, and that isn't really portable to all users. I'd love a simple UNIX utility that would just do the basic job of scanning PATH for an executable and reporting the full path of the first match.
But I'll settle for any /bin/sh solution that is stable for all users.
I'm looking for a solution that is better than writing my own /bin/sh loop that chops up $PATH and searches it one line at a time. It would seem that this is common enough that there should be an reusable way to do it.
My first approximation of the "long way" is this:
IFS=:
for i in $PATH; do
if [ -x $i/$cmd ]; then
echo $i/$cmd
fi
done
Is there something simpler and portable?
The answer seems to be the 'type' built-in.
% /bin/sh
$ type ls
ls is /bin/ls
Maybe the whereis command will work for you?
whereis -b -B `echo $PATH | sed 's/:/ /g'` -f [commands]
e.g. on my computer, this works:
whereis -b -B `echo $PATH | sed 's/:/ /g'` -f find man fsc
And results in:
find: /usr/bin/find
man: /usr/bin/man
fsc: /opt/FSharp-2.0.0.0/bin/fsc.exe /opt/FSharp-2.0.0.0/bin/fsc
One caveat from the whereis man page:
Since whereis uses chdir(2V) to run faster, pathnames given
with the -M, -S, or -B must be full; that is, they must begin
with a `/'.
This question is answered in details here: https://unix.stackexchange.com/questions/85249/why-not-use-which-what-to-use-then. Bottom line: use command -v ls.
Related
Dipping my toes into Bash coding for the first time (not the most experienced person with Linux either) and I'm trying to read the version from the version.php inside a container at:
/config/www/nextcloud/version.php
To do so, I run:
docker exec -it 1c8c05daba19 grep -eo "(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?" /config/www/nextcloud/version.php
This uses a semantic versioning RegEx pattern (I know, a bit overkill, but it works for now) to read and extract the version from the line:
$OC_VersionString = '20.0.1';
However, when I run the command it tells me No such file or directory, (I've confirmed it does exist at that path inside the container) and then proceeds to spit out the entire contents of the file it just said doesn't exist?
grep: (0|[1-9]\d*).(0|[1-9]\d*).(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-])(?:.(?:0|[1-9]\d|\d*[a-zA-Z-][0-9a-zA-Z-]))))?(?:+([0-9a-zA-Z-]+(?:.[0-9a-zA-Z-]+)*))?: No such file or directory
/config/www/nextcloud/version.php:$OC_Version = array(20,0,1,1);
/config/www/nextcloud/version.php:$OC_VersionString = '20.0.1';
/config/www/nextcloud/version.php:$OC_Edition = '';
/config/www/nextcloud/version.php:$OC_VersionCanBeUpgradedFrom = array (
/config/www/nextcloud/version.php: 'nextcloud' =>
/config/www/nextcloud/version.php: 'owncloud' =>
/config/www/nextcloud/version.php:$vendor = 'nextcloud';
Anyone able to spot the problem?
Update 1:
For the sake of clarity, I'm trying to run this from a bash script. I just want to fetch the version number from that file, to use it in other areas of the script.
Update 2:
Responding to the comments, I tried to login to the container first, and then run the grep, and still get the same result. Then I cat that file and it shows it's contents no problem.
Many containers don't have the GNU versions of Unix tools and their various extensions. It's popular to base containers on Alpine Linux, which in turn uses a very lightweight single-binary tool called BusyBox to provide the base tools. Those tend to have the set of options required in the POSIX specs, and no more.
POSIX grep(1) in particular doesn't have an -o option. So the command you're running is
grep \
-eo \ # specify "o" as the regexp to match
"(regexps are write-only)" \ # a filename
/config/www/nextcloud/version.php # a second filename
Notice that the grep output in the interactive shell only contains lines with the letter "o", but not for example the line just containing array.
POSIX grep doesn't have an equivalent for GNU grep's -o option
Print only the matched (non-empty) parts of matching lines, with each such part on a separate output line. Output lines use the same delimiters as input....
but it's easy to do that with sed(1) instead. Ask it to match some stuff, the regexp in question, and some stuff, and replace it with the matched group.
sed -e 's/.*\(any regexp here\).*/\1/' input-file
(POSIX sed only accepts basic regular expressions, so you'll have to escape more of the parentheses.)
Well, for any potential future readers, I had no luck getting grep to do it, I'm sure it was my fault somehow and not grep's, but thanks to the help in this post I was able to use awk instead of grep, like so:
docker exec -it 1c8c05daba19 awk '/^\$OC_VersionString/ && match($0,/\047[0-9]+\.[0-9]+\.[0-9]+\047/){print substr($0,RSTART+1,RLENGTH-2)}' /config/www/nextcloud/version.php
That ended up doing exactly what I needed:
It logs into a docker container.
Scans and returns just the version number from the line I am looking for at: /config/www/nextcloud/version.php inside the container.
Exits stage left from the container with just the info I needed.
I can get right back to eating my Hot Cheetos.
after the installation following the instructions with
curl https://nixos.org/nix/install | sh
and logout/login, nix-env and nix-build are not found.
I had the problem with debian stretch and now with buster. What am I doing wrong?
The nix manual instructs to execute
source ~/.nix-profile/etc/profile.d/nix.sh
but the instructions printed after the execution say to do (I do not remember exactly)
./~/.nix-profile/etc/profile.d/nix.sh
and the same command is inserted into ~/.profile. The cause of the problem is the difference between . and source (see this superuser question). The script is setting up the $PATH variable in the environment and has the desired effect wtih source but no effect with . (which operates in its own shell and closes it at the end).
Cure:
change the line in .profile (or better move it to .bashrc) to
if [ -e /home/xxx/.nix-profile/etc/profile.d/nix.sh ]; then source /home/xxx/.nix-profile/etc/profile.d/nix.sh; fi
(xxx is your user name),
You need to add this recommended script.
For me only setting $PATH like this worked (in .profile)
export PATH="$PATH:/nix/var/nix/profiles/default/bin"
when checking my $PATH on ubuntu (16.04)
I get a long list of directories, few of which even do not exist in my file
system, and some of them I just don't need:
echo $PATH
.../usr/games:/usr/local/games:/snap/bin
where they are created and how can I remove them?
I wnant to control the creation of the $PATH, rather than
correct it later by the tricks described in
https://unix.stackexchange.com/questions/108873/removing-a-directory-from-path
Some typical places where $PATH can be set when starting up a bash shell on Ubuntu include:
/etc/profile
~/.profile
~/.bashrc
where ~ represents your home directory.
Also look at any scripts called by those scripts.
There may be other things that get called when starting up a bash shell, depending on various conditions. For details, take a look at the INVOCATION section from the command:
$ man bash
See this answer from askubuntu.com to edit the path either using a text editor or the command line.
I found the answer to your question today. The path you want to edit is in /etc/environment.
I've tried installing GNU grep on OSX, and it seems to be installed, but I can't use it.. I've done so using homebrew, Macports is having some issues currently, so I can't use that.
To install: brew tap homebrew/dupes; brew install grep
Which returns: Warning: homebrew/dupes already tapped! Warning: homebrew/dupes/grep-2.21 already installed
Symlinking seems to work to /usr/local/bin/ggrep. When I add the alias alias grep="ggrep" and do grep --version, I get -bash: ggrep: command not found. Which is true, since there is no ggrep in the folder. I've tried installing with and without --with-default-names.
The folder /usr/local/Cellar/grep/2.21/bin/ contains the following:
-r-xr-xr-x 1 Wes admin 158 Oct 14 09:27 egrep
-r-xr-xr-x 1 Wes admin 158 Oct 14 09:27 fgrep
Which is strange to me, since the documentation implies that The command has been installed with the prefix "g".
I've seen the following post, but none of the solutions work for me. Updating grep for Mac OS 10.7
Does anyone have any solutions? I really want to use GNU grep.
Output of brew unlink grep && brew link grep -v:
Unlinking /usr/local/Cellar/grep/2.21...
6 symlinks removed
Linking /usr/local/Cellar/grep/2.21...
ln -s ../Cellar/grep/2.21/bin/egrep egrep
ln -s ../Cellar/grep/2.21/bin/fgrep fgrep
ln -s ../../Cellar/grep/2.21/share/info/grep.info grep.info info /usr/local/share/info/grep.info
ln -s ../../../Cellar/grep/2.21/share/man/man1/egrep.1 egrep.1
ln -s ../../../Cellar/grep/2.21/share/man/man1/fgrep.1 fgrep.1
ln -s ../../../Cellar/grep/2.21/share/man/man1/grep.1 grep.1
6 symlinks created`
New:
brew uninstall grep; brew install grep
$ which -a grep
/usr/bin/grep
$ which -a ggrep
/usr/local/bin/ggrep
$ echo $PATH
/opt/local/bin:/opt/local/sbin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/git/bin
This time, it seems something is different. ggrep is finally installed! I think the unlink/link straightened some things out.
How can I set ggrep as the default? With alias?
To make GNU grep the default install it with --with-default-names:
$ brew install grep --with-default-names
If you already have it installed use reinstall instead of install.
Ensure that /usr/local/bin (the location of GNU grep) is before /usr/bin (the location of the BSD grep) in your $PATH; which seems to be the case here.
You might have to start a new shell session afterward because Bash caches the binaries paths for the current session. This means that the first time you use grep it’ll determine which binary it’ll use depending on your $PATH and cache it. The next time it’ll use the cached value so changing your $PATH won’t change anything until you reload the shell.
Offically out of date for the answer above.
As of Homebrew version 2.0.0 the --with-default-names flag is no longer available.
from the official documentation
--with-default-names is no longer supported. It is now installed into its own directory and you will need to adjust your PATH to use it.
What you need to do is to add this command to your shell
PATH="/usr/local/opt/grep/libexec/gnubin:$PATH"
As of Febuary 8, 2020, You could either prepend the PATH and MANPATH variables as eleijonmarck proposes, or use symbolic links to a path loaded early on the PATH. If you choose to use eleijonmarck's method, I would do it differently.
You need to know where Homebrew installs grep binaries.
PATH=/usr/local/opt/grep/bin:$PATH
as opposed to libexec. The Filesystem Hierarchy specifies that the libexec directory
includes internal binaries that are not intended to be executed directly by users or shell scripts.
Also, if you are doing that, then you might want the MAN pages too.
MANPATH=/usr/local/opt/grep/share/man:$MANPATH
The other option is to use symbolic links. You might think to do this:
ln -s /usr/local/opt/grep/bin/grep /usr/local/bin/
The thing is, this won't work, because MacOS has grep as a built-in. Instead, would need to remove or replace the built-in at /usr/bin. You could rename it mv /usr/bin/grep /usr/bin/bsdgrep. This effectively makes grep not findable at /usr/bin, but still usable as bsdgrep (advantage of this technique). Assuming you added grep to /usr/local/bin, you could test your change with:
which grep && grep -V # prove old grep found first
which grep && grep -V # prove new grep found first
The disadvantage of this technique is that you must add a symbolic link per binary.
Is there a way to run shell commands without output buffering?
For example, hexdump file | ./my_script will only pass input from hexdump to my_script in buffered chunks, not line by line.
Actually I want to know a general solution how to make any command unbuffered?
Try stdbuf, included in GNU coreutils and thus virtually any Linux distro. This sets the buffer length for input, output and error to zero:
stdbuf -i0 -o0 -e0 command
The command unbuffer from the expect package disables the output buffering:
Ubuntu Manpage: unbuffer - unbuffer output
Example usage:
unbuffer hexdump file | ./my_script
AFAIK, you can't do it without ugly hacks. Writing to a pipe (or reading from it) automatically turns on full buffering and there is nothing you can do about it :-(. "Line buffering" (which is what you want) is only used when reading/writing a terminal. The ugly hacks exactly do this: They connect a program to a pseudo-terminal, so that the other tools in the pipe read/write from that terminal in line buffering mode. The whole problem is described here:
http://www.pixelbeat.org/programming/stdio_buffering/
The page has also some suggestions (the aforementioned "ugly hacks") what to do, i.e. using unbuffer or pulling some tricks with LD_PRELOAD.
You could also use the script command to make the output of hexdump line-buffered (hexdump will be run in a pseudo terminal which tricks hexdump into thinking its writing its stdout to a terminal, and not to a pipe).
# cf. http://unix.stackexchange.com/questions/25372/turn-off-buffering-in-pipe/
stty -echo -onlcr
script -q /dev/null hexdump file | ./my_script # FreeBSD, Mac OS X
script -q -c "hexdump file" /dev/null | ./my_script # Linux
stty echo onlcr
One should use grep or egrep "--line-buffered" options to solve this. no other tools needed.