I need to grep in to multiple file extensions that I don't know before hand. So basically,
grep -ri --include=\*.{$1} $2 ./
Assume that first argument is supplied by the caller of the script. ABove doesn't work however because {} is not actually supported by grep by instead is expanded by bash in to multiple --include. Any idea how can I use something like above in script?
Related
For some background, my specific use case is actually in Rules Python's compile_pip_requirements rule. I do not want to use the default caching location for pip packages, so I am passing args to pip through compile_pip_requirements via:
compile_pip_requirements(
...
extra_args = [
"--allow-unsafe",
"--cache-dir",
"/tmp/pip-compile",
],
...)
I can easily hardcode the tmp dir there, but I would like to specifically use the same directory that Bazel uses to cache everything else, without making any magic assumptions about where that cache dir is.
I have tried to use $(bazel info | grep output_base | awk '{print $2}') but Bazel appears to only accept its own Make Variables within a variable expression, not to mention that this does not feel like a safe option anyway.
I have also tangled a bit with those make variables (most promisingly, $(location //foo)), but it seems this only returns the location of a given module relative to WORKSPACE.
Thanks!
Dipping my toes into Bash coding for the first time (not the most experienced person with Linux either) and I'm trying to read the version from the version.php inside a container at:
/config/www/nextcloud/version.php
To do so, I run:
docker exec -it 1c8c05daba19 grep -eo "(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*)(?:\.(?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-]*))*))?(?:\+([0-9a-zA-Z-]+(?:\.[0-9a-zA-Z-]+)*))?" /config/www/nextcloud/version.php
This uses a semantic versioning RegEx pattern (I know, a bit overkill, but it works for now) to read and extract the version from the line:
$OC_VersionString = '20.0.1';
However, when I run the command it tells me No such file or directory, (I've confirmed it does exist at that path inside the container) and then proceeds to spit out the entire contents of the file it just said doesn't exist?
grep: (0|[1-9]\d*).(0|[1-9]\d*).(0|[1-9]\d*)(?:-((?:0|[1-9]\d*|\d*[a-zA-Z-][0-9a-zA-Z-])(?:.(?:0|[1-9]\d|\d*[a-zA-Z-][0-9a-zA-Z-]))))?(?:+([0-9a-zA-Z-]+(?:.[0-9a-zA-Z-]+)*))?: No such file or directory
/config/www/nextcloud/version.php:$OC_Version = array(20,0,1,1);
/config/www/nextcloud/version.php:$OC_VersionString = '20.0.1';
/config/www/nextcloud/version.php:$OC_Edition = '';
/config/www/nextcloud/version.php:$OC_VersionCanBeUpgradedFrom = array (
/config/www/nextcloud/version.php: 'nextcloud' =>
/config/www/nextcloud/version.php: 'owncloud' =>
/config/www/nextcloud/version.php:$vendor = 'nextcloud';
Anyone able to spot the problem?
Update 1:
For the sake of clarity, I'm trying to run this from a bash script. I just want to fetch the version number from that file, to use it in other areas of the script.
Update 2:
Responding to the comments, I tried to login to the container first, and then run the grep, and still get the same result. Then I cat that file and it shows it's contents no problem.
Many containers don't have the GNU versions of Unix tools and their various extensions. It's popular to base containers on Alpine Linux, which in turn uses a very lightweight single-binary tool called BusyBox to provide the base tools. Those tend to have the set of options required in the POSIX specs, and no more.
POSIX grep(1) in particular doesn't have an -o option. So the command you're running is
grep \
-eo \ # specify "o" as the regexp to match
"(regexps are write-only)" \ # a filename
/config/www/nextcloud/version.php # a second filename
Notice that the grep output in the interactive shell only contains lines with the letter "o", but not for example the line just containing array.
POSIX grep doesn't have an equivalent for GNU grep's -o option
Print only the matched (non-empty) parts of matching lines, with each such part on a separate output line. Output lines use the same delimiters as input....
but it's easy to do that with sed(1) instead. Ask it to match some stuff, the regexp in question, and some stuff, and replace it with the matched group.
sed -e 's/.*\(any regexp here\).*/\1/' input-file
(POSIX sed only accepts basic regular expressions, so you'll have to escape more of the parentheses.)
Well, for any potential future readers, I had no luck getting grep to do it, I'm sure it was my fault somehow and not grep's, but thanks to the help in this post I was able to use awk instead of grep, like so:
docker exec -it 1c8c05daba19 awk '/^\$OC_VersionString/ && match($0,/\047[0-9]+\.[0-9]+\.[0-9]+\047/){print substr($0,RSTART+1,RLENGTH-2)}' /config/www/nextcloud/version.php
That ended up doing exactly what I needed:
It logs into a docker container.
Scans and returns just the version number from the line I am looking for at: /config/www/nextcloud/version.php inside the container.
Exits stage left from the container with just the info I needed.
I can get right back to eating my Hot Cheetos.
I am using grep (BSD grep) 2.5.1-FreeBSD in MacOS and I have found the following behavior.
I have two *.tex files. Each one of these contains the following lines
$k$-th bit of
$(i-m)$-th bit of
respectively. When I ran
grep --color -rnw . -e '\$-th bit of' --include="*.tex"
I got only the second file, i.e., $(i-m)$-th bit of, while I expect the two lines. Could you help me please to understand this behavior?
Never use -r or --include or any other grep option to find files. The GNU guys really screwed up by adding those options to grep when there's a perfectly good tool named find for finding files and now they've turned grep into a convoluted mush of finding files and Globally matching a Regular Expression within a file and Printing the result (G/RE/P).
Keep it simple - find the files with find then g/re/p within then using grep:
find . -name '*.tex' -exec grep --color -n '\$-th bit of' {} +
As others pointed out your g/re/p problem was the -w arg so I've removed that above.
I have the same version of grep.
It is caused by your use of the -w option:
-w, --word-regexp
The expression is searched for as a word (as if surrounded by `[[:<:]]' and `[[:>:]]'; see re_format(7)).
The matched part of the string $k$-th bit of is bounded on the left-hand side by a word character (i.e. k) so the match is treated as being inside a "word" and it can't therefore satisfy the "searched for as a whole word" requirement.
Try without -w and it will work fine.
while
grep -ir "xyz" * recursively searches through the directories and tell me that the text is present in ./x/y/z/abc.cpp
However ,
grep -ir "xyz" *.cpp offers no result.
Isn't the second command supposed to recursively grep all cpp files inside the directory ?
What am I missing here?
Grep will recurse through any directories you match with your glob pattern. (In your case, you probably do not have any directories that match the pattern "*.cpp") You could explicitly specify them: grep -ir "xyz" *.cpp */*.cpp */*/*.cpp */*/*/*.cpp, etc. You can also use the --include option (see the example below)
If you are using GNU grep, then you can use the following:
grep -ir --include "*.cpp" "xyz" .
The command above says to search recursively starting in current directory ignoring case on the pattern and to only search in files that match the glob pattern "*.cpp".
OR if you are on some other Unix platform, you can use this:
find ./ -type f -name "*.cpp" -print0 | xargs -0 grep -i "xyz"
If you are sure that none of your files have spaces in their names, you can omit the -print0 argument to find and the -0 to xargs
The command above says the following: find all files (-type f) under the current directory (./) that match the name glob/wildcard "*.cpp" (-name "*.cpp") and then print them out delimited by a null (-print0). That list of files found should be written to the stdin of the next command: xargs. xargs should read from stdin (default behavior) and split its input on nulls (-0) and then call the grep command with the specified options (grep -i "xyz") on that list of files.
If you are interested in learning more about why grep -ir "xyz" *.cpp does not work the way you think it should, you should search for "shell globbing" (here is a good first article on the subject). I'll also try to provide a quick explanation. When you type in the command grep -ir "xyz" *.cpp and hit enter, there are two programs that are involved in executing your command. The first program is your shell (and unless you've done something to customize things, you are probably usually the bash shell - if you've never heard of a shell or bash, that's where you should start looking, there are tons of good articles). Suffice it say that a shell is just a program that is designed to let you navigate the filesystem on your computer and run other programs. (In Windows, when you double click on an icon to launch a program, or open a folder to access a file, the program that you are running is explorer.exe and it is the Windows graphical shell). So, when you type the command grep -ir "xyz" *.cpp, before grep is run, the shell handles reading your command and does a few things. One of the things is does is expand glob patterns (things like *.txt or [0-9]+.pdf). Like I said, if you want to understand it, go read more about it, but the thing you should take away is that the grep command never sees the *.cpp. What happens is, the shell looks in the current directory for any files or directories with a name that match the pattern *.cpp and then replaces them on the command line BEFORE it runs the grep command. (If it doesn't find anything that matches, then it will leave the *.cpp there and grep will see it, but grep because doesn't normally do glob matching, this doesn't do anything for you).
Alternatively, when you type in grep -ir "xyz" *, what happens is that the shell replaces the * with the name of every file and directory in the current directory (because * matches anything). Let's say you had a directory that contained file1, file2, and dir1, and dir2, then the shell would perform its replacements and then execute a command that looked like this grep -ir "xyz" file1 file2 dir1 dir2, which means grep would search file1 and file2 for a line with the string xyz, and because of the -ir it also search recursively through dir1 and dir2 and search any files found for that string as well. Lastly, if you've followed everything I've said so far, then it will make sense to you that grep does have a way to use glob patterns on recursive searches, and that is to use the --include option, as in the command I described earlier: grep -ir --include "*.cpp" "xyz" ., and the reason why we put the *.cpp in quotes in that command is to prevent the shell from trying to expand the glob pattern before we run the command.
I am trying to create a script, and one part requires showing lines with numeric values.
My basic syntax is:
echo $i | grep [0-9]
For example, I set i=12345, it should output 12345.
But on one server, it doesn't output anything (exactly the same commands).
I do not know how to Google this issue, I have tried "grep output different on other server", to no avail.
When using a regexp, either use egrep or grep -e to make sure the pattern is not treated as a plain string.
maybe it's a shell issue? some shells interpert [] differently
try
echo "1234" | grep "[0-9]"
(with quotes)
also try
grep --version
to see if there is a different grep version