grep -f does not find files in loop - grep

I want to use grep -f in a loop but it's not seeing the files I give to -f. My grep version from grep -V:
grep (GNU grep) 3.6
Copyright (C) 2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <https://gnu.org/licenses/gpl.html>.
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.
Written by Mike Haertel and others; see
<https://git.sv.gnu.org/cgit/grep.git/tree/AUTHORS>.
Example:
echo "line1" > searchfile.1
echo "line2" > searchfile.2
echo "line1" > targetfile
for file in `ls searchfile*`; do echo $file; ls $file; grep -f $file targetfile; done
gives the output
searchfile.1
ls: cannot access ''$'\033''[0m'$'\033''[32msearchfile.1'$'\033''[0m': No such file or directory
grep: searchfile.1: No such file or directory
searchfile.2
ls: cannot access ''$'\033''[32msearchfile.2'$'\033''[0m': No such file or directory
grep: searchfile.2: No such file or directory
But if I do it manually like
grep -f searchfile.1 targetfile
I get
line1
Any ideas what could be going on?

Don't parse ls output, use find:
find . -mindepth 1 -maxdepth 1 -name 'searchfile*' -exec echo {} \; -exec grep -f {} targetfile \;
Output:
./searchfile.2
./searchfile.1
line1
ls outputs other characters in addition to the file names (for the colored ls output). Plus, there may be whitespace in the file names (not in your case, though). See also:
Why not parse ls (and what to do instead)? - https://unix.stackexchange.com/q/128985/13411

Two alternatives to Timur Shatland's find solution.
Use shell wildcard expansion
for file in searchfile*; do ...
Work around the ls alias by putting it inside quotes
for file in `"ls" searchfile*`; do ...

Related

Linux: Search through sub-folders recursively for a file that contains a string and move it to another file

So far, I have this command on my terminal and it doesn't do anything.
Essentially it's to look for any file that contains the word bango and move it to another directory.
grep -r ".*bango.*" /Users/user/Desktop/drums | xargs mv /Users/user/Desktop/bango
Grep has a function to list the filename only you should use that to list the name of the files.
Also xargs can build commands with positional arguments.
Try to use
grep -rlE ".*bango.*" /Users/user/Desktop/drums | xargs -I # mv # /Users/user/Desktop/bango
The option -E allows to use regular expressions.
However, a regular expression is not needed, you can activate a fast grep algorithm for fixed strings:
grep -rlF "bango" /Users/user/Desktop/drums | xargs -I # mv # /Users/user/Desktop/bango

How do you exclude symlinks in a grep?

I want to grep -R a directory but exclude symlinks how dow I do it?
Maybe something like grep -R --no-symlinks or something?
Thank you.
Gnu grep v2.11-8 and on if invoked with -r excludes symlinks not specified on the command line and includes them when invoked with -R.
If you already know the name(s) of the symlinks you want to exclude:
grep -r --exclude-dir=LINK1 --exclude-dir=LINK2 PATTERN .
If the name(s) of the symlinks vary, maybe exclude symlinks with a find command first, and then grep the files that this outputs:
find . -type f -a -exec grep -H PATTERN '{}' \;
The '-H' to grep adds the filename to the output (which is the default if grep is searching recursively, but is not here, where grep is being handed individual file names.)
I commonly want to modify grep to exclude source control directories. That is most efficiently done by the initial find command:
find . -name .git -prune -o -type f -a -exec grep -H PATTERN '{}' \;
For now.. here is how I would exclude symbolic links when using grep
If you want just file names matching your search:
for f in $(grep -Rl 'search' *); do if [ ! -h "$f" ]; then echo "$f"; fi; done;
Explaination:
grep -R # recursive
grep -l # file names only
if [ ! -h "file" ] # bash if not a symbolic link
If you want the matched content output, how about a double grep:
srch="whatever"; for f in $(grep -Rl "$srch" *); do if [ ! -h "$f" ]; then
echo -e "\n## $f";
grep -n "$srch" "$f";
fi; done;
Explaination:
echo -e # enable interpretation of backslash escapes
grep -n # adds line numbers to output
.. It's not perfect of course. But it could get the job done!
If you're using an older grep that does not have the -r behavior described in Aryeh Leib Taurog's answer, you can use a combination of find, xargs and grep:
find . -type f | xargs grep "text-to-search-for"
If you are using BSD grep (Mac) the following works similar to '-r' option of Gnu grep.
grep -OR <PATTERN> <PATH> 2> /dev/null
From man page
-O If -R is specified, follow symbolic links only if they were explicitly listed on the command line.

Delete a list of files with find and grep

I want to delete all files which have names containing a specific word, e.g. "car".
So far, I came up with this:
find|grep car
How do I pass the output to rm?
find . -name '*car*' -exec rm -f {} \;
or pass the output of your pipeline to xargs:
find | grep car | xargs rm -f
Note that these are very blunt tools, and you are likely to remove files that you did not intend to remove. Also, no effort is made here to deal with files that contain characters such as whitespace (including newlines) or leading dashes. Be warned.
To view what you are going to delete first, since rm -fr is such a dangerous command:
find /path/to/file/ | grep car | xargs ls -lh
Then if the results are what you want, run the real command by removing the ls -lh, replacing it with rm -fr
find /path/to/file/ | grep car | xargs rm -fr
I like to use
rm -rf $(find . | grep car)
It does exactly what you ask, logically running rm -rf on the what grep car returns from the output of find . which is a list of every file and folder recursively.
You can use ls and grep to find your files and rm -rf to delete the files.
rm -rf $(ls | grep car)
But this is not a good idea to use this command if there is a chance of directories or files, you don't want to delete, having names with the character pattern you are specifying with grep.
You really want to use find with -print0 and rm with --:
find [dir] [options] -print0 | grep --null-data [pattern] | xargs -0 rm --
A concrete example (removing all files below the current directory containing car in their filename):
find . -print0 | grep --null-data car | xargs -0 rm --
Why is this necessary:
-print0, --null-data and -0 change the handling of the input/output from parsed as tokens separated by whitespace to parsed as tokens separated by the \0-character. This allows the handling of unusual filenames (see man find for details)
rm -- makes sure to actually remove files starting with a - instead of treating them as parameters to rm. In case there is a file called -rf and do find . -print0 | grep --null-data r | xargs -0 rm, the file -rf will possibly not be removed, but alter the behaviour of rm on the other files.
This finds a file with matching pattern (*.xml) and greps its contents for matching string (exclude="1") and deletes that file if a match is found.
find . -type f -name "*.xml" -exec grep exclude=\"1\" {} \; -exec rm {} \;
Most of the other solutions presented here have problems with handling file names with spaces in them. Here's a solution that handles spaces properly.
grep -lRZ car . | xargs -0 rm
Notes on arguments used:
-l tells grep to print only filenames
-R enables grep recursive search in subfolders
-Z tells grep to separate results by \0 instead of \n
-0 tells xargs to separate input arguments by \0 instead of whitespace
car is the regular expression to search for
. is the folder where to search
Can also use rm -f to force the removal (as usual).
A bit of necromancy, but you can also use find, grep, and xargs
find . -type f | grep -e "pattern1" -e "pattern2" | xargs rm -rf
^ Find will need some attention to make it work for your needs potentially, such as is a file, mindepth, maxdepth and any globbing.
when find | grep car | xargs rm -f get results:
/path/to/car
/path/to/car copy
some files which contain whitespace will not be removed.
So my answer is:
find | grep car | while read -r line ; do
rm -rf "${line}"
done
So the file contains whitespace could be removed.
find start_dir -iname \*car\* -exec rm -v {} \;
I use:
find . | grep "car" | while read i; do echo $i; rm -f "$i"; done
This works even if there are spaces in the filename as well as in recursive manner, searching for directories as well.
Use rm with wildcard *
rm * will delete all files
rm *.ext will delete all files which have ext as extension
rm word* will delete all files which starts with word.

use cat and grep to look for a bunch of files and give me info while retaining file name

so I am running the following command
cat /Users/sars/logs/testlogs/2012-04-02*/*/top |grep -H "httpd"
I'm using the * because there are a bunch of directories (which is actually the information I am looking for) and looking for the phrase httpd in the top output
But when I do this I get (standard input): 4951 root 1 96 0 14052K 6844K select 2 0:12 0.00% httpd
instead of the filename
how do I go through these directories look in the top file and find the lines with httpd in them while maintaining the name and path of the file it is found in?
grep can take the filenames as arguments:
grep -H "httpd" /Users/sars/logs/testlogs/2012-04-02*/*/top
ack is a good tool for this sort of thing, but is nonstandard. To do it with grep, you probably want to use find:
find /Users/sars/logs/testlogs -type f \
-wholename '/Users/sars/logs/testlogs/2012-04-02*/*/top/*' \
-exec grep -H httpd {} \;

grep multiple extension current and subfolders

I'm trying to grep multiple extensions within the current and all sub-folders.
grep -i -r -n 'hello' somepath/*.{php,html}
This is only grepping the current folder but not sub-folders.
What would be a good way of doing this?
Using only grep:
grep -irn --include='*.php' --include='*.html' 'hello' somepath/
One of these:
find '(' -name '*.php' -o -name '*.html' ')' -exec grep -i -n hello {} +
find '(' -name '*.php' -o -name '*.html' ')' -print0 | xargs -0 grep -i -n hello
I was looking the same and when decided to do a bash script I started with vim codesearch and surprise I already did this before!
#!/bin/bash
context="$3"
#ln = line number mt = match mc = file
export GREP_COLORS="sl=32:mc=00;33:ms=05;40;31:ln="
if [[ "$context" == "" ]]; then context=5; fi
grep --color=always -n -a -R -i -C"$context" --exclude='*.mp*'\
--exclude='*.avi'\
--exclude='*.flv'\
--exclude='*.png'\
--exclude='*.gif'\
--exclude='*.jpg'\
--exclude='*.wav'\
--exclude='*.rar'\
--exclude='*.zip'\
--exclude='*.gz'\
--exclude='*.sql' "$2" "$1" | less -R
paste this code into in a file named codesearch and set the chmod to 700 or 770
I guess this could be better here for the next time that I forgot
this script will show with colors the matches and the context around
./codesearch '/full/path' 'string to search'
and optional defining the number of context line around default 5
./codesearch '/full/path' 'string to search' 3
I edited the code and added some eye candy
example ./codesearch ./ 'eval' 2
Looks like this when you have enabled "allow blinking text" in terminal

Resources