One of our shared hosting sites got moved recently. New server is Red Hat 4.8.5-36. The other binaries' versions are grep (GNU grep) 2.20 and find (GNU findutils) 4.5.11
This cron job had previously functioned fine for at least 6 years and gave us a list of updated files which did not match logs, cache etc.
find /home/example/example.com/public_html/ -mmin -12 \
| grep -v 'error_log|logs|cache'
After the move the -v seems to be ineffectual and we get results like
/home/example/example.com/public_html/products/cache/ssu/pc/d/5/c
The change in results occurred immediately after the move. Anyone have an idea why it is now broken? Additionally - how do I restore the filtered output?
If you like to exclude a group of words.
grep -v -e 'error_log' -e 'logs' -e 'cache' file
With awk you can do:
awk '!/error_log|logs|cache/' file
It will exclude all lines with these words.
grep -v 'error_log|logs|cache'
only excludes strings that contain literally error_log|logs|cache. To use alternation, use extended regular expressions:
grep -Ev 'error_log|logs|cache'
GNU grep supports alternation as an extension to Basic Regular Expressions, but | needs to be escaped, so this might work as well:
grep -v 'error_log\|logs\|cache'
However, grep isn't required in the first place, we can use (GNU) find to do all the work:
find /home/example/example.com/public_html/ -mmin -12 \
-not \( -name '*error_log*' -or -name '*logs*' -or -name '*cache*' \)
or, POSIX compliant:
find /home/example/example.com/public_html/ -mmin -12 \
\! \( -name '*error_log*' -o -name '*logs*' -o -name '*cache*' \)
or, if your find supports -regex (both GNU and BSD find do):
find /home/example/example.com/public_html/ -mmin -12 \
-not -regex '.*\(error_log\|logs\|cache\).*'
Related
I was wondering if I can treat a deprecation with a specific message as an error? Say I'm trying to cut out some part of the codebase so I added DEPRECATED_MSG_ATTRIBUTE("my custom message"). Now I'd like to treat only these custom deprecations as errors.
I have a script run at the build phase which I think will do what you want called 'todoToWarning.sh' :
TAGS="TODO:|FIXME:"
echo "searching ${SRCROOT} for ${TAGS}"
find "${SRCROOT}" \( -name "*.h" -or -name "*.m" \) -print0 | xargs -0 egrep --with-filename --line-number --only-matching "($TAGS).*\$" | perl -p -e "s/($TAGS)/ warning: \$1/"
This tags all my "TODO:" as Warnings in Xcode. So simply replace the TAGS with your identifiers and change the 'warning' to 'error' on the last line. Then add the "Run Script" to your build phase and build the project!
I have written script that shows Xcode warnings. e.g TODO warnings. This script will run on each build of XCode. (I have written script in "Run Phase" option).
Now I want to collect and export all these warnings to text files. Is there any way to export all warnings or build errors to a text file?
(the first bit of this is what you've already done, or something like it)
Outputting TODO, etc, as warnings.
Select your project, click the Build Phases tab, and select 'Add Build Phase > Add Run Script Build Phase' from the 'Editor' menu.
In the script box use a script something like this:
KEYWORDS="TODO:|FIXME:|\?\?\?:|\!\!\!:"
find "${SRCROOT}" \( -name "*.h" -or -name "*.m" \) -print0 | xargs -0 egrep --with-filename --line-number --only-matching "($KEYWORDS).*\$" | perl -p -e "s/($KEYWORDS)/ warning: \$1/"
(courtesy of: http://deallocatedobjects.com/posts/show-todos-and-fixmes-as-warnings-in-xcode-4)
The KEYWORDS regular expression matches TODO:, FIXME:, ???: and !!!:, but could be adjusted to find whichever indicators you want.
Making this output to a file.
The script currently outputs to stdout, which is picked up by XCode and parsed. To make it also log to a file, use tee as part of the script (see the end of line 2 for the change):
KEYWORDS="TODO:|FIXME:|\?\?\?:|\!\!\!:"
find "${SRCROOT}" \( -name "*.h" -or -name "*.m" \) -print0 | xargs -0 egrep --with-filename --line-number --only-matching "($KEYWORDS).*\$" | perl -p -e "s/($KEYWORDS)/ warning: \$1/" | tee "${SRCROOT}/NOTICES.txt"
This approach can be as complex as you like, of course, as well as teeing to a file, we can augment the script to do anything we choose:
KEYWORDS="TODO:|FIXME:|\?\?\?:|\!\!\!:"
find "${SRCROOT}" \( -name "*.h" -or -name "*.m" \) -print0 | xargs -0 egrep --with-filename --line-number --only-matching "($KEYWORDS).*\$" | perl -p -e "s/($KEYWORDS)/ warning: \$1/" | tee ${SRCROOT}/NOTICES.txt
mail -s NOTICES idmillington#example.com < ${SRCROOT}/NOTICES.txt
That emails it to me.
I've confirmed this works with XCode 5.0.2, including emailing.
Note that this does not export all warnings from the build to a file, which is strictly what you asked. I can't find a way to automate this in XCode 5.0.2, though you can do it with xcodebuild. From within the UI, the only option is to copy the log text from the log navigator to the clipboard, it seems.
I'm grepping through a large pile of code managed by git, and whenever I do a grep, I see piles and piles of messages of the form:
> grep pattern * -R -n
whatever/.git/svn: No such file or directory
Is there any way I can make those lines go away?
You can use the -s or --no-messages flag to suppress errors.
-s, --no-messages suppress error messages
grep pattern * -s -R -n
If you are grepping through a git repository, I'd recommend you use git grep. You don't need to pass in -R or the path.
git grep pattern
That will show all matches from your current directory down.
Errors like that are usually sent to the "standard error" stream, which you can pipe to a file or just make disappear on most commands:
grep pattern * -R -n 2>/dev/null
I have seen that happening several times, with broken links (symlinks that point to files that do not exist), grep tries to search on the target file, which does not exist (hence the correct and accurate error message).
I normally don't bother while doing sysadmin tasks over the console, but from within scripts I do look for text files with "find", and then grep each one:
find /etc -type f -exec grep -nHi -e "widehat" {} \;
Instead of:
grep -nRHi -e "widehat" /etc
I usually don't let grep do the recursion itself. There are usually a few directories you want to skip (.git, .svn...)
You can do clever aliases with stances like that one:
find . \( -name .svn -o -name .git \) -prune -o -type f -exec grep -Hn pattern {} \;
It may seem overkill at first glance, but when you need to filter out some patterns it is quite handy.
Have you tried the -0 option in xargs? Something like this:
ls -r1 | xargs -0 grep 'some text'
Use -I in grep.
Example: grep SEARCH_ME -Irs ~/logs.
I redirect stderr to stdout and then use grep's invert-match (-v) to exclude the warning/error string that I want to hide:
grep -r <pattern> * 2>&1 | grep -v "No such file or directory"
I was getting lots of these errors running "M-x rgrep" from Emacs on Windows with /Git/usr/bin in my PATH. Apparently in that case, M-x rgrep uses "NUL" (the Windows null device) rather than "/dev/null". I fixed the issue by adding this to .emacs:
;; Prevent issues with the Windows null device (NUL)
;; when using cygwin find with rgrep.
(defadvice grep-compute-defaults (around grep-compute-defaults-advice-null-device)
"Use cygwin's /dev/null as the null-device."
(let ((null-device "/dev/null"))
ad-do-it))
(ad-activate 'grep-compute-defaults)
One easy way to make grep return zero status all the time is to use || true
→ echo "Hello" | grep "This won't be found" || true
→ echo $?
0
As you can see the output value here is 0 (Success)
Is there any way I could use grep to ignore some files when searching something, something equivalent to svnignore or gitignore? I usually use something like this when searching source code.
grep -r something * | grep -v ignore_file1 | grep -v ignore_file2
Even if I could set up an alias to grep to ignore these files would be good.
--exclude option on grep will also work:
grep perl * --exclude=try* --exclude=tk*
This searches for perl in files in the current directory excluding files beginning with try or tk.
You might also want to take a look at ack which, among many other features, by default does not search VCS directories like .svn and .git.
find . -path ./ignore -prune -o -exec grep -r something {} \;
What that does is find all files in your current directory excluding the directory (or file) named "ignore", then executes the command grep -r something on each file found in the non-ignored files.
Use shell expansion
shopt -s extglob
for file in !(file1_ignore|file2_ignore)
do
grep ..... "$file"
done
I thinks grep does not have filename filtering.
To accomplish what you are trying to do, you can combine find, xargs, and grep commands.
My memory is not good, so the example might not work:
find -name "foo" | xargs grep "pattern"
Find is flexible, you can use wildcards, ignore case, or use regular expressions.
You may want to read manual pages for full description.
after reading next post, apparently grep does have filename filtering.
Here's a minimalistic version of .gitignore. Requires standard utils: awk, sed (because my awk is so lame), egrep:
cat > ~/bin/grepignore #or anywhere you like in your $PATH
egrep -v "`awk '1' ORS=\| .grepignore | sed -e 's/|$//g' ; echo`"
^D
chmod 755 ~/bin/grepignore
cat >> ./.grepignore #above set to look in cwd
ignorefile_1
...
^D
grep -r something * | grepignore
grepignore builds a simple alternation clause:
egrep -v ignorefile_one|ignorefile_two
not incredibly efficient, but good for manual use
I'm getting frustrated enough that I figured it was time to ask a question.
I'm trying to replace an email address across a website that is hard coded into 1000's of pages. It's on a FreeBSD 6.3 server.
Here is the command I am using:
grep -R --files-with-matches 'Email\#domain.com' . | sort | uniq | xargs perl -pi -e 's/Email\#domain.com/Email\#newdomain.com/' *.html
And here is the error that I keep getting:
xargs: unterminated quote
Oddly enough, when I run that command on a test case of 3 files (in a nested structure) it works just fine. I've been googling and most solutions seem to deal with adding a -print0 after the . and a -0 after the xargs. However, this yields a different set of errors that lead me to believe I'm putting things in the wrong places.
thanks in advance for your help
Pax is correct. I would further correct it to something like:
grep -R --files-with-matches 'Email\#domain.com' . -print0 | xargs -0 perl -pi -e 's/Email\#domain.com/Email\#newdomain.com/'
EDIT:
Thanks to kcwu, this is the full FreeBSD:
grep -R --files-with-matches 'Email\#domain.com' . --null | xargs -0 perl -pi -e 's/Email\#domain.com/Email\#newdomain.com/'
Note that I've removed sort and uniq. --files-without-match is documented to "stop on the first match" so you will not get duplicate files. -print0 and -0 ensure (and handle) a null-terminated file list, which is vital, because POSIX allows filenames to contain newlines.
Note that I don't know perl, but I'm assuming that part's roughly equivalent to:
sed -i s/Email\#domain.com/Email\#newdomain.com/g
Why are you giving a list of HTML files to xargs? That program takes its file list from the pipeline (output of grep).
Use GNU Parallel:
grep -R --files-with-matches 'Email\#domain.com' . | sort | uniq | parallel -q perl -pi -e 's/Email\#domain.com/Email\#newdomain.com/g'
Watch the intro video to learn more: http://www.youtube.com/watch?v=OpaiGYxkSuQ