How can I make grep read the expression from standard input (stdin)?
For example (the following doesn't work):
grep -i -f &0 /path/to/text/file < "/regexp/"
Use -f with a single dash to denote the standard input:
$ echo Content | grep -f - notice.html
<meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1">
...
Note: This has been tested with GNU grep - I am not sure if it's specified by POSIX.
Related
I want the output of the sed file edit to go into my log file name d_selinuxlog.txt. Currently, grep outputs the specified string as well as 3 other strings above and below in the edited file.
#!/bin/bash
{ getenforce;
sed -i s/SELINUX=enforcing/SELINUX=disabled /etc/selinux/config;
grep "SELINUX=*" /etc/selinux/config > /home/neb/scropts/logs/d_selinuxlog.txt;
setenforce 0;
getenforce; }
I want to be seeing just SELINUX=disabled in the log file
All the lines with the lines SELINUX are going to match, even the commented ones, so, you need to omit that ones, and the * from the match.
grep "SELINUX=" /etc/selinux/config | grep -v "#"
This is my output
17:52:07 alvaro#lykan /home/alvaro
$ grep "SELINUX=" /etc/selinux/config | grep -v "#"
SELINUX=disabled
17:52:22 alvaro#lykan /home/alvaro
I have a space-separated file that looks like this:
$ cat in_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004927566.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004919950.1 FAD_binding_3
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
I am using the following shell script utilizing grep to search for strings:
$ cat search_script.sh
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
The problem is that I want each grep command to return only the first instance of the string it finds exclusive of the previous identical grep command's output.
I need an output which would look like this:
$ cat out_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
in which line 1 is exclusively the output of the first grep command and line 2 is exclusively the output of the second grep command. How do I do it?
P.S. I am running this on a big file (>125,000 lines). So, search_script.sh is mostly composed of unique grep commands. It is the identical commands' execution that is messing up my downstream analysis.
I'm assuming you are generating search_script.sh automatically from the contents of in_file. If you can count how many times you'll repeat the same grep command you can just use grep once and use head, for example if you know you'll be using it 2 times:
grep "foo" bar.txt | head -2
Will output the first 2 occurrences of "foo" in bar.txt.
If you have to do the grep commands separately, for example if you have other code in between the grep commands, you can mix head and tail:
grep "foo" bar.txt | head -1 | tail -1
Some other commands...
grep "foo" bar.txt | head -2 | tail -1
head -n displays the first n lines of the input
tail -n displays the last n lines of the input
If you really MUST always use the same command, but ensure that the outputs always differ, the only way I can think of to achieve this is using temporary files and a complex sequence of commands:
cat foo.bar.txt.tmp 2>&1 | xargs -I xx echo "| grep -v \\'xx\\' " | tr '\n' ' ' | xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp"
So to explain this command, given foo as a search string and bar.txt as the filename, then foo.bar.txt.tmp is a unique name for a temporary file. The temporary file will hold the strings that have already been output:
cat foo.bar.txt.tmp 2>&1 : outputs the contents of the temporary file. If none is present, will output an error message to stdout, (important because if the output was empty the rest of the command wouldn't work.)
xargs -I xx echo "| grep -v \\'xx\\' " adds | grep -v to the start of each line in the temporary file, grep -v something excludes lines that include something.
tr '\n' ' ' replaces newlines with spaces, to have on a single string a sequence of grep -vs.
xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp" runs a new command, grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp, replacing xx with the previous output. xx should be the sequence of grep -vs that exclude previous outputs.
head -1 makes sure only one line is output at a time
tee -a foo.bar.txt.tmp appends the new output to the temporary file.
Just be sure to clear the temporary files, rm *.tmp, at the end of your script.
If I am getting question right and you want to remove duplicates based on last field of each line then try following(this should be easy task for awk).
awk '!a[$NF]++' Input_file
grep -w "ing_[0-9][0-9][0-9][0-9]"
The command mentioned above is working. But is there a short version of 4 digits?
This does not work:
grep -w "ing_[0-9]\+ {4}"
Grep by default use Basic Regular expressions. In BRE , you need to escape the curly braces so that it would consider the curly braces as repetition quantifier.
grep -w "ing_[0-9]\{4\}" file
Example:
$ echo 'ing_6786 says' | grep -w "ing_[0-9]{4}"
$ echo 'ing_6786 says' | grep -w "ing_[0-9]\{4\}"
ing_6786 says
If you are lucky and your grep supports modern (Perl) regular expressions, try -P argument
grep -wP "ing_[0-9]{4}"
How to filter anything inside "<torrent:magnetURI><![CDATA[" and "]]></torrent:magnetURI>" so it would output the string "EXAMPLE" using grep?
<torrent:magnetURI><![CDATA[EXAMPLE]]></torrent:magnetURI>
I'm trying to get all the magnet url in the web and add them to transmission.
for url in $(wget -q -O- "http://sample.com/rss.xml" | grep -o '<torrent:magnetURI><![CDATA["[^"]*' | grep -o '[^>]*$'); do
transmission-remote localhost:9091 -a "$url";
done
You can use:
$ grep -Po '(?<=<torrent:magnetURI><!\[CDATA\[)\w*(?=\]\]>)' file
EXAMPLE
Note it is using a look behind and look forward (?<=before)\w*(?=after), also escaping the [:
(?<=<torrent:magnetURI><!\[CDATA\[)\w*(?=\]\]>)
------------------------------- --- -----
string to find before | string after
string matched
The following command gives me a list of matching expressions:
grep -f /tmp/list Filename* > /tmp/output
The list file is then parsed and used to search Filename* for the parsed string. The results are then saved to output.
How would I output the parsed string from list in the case where there is no match in Filename*?
Contents of the list file could be:
ABC
BLA
ZZZ
HJK
Example Files:
Filename1:5,ABC,123
Filename2:5,ZZZ,342
Result of Running Command:
BLA
HJK
Stack overflow question 2480584 looks like it may be relevant, through the use of an if statement. However I'm not sure how to output the parsed string to the output file. Would require some type of read line?
TIA,
Mic
Obviously, grep -f list Filename* gives all matches of patterns from the file list in the files specified by Filename*, i.e.,
Filename1:5,ABC,123
Filename2:5,ZZZ,342
in your example.
By adding the -o (only print matching expression) and -h (do not print filename) flags, we can turn this into:
ABC
ZZZ
Now you want all patterns from list that are not contained in this list, which can be achieved by
grep -f list Filename* -o -h | grep -f /dev/stdin -v list
where the second grep takes it's patterns from the output of the first and by using the -v flag gives all the lines of file list that do not match those patterns.
This makes it:
$ grep -v "$(cat Filename* | cut -d, -f2)" /tmp/list
BLA
HJK
Explanation
$ cat Filename* | cut -d, -f2
ABC
ZZZ
And then grep -v looks for the inverse matching.