# spam.txt
src/blahblah/potato.py
src/migrations/helloworld.py
spam/blahblah/thing.py
src/thingy/other.cpp
I have this:
$ cat spam.txt | grep 'src.*\.py' | grep --invert-match '/migrations/'
src/blahblah/potato.py
How can I chain the exclude into the same pattern? i.e. how can I write the 'pattern???' below so that I get the same result?
$ grep 'pattern???' spam.txt
Better
awk '/src/ && /\.py/ && ! /migrations/' spam.txt
Example
Related
I have a problem with this Linux command:
ls | grep -E 'i{2,3}'
.It should take a file that has at least 2 i and max 3 i, but it doesn't work.
This is the output
ls:
life.py, viiva.txt, viiiiiiiiiva.txt
grep:
viiva.txt, viiiiiiiiiva.txt (with the first 3 I highlighted)
Thanks for the help.
Issue with OP's attempt grep -E 'i{2,3}' will match two or three consecutive occurrences of i anywhere in the input, so 4 or more consecutive i is also a valid match.
Parsing ls output is not recommended, see Why not parse ls (and what to do instead)?. If you wish to pass the filenames after filtering to some other command, find is a good option.
$ ls
1i2i3i.txt aibi.txt II.txt life.py viiiiiiiiiva.txt viiva.txt
$ # files with 2 or 3 consecutive i
$ # note that the regex will act on entire filename, thus anchors are not needed
$ find -type f -regextype egrep -regex '[^i]*i{2,3}[^i]*'
./viiva.txt
$ # files with 2 or 3 i anywhere in the name
$ find -type f -regextype egrep -regex '[^i]*i[^i]*i[^i]*(i[^i]*)?'
./aibi.txt
./1i2i3i.txt
./viiva.txt
$ # files with 2 or 3 i anywhere in the name, ignoring case
$ find -type f -regextype egrep -iregex '[^i]*i[^i]*i[^i]*(i[^i]*)?'
./II.txt
./aibi.txt
./1i2i3i.txt
./viiva.txt
If filenames won't cause an issue, you can grep -xE or grep -ixE with above regex, where x option will make sure the regex matches the whole line, instead of anywhere in the line. Or you can also use awk:
$ # NF will give number of fields after splitting on i
$ ls | awk -F'i' 'NF>=3 && NF<=4'
1i2i3i.txt
aibi.txt
viiva.txt
$ ls | awk -F'[iI]' 'NF>=3 && NF<=4'
1i2i3i.txt
aibi.txt
II.txt
viiva.txt
I am using grep to find match of 2 patterns with condition OR like this, classically:
grep -E 'C_matrix|F_matrix' triplot_XC_dev_for_right_order_with_FoM.py | wc -l
I would like now to exclude the cases when both pattern are matched on the same line, i.e I would like to use a XOR operator with grep.
How can I do this operation ? Maybe another trick is possible (I think about grep -v to exclude but this would be nice to do this operation in ony one command line with grep -E).
When you want to make such a special case, it is better to make use of awk:
$ awk '(/C_matrix/ && !/F_matrix/) || (!/C_matrix/ && !/F_matrix/)' file
Using GNU awk, you can use the bit-manipulation function xor:
$ awk 'xor(/C_matrix/,/F_matrix/)' file
I have a bunch of strings that I have to fetch the 'port_num' from -
"76 : client=new; tags=circ, LINK; port_num=switch01; far_port=Gi1/0"
The word might be in a different place in the string and it might be a different length, but it always says 'port_num=' before it and ';' after it...
I only want this bit- 'switch01'
Currently I use-
| grep -Eo 'port_num=.+' | cut -d"=" -f2 | cut -d";" -f1'
But there has got to be a better way
You can try grep -oP '(?<=port_num=).+(?=;)', if you run this:
echo "76 : client=new; tags=circ, LINK; port_num=switch01; far_port=Gi1/0" \
| grep -oP '(?<=port_num=).+(?=;)'
result will be:
switch01
Updated answer: grep -oP '(?<=port_num=)[^;]+(?=;)'
This is what I would use:
... | grep -E 'port_num=.+' | sed 's/^.*port_num=\([^;]*\).*$/\1/'
This works with or without the -o on grep, and the availability of -P will depend on the version of grep you have. (e.g., my grep does not have it). I'm not saying the other answers that rely on -P aren't any good -- they look fine to me. But grep -P will be less portable.
IMHO, piping grep with sed allows each utility to do what it specializes in -- grep is for selecting lines, sed is for modifying lines.
This can be done in a simple sed command:
s="76 : client=new; tags=circ, LINK; port_num=switch01; far_port=Gi1/0"
sed 's/.*port_num=\([^;]*\);.*/\1/' <<< "$s"
switch01
... | grep -Po 'port_num.+(?=;)'
This uses grep's Perl Compatible Regular Expression (PCRE) syntax. The (?=;) is a look-ahead assertion which looks for a match with ";" but doesn't include it in the matched output.
This produces:
port_num=switch01
As #Vladimir Kovpak noted, if you want to exclude the "port_num=" string from this output, add a look-behind assertion:
... | grep -Po '(?<=port_num).+(?=;)'
I am tasked with taking a file that has line entries that include string username=xxxx:
$ cat file.txt
Yadayada username=jdoe blablabla
Yadayada username=jdoe blablabla
Yadayada username=jdoe blablabla
Yadayada username=dsmith blablabla
Yadayada username=dsmith blablabla
Yadayada username=sjones blablabla
And finding how many times each user in the file shows up, which I can do manually by feeding username=jdoe for example:
$ grep -r "username=jdoe" file.txt | wc -l | tr -d ' '
3
What's the best way to report each user in the file, and the number of lines for each user, sorted from highest to lowest instances:
3 jdoe
2 dsmith
1 sjones
Been thinking of how to approach this, but drawing blanks, figured I'd check with our gurus on this forum. :)
TIA,
Don
In GNU awk:
$ awk '
BEGIN { RS="[ \n]" }
/=/ {
split($0,a,"=")
u[a[2]]++ }
END {
PROCINFO["sorted_in"]="#val_num_desc"
for(i in u)
print u[i],i
}' file
3 jdoe
2 dsmith
1 sjones
Using grep :
$ grep -o 'username=[^ ]*' file | cut -d "=" -f 2 | sort | uniq -c | sort -nr
Awk alone:
awk '
{sub(/.*username=/,""); sub(/ .*/,"")}
{a[$0]++}
END {for(i in a) printf "%d\t%s\n",a[i],i | "sort -nr"}
' file.txt
This uses awk's sub() function to achieve what grep -o does in other answers. It embeds the call to sort within the awk script. You could of course use that pipe after the awk script rather than within it if you prefer.
Oh, and unlike the other awk solutions presented here, this one (1) is portable to non-GNU-awk environments (like BSD, macOS) and doesn't depend on the username being in a predictable location on each line (i.e. $2).
Why might awk be a better choice than simpler tools like uniq? It probably wouldn't, for a super simple requirement like this. But good to have in your toolbox if you want something with the capability of a little more text processing.
Using sed, uniq, and sort:
sed 's/.*username=\([^ ]*\).*/\1/' file.txt | sort | uniq -c | sort -nr
If there are lines without usernames:
sed -n 's/.*username=\([^ ]*\).*/\1/p' input | sort | uniq -c | sort -nr
$ awk -F'[= ]' '{print $3}' file | sort | uniq -c | sort -nr
3 jdoe
2 dsmith
1 sjones
Following awk may help you on same too.
awk -F"[ =]" '{a[$3]++} END{for(i in a){print a[i],i | "sort -nr"}}' Input_file
I am trying to parse items out of a file I have. I cant figure out how to do this with grep
here is the syntax
<FQDN>Compname.dom.domain.com</FQDN>
<FQDN>Compname1.dom.domain.com</FQDN>
<FQDN>Compname2.dom.domain.com</FQDN>
I want to spit out just the bits between the > and the <
can anyone assist?
Thanks
grep can do some text extraction. however not sure if this is what you want:
grep -Po "(?<=>)[^<]*"
test
kent$ echo "<FQDN>Compname.dom.domain.com</FQDN>
dquote>
dquote> <FQDN>Compname1.dom.domain.com</FQDN>
dquote>
dquote> <FQDN>Compname2.dom.domain.com</FQDN>"|grep -Po "(?<=>)[^<]*"
Compname.dom.domain.com
Compname1.dom.domain.com
Compname2.dom.domain.com
Grep isn't what you are looking for.
Try sed with a regular expression : http://unixhelp.ed.ac.uk/CGI/man-cgi?sed
You can do it like you want with grep :
grep -oP '<FQDN>\K[^<]+' FILE
Output:
Compname.dom.domain.com
Compname1.dom.domain.com
Compname2.dom.domain.com
As others have said, grep is not the ideal tool for this. However:
$ echo '<FQDN>Compname.dom.domain.com</FQDN>' | egrep -io '[a-z]+\.[^<]+'
Compname.dom.domain.com
Remember that grep's purpose is to MATCH things. The -o option shows you what it matched. In order to make regex conditions that are not part of the expression that is returned, you'd need to use lookahead or lookbehind, which most command-line grep does not support because it's part of PCRE rather than ERE.
$ echo '<FQDN>Compname.dom.domain.com</FQDN>' | grep -Po '(?<=>)[^<]+'
Compname.dom.domain.com
The -P option will work in most Linux environments, but not in *BSD or OSX or Solaris, etc.