Awk processing and parsing - parsing

Trying to use STRICTLY awk to solve a little issue and I can't wrap myself around the solution:
# more connections
0x828ac008 127.0.0.1:5152 127.0.0.1:1387 2000
0x82cc28f8 10.4.4.63:1435 10.4.4.72:22 1132
0x82ec1de0 10.4.4.63:1524 50.28.90.36:8080 3248
# awk -F":" '/[0-9]/{print $1,$2}' connections | awk '!/^127/{print "nslookup "$4}'
nslookup 127.0.0.1
nslookup 10.4.4.72
nslookup 50.28.90.36
I am looking for a streamlined method to just parse out anything aside from loopbacks ^127 and the 10. addresses in my netblock. Yes, I know I can use sed, grep, cut, etc, but I'm hoping to see how someone else would do this in awk. Its more of a learning curve/a-ha! thing
Clarifying: Output would omit 10.x.x.x and 127.x.x.x
Output would be
nslookup 50.28.90.36
I tried awk !/^127\.|^10\./ but I couldn't get it to ignore the values.

This awk should work:
awk -F "[: ]+" 'NR>1 && !($4 ~ /^(127|10)\./){print "nslookup", $4}' connections
OUTPUT:
nslookup 50.28.90.36

Anubhava offers a great solution by setting Field Separators. Here is an alternate way of doing it using the default FS:
Using split function:
awk '$3!~/^(127|10)/{split($3,ip,/:/);print "nslookup",ip[1]}' connections
Using sub function:
awk '$3!~/^(127|10)/{sub(/:.*/,"",$3);print "nslookup", $3}' connections
Also a bash way of doing it:
while read -ra lines; do
[[ ! ${lines[2]} =~ "^(127|10).*" ]] && echo "nslookup ${lines[2]%:*}";
done < connections

Related

XOR operator with grep when we grep on 2 patterns

I am using grep to find match of 2 patterns with condition OR like this, classically:
grep -E 'C_matrix|F_matrix' triplot_XC_dev_for_right_order_with_FoM.py | wc -l
I would like now to exclude the cases when both pattern are matched on the same line, i.e I would like to use a XOR operator with grep.
How can I do this operation ? Maybe another trick is possible (I think about grep -v to exclude but this would be nice to do this operation in ony one command line with grep -E).
When you want to make such a special case, it is better to make use of awk:
$ awk '(/C_matrix/ && !/F_matrix/) || (!/C_matrix/ && !/F_matrix/)' file
Using GNU awk, you can use the bit-manipulation function xor:
$ awk 'xor(/C_matrix/,/F_matrix/)' file

awk help > Sending output to a command

I am by no means a programmer. I'll put that right out there. However, I'm trying to write a script that writes fstab entries after grabbing uuid data. This is in a OpenWRT environment on my router. My goal:
Grab uuid info using awk, like:
blkid /dev/sdb2 | awk -F'UUID="|"' '{print $2}'
Send the full uuid output to the end of this command:
uci set fstab.#mount[-1].uuid=
Execute that command with the correct uuid.
This command writes that uuid to the correct place in fstab. How can this be accomplished in a bash-script?
Thanks,
KG
You can either execute the command from within awk (using system(...)), or from within the shell (using uci set fstab.#mount[-1].uuid=$(blkid ...)).
I can't test it with the uci command but I believe this should do what you need.
bblkid /dev/sda1 | uci set fstab.#mount[-1].uuid=$( awk -F'UUID="|"' '{print $2}' )
Edit: this would probably be better.
uci set fstab.#mount[-1].uuid=$( bblkid /dev/sda1 | awk -F'UUID="|"' '{print $2}' )
Edit 2: little bit shorter but should get the same result. Promise this is the last one
uci set fstab.#mount[-1].uuid=$( bblkid /dev/sda1 | tr -d UUID=\" )
The above options didn't work quite right.
What if I have this in fstab:
option uuid 'hotdog'
I want to replace the word "hotdog" with the output of the uuid from this command:
On my router that output looks like:
root#OpenWrt:/etc/config# blkid /dev/sdb2 | awk -F'UUID="|"' '{print $2}'
8618b8fe-93a9-488c-a901-0df30898c82e
or
root#OpenWrt:/etc/config# block info | grep sdb2
/dev/sdb2: UUID="8618b8fe-93a9-488c-a901-0df30898c82e" NAME="EXT_JOURNAL" VERSION="1.0" TYPE="ext4"
I want to then replace the word hotdog with the random uuid, and keep the semi-colons. Keep in mind that the uuid will change each time I format that partition.

Find parameters of a method with grep

I need some help with a grep command (in the Bash).
In my source files, I want to list all unique parameters of a function. Background: I want to search through all files, to see, which permissions ([perm("abc")] are used.
Example.txt:
if (x) perm("this"); else perm("that");
perm("what");
I'd like to have my grep output:
this
that
what
If I do my grep with this search expression
perm\(\"(.*?)\"\)
I'll get perm("this), perm("that"), etc. but I'd like to have just the permissions: this and that and what.
How can I do that?
Use a look-behind:
$ grep -Po '(?<=perm\(")[^"]*' file
this
that
what
This looks for all the text occurring after perm(" and until another " is found.
Note -P is used to allow this behaviour (it is a Perl regex) and -o to just print the matched item, instead of the whole line.
Here is a gnu awk version (due to multiple characters in RS)
awk -v RS='perm\\("' -F\" 'NR>1 {print $1}' file
this
that
what

How to grep in one line starting from particular string to end with particular string

I want to grep "[calleruid]=aab01b055-89e3-49f3-839e-507bb128d07e&smscresponse"
in Below file
2014-10-15 18:38:32,831 plivo-rest[2781]: INFO: Fetching GET http://*******/outbound_callback.aspx with smscresponse[to]=8912722fsf9&smscresponse[ALegUUID]=5bb516fsd64-546c-11e4-879f-551816a551303677&smscresponse[calluid]=aab01b055-89e3-49f3-839e-507bb128d07e&smscresponse[direction]=outbosund&smscresfdsponse[endreason]=UNALLOCATED_NUMBER&smscresponse[from]=83339995896999&smscresponse[starttime]=0&smscresponse[ALegRequestUUID]=5bb4bafc-546c-11e4-891d-000c29ec6e41&smscresponse[RequestUUID]=5bb4bafc-546c-11e4-891d-000c29ec6e41&smscresponse[callstatus]=completed&smscresponse[endtime]=1413378509&smscresponse[ScheduledHangupId]=5bb4c15a-546c-11e4-891d-000c29ec6e41&smscresponse[event]=missed_call_hangup
I used this command
$ grep -oP '(calluid).*$'
this greps upto end of file
I used this command
$ grep -oP '(calluid).{40}'
it fetches 40 characters but i have 1000's of calleruid's so each have different no.s of characters
So please guide me to grep exact callerid data
Use a lookahead to force the regex engine to do the match upto a specific character or a boundary.
$ grep -oP '\[calluid\][^\]\[]*(?=\[|$)' file
[calluid]=aab01b055-89e3-49f3-839e-507bb128d07e&smscresponse
Here is an gnu awk (due to multiple characters in RS) version:
awk -v RS="[[]calluid[]]=" -F[ 'NR==2 {print $1}' file
aab01b055-89e3-49f3-839e-507bb128d07e&smscresponse
You can also set RS like this: RS="\\\[calluid]="

Simple Grep Issue

I am trying to parse items out of a file I have. I cant figure out how to do this with grep
here is the syntax
<FQDN>Compname.dom.domain.com</FQDN>
<FQDN>Compname1.dom.domain.com</FQDN>
<FQDN>Compname2.dom.domain.com</FQDN>
I want to spit out just the bits between the > and the <
can anyone assist?
Thanks
grep can do some text extraction. however not sure if this is what you want:
grep -Po "(?<=>)[^<]*"
test
kent$ echo "<FQDN>Compname.dom.domain.com</FQDN>
dquote>
dquote> <FQDN>Compname1.dom.domain.com</FQDN>
dquote>
dquote> <FQDN>Compname2.dom.domain.com</FQDN>"|grep -Po "(?<=>)[^<]*"
Compname.dom.domain.com
Compname1.dom.domain.com
Compname2.dom.domain.com
Grep isn't what you are looking for.
Try sed with a regular expression : http://unixhelp.ed.ac.uk/CGI/man-cgi?sed
You can do it like you want with grep :
grep -oP '<FQDN>\K[^<]+' FILE
Output:
Compname.dom.domain.com
Compname1.dom.domain.com
Compname2.dom.domain.com
As others have said, grep is not the ideal tool for this. However:
$ echo '<FQDN>Compname.dom.domain.com</FQDN>' | egrep -io '[a-z]+\.[^<]+'
Compname.dom.domain.com
Remember that grep's purpose is to MATCH things. The -o option shows you what it matched. In order to make regex conditions that are not part of the expression that is returned, you'd need to use lookahead or lookbehind, which most command-line grep does not support because it's part of PCRE rather than ERE.
$ echo '<FQDN>Compname.dom.domain.com</FQDN>' | grep -Po '(?<=>)[^<]+'
Compname.dom.domain.com
The -P option will work in most Linux environments, but not in *BSD or OSX or Solaris, etc.

Resources