How to do operations with form values? - ruby-on-rails

so i have a form with 3 text fields, in two of them, users enters a code (and OID), which i use to run a command (snmpbulkwalk). So i want to store the outputs of those commands in a variable, lets say var1 and var2.
[]
But then, in the last text field, i want to do and operation with these variables, like: var1 - var2.
So far i tried this:
#var1 = "nice -n 19 snmpbulkwalk -v 2c -c %snmp_community
%sensor_ip "+ "#{monitor_category_params[:oid].strip}"+" | awk
'/kB$/{ $(NF-1)= $(NF-1)*1024;} {print $0}' | sed 's/ [A-Za-
z]*$//' | awk '{print $NF}' | grep -o '[[:digit:]]*' | paste -s
-d';' -"
puts("var1 = " + "#{#var1}");
#var2 = "nice -n 19 snmpbulkwalk -v 2c -c %snmp_community
%sensor_ip "+ "#{monitor_category_params[:oid2]}"+" | awk
'/kB$/{ $(NF-1)= $(NF-1)*1024;} {print $0}' | sed 's/ [A-Za-
z]*$//' | awk '{print $NF}' | grep -o '[[:digit:]]*' | paste -s
-d';' -"
puts("var2 = " + "#{#var2}");
#var3 = "#{monitor_category_params[:snmp_oper].strip}"
puts("var3 with {} = " + "#{#var3}");
system = '$(#{#var3})'
puts(system(system));
The thing is i dont know how to store the output of the shell command "nice -n 19 ...blabla" in the variable. I used exec and backticks like this:
exec "nice -n 19 snmpbulkwalk -v 2c -c %snmp_community %sensor_ip "+ "#{monitor_category_params[:oid].strip}"+" | awk '/kB$/{ $(NF-1)= $(NF-1)*1024;} {print $0}' | sed 's/ [A-Za-z]*$//' | awk '{print $NF}' | grep -o '[[:digit:]]*' | paste -s -d';' -"
but it gives me this error, which i dont get it since the command is well formatted.
sh: -c: línea 0: EOF inesperado mientras se buscaba un `)' coincidente
sh: -c: línea 1: error sintáctico: no se esperaba el final del fichero
I dont know if im losing arguments (%snmp_community or %sensor_ip), but the final result should be something like system = number - number. but i only get system = var1 - var2, which is not doing nothing since i want the outputs of the commands, not the name of the variables.
sorry if didnt explain myself good, thank you in advance.

So assuming this is related with ruby or ruby on rails
#var1 and #var2 are instance variable of a method in some class.
You can directly use the backticks directly to save to output from the shell command
#var1 = `ls`
p #var1
However, some bash commands do not return a string value directly.
I have not used snmpbulkwalk till now but $PATH might be similar in a way
#var1 = `echo $PATH` #add you own shell code here to check
p #var1

Related

How to grep lines non-repeatedly for same command?

I have a space-separated file that looks like this:
$ cat in_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004927566.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004919950.1 FAD_binding_3
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
I am using the following shell script utilizing grep to search for strings:
$ cat search_script.sh
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
The problem is that I want each grep command to return only the first instance of the string it finds exclusive of the previous identical grep command's output.
I need an output which would look like this:
$ cat out_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
in which line 1 is exclusively the output of the first grep command and line 2 is exclusively the output of the second grep command. How do I do it?
P.S. I am running this on a big file (>125,000 lines). So, search_script.sh is mostly composed of unique grep commands. It is the identical commands' execution that is messing up my downstream analysis.
I'm assuming you are generating search_script.sh automatically from the contents of in_file. If you can count how many times you'll repeat the same grep command you can just use grep once and use head, for example if you know you'll be using it 2 times:
grep "foo" bar.txt | head -2
Will output the first 2 occurrences of "foo" in bar.txt.
If you have to do the grep commands separately, for example if you have other code in between the grep commands, you can mix head and tail:
grep "foo" bar.txt | head -1 | tail -1
Some other commands...
grep "foo" bar.txt | head -2 | tail -1
head -n displays the first n lines of the input
tail -n displays the last n lines of the input
If you really MUST always use the same command, but ensure that the outputs always differ, the only way I can think of to achieve this is using temporary files and a complex sequence of commands:
cat foo.bar.txt.tmp 2>&1 | xargs -I xx echo "| grep -v \\'xx\\' " | tr '\n' ' ' | xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp"
So to explain this command, given foo as a search string and bar.txt as the filename, then foo.bar.txt.tmp is a unique name for a temporary file. The temporary file will hold the strings that have already been output:
cat foo.bar.txt.tmp 2>&1 : outputs the contents of the temporary file. If none is present, will output an error message to stdout, (important because if the output was empty the rest of the command wouldn't work.)
xargs -I xx echo "| grep -v \\'xx\\' " adds | grep -v to the start of each line in the temporary file, grep -v something excludes lines that include something.
tr '\n' ' ' replaces newlines with spaces, to have on a single string a sequence of grep -vs.
xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp" runs a new command, grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp, replacing xx with the previous output. xx should be the sequence of grep -vs that exclude previous outputs.
head -1 makes sure only one line is output at a time
tee -a foo.bar.txt.tmp appends the new output to the temporary file.
Just be sure to clear the temporary files, rm *.tmp, at the end of your script.
If I am getting question right and you want to remove duplicates based on last field of each line then try following(this should be easy task for awk).
awk '!a[$NF]++' Input_file

How to grep repeated strings on a single line?

I have this a file.txt with one line, whose content is
/app/jdk/java/bin/java -server -Xms3g -Xmx3g -XX:MaxPermSize=256m -Dweblogic.Name=O2pPod8_mapp_msrv1_1 -Djava.security.policy=/app/Oracle/Middleware/Oracle_Home/wlserver/server/lib/weblogic.policy -Djava.security.egd=file:/dev/./urandom -Dweblogic.ProductionModeEnabled=true -Dweblogic.system.BootIdentityFile=/app/Oracle/Middleware/Oracle_Home/user_projects/domains/O2pPod8_domain/servers/O2pPod8_mapp_msrv1_1/data/nodemanager/boot.properties -Dweblogic.nodemanager.ServiceEnabled=true -Dweblogic.nmservice.RotationEnabled=true -Dweblogic.security.SSL.ignoreHostnameVerification=false -Dweblogic.ReverseDNSAllowed=false -Xms8192m -Xmx8192m -XX:MaxPermSize=2048m -XX:NewSize=1300m -XX:MaxNewSize=1300m -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled
and when I do
cat file.txt | grep -io "Xms.*" | awk '{FS" "; print $1} ' | cut -d "s" -f2
output:
3g
why is grep not reading the second occurrence, i.e. I expect 3g and 8192m.
Infact, how do I print only 8192m in this case?
Your regex just says "find Xms followed by anything repeated 0 to n times". That returns the rest of the row from Xms onward.
What you actually want is something like "find Xms followed by anything until there's a whitespace repeated 0 to n times".
grep -io "Xms[^ ]*" file.txt | awk '{FS" "; print $1} ' | cut -d "s" -f2
In [^ ] the ^ means "not"
I'm not really sure what you are trying to achieve here but if you want the endings of all space-separated strings starting with -Xms, using bare awk is:
$ awk -v RS=" " '/^-Xms/{print substr($0,5)}' file
3g
8192m
Explained:
$ awk -v RS=" " ' # space separated records
/^-Xms/ { # strings starting with -Xms
print substr($0,5) # print starting from 5th position
}' file
If you wanted something else (word repeated in the title puzzles me a bit), please update the question with more detailed requirements.
Edit: I just noticed how do I print only 8192m in this case (that's the repeated maybe). Let's add a counter c and not print the first instance:
$ awk -v RS=" " '/^-Xms/&&++c>1{print substr($0,5)}' file
8192m
You could use grep -io "Xms[0-9]*[a-zA-Z]" instead of grep -io "Xms.*" to match a sequence of digits followed by a single character instead the entire line within a single group:
cat file.txt | grep -io "Xms[0-9]*[a-zA-Z]" | awk '{FS" "; print $1} ' | cut -d "s" -f2
Hope this helps!
The .* in your regexp is matching the rest of the line, you need [^ ]* instead. Look:
$ grep -o 'Xms.*' file
Xms3g -Xmx3g -XX:MaxPermSize=256m -Dweblogic.Name=O2pPod8_mapp_msrv1_1 -Djava.security.policy=/app/Oracle/Middleware/Oracle_Home/wlserver/server/lib/weblogic.policy -Djava.security.egd=file:/dev/./urandom -Dweblogic.ProductionModeEnabled=true -Dweblogic.system.BootIdentityFile=/app/Oracle/Middleware/Oracle_Home/user_projects/domains/O2pPod8_domain/servers/O2pPod8_mapp_msrv1_1/data/nodemanager/boot.properties -Dweblogic.nodemanager.ServiceEnabled=true -Dweblogic.nmservice.RotationEnabled=true -Dweblogic.security.SSL.ignoreHostnameVerification=false -Dweblogic.ReverseDNSAllowed=false -Xms8192m -Xmx8192m -XX:MaxPermSize=2048m -XX:NewSize=1300m -XX:MaxNewSize=1300m -XX:SurvivorRatio=4 -XX:TargetSurvivorRatio=90 -XX:+UseParNewGC -XX:+UseConcMarkSweepGC -XX:+CMSParallelRemarkEnabled
$ grep -o 'Xms[^ ]*' file
Xms3g
Xms8192m
$ grep -o 'Xms[^ ]*' file | cut -d's' -f2
3g
8192m
$ grep -o 'Xms[^ ]*' file | cut -d's' -f2 | tail -1
8192m
or more concisely:
$ sed 's/.*Xms\([^ ]*\).*/\1/' file
8192m
The positive lookbehind of PCRE (the form: (?<=RE1)RE2) can resolve the problem easily:
$ grep -oP '(?<=Xms)\S+' file.txt
3g
8192m
Explains:
-o: show only the part of a line matching PATTERN.
-P: PATTERN is a Perl regular expression.
(?<=Xms)\S+: matches all continuous non-whitespace strings which are just following the string Xms.

grep -v under double quotes query

We have a portion of code which states,
"diff file1 file2 | /usr/bin/grep -v "#" | /usr/bin/grep ^\> | /usr/bin/awk '{print $3}' | /usr/bin/xargs mkdir"
The whole statement is enclosed in double quotes(is a requirement of the application syntax). When the application reaches this stage , it gives the grep error.
This statement works well on the command line. But through application, gives error for grep.
Usage: grep [OPTION]... PATTERN [FILE]...
Try `grep --help' for more information.
So not sure if it is first grep or second grep which is a problem.
Seems like a problem with double quotes. Try changing your first grep to /usr/bin/grep -v '#' and the second grep to /usr/bin/grep '^>'
You are using grep -v ^> and > means "redirect".
If you for example do:
grep ^>output
all the output will be stored in the file output.
So what you need to do is to quote ^> so that it is interpreted as the pattern you are looking for:
"diff file1 file2 | /usr/bin/grep -v "#" | /usr/bin/grep "^>" | /usr/bin/awk '{print $3}' | /usr/bin/xargs mkdir"
^ ^
By the way, note all your greps can be reduced like this:
diff file1 file2 | awk '/#/ || /^>/ {print $3}' | /usr/bin/xargs mkdir
^^^ ^^ ^^^^
either contains # | |
or starts with >

How to grep and execute a command (for every match)

How to grep in one file and execute for every match a command?
File:
foo
bar
42
foo
bar
I want to execute to execute for example date for every match on foo.
Following try doesn't work:
grep file foo | date %s.%N
How to do that?
grep file foo | while read line ; do echo "$line" | date %s.%N ; done
More readably in a script:
grep file foo | while read line
do
echo "$line" | date %s.%N
done
For each line of input, read will put the value into the variable $line, and the while statement will execute the loop body between do and done. Since the value is now in a variable and not stdin, I've used echo to push it back into stdin, but you could just do date %s.%N "$line", assuming date works that way.
Avoid using for line in `grep file foo` which is similar, because for always breaks on spaces and this becomes a nightmare for reading lists of files:
find . -iname "*blah*.dat" | while read filename; do ....
would fail with for.
What you really need is a xargs command. http://en.wikipedia.org/wiki/Xargs
grep file foo | xargs date %s.%N
example of matching some files and converting matches to the full windows path in Cygwin environment
$ find $(pwd) -type f -exec ls -1 {} \; | grep '\(_en\|_es\|_zh\)\.\(path\)$' | xargs cygpath -w
grep command_string file | sh -
There is an interesting command in linux for that: xargs, It allows You to use the output from previous command(grep, ls, find, etc.) as the input for a custom execution but with several options that allows You to even execute the custom command in parallel. Below some examples:
Based in your question, here is how to print the date with format "%s.%N" for each "foo" match in file.txt:
grep "foo" file.txt | xargs -I {} date +%s.%N
A more interesting use is creating a file for each match, but in this case if matches are identical the file will be override:
grep "foo" file.txt | xargs -I {} touch {}
If You want to concatenate a custom date to the file created
grep "foo" file.txt | xargs -I {} touch "{}`date +%s.%N`"
Imagine the matches are file names and You want to make a backup of them:
grep "foo" file.txt | xargs -I {} cp {} "{}.backup"
And finally for xargs using the custom date in the backupName
grep "foo" file.txt | xargs -I {} cp {} "{}`date +%s.%N`"
For more info about options like parallel execution of xargs visit: https://en.wikipedia.org/wiki/Xargs and for date formats: https://www.thegeekstuff.com/2013/05/date-command-examples/
Extra I have found also a normal for command useful in this scenarios It is simpler but less versatile below are the equivalent for above examples:
for i in `grep "foo" test.txt`; do date +%s.%N; done
for i in `grep "foo" test.txt`; do touch ${i}; done
for i in `grep "foo" test.txt`; do touch "${i}`date +%s.%N`"; done
for i in `grep "foo" test.txt`; do cp ${i} "${i}.backup2"; done
for i in `grep "foo" test.txt`; do cp ${i} "${i}.backup2`date +%s.%N`"; done
Have Fun!!!
grep may need --line-buffered option to emit each matching line when it matches it, otherwise it buffers up to 4K byte before printing match lines, which defeats the goal here, e.g.
tail -f source | grep --line-buffered "expression | xargs ...
grep search_string files_to_search | sh

xargs: String concatenation

zgrep -i XXX XXX | grep -o "RID=[0-9|A-Z]*" |
uniq | cut -d "=" -f2 |
xargs -0 -I string echo "RequestID="string
My output is
RequestID=121212112
8127127128
8129129812
But my requirement is to have the request ID prefixed before all the output.
Any help is appreciated
I had a similar task and this worked for me. It might be what you are looking for:
zgrep -i XXX XXX | grep -o "RID=[0-9|A-Z]*" |
uniq | cut -d "=" -f2 |
xargs -I {} echo "RequestID="{}
Try -n option of xargs.
-n max-args
Use at most max-args arguments per command line. Fewer than max-args arguments will be used if the size (see the -s option)
is exceeded,
unless the -x option is given, in which case xargs will exit.
Example:
$ echo -e '1\n2' | xargs echo 'str ='
str = 1 2
$ echo -e '1\n2' | xargs -n 1 echo 'str ='
str = 1
str = 2

Resources