I would like to compare 2 file and show the difference between files
when I use the following :
exec sh -c "diff -w file1 file2 \|grep \<"
but no result is shown and the script is brokeen
what's wrong with the tcl command ?
The following will work:
puts [exec sh -c "diff -w file1 file2 |grep \\<"]
The < needs to be double-backslashed, and the output should be printed.
Related
I am running this command "ls -lah -lF | grep /$ | awk '{print $2,$5,$9}' >> test.txt"
when it runs and sends the results to the text file the output is all in one row. How do I get it to formate each directory in a row?
It isn't in one line. Just checked. Check, how your editor handles Unix line endings.
File1 contains
hello
hello
I need to write a grep command to print the filename if this file contains more than one "hello". Otherwise, I need grep to exit on failure.
So far I have
grep -c "hello" File1 | grep -v :0
but it outputs
2. How do I get the desired output, which should either be filename File1 or no output at all (from what I understand, no match is a non zero exit code for grep)
with GNU grep for -z:
grep -lz 'hello.*hello' file
e.g.:
$ seq 15 | grep -lz '3.*3'
(standard input)
$ echo $?
0
$ seq 5 | grep -lz '3.*3'
$ echo $?
1
Like this:
#!/bin/bash
count=$(grep -c "hello" "$1")
if ((count > 1)); then
echo "$1"
else
exit 1
fi
Usage:
chmod +x script.sh
./script.sh File1
Explanations:
((...)) is an arithmetic command, which returns an exit status of 0 if the expression is nonzero, or 1 if the expression is zero. Also used as a synonym for "let", if side effects (assignments) are needed. See http://mywiki.wooledge.org/ArithmeticExpression
Using perl in a shell:
perl -0 -le '
my $filename = $ARGV[0];
print $filename if grep { /hello\nhello/ } <>
' file
I am trying to execute a program (say, biotool) using GNU-parallel which takes 3 arguments, i, o and a :
the input files (i)
output file name to be written in (o)
an argument which takes a sub string from the input file name (a)
for example, say i have 10 text files like this
1_a_test.txt
2_b_test.txt
3_c_test.txt
...
10_j_test.txt
I want to run my tool (say biotool) on all the 10 text files. I tried this
parallel biotool -i {} -o {.}.out -a {} ::: *.txt
I want to pass the charachter/letter/whatever before the first underscore from the input text file name as an argument to -a option like this (dry run):
parallel biotool -i 1_a_test.txt -o 1_a_test.out -a 1 ::: *.txt`
parallel biotool -i 2_b_test.txt -o 2_b_test.out -a 2 ::: *.txt`
parallel biotool -i 3_c_test.txt -o 3_c_test.out -a 3 ::: *.txt`
...
{} supplies the complete file name to -a but I only want the sub string before the first underscore to be supplied to -a
The easiest, but harder to read is this:
parallel --dry-run biotool -i {} -o {.}.out -a '{= s/_.*// =}' ::: *test.txt
Alternatively, you can make a bash function that uses bash Parameter Substitution to extract the part before the underscore. Then export that to make it known to GNU Parallel
#!/bin/bash
doit(){
i=$1
o=$2
# Use internal bash parameter substitution to extract whatever precedes "_"
# See https://www.tldp.org/LDP/abs/html/parameter-substitution.html
a=${i/_*/}
echo biotool -i "$i" -o "$o" -a "$a"
}
export -f doit
parallel doit {} {.}.out ::: *test.txt
Sample Output
biotool -i 10_j_test.txt -o 10_j_test.out -a 10
biotool -i 1_a_test.txt -o 1_a_test.out -a 1
biotool -i 2_b_test.txt -o 2_b_test.out -a 2
I have a space-separated file that looks like this:
$ cat in_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004927566.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004919950.1 FAD_binding_3
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
I am using the following shell script utilizing grep to search for strings:
$ cat search_script.sh
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
grep "GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1" Pfam_anntn_temp.txt
The problem is that I want each grep command to return only the first instance of the string it finds exclusive of the previous identical grep command's output.
I need an output which would look like this:
$ cat out_file
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 Chal_sti_synt_C
GCF_000046845.1_ASM4684v1_protein.faa WP_004920342.1 FAD_binding_3
in which line 1 is exclusively the output of the first grep command and line 2 is exclusively the output of the second grep command. How do I do it?
P.S. I am running this on a big file (>125,000 lines). So, search_script.sh is mostly composed of unique grep commands. It is the identical commands' execution that is messing up my downstream analysis.
I'm assuming you are generating search_script.sh automatically from the contents of in_file. If you can count how many times you'll repeat the same grep command you can just use grep once and use head, for example if you know you'll be using it 2 times:
grep "foo" bar.txt | head -2
Will output the first 2 occurrences of "foo" in bar.txt.
If you have to do the grep commands separately, for example if you have other code in between the grep commands, you can mix head and tail:
grep "foo" bar.txt | head -1 | tail -1
Some other commands...
grep "foo" bar.txt | head -2 | tail -1
head -n displays the first n lines of the input
tail -n displays the last n lines of the input
If you really MUST always use the same command, but ensure that the outputs always differ, the only way I can think of to achieve this is using temporary files and a complex sequence of commands:
cat foo.bar.txt.tmp 2>&1 | xargs -I xx echo "| grep -v \\'xx\\' " | tr '\n' ' ' | xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp"
So to explain this command, given foo as a search string and bar.txt as the filename, then foo.bar.txt.tmp is a unique name for a temporary file. The temporary file will hold the strings that have already been output:
cat foo.bar.txt.tmp 2>&1 : outputs the contents of the temporary file. If none is present, will output an error message to stdout, (important because if the output was empty the rest of the command wouldn't work.)
xargs -I xx echo "| grep -v \\'xx\\' " adds | grep -v to the start of each line in the temporary file, grep -v something excludes lines that include something.
tr '\n' ' ' replaces newlines with spaces, to have on a single string a sequence of grep -vs.
xargs -I xx sh -c "grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp" runs a new command, grep 'foo' bar.txt xx | head -1 | tee -a foo.bar.txt.tmp, replacing xx with the previous output. xx should be the sequence of grep -vs that exclude previous outputs.
head -1 makes sure only one line is output at a time
tee -a foo.bar.txt.tmp appends the new output to the temporary file.
Just be sure to clear the temporary files, rm *.tmp, at the end of your script.
If I am getting question right and you want to remove duplicates based on last field of each line then try following(this should be easy task for awk).
awk '!a[$NF]++' Input_file
What i wanted was to make a little Script that makes a grep over files that successfully passed another grep.
Here is that i wrote:
#!/bin/bash -f
if (( $# != 3 ))
then
echo "Usage: $0 <directory> <grep1> <grep2>"
exit 1
fi
dir=$1
grep1=$2
grep2=$3
files=`grep -Ril "$grep1" $dir`
for file in $files; do
grep -iH "$grep2" $file
done
so this works, in case the directory is a specific one.
But It doesnt work in the way i wanted to use it:
./grepIfSucceededGrep.sh /mnt/logs/\*/20130929/\* test1 test2
grep: /mnt/logs/*/20130929/*: No such file or directory
But when i enter the command manually it works perfectly. Must have to do something with the asterisk.
grep -Ril "segment1" /mnt/logs/*/20130929/*
/mnt/logs/1111/20130929/000033.00.log
/mnt/logs/1112/20130929/000033.00.log
/mnt/logs/1113/20130929/154852.00.log
/mnt/logs/1114/20130929/171227.00.log
Why?! :( just searched for 2 hours to get a workarround but had no fix for that issue.
Do you really need a script to achieve what you need? You can say:
find /path -type f -exec sh -c "grep -iq string1 {} && grep -iq string2 {} && echo {}" \;
This would list the files in /path and subdirectories that contain both string1 and string2.
If you need to put it into a script so that you could pass parameters, say:
find $1 -type f -exec sh -c "grep -iq $2 {} && grep -iq $3 {} && echo {}" \;
within a file, say myfind.sh that could be invoked by saying sh myfind.sh /my/path/to/search firststring secondstring.