I am a beginner of bash script. I just started to write a script where it checks the contents of b.txt can all be found in a.txt. (line by line preferably). My code is as following:
grep -Ffw b.txt a.txt
As you can see, I want to do fixed string instead of REGEX, I want to check everything from the b.txt file, because there are some strings inside the b.txt and I want to check if all of them exist in a.txt. And I also want to match the whole word only of course. So these are the requirements, however when I run this command it returns me an error says: grep: w: No such file or directory
I am thinking that maybe there are some limitations of the flags in bash? Sorry I am not really familiar with the language, didn't read much about the MAN page etc. If anyone could help me to solve the puzzle it would be appreciated :) In addition, i think if possible I would like to add a -q to surpress the output when there is a match also, right now I didn't add it in the example since it couldn't make it through with 3 flags even. So can anyone give me some hints here? Thanks in advance!
Hereby some explanation from the manpage:
OPTIONS
Generic Program Information
...
-F, --fixed-strings
Interpret PATTERNS as fixed strings, ...
-f FILE, --file=FILE
Obtain patterns from FILE, ...
-w, --word-regexp
Select only those lines ...
As you can see, the options -F and -w are indicated ending immediately (hence the comma in -F, and -w,), but the -f switch is followed by FILE, with means they belong together.
I you want to preserve the order Ffw, that's possible, but then you need to do something like:
grep -Ff b.txt -w a.txt
As mentioned by #kvantour, the solution is simply placing the -f before the b.txt file. grep -Fwf b.txt a.txt
Should have thought it when it says 'no such file or directory' as it was a clear indication that flags after the -f were treated as the path already.
Related
I am using grep (BSD grep) 2.5.1-FreeBSD in MacOS and I have found the following behavior.
I have two *.tex files. Each one of these contains the following lines
$k$-th bit of
$(i-m)$-th bit of
respectively. When I ran
grep --color -rnw . -e '\$-th bit of' --include="*.tex"
I got only the second file, i.e., $(i-m)$-th bit of, while I expect the two lines. Could you help me please to understand this behavior?
Never use -r or --include or any other grep option to find files. The GNU guys really screwed up by adding those options to grep when there's a perfectly good tool named find for finding files and now they've turned grep into a convoluted mush of finding files and Globally matching a Regular Expression within a file and Printing the result (G/RE/P).
Keep it simple - find the files with find then g/re/p within then using grep:
find . -name '*.tex' -exec grep --color -n '\$-th bit of' {} +
As others pointed out your g/re/p problem was the -w arg so I've removed that above.
I have the same version of grep.
It is caused by your use of the -w option:
-w, --word-regexp
The expression is searched for as a word (as if surrounded by `[[:<:]]' and `[[:>:]]'; see re_format(7)).
The matched part of the string $k$-th bit of is bounded on the left-hand side by a word character (i.e. k) so the match is treated as being inside a "word" and it can't therefore satisfy the "searched for as a whole word" requirement.
Try without -w and it will work fine.
I stopped a process to troubleshoot something. Now, I would like to start the process where it left off in CentOS 6.4.
This script I stopped runs a perl script in a loop to process all of the files in /dev/shm/split/. These files were split into many parts from a larger file. An example of how they are named are as follows:
file.txt.aa
file.txt.ab
file.txt.ac
...and so on.
I have identified that the script left off at file.txt.fy. So, I would like to remove all of the files in /dev/shm/split/ that are from file.txt.aa through file.txt.fy.
I tried to create a whitelist for the rm command by doing:
ls /dev/shm/split/ > whitelist
cat whitelist | egrep -v 'file.txt.[aa-fz]' | tee whitelist.tmp
This did not do what I had intended.
Please help me! Thank you!
The problem with your command is that you cannot match two characters with the square bracket pattern in bash. You should use something like that instead:
ls file.txt.[a-e]? file.txt.f[a-y]
Basically decompose your range into two ranges, the first will match .aa to .ez, and the second .fa to .fy (included).
Note that I have used the ls command here. I always find it a good idea to first echo or ls the commands/files you're going to run when the operations you do are potentially destructive. When you're sure it produces the right output, go on and use rm instead of ls.
I used the following syntax in order to find IP address under /etc
(answered by Dennis Williamson in superuser site)
but I get the message "grep: line too long".
Someone have idea how to ignore this message and why I get this?
grep -Er '\<([0-9]{1,3}\.){3}[0-9]{1,3}\>' /etc/
grep: line too long
The find/xargs solution didn't work for me, but resulted in the same error.
I solved this problem by using the -I grep option (ignore binary files). In my case there must have been a binary file in the list of files to search that had no linebreaks, so grep tries to read in a gigantic line that is too big. That's my guess at what this error means.
I got the idea from: http://web.archiveorange.com/archive/v/am8x7wI0r0243prrmYd4
This might not work for you of course if there's a text file with a line that is too long.
Use find to build a list of files to grep,
find /etc -type f -print0 | xargs -r0 grep -E '\<([0-9]{1,3}\.){3}[0-9]{1,3}\>'
In general find is a more flexible way of traversing the filesystem and building lists of files for other programs.
Perhaps your grep has a bug and scans by accident a binary file with too long lines (i.e. too much characters for grep to handle between two newlines). See this red hat page for more details (bug page).
Well, I have a file test.txt
#test.txt
odsdsdoddf112 test1_for_grep
dad23392eeedJ test2 for grep
Hello World test
garbage
I want to extract strings which have got a space after them. I used following expression and it worked
grep -o [[:alnum:]]*.[[:blank:]] test.txt
Its output is
odsdsdoddf112
dad23392eeedJ
test2
for
Hello
World
But problem is grep prints all the strings that have got space after them, where as I want it to stop after first match on a line and then proceed to second line.
Which expression should I use here, in order to make it stop after first match and move to next line?
This problem may be solved with gawk or some other tool, but I will appreciate a solution which uses grep only.
Edit
I using GNU grep 2.5.1 on a Linux system, if that is relevant.
Edit
With the help of the answers given below, I tried my luck with
grep -o ^[[:alnum:]]* test.txt
grep -Eo ^[[:alnum:]]+ test.txt
and both gave me correct answers.
Now what surprises me is that I tried using
grep -Eo "^[[:alnum:]]+[[:blank:]]" test.txt
as suggested here but didn't get the correct answer.
Here is the output on my terminal
odsdsdoddf112
dad23392eeedJ
test2
for
Hello
World
But comments from RichieHindle and Adrian Pronk, shows that they got correct output on their systems. Anyone with some idea that why I too am not getting the same result on my system. Any idea? Any help will be appreciated.
Edit
Well, it seems that grep 2.5.1 has some bug because of which my output wasn't correct. I installed grep 2.5.4, now it is working correctly. Please see this link for details.
If you're sure you have no leading whitespace, add a ^ to match only at the start of a line, and change the * to a + to match only when you have one or more alphanumeric characters. (That means adding -E to use extended regular expressions).
grep -Eo "^[[:alnum:]]+[[:blank:]]" test.txt
(I also removed the . from the middle; I'm not sure what that was doing there?)
As the questioner discovered, this is a bug in versions of GNU grep prior to 2.5.3. The bug allows a caret to match after the end of a previous match, not just at beginning of line.
This bug is still present in other versions of grep, for instance in Mac OS X 10.9.4.
There isn't a universal workaround, but in the some examples, like non-spaces followed by a space, you can often get the desired behavior by leaving off the delimiter. That is, search for '[^ ]*' rather than '[^ ]* '.
grep -oe "^[^ ]* " test.txt
If we want to extract all meaningful input before garbage and actually stop on first match then -B NUM, --before-context=NUM option may be useful to "print NUM lines of leading context before matching lines".
Example:
grep --before-context=999999 "Hello World test"
I am a newbie to shell scripting and to Linux environment as well.
In my project I am trying to search for following text from the httpd.conf file
<Directory '/somedir/someinnerdir'>
AllowOverride All
</Directory>
and then remove this text and again rewrite the same text.
The reason to do this rewriting is that the script will be run on first installation of the web app, but it may again be run some time later as other part of this shell script is performing other tasks as well. So for first time this text wont be found and will simply be written but later again when script is run this text will be found and will need to be removed and the written again.
So the part of my script with which I am trying to achieve this is something like :
grep -ve "<Directory '/somedir/someinnerdir'>\\nAllowOverride All\\n</Directory>" /etc/httpd/conf/httpd.conf > tmp_direct
echo -e "<Directory '/somedir/someinnerdir'>\\nAllowOverride All\\n</Directory>" >> tmp_direct
mv tmp_direct /etc/httpd/conf/httpd.conf
I dont have the code in front of me currently so there may be some syntactical errors above but the logic/coding is same.
Above code fragment is not able to do what I want to achieve as the grep command doesnt support multiline searching.
My OS is Fedora 8.
Can you please suggest something in this code to achieve what is needed or may be some other alternative.
Any help in this regard will be highly appreciated.
Thanks in advance.
Thanks for your replies.
Sorry for the previous bad code. Its corrected now.
Charlie and i-moan, due to workability constraints I wont be able to implement sed or perl as it will need to be added to the environment we will distribute this project in.
Steve, I want to do the check for multiple lines. I didnt put it in code blocks so it removed the directory tags. :(
So I will need to find some other way out.
Thanks again.
Best regards.
This sounds like a job for sed(1). What you want will be very close to
sed -i.bak -e '/AllowOverride All/i# ' file
to comment out the line (remember httpd.conf has comments) followed by
sed -i.bak -e 's/# AllowOverride All/AllowOverride All ' file
to pujt it back.
WARNING I haven't tried it, you want to read the man page and test it youself.
If you want to make sure you're matching an entire line, you could do:
grep -v "^AllowOverride All$"
The ^ matches the start of line and $ matches the end of line.
Does that help? I'm not exactly clear on what you're trying to do with the grep.
I'm unsure I understand the question.
Correct me if I'm wrong but your trying to add 'AllowOverride All' to the httpd.conf on the 1st running of your script. If so do you want it enabled?
Then on a 2nd pass you may want to enable / disable the command?
If I remember correctly 'AllowOverride' is only allowed in a < Directory > section So it only makes sense to add / enable in < Directory >< /Directory > definition:
If above is true you could do this in perl..
1st run: to add
perl -0777 -pi
-e 's{(.?(?!AllowOverride\s+All).?)}[\nAllowOverride
All\n$1]ixgs' httpd.conf
This will add 'AllowOverride All' to a Directory section only if it does not already contain one.
2nd run: to alter
On the 2nd run you could comment out as Charlie's suggested.. or if you like perl
perl -0777 -pi
-e's{(.?)(AllowOverride\s+All)(.?)}[$1#$2$3]ixgs' httpd.conf
and to comment in
perl -0777 -pi
-e's{(.?)(#AllowOverride\s+All)(.?)}[$1AllowOverride\s+All$3]ixgs'
httpd.conf
The regex s{}[] is quite verbose and there are better regex's to do the same thing but this is easier ( hopefully ) for a beginner to understand :).
Explanation of the perl command options
-0777 force Perl to read the whole file in one shot because 0777 is not a legal character value
-p places a loop around your script
-i lets you edit files in-place ( so your command in '-e' change the file )
-e lets you specify a single line of code on the command line
In that single line we have a regex. see perldoc perlre for info on that.
Hope that helps