a log generates errors. it generates two very similar lines for 1 error
I want to grep these errors but only one of the lines
for example:
Line is : Mar 21 15:33:04 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 21/03/17 15:33:04 VMD25 DIR_CHECK 0 FATAL File /data1/gmq6/in/29920991077061 is more than 10 minutes old
Line is : Mar 21 15:33:04 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 21/03/17 15:33:04 VMD18 DIR_CHECK 0 FATAL File /data1/sftp/out/26515991064454 is more than 10 minutes old
Mar 21 15:33:04 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 21/03/17 15:33:04 VMD25 DIR_CHECK 0 FATAL File /data1/gmq6/in/29920991077061 is more than 10 minutes old
Mar 21 15:33:04 VMP05 SMC_User: FATAL ECSDPROD 5210/SUPPORT/ECSD 21/03/17 15:33:04 VMD18 DIR_CHECK 0 FATAL File /data1/sftp/out/26515991064454 is more than 10 minutes old
but I only want to grep the lines without 'Line is'. I am using Hewlett Packard Linux
EDIT:
this grep is needed within a tail -f :
#!/usr/bin/ksh
echo "checking for last 10 fatals"
grep "FATAL ECSDPROD" /data1/log/startstop/MonitorDaemon.log|tail > /tmp/AH/linesDP.txt
grep "FATAL ECSD" /tmp/AH/linesDP.txt | grep -v "Line is"
echo "\n\n----------\n"
echo "checking for new fatals"
tail -f /data1/log/startstop/MonitorDaemon.log | grep "FATAL ECSD" | grep -v "Line is"
echo "about to exit"
exit 0
with the above the tail isn't updating, the script gets all the way down to the echo "checking for new fatals" then it won't tail the log
Maybe this helps.
tail -f /data1/log/startstop/MonitorDaemon.log | grep --line-buffered "FATAL ECSD" | grep -v "Line is"
Elaboration: You can use the pipe symbol ( | ) multiple times in one command!
Source
I think I might of solved this by using a combination of awk and grep.
I am testing it at the moment but so far it works.
#!/usr/bin/ksh
echo "Checking For The Last 5 Fatals"
grep "FATAL ECSDPROD" /data1/log/startstop/MonitorDaemon.log|tail > /tmp/AH/linesDP.txt
grep "FATAL ECSD" /tmp/AH/linesDP.txt | grep -v "Line is"
echo "\n\n----------\n"
echo "Checking For New Fatals"
tail -f /data1/log/startstop/MonitorDaemon.log | awk '/FATAL ECS/' | grep -v "Line is"
echo "about to exit"
exit 0
Related
I am using grep to printout the matching lines from a very large file
from which i got hundreds of matches, some of them are not interesting i want to exclude those matching which are not interesting
grep "WARNING" | grep -v "WARNING_HANDLING_THREAD" path # i tried this
When I grep the file for warning I get
0-00:00:33.392 (2127:127:250:02 = 21.278532 Fri Feb 1 10:17:22 2019) <3:0x000a>:[89]:[enter]: cest_handleFreeReq.c:116: [WARNING]: cest_handleFreeReq: sent from DECA ->UCS
0-00:00:38.263 (2189:022:166:06 = 21.891510 Fri Feb 1 10:17:28 2019) <3:0x000a>:[89]:[enter]: cest_handleConfigReq.c:176: [WARNING]: cest_handleConfigReq.c: GroupConfig NOT present.
0-00:00:38.263 (2189:022:167:03 = 21.891510 Fri Feb 1 10:17:28 2019) <3:0x000a>:[89]:[enter]: cest_handleConfigReq.c:194: [WARNING]: cest_handleConfigReq: physicalConfig NOT present.
60 0x6d77 0 0x504ea | 2 18 | 0 0 | 4 12 | 647 | 14685 0 0.0 0 500 500 | 0 | 0 | 38 | ETH_DRV_WARNING_HANDLING_thread
60 0 | 0 0 | 0 0 0 | 0 0 0 0 0 0 ! N/A N/A N/A N/A N/A N/A |ETH_DRV_WARNING_HANDLING_thread
WARNING: List of threads violating the heap & stack limit
I want to exclude the last lines which are not interesting
0-00:00:33.392 (2127:127:250:02 = 21.278532 Fri Feb 1 10:17:22 2019) <3:0x000a>:[89]:[enter]: cest_handleFreeReq.c:116: [WARNING]: cest_handleFreeReq: sent from DECA ->UCS
0-00:00:38.263 (2189:022:166:06 = 21.891510 Fri Feb 1 10:17:28 2019) <3:0x000a>:[89]:[enter]: cest_handleConfigReq.c:176: [WARNING]: cest_handleConfigReq.c: GroupConfig NOT present.
0-00:00:38.263 (2189:022:167:03 = 21.891510 Fri Feb 1 10:17:28 2019) <3:0x000a>:[89]:[enter]: cest_handleConfigReq.c:194: [WARNING]: cest_handleConfigReq: physicalConfig NOT present.
Is there a way to do this using grep find or any other tool?
Thank you
Note that the substring thread is in lower case in the data, but in upper case in your expression.
Instead, use
grep -F 'WARNING' logfile | grep -F -v 'WARNING_HANDLING_thread'
The -F make grep use string comparisons rather than regular expression matching (this is not really related to your current issue, but just a way of showing that we know what type of pattern we're matching with).
Another option would be to make the second grep do case insensitive matching with -i:
grep -F 'WARNING' logfile | grep -Fi -v 'WARNING_HANDLING_THREAD'
In this case though, I would probably match the [WARNING] tag instead:
grep -F '[WARNING]:' logfile
Note that here we need the -F so that grep interprets the pattern as a string and not as a regular expression matching any single character out of the W, A, R, N, I, G set, followed by a :.
I am a new leaner and I am running a program with following script:
./Multiwfn >HF-Dr.out << EOF
HF.fchk
3
21
2
2
grep 'Global surface minimum:' HF-ESP.out | awk '{print $7,$8,$9,$7,$8,$9}'
EXIT
EOF
The output of this grep is something like:
0.043532 -0.032964 1.960094 0.043532 -0.032964 1.960094
I want to use the output of grep instead of input in the script, i.e. I want the script like:
./Multiwfn >HF-Dr.out << EOF
HF.fchk
3
21
2
2
0.043532 -0.032964 1.960094 0.043532 -0.032964 1.960094
EXIT
EOF
Is there any way that I hide (make non-executable) the grep input from my program and use only its output? Thank you in advance.
I do not understand the 'make non-executable' part of your question. As I understand it, you want it executed before it is given to ./Multiwfn.
There are several solutions:
1) PesaThe's comment:
./Multiwfn >HF-Dr.out << EOF
HF.fchk
3
21
2
2
$(grep 'Global surface minimum:' HF-ESP.out | awk '{print $7,$8,$9,$7,$8,$9}')
EXIT
EOF
2) Create the input in a pipe:
(echo -e "HF.fchk\n3\n21\n2\n2;grep 'Global surface minimum:' HF-ESP.out | awk '{print $7,$8,$9,$7,$8,$9}';echo EXIT) | ./Multiwfn >HF-Dr.out
3) use an intermediate file:
cat > file <<EOF
HF.fchk
3
21
2
2
EOF
grep 'Global surface minimum:' HF-ESP.out | awk '{print $7,$8,$9,$7,$8,$9}' >> file
echo EXIT >> file
./Multiwfn < file >HF-Dr.out
rm file
I'm sure that there are more ways, but this should put you on the right track.
I am trying to use grep to just capture a number in a string but I am having difficulty.
echo "There are <strong>54</strong> cities | grep -o "([0-9]+)"
How am I suppose to just have it return "54"? I have tried the above grep command and it doesn't work.
echo "You have <strong>54</strong>" | grep -o '[0-9]' seems to sort of work but it prints
5
4
instead of 54
Don't parse HTML with regex, use a proper parser :
$ echo "There are <strong>54</strong> cities " |
xmllint --html --xpath '//strong/text()' -
OUTPUT:
54
Check RegEx match open tags except XHTML self-contained tags
You need to use the "E" option for extended regex support (or use egrep). On my Mac OSX:
$ echo "There are <strong>54</strong> cities" | grep -Eo "[0-9]+"
54
You also need to think if there are going to be more than one occurrence of numbers in the line. What should be the behavior then?
EDIT 1: since you have now specified the requirement to be a number between <strong> tags, I would recommend using sed. On my platform, grep does not have the "P" option for perl style regexes. On my other box, the version of grep specifies that this is an experimental feature so I would go with sed in this case.
$ echo "There are <strong>54</strong> 12 cities" | sed -rn 's/^.*<strong>\s*([0-9]+)\s*<\/strong>.*$/\1/p'
54
Here "r" is for extended regex.
EDIT 2: If you have the "PCRE" option in your version of grep, you could also utilize the following with positive lookbehinds and lookaheads.
$ echo "There are <strong>54 </strong> 12 cities" | grep -o -P "(?<=<strong>)\s*([0-9]+)\s*(?=<\/strong>)"
54
RegEx Demo
Trying to grep a phrase out of multiple files as they are constantly populated (logs), but with hint as to which file was updated with the phrase.
For example:
grep bindaddr /vservers/*/var/log
gets me:
/vservers/11010/var/log:bindaddr=xxx.xxx.xxx.xxx
/vservers/12525/var/log:bindaddr=xxx.xxx.xxx.xxx
/vservers/12593/var/log:bindaddr=xxx.xxx.xxx.xxx
Which is cool, but I need this for tail -f.
tail -fn 100 /vservers/*/var/log | grep bindaddr
gets me the lines needed but no indicator in which file, so I need a mix of the two.
If you use -v in tail, you get a verbose mode: from man tab --> "always output headers giving file names". This way, whenever something happens in a file, you will get the header on the preceding line.
Together with this, you can use grep -B1 to show the match + the previous line.
All together, this should do:
tail -fvn 100 /vservers/*/var/log | grep -B1 bindaddr
Test
Doing this in one tab:
$ echo "hi" >> a2
$ echo "hi" >> a2
$ echo "hi" >> a1
$ echo "hi" >> a2
I got this in the other one:
$ tail -vfn 100 /tmp/a* | grep -B1 "h"
==> /tmp/a1 <==
==> /tmp/a2 <==
hi
hi
==> /tmp/a1 <==
hi
==> /tmp/a2 <==
hi
Something like this to put the filename in the front of each line from tail:
#!/bin/bash
# Arrange to kill all descendants on exit/interrupt
trap "kill 0" SIGINT SIGTERM EXIT
for f in *.txt; do
tail -f "$f" | sed "s/^/"$f": /" > /dev/tty &
done
# grep in stdin (i.e. /dev/tty)
grep bina -
I think some people are coming to this post looking for a way to display the filename while grepping the tail of multiple files:
for f in path/to/files*.txt; do echo $f; tail $f | grep 'SEARCH-THIS'; done;
This will display an output like this
filename1.txt
search result 1
search result 2
filenam2.txt
search result 3
search result 4
...
tail /mnt/mysqld_log/mysql_error_log.err | grep -e `date +'%y%m%d' --date='4 hour ago'` | more
120824 11:25:06 [ERROR] /usr/libexec/mysqld: Table '.zone_assoc' is marked as crashed and should be repaired
120824 18:03:23 [ERROR] /usr/libexec/mysqld: Incorrect key file for table '.ad_zone_assoc.MYI'; try to repair it
120824 18:08:38 [ERROR] /usr/libexec/mysqld: Incorrect key file for table '.ad_zone_assoc.MYI'; try to repair it
The above works as expected and shows the results for today's date. But the following does not work.
tail /mnt/mysqld_log/mysql_error_log.err | grep -e `date +'%y%m%d %k' --date='4 hour ago'` | more
grep: 18: No such file or directory
I tried to escape the space and that did not work as well.
tail /mnt/mysqld_log/mysql_error_log.err | grep -e `date +'%y%m%d\ %k' --date='4 hour ago'` | more
grep: Trailing backslash
You need to wrap the date command with quotes:
tail /mnt/mysqld_log/mysql_error_log.err | grep -e "`date +'%y%m%d %k' --date='4 hour ago'`" | more
The space is being interpreted as the file you are trying to read from, by encasing it in quotes, this will not happen.