Hi I'm trying to search a log file for the following words and assign the number of matches to a variable as a number.
errors
error
fail
failure
can't
But I don't want to match the words error or errors if its preceeded by "No "
So ignore error if its "no error" and ignore errors if its "no errors"
Here's what I have so far
ErrorCheck=$(grep -vi "No errors" $LOGFILE | grep -ciE "error|fail|can't" $LOGFILE)
Its not working out for me unfortunatly, any suggestions would be great.
PS I'm using microcore running busybox shell, so I have a slightly lean environment to work in.
All comments and suggestions welcome.
Thanks for your input.
Well, I think one problem is that the "-v" flag for grep will basically omit the entire line containing 'No errors' or any other string you specify. So if a single line contains both "no failures" and "can't" for example, you'd have a problem.
One possible (sort of janky) way to do this could be to store values in three different variables: NUM_COUNTED minus NUM_NOTS = ErrorCheck, which should account for having a "no failure" and an actual failure indicator in the same line.
NUM_COUNTED=grep -ciF 'error
fail
can't' $LOGFILE
NUM_NOTS=grep -ciF 'no error
no fail' $LOGFILE
ErrorCheck=`expr $NUM_COUNTED - $NUM_NOTS`
Alternatively, this seems to be giving (mostly accurate) results:
ErrorCheck=$(grep -vi 'No errors' $LOGFILE | grep -ciF 'error
fail
can't' $LOGFILE)
The -F flag just tells grep to look for string literals (error, fail, and can't) which are newline separated.
Hope this helps.
Related
I've file containing various log lines, and I want to grep for a pattern but exclude when message:com.mycompany.excluded , so basically following should be returned :
"The log found this message blah, message:com.blahblah"
"The log found this message blah2, message:com.blahblah2"
"The log found this message foobar, message:com.mycompany"
"The log found this message blah, message:com.mycompany.included"
but not :
"The log found this message blah, message:com.mycompany.excluded"
I'm using this pattern, but it would not work for excluding com.mycompany.excluded
grep "The log found this message.*message:.*" "mylogs.txt"
awk '/The log found this message.*message:/ && !/com.mycompany.excluded/' "mylogs.txt"
There's various ways to make it more robust but they're probably not necessary (depending on what the rest of your log file looks like).
What you want here is sed, not grep:
sed -i .bak 's/^.*excluded//' mylogs.txt
(The -i .bak creates a backup of your original file with the extension .bak; this would leave you with mylogs.txt and mylogs.txt.bak, with mylogs.txt having the undesired lines removed.)
I have a custom package I want to install automatically in my docker using expect.
The first thing the package asks me to do is press Enter to continue, then it prints another 2 empty lines then it waits for an input.
My expect script :
#!/usr/bin/expect -f
set timeout -1
spawn ./install
expect "\n"
send -- "\n"
But as you can see in the image, it just runs the installer and exits.
I tried removing the expect "\n" so only send -- "\n" will execute but now even the install message doesn't appear (tried with set timeout 1000 before send and it also didn't work)
Any ideas?
P.S : This is a link to the package if anyone wants to have a go at it:
https://www.bayometric.com/downloads/digital-persona/DP_UareU_Linux223_20140429.2.zip
(the installer is inside DP-UareU-RTE-2.2.3-1.20140429_1533.tar.gz)
expect "\n" match a linefeed exactly, I think this is not what your program is sending.
To wait for a Shell prompt you can use expect "%" or expect "*" to match anything.
If you need to make sure you're dealing with the right prompt you may be able to use something like expect "*Linux Installation*".
Also don't send \n but \r for the enter key :
#!/usr/bin/expect
spawn ./install
expect "*Linux Installation*"
send "\r"
expect eof
Note that the default flag is -gl for glob pattern matching but you can also use the -re flag for regular expression matching.
We have a error log directory structure wherein we store all errors log files for a particular day in datewise directories -
errorbackup/20150629/errorlogFile3453123.log.xml
errorbackup/20150629/errorlogFile5676934.log.xml
errorbackup/20150629/errorlogFile9812387.log.xml
errorbackup/20150628/errorlogFile1097172.log.xml
errorbackup/20150628/errorlogFile1908071_log.xml
errorbackup/20150627/errorlogFile5675733.log.xml
errorbackup/20150627/errorlogFile9452344.log.xml
errorbackup/20150626/errorlogFile6363446.log.xml
I want to search for a particular string in the error log file and get the output such that I will get directory wise search result of a count of that string's occurrence. For example grep "blahblahSQLError" should output something like-
20150629:0
20150628:0
20150627:1
20150626:1
This is needed because we fixed some errors in one of the release and I want to make sure that there are no occurrences of that error since the day it was deployed to Prod. Also note that there are thousands of error log files created every day. Each error log file is created with a random number in its name to ensure uniqueness.
If you are sure the filenames of the log files will not contain any "odd" characters or newlines then something like the following should work.
for dir in errorbackup/*; do
printf '%s:%s\n' "${dir#*/}" "$(grep -l blahblahSQLError "$dir/"*.xml | wc -l)"
done
If they can have unexpected names then you would need to use multiple calls to grep and count the matching files manually I believe. Something like this.
for dir in errorbackup/*; do
_dcount=0;
for log in "$dir"/*.xml; do
grep -l blahblahSQLError "$log" && _dcount=$((_dcount + 1));
done
done
Something like this should do it:
for dir in errorbackup/*
do
awk -v dir="${dir##*/}" -v OFS=':' '/blahblahSQLError/{c++} END{print dir, c+0}' "$dir"/*
done
There's probably a cuter way to do it with find and xargs to avoid the loop and you could certainly do it all within one awk command but life's too short....
I'm working on the first steps towards creating a powershell script that will read through printer logs (probably using get-WMI cmdlet), and parse through the logs. Afterwards, I plan on having the script output to a .txt file the name of the printer, a counter of the number of times a printer was used (if possible), and specific info found in the logs.
In order to do this, I've decided to try working backwards. Below is a small portion of what the logs will look like:
10 Document 81, A361058/GPR0000151814_1: owned by A361058 was printed on R3556 via port IP_***.***.***.***. Size in bytes: 53704; pages printed: 2 20130219123105.000000-300
10 Document 80, A361058/GPR0000151802_1: owned by A361058 was printed on R3556 via port IP_***.***.***.***. Size in bytes: 53700; pages printed: 2
Working backwards and just focusing on parsing first, I'd like to be able to specifically get the "/GRP", "R3446 (in general, R** as this is the printer name)", and get a counter that shows how often a specific printer appeared in the log files.
It has been a while since I last worked with Powershell, however at the moment this is what I've managed to create in order to try accomplishing my goal:
Select-String -Path "C:\Documents and Settings\a411882\My Documents\Scripts\Print Parse Test.txt" -Pattern "/GPR", " R****" -AllMatches -SimpleMatch
The code does not produce any errors, however I'm also unable to get any output to appear on screen to see if I'm capturing the /GRP and printer name. At the moment I'm trying to just ensure I'm gathering the right output before worrying about any counters. Would anyone be able to assist me and tell me what I'm doing wrong with my code?
Thanks!
EDIT: Fixed a small error with my code that was causing no data to appear on screen. At the moment this code outputs the entire two lines of test text instead of only outputting the /GPR and server name. The new output is the following:
My Documents\Scripts\Print Parse Test.txt:1:10 Document 81, A361058/GPR0000151814_1: owned by A361058 was printed on
R3556 via port IP_***.***.***.***. Size in bytes: 53704; pages printed: 2
20130219123105.000000-300
My Documents\Scripts\Print Parse Test.txt:2:10 Document 80, A361058/GPR0000151802_1: owned by A361058 was printed on
R3556 via port IP_***.***.***.***. Size in bytes: 53700; pages printed: 2
I'd like to try having it eventually look something like the following:
/GPR, R****, count: ## (although for now I'm less concerned about the counter)
You can try this. It only returns a line when /GPR (and "on" from "printed on") is present.
Get-Content .\test.txt | % {
if ($_ -match '(?:.*)(/GPR)(?:.*)(?<=on\s)(\w+)(?:.*)') {
$_ -replace '(?:.*)(/GPR)(?:.*)(?<=on\s)(\w+)(?:.*)', '$1,$2'
}
}
Output:
/GPR,R3556
/GPR,R3556
I'm sure there are better regex versions. I'm still learning it :-)
EDIT this is easier to read. The regex is still there for extraction, but I filter out lines with /GPR first using select-string instead:
Get-Content .\test.txt | Select-String -SimpleMatch -AllMatches -Pattern "/GPR" | % {
$_.Line -replace '(?:.*)(/GPR)(?:.*)(?<=on\s)(\w+)(?:.*)', '$1,$2'
}
I generally start with an example of the line I'm matching, and build a regex from that, substituting regex metacharacters for the variable parts of the text. This makes makes the regex longer, but much more intuitive to read later.
Assign the regex to a variable, and then use that variable in subsequent code to keep the messy details of the regex from cluttering up the rest of the code:
[regex]$DocPrinted =
'Document \d\d, \w+/(\D{3})[0-9_]+: owned by \w+ was printed on (\w+) via port IP_[0-9.]+ Size in bytes: \d+; pages printed: \d+'
get-content <log file> |
foreach {
if ($_ -match $DocPrinted)
{
$line -match $docprinted > $null
$matches
}
}
all! I'm trying to compile a program using PowerShell, but the command is being parsed strangely.
This command executes correctly in cmd.exe:
dmd -od"bin" -of"bin\convHull.exe" -I"src" "src\concSort.d" "src\fileParser.d" "src\main.d" "src\pointLogic.d" "src\quickHull.d" "src\stupidHull.d" -D -O -release
But PowerShell executes it as: (blue, navy, purple texts as they appear in PowerShell ISE)
dmd -od"bin" -of"bin\convHull.exe" -I"src" "src\concSort.d" "src\fileParser.d" "src\main.d" "src\pointLogic.d" "src\quickHull.d" "src\stupidHull.d" -D -O -release
This spits the following error:
The string starting:
At line:1 char:147
+ dmd -od"bin" -of"bin\convHull.exe" -I"src" "src\concSort.d" "src\fileParser.d" "src\main.d"
"src\pointLogic.d" "src\quickHull.d" "src\stupidHull.d <<<< " -D -O -release
is missing the terminator: ".
At line:1 char:163
So it seems to be interpreting a period as a quote. This is peculiar. Has anyone else had this problem with PowerShell?
Things I've tried:
escaping quotes
making sure all quotes are "straight quotes" and not angled
putting a space before quotes (parses correctly, but the program doesn't understand the arguments.)
Thanks,
Charles.
I believe this should do the trick (newlines added for clarity only, and removal of extra quotes):
dmd '-od"bin"' '-of"bin\convHull.exe"' '-I"src"'
src\concSort.d src\fileParser.d src\main.d src\pointLogic.d src\quickHull.d src\stupidHull.d
-D -O -release
Note that in the case where a quote (") is to be passed as part of the argument itself, I surrounded the entire argument with single quotes ('). From the experimentation below it can be seen that only -of"..." needs the quotes about it.
Happy coding.
I can't find a good reference on this exact production, however note the following parsings:
-x"w." -> error: " expected (last " is special)
-x"w."" -> -x"w and ."" (the . starts a new token and the " in that starts
a quote; however, the quotes are not removed)
'-x"w."' -> -x"w." (extra quote fine, neither special)
-x"w" -> -x"w" (no . and " not special)
-x"w"" -> -x"w"" (no . and " not special)
a".b" -> a.b (didn't start with `-`, quotes removed)
a".b -> error: " expected (" is special)
So it does indeed appear to have something to do with the . and - combination (and it might not be exclusive). From the above I believe that a token starting with - does not include the . character as a valid character in the token so the lexer terminates said token and starts a new token -- easily provable with a good EBNF reference, which I don't have.
The best I can find is Appendix C: The PowerShell Grammar:
The ParameterToken rule is used to match cmdlet parameters such as -foo or -
boolProp: . Note that this rule will also match --foobar, so this rule has
to be checked before the --token rule.
<ParameterToken> = -[:letter:]+[:]{0 |1}
However, this is incomplete at best and does not even include a definition of "letter".
I don't have the executable, but this seems to want to work.
$cmd = #'
dmd -od"bin" -of"bin\convHull.exe" -I"src" "src\concSort.d" "src\fileParser.d" "src\main.d" "src\pointLogic.d" "src\quickHull.d" "src\stupidHull.d" -D -O -release
'#
&$cmd