I am using simple grep like
grep -E "input files : 0" *.stdout
and its output is following:
QCD_HT200to300_13TeV_ext.stdout:==> Total number of input files : 0
WJetsToLNu_HT_600To800_13TeV_ext1.stdout:==> Total number of input files : 0
If I use the same grep command within the for loop then its output is not nice. My script is
foreach file ( `grep -E "input files : 0" *.stdout` )
echo $file
end
the output of this is like:
QCD_HT200to300_13TeV_ext.stdout:==>
Total
number
of
input
files
:
0
WJetsToLNu_HT_600To800_13TeV_ext1.stdout:==>
Total
number
of
input
files
:
0
How I can get the same output within foreach loop as I get from grep command running independently.
Use the list of files to iterate through in the foreach loop and do the grep inside the foreach loop.
set files = `ls *.stdout`
foreach file ( $files )
grep -E "input files : 0” $file
end
Related
I'd like to write a bit of code that looks like this:
while ( `ls -1 ${JOB_PREFIX}_${job_counter}_*.out | wc -l` > 0 )
echo "Do something here."
end
But every time there is no ls -1 ${JOB_PREFIX}_${job_counter}*.csh it gives an annoying ls: No match.
Is it possible to suppress this error message but still pipe STDOUT to wc ? I went through the existing questions on this but most of the answers are either not working on csh or tries to combine STDOUT and STDERR with |& which I do not want.
The cleanest way would have been to mute the ls itself, but it is not possible.
You can suppress your entire script, meaning:
If your loops is in a file called myscript.cs and you call your file with some args myscript.cs -arg1 blabla -arg2 bla, add >& to your shell command to redirect stderr to wherever you want. e.g.
myscript.cs -arg1 blabla -arg2 bla >& /dev/null
It's not answering your answer directly, but it should solve your problem.
Edit
Since the comment added that the script should be redirected to another script, you can split your line to two lines:
if `ls -1 ${JOB_PREFIX}_${job_counter}_*.out >& /dev/null` then
while < your while here >
end
endif
csh and tcsh cannot redirect stderr by itself, one of many reasons you should not use it for scripting.
If you're okay with spawning another shell, you can change JOB_COUNTER into an environment variable and do:
while ( `sh -c '2>/dev/null ls -1 ${JOB_PREFIX}_${JOB_COUNTER}_*.out | wc -l'` > 0 )
echo "Do something here."
end
If you want to use a test as a condition in a while loop instead of an expression, you have to get creative.
#!/bin/tcsh -f
set i = 4
while ( 1 )
if ( $i <= 0 ) break
echo hi
# i = ($i - 1)
end
In your case, this might look something like: I changed the wc -l to a grep -q . so that we can use the exit status instead of the result printed to stderr.
#!/bin/tcsh -f
set dir_exit_status = 0
while ( $dir_exit_status = 0 )
( ls -1 ${JOB_PREFIX}_${job_counter}_*.out | grep -q '.') >& /dev/null
set dir_exit_status = $status
end
I am running a Teamspeak 3 server on a Ubuntu server and I would like to fetch the clients currently connected using a script.
The script currently outputs this from the Teamspeak Server Query:
clid=1 cid=11 client_database_id=161 client_nickname=Music client_type=1|clid=3 cid=11 client_database_id=153 client_nickname=Music\sBot client_type=0|clid=5 cid=1 client_database_id=68 client_nickname=Unknown\sfrom\s127.0.0.1:52537 client_type=1|clid=12 cid=11 client_database_id=3 client_nickname=FriendlyMan client_type=0|clid=16 cid=11 client_database_id=161 client_nickname=Windows\s10\sUser client_type=0|clid=20 cid=11 client_database_id=225 client_nickname=3C2J0N47H4N client_type=0
How can I extract the nicknames from this mess?
More Specifically only the ones that contain "client_type=0".
Played around with GREP (grep -E -o 'client_nickname=\w+'), close to what I want.
client_nickname=Music
client_nickname=Music
client_nickname=Unknown
client_nickname=FriendlyMan
client_nickname=Windows
client_nickname=3C2J0N47H4N
Desired Output:
Music Bot,FriendlyMan,Windows 10 User,3C2J0N47H4N
Our input consists of a single line:
$ cat file
clid=1 cid=11 client_database_id=161 client_nickname=Music client_type=1|clid=3 cid=11 client_database_id=153 client_nickname=Music\sBot client_type=0|clid=5 cid=1 client_database_id=68 client_nickname=Unknown\sfrom\s127.0.0.1:52537 client_type=1|clid=12 cid=11 client_database_id=3 client_nickname=FriendlyMan client_type=0|clid=16 cid=11 client_database_id=161 client_nickname=Windows\s10\sUser client_type=0|clid=20 cid=11 client_database_id=225 client_nickname=3C2J0N47H4N client_type=0
Using grep + sed
Here is one approach that starts with grep and then uses sed to cleanup to the final format:
$ grep -oP '(?<=client_nickname=)[^=]+(?=client_type=0)' file | sed -nE 's/\\s/ /g; H;1h; ${x; s/ *\n/,/g;p}'
Music Bot,FriendlyMan,Windows 10 User,3C2J0N47H4N
Using awk
Here is another approach that just uses awk:
$ awk -F'[= ]' '/client_type=0/{gsub(/\\s/, " ", $8); printf (f?",":"")$8; f=1} END{print ""}' RS='|' file
Music Bot,FriendlyMan,Windows 10 User,3C2J0N47H4N
The awk code uses | as the record separator and awk reads in one record at a time. Each record is divided into fields with the field separator being either a space or an equal sign. If the record contains the text client_type=0, then we replace all occurrences of \s in field 8 with space and then print the resulting field 8.
Using bash
#!/bin/bash
sep=
( cat file; echo "|"; ) | while read -r -d\| clid cid db name type misc
do
[ "$type" = "client_type=0" ] || continue
name=${name//\\s/ }
printf "%s%s" "$sep" "${name#client_nickname=}"
sep=,
done
echo ""
This produces the output:
Music Bot,FriendlyMan,Windows 10 User,3C2J0N47H4N
I am interested in finding a string within a specific file type.
The command below serves my purpose.
find /any/path -type f -name "*.log" | xargs grep -B2 -A2 'SUMMARY' {} \;
It gives the following output:
--
/path/to/file.log-line1
/path/to/file.log-line2
/path/to/file.log:text SUMMARY text
/path/to/file.log-line1
/path/to/file.log-line2
--
I would like the file name not to be prepended to each line. Is it possible to have the output as below?
--
/path/to/file.log
line1
line2
text SUMMARY text
line1
line2
--
If you're running this under linux with bash, you could use a bash script like this:
#!/bin/bash
for fn in `grep --include \*.txt -lr 'SUMMARY' /any/path`; do
echo $fn
grep -A2 -B2 'SUMMARY' $fn
done
This will find all files containing the word "SUMMARY" in a recursive manner starting from the directory "/any/path". All matched files are then printed by name and the matched portion is printed with the second grep line.
I use the following command to my liking, but perfection is better;-)
grep -w -i -r -n -f all.txt . > output.txt
./index.php:86:complete paragraph1
./index.php:89:complete paragraph2
With this:
grep -w -i -r -o -n -f all.txt . > output.txt
We get :
./index.php:86:match1
./index.php:89:match2
Is it also possible to get a combination of that? Like this:
./index.php:86:match1:complete paragraph1
./index.php:89:match2:complete paragraph2
Would be great, still better than that would be even a part ofthe paragraph, but i guess that is a little much to ask for with such a simple tool;-)
Thanks!
grep doesn't have a facility for this, but it's easy to reimplement the useful parts in a simple Awk script.
awk 'NR==FNR { p[++i] = tolower($0); next }
{ line = tolower($0); for (j=1; j<=i; ++j) if (match(line, p[j]))
{ printf "%s:%i:%s:%s\n", FILENAME, FNR, substr($0, RSTART, RLENGTH), $0;
next } }' all.txt files...
The NR==FNR condition matches on the first input file. Each line in that file is converted to lowercase and read into the array p.
The second action only applies to the second and subsequent files. It loops over the items in p and checks whether the current line matches. If so, a match message is printed, and we skip to the next input line.
Trying to grep a phrase out of multiple files as they are constantly populated (logs), but with hint as to which file was updated with the phrase.
For example:
grep bindaddr /vservers/*/var/log
gets me:
/vservers/11010/var/log:bindaddr=xxx.xxx.xxx.xxx
/vservers/12525/var/log:bindaddr=xxx.xxx.xxx.xxx
/vservers/12593/var/log:bindaddr=xxx.xxx.xxx.xxx
Which is cool, but I need this for tail -f.
tail -fn 100 /vservers/*/var/log | grep bindaddr
gets me the lines needed but no indicator in which file, so I need a mix of the two.
If you use -v in tail, you get a verbose mode: from man tab --> "always output headers giving file names". This way, whenever something happens in a file, you will get the header on the preceding line.
Together with this, you can use grep -B1 to show the match + the previous line.
All together, this should do:
tail -fvn 100 /vservers/*/var/log | grep -B1 bindaddr
Test
Doing this in one tab:
$ echo "hi" >> a2
$ echo "hi" >> a2
$ echo "hi" >> a1
$ echo "hi" >> a2
I got this in the other one:
$ tail -vfn 100 /tmp/a* | grep -B1 "h"
==> /tmp/a1 <==
==> /tmp/a2 <==
hi
hi
==> /tmp/a1 <==
hi
==> /tmp/a2 <==
hi
Something like this to put the filename in the front of each line from tail:
#!/bin/bash
# Arrange to kill all descendants on exit/interrupt
trap "kill 0" SIGINT SIGTERM EXIT
for f in *.txt; do
tail -f "$f" | sed "s/^/"$f": /" > /dev/tty &
done
# grep in stdin (i.e. /dev/tty)
grep bina -
I think some people are coming to this post looking for a way to display the filename while grepping the tail of multiple files:
for f in path/to/files*.txt; do echo $f; tail $f | grep 'SEARCH-THIS'; done;
This will display an output like this
filename1.txt
search result 1
search result 2
filenam2.txt
search result 3
search result 4
...