A lot of times when I use ps and pipe it into grep, I put one of the grep characters in [ ] square brackets. I do this do the actual call to grep is not included in the ps readout. It helps when I am scripting. Like if I redirect the output to a file, and that file is empty, the process I am looking at is not on. I do not know why bracketing one character of the grep works, and was amused to see that using the brackets is actually faster and takes less processing time.
-bash-3.2$ time ps -ef | grep *.[j]il
real 0m0.027s
user 0m0.012s
sys 0m0.017s
-bash-3.2$ time ps -ef | grep *.jil
zksuy7k 11528 18285 0 20:54 pts/7 00:00:00 grep *.jil
real 0m0.040s
user 0m0.015s
sys 0m0.016s
-bash-3.2$
Related
I want the output of the sed file edit to go into my log file name d_selinuxlog.txt. Currently, grep outputs the specified string as well as 3 other strings above and below in the edited file.
#!/bin/bash
{ getenforce;
sed -i s/SELINUX=enforcing/SELINUX=disabled /etc/selinux/config;
grep "SELINUX=*" /etc/selinux/config > /home/neb/scropts/logs/d_selinuxlog.txt;
setenforce 0;
getenforce; }
I want to be seeing just SELINUX=disabled in the log file
All the lines with the lines SELINUX are going to match, even the commented ones, so, you need to omit that ones, and the * from the match.
grep "SELINUX=" /etc/selinux/config | grep -v "#"
This is my output
17:52:07 alvaro#lykan /home/alvaro
$ grep "SELINUX=" /etc/selinux/config | grep -v "#"
SELINUX=disabled
17:52:22 alvaro#lykan /home/alvaro
If I cat text file containing mysqld_safe and mysqld then grep -w works as expected. But when piped with PS output it does not work.
ps -ef | grep -w mysqld
Output's both lines.
/usr/bin/mysqld_safe
/usr/libexec/mysqld
I am expecting only mysqld. I'm aware of exclude option grep -v mysqld_safe.
Version - grep (GNU grep) 2.5.1
If you have pgrep, using pgrep -x mysqld would be better choice than ps + grep
From man pgrep
pgrep, pkill - look up or signal processes based on name and other attributes
-x, --exact
Only match processes whose names (or command line if -f is specified) exactly match the pattern.
-l, --list-name
List the process name as well as the process ID. (pgrep only.)
-c, --count
Suppress normal output; instead print a count of matching processes. When count does not match any‐
thing, e.g. returns zero, the command will return non-zero value.
-n, --newest
Select only the newest (most recently started) of the matching processes.
With docker stats you can see the memory usage of a container over time.
Is there a way to find what the highest value of memory usage was while running docker stats?
If you need to find the peak usage you are better off requesting the .MemPerc option and calculating based on the total memory (unless you restricted the memory available to the container). .MemUsage has units which change during the life of the container which mess with the result.
docker stats --format 'CPU: {{.CPUPerc}}\tMEM: {{.MemPerc}}'
You can stream an ongoing log to a file (or script).
To get just the max memory as originally requested:
(timeout 120 docker stats --format '{{.MemPerc}}' <CONTAINER_ID> \
| sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' ; echo) \
| tr -d '%' | sort -k1,1n | tail -n 1
And then you can ask the system for its total RAM (again assuming you didn't limit the RAM available to docker) and calculate:
awk '/MemTotal/ {print $2}' /proc/meminfo
You would need to know how long the container is going to run when using timeout as above, but if docker stats was run without this in background submitted by a script it could kill it once the container completed.
...
This command allows you to generate a time-series of the cpu/memory load:
(timeout 20 docker stats --format \
'CPU: {{.CPUPerc}}\tMEM: {{.MemPerc}}' <CONTAINER_ID> \
| sed 's/\x1b\[[0-9;]*[a-zA-Z]//g' ; echo) \
| gzip -c > monitor.log.gz
Note that it pipes into gzip. In this form you get ~2 rows per second so the file would get large rapidly if you don't.
I'd advise this for benchmarking and trouble shooting rather than use on production containers
I took a sampling script from here and aggregated data by #pl_rock. But be careful - the sort command only compares string values - so the results are usually wrong (but ok for me).
Also mind that docker is sometimes reporting wrong numbers (ie. more allocated mem than physical RAM).
Here is the script:
#!/bin/bash
"$#" & # Run the given command line in the background.
pid=$!
echo "" > stats
while true; do
sleep 1
sample="$(ps -o rss= $pid 2> /dev/null)" || break
docker stats --no-stream --format "{{.MemUsage}} {{.Name}} {{.Container}}" | awk '{ print strftime("%Y-%m-%d %H:%M:%S"), $0 }' >> stats
done
for containerid in `awk '/.+/ { print $7 }' stats | sort | uniq`
do
grep "$containerid" stats | sort -r -k3 | tail -n 1
# maybe: | sort -r -k3 -h | head -n 1
# see comment below (didnt tested)
done
In my case I wanted to monitor a docker container which runs tests for my web application. The test suite is pretty big, it includes javascript tests in a real browser and consume significant amount of both, memory and time.
Ideally, I wanted to watch the current memory usage real time, but to also keep the history for later analysis.
I ended up using a modified and simplified version of the Keiran's solution:
CONTAINER=$(docker ps -q -f name=CONTAINER_NAME)
FORMAT='{{.MemPerc}}\t{{.MemUsage}}\t{{.Name}}'
docker stats --format $FORMAT $CONTAINER | sed -u 's/\x1b\[[0-9;]*[a-zA-Z]//g' | tee stats
Notes:
CONTAINER=$(docker ps -q -f name=NAME) find container by name, but there are other options
FORMAT='{{.MemPerc}} ...}} MemPerc goes first (for sorting); otherwise you can be creative
sed -u the -u flag is important, it turns off buffering
| sed -u 's/\x1b\[[0-9;]*[a-zA-Z]//g' removes ANSI escape sequences
| tee stats not only display in real time, but also write into the stats file
I Ctrl-C manually when it's ready – not ideal, but OK for me
after that it's easy to find the max with something like sort -n stats | tail
you can use command:
docker stats --no-stream | awk '{ print $3 }' | sed '1d'|sort | tail -1
It will give highest memory by container.
Let me Explain command:
--no-stream : Disable streaming stats and only pull the first result
awk '{ print $3 }' : will print MEM USAGE
sed '1d' : will delete first entry that is %
sort : it will sort the result
tail -1 : it will give last entry that is highest.
If I use a grep command like this ls | grep '^[-[:alnum:]\._]+$' to match filenames, it outputs no result but when the command changes to ls | grep '^[-[:alnum:]\._]*$', it works correctly. What's going on?
grep uses "basic" regexes by default, where + is a normal character. You need \+ to match 1 or more things (or use grep -E).
I often have to look for specific strings in a big set of log files with grep. And I get lots of results, on what I must scroll a lot.
Today, the results of grep list the results in alphabetical order. I would like to have my grep results reversed ordered by time, like a ls -ltr would do.
I know I could take the result of ls -ltr and grep file by file. I do it like this:
ls -ltr ${my_log_dir}\* | awk '{print $9}' |xargs grep ${my_pattern}
But I wonder: Is there a simpler way?
PS: I'm using ksh on AIX.
The solution I found (thanks to Fedorqui) was to simply use ls -tr. It assumes the results are passed in the right order through the | to xargs allowing then to do the grep.
My misconception was that since when I us ls, the results arrive not as a single column list but as a multiple column list, it couldn't work as an input for xargs.
Here is the simplest solution to date, since it avoids any awk parsing:
ls -tr ${my_log_dir}\* | xargs grep ${my_pattern}
I checked, and every result of the ls -t are passed to xargs even though they look not as I expected they would enter easily in it:
srv:/papi/tata $ ls -t
addl ieet rrri
ooij lllr sss
srv:/papi/tata $ ls -t |xargs -I{} echo {}
addl
ieet
rrri
ooij
lllr
sss
This will work too use find command:-
find -type f -print0 | xargs -r0 stat -c %y\ %n | sort -r | awk '{print $4}' | sed "s|^\./||"
-print0 in find to preserve files having special characters(whitespaces, tabs)
Print file status (stat with %y (Time of last modification) and %n (%n File name) with output having new-separated (-c)
Reverse sort the output from previous command. (-r for reverse)
awk '{print $4}' printing only the file-name (can be optimized as needed)
Removing the leading ./ from the file-names.