As for linux, I can get the real-time of a process through /proc (process file system). But how can I get the data of the process on MacOS 12.4 since there is no /proc dir.
When using top command, I want to save the read-time output of a process to a text file (only the process info), but the interval is too large to get accurate result and I cannot strip the header of top via options. Is there any possible way to achieve this? On a phone, I can get the output of the top via the following script:
#!/bin/sh
export LANG="en_US.UTF-8"
if [ "$#" -ne 1 ]; then
echo "Illegal number of parameters" $#
exit 0
fi
while true; do
pid=`pgrep COMMAND`
if [ "$pid" -ne 0 ]; then
echo "${pid}\t`top -q -p $pid -n 1 -d 1`" >> $1
fi
usleep 100
done
I wonder whether it is possible to get the detailed info via /proc-like file system on MacOS?
Related
I have couple of python docker containers, processing data from kafka topics every X seconds.
If the data isn't appearing for i.e. more than 2X time, I should log it somewhere (so I can check which kafka producers to restart - doesn't matter here).
I was thinking about creating some file, where I would keep last message timestamp, and another script/container would check it every X seconds, to find out, if the data should already have appeared. I could check docker logs aswell, if "amount" of logs is increasing (if not, then it means, that no data is appearing).
While those solutions would probably work, I'm not sure, if there isn't already another "ready-made" solution for that issue. Is there something, that I can use to check if the data is being processed?
You can use some sort of health check mechanism by putting logs inside: <filename_1>
then the healthcheck would be:
if [[ `[ -r <filename_1> ] && cat <filename_1> | wc -l` -ge 1 ]]; then > <filename_1> && echo 0; else echo 1; fi
By doing this you check if the file exist then check the number of lines. If greater/equal than 1 you empty it then return 0 .If empty or does not exist you return 1
This will mark your docker as healthy or unheathly.
If you want to restart by itself you can replace echo 1 by kill 1 kill 1 will kill process with pid 1 in container and force container exit. Usually the process started by CMD or ENTRYPOINT has pid 1.
[[ `[ -r <filename_1> ] && cat <filename_1> | wc -l` -ge 1 ]] && > <filename_1> && echo 0 || kill 1
Don't forget --restart always option.
I have a legacy Docker application I'm working with that uses multiple Celery workers. There is a long running process I need to track. I'm able to write data to a file that is visible from the CLI interface of the worker thread:
I'm writing to the file like this:
def log(msg):
now = datetime.now()
dt_string = now.strftime("%Y-%m-%d %H:%M:%S")
fu.mkdirs(defs.LRP_LOG_DIR)
fu.append_string_to_file(dt_string + ": " + msg + "\n", defs.LRP_LOG_FILE)
def append_string_to_file(string, file_path):
with open(file_path, "a") as text_file:
text_file.write(string)
LRP_LOG_DIR = "/opt/project/backend"
LRP_LOG_FILE = LRP_LOG_DIR + "/lrp-log.txt"
The question is: If I add multiple Celery workers will they each write to their own file (not the desired behaviory) or will they all write to a common /opt/project/backend/lrp-log.txt file (the desired behavior)?
If they don't write to a common file, what do I need to do to get multiple Celery workers to write to the same file?
Also, it would be nice if this file was available on the host file system (I'm running on a Windows machine).
I ended up writing a couple of .sh scripts for Cygwin (I'm on windows). I would like to get the tail to work in the same script but this is good enough for now.
Script to start Docker and write to log file
echo
echo
echo
# STOP CONTAINERS
echo "Stopping all Containers..."
docker kill $(docker ps -q)
# DELETE CONTAINERS
echo "Deleting Containers..."
docker rm $(docker ps -aq)
echo
# PRUNE VOLUMES
echo "Pruning orphaned volumes"
docker volume prune -f
echo
# CREATE LOG DIR
mkdir ./logs
# DELETE OLD FULL LOG FILE
echo "Deleting old full log file..."
touch ./logs/full-log.txt
rm ./logs/full-log.txt
touch ./logs/full-log.txt
# SET UP LRP LOG FILE
echo "Deleting old lrp log file..."
touch ./logs/lrp-log.txt
rm ./logs/lrp-log.txt
# TAIL THE LOG FILE (display the running process in a cygwin window)
cygstart tail -f ./logs/full-log.txt
cygstart tail -f ./logs/lrp-log.txt
# START AES
echo "Starting anonlink entity service (aes)..."
echo "Process is running and writing log to ./full-log.txt"
echo "Long Running Process Log (LRP) is being written to lrp-log.txt"
echo "! ! ! DO NOT CLOSE THIS WINDOW ! ! !"
echo "(<ctrl-c> to quit the process)"
docker-compose -p anonlink -f ../tools/docker-compose.yml up --remove-orphans > ./logs/full-log.txt
echo
echo
echo "Done."
echo
echo
Script to create truncated log file to track long running processes
tail -f ./logs/full-log.txt | grep --line-buffered "LOG_FILE:" > ./logs/lrp-log.txt
Certain processes (like git gc --aggressive) take a long time to run, take up lots of disk space, and die if I run out of disk space. I'd like to pause them if the disk will run out of space soon, so I have time to free up some memory. How can I do this?
Here is an initial solution I came up with. Tested with Mac OS X. Suggestions welcome!
#!/bin/bash
# Change these variables as necessary.
FILESYSTEM="/dev/disk1"
DF=/usr/local/opt/coreutils/libexec/gnubin/df
OSASCRIPT=/usr/bin/osascript
if ! [[ -x $DF ]]; then echo "Error: $DF isn't executable."; exit 1; fi
PID=$1
STOPAT=$2
# Verify input
if [[ -n ${PID//[0-9]/} ]]; then echo "Error: The first parameter should be an integer"; exit 1; fi
if [[ -n ${STOPAT//[0-9]/} ]]; then echo "Error: The second parameter should be an integer"; exit 1; fi
RED='\033[0;31m'; PURPLE='\033[0;35m'; BLUE='\033[0;36m'
NC='\033[0m' # No Color
echo -e "Will pause the following process when there are ${PURPLE}$STOPAT${NC} bytes left on ${PURPLE}$FILESYSTEM${NC}"
PROCESS=`ps -p $PID | grep $PID`
echo -e "${BLUE}$PROCESS${NC}"
# Check every second to see if FILESYSTEM has more than STOPAT bytes left.
while true; do
left=`$DF | grep -m 1 $FILESYSTEM | tr -s ' ' | cut -d" " -f4`
echo -ne "$left bytes left\r";
if [[ $left -lt $STOPAT ]]; then
MSG="pausing process...$PID";
echo $MSG;
if [[ -x $OSASCRIPT ]]; then
$OSASCRIPT -e "display notification \"$MSG\""
fi
kill -TSTP $PID
break
fi
sleep 1s
done
Sun OS 5.8
Bash shell script
Oracle 10g database
Error 1 the command executing at the time of the error was egrep ORA-\|TNS-\|PLS-\|Error\|PLW-\|IMP-\|EXP-\|RMAN-\|SQL- alert_work.log > alert.err on line 11
The "egrep" line runs successfully when I run it manually. But in a bash script (cron job) it gets the above error. Here is the script:
#!/bin/bash
SID=$ORACLE_SID
DOMAIN=$(uname -n)
DBALIST='dbak#xxx.com'
YESTERDAY=`TZ=CST+24 date +%Y-%m-%d`
cd $ORACLE_HOME/admin/$SID/bdump
mv alert_${SID}.log alert_work.log
touch alert_${SID}.log
cat alert_work.log >> alert_${SID}.hist
egrep ORA-\|TNS-\|PLS-\|Error\|PLW-\|IMP-\|EXP-\|RMAN-\|SQL- alert_work.log > alert.err
if [ `cat alert.err|wc -l` -gt 0 ]
then
mailx -s "${DOMAIN}.${SID} ALERT LOG ERRORS FOUND" $DBALIST < alert.err.log
fi
/usr/bin/mv alert_work.log $ORACLE_HOME/admin/$SID/bdump/hist/alert_${SID}_${YESTERDAY}.log
exit
I am suspicious of your egrep regular expression, The fact that you have not quoted it, and create a script from within a Bash script leads and then run the script, leads me to think that you will end up with:
egrep ORA-|TNS-|PLS-|Error|PLW-|IMP-|EXP-|RMAN-|SQL- alert_work.log > alert.err
which is not what you intended. Try:
egrep 'ORA-\|TNS-\|PLS-\|Error\|PLW-\|IMP-\|EXP-\|RMAN-\|SQL-' alert_work.log > alert.err
That should preserve the back slashes.
I have a script which has to move uploaded files from the first directory to the second directory.
The problem is that the script already moves the files during upload.
Anyone who can help?
#!/bin/sh
lockfile=/home/mediaze111/cronjobs/zenon_move.lock
if ( set -o noclobber; echo "$$" > "$lockfile") 2> /dev/null; then
trap 'rm -f "$lockfile"; exit $?' INT TERM EXIT
if [ "$(ls -A /home/mediaze111/domains/optimaal.fm/ZenonImport/Import1/)" ]; then
ps faux | grep -E 'UPLOAD' | grep -v 'grep' > /dev/null || mv -f /home/mediaze111/domains/optimaal.fm/ZenonImport/Import1/*.* /home/mediaze111/domains/optimaal.fm/ZenonImport/Import2/
fi
rm -f "$lockfile"
trap - INT TERM EXIT
fi
u can get the file size before upload . then check the destination file size . if same move uploaded. pul all the script in a loop so when was the same break it an move files.