I have this simple Dockerfile for testing, but this is also same in my LEMP stack in a PHP image: cron jobs simply not being executed in Docker.
This is my testing Dockerfile:
FROM debian:latest
MAINTAINER XY <info#domain.com>
LABEL Description="Cron" Vendor="Istvan Lantos" Version="1.0"
RUN apt-get -y update && apt-get -y dist-upgrade \
&& apt-get -y install \
cron \
rsyslog \
vim
RUN rm -rf /var/lib/apt/lists/*
#cron fixes
RUN touch /etc/crontab /etc/cron.d/* /var/spool/cron/crontabs/*
#COPY etc/cron.d /etc/cron.d
COPY etc/crontab /etc/crontab
#COPY var/spool/cron/crontabs /var/spool/cron/crontabs
RUN chmod 600 /etc/crontab /etc/cron.d/* /var/spool/cron/crontabs/*
RUN touch /etc/crontab /etc/cron.d/* /var/spool/cron/crontabs/*
RUN rm -rf /var/lib/apt/lists/*
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
CMD ["/docker-entrypoint.sh"]
docker-entrypoint.sh:
#!/bin/bash
set -e
echo PID1 > /dev/null
/etc/init.d/rsyslog start
#Stay in foreground mode, don’t daemonize.
/usr/sbin/cron -f
And this is the Crontab file. I also placed one liners in /etc/cron.d or /var/spool/cron/crontabs with the name of the user, but the effect was the same just like if I modified this base crontab file: cron jobs not will be executed:
MAILTO=""
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
SHELL=/bin/sh
#PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/php7/bin:/usr/local/php7/sbin
# m h dom mon dow user command
#17 * * * * root cd / && run-parts --report /etc/cron.hourly
#25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
#47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
#52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#
*/1 * * * * root date >> /var/log/cron-test.log 2>&1
This is the output of the /var/log/syslog file:
Jan 23 09:38:39 1ab854e8d9a7 rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="14" x-info="http://www.rsyslog.com"] start
Jan 23 09:38:39 1ab854e8d9a7 rsyslogd: imklog: cannot open kernel log(/proc/kmsg): Operation not permitted.
Jan 23 09:38:39 1ab854e8d9a7 rsyslogd-2145: activation of module imklog failed [try http://www.rsyslog.com/e/2145 ]
Jan 23 09:38:39 1ab854e8d9a7 cron[19]: (CRON) INFO (pidfile fd = 3)
Jan 23 09:38:39 1ab854e8d9a7 cron[19]: (*system*) NUMBER OF HARD LINKS > 1 (/etc/crontab)
Jan 23 09:38:39 1ab854e8d9a7 cron[19]: (*) ORPHAN (no passwd entry)
Jan 23 09:38:39 1ab854e8d9a7 cron[19]: (CRON) INFO (Running #reboot jobs)
/var/log/cron-test.log won't be created by the cron job.
I have a question for those who flagged this as "off topic" and SuperUser material, plus this is about general computing HARDWARE AND SOFTWARE: really? Docker questions when become sysadmin stuff? This way every Docker related question here have at least one flag. I'm not against promoting more users to the less known child sites, but we have more change to get the answer here than their.
UPDATE:
This is what I come up with until cron jobs not working:
End of Dockerfile:
COPY cron-jobs.sh /
RUN chmod +x /cron-jobs.sh
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
CMD ["/docker-entrypoint.sh"]
docker-entrypoint.sh:
#!/bin/bash
set -e
echo PID1 > /dev/null
# Run script in the background (this is not daemonized)
/cron-jobs.sh &
/usr/local/php7/sbin/php-fpm --nodaemonize --fpm-config /usr/local/php7/etc/php-fpm.conf
cron-jobs.sh:
#!/bin/bash
while true; do
date >> /var/log/cron-test.log 2>&1
sleep 60
done
Cron (at least in Debian) does not execute crontabs with more than 1 hardlink, see bug 647193. As Docker uses overlays, it results with more than one link to the file, so you have to touch it in your startup script, so the link is severed:
touch /etc/crontab /etc/cron.*/*
Related
So I am pretty new into creating containers and I have the simple Dockerfile where I would like to run a simple python script every minute:
FROM python:3.8-buster
RUN apt-get update && apt-get install -y cron
COPY my_python /bin/my_python
COPY root /var/spool/cron/crontabs/root
RUN chmod +x /bin/my_python
CMD cron -l 2 -f
where my_python:
print("hi world!!")
and root:
* * * * * python3 /bin/my_python
then I just create the image and the container:
docker image build -t python-test
docker container run -it --name python-test python-test
I was supposed to see every minute a print with the hi world, however when running the container ( after the image build) no logs seem to appear.
What am i doing wrong?
First, I believe you want -L 2 rather than -l 2 in your cron command line; see the man page for details.
The cron daemon logs to syslog, so if something isn't work as intended, it's a good idea to arrange to receive those messages. The busybox tool provides a simple syslog daemon that can log to an in-memory buffer and a tool for reading those logs, so I modified your Dockerfile to look like this:
FROM python:3.8-buster
RUN apt-get update && apt-get install -y cron busybox
COPY my_python /bin/my_python
COPY root /var/spool/cron/crontabs/root
RUN chmod +x /bin/my_python
CMD busybox syslogd -C; cron -L 2 -f
After starting this, I docker exec'd into the container and ran busybox logread and found:
Jan 24 16:50:45 7f516db86417 cron.info cron[4]: (CRON) INFO (pidfile fd = 3)
Jan 24 16:50:45 7f516db86417 cron.info cron[4]: (root) INSECURE MODE (mode 0600 expected) (crontabs/root)
Jan 24 16:50:45 7f516db86417 cron.info cron[4]: (CRON) INFO (Running #reboot jobs)
So there's your problem: the permissions on the root crontab are incorrect. There are two ways to fix this problem:
We could explicitly chmod the file when we copy it into place, or
We can use the crontab command to install the file, which takes care of that for us
I like option 2 because it means we don't need to know the specifics of what cron expects in terms of permissions. That gets us:
FROM python:3.8-buster
RUN apt-get update && apt-get install -y cron busybox
COPY my_python /bin/my_python
COPY root /tmp/root.crontab
RUN crontab /tmp/root.crontab
RUN chmod +x /bin/my_python
CMD busybox syslogd -C; cron -L 2 -f
With that change, we can confirm that the cron job is now running as expected:
Jan 24 16:59:50 8aa688ad31cc syslog.info syslogd started: BusyBox v1.30.1
Jan 24 16:59:50 8aa688ad31cc cron.info cron[4]: (CRON) INFO (pidfile fd = 3)
Jan 24 16:59:50 8aa688ad31cc cron.info cron[4]: (CRON) INFO (Running #reboot jobs)
Jan 24 17:00:01 8aa688ad31cc authpriv.err CRON[7]: pam_env(cron:session): Unable to open env file: /etc/default/locale: No such file or directory
Jan 24 17:00:01 8aa688ad31cc authpriv.info CRON[7]: pam_unix(cron:session): session opened for user root by (uid=0)
Jan 24 17:00:02 8aa688ad31cc cron.info CRON[7]: (root) END (python3 /bin/my_python)
Jan 24 17:00:02 8aa688ad31cc authpriv.info CRON[7]: pam_unix(cron:session): session closed for user root
But...there's still no output from the container! If you read through that man page, you'll find this:
cron then wakes up every minute, examining all stored crontabs,
checking each command to see if it should be run in the current
minute. When executing commands, any output is mailed to the owner of
the crontab (or to the user named in the MAILTO environment
variable in the crontab, if such exists)...
In other words, cron collects the output from programs and attempts
to mail to the user who owns the cron job. If you want to see the
output from the cron job on the console, you will need to explicitly
redirect stdout, like this:
* * * * * python3 /bin/my_python > /dev/console
With this change in place, running the image results in the message...
hi world!
...printing to the console once a minute.
I'm fairly new to Docker so please bear with me:
I have a collection of 5 java applications that I'm trying to run inside a docker container (via cronjobs inside the container).
I've created my Dockerfile and a docker-compose file as well. I'm able to launch my docker container and I'm able to manually run my applications inside the container. The problem is, it doesn't seem to be running my stuff via the cron. To test, I added a cron entry to echo hello world to a file every minute. It doesn't seem to run. Here are my files:
Dockerfile
FROM ubuntu:latest
ENV LANG C.UTF-8
ENV TZ=EST
ENV DEBIAN_FRONTEND=noninteractive
RUN mkdir -p /wrds/ingesters/logs
WORKDIR /wrds/ingesters
RUN apt-get update && apt-get install -y \
tzdata \
apt-utils \
cron \
default-jdk \
bash \
postgresql \
bsd-mailx \
postfix \
vim
COPY *.jar /wrds/ingesters/
COPY run* /wrds/ingesters/
COPY *.properties /wrds/ingesters/
ADD ingester_cron /etc/cron.d/root
RUN chmod 0644 /etc/cron.d/root
RUN touch /var/log/cron.log
RUN crontab /etc/cron.d/root
RUN mkfifo /var/spool/postfix/public/pickup
RUN echo 'alias ll="ls -la"' >> ~/.bashrc
RUN echo 'stty erase ^H' >> ~/.bashrc
CMD cron -L15 && tail -f /var/log/cron.log
docker-compose.yml (anything in all caps is edited out by me for security issues)
version: '3.5'
services:
wrds_hylo_db_ingesters:
image: HOSTNAME/wrds/CONTAINERNAME
container_name: CONTAINERNAME
restart: always
env_file: env.dev-v2
volumes:
- /ingester_logs:/wrds/ingesters/logs
pid: host
ingester_cron
0 17 * * 1 root /wrds/ingesters/run_script1
0 20 * * 1 root /wrds/ingesters/run_script2
0 23 * * 1 root /wrds/ingesters/run_script3
0 2 * * 2 root /wrds/ingesters/run_script4
* * * * * root echo "Hello World" >> /var/log/cron.log 2>&1
When I run my docker container and ssh in, I get the following:
root#1b0d719feada:/var/log# service cron status
* cron is running
crontab -l
0 17 * * 1 root /wrds/ingesters/run_script1
0 20 * * 1 root /wrds/ingesters/run_script2
0 23 * * 1 root /wrds/ingesters/run_script3
0 2 * * 2 root /wrds/ingesters/run_script4
* * * * * root echo "Hello World" >> /var/log/cron.log 2>&1
Any help would be appreciated!
Maybe you take a look at the logs. Maybe you can see more of what your crons are doing.
Cron: */3 * * * * YOUR-CRON_BLABLA > /dev/stdout logs to the container output
ex.: docker-compose logs -f
I load my cron jobs like this:
COPY ./MY-CRON-FILE.cron /
RUN crontab /tmp/MY-CRON-FILE.cron
...
Depending on the application also sometimes with a certain user:
RUN crontab -u MY-USER /MY-CRON-FILE.cron
I am trying to create docker image that will execute cronjobs from root and custom made user. So far only root user is working:
FROM amazonlinux:2017.09
RUN yum -y install ca-certificates shadow-utils cronie && yum -y clean all
# root cronjob
RUN echo '* * * * * echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2' > ~/cronjob
RUN chmod 0644 ~/cronjob
RUN crontab ~/cronjob
RUN useradd -ms /bin/bash ansible
# Ansible cronjob
USER ansible
RUN echo '* * * * * echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2' > ~/cronjob
RUN chmod 0644 ~/cronjob
RUN crontab ~/cronjob
USER root
CMD ["/usr/sbin/crond", "-n"]
I build this docker using this command docker build -t demotest -f Dockerfile .
I execute the created image using this command docker run -t -i demotest:latest
Result of execution:
> docker run -t -i demotest:latest
root - Working in a coal mine...
root - Working in a coal mine...
root - Working in a coal mine...
root - Working in a coal mine...
Few details:
docker run -t -i demotest:latest bash -c 'ls -l /home/ansible/cronjob'
-rw-r--r-- 1 ansible ansible 81 Jul 11 16:06 /home/ansible/cronjob
docker run -t -i demotest:latest bash -c 'ls -l /root/cronjob'
-rw-r--r-- 1 root root 81 Jul 11 16:06 /root/cronjob
docker run -t -i demotest:latest bash -c 'crontab -u root -l'
* * * * * echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2
docker run -t -i demotest:latest bash -c 'crontab -u ansible -l'
* * * * * echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2
What am I doing wrong?
Updated information
I changed CMD to look like this: CMD ["/usr/sbin/crond", "-n", "-x", "sch"]
This generated the output:
docker run -t -i demotest:latest
debug flags enabled: sch
[1] cron started
log_it: (CRON 1) INFO (RANDOM_DELAY will be scaled with factor 85% if used.)
log_it: (CRON 1) INFO (running with inotify support)
[1] GMToff=0
[1] Target time=1562865060, sec-to-wait=12
user [root:0:0:...] cmd="run-parts /etc/cron.hourly"
user [root:0:0:...] cmd="[ ! -f /etc/cron.hourly/0anacron ] && run-parts /etc/cron.monthly"
user [root:0:0:...] cmd="[ ! -f /etc/cron.hourly/0anacron ] && run-parts /etc/cron.weekly"
user [root:0:0:...] cmd="[ ! -f /etc/cron.hourly/0anacron ] && run-parts /etc/cron.daily"
user [ansible:500:500:...] cmd="echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2"
user [root:0:0:...] cmd="echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2"
Minute-ly job. Recording time 1562865061
[1] Target time=1562865120, sec-to-wait=60
Minute-ly job. Recording time 1562865061
log_it: (root 8) CMD (echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2)
log_it: (ansible 9) CMD (echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2)
root - Working in a coal mine...
This shows that the custom user cronjob is executed but output is not redirected.
So the questions changes to - How to properly redirect stdout to host console?
Change last line of your Dockerfile:
from
CMD ["/usr/sbin/crond", "-n", "-x", "sch"]
to
CMD ["cron", "-f"]
Note:
cron -f --> cron foreground
I am trying to create a Docker container with a custom D-Bus bus running inside.
I configured my Dockerfile as follow:
FROM ubuntu:16.04
COPY myCustomDbus.conf /etc/dbus-1/
RUN apt-get update && apt-get install -y dbus
RUN dbus-daemon --config-file=/etc/dbus-1/myCustomDbus.conf
After building, the socket is created but it is flagged as "file", not as "socket", and I can not use it as a bus...
-rwxrwxrwx 1 root root 0 Mar 20 07:25 myCustomDbus.sock
If I remove this file and run the dbus-daemon command again in a terminal, the socket is successfully created :
srwxrwxrwx 1 root root 0 Mar 20 07:35 myCustomDbus.sock
I am not sure if it is a D-Bus problem or a docker one.
Instead of using the "RUN" command, you should use the "ENTRYPOINT" one to run a startup script.
The Dockerfile should look like that :
FROM ubuntu:14.04
COPY myCustomDbus.conf /etc/dbus-1/
COPY run.sh /etc/init/
RUN apt-get update && apt-get install -y dbus
ENTRYPOINT ["/etc/init/run.sh"]
And run.sh :
#!/bin/bash
dbus-daemon --config-file=/etc/dbus-1/myCustomDbus.conf --print-address
You should use a startup script. The "run" command is executed only when the container is created and then stopped.
my run.sh:
if ! pgrep -x "dbus-daemon" > /dev/null
then
# export DBUS_SESSION_BUS_ADDRESS=$(dbus-daemon --config-file=/usr/share/dbus-1/system.conf --print-address | cut -d, -f1)
# or:
dbus-daemon --config-file=/usr/share/dbus-1/system.conf
# and put in Dockerfile:
# ENV DBUS_SESSION_BUS_ADDRESS="unix:path=/var/run/dbus/system_bus_socket"
else
echo "dbus-daemon already running"
fi
if ! pgrep -x "/usr/lib/upower/upowerd" > /dev/null
then
/usr/lib/upower/upowerd &
else
echo "upowerd already running"
fi
then chrome runs with
--use-gl=swiftshader
without errors
I use supervisor to run cron and nginx, the problem is when i try to COPY or VOLUME mount my cron files, it does not run my cron files in /etc/cron.d
But when I exec -it <container_id> bash into the container and create the exact same cron file from inside, it is immediately recognized and runs as it should.
Dockerfile :
FROM phusion/baseimage:latest
ENV TERM xterm
ENV HOME /root
RUN apt-get update && apt-get install -y \
nginx \
supervisor \
curl \
nano \
net-tools
RUN rm -rf /etc/nginx/*
COPY nginx_conf /etc/nginx
COPY supervisor_conf /etc/supervisor/
RUN mkdir -p /var/log/supervisor
COPY crontabs /etc/cron.d/
RUN chmod -R 644 /etc/cron.d/
CMD /usr/bin/supervisord
The cron itself
* * * * * root curl --silent http://127.0.0.1/cronjob/cron_test_docker.php >> /var/www/html/log/docker_test.log 2>&1
cron and nginx run through supervisor
[supervisord]
nodaemon = true
[program:nginx]
command = /usr/sbin/nginx -g "daemon off;"
autostart = true
[program:cron]
command = /usr/sbin/cron -f
autostart = true
The logs inside /var/log/supervisor/ relating to cron for stdout and stderr are empty.
I also tried stripping out supervisor and running cron on its own through phusion and CMD cron -f but got the same issue of it not working when the source is external(COPY or VOLUME) and magically works when created inside the container.
Initially believed it to be a permissions issue and tried chmod 644 (as this was the permission a file created in the container had) on all files that were the result of COPY into.
RUN chmod 644 /etc/cron.d/
After which tried every possible combination of permissions with rwx to no avail.
Also, tried to append the line of the cronjob into /etc/crontab but it is not recognized in crontab -l.
COPY crontab /tmp/crontab
RUN cat crontab >> /etc/crontab
It would be really handy if it worked just when it was created through COPY or VOLUME as it is a hassle to create it manually in the container everytime.
Any help would be greatly appreciated!
Edit 1 :
Some additional information about the file permissions after COPY or VOLUME.
When I perform
COPY crontabs /etc/cron.d/
RUN chmod -R 644 /etc/cron.d/
Inside the container running ls -l inside /etc/cron.d/ shows
-rw-r--r-- 1 root root 118 Jul 20 11:03 wwwcron-cron-docker_test
When I mount the folder through my docker-compose through VOLUME
volumes:
- ./server/crontabs:/etc/cron.d
ls -l shows
-rwxrwxrwx 1 1000 staff 118 Jul 20 11:03 wwwcron-cron-docker_test
In addition if I manually create the cron file in the container it looks like this and this works
-rw-r--r-- 1 root root 118 Jul 22 15:50 wwwcron-cron-docker_test_inside_docker
Clearly there are very different permissions and ownership when making COPY or VOLUME. But making a COPY with exact permissions does not work but seems to work when created in the container.
Thanks to #BMitch was able to find the issue which was related to line endings since my host machine was windows and the cron file origin was windows as well there was a disparity in the line endings thereby cron did not pick it up automatically.
I added this line to my Dockerfile and it works like a charm
RUN find /etc/cron.d/ -type f -print0 | xargs -0 dos2unix
And iterating on that the size of the file is indeed 1 byte smaller when a dos2unix is run, so you can verify if this operation indeed occurred.
-rw-r--r-- 1 root root 117 Jul 25 08:33 wwwcron-cron-docker_test
Have you tried installing the crontab as a separate command in the Dockerfile?
i.e.
...
COPY crontabs /path/to/crontab.txt
RUN crontab -u myUser /path/to/crontab.txt
...