I am trying to create docker image that will execute cronjobs from root and custom made user. So far only root user is working:
FROM amazonlinux:2017.09
RUN yum -y install ca-certificates shadow-utils cronie && yum -y clean all
# root cronjob
RUN echo '* * * * * echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2' > ~/cronjob
RUN chmod 0644 ~/cronjob
RUN crontab ~/cronjob
RUN useradd -ms /bin/bash ansible
# Ansible cronjob
USER ansible
RUN echo '* * * * * echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2' > ~/cronjob
RUN chmod 0644 ~/cronjob
RUN crontab ~/cronjob
USER root
CMD ["/usr/sbin/crond", "-n"]
I build this docker using this command docker build -t demotest -f Dockerfile .
I execute the created image using this command docker run -t -i demotest:latest
Result of execution:
> docker run -t -i demotest:latest
root - Working in a coal mine...
root - Working in a coal mine...
root - Working in a coal mine...
root - Working in a coal mine...
Few details:
docker run -t -i demotest:latest bash -c 'ls -l /home/ansible/cronjob'
-rw-r--r-- 1 ansible ansible 81 Jul 11 16:06 /home/ansible/cronjob
docker run -t -i demotest:latest bash -c 'ls -l /root/cronjob'
-rw-r--r-- 1 root root 81 Jul 11 16:06 /root/cronjob
docker run -t -i demotest:latest bash -c 'crontab -u root -l'
* * * * * echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2
docker run -t -i demotest:latest bash -c 'crontab -u ansible -l'
* * * * * echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2
What am I doing wrong?
Updated information
I changed CMD to look like this: CMD ["/usr/sbin/crond", "-n", "-x", "sch"]
This generated the output:
docker run -t -i demotest:latest
debug flags enabled: sch
[1] cron started
log_it: (CRON 1) INFO (RANDOM_DELAY will be scaled with factor 85% if used.)
log_it: (CRON 1) INFO (running with inotify support)
[1] GMToff=0
[1] Target time=1562865060, sec-to-wait=12
user [root:0:0:...] cmd="run-parts /etc/cron.hourly"
user [root:0:0:...] cmd="[ ! -f /etc/cron.hourly/0anacron ] && run-parts /etc/cron.monthly"
user [root:0:0:...] cmd="[ ! -f /etc/cron.hourly/0anacron ] && run-parts /etc/cron.weekly"
user [root:0:0:...] cmd="[ ! -f /etc/cron.hourly/0anacron ] && run-parts /etc/cron.daily"
user [ansible:500:500:...] cmd="echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2"
user [root:0:0:...] cmd="echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2"
Minute-ly job. Recording time 1562865061
[1] Target time=1562865120, sec-to-wait=60
Minute-ly job. Recording time 1562865061
log_it: (root 8) CMD (echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2)
log_it: (ansible 9) CMD (echo "$USER - Working in a coal mine..." > /proc/1/fd/1 2>/proc/1/fd/2)
root - Working in a coal mine...
This shows that the custom user cronjob is executed but output is not redirected.
So the questions changes to - How to properly redirect stdout to host console?
Change last line of your Dockerfile:
from
CMD ["/usr/sbin/crond", "-n", "-x", "sch"]
to
CMD ["cron", "-f"]
Note:
cron -f --> cron foreground
Related
I have setup a cron inside a docker php:8.1.0-fpm-buster. It's running like expected, but there is no log showing up inside the docker desktop, it's a black screen.
Here is the docker file
FROM php:8.1.0-fpm-buster
ARG ENV
RUN apt-get update && apt-get -y install cron
RUN touch /var/log/cron.log
RUN chmod 777 /var/log/cron.log
COPY ./crontab /etc/cron.d/crontab
RUN chmod 0644 /etc/cron.d/crontab
RUN /usr/bin/crontab /etc/cron.d/crontab
CMD [ "cron", "-f", "-L", "2" ]
What I was expecting inside the logs was something similar to linux logs of cron, like this example:
Jan 20 09:32:01 ns555555 CRON[21051]: (root) CMD (echo 'my command')
I tried differents commands:
I added bash -c before the cron command
I remove the -L 2
I have also found other stackoverflow posts, but eachtime it's not the same cron:
See cron output via docker logs, without using an extra file
Docker - Using PHP Cli base image for Cron container
What am I doing wrong ? Did I install the wrong cron ?
I found out the solution inside this post:
How to run a cron job inside a docker container?
I added > /proc/1/fd/1 2>/proc/1/fd/2 after the command, now I have the command output inside the logs
* * * * * root echo hello > /proc/1/fd/1 2>/proc/1/fd/2
Just recently I stumbled on an SSH issue that I cannot figure out what is missing. We use GitLab CI to build and deploy the project to one of our remote servers. As a part of the upgrade plan, we need to replace the degrading Debian 6 server with a new RHEL 7 server. I cannot get the passwordless SSH to work right from GitLab Runner to a remote machine.
I created a reproducible example in a Dockerfile, the IP of the remote server and the user is replaced with non-sensitive data.
FROM centos:7
RUN yum install -y epel-release
RUN yum update -y
RUN yum install -y openssh-clients
RUN useradd -m joe
RUN mkdir -p /home/joe/.ssh
COPY id_rsa_shared /home/joe/.ssh/id_rsa
RUN echo "Host *\n\tStrictHostKeyChecking no\n" >> /home/joe/.ssh/config
RUN ssh-keyscan 10.x.x.x >> /home/joe/.ssh/known_hosts
RUN chown -R joe:joe /home/joe/.ssh
USER joe
CMD ["/bin/bash"]
The file id_rsa_shared is created on local machine with the following command:
ssh-keygen -t rsa -b 2048 -f ./id_rsa_shared
ssh-copy-id -i ./id_rsa_shared joe#10.x.x.x
This works on local. A simple ssh joe#10.x.x.x uname -a in the docker container will output the following:
Linux newweb01p.company.local 3.10.0-1160.25.1.el7.x86_64 #1 SMP Tue Apr 13 18:55:45 EDT 2021 x86_64 x86_64 x86_64 GNU/Linux
However, if I commit this to a branch as GitLab CI script, as shown:
image: centos:7
stages:
- deploy
dev-www:
stage: deploy
tags:
- docker
environment:
name: dev-www
url: http://dev-www.company.local
variables:
DEV_HOST: 10.x.x.x
APP_ENV: dev
DEV_USER: joe
script:
- whoami
- yum install -y epel-release
- yum update -y
- yum install -y openssh-clients
- useradd -m joe
- mkdir -p /home/joe/.ssh
- cp "./gitlab/known_hosts" /home/joe/.ssh/known_hosts
- echo "$DEV_USER_OPENSSH_KEY" >> /home/joe/.ssh/id_rsa
- echo "Host *\n\tStrictHostKeyChecking no\n" >> /home/joe/.ssh/config
- chown -R joe:joe /home/joe/.ssh/
- chmod 600 /home/joe/.ssh/*
- chmod 700 /home/joe/.ssh
- ls -Fsal /home/joe/.ssh
- su - joe
- ssh -oStrictHostKeyChecking=no "${DEV_USER}#${DEV_HOST}" uname -a
when: manual
The pipeline will fail authentication as shown:
Running with gitlab-runner 13.12.0 (7a6612da)
on docker.hqgitrunner01d.company.local K47w1s77
Preparing the "docker" executor
Using Docker executor with image centos:7 ...
Authenticating with credentials from job payload (GitLab Registry)
Pulling docker image centos:7 ...
Using docker image sha256:xxx for centos:7 with digest centos:7#sha256:xxxx ...
Preparing environment
Running on runner-k47w1s77-project-93-concurrent-0 via hqgitrunner01d.company.local...
Getting source from Git repository
Fetching changes...
Reinitialized existing Git repository in /builds/webversion3/API/.git/
Checking out 6a7c193b as tdr/psr4-composer...
Updating/initializing submodules recursively...
Executing "step_script" stage of the job script
Using docker image sha256:xxx for centos:7 with digest centos:7#sha256:xxx ...
$ whoami
root
$ useradd -m joe
$ mkdir -p /home/joe/.ssh
$ cp "./gitlab/known_hosts" /home/joe/.ssh/known_hosts
$ echo "$DEV_USER_OPENSSH_KEY" >> /home/joe/.ssh/id_rsa
$ echo "Host *\n\tStrictHostKeyChecking no\n" >> /home/joe/.ssh/config
$ chown -R joe:joe /home/joe/.ssh/*
$ chmod 600 /home/joe/.ssh/*
$ chmod 700 /home/joe/.ssh
$ ls -Fsal /home/joe/.ssh
total 16
0 drwx------ 2 root root 53 Apr 1 15:19 ./
0 drwx------ 3 joe joe 74 Apr 1 15:19 ../
4 -rw------- 1 joe joe 37 Apr 1 15:19 config
4 -rw------- 1 joe joe 3414 Apr 1 15:19 id_rsa
8 -rw------- 1 joe joe 6241 Apr 1 15:19 known_hosts
$ su - joe
$ ssh -oStrictHostKeyChecking=no "${DEV_USER}#${DEV_HOST}" uname -a
Warning: Permanently added '10.x.x.x' (ECDSA) to the list of known hosts.
Permission denied, please try again.
Permission denied, please try again.
Permission denied (publickey,gssapi-keyex,gssapi-with-mic,password).
Cleaning up file based variables
ERROR: Job failed: exit code 1
Maybe there’s a step I missed because I get a ‘Permission denied, please try again’ message. How do I get Docker Executor to use passwordless SSH to a remote server?
The solution was really simple, and straightforward. The important part is understanding SSH.
The solution works. A snippet from the .gitlab-ci.yml for those who has the same problem as I do.
...
- mkdir -p ~/.ssh
- touch ~/.ssh/id_rsa ~/.ssh/config ~/.ssh/known_hosts
- chmod 600 ~/.ssh/id_rsa ~/.ssh/config ~/.ssh/known_hosts
- echo "$OPENSSH_KEY" >> ~/.ssh/id_rsa
- echo "Host *\n\tStrictHostKeyChecking no" >> ~/.ssh/config
- ssh-keyscan ${DEV_HOST} >> ~/.ssh/known_hosts
Just inline all your ssh options. Use -i to specify your key file. You can also use -o UserKnownHostsFile to specify your known hosts file -- you don't need to copy all that it into an ssh configuration.
This should be enough to ssh successfully:
# ...
- echo "$DEV_USER_OPENSSH_KEY" > "${CI_PROJECT_DIR}/id_rsa.key"
- chmod 600 "${CI_PROJECT_DIR}/id_rsa.key"
- |
ssh -i "${CI_PROJECT_DIR}/id_rsa.key" \
-o IdentitiesOnly=yes \
-o UserKnownHostsFile="${CI_PROJECT_DIR}/gitlab/known_hosts" \
-o StrictHostKeyChecking=no \
user#host ...
Also, since you're disabling StrictHostKeyChecking, you can also just use /dev/null for your UserKnownHostsFile. If you want key checking, omit the StrictHostKeyChecking=no option.
I'm fairly new to Docker so please bear with me:
I have a collection of 5 java applications that I'm trying to run inside a docker container (via cronjobs inside the container).
I've created my Dockerfile and a docker-compose file as well. I'm able to launch my docker container and I'm able to manually run my applications inside the container. The problem is, it doesn't seem to be running my stuff via the cron. To test, I added a cron entry to echo hello world to a file every minute. It doesn't seem to run. Here are my files:
Dockerfile
FROM ubuntu:latest
ENV LANG C.UTF-8
ENV TZ=EST
ENV DEBIAN_FRONTEND=noninteractive
RUN mkdir -p /wrds/ingesters/logs
WORKDIR /wrds/ingesters
RUN apt-get update && apt-get install -y \
tzdata \
apt-utils \
cron \
default-jdk \
bash \
postgresql \
bsd-mailx \
postfix \
vim
COPY *.jar /wrds/ingesters/
COPY run* /wrds/ingesters/
COPY *.properties /wrds/ingesters/
ADD ingester_cron /etc/cron.d/root
RUN chmod 0644 /etc/cron.d/root
RUN touch /var/log/cron.log
RUN crontab /etc/cron.d/root
RUN mkfifo /var/spool/postfix/public/pickup
RUN echo 'alias ll="ls -la"' >> ~/.bashrc
RUN echo 'stty erase ^H' >> ~/.bashrc
CMD cron -L15 && tail -f /var/log/cron.log
docker-compose.yml (anything in all caps is edited out by me for security issues)
version: '3.5'
services:
wrds_hylo_db_ingesters:
image: HOSTNAME/wrds/CONTAINERNAME
container_name: CONTAINERNAME
restart: always
env_file: env.dev-v2
volumes:
- /ingester_logs:/wrds/ingesters/logs
pid: host
ingester_cron
0 17 * * 1 root /wrds/ingesters/run_script1
0 20 * * 1 root /wrds/ingesters/run_script2
0 23 * * 1 root /wrds/ingesters/run_script3
0 2 * * 2 root /wrds/ingesters/run_script4
* * * * * root echo "Hello World" >> /var/log/cron.log 2>&1
When I run my docker container and ssh in, I get the following:
root#1b0d719feada:/var/log# service cron status
* cron is running
crontab -l
0 17 * * 1 root /wrds/ingesters/run_script1
0 20 * * 1 root /wrds/ingesters/run_script2
0 23 * * 1 root /wrds/ingesters/run_script3
0 2 * * 2 root /wrds/ingesters/run_script4
* * * * * root echo "Hello World" >> /var/log/cron.log 2>&1
Any help would be appreciated!
Maybe you take a look at the logs. Maybe you can see more of what your crons are doing.
Cron: */3 * * * * YOUR-CRON_BLABLA > /dev/stdout logs to the container output
ex.: docker-compose logs -f
I load my cron jobs like this:
COPY ./MY-CRON-FILE.cron /
RUN crontab /tmp/MY-CRON-FILE.cron
...
Depending on the application also sometimes with a certain user:
RUN crontab -u MY-USER /MY-CRON-FILE.cron
I am trying to create a Docker container with a custom D-Bus bus running inside.
I configured my Dockerfile as follow:
FROM ubuntu:16.04
COPY myCustomDbus.conf /etc/dbus-1/
RUN apt-get update && apt-get install -y dbus
RUN dbus-daemon --config-file=/etc/dbus-1/myCustomDbus.conf
After building, the socket is created but it is flagged as "file", not as "socket", and I can not use it as a bus...
-rwxrwxrwx 1 root root 0 Mar 20 07:25 myCustomDbus.sock
If I remove this file and run the dbus-daemon command again in a terminal, the socket is successfully created :
srwxrwxrwx 1 root root 0 Mar 20 07:35 myCustomDbus.sock
I am not sure if it is a D-Bus problem or a docker one.
Instead of using the "RUN" command, you should use the "ENTRYPOINT" one to run a startup script.
The Dockerfile should look like that :
FROM ubuntu:14.04
COPY myCustomDbus.conf /etc/dbus-1/
COPY run.sh /etc/init/
RUN apt-get update && apt-get install -y dbus
ENTRYPOINT ["/etc/init/run.sh"]
And run.sh :
#!/bin/bash
dbus-daemon --config-file=/etc/dbus-1/myCustomDbus.conf --print-address
You should use a startup script. The "run" command is executed only when the container is created and then stopped.
my run.sh:
if ! pgrep -x "dbus-daemon" > /dev/null
then
# export DBUS_SESSION_BUS_ADDRESS=$(dbus-daemon --config-file=/usr/share/dbus-1/system.conf --print-address | cut -d, -f1)
# or:
dbus-daemon --config-file=/usr/share/dbus-1/system.conf
# and put in Dockerfile:
# ENV DBUS_SESSION_BUS_ADDRESS="unix:path=/var/run/dbus/system_bus_socket"
else
echo "dbus-daemon already running"
fi
if ! pgrep -x "/usr/lib/upower/upowerd" > /dev/null
then
/usr/lib/upower/upowerd &
else
echo "upowerd already running"
fi
then chrome runs with
--use-gl=swiftshader
without errors
I have this simple Dockerfile for testing, but this is also same in my LEMP stack in a PHP image: cron jobs simply not being executed in Docker.
This is my testing Dockerfile:
FROM debian:latest
MAINTAINER XY <info#domain.com>
LABEL Description="Cron" Vendor="Istvan Lantos" Version="1.0"
RUN apt-get -y update && apt-get -y dist-upgrade \
&& apt-get -y install \
cron \
rsyslog \
vim
RUN rm -rf /var/lib/apt/lists/*
#cron fixes
RUN touch /etc/crontab /etc/cron.d/* /var/spool/cron/crontabs/*
#COPY etc/cron.d /etc/cron.d
COPY etc/crontab /etc/crontab
#COPY var/spool/cron/crontabs /var/spool/cron/crontabs
RUN chmod 600 /etc/crontab /etc/cron.d/* /var/spool/cron/crontabs/*
RUN touch /etc/crontab /etc/cron.d/* /var/spool/cron/crontabs/*
RUN rm -rf /var/lib/apt/lists/*
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
CMD ["/docker-entrypoint.sh"]
docker-entrypoint.sh:
#!/bin/bash
set -e
echo PID1 > /dev/null
/etc/init.d/rsyslog start
#Stay in foreground mode, don’t daemonize.
/usr/sbin/cron -f
And this is the Crontab file. I also placed one liners in /etc/cron.d or /var/spool/cron/crontabs with the name of the user, but the effect was the same just like if I modified this base crontab file: cron jobs not will be executed:
MAILTO=""
# /etc/crontab: system-wide crontab
# Unlike any other crontab you don't have to run the `crontab'
# command to install the new version when you edit this file
# and files in /etc/cron.d. These files also have username fields,
# that none of the other crontabs do.
SHELL=/bin/sh
#PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin:/usr/local/php7/bin:/usr/local/php7/sbin
# m h dom mon dow user command
#17 * * * * root cd / && run-parts --report /etc/cron.hourly
#25 6 * * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.daily )
#47 6 * * 7 root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.weekly )
#52 6 1 * * root test -x /usr/sbin/anacron || ( cd / && run-parts --report /etc/cron.monthly )
#
*/1 * * * * root date >> /var/log/cron-test.log 2>&1
This is the output of the /var/log/syslog file:
Jan 23 09:38:39 1ab854e8d9a7 rsyslogd: [origin software="rsyslogd" swVersion="8.4.2" x-pid="14" x-info="http://www.rsyslog.com"] start
Jan 23 09:38:39 1ab854e8d9a7 rsyslogd: imklog: cannot open kernel log(/proc/kmsg): Operation not permitted.
Jan 23 09:38:39 1ab854e8d9a7 rsyslogd-2145: activation of module imklog failed [try http://www.rsyslog.com/e/2145 ]
Jan 23 09:38:39 1ab854e8d9a7 cron[19]: (CRON) INFO (pidfile fd = 3)
Jan 23 09:38:39 1ab854e8d9a7 cron[19]: (*system*) NUMBER OF HARD LINKS > 1 (/etc/crontab)
Jan 23 09:38:39 1ab854e8d9a7 cron[19]: (*) ORPHAN (no passwd entry)
Jan 23 09:38:39 1ab854e8d9a7 cron[19]: (CRON) INFO (Running #reboot jobs)
/var/log/cron-test.log won't be created by the cron job.
I have a question for those who flagged this as "off topic" and SuperUser material, plus this is about general computing HARDWARE AND SOFTWARE: really? Docker questions when become sysadmin stuff? This way every Docker related question here have at least one flag. I'm not against promoting more users to the less known child sites, but we have more change to get the answer here than their.
UPDATE:
This is what I come up with until cron jobs not working:
End of Dockerfile:
COPY cron-jobs.sh /
RUN chmod +x /cron-jobs.sh
COPY docker-entrypoint.sh /
RUN chmod +x /docker-entrypoint.sh
CMD ["/docker-entrypoint.sh"]
docker-entrypoint.sh:
#!/bin/bash
set -e
echo PID1 > /dev/null
# Run script in the background (this is not daemonized)
/cron-jobs.sh &
/usr/local/php7/sbin/php-fpm --nodaemonize --fpm-config /usr/local/php7/etc/php-fpm.conf
cron-jobs.sh:
#!/bin/bash
while true; do
date >> /var/log/cron-test.log 2>&1
sleep 60
done
Cron (at least in Debian) does not execute crontabs with more than 1 hardlink, see bug 647193. As Docker uses overlays, it results with more than one link to the file, so you have to touch it in your startup script, so the link is severed:
touch /etc/crontab /etc/cron.*/*