I need to add environment variable to a crontab task in a Dockerfile. This is so i can pass the value as a runtime value which Docker can pick up.
But when i start a container from the image and pass environment variables, it is not picked up in the container.
How do i fix it.
Example below:
RUN yum install -y cronie
RUN pip install --upgrade pip
ADD dist/*.whl /opt/
RUN pip install /opt/*.whl
WORKDIR /opt/setup
VOLUME /var/docker-share
# Add task to crontab
RUN echo '0 * * * * run.sh --var1 $VAR1 --var2 $VAR2' | crontab -
# Run the command on container startup
CMD crond && while true; do sleep 3600; done
Related
I want to preface this in saying that I am very new to docker and have just got my feet wet with using it. In my Docker file that I run to build the container I install a program that sets some env variables. Here is my Docker file for context.
FROM python:3.8-slim-buster
COPY . /app
RUN apt-get update
RUN apt-get install wget -y
RUN wget http://static.matrix-vision.com/mvIMPACT_Acquire/2.40.0/install_mvGenTL_Acquire.sh
RUN wget http://static.matrix-vision.com/mvIMPACT_Acquire/2.40.0/mvGenTL_Acquire-x86_64_ABI2-2.40.0.tgz
RUN chmod +x ./install_mvGenTL_Acquire.sh
RUN ./install_mvGenTL_Acquire.sh -u
RUN apt-get install -y python3-opencv
RUN pip3 install USSCameraTools
WORKDIR /app
CMD python3 main.py
After executing the build docker command the program "mvGenTL_Acquire.sh" sets env inside the container. I need these variables to be set when executing the run docker command. But when checking the env variables after running the image it is not set. I know I can pass them in directly but would like to use the ones that are set from the install in the build.
Any help would be greatly appreciated, thanks!
For running a bash script when your container is creating:
make an script.sh file:
#!/bin/bash
your commands here
If you are using an alpine image, you must use #!/bin/sh instead of #!/bin/bash on the first line of your bash file.
Now in your Dockerfile copy your bash file in the container and use the ENTRYPOINT instruction for running this file when the container is creating:
.
.
.
COPY script.sh /
RUN chmod +x /script.sh
.
.
.
ENTRYPOINT ["/script.sh"]
Notice that in the ENTRYPOINT instruction use your bash file address in your image.
Now when you create a container, the script.sh file will be executed.
I've been trying to run a go script with cron under Ubuntu 16.04 Docker image. Here are the files that I've
Dockerfile
FROM couchbase
RUN apt-get update
RUN apt-get install gcc make -y
RUN apt-get install golang-1.10 git -y
ADD src/crontab.txt /crontab.txt
ADD src/backup.sh /backup.sh
ADD src/backup.go /backup.go
ADD src/file.txt /file.txt
COPY entry.sh /entry.sh
RUN chmod 755 /backup.sh /entry.sh
RUN /usr/bin/crontab /crontab.txt
RUN apt-get install vim -y
CMD ["/entry.sh"]
entry.sh
#!/bin/sh
/usr/sbin/cron -f -l 8
src/crontab.txt
* * * * * /backup.sh >> /var/log/backup.log
src/backup.sh
#!/bin/sh
chmod 666 /var/log/backup.log
/usr/lib/go-1.10/bin/go run backup.go
backup.go
package main
import (
"log"
"os"
"strings"
)
func init() {
file, err := os.OpenFile("/var/log/backup.log", os.O_APPEND|os.O_CREATE|os.O_WRONLY, 0666)
if err != nil {
log.Fatal(err)
}
log.SetOutput(file)
}
func main() {
log.Println("Writing log")
}
I checked and the cron task is running each minute. The go installation is there in the folder and when I exec into the container it works, but the backup.go script is not logging anything. When I trigger the script manually it works though. The container that I'm using has Ubuntu 16.04 and I want it cause I don't have to do a couchbase installation.
You can do this simpler by using a multi stage build. First use a Go image to compile a standalone executable from your src/backup.go. Then switch to a coughbase image and copy the executable from the previous step.
Dockerfile:
# use a first-stage image to build the go code
# we'll change it later
FROM golang:1.10 AS build
# for now we only need the go code
COPY src/backup.go backup.go
# build a standalone executable
RUN go build -o /backup backup.go
# switch to a second-stage production image
FROM couchbase
# setup cronjob
COPY src/crontab.txt /crontab.txt
RUN /usr/bin/crontab /crontab.txt
# copy the executable from the first stage
# into the production image
COPY --from=build /backup /backup
CMD ["/usr/sbin/cron", "-f", "-l", "8"]
src/crontab.txt:
* * * * * /backup >> /var/log/backup.log
Build and run like this:
docker build . -t backup
# start in backgroud
docker run --name backup -d test
# check if it works
docker exec backup tail -f /var/log/backup.log
On the next minute :
2021/04/09 19:05:01 Writing log
I have a docker file like this and I have to pass the arguments to docker run command dynamically
FROM ubuntu:14.04
ENV IRONHIDE_SOURCE /var/tmp/ironhide-setup
RUN apt-get update && apt-get install -y openssh-server supervisor cron syslog-ng-core logrotate libapr1 libaprutil1 liblog4cxx10 libxml2 psmisc xsltproc ntp
RUN sed -i -E 's/^(\s*)system\(\);/\1unix-stream("\/dev\/log");/' /etc/syslog-ng/syslog-ng.conf
ADD ironhide-setup/supervisord.conf /etc/supervisor/conf.d/supervisord.conf
RUN mkdir -p /var/log/supervisor & mkdir -p /opt/ibm/
COPY /ironhide-setup/etc/cron.d/* /etc/cron.d
ADD ironhide-setup $IRONHIDE_SOURCE
ENV JAVA_HOME /usr/java/default
ENV PATH $JAVA_HOME/bin:$PATH
ENV IRONHIDE_ROOT /usr/ironhide
ENV LD_LIBRARY_PATH /usr/ironhide/lib
ENV IH_ROOT /usr/ironhide
ENV IRONHIDE_BACKUP_PATH /var/tmp/ironhide-backup
ENV PATH $IH_ROOT/bin:$PATH
RUN echo 'PS1="[AppConnect-Container#\h \w]: "' >> ~/.bashrc
CMD ["/usr/bin/supervisord"]
and my supervisord.conf is this
[supervisord]
nodaemon=true
[program:cron]
command = cron -f -L 15
priority=1
[program:syslog-ng]
command=/usr/sbin/syslog-ng -F -p /var/run/syslog-ng.pid --no-caps
[program:InstallCastIron]
command = %(ENV_IRONHIDE_SOURCE)s/scripts/var_setup
priority=2
I have to pass the arguments to "docker run" command so internally one of the script under scripts location should be using the argument when the docker container comes up.
Please let me know how can I do this and how to achieve" this
To achieve this feat, you will need to use environment variables.
First you will need to make sure that the service you want to pass arguments to consumes those environment variables.
Second you will need to have those variables defined in your dockerfile. For example:-
Third make sure you use entrypoint script For example:-
Last you can use the docker run -e DEFINE_THOSE_VARS=<value>. Or alternatively you can use docker-compose like this
You can traverse through my repo here which achieves this feat.
Please feel free to ask any Question.
Cheers!
I am in the process of creating a docker container which has a miniconda environment setup with some packages (pip and conda). Dockerfile :
# Use an official Miniconda runtime as a parent image
FROM continuumio/miniconda3
# Create the conda environment.
# RUN conda create -n dev_env Python=3.6
RUN conda update conda -y \
&& conda create -y -n dev_env Python=3.6 pip
ENV PATH /opt/conda/envs/dev_env/bin:$PATH
RUN /bin/bash -c "source activate dev_env" \
&& pip install azure-cli \
&& conda install -y nb_conda
The behavior I want is that when the container is launched, it should automatically switch to the "dev_env" conda environment but I haven't been able to get this to work. Logs :
dparkar#mymachine:~/src/dev/setupsdk$ docker build .
Sending build context to Docker daemon 2.56kB
Step 1/4 : FROM continuumio/miniconda3
---> 1284db959d5d
Step 2/4 : RUN conda update conda -y && conda create -y -n dev_env Python=3.6 pip
---> Using cache
---> cb2313f4d8a8
Step 3/4 : ENV PATH /opt/conda/envs/dev_env/bin:$PATH
---> Using cache
---> 320d4fd2b964
Step 4/4 : RUN /bin/bash -c "source activate dev_env" && pip install azure-cli && conda install -y nb_conda
---> Using cache
---> 3c0299dfbe57
Successfully built 3c0299dfbe57
dparkar#mymachine:~/src/dev/setupsdk$ docker run -it 3c0299dfbe57
(base) root#3db861098892:/# source activate dev_env
(dev_env) root#3db861098892:/# exit
exit
dparkar#mymachine:~/src/dev/setupsdk$ docker run -it 3c0299dfbe57 source activate dev_env
[FATAL tini (7)] exec source failed: No such file or directory
dparkar#mymachine:~/src/dev/setupsdk$ docker run -it 3c0299dfbe57 /bin/bash source activate dev_env
/bin/bash: source: No such file or directory
dparkar#mymachine:~/src/dev/setupsdk$ docker run -it 3c0299dfbe57 /bin/bash "source activate dev_env"
/bin/bash: source activate dev_env: No such file or directory
dparkar#mymachine:~/src/dev/setupsdk$ docker run -it 3c0299dfbe57 /bin/bash -c "source activate dev_env"
dparkar#mymachine:~/src/dev/setupsdk$
As you can see above, when I am within the container, I can successfully run "source activate dev_env" and the environment switches over. But I want this to happen automatically when the container is launched.
This also happens in the Dockerfile during build time. Again, I am not sure if that has any effect either.
You should use the command CMD for anything related to runtime.
Anything typed after RUN will only be run at image creation time, not when you actually run the container.
The shell used to run such commands is closed at the end of the image creation process, making the environment activation non-persistent in that case.
As such, your additional line might look like this:
CMD ["conda activate <your-env-name> && <other commands>"]
where <other commands> are other commands you might need at runtime after the environment activation.
This docker build file worked for me.
# start with miniconda image
FROM continuumio/miniconda3
# setting the working directory
WORKDIR /usr/src/app
# Copy the file from your host to your current location in container
COPY . /usr/src/app
# Run the command inside your image filesystem to create an environment and name it in the requirements.yml file, in this case "myenv"
RUN conda env create --file requirements.yml
# Activate the environment named "myenv" with shell command
SHELL ["conda", "run", "-n", "myenv", "/bin/bash", "-c"]
# Make sure the environment is activated by testing if you can import flask or any other package you have in your requirements.yml file
RUN echo "Make sure flask is installed:"
RUN python -c "import flask"
# exposing port 8050 for interaction with local host
EXPOSE 8050
#Run your application in the new "myenv" environment
CMD ["conda", "run", "-n", "myenv", "python", "app.py"]
To preface, I have been referencing these two articles for help:
Run a cron job with Docker - Julien Boulay
Running cron jobs inside a Docker container - Chris S.
My goal is to have a cron job automatically start when I start my docker container. Currently, it doesn't automatically start, but I can manually go into my container and run service cron start, which starts the job, and it works correctly.
So the problem is: How do I get my cron job to start automatically when my container starts up?
Dockerfile
FROM microsoft/dotnet:latest
RUN apt-get update && apt-get install -y cron
COPY . /app
WORKDIR /app
ADD crontab /etc/cron.d/crontab
RUN chmod 0600 /etc/cron.d/crontab
RUN crontab -u root /etc/cron.d/crontab
RUN touch /var/log/cron.log
RUN ["dotnet", "restore"]
RUN ["dotnet", "build"]
EXPOSE 5000/tcp
CMD cron && tail -f /var/log/cron.log
CMD service cron start
crontab
* * * * * echo "Hello world" >> /var/log/cron.log 2>&1
# Empty space
Though I wasn't able to get cron working in that particular container, I was able to create a standalone docker container specifically for cron, and was successful in getting it to run automatically.
As far as setup for the cron container, I followed the linked article, Run a cron job with Docker - Julien Boulay, and was able to get it working.
What I'm doing is have the CMD call cron directly like this:
CMD /usr/sbin/cron -f
Before that I'm adding the crontab to the container and assigning it as the root crontab with the command:
RUN crontab /root/mycrontab
You don't need to call the crontab command on files that are located in /etc/cron.d, but you do need those files to have the correct syntax. Using your example, instead of this:
* * * * * echo "Hello world" >> /var/log/cron.log 2>&1
You should have this:
* * * * * root echo "Hello world" >> /var/log/cron.log 2>&1
On your crontab file. This only applies to crontab files located within /etc/cron.d, otherwise your crontab file syntax is correct and you use the crontab command to load it.
Starting from your example, I think you should modify your files like this:
Dockerfile
FROM microsoft/dotnet:latest
RUN apt-get update && apt-get install -y cron
COPY . /app
WORKDIR /app
ADD crontab /etc/cron.d/crontab
RUN chmod 0600 /etc/cron.d/crontab
RUN touch /var/log/cron.log
RUN ["dotnet", "restore"]
RUN ["dotnet", "build"]
EXPOSE 5000/tcp
CMD /usr/sbin/cron -f
crontab
* * * * * root echo "Hello world" >> /var/log/cron.log 2>&1
Another alternative would be:
Dockerfile
FROM microsoft/dotnet:latest
RUN apt-get update && apt-get install -y cron
COPY . /app
WORKDIR /app
ADD crontab /root/
RUN crontab /root/crontab
RUN touch /var/log/cron.log
RUN ["dotnet", "restore"]
RUN ["dotnet", "build"]
EXPOSE 5000/tcp
CMD /usr/sbin/cron -f
crontab
* * * * * echo "Hello world" >> /var/log/cron.log 2>&1
We had a problem with php-fpm and docker where our cronjob tasks were not be executed. There were two problems we solved:
We tried to copy a crontab file into the docker container by using COPY config/custom-cron /etc/cron.d/custom-cron. The problem is, that our line endings were in windows format. This did break our crontab file, because this line endings are not converted while copy that file into the container.
The second problem was, that we tried to start the cron via CMD ["cron", "-f"] which did block the main php-fpm process. This results in a 502 Bad gateway error when calling our web application.
Finaly we made it work by editing the crontab file manually while building the docker image instead of copy-pasting and using supervisord to have multiple tasks running inside docker. This should work on all supported operating systems.
dockerfile
FROM php:7.1.16-fpm
RUN apt-get update && apt-get install -y cron supervisor
# Configure cron
RUN crontab -l | { cat; echo "* * * * * echo 'Hello world' >> /var/log/cron-test.log 2>&1"; } | crontab -
# Configure supervisor
COPY config/supervisord.conf /etc/supervisor/supervisord.conf
supervisord.conf
[supervisord]
logfile = /dev/null
loglevel = info
pidfile = /var/run/supervisord.pid
nodaemon = true
[program:php-fpm]
command = php-fpm
autostart = true
autorestart = true
stdout_logfile = /dev/stdout
stdout_logfile_maxbytes = 0
stderr_logfile = /dev/stderr
stderr_logfile_maxbytes = 0
[program:cron]
command = cron -f
autostart = true
autorestart = true
stdout_logfile = /dev/stdout
stdout_logfile_maxbytes = 0
stderr_logfile = /dev/stderr
stderr_logfile_maxbytes = 0
There is a bug in Debian based distributions which will cause cronjobs to fail because docker uses layered filesystem and cron doesn't start and says NUMBER OF HARD LINKS > 1 (/etc/crontab).
The fix is simple, add touch /etc/crontab /etc/cron.*/* to the entrypoint of your container.
I have made a blog post explaining how to setup cron in a Docker container here : https://esc.sh/blog/cron-jobs-in-docker/
I know this is an old question but I found a fix to this on Debian and it solved my problem. Cron pam auth with uid was breaking my cron from being able to run.
RUN sed -i '/session required pam_loginuid.so/c\#session required pam_loginuid.so/' /etc/pam.d/cron