Docker cron job - docker

I need to schedule some cron jobs in docker/kubernetes environment, which will make some external service calls using curl commands. I was trying to use plain alpine:3.6 image but it doesn't have curl.
Any suggestion what base image will be good for this purpose? Also, it will be helpful if there are some examples.

Prepare your own docker image which will includes packages you need or just use something like this https://github.com/aylei/kubectl-debug

I am able to run curl as cronjob job using alpine image as
FROM alpine:3.6
RUN apk --no-cache add curl bash
RUN apk add --no-cache tzdata
ENV TZ=Asia/Kolkata
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ /etc/timezone
RUN mkdir -p /var/log/cron \
&& touch /var/log/cron/cron.log \
&& mkdir -m 0644 -p /etc/cron.d
ADD curlurl /etc/crontabs/root
CMD bash -c "crond -L /var/log/cron/cron.log && tail -F /var/log/cron/cron.log"
#ENTRYPOINT /bin/bash
Where curlurl is having simple curl command as cron job
* * * * * /usr/bin/curl http://dummy.restapiexample.com/api/v1/employees >> /var/log/cron/cron.log
# Leave one blank line below or cron will fail
This is working fine . But now I have tried to make alpine base image with push in private repo to avoid downloading every time. But it stopped working. How can i debug why cron job is not running anymore. Below is the way I have used.
BaseImage
FROM alpine:3.6
RUN apk --no-cache add curl bash
RUN apk add --no-cache tzdata
ENTRYPOINT /bin/bash
Docker for curl
FROM my/alpine:3.6
ENV TZ=Asia/Kolkata
RUN ln -snf /usr/share/zoneinfo/$TZ /etc/localtime && echo $TZ /etc/timezone
RUN mkdir -p /var/log/cron \
&& touch /var/log/cron/cron.log \
&& mkdir -m 0644 -p /etc/cron.d
ADD curlurl /etc/crontabs/root
CMD bash -c "crond -L /var/log/cron/cron.log && tail -F /var/log/cron/cron.log"
After making it like this I am not able to get any cron output . Any suggestion on how can I debug it.

Related

sh: ./filebeat: not found in Docker container

Im trying to run filebeat in a docker container with the s6 overlay.
When s6 executes or when i manually execute the filebeat binary i get sh: ./filebeat: not found
This is my Dockerfile:
FROM alpine:3.15
ENV AM_I_IN_A_DOCKER_CONTAINER Yes
COPY root/ /
ADD https://github.com/just-containers/s6-overlay/releases/download/v1.21.8.0/s6-overlay-amd64.tar.gz /tmp/
ADD https://artifacts.elastic.co/downloads/beats/filebeat/filebeat-8.0.0-linux-x86_64.tar.gz /tmp/
ADD requirements.txt /etc/services.d/01_instabot/requirements.txt
ADD src/ /etc/services.d/01_instabot/
RUN chmod +x /usr/local/bin/install.sh
RUN /usr/local/bin/install.sh
#ENTRYPOINT ["/init"]
This is my install.sh:
#!/bin/sh
echo "Unpacking s6 overlay"
gunzip -c /tmp/s6-overlay-amd64.tar.gz | tar -xf - -C /
echo "Creating user"
adduser -D -u 2000 -s /sbin/nologin -D -H botuser
adduser -D -u 2001 -s /sbin/nologin -D -H filebeatuser
echo "Set time"
ln -snf /usr/share/zoneinfo/"$TZ" /etc/localtime && echo "$TZ" > /etc/timezone
apk add --no-cache tzdata
echo "Install filebeat"
gunzip -c /tmp/filebeat-8.0.0-linux-x86_64.tar.gz | \
tar -xf - -C /etc/services.d/00_filebeat/ --strip-components=1
mv /etc/services.d/00_filebeat/my_filebeat.yml /etc/services.d/00_filebeat/filebeat.yml
echo "Install app dependencies"
apk add --no-cache python3 py3-pip
pip3 install --no-cache-dir -r /etc/services.d/01_instabot/requirements.txt
mv /etc/services.d/01_instabot/settings_docker.py /etc/services.d/01_instabot/settings.py
echo "Cleanup"
rm -rf /tmp/*
If i take a look inside the docker container with the docker run command i see the binary present.
/etc/services.d/00_filebeat # ls
LICENSE.txt README.md filebeat filebeat.yml module run
NOTICE.txt fields.yml filebeat.reference.yml kibana modules.d
But when i execute it using ./filebeat i get the not found error.
/etc/services.d/00_filebeat # ./filebeat
sh: ./filebeat: not found
Why is this? And how do i fix it? Is it because of busybox or something?
libc6-compat was missing from my alpine image.

Run Python scripts on command line running Docker images

I built a docker image using Dockerfile with Python and some libraries inside (no my project code inside). In my local work dir, there are some scripts to be run on the docker. So, here what I did
$ cd /path/to/my_workdir
$ docker run -it --name test -v `pwd`:`pwd` -w `pwd` my/code:test python src/main.py --config=test --results-dir=/home/me/Results
The command python src/main.py --config=test --results-dir=/home/me/Results is what I want to run inside the Docker container.
However, it returns,
/home/docker/miniconda3/bin/python: /home/docker/miniconda3/bin/python: cannot execute binary file
How can I fix it and run my code?
Here is my Dockerfile
FROM nvidia/cuda:10.1-cudnn7-runtime-ubuntu18.04
MAINTAINER Me <me#me.com>
RUN apt update -yq && \
apt install -yq curl wget unzip git vim cmake sudo
RUN adduser --disabled-password --gecos '' docker && \
adduser docker sudo && \
echo '%sudo ALL=(ALL) NOPASSWD:ALL' >> /etc/sudoers
USER docker
WORKDIR /home/docker/
RUN chmod a+rwx /home/docker/ && \
wget https://repo.anaconda.com/miniconda/Miniconda3-latest-Linux-x86_64.sh && \
bash Miniconda3-latest-Linux-x86_64.sh -b && rm Miniconda3-latest-Linux-x86_64.sh
ENV PATH /home/docker/miniconda3/bin:$PATH
Run pip install absl-py==0.5.0 atomicwrites==1.2.1 attrs==18.2.0 certifi==2018.8.24 chardet==3.0.4 cycler==0.10.0 docopt==0.6.2 enum34==1.1.6 future==0.16.0 idna==2.7 imageio==2.4.1 jsonpickle==1.2 kiwisolver==1.0.1 matplotlib==3.0.0 mock==2.0.0 more-itertools==4.3.0 mpyq==0.2.5 munch==2.3.2 numpy==1.15.2 pathlib2==2.3.2 pbr==4.3.0 Pillow==5.3.0 pluggy==0.7.1 portpicker==1.2.0 probscale==0.2.3 protobuf==3.6.1 py==1.6.0 pygame==1.9.4 pyparsing==2.2.2 pysc2==3.0.0 pytest==3.8.2 python-dateutil==2.7.3 PyYAML==3.13 requests==2.19.1 s2clientprotocol==4.10.1.75800.0 sacred==0.8.1 scipy==1.1.0 six==1.11.0 sk-video==1.1.10 snakeviz==1.0.0 tensorboard-logger==0.1.0 torch==0.4.1 torchvision==0.2.1 tornado==5.1.1 urllib3==1.23
USER docker
ENTRYPOINT ["/bin/bash"]
Try making the file executable before running it.
as John mentioned to do in the dockerfile
FROM python:latest
COPY src/main.py /usr/local/share/
RUN chmod +x /usr/local/share/src/main.py #<-**--- just add this also
# I have some doubts about the pathing
CMD ["/usr/local/share/src/main.py", "--config=test --results-dir=/home/me/Results"]
You can run a python script in docker by adding this to your docker file:
FROM python:latest
COPY src/main.py /usr/local/share/
CMD ["src/main.py", "--config=test --results-dir=/home/me/Results"]

Unable to run Heroku exec on a docker container

I've followed the steps here (https://devcenter.heroku.com/articles/exec#enabling-docker-support) to install curl (curl 7.61.1 (x86_64-alpine-linux-musl) libcurl/7.61.1 LibreSSL/2.0.0 zlib/1.2.11 libssh2/1.8.0 nghttp2/1.32.0), python, bash, and openssh (OpenSSH_7.2p2-hpn14v4, OpenSSL 1.0.2p 14 Aug 2018). I've created the /app/.profile.d/heroku-exec.sh file in my container and added a sym link.
However, running heroku ps:exec -a my-app returns the following message:
Establishing credentials... error
! Could not connect to dyno!
! Check if the dyno is running with `heroku ps'
I've verified my application is in fact running (& has the runtime-heroku-exec feature enabled):
web (Free): /bin/sh -c exec\ java\ \$JAVA_OPTS\ -Dserver.port\=\$PORT\ -Djava.security.egd\=file:/dev/./urandom\ -jar\ /app.jar (1)
web.1: up 2018/09/30 09:34:55 -0600 (~ 4m ago)
I've verified that the heroku-exec.sh exists on my deployed container by doing heroku run -a my-app bash and cat /app/.profile.d/heroku-exec.sh
At this point, I'm not sure what to try in order to troubleshoot why heroku exec won't work on my container. Here's what my Dockerfile looks like in case there's something off with how I've put together my application:
FROM openjdk:8-jdk-alpine
RUN apk add --no-cache openssh-keygen
RUN apk add --no-cache openssh-client=7.2_p2-r5 --repository=http://dl-cdn.alpinelinux.org/alpine/v3.4/main --allow-untrusted
RUN apk add --no-cache openssh-sftp-server=7.2_p2-r5 --repository=http://dl-cdn.alpinelinux.org/alpine/v3.4/main --allow-untrusted
RUN apk add --no-cache openssh=7.2_p2-r5 --repository=http://dl-cdn.alpinelinux.org/alpine/v3.4/main --allow-untrusted
RUN apk add --no-cache curl
RUN apk add --no-cache bash
RUN apk add --update --no-cache python
RUN rm /bin/sh && ln -s /bin/bash /bin/sh
RUN adduser -D app-user
USER app-user
ARG JAR_FILE
ARG PORT
ARG HEROKU_FILE_NAME
COPY ${JAR_FILE} app.jar
COPY ${HEROKU_FILE_NAME} /app/.profile.d/heroku-exec.sh
ENV JAVA_OPTS -agentlib:jdwp=transport=dt_socket,server=y,suspend=n,address=8000
HEALTHCHECK --interval=15m --timeout=10s --retries=3 --start-period=1m CMD curl --fail http://localhost:8080/restaurantscores/actuator/health || exit 1
CMD exec java $JAVA_OPTS -Dserver.port=$PORT -Djava.security.egd=file:/dev/./urandom -jar /app.jar
The "Using with Docker" section of the "Heroku Exec (SSH Tunneling)" documentation doesn't address Alpine usage.
Rather than using /app/.profile.d Alpine uses /etc/profile.d.
So change
COPY ${HEROKU_FILE_NAME} /app/.profile.d/heroku-exec.sh
to
COPY ${HEROKU_FILE_NAME} /etc/profile.d/heroku-exec.sh
Also ensure that heroku-exec.sh is executable. You can do this using chmod +x heroku-exec.sh.
I found that I had to additionally add a python symlink to my alpine-based dockerfile, to get it working:
ln -s /usr/bin/python3 /usr/bin/python
here is an example of a dockerfile, for deploying a very simple nginx application, that definitely works with heroku-exec (as of October 2020):
FROM alpine:latest
# install required packages
RUN apk add --no-cache --update python3 bash curl openssh nginx
# simplfy nginx config to enable ENV variable substitution
RUN echo 'server { listen PORT_NUMBER; }' > /etc/nginx/conf.d/default.conf \
&& mkdir /run/nginx
# add config required for HEROKU_EXEC
# ENV HEROKU_EXEC_DEBUG=1
RUN rm /bin/sh \
&& ln -s /bin/bash /bin/sh \
&& mkdir -p /app/.profile.d/ \
&& printf '#!/usr/bin/env bash\n\nset +o posix\n\n[ -z "$SSH_CLIENT" ] && source <(curl --fail --retry 7 -sSL "$HEROKU_EXEC_URL")\n' > /app/.profile.d/heroku-exec.sh \
&& chmod +x /app/.profile.d/heroku-exec.sh \
&& ln -s /usr/bin/python3 /usr/bin/python
# configure NGINX to listen on dynamic $PORT env variable supplied by Heroku.
CMD sed -i 's/PORT_NUMBER/'"$PORT"'/g' /etc/nginx/conf.d/default.conf; nginx -g 'daemon off;'

My docker starts zookeeper, but it then automatically exists

i write dockfile start zookeeper
FROM buildpack-deps:sid-scm
RUN apt-get update && apt-get install -y --no-install-recommends \
bzip2 \
unzip \
xz-utils \
gettext-base \
&& rm -rf /var/lib/apt/lists/*
COPY zookeeper-3.4.12.tar.gz /opt
COPY config.template.properties /opt
RUN tar xfz /opt/zookeeper-3.4.12.tar.gz -C /opt
ENV ZK_HOME /opt/zookeeper-3.4.12
COPY startzookeeper.sh /opt
RUN chmod a+x /opt/startzookeeper.sh $ZK_HOME
CMD ["/opt/startzookeeper.sh"]
the startzookeeper.sh file is
#!/usr/bin/env bash
eval "cat <<EOF
$(</opt/config.template.properties)
EOF
" | tee /opt/zoo.cfg 2> /dev/null
#echo "$ZK_HOME" > 2.txt
cp /opt/zoo.cfg "$ZK_HOME"/conf
#
exec "$ZK_HOME/bin/zkServer.sh" start
but when i run docker ps,it is empty.
i try add tail -f /dev/null,but it does not work.
i don't know why,the zookeeper should run always,why it exist?
thanks any suggestions.
You could adapt your script to imitate the one from the official zookeeper-docker
(from hub.docker.com)
Its docker-entrypoint.sh ends with exec "$#", which executes "zkServer.sh", "start-foreground".
The important part is the start-foreground option, which ensures the process does not exit immediately, as that would exit your container as well.

How can i use volume in my dockerfile for copy the jmeter result in my local

How can I use volume in my dockerfile for copy the JMeter result in my local?
Need to display the result in local, how can I copy the result and paste in local with the help of VOLUME.
For example:- I am saving the JMeter HTML report in my container but after that container is automatically stopped. So someone suggests me to use the docker VOLUME command for RUN the HTML.
FROM alpine
ARG JMETER_VERSION="4.0"
ENV JMETER_HOME /opt/apache-jmeter-${JMETER_VERSION}
ENV JMETER_BIN ${JMETER_HOME}/bin
ENV JMETER_DOWNLOAD_URL https://archive.apache.org/dist/jmeter/binaries/apache-jmeter-${JMETER_VERSION}.tgz
ENV JMETER_PLUGINS_DOWNLOAD_URL http://repo1.maven.org/maven2/kg/apc/jmeter-plugins-functions/2.0/jmeter-plugins-functions-2.0.jar
ENV JMETER_PLUGINS_FOLDER ${JMETER_HOME}/lib/ext/
# Change TimeZone TODO: TZ still is not set!
ARG TZ="Australia/Melbourne"
RUN apk update \
&& apk upgrade \
&& apk add ca-certificates \
&& update-ca-certificates \
&& apk add --update openjdk8-jre tzdata curl unzip bash \
&& rm -rf /var/cache/apk/* \
&& mkdir -p /tmp/dependencies \
&& curl -L --silent ${JMETER_DOWNLOAD_URL} > /tmp/dependencies/apache-jmeter-${JMETER_VERSION}.tgz \
&& mkdir -p /opt \
&& tar -xzf /tmp/dependencies/apache-jmeter-${JMETER_VERSION}.tgz -C /opt \
&& rm -rf /tmp/dependencies
RUN curl -L --silent ${JMETER_PLUGINS_DOWNLOAD_URL}/jmeter-plugins-dummy/0.2/jmeter-plugins-dummy-0.2.jar -o ${JMETER_PLUGINS_FOLDER}/jmeter-plugins-dummy-0.2.jar
RUN curl -L --silent ${JMETER_PLUGINS_DOWNLOAD_URL}/jmeter-plugins-cmn-jmeter/0.5/jmeter-plugins-cmn-jmeter-0.5.jar -o ${JMETER_PLUGINS_FOLDER}/jmeter-plugins-cmn-jmeter-0.5.jar
# TODO: plugins (later)
# && unzip -oq "/tmp/dependencies/JMeterPlugins-*.zip" -d $JMETER_HOME
# Set global PATH such that "jmeter" command is found
ENV PATH $PATH:$JMETER_BIN
ENV URL_PATH=${URL}
WORKDIR ${JMETER_HOME}
#RUN export DATETIME=$(date +%Y%m%d)
RUN mkdir -p /var/www/html/"$(date +%Y%m%d)"
VOLUME /var/www/html/
#Copy the *.jmx file jmeter bin file
COPY Get_Ping_Node_API.jmx ./bin
CMD ./bin/jmeter -n -t ./bin/Get_Ping_Node_API.jmx -l ./bin/result.jtl -e -o ./bin/result_html'
You can use docker volume while you run the docker image.
You can add --volume , -v flag to your docker run command.
docker run -v "HOST DIR":"CONTAINER DIR"

Resources