I was trying to schedule a cron job inside docker based logstash application.
The cron job is as follows:
30 10 * * * root logrotate -f /etc/logrotate.d/logstash
The cron is not getting executed inside container but when I execute the above command manually it works fine.
# logrotate -f /etc/logrotate.d/logstash
# ls -l /usr/share/logstash/logs/
total 36
-rw-r--r-- 1 logstash logstash 17 Jan 2 10:16 logstash.log
-rw-r--r-- 1 logstash logstash 10701 Jan 2 10:16 logstash.log.1
This might be a duplicate of Cronjobs in Docker container how get them running?
It basically says, that you need to make sure, that
/etc/init.d/cron start
is running.
Related
I am trying to set up regular task on an amazon ec2 instance that will launch a few docker containers.
I ve created a startup_service.sh script in my home directory :
cd ~
docker-compose pull && docker-compose up
In that very same home directory, I have a docker-compose.yml file that defines my containers and image.
I have tested this file sh startup_service.sh and it is working as expected
Ive added execute permission to this startup_service.sh file and created a cronjob with crontab -e :
50 11 * * * /usr/bin/sh /home/ec2-user/startup_service.sh
However it is not working ( running docker ps doesnt show any containers being created or anything).
However checking the cron logs, it seems like the task is actually executed sudo grep -C 3 "startup" /var/log/cron :
Feb 27 11:42:01 ip-172-31-27-90 crond[3188]: (ec2-user) RELOAD (/var/spool/cron/ec2-user)
Feb 27 11:44:00 ip-172-31-27-90 crontab[18011]: (ec2-user) LIST (ec2-user)
Feb 27 11:48:54 ip-172-31-27-90 crontab[18064]: (ec2-user) LIST (ec2-user)
Feb 27 11:50:01 ip-172-31-27-90 CROND[18153]: (ec2-user) CMD (/usr/bin/sh /home/ec2-user/startup_service.sh)
Feb 27 11:50:02 ip-172-31-27-90 CROND[18156]: (root) CMD (/usr/lib64/sa/sa1 1 1)
Feb 27 11:53:05 ip-172-31-27-90 crontab[18232]: (ec2-user) LIST (ec2-user)
Feb 27 12:00:01 ip-172-31-27-90 CROND[18343]: (root) CMD (/usr/lib64/sa/sa1 1 1)
How can I correctly setup this cron job ?
I am trying to set up a multi-container service with docker-compose.
Some of the containers need to be restarted from a fresh container (eg. the file system should be like in the image) when they restart.
How can I achieve this?
I've found the restart: always option I can put on my service in the docker-compose.yml file, but that doesn't give me a fresh file system as it uses the same container.
I've also seen the --force-recreate option of docker-compose up, but that doesn't apply as that only recreates the containers when the command is runned.
EDIT:
This is probably not a docker-compose issue, but more of a general docker question: What is the best way to make sure a container is in a fresh state when it is restarted? With fresh state, I mean a state identical to that of a brand new container from the same image. Restarted is the docker command docker restart or docker stop and docker start.
In docker, immutability typically refers to the image layers. They are immutable, and any changes are pushed to a container specific copy-on-write layer of the filesystem. That container specific layer will last for the lifetime of the container. So to have those files not-persist, you have two options:
Recreate the container instead of just restart it
Don't write the changes to the container filesystem, and don't write them to any persistent volumes.
You cannot do #1 with a restart policy by it's very definition. A restart policy gives you the same container filesystem, with the application restarted. But if you use docker's swarm mode, it will recreate containers when they exit, so if you can migrate to swarm mode, you can achieve this result.
Option #2 looks more difficult than it is. If you aren't writing to the container filesystem, or to a volume, then where? The answer is a tmpfs volume that is only stored in memory and is lost as soon as the container exits. In compose, this is a tmpfs: /data/dir/to/not/persist line. Here's an example on the docker command line.
First, let's create a container with a tmpfs mounted at /data, add some content, and exit the container:
$ docker run -it --tmpfs /data --name no-persist busybox /bin/sh
/ # ls -al /data
total 4
drwxrwxrwt 2 root root 40 Apr 7 21:50 .
drwxr-xr-x 1 root root 4096 Apr 7 21:50 ..
/ # echo 'do not save' >>/data/tmp-data.txt
/ # cat /data/tmp-data.txt
do not save
/ # ls -al /data
total 8
drwxrwxrwt 2 root root 60 Apr 7 21:51 .
drwxr-xr-x 1 root root 4096 Apr 7 21:50 ..
-rw-r--r-- 1 root root 12 Apr 7 21:51 tmp-data.txt
/ # exit
Easy enough, it behaves as a normal container, let's restart it and check the directory contents:
$ docker restart no-persist
no-persist
$ docker attach no-persist
/ # ls -al /data
total 4
drwxr-xr-x 2 root root 40 Apr 7 21:51 .
drwxr-xr-x 1 root root 4096 Apr 7 21:50 ..
/ # echo 'still do not save' >>/data/do-not-save.txt
/ # ls -al /data
total 8
drwxr-xr-x 2 root root 60 Apr 7 21:52 .
drwxr-xr-x 1 root root 4096 Apr 7 21:50 ..
-rw-r--r-- 1 root root 18 Apr 7 21:52 do-not-save.txt
/ # exit
As you can see, the directory returned empty, and we can add data as needed back to the directory. The only downside of this is the directory will be empty even if you have content in the image at that location. I've tried combinations of named volumes, or using the mount syntax and passing the volume-nocopy option to 0, without luck. So if you need the directory to be initialized, you'll need to do that as part of your container entrypoint/cmd by copying from another location.
In order to not persist any changes to your containers it is enough that you don't map any directory from host to the container.
In this way, every time the containers runs (with docker run or docker-compose up ), it starts with a fresh file system.
docker-compose down also removes the containers, deleting any data.
The best solution I have found so far, is for the container itself to make sure to clean up when starting or stopping. I solve this by cleaning up when starting.
I copy my app files to /srv/template with the docker COPY directive in my Dockerfile, and have something like this in my ENTRYPOINT script:
rm -rf /srv/server/
cp -r /srv/template /srv/server
cd /srv/server
Simple question: Is there a docker command to view the files inside a volume?
I run docker for windows which creates a MobyLinuxVM on my machine to run Docker. I can't get a remote desktop connection onto this machine like I can with an Ubuntu VM (which I also have running on my machine).
Therefore, I can't see a way to see what is inside my host volumes (as they are actually inside the MobyLinuxVM), where as if I ran docker on my Ubuntu VM I could remote onto the machine and take a look.
Therefore, is there a way I can run some sort of docker volume command to list what's inside each volume?
You can use a temporary container for this. I tend to use busybox for these temporary containers:
$ docker volume ls
DRIVER VOLUME NAME
local jenkins-home
local jenkins-home2
local jenkinsblueocean_jenkins-data
...
$ docker run -it --rm -v jenkins-home:/vol busybox ls -l /vol
total 428
-rw-r--r-- 1 1000 1000 327 Jul 14 2016 com.dabsquared.gitlabjenkins.GitLabPushTrigger.xml
-rw-r--r-- 1 1000 1000 276 Aug 17 2016 com.dabsquared.gitlabjenkins.connection.GitLabConnectionConfig.xml
-rw-r--r-- 1 1000 1000 256 Aug 17 2016 com.nirima.jenkins.plugins.docker.DockerPluginConfiguration.xml
drwxr-xr-x 28 1000 1000 4096 Aug 17 2016 config-history
-rw-r--r-- 1 1000 1000 6460 Aug 17 2016 config.xml
-rw-r--r-- 1 1000 1000 174316 Jun 2 18:50 copy_reference_file.log
-rw-r--r-- 1 1000 1000 2875 Aug 9 2016 credentials.xml
...
For a host volume, you can just replace the volume mount with the host directory name (fully qualified) in the docker run cli.
$ docker run -it --rm -v /path/on/host:/vol busybox ls -l /vol
This isn't a direct answer to the question (because it was asking about a docker command) but in case anyone arrives here like I did:
If you have Docker Desktop (on Windows at least) you can explore into a volume using the Docker Desktop GUI. Just click on the volume, then switch to the "Data" tab at the top.
Quick and easy if you are just wanting to take a look around or copy out a file.
Not sure how widely applicable this is, but if you have root access I've just discovered that you can browse the contents of a volume at /var/lib/docker/volumes/<VOLUME_NAME>/_data. VOLUME_NAME is as shown by docker volume ls.
I'm looking at an Ubuntu 18.04 VM running Docker 19.03.5 - YMMV.
Is there a way to associate existing docker volumes (located in /etc/docker/volumes) to containers?
One way to do this is to use docker inspect :conatiner_id but this assumes that the container exists. How can you find to which container the volume belonged to, in the scenario that the container does no longer exist?
Check docker volumes
$ ls -l /var/lib/docker/volumes/
total 72
drwxr-xr-x 3 root root 4096 Nov 14 14:27 0f801819cf76b04b6794163b65df5d649bd795e23f4fc778f78db9ac60a0180d
drwxr-xr-x 3 root root 4096 Nov 29 14:29 my-jenkins
For more info about your volume you can perform docker volume inspect but this tells you nothing about what's really inside your volume. The only way to know this is by going inside the volume-folder and check.
So I'll check "unamed" volume:
$ ls -l /var/lib/docker/volumes/0f801819cf76b04b6794163b65df5d649
bd795e23f4fc778f78db9ac60a0180d/_data
...
drwx------ 2 999 ping 4096 Nov 14 14:27 pg_tblspc
drwx------ 2 999 ping 4096 Nov 14 14:27 pg_twophase
drwx------ 3 999 ping 4096 Nov 14 14:27 pg_xlog
-rw------- 1 999 ping 88 Nov 14 14:27 postgresql.auto.conf
-rw------- 1 999 ping 20791 Nov 14 14:27 postgresql.conf
-rw------- 1 999 ping 37 Nov 14 14:27 postmaster.opts
Normally you should be able to link your volume to the old container you've used. You can check everything what's inside. There isn't a better way at the moment. This is actually the answer on your question but I'll give some more explanation to make it easier in the future.
The best way is to create named volumes. After deleting your container the volume will remain easy to recognize:
docker volume create --name my-jenkins
So in /var/lib/docker/volumes you'll see my-jenkins.
Now I start my jenkins container and link it with my named volume.
Everything which is in /var/jenkins_home will be stored in the named volume.
docker run -d -p 8080:8080 -v my-jenkins:/var/jenkins_home jenkins
I'll create a job in jenkins with the name firstjob. You'll see this job in my named docker volume.
$ ls -l /var/lib/docker/volumes/my-jenkins/_data/jobs/
total 4
drwxr-xr-x 3 dockrema dockrema 4096 Nov 29 14:47 firstjob
Now I will delete my container (id = fa1003894dbc). The container is gone:
$ docker rm -fv fa1003894dbc
I'm a bit later. I want to reuse the named docker volume which still exists to start a new jenkins container which will immediatly container the job "firstjob".
$ docker run -d -p 8080:8080 -v my-jenkins:/var/jenkins_home jenkins
If you have an unnamed docker volume (created automatically with name 0f8018x9cf76b04x163b6xdf) you can use
docker run -d -v 0f8018x9cf76b04x163b6xdf:/var/jenkins_home jenkins
Now your jenkins will use everything which is inside that volume (it's only not a named volume, what makes it more difficult to see with which type of container it was linked. But by accessing the volume folder you will find it in most cases.)
I've been looking into setting up a data volume for a Docker container that I'm running on my server. The container is from this FreePBX image https://hub.docker.com/r/jmar71n/freepbx/
Basically I want persistent data so I don't lose my VoIP extensions and settings in the case of Docker stopping. I've tried many guides, ones here on stack overflow, and on the Docker manpages, but I just can't quite get it to work.
Can anyone help me with what commands I need to run in order to attach a volume to the FreePBX image I linked above?
You can do this by running a container with the -v option and mapping to a host directory - you just need to know where the container's storing the data.
Looking at the Dockerfile for that image, I'm assuming that the data you're interested in is stored in MySql. In the MySql config the data directory the container's using is /var/lib/mysql.
So you can start your container like this, mapping the MySql data directory to /docker/pbx-data on your host:
> docker run -d -t -v /docker/pbx-data:/var/lib/mysql jmar71n/freepbx
20b45b8fb2eec63db3f4dcab05f89624ef7cb1ff067cae258e0f8a910762fb1a
Use docker inpect to confirm that the mount is mapped as expected:
> docker inspect --format '{{json .Mounts}}' 20b
[{"Source":"/docker/pbx-data",
"Destination":"/var/lib/mysql",
"Mode":"","RW":true,"Propagation":"rprivate"}]
When the container runs it bootstraps the database, so on the host you'll be able to see the contents of the MySql data directory the container is using:
> ls -l /docker/pbx-data
total 28684
-rw-r----- 1 103 root 2062 Sep 21 09:30 20b45b8fb2ee.err
-rw-rw---- 1 103 messagebus 18874368 Sep 21 09:30 ibdata1
-rw-rw---- 1 103 messagebus 5242880 Sep 21 09:30 ib_logfile0
-rw-rw---- 1 103 messagebus 5242880 Sep 21 09:30 ib_logfile1
drwx------ 2 103 root 4096 Sep 21 09:30 mysql
drwx------ 2 103 messagebus 4096 Sep 21 09:30 performance_schema
If you kill the container and run another one with the same volume mapping, it will have all the data files from the previous container, and your app state should be preserved.
I'm not familiar with FreePBX, but if there is state being stored in other directories, you can find the locations in config and map them to the host in the same way, with multiple -v options.
Hi Elton Stoneman and user3608260!
Yes, you assuming correctly for data saves in Mysql (records, users, configs, etc.).
But in asterisk, all configurations are saved in files '.conf' and similars.
In this case, the archives looked for user3608260 are storaged in '/etc/asterisk/*'
Your answer is perfectly with more one command: -v /local_to_save:/etc/asterisk
the final docker command:
docker run -d -t -v /docker/pbx-data:/var/lib/mysql -v /docker/pbx-asterisk:/etc/asterisk jmar71n/freepbx
[Assuming /docker/pbx-asterisk is a host directory. ]