I want to run a lot of docker containers in foreground mode like docker run johndoe/example doSomething.
I don't want to waste disk space on exited containers, so I have to remove them. But it takes about 9 seconds to remove an exited container, so I should remove the container after the command execution.
That's why I have two concurrent processes: one runs containers, the other removes them.
The problem is that docker rm appears to be blocking - it doesn't run while removing.
Here is a minimal working example. Following command runs a docker container every second, then print current date to the console:
while true; do docker run ubuntu ls > /dev/null; sleep 1; date; done
And the output is like:
Tue Sep 30 14:25:18 MSK 2014
Tue Sep 30 14:25:20 MSK 2014
Tue Sep 30 14:25:22 MSK 2014
Tue Sep 30 14:25:24 MSK 2014
But when I run docker rm some_id in a separate console, then I see that the time span increases like:
Tue Sep 30 14:26:53 MSK 2014
Tue Sep 30 14:26:55 MSK 2014
Tue Sep 30 14:27:03 MSK 2014
Tue Sep 30 14:27:10 MSK 2014
Am I getting something wrong? Why is it so? How can I deal with it?
Have you tried removing multiple containers in one rm command? This may reduce the number of hiccups. Perhaps you could do that once a day at a time where a hiccup is not important to you?
Also, have you measured how much disk space gets consumed, it may not matter for a long time thanks to the nature of the overlay filesystems. Of course this depends on what your doSomething actually does...
As in the docker-user mailing list - can you please post the output from
docker version ; docker info
as this may well be storage driver and setup specific
(though perhaps you just have so many containers or images that your system is going to take time to process them.
Related
Using macos Catalina and docker desktop.
The time of the conteiners perfectly syncs with the time in Vm Docker Desktop.
But I need to test one conteiner with date in the future.
I dont want to advance the clock of my mac because of iCloud services.
So I can achieve this just changing the hour in VM docker-desktop
I run:
docker run --privileged --rm alpine date -s "2023-02-19 11:27"
It changes the time ok. But it last just some seconds. Clearly there is some type of "syncronizer" that keeps changing back the time.
How do I disable this "syncronizer"?
There's only one time in Linux, it's not namespaced, so when Docker runs ntp on the VM to keep it synchronized (in the past it would get out of sync, especially after the parent laptop was put to sleep), that sync applies to the Linux kernel, which applies to every container since it's the same kernel value for everything. Therefore it's impossible to set this on just one container in the Linux kernel.
Instead, I'd recommend going with something like libfaketime that can be used to alter the response applications see when the query that time value. It basically sits as a layer between the kernel and application, and injects an offset based on an environment variable you set.
FROM debian
ARG DEBIAN_FRONTEND=noninteractive
RUN apt-get update \
&& apt-get install -y libfaketime \
&& rm -rf /var/lib/apt/lists*
ENV LD_PRELOAD=/usr/lib/x86_64-linux-gnu/faketime/libfaketime.so.1
And then to run it, set FAKETIME:
$ docker run --rm test-faketime date
Thu Feb 17 14:59:48 UTC 2022
$ docker run -e FAKETIME="+7d" --rm test-faketime date
Thu Feb 24 14:59:55 UTC 2022
$ date
Thu 17 Feb 2022 09:59:57 AM EST
I found that you can kill the NTP service which syncs the VM time to the host's time. Details of how service works.
First, use this guide to get a shell inside the VM.
Then, find the sntpc service:
/ # ps a | grep sntpc
1356 root 0:00 /usr/bin/containerd-shim-runc-v2 -namespace services.linuxkit -id sntpc -address /run/containerd/containerd.sock
1425 root 0:00 /usr/sbin/sntpc -v -i 30 127.0.0.1
3465 root 0:00 grep sntpc
Take the number at the beginning of the /usr/sbin/sntpc line, and use kill to stop the process.
/ # kill 1425
I have found that Docker Desktop does not seem to restart this process if it dies, and you can change the VM time without SNTPC changing it back.
I am using rails and want to run sidekiq and running sidekiq requires a Redis server to be installed. I installed Redis for my KDE Neon by following the instructions from the digital ocean. Here is the error that is displayed when I try to run sudo systemctl status redis :
redis.service - Redis In-Memory Data Store
Loaded: loaded (/etc/systemd/system/redis.service; enabled; vendor preset: enabled)
Active: failed (Result: exit-code) since Wed 2021-03-24 17:24:12 IST; 6s ago
Process: 47334 ExecStart=/usr/local/bin/redis-server /etc/redis/redis.conf (code=exited, status=203/EXEC)
Main PID: 47334 (code=exited, status=203/EXEC)
Mar 24 17:24:12 maxagno3 systemd[1]: redis.service: Scheduled restart job, restart counter is at 5.
Mar 24 17:24:12 maxagno3 systemd[1]: Stopped Redis In-Memory Data Store.
Mar 24 17:24:12 maxagno3 systemd[1]: redis.service: Start request repeated too quickly.
Mar 24 17:24:12 maxagno3 systemd[1]: redis.service: Failed with result 'exit-code'.
Mar 24 17:24:12 maxagno3 systemd[1]: Failed to start Redis In-Memory Data Store.
Using redis-cli works fine. I assume when I first ran the command sudo systemctl disable redis it deleted the redis.service file. Since then I have uninstalled and installed the Redis but still, the error persists.
Quick help would greatly be appreciated.
The error your showing is hiding your originally error. Redis is basically in a reboot loop, which your error is eluding to. What you need to do is disable this restart functionality to get the underlining problem.
This can be done by doing the following:
Edit the /etc/systemd/system/redis.service
Edit the restart line to Restart=no. The possible options are no, on-success, on-failure, on-abnormal, on-watchdog, on-abort or always.
Edit the start limit interval to StartLimitInterval=0. This is normally set really high to prevent load from spiking by a service constantly restarting
Lastly reload your services for your changes to take affect. This is done by running systemctl daemon-reload
Once your service stops looping, you can try to manually start the service to get the actual error. If the error is too large you can look in your OS's general log greping specifically for Redis, or by running journalctl: journalctl -u redis.service
Hopefully this helps!
If you want clean and repeatable approach I suggest you to always use docker, especially for dev environment.
So starting docker redis as simple as:
docker run -d -p 6379:6379 --name my-redis redis
I'm trying to create a Docker image which will forward a port through a VPN. I've created a simple image which exposes port 5144, and tested that it works properly:
sudo docker run -t -d -p 5144:5144 \
--name le-bridge \
--cap-add=NET_ADMIN \
--device=/dev/net/tun \
bridge
sudo docker exec -it le-bridge /bin/bash
I check that the port is exposed correctly like this:
[CONTAINER] root#6116787b1c1e:~# nc -lvvp 5144
[HOST] user$ nc -vv 127.0.0.1 5144
Then, whatever I type is correctly echoed in the container's terminal. However, as soon as I start the openvpn daemon, this doesn't work anymore:
[CONTAINER] root#6116787b1c1e:~# openvpn logger.ovpn &
[1] 33
Sun Apr 5 22:52:54 2020 OpenVPN 2.4.4 x86_64-pc-linux-gnu [SSL (OpenSSL)] [LZO] [LZ4] [EPOLL] [PKCS11] [MH/PKTINFO] [AEAD] built on May 14 2019
Sun Apr 5 22:52:54 2020 library versions: OpenSSL 1.1.1 11 Sep 2018, LZO 2.08
Sun Apr 5 22:52:54 2020 TCP/UDP: Preserving recently used remote address: [AF_INET]
Sun Apr 5 22:52:54 2020 UDPv4 link local (bound): [AF_INET][undef]:1194
Sun Apr 5 22:52:54 2020 UDPv4 link remote:
Sun Apr 5 22:52:54 2020 WARNING: this configuration may cache passwords in memory -- use the auth-nocache option to prevent this
Sun Apr 5 22:52:55 2020 [] Peer Connection Initiated with [AF_INET]
Sun Apr 5 22:53:21 2020 TUN/TAP device tun0 opened
Sun Apr 5 22:53:21 2020 do_ifconfig, tt->did_ifconfig_ipv6_setup=0
Sun Apr 5 22:53:21 2020 /sbin/ip link set dev tun0 up mtu 1500
Sun Apr 5 22:53:21 2020 /sbin/ip addr add dev tun0 10.X.0.2/24 broadcast 10.X.0.255
Sun Apr 5 22:53:21 2020 Initialization Sequence Completed
root#6116787b1c1e:~#
root#6116787b1c1e:~# nc -lvvp 5144
listening on [any] 5144 ...
From here, using the exact same netcat command, I cannot reach the exposed port anymore from the host.
What am I missing?
EDIT: It's maybe worth mentioning that after the VPN is started, the connexion still succeeds from the host ; it just never reaches the netcat process inside the container.
I'm not exactly sure why, but it turns out that routes need to be fixed inside the container. In my case, the following command solves the issue:
ip route add 192.168.0.0/24 via 172.17.42.1 dev eth0
...where 172.17.42.1 is the IP of the docker0 interface on my host.
Hopefully this is helpful to someone one day.
Testing with Jenkins & Docker, I don't understand completely what is happening with my containers and images.
Firstly, I built my first docker container from jenkins/jenkins:tls
docker run --name myjenkins -p 8080:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts
And I received the typical messsage from jenkins installation with the initial password:
INFO:
*************************************************************
*************************************************************
*************************************************************
Jenkins initial setup is required. An admin user has been created and a password generated.
Please use the following password to proceed to installation:
xxxxxxxxxxxxxxxxxxxxxxxxxxxx
This may also be found at: /var/jenkins_home/secrets/initialAdminPassword
*************************************************************
*************************************************************
*************************************************************
I completed the installation process and I were playing with Jenkins for a while. Everything ok.
My missunderstanding come repeating the process from the beggining. I deleted my container and built the same container for a second time.
docker container stop myjenkins <- Stop container
docker container rm myjenkins <- Remove myjenkins container
docker image rm 95bf220e341a <- Remove jenkins/jenkins image
docker run --name myjenkins -p 8080:8080 -p 50000:50000 -v jenkins_home:/var/jenkins_home jenkins/jenkins:lts
But in this case, Jenkins doesn't show me a new initial password for this second time:
Jun 18, 2019 7:43:17 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory#83bb567: defining beans [authenticationManager]; root of factory hierarchy
<-- I was expecting the message just here -->
Jun 18, 2019 7:43:17 PM org.springframework.context.support.AbstractApplicationContext prepareRefresh
INFO: Refreshing org.springframework.web.context.support.StaticWebApplicationContext#5bfdcaf3: display name [Root WebApplicationContext]; startup date [Tue Jun 18 19:43:17 UTC 2019]; root of context hierarchy
Jun 18, 2019 7:43:17 PM org.springframework.context.support.AbstractApplicationContext obtainFreshBeanFactory
INFO: Bean factory for application context [org.springframework.web.context.support.StaticWebApplicationContext#5bfdcaf3]: org.springframework.beans.factory.support.DefaultListableBeanFactory#1f98db0a
Jun 18, 2019 7:43:17 PM org.springframework.beans.factory.support.DefaultListableBeanFactory preInstantiateSingletons
INFO: Pre-instantiating singletons in org.springframework.beans.factory.support.DefaultListableBeanFactory#1f98db0a: defining beans [filter,legacy]; root of factory hierarchy
Jun 18, 2019 7:43:18 PM jenkins.InitReactorRunner$1 onAttained
INFO: Completed initialization
Jun 18, 2019 7:43:19 PM hudson.WebAppMain$3 run
INFO: Jenkins is fully up and running
I tried with docker system prune -a but nothing changed. Every time that I tried to rebuild my container, I couldn't get the Initial Admin Password message again.
What's happening? If I delete a container.. How Docker/Jenkins knows that is not the first time I try to install jenkins?
-v jenkins_home:/var/jenkins_home
You're mapping that directory to somewhere so the docker image is (hopefully) immutable. Recreating it makes no difference - if you don't expunge that folder then Jenkins' config data remains intact the next time the image is started.
Also, this means your bootstrap pwd should be available on the docker host at:
jenkins_home/secrets/initialAdminPassword
I have 3 nodes that I want to play with managers and workers. My first one (Debian) I made it into a swarm manager:
root#debiancli:~# docker swarm init --advertise-addr 192.168.182.129
Swarm initialized: current node (mkg6ecl3x28uyyqx7gvzz0ja3) is now a manager.
To add a worker to this swarm, run the following command:
docker swarm join --token SWMTKN-1-47h52q7mpdkhbi4dsqyjt7pnjqgvm4oxxfh87k6e2hoj8f4op0-2p1zkg309owyophvk95bw7rj0 192.168.182.129:2377
To add a manager to this swarm, run 'docker swarm join-token manager' and follow the instructions.
On my second soon-to-be node (CentOS), I tried to join it to the cluster:
[root#centostraining ~]# docker swarm join --token SWMTKN-1-47h52q7mpdkhbi4dsqyjt7pnjqgvm4oxxfh87k6e2hoj8f4op0-2p1zkg309owyophvk95bw7rj0 192.168.182.129:2377
Error response from daemon: error while validating Root CA Certificate: x509: certificate has expired or is not yet valid
but it said, as shown above, that the certificate is not valid (date issue). I checked the date on my Debian and it is fine
root#debiancli:~# date
Tue Aug 14 22:02:29 EDT 2018
I also checked the date in my CentOS:
[root#centostraining ~]# date
Ter Ago 14 22:05:05 -03 2018
Now, I checked my swarm manager CA cert date:
root#debiancli:~# docker swarm ca | openssl x509 -noout -text | grep -E "Before|After"
Not Before: Aug 15 01:58:00 2018 GMT
Not After : Aug 10 01:58:00 2038 GMT
So, weirdly enough, my certificate was generated to start the day after it was generated?
Then on my future node (CentOS), if I change the date:
[root#centostraining ~]# date +%Y%m%d -s "20180816"
20180816
[root#centostraining ~]# date
Qui Ago 16 00:00:01 -03 2018
[root#centostraining ~]# docker swarm join --token SWMTKN-1-47h52q7mpdkhbi4dsqyjt7pnjqgvm4oxxfh87k6e2hoj8f4op0-2p1zkg309owyophvk95bw7rj0 192.168.182.129:2377
This node joined a swarm as a worker.
Voilá, it now works as expected. Can anyone explain why my swarm ca cert is "in the future"?
Docker's certificate is not in the future. Instead, your timezones are setup incorrectly. On one, you are showing EDT, or -4. And on the other node, it's running an hour behind since it shows the same local time but in a -3 timezone. The output from the certificate is GMT or +0.
You should be using a tool like NTP to keep your clocks in sync, which would then show the -3 machine running an hour ahead of the other in local time, but the same time if you compare the same timezone.