I'm trying to start the Docker daemon:
sudo systemctl start docker
But nothing happens, the cursor just blinks and the process never ends.
Yesterday it was working properly :(
sudo journalctl -fu docker
ago 18 16:05:24 host docker[1602]: time="2016-08-18T16:05:24.467635627-05:00" level=info msg="New containerd process, pid: 1609\n"
ago 18 16:05:24 host docker[1602]: time="2016-08-18T16:05:24.482107319-05:00" level=fatal msg="bad listen address format /var/run/docker/libcontainerd/docker-containerd.sock, expected proto://address"
ago 18 16:05:30 host docker[1602]: time="2016-08-18T16:05:30.470570243-05:00" level=info msg="New containerd process, pid: 1620\n"
ago 18 16:05:30 host docker[1602]: time="2016-08-18T16:05:30.491495106-05:00" level=fatal msg="bad listen address format /var/run/docker/libcontainerd/docker-containerd.sock, expected proto://address"
ago 18 16:08:06 host systemd[1]: Stopped Docker Application Container Engine.
-- Reboot --
ago 18 16:16:52 host systemd[1]: Starting Docker Application Container Engine...
ago 18 16:16:54 host docker[2294]: time="2016-08-18T16:16:54.360878396-05:00" level=info msg="New containerd process, pid: 2327\n"
ago 18 16:16:54 host docker[2294]: time="2016-08-18T16:16:54.686503187-05:00" level=fatal msg="bad listen address format /var/run/docker/libcontainerd/docker-containerd.sock, expected proto://address"
ago 18 16:17:00 host docker[2294]: time="2016-08-18T16:17:00.664023288-05:00" level=info msg="New containerd process, pid: 2368\n"
ago 18 16:17:00 host docker[2294]: time="2016-08-18T16:17:00.67708602-05:00" level=fatal msg="bad listen address format /var/run/docker/libcontainerd/docker-containerd.sock, expected proto://address"
One interesting thing with systemd is that if it thinks that a daemon is running, then the start command does nothing.
I have had to do the following to make sure I cleanly restart certain daemons:
sudo systemctl stop service-name
# wait a little if the service is slow to stop like the Cassandra database
sudo systemctl start service-name
That has worked for me with various services.
One way to know whether the service is considered running, is to check the status like so:
systemctl status service-name
Related
I have couple of docker swarm nodes, When tried to create the service on Leader with below command. Service creation process still going on it is more-than 40 minutes now.
docker service create \
--mode global \
--mount type=bind,src=/project/m32/,dst=/root/m32/ \
--publish mode=host,target=310,published=310 \
--publish mode=host,target=311,published=311 \
--publish mode=host,target=312,published=312 \
--publish mode=host,target=313,published=313 \
--constraint "node.labels.m32 == true" \
--name m32 \
local-registry/ubuntu:07
overall progress: 1 out of 2 tasks
ew0edluvz39p: ready [======================================> ]
kzc7jf7irsrh: running [==================================================>]
From service process, it keep on showing as Ready and Shutdown
$ docker service ps m32
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
s4q0rqrqbpdn m32.ew0edluvz39pazold0wnv2ean local-registry/ubuntu:07 sl-089 Ready Ready 1 second ago
r6vibgptm5oc \_ m32.ew0edluvz39pazold0wnv2ean local-registry/ubuntu:07 sl-089 Shutdown Complete 1 second ago
joq2p6c9jpnx \_ m32.ew0edluvz39pazold0wnv2ean local-registry/ubuntu:07 sl-089 Shutdown Complete 7 seconds ago
a5h8gac02vfx \_ m32.ew0edluvz39pazold0wnv2ean local-registry/ubuntu:07 sl-089 Shutdown Complete 13 seconds ago
f51stfsdlhvp \_ m32.ew0edluvz39pazold0wnv2ean local-registry/ubuntu:07 sl-089 Shutdown Complete 19 seconds ago
zqcbxkm4fwhr m32.kzc7jf7irsrhnx3kurcwqjb2j local-registry/ubuntu:07 sl-090 Ready Ready less than a second ago
za8efvi9x4yw \_ m32.kzc7jf7irsrhnx3kurcwqjb2j local-registry/ubuntu:07 sl-090 Shutdown Complete less than a second ago
$ sudo systemctl status docker.service
Nov 24 19:58:48 svr2 dockerd[2797]: time="2021-11-24T19:58:48.200421563+05:30" level=info msg="ignoring event" container=ea8b76fedb18159ba0cd8f279a9ca4264399c>
Nov 24 20:01:39 svr2 dockerd[2797]: time="2021-11-24T20:01:39.602028420+05:30" level=info msg="NetworkDB stats svr2(00bbf0799aa6) - netID:ubuzyty9mq4tb7xyb>
Nov 24 20:06:39 svr2 dockerd[2797]: time="2021-11-24T20:06:39.802013427+05:30" level=info msg="NetworkDB stats svr2(00bbf0799aa6) - netID:ubuzyty9mq4tb7xyb>
Nov 24 20:11:40 svr2 dockerd[2797]: time="2021-11-24T20:11:40.001992437+05:30" level=info msg="NetworkDB stats svr2(00bbf0799aa6) - netID:ubuzyty9mq4tb7xyb>
Nov 24 20:14:17 svr2 dockerd[2797]: time="2021-11-24T20:14:17.871605342+05:30" level=error msg="Error getting service xkauq9a599iv: service xkauq9a599iv not f>
Nov 24 20:14:52 svr2 dockerd[2797]: time="2021-11-24T20:14:52.833890158+05:30" level=error msg="Error getting service xkauq9a599iv: service xkauq9a599iv not f>
Nov 24 20:15:12 svr2 dockerd[2797]: time="2021-11-24T20:15:12.395692837+05:30" level=error msg="Error getting service pwaa8cvdd683: service pwaa8cvdd683 not f>
Nov 24 20:15:17 svr2 dockerd[2797]: time="2021-11-24T20:15:17.773200054+05:30" level=error msg="Error getting service xk0v0g2roypx: service xk0v0g2roypx not f>
Nov 24 20:16:18 svr2 dockerd[2797]: time="2021-11-24T20:16:18.529344060+05:30" level=error msg="Error getting service xk0v0g2roypx: service xk0v0g2roypx not f>
Nov 24 20:16:40 svr2 dockerd[2797]: time="2021-11-24T20:16:40.201888504+05:30" level=info msg="NetworkDB stats svr2(00bbf0799aa6) - netID:ubuzyty9mq4tb7xyb>
It looks loop process keep on creating containers. What is wrong in my way? Any help to fix this problem will be highly appreciated. Thanks
You really need to pass --restart-max-attempts 5 to your docker service create to ensure that services don't start too many times in a loop. Its bad for the stability of docker, and hard to debug. Rather have a task just give up and stop so you can see something is wrong and diagnose it.
To see specifically what is wrong you would want to look at the logs of each task. You use the individual task id's to see why each one failed:
# The logs for a task
docker service logs s4q0rqrqbpdn
# A general breakdown of a task
docker inspect s4q0rqrqbpdn
Sometimes you need to track down the actual container for the task and inspect that. docker container is not swarm aware, so
# list the service showing the full task id.
docker service ps <service> --no-trunc
# then docker context use <node> / ssh <node> to switch to a node of interest.
# Then, the container name is the "ID"."NAME" from the PS list. For example:
docker context use sl-089
docker container inspect m32.ew0edluvz39pazold0wnv2ean.s4q0rqrqbpdnABCDEFGABCDEFG
Inspecting the container can show if it was killed because of an OOM or certain other reasons that don't otherwise show up.
we have docker-compose.yml that contain configuration for Kafka , zookeeper and schema registry
when we start the docker compose we get the following errors
docker-compose up -d
Starting kafka-docker-final_zookeeper3_1 ... error
ERROR: for kafka-docker-final_zookeeper3_1 Cannot start service zookeeper3: network dd321821f3cb4a715c31e04b32bff2cf206c85ed5581b01b1c6a94ffa45f330e not found
ERROR: for zookeeper3 Cannot start service zookeeper3: network dd321821f3cb4a715c31e04b32bff2cf206c85ed5581b01b1c6a94ffa45f330e not found
ERROR: Encountered errors while bringing up the project.
and
systemctl status docker
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Active: active (running) since Thu 2020-03-19 07:57:29 UTC; 1h 55min ago
Docs: https://docs.docker.com
Main PID: 12105 (dockerd)
Tasks: 30
Memory: 654.6M
CGroup: /system.slice/docker.service
└─12105 /usr/bin/dockerd -H fd:// --containerd=/run/containerd/containerd.sock
Mar 19 07:57:29 master3 dockerd[12105]: time="2020-03-19T07:57:29.610005717Z" level=info msg="Daemon has completed initialization"
Mar 19 07:57:29 master3 dockerd[12105]: time="2020-03-19T07:57:29.631338594Z" level=info msg="API listen on /var/run/docker.sock"
Mar 19 07:57:29 master3 systemd[1]: Started Docker Application Container Engine.
Mar 19 07:58:12 master3 dockerd[12105]: time="2020-03-19T07:58:12.352833676Z" level=warning msg="Error getting v2 registry: Get https://registry-1.docker.io/v2/: net/http: re...ng headers)"
Mar 19 07:58:12 master3 dockerd[12105]: time="2020-03-19T07:58:12.352916724Z" level=info msg="Attempting next endpoint for pull after error: Get https://registry-1.docker.io/...ng headers)"
Mar 19 07:58:12 master3 dockerd[12105]: time="2020-03-19T07:58:12.353019409Z" level=error msg="Handler for POST /v1.22/images/create returned error: Get https://registry-1.do...ng headers)"
Mar 19 08:03:47 master3 dockerd[12105]: time="2020-03-19T08:03:47.255058871Z" level=warning msg="error locating sandbox id 20ce3c5b6383ad92dae848c3de1d91bbfff9306ca86fdc90fae...c not found"
Mar 19 08:03:47 master3 dockerd[12105]: time="2020-03-19T08:03:47.263976715Z" level=error msg="ef808aa411ae0aaef0920397c77b6d9a327bdd1651877402fe1fc142a513af8a cleanup: faile...h container"
Mar 19 09:50:43 master3 dockerd[12105]: time="2020-03-19T09:50:43.920457464Z" level=warning msg="error locating sandbox id 20ce3c5b6383ad92dae848c3de1d91bbfff9306ca86fdc90fae...c not found"
Mar 19 09:50:43 master3 dockerd[12105]: time="2020-03-19T09:50:43.927744636Z" level=error msg="ef808aa411ae0aaef0920397c77b6d9a327bdd1651877402fe1fc142a513af8a cleanup: faile...h container"
Hint: Some lines were ellipsized, use -l to show in full.
regarding to
Cannot start service zookeeper3: network dd321821f3cb4a715c31e04b32bff2cf206c85ed5581b01b1c6a94ffa45f330e not found
Cannot start service zookeeper3: network dd321821f3cb4a715c31e04b32bff2cf206c85ed5581b01b1c6a94ffa45f330e not found
how to fix this issue?
docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6c729cb0bb2c confluentinc/cp-schema-registry:latest "/etc/confluent/dock…" 3 months ago Exited (255) 2 hours ago 0.0.0.0:8081->8081/tcp kafka-docker-schemaregistry_1
ef808aa411ae confluentinc/cp-zookeeper:latest "/etc/confluent/dock…" 3 months ago Exited (255) 2 hours ago
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
docker network ls
NETWORK ID NAME DRIVER SCOPE
e5566ab8ca6d bridge bridge local
2467d9664593 host host local
c509e32d0d67 kafka-docker-default bridge local
08966157382c none null local
we fixed the issue by the following procedure
# docker container ls -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
6c729cb0bb2c confluentinc/cp-schema-registry:latest "/etc/confluent/dock…" 3 months ago Exited (255) 5 hours ago 0.0.0.0:8081->8081/tcp kafka-docker-schemaregistry_1
ef808aa411ae confluentinc/cp-zookeeper:latest "/etc/confluent/dock…" 3 months ago Exited (255) 5 hours ago kafka-docker-zookeeper3_1
# docker container rm 6c729cb0bb2c
# docker container rm ef808aa411ae
systemctl stop docker
systemctl start docker
docker-compose up -d
Creating kafka-docker-zookeeper3_1 ... done
Creating kafka-docker-kafka3_1 ... done
Creating kafka-docker-schemaregistry_1 ... done
docker-compose ps
Name Command State Ports
------------------------------------------------------------------------------------------------------------------------------------------------
kafka-docker-kafka3_1 /etc/confluent/docker/run Up 0.0.0.0:9092->9092/tcp
kafka-docker-schemaregistry_1 /etc/confluent/docker/run Up 0.0.0.0:8081->8081/tcp
kafka-docker-zookeeper3_1 /etc/confluent/docker/run Up 0.0.0.0:2181->2181/tcp, 0.0.0.0:2888->2888/tcp, 0.0.0.0:3888->3888/tcp
I had the same problem and for me it was enough to:
sudo docker-compose down
sudo docker-compose up
Installed Docker 17.x version in RHEL and we are getting below excetption.
-bash-4.2$ docker version
Client:
Version: 17.09.1-ce
API version: 1.32
Go version: go1.8.3
Git commit: 19e2cf6
Built: Thu Dec 7 22:23:40 2017
OS/Arch: linux/amd64
Got permission denied while trying to connect to the Docker daemon socket at unix:///var/run/docker.sock: Get http://%2Fvar%2Frun%2Fdocker.sock/v1.32/version: dial unix /var/run/docker.sock: connect: permission denied
-bash-4.2$
to solve this , we introduce another user group (docker-user) and we added all the users in this group. after that we ran this command and able to ran docker .
sudo systemctl stop docker
sudo systemctl start docker
cd /var/run
sudo chown :docker-user docker.sock
But we are facing another issue that whenever VM is getting restarted ,DOCKER is not running. So we decided to setup run docker as daemon process and we followed
below steps.
1. create docker.conf file under /etc/systemd/system/docker.service.d folder.
2. added this entry in docker.conf file
[Service]
ExecStart=
ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock
ExecStartPost=/bin/chown :docker-user /var/run/docker.sock
After adding this entry and we ran
1. sudo systemctl daemon-reload
2. sudo systemctl stop docker
3. sudo systemctl start docker
We are getting below exception
-bash-4.2$ sudo systemctl status docker.service
● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; disabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/docker.service.d
└─docker.conf
Active: failed (Result: start-limit) since Wed 2018-03-28 09:10:50 PDT; 12s ago
Docs: https://docs.docker.com
Process: 23395 ExecStart=/usr/bin/docker daemon -H tcp://0.0.0.0:2375 -H unix:///var/run/docker.sock (code=exited, status=1/FAILURE)
Main PID: 23395 (code=exited, status=1/FAILURE)
Mar 28 09:10:50 hostname systemd[1]: Failed to start Docker Application Container Engine.
Mar 28 09:10:50 hostname systemd[1]: Unit docker.service entered failed state.
Mar 28 09:10:50 hostname systemd[1]: docker.service failed.
Mar 28 09:10:50 hostname systemd[1]: docker.service holdoff time over, scheduling restart.
Mar 28 09:10:50 hostname systemd[1]: start request repeated too quickly for docker.service
Mar 28 09:10:50 hostname systemd[1]: Failed to start Docker Application Container Engine.
Mar 28 09:10:50 hostname systemd[1]: Unit docker.service entered failed state.
Mar 28 09:10:50 hostname systemd[1]: docker.service failed.
Guide me how to setup docker as daemon process
So, you have already dug in pretty good.
However, this behavior is built into Docker. For user groups, the docker daemon will allow users with the docker group to access the server (important to remember, this is exactly the same as giving root access to any users in that group!). If you wanted to specify a different group, you can start the daemon with the -g option.
Installing docker also installs a systemd service unit to run the daemon. The correct way to enable that (so that it restarts automatically) is
sudo systemctl enable docker
At this point, you didn't include enough of the journalctl logs to actually say why the daemon is not starting for you- if it is an option, I would try starting over since I don't know everything you have messed with, but if that is not an option the journalctl logs will likely explain the problem (probably that the user 'docker' no longer has access to the socket after you chowned it, but that is just a guess)
I am new to Docker, so don't have much idea about it.
I tried restarting Docker service using command
service docker restart
As the command was taking too much of time I did a CTL+C
Now I am not able to start docker deamon
Any docker command gives following op
Cannot connect to the Docker daemon. Is the docker daemon running on this host?
I tried starting Docker deamon using
systemctl start docker
But it outputs:
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
Output of command
**systemctl status docker.service**
`● docker.service - Docker Application Container Engine
Loaded: loaded (/usr/lib/systemd/system/docker.service; enabled; vendor preset: disabled)
Drop-In: /etc/systemd/system/docker.service.d
└─docker.conf, http-proxy.conf, https-proxy.conf
Active: failed (Result: exit-code) since Mon 2018-03-05 17:17:54 IST; 2min 23s ago
Docs: https://docs.docker.com
Process: 11331 ExecStart=/usr/bin/dockerd --graph=/app/dockerRT (code=exited, status=1/FAILURE)
Main PID: 11331 (code=exited, status=1/FAILURE)
Memory: 76.9M
CGroup: /system.slice/docker.service
└─4593 docker-containerd-shim 3bda33eac892d14adda9f3b1fc8dc52173e26ce60ca949075227d903399c7517 /var/run/docker/libcontainerd/3bda33eac892d14adda9f3b1fc8dc52173e26c...
Mar 05 17:17:05 hj-fsbfsd9761.persistent.co.in systemd[1]: Starting Docker Application Container Engine...
Mar 05 17:17:05 hj-fsbfsd9761.persistent.co.in dockerd[11331]: time="2018-03-05T17:17:05.126009059+05:30" level=info msg="libcontainerd: new containerd process, pid: 11337"
Mar 05 17:17:06 hj-fsbfsd9761.persistent.co.in dockerd[11331]: time="2018-03-05T17:17:06.346599571+05:30" level=warning msg="devmapper: Usage of loopback devices is ...section."
Mar 05 17:17:10 hj-fsbfsd9761.persistent.co.in dockerd[11331]: time="2018-03-05T17:17:10.889378989+05:30" level=warning msg="devmapper: Base device already exists an...ignored."
Mar 05 17:17:10 hj-fsbfsd9761.persistent.co.in dockerd[11331]: time="2018-03-05T17:17:10.976695025+05:30" level=info msg="[graphdriver] using prior storage driver \"...mapper\""
Mar 05 17:17:54 hj-fsbfsd9761.persistent.co.in dockerd[11331]: time="2018-03-05T17:17:54.312812069+05:30" level=fatal msg="Error starting daemon: timeout"
Mar 05 17:17:54 hj-fsbfsd9761.persistent.co.in systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Mar 05 17:17:54 hj-fsbfsd9761.persistent.co.in systemd[1]: **Failed to start Docker Application Container Engine.**
Mar 05 17:17:54 hj-fsbfsd9761.persistent.co.in systemd[1]: Unit docker.service entered failed state.
Mar 05 17:17:54 hj-fsbfsd9761.persistent.co.in systemd[1]: docker.service failed.
Hint: Some lines were ellipsized, use -l to show in full.
journalctl -xe
loop: Write error at byte offset 63585648640, length 4096.
How would I be able to start Docker without losing any containers and using previous configurations?
I had the same issue (Fedora 30 x86_64, kernel 5.2.9) and it turned out that being connected to a VPN was the problem. Apparently having a changed gateway address causes an "Error initializing network controller" error which I was able to see when I tried starting docker via sudo dockerd instead of sudo systemctl start docker.
I found the note here about the VPN being a possible problem, disconnecting immediately allowed me to start docker with systemctl start docker.
"Failed to start Docker Application Container Engine" is a general error message.
You should inspect journal for more details:
journalctl -eu docker
In my case it was: "error initializing graphdriver: /var/lib/docker contains several valid graphdrivers: devicemapper, overlay2"
Changing graphdriver to overlay2, fixed it:
$ sudo systemctl stop docker
$ vi /etc/docker/daemon.json # Create the file if it does not exist, and add:
{
"storage-driver": "overlay2"
}
$ sudo systemctl start docker
$ systemctl status docker.service # Hopefully it's running now
I removed the file /etc/docker/daemon.json and started it with sudo systemctl start docker and it worked!!
For the googlers:
For me what worked was doing a killall dockerd and a sudo rm /var/run/docker.pid
I got the error message that recommended that from running dockerd
I ran into this too but all I did after receiving the error was sudo systemctl start docker and then ran sudo systemctl status docker again and that took care of it while on a vpn.
You may have the issue with the current docker installation.
If you don't have much done already you may want to try to reinstall using the install script provided by Docker: https://docs.docker.com/install/linux/docker-ce/ubuntu/#install-using-the-convenience-script this will help you to investigate the errors.
I faced the same problem, in my case the disk was full. I removed docker volumes from /var/lib/docker/volumes/ and started docker with sudo systemctl start docker.
I have followed the installation instructions of Docker CE for CentOS. Initially this worked. At some point the system was restarted and now starting Docker fails. Appreciate expert eyes on this matter...
systemctl start docker produces:
Job for docker.service failed because the control process exited with error code. See "systemctl status docker.service" and "journalctl -xe" for details.
systemctl status docker.service produces:
Apr 21 11:25:23 sec-services-build-1 systemd[1]: Starting Docker Application Container Engine...
Apr 21 11:25:23 sec-services-build-1 dockerd[9693]: time="2017-04-21T11:25:23.370390797+03:00" level=info msg="libcontainerd: previous instance of containerd still alive (8908)"
Apr 21 11:25:23 sec-services-build-1 dockerd[9693]: time="2017-04-21T11:25:23.382492171+03:00" level=warning msg="overlay: the backing xfs filesystem is formatted without d_type support, which leads to incorrect behavior. Reformat the filesystem with ftype=1 to enable d_type support. Running without d_type support will no longer be supported in Docker 17.12."
Apr 21 11:25:23 sec-services-build-1 dockerd[9693]: time="2017-04-21T11:25:23.382547668+03:00" level=info msg="[graphdriver] using prior storage driver: overlay"
Apr 21 11:25:24 sec-services-build-1 dockerd[9693]: Error starting daemon: error while opening volume store metadata database: timeout
Apr 21 11:25:24 sec-services-build-1 systemd[1]: docker.service: main process exited, code=exited, status=1/FAILURE
Apr 21 11:25:24 sec-services-build-1 systemd[1]: Failed to start Docker Application Container Engine.
Apr 21 11:25:24 sec-services-build-1 systemd[1]: Unit docker.service entered failed state.
Apr 21 11:25:24 sec-services-build-1 systemd[1]: docker.service failed.
From here: https://github.com/moby/moby/issues/22507
I ran:
ps axf | grep docker | grep -v grep | awk '{print "kill -9 " $1}' | sudo sh
I was then able to restart docker using:
sudo systemctl start docker
Step 1: systemctl status docker (if docker is running) stop the docker.
step 2: systemctl stop docker.
step 3: dockerd
i got this message when copying volumes from production machine, ended up to overwrite metadata.db inside /var/lib/docker/volumes, then it crashes. A fix is so simple
docker system prune --volumes -f && rm /var/lib/docker/volumes/metadata.db && docker-compose up -d
I encountered the same error.
❶tried
sudo kill -9 1452
multiple times, but it doesn't work. There's still a dockerd process active.
1452 ? Zsl 127:42 [dockerd] <defunct>
❷tried as #Artur Mustafin suggested:
sudo mv /var/lib/docker/volumes/metadata.db /var/lib/docker/volumes/metadata.db.bk
it worked.
so I tried all of these and nothing worked. However what worked was removing all the containers from /var/lib/docker/containers. Then i killed all docker processes (ps -ef | grep docker) then restarted docker and the docker socket. When docker became active I added the containers one at a time and 1 container was what caused the issues