I use this method below to port data out of one container.
docker run --volumes-from <data container> ubuntu tar -cO <volume path> | gzip -c > volume.tgz
But there is one problem with it is every time it performs a backup, there will be a zombie container left. What is the good way to get that id and remove the zombie container afterward.
Thanks
Apparently, you just want to be able to export volume data. To do that, you just need to start your initial container with a volume pointing to a directory on the host with the -v option. You can tar on the host without creating a container for it. Your current tactic seems a bit over-engineered ;)
The easy way to remove the container after executing the command, is to use the option --rm, from here
However, if you feel that the container you are creating will have data that you will need to
1. update in real time
2. access after the container has been created
then you may also mount a host directory as a container volume and access the contents of that directory from the host.
If you start a container using the -volume option, you can also call reference the directory created on this host
$ docker run -v /volume_directory ubuntu
$ container=$(docker ps -n=1 -q)
$ docker inspect -f '{{.Volumes}}' $container
Related
Running the docker registry with below command always throws an error:
dev:tmp me$ docker run \
-d --name registry-v1 \
-e SETTINGS_FLAVOR=local \
-e STORAGE_PATH=/registry \
-e SEARCH_BACKEND=sqlalchemy \
-e LOGLEVEL=DEBUG \
-p 5000:5000 \
registry:0.9.1
Error response from daemon: Conflict. The name "registry-v1" is already in use by container f9e5798a82e0. You have to delete (or rename) that container to be able to reuse that name.
How can I prevent this error ?
I got confused by this also. There are two commands relevant here:
docker run # Run a command in a **new** container
docker start # Start one or more stopped containers
That means you have already started a container in the past with the parameter
docker run --name registry-v1 ...
You need to delete that first before you can re-create a container with the same name with
docker rm registry-v1
When that container is sill running you need to stop it first before you can delete it with
docker stop registry-v1
Or simply choose a different name for the new container.
To get a list of existing containers and their names simply invoke
docker ps -a
Here what i did, it works fine.
step 1:(it lists docker container with its name)
docker ps -a
step 2:
docker rm name_of_the_docker_container
When you are building a new image you often want to run a new container each time and with the same name. I found the easiest way was to start the container with the --rm option:
--rm Automatically remove the container when it exits
e.g.
docker run --name my-micro-service --rm <image>
Sadly it's used almost randomly in the examples from the docs
Edit: Read Lepe's comment below.
Just to explain what others are saying (it took me some time to understand) is that, simply put, when you see this error, it means you already have a container and what you have to do is run it. While intuitively docker run is supposed to run it, it doesn't. The command docker run is used to only START a container for the very first time. To run an existing container what you need is docker start $container-name. So much for asking developers to create meaningful/intuitive commands.
You have 2 options to fix this...
Remove previous container using that name, with the command docker rm $(docker ps -aq --filter name=myContainerName)
OR
Rename current container to a different name i.e change this portion --name registry-v1 to something like --name myAnotherContainerName
You are getting this error because that container name ( i.e registry-v1) was used by another container in the past...even though that container may have exited i.e (currently not in use).
Cause
A container with the same name is still existing.
Solution
To reuse the same container name, delete the existing container by:
docker rm <container name>
Explanation
Containers can exist in following states, during which the container name can't be used for another container:
created
restarting
running
paused
exited
dead
You can see containers in running state by using :
docker ps
To show containers in all states and find out if a container name is taken, use:
docker ps -a
Here is how I solved this on ubuntu 18:
$ sudo docker ps -a
copy the container ID
For each container do:
$ sudo docker stop container_ID
$ sudo docker rm container_ID
removing all the exited containers
docker rm $(docker ps -a -f status=exited -q)
The Problem: you trying to create new container while in background container with same name is running and this situation causes conflicts.
The error would be like:
Cannot create continer for service X :Conflict. The name X is already in use by container abc123xyz. You have to remove ot delete (or rename) that container to be able to reuse that name.
Solution rename the service name in docker-compose.yml or delete the running container and rebuild it again (this solution related to Unix/Linux/macOS systems):
get all running containers sudo docker ps -a
get the specific container id
stop and remove the duplicated container / force remove it
sudo docker stop <container_id>
sudo docker rm <container_id>
or
sudo docker rm --force <container_id>
You can remove it with command sudo docker rm YOUR_CONTAINER_ID, then run a new container with sudo docker run ...;
or restart an existing container with sudo docker start YOUR_CONTAINER_ID
I was running into this issue that when I run docker rm (which usually works) I would get:
Error: No such image
The easiest solution to this is removing all stopped containers by running:
docker container prune
I have solved the issue by doing following steps and I hope it helps.
Type docker ps -a to list all the containers in your system.
Check the NAMES part where you have initialized your docker container.
Then type docker rm --force name_of_container
Install the docker container as you wish.
I had problem using NIFI and I have removed and reinstalled using docker. Good luck.
TL:DR;
List all containers:
docker ps -a
Remove the concerned container by id:
docker container rm <container_id>
I'm just learning docker and this got me as well. I stopped the container with that name already and therefore I thought I could run a new container with that name.
Not the case. Just because the container is stopped, doesn't mean it can't be started again, and it keeps all the same parameters that it was created with (including the name).
when I ran docker ps -a that's when I saw all the dummy test containers I created while I was playing around.
No problem, since I don't want those any more I just did docker rm containername at which point my new container was allowed to run with the old name.
Ah, and now that I finish writing this answer, I see Slawosz's comment on Walt Howard's answer above suggesting the use of docker ps -a
The OP's problem is the error. Deleting state isn't the only solution - or even a good one. The problem is docker run isn't re-entrant, and docker start is impotent w/o run. So we have to combine them.
For example to run Postgres w/o destroying previous state, try this:
docker start postgres || docker run -d -p 5432:5432 --name postgres -e POSTGRES_PASSWORD=password postgres:13-alpine
Ok, so I didn't understand either, then I left my pc, went to do other things, and upon my return, it clicked :D
You download a docker image file. docker pull *image-name* will just pull the image from docker hub without running it.
Now, you use docker run, and give it a name (e.g. newWebServer).
docker run -d -p 8080:8080 -v volume --name newWebServer image-name/version
You perhaps only need docker run --name *name* *image*, but the other stuff will become useful quickly.
-d (detached) - means the container will exit when the root process used to run the container exits.
-p (port) - specify the container port and the host port. Kind of the internal and external port. The internal one being the port the container uses, and the external one is the port you use outside of it and probably the one you need to put in your web browser if that's how you access your app.
--name (what you want to call this instance of the container) - you could have several instances of the same container all with different names, which is useful when you're trying to test something.
image-name/version is the actual image you want to create the container from. You can see a list of all the images on your system with docker images -a. You may have more than one version, so make sure you choose the correct one/tag.
-v (volume) - perhaps not needed initially, but soon you'll want to persist data after your container exits.
OK. So now, docker run just created a container from your image. If it isn't running, you can now start it with it's name:
docker start newWebServer
You can check all your containers (they may or may not be running) with
docker ps -a
You can stop and start them (or pause them) with their name or the container id (or just the first few characters of it) from the CONTAINER ID column e.g:
docker stop newWebServer
docker start c3028a89462c
And list all your images, with
docker images -a
In a nutshell, download an image; docker run creates a container from it; start it with docker start (name or container id); stop it with docker stop (name or container id).
I had this issue because I had two or more containers with the same container_name in the docker-compose.yml file.
Simple Solution: Goto your docker folder in the system and delete .raw file or docker archive with large size.
For me, the issue was that I used an image name more than once in the dockerfile.
This happened to me on the docker tutorial! The port I tried to use was taken, but docker still created.. an image? A process to run docker? I'll find out soon. Anyways, to choose a different port, I had to remove the older image, and then docker run again.
Sometimes a tutorial can be too terse. What you want is concise, not terse, or even succinct.
I have a docker image that runs TensorFlow/Keras on my GPU.
I start the session like this which opens a Jupyter Notebook -
sudo systemctl daemon-reload
sudo systemctl restart docker
sudo docker run --rm --runtime=nvidia -v -it -p 8888:8888 tensorflow/
tensorflow:2.0.0a0-gpu-py3-jupyter
The issue is everytime I start it, it will launch a fresh instance and all previous session's work will be lost. How do I maintain the state every time I start it?
You can start a container like:
sudo docker run --runtime=nvidia -it -v /homepath:/workspace -p 8888:8888 tensorflow/tensorflow:2.0.0a0-gpu-py3-jupyter
where /hometpath is the folder that you want to share between your machine and the docker container, /workspace is the path inside the container where /homepath will be copied.
Then after you disconnect you can type :
docker attach name
where name is a name of a container that you can find by typing:
docker container ls
The issue is everytime I start it, it will launch a fresh instance and all previous session's work will be lost. How do I maintain the state every time I start it?
A container is ephemeral...when you call docker run, you are creating a new container that doesn't inherit any state from previous containers. If you want to maintain data outside the lifecycle of the container, then you use a docker volume.
You can either allocate a named volume from docker, or bind-mount a host directory into your container.
You will need to discover the appropriate path inside the container at which to mount the storage.
I have the following Dockerfile :
FROM jboss/wildfly
USER jboss
RUN mkdir -p /opt/jboss/wildfly/standalone/log
VOLUME /opt/jboss/wildfly/standalone/log
CMD /bin/bash
# CMD true
This resulting image is started with docker run -ti --name=data_volume data/volume. The next Dockerfile
FROM jboss/wildfly
RUN sed -i 's|<file relative-to="jboss.server.log.dir"
path="server.log"/>|\<file relative-to="jboss.server.log.dir"
path="\${jboss.host.name}-server.log"/\>|'
/opt/jboss/wildfly/standalone/configuration/standalone.xml
overrides the logging of the resulting jboss to log to "servername"-server.log in the logging dir. When I start the resulting image with docker run -ti --name=wild-01 --volumes-from=data_volume my/wildfly and docker run -ti --name=wild-02 --volumes-from=data_volume my/wildfly I have two log files in my data_colume container. So fine so good.
I would like to point my volume to a directory on the host eg. /var/log/wildfly.
How can I achieve this in Dockerfiles and not with the -v parameter when running data/volume
Thanks a lot in advance
Inside dockerfiles you can only define volumes in /var/lib/docker/volumes. This is because every host can be different from the other.
Docker uses /var/lib/docker as "docker area" where it stores all docker-related data. It's the directory that's guaranteed on every host because it gets created on installation.
If you were to point out a volume in the dockerfile, let's say to /home/mbieren/docker_vol, the image would result in multiple errors when executed on a different host, as that directory does not exist and the user probably has insufficient permissions to create it.
Docker goes around that problem by not allowing custom mount-paths to be set in the dockerfile.
I would like to point my volume to a directory on the host eg. /var/log/wildfly.
remove all mention of volumes from your Dockerfile ... launch your container using
docker run -d -v /var/log/wildfly:/var/log/wildfly your-image-name
then in your code just reference the normal path
/var/log/wildfly
Your syntax to launch the container using docker run -ti makes the container shell interactive whereas -d is the normal mode to spin it up as a daemon running in the background
I'm new to docker, and I'm trying mount the root directory of docker container as a NFS mount point.
for example, I had a NFS mount point test:/home/user/3243, and I'm trying:
docker run -it -v "test:/home/user/3243":/ centos7 /bin/bash
absolutely, it's failed. So I tried this:
mount -t nfs test:/home/user/3243 /mnt/nfs/3243
docker run -it -v /mnt/nfs/3243:/ centos7 /bin/bash
but failed again, so how to do this? Could it be worked out?
A couple of issues here:
You cannot mount to the root directory of a container. So docker run -v /foo:/ will never work.
With the syntax of your first attempt, -v test:/foo:bar, Docker would see this as wanting to create a "named" volume called "test".
You should be able to first do the NFS mount, then do docker run -v /mnt/nfs/3243:/foo to have the nfs path mounted to /foo.
But again, you can't mount to /.
That is currently discussed (since mid 2014) in issue 4213.
One recent workaround by Jeroen van Bemmel (jbemmel) was:
It appears that NFS functionality depends on the underlying storage driver ( aufs, devicemapper, etc. ), as well as the sharing of file handles between processes ( see blog post "docker: devicemapper fix for “device or resource busy” (EBUSY)") i.e. 'unshare' may have an impact on NFS mounts.
I've moved away from using the 'MOUNTPOINT=/vm/nfs' as I am not sure if that event is even emitted.
Instead I created an upstart file like this:
cat > /etc/init/ecdn.conf << EOF
description "eCDN container"
author "Jeroen van Bemmel"
# mounted MOUNTPOINT=/vm/nfs doesn't seem to work, at least not the first time
start on started docker and virtual-filesystems
stop on starting rc RUNLEVEL=[016]
respawn
script
exec /usr/bin/docker start -a ecdn
end script
pre-stop script
/usr/bin/docker stop ecdn
# dont /usr/bin/docker rm ecdn
end script
EOF
and then create the container like this:
script -c "docker create -it --name='ecdn' --volume /vm:/usr/share/nginx/html/vm:ro image/name"
Running the docker registry with below command always throws an error:
dev:tmp me$ docker run \
-d --name registry-v1 \
-e SETTINGS_FLAVOR=local \
-e STORAGE_PATH=/registry \
-e SEARCH_BACKEND=sqlalchemy \
-e LOGLEVEL=DEBUG \
-p 5000:5000 \
registry:0.9.1
Error response from daemon: Conflict. The name "registry-v1" is already in use by container f9e5798a82e0. You have to delete (or rename) that container to be able to reuse that name.
How can I prevent this error ?
I got confused by this also. There are two commands relevant here:
docker run # Run a command in a **new** container
docker start # Start one or more stopped containers
That means you have already started a container in the past with the parameter
docker run --name registry-v1 ...
You need to delete that first before you can re-create a container with the same name with
docker rm registry-v1
When that container is sill running you need to stop it first before you can delete it with
docker stop registry-v1
Or simply choose a different name for the new container.
To get a list of existing containers and their names simply invoke
docker ps -a
Here what i did, it works fine.
step 1:(it lists docker container with its name)
docker ps -a
step 2:
docker rm name_of_the_docker_container
When you are building a new image you often want to run a new container each time and with the same name. I found the easiest way was to start the container with the --rm option:
--rm Automatically remove the container when it exits
e.g.
docker run --name my-micro-service --rm <image>
Sadly it's used almost randomly in the examples from the docs
Edit: Read Lepe's comment below.
Just to explain what others are saying (it took me some time to understand) is that, simply put, when you see this error, it means you already have a container and what you have to do is run it. While intuitively docker run is supposed to run it, it doesn't. The command docker run is used to only START a container for the very first time. To run an existing container what you need is docker start $container-name. So much for asking developers to create meaningful/intuitive commands.
You have 2 options to fix this...
Remove previous container using that name, with the command docker rm $(docker ps -aq --filter name=myContainerName)
OR
Rename current container to a different name i.e change this portion --name registry-v1 to something like --name myAnotherContainerName
You are getting this error because that container name ( i.e registry-v1) was used by another container in the past...even though that container may have exited i.e (currently not in use).
Cause
A container with the same name is still existing.
Solution
To reuse the same container name, delete the existing container by:
docker rm <container name>
Explanation
Containers can exist in following states, during which the container name can't be used for another container:
created
restarting
running
paused
exited
dead
You can see containers in running state by using :
docker ps
To show containers in all states and find out if a container name is taken, use:
docker ps -a
Here is how I solved this on ubuntu 18:
$ sudo docker ps -a
copy the container ID
For each container do:
$ sudo docker stop container_ID
$ sudo docker rm container_ID
removing all the exited containers
docker rm $(docker ps -a -f status=exited -q)
The Problem: you trying to create new container while in background container with same name is running and this situation causes conflicts.
The error would be like:
Cannot create continer for service X :Conflict. The name X is already in use by container abc123xyz. You have to remove ot delete (or rename) that container to be able to reuse that name.
Solution rename the service name in docker-compose.yml or delete the running container and rebuild it again (this solution related to Unix/Linux/macOS systems):
get all running containers sudo docker ps -a
get the specific container id
stop and remove the duplicated container / force remove it
sudo docker stop <container_id>
sudo docker rm <container_id>
or
sudo docker rm --force <container_id>
You can remove it with command sudo docker rm YOUR_CONTAINER_ID, then run a new container with sudo docker run ...;
or restart an existing container with sudo docker start YOUR_CONTAINER_ID
I was running into this issue that when I run docker rm (which usually works) I would get:
Error: No such image
The easiest solution to this is removing all stopped containers by running:
docker container prune
I have solved the issue by doing following steps and I hope it helps.
Type docker ps -a to list all the containers in your system.
Check the NAMES part where you have initialized your docker container.
Then type docker rm --force name_of_container
Install the docker container as you wish.
I had problem using NIFI and I have removed and reinstalled using docker. Good luck.
TL:DR;
List all containers:
docker ps -a
Remove the concerned container by id:
docker container rm <container_id>
The OP's problem is the error. Deleting state isn't the only solution - or even a good one. The problem is docker run isn't re-entrant, and docker start is impotent w/o run. So we have to combine them.
For example to run Postgres w/o destroying previous state, try this:
docker start postgres || docker run -d -p 5432:5432 --name postgres -e POSTGRES_PASSWORD=password postgres:13-alpine
I'm just learning docker and this got me as well. I stopped the container with that name already and therefore I thought I could run a new container with that name.
Not the case. Just because the container is stopped, doesn't mean it can't be started again, and it keeps all the same parameters that it was created with (including the name).
when I ran docker ps -a that's when I saw all the dummy test containers I created while I was playing around.
No problem, since I don't want those any more I just did docker rm containername at which point my new container was allowed to run with the old name.
Ah, and now that I finish writing this answer, I see Slawosz's comment on Walt Howard's answer above suggesting the use of docker ps -a
Ok, so I didn't understand either, then I left my pc, went to do other things, and upon my return, it clicked :D
You download a docker image file. docker pull *image-name* will just pull the image from docker hub without running it.
Now, you use docker run, and give it a name (e.g. newWebServer).
docker run -d -p 8080:8080 -v volume --name newWebServer image-name/version
You perhaps only need docker run --name *name* *image*, but the other stuff will become useful quickly.
-d (detached) - means the container will exit when the root process used to run the container exits.
-p (port) - specify the container port and the host port. Kind of the internal and external port. The internal one being the port the container uses, and the external one is the port you use outside of it and probably the one you need to put in your web browser if that's how you access your app.
--name (what you want to call this instance of the container) - you could have several instances of the same container all with different names, which is useful when you're trying to test something.
image-name/version is the actual image you want to create the container from. You can see a list of all the images on your system with docker images -a. You may have more than one version, so make sure you choose the correct one/tag.
-v (volume) - perhaps not needed initially, but soon you'll want to persist data after your container exits.
OK. So now, docker run just created a container from your image. If it isn't running, you can now start it with it's name:
docker start newWebServer
You can check all your containers (they may or may not be running) with
docker ps -a
You can stop and start them (or pause them) with their name or the container id (or just the first few characters of it) from the CONTAINER ID column e.g:
docker stop newWebServer
docker start c3028a89462c
And list all your images, with
docker images -a
In a nutshell, download an image; docker run creates a container from it; start it with docker start (name or container id); stop it with docker stop (name or container id).
I had this issue because I had two or more containers with the same container_name in the docker-compose.yml file.
Simple Solution: Goto your docker folder in the system and delete .raw file or docker archive with large size.
For me, the issue was that I used an image name more than once in the dockerfile.
This happened to me on the docker tutorial! The port I tried to use was taken, but docker still created.. an image? A process to run docker? I'll find out soon. Anyways, to choose a different port, I had to remove the older image, and then docker run again.
Sometimes a tutorial can be too terse. What you want is concise, not terse, or even succinct.