I observe an odd behavior of swarm when I create a service with docker in swarm mode.
basically, I create a service from a private registry, with a binding mount :
docker service create --mount type=bind,src=/some/shared/filesystem/mod_tile,dst=/osm/mod_tile,ro --name="mod_tile" --publish 8082:80 --replicas 3 --with-registry-auth my-registry:5050/repo1/mod_tile
This goes well... and my services are replicated the way I expected...
But When I perform a docker ps on the manager, I see my expected container, as well as an unexpected second container, running from the same image, with a different name :
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
ca33d my-registry:5050/mod_tile:latest "apachectl -D FOREGRâ¦" About a minute ago Up About a minute vigilant_kare.1.fn5u
619e7 my-registry:5050/mod_tile:latest "apachectl -D FOREGRâ¦" 3 minutes ago Up 3 minutes mod_tile.3.dyismrc
4f1ebf demo/demo-tomcat:0.0.1 "./entrypoint.sh" 7 days ago Up 7 days (healthy) 9900/tcp, 0.0.0.0:8083->8080/tcp tomcatgeoserver
d3adf some.repo:5000/manomarks/visualizer:latest "npm start" 8 days ago Up 8 days 8080/tcp supervision_visualizer.1.ok27kbz
673c1 some.repo:5000/grafana/grafana:latest "/run.sh" 8 days ago Up 8 days 3000/tcp supervision_grafana.1.pgqko8 some.repo:5000/portainer:latest "/portainer --externâ¦" 8 days ago Up 8 days 9000/tcp supervision_portainer.1.vi90w6
bd9b1 some.repo:5000/prom/prometheus:latest "/bin/prometheus -coâ¦" 8 days ago Up 8 days 9090/tcp supervision_prometheus.1.j4gyn02
d8a8b some.repo:5000/cadvisor:0.25.0 "/usr/bin/cadvisor -â¦" 8 days ago Up 8 days 8080/tcp supervision_cadvisor.om7km
bd46d some.repo:5000/prom/node-exporter:latest "/bin/node_exporter â¦" 8 days ago Up 8 days 9100/tcp supervision_nodeexporter.om7kmd
04b53 some.repo:5000/sonatype/nexus3 "sh -c ${SONATYPE_DIâ¦" 9 days ago Up 2 hours 0.0.0.0:5050->5050/tcp, 0.0.0.0:8081->8081/tcp nexus_registry
At first, I thought it was a remaining container from previous attempts, so I stoped it... but a few seconds later, it was up again! No matter how many time I stop it, it will be restarted.
So, I guess it is there on purpose... but I don't understand : I already have my 3 replicas running (I checked on all nodes), and even though I promote another node, the extra container appears only on the leader...
This may come from one of my other containers (used for supervision), but so far, I couldn't figure out from which one...
Does any one have an idea why this extra container is created?
EDIT 05/07
Here are the result of a docker ps on the mod_tile service. The 3 replicas are here, one one each node. The extra service is not considered by the "ps" command.
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
c77gc mod_tile.1 my-registry:5050/mod_tile:latest VM3 Running Running 15 hours ago
u7465 mod_tile.2 my-registry:5050/mod_tile:latest VM4 Running Running 15 hours ago
dyism mod_tile.3 my-registry:5050/mod_tile:latest VM2 Running Running 15 hours ago
It looks like you have a second service defined with the name "vigilant_kare", possibly automatically named if you didn't provide a name.
Swarm mode will automatically correct a down or deleted container to return you to the target state. To delete a container managed by swarm mode, you need to delete the service that manages it:
docker service rm vigilant_kare
Related
When i run docker save nifi > nifi_backup.tar i get
Error response from daemon: No such image: nifi
When i run docker ps -a i got
$docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
a8796cbf6cb1 apache/nifi:1.9.2 "../scripts/start.sh" 11 months ago Exited (0) 33 minutes ago nifi
f4ecf6ca0d16 apache/nifi "../scripts/start.sh" 12 months ago Exited (0) 12 months ago nervous_poincare
9a68c235bb3a apache/nifi "../scripts/start.sh" 12 months ago Exited (0) 12 months ago jolly_mendel
d608287fe560 superset_superset "/entrypoint.sh" 2 years ago Up 27 hours (healthy) 0.0.0.0:8088->8088/tcp, :::8088->8088/tcp superset_superset_1
fcea620b1983 postgres:10 "docker-entrypoint.s…" 2 years ago Up 27 hours 0.0.0.0:5433->5432/tcp, :::5433->5432/tcp superset_postgres_1
380782e0a024 redis:3.2 "docker-entrypoint.s…" 2 years ago Up 27 hours 0.0.0.0:6379->6379/tcp, :::6379->6379/tcp superset_redis_1
Also, i cant start nifi image
if i run docker start nifi
it try to start about a minute, but then stop silently.
And there is no logs in result.
docker events says exit code = 0:
Maybe it has some relation for unavailable saving, so i inform about it too.
How to fix No such image in such a case?
Use the command docker images to see if there is an image named nifi. If not first build that image and than execute the command:
docker save nifi > nifi_backup.tar
It should not show the error No such image: nifi if the command docker images has an image named nifi.
As per documentation here, docker save command expects an image name, not a container. You're probably looking for docker export (export docs) command to achieve what you want.
The error about starting your container could be container-specific. As it exited already a year ago, maybe data is lost and the container cannot be started again. Using command docker events&, you can start the docker event listener in the background. This way you can obtain the hex value of this very start attempt and use it to search specific logs: docker logs <startId hex>. Maybe it can tell you more details about why the container did not properly start.
I'm forced to use elastic beanstalk (eb) and Docker in deploying. When I build & run my container locally it boots up and runs well. I'm using supervisord to boot some ruby code (clockwork and Rails/puma)
When deploying using eb, I see how eb spawns several consecutive containers until all just chokes down:
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
232bbe498977 a4a6fd70537b "supervisord -c /etc…" About a minute ago Up About a minute 80/tcp silly_williams
a9e21774575e a4a6fd70537b "supervisord -c /etc…" 2 minutes ago Up 2 minutes 80/tcp trusting_murdock
945f51ef510f a4a6fd70537b "supervisord -c /etc…" 3 minutes ago Up 3 minutes 80/tcp blissful_stonebraker
6e51470ddce8 a4a6fd70537b "supervisord -c /etc…" 4 minutes ago Up 4 minutes 80/tcp lucid_ramanujan
2689568ceb6d a4a6fd70537b "supervisord -c /etc…" 4 minutes ago Up 4 minutes 80/tcp keen_mestorf
Where should I be looking for the root to this behavior? Can the container be creating this behaviour or is eb configured in a wrong way?
(I apologize that I'm a bit too unspecific with details since I'm not in full control of the environment)
I eventually realized I had been tampering with some settings, and had set monitoring to basic. Once put to Enhanced it only booted one container and things started to work again!
In:
Elastic Beanstalk > [my application] > Configuration > monitoring > System: Enhanced.
I was just playing around with docker!!
Ran ps command -a option
docker run -a
C:\Users\sarkan1>docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
3d6db385321c busybox:1.24 "sleep 1000" 26 seconds ago Up 24 seconds lucid_ramanujan
e6acd65398b2 hello-world "/hello" 43 minutes ago Exited (0) 43 minutes ago nifty_brattain
c5576137580d hello-world "/hello" 4 days ago Exited (0) 4 days ago dreamy_aryabhata
2594fbf1fa82 hello-world "/hello" 4 days ago Exited (0) 4 days ago nostalgic_hopper
c0102bc64c45 hello-world "/hello" 4 days ago Exited (0) 4 days ago vibrant_khorana
c4af79ea96e9 hello-world "/hello" 4 days ago Exited (0) 4 days ago cranky_heyrovsky
Questions :
Why did I get the values in the names column always different? I ran the same container I guess!!
What is command column? Does the slash before the hello (/hello) have any significance?
Unless you launch a container specifying its name (docker run --name), docker will generate one for you.
That is easier than referencing said container with its ID.
You can then make docker commands with its (generated) name instead.
See more at "How does Docker generate default container names?".
(and the source code: moby/moby pkg/namesgenerator/names-generator.go)
The command column is the full command executed by the container: once this command stops, the container stops.
That command is specified in the docker image, and is a combination of:
Dockerfile ENTRYPOINT
Dockerfile CMD
(See "difference between cmd and entrypoint in dockefile")
Basically they are random container names generated!! The left and right parts of the names are hardcoded and can be found in
https://github.com/moby/moby/blob/master/pkg/namesgenerator/names-generator.go
Docker engine generates it if default name is not provided when you create a new docker container.
I have several docker containers running:
docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
736caaa764f4 ubuntu "/bin/bash" 2 hours ago Up 2 hours quirky_morse
e2869c98ee1a ubuntu "/bin/bash" 2 hours ago Up 2 hours sleepy_wilson
e4149472a2da ubuntu "/bin/bash" 2 hours ago Up 2 hours cranky_booth
70bb44ac5d24 grafana/grafana "/run.sh" 2 hours ago Up 2 hours 0.0.0.0:3000->3000/tcp microservicemonitoring_grafana_1
e4b30881a83e prom/prometheus "/bin/prometheus -..." 2 hours ago Up 2 hours 0.0.0.0:9090->9090/tcp prometheus
281f792380f9 prom/node-exporter "/bin/node_exporte..." 2 hours ago Up 2 hours 9100/tcp node-exporter
17810c718b29 google/cadvisor "/usr/bin/cadvisor..." 2 hours ago Up 2 hours 8080/tcp microservicemonitoring_cadvisor_1
77711de421e2 prom/alertmanager "/bin/alertmanager..." 2 hours ago Up 2 hours 0.0.0.0:9093->9093/tcp microservicemonitoring_alertmanager_1
What I want to do is to build graphs for containers filtered by name and image.
Example: built from ubuntu container (quirky_morse, sleepy_wilson, cranky_booth) and prometheus container.
I can filter containers by image with this type of query:
sum by (name) (rate(container_network_receive_bytes_total{image="ubuntu"} [1m] ) )
As you can see I get graphs of three containers (flatlines because they a re doing nothing).
Now I want to add additional filter parameter name and it dows not work
sum by (name) (rate(container_network_receive_bytes_total{image="ubuntu", name="prometheus"} [1m] ) )
What I want to get is: three graphs for containers derived from image "ubuntu" and the one with name "prometheus" no matter the origin image
You can't do this with one selector.
The proper solution here is to use Grafana, which supports graphing multiple expressions on one graph.
At this level the best you can do is rate(container_network_receive_bytes_total{image="ubuntu"} [1m] or rate(container_network_receive_bytes_total{name="prometheus"}[1m]
I've been toying with a docker image for Tensorflow.
To summarize, I first installed the standard image, then realized I needed nodejs, so added it and did a docker commit. Then realized I needed expressJS, added it an did a commit
I am running docker v1.12.5 (so the new gc/prune commands are not there)
At this stage, docker images -a shows:
REPOSITORY TAG IMAGE ID CREATED SIZE
tensor-node-express latest f2f59eb61aae 15 hours ago 2.104 GB
gcr.io/tensorflow/tensorflow latest-devel 308238445d5c 2 days ago 1.995 GB
gcr.io/tensorflow/tensorflow <none> 74435614a991 9 days ago 1.52 GB
I only want to keep tensor-node-express and delete the older images.
$ docker rmi 308238445d5c
Error response from daemon: conflict: unable to delete 308238445d5c (cannot be forced) - image has dependent child images
$docker rmi gcr.io/tensorflow/tensorflow:latest-devel
Error response from daemon: conflict: unable to remove repository reference "gcr.io/tensorflow/tensorflow:latest-devel" (must force) - container 03de9d864e31 is using its referenced image 308238445d5c
I assumed that this means docker commits store differential images, but when I go to ~/.docker/machine/machines/default, I see:
40894464 Mar 13 13:57 boot2docker.iso
5043847168 Mar 16 08:34 disk.vmdk
I suppose the 5G file is a composite of my images, which seems to show each docker commit is the full image!
Any thoughts on how I can only use the latest docker image (tensor-node-express) and free my HD of the invasion of docker?
Supplementary info - here is the output of docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e6dcd2915991 tensor-node-express "/bin/bash" 15 hours ago Exited (130) 15 hours ago flamboyant_bose
fb44b19a21c2 gcr.io/tensorflow/tensorflow:latest-devel "/bin/bash" 18 hours ago Exited (130) 15 hours ago compassionate_bose
075001a687e3 gcr.io/tensorflow/tensorflow:latest-devel "/bin/bash" 18 hours ago Exited (0) 18 hours ago nervous_sinoussi
a80ce2d2e688 gcr.io/tensorflow/tensorflow:latest-devel "/bin/bash" 19 hours ago Exited (130) 18 hours ago happy_euclid
f493bd3c8712 gcr.io/tensorflow/tensorflow:latest-devel "/bin/bash" 19 hours ago Exited (1) 19 hours ago friendly_cori
03de9d864e31 gcr.io/tensorflow/tensorflow:latest-devel "/bin/bash" 2 days ago Exited (255) 23 minutes ago 6006/tcp, 8888/tcp tender_hopper
2dd1e83d62d3 gcr.io/tensorflow/tensorflow:latest-devel "/bin/bash" 2 days ago Exited (0) 15 hours ago modest_einstein
3067ed171b1c gcr.io/tensorflow/tensorflow:latest-devel "/bin/bash" 2 days ago Exited (0) 2 days ago dazzling_bhabha
62c699afd3fd 74435614a991 "/bin/bash" 2 days ago Exited (127) 2 days ago inspiring_austin
9523ffe2945c 74435614a991 "/bin/bash" 2 days ago Exited (0) 2 days ago kickass_leakey
e06958ea517c 74435614a991 "/bin/bash" 2 days ago Exited (0) 2 days ago objective_euler
ccf922954667 74435614a991 "/bin/bash" 2 days ago Exited (255) 2 days ago dreamy_bartik
fad0d92a07a3 74435614a991 "/bin/bash" 2 days ago Exited (130) 2 days ago elastic_dubinsky
f2a98d4e11ea 74435614a991 "/bin/bash" 2 days ago Exited (0) 2 days ago heuristic_kilby
f07e46367b17 74435614a991 "/bin/bash" 2 days ago Exited (130) 2 days ago trusting_darwin
5bbf9cf992b8 74435614a991 "/bin/bash" 2 days ago Exited (0) 2 days ago flamboyant_knuth
I tried
docker ps --filter "status=exited" | grep "days ago" | awk '{print $1}' | xargs docker rm (credit)
I ran the above manually as well for some of the containers it missed
That pruned the ps list to:
docker ps -a
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
e6dcd2915991 tensor-node-express "/bin/bash" 15 hours ago Exited (130) 15 hours ago flamboyant_bose
But even then I can't delete old images - same error.
Further update, I tried to list dependencies in images (credit)
using this script:
for i in $(docker images -q)
do
docker history $i | grep -q 74435614a991 && echo $i
done | sort -u
And it told me:
308238445d5c
74435614a991
f2f59eb61aae
This means my new images are child images of the old image. But the size is not a differential looking at the disk size.
Thoughts?
docker-machine uses a Linux VM
When you looked at the docker-machine .vdmk and .iso files, what you are looking at is files for a Linux VM running on your Mac. This is needed because Docker requires Linux kernel features to run, it cannot run directly on the Mac's microkernel.
So your Mac is running a Linux virtual machine, and inside that virutal machine is running the Docker daemon and all of your containers.
Therefore the file size of the .vmdk and .iso tell you nothing about any one image.
docker images have parent/child relationships
As you may already know, docker images have parents and/or children. For instance when you build an image with a Dockerfile like this:
FROM ubuntu:latest
RUN apt-get update && apt-get install nginx
You will end up with a new image that you have perhaps tagged my-nginx. But it requires the ubuntu:latest image as its parent; you cannot delete ubuntu:latest with this image still around, as it requires its parent.
docker commit creates those relationships
When you use docker commit, you are basically doing a dynamic snapshot build. It is similar to the above, except there's no Dockerfile involved.
The above example has a FROM line which indicates the image to use as a base. When using commit, there is a base implied - whatever image was used to launch the running container that you are committing.
The above example has a RUN command which will create new contents in the built image, above and beyond the base image. In a real Dockerfile there are usually multiple commands that do various things which build on the base image. When you use commit, you don't have that. Instead, anything that has been written to the container on top of the base image is your new content. It exists in a read-write filesystem layer in the container. That is the thing you are committing; it is written as a new read-only layer and you get that back as a new (immutable, read-only) docker image. With a parent.
Based on your comments, and the question itself, you appear to have believed that using docker commit would create a new full image that had no dependencies on other images. That is not true. You can craft images like that if you build them yourself from scratch, just not this way.
You can untag the image
If what you want is for the image to not show up in your list, that's easy. Just untag it.
docker rmi gcr.io/tensorflow/tensorflow:latest-devel
However, this is more or less cosmetic. The image will still be there, as another image requires it. All this does is remove the tag, so it doesn't appear in the docker images list anymore without the -a flag.
The reason trying this did not work for you is you tried to rmi the image using its ID, not using its tag.