I'm using docker-compose and docker swarm. I have created a docker-mgt server from which I did a docker-machine to setup a swarm cluster consisting of a swarm master and two swarm-agents.
I have created an Express app that gets started in a container and it seems the swarm master decided to have it on agent1. Then I have also a Wordpress site (example from the docker compose site) that I'm running with docker-compose on Docker Swarm.
cloud-user#docker-mgt:~/wordpress$ docker-compose up
wordpress_db_1 is up-to-date
Creating wordpress_web_1
ERROR: Error: image library/wordpress_web:latest not found
The following images are available:
cloud-user#docker-mgt:~/wordpress$ docker images
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
orchardup/php5 latest c385b8a81cee 18 months ago 330.1 MB
swarm latest a9975e2cc0a3 19 hours ago 17.15 MB
orchardup/mysql latest 5a45a5a953bb 16 months ago 292.4 MB
wordpress_web latest e484f88dc8c8 11 minutes ago 350.9 MB
node-ip latest d177af00338b 39 minutes ago 549.5 MB
centos centos6 1a895dd3954a 8 weeks ago 190.6 MB
So it does have the wordpress_web:latest available and still it complains.
When running this wordpress app on a seperate docker host (not part of the swarm cluster) it runs just fine. To exclude a mistake on the wordpress app, I also tried another complete different app consisting of two containers (with docker-compose) and it is exactly the same issue.
I'm beginning to think that docker compose and swarm can not work together smoothly but that's hard to believe of course.
You might want to try out Rancher, it uses compose and makes it simple to deploy applications across broad types of infrastructure, while keeping native docker syntax and using compose. GitHub Website
The problem is that swarm needs some way to retrieve the image to run the container. If the image doesn't exist in a registry (or the docker hub) then it doesn't (currently) have any way to get that image , and you end up with this error.
One option is to tag the images with a full name and push them to the hub. Then swarm will be able to pull the images on all the nodes.
Another options is to propagate the image using docker save and docker load, which will send the image to all nodes.
Another option is to add an image affinity to force the container to run on the node that has the image.
In the future, swarm should gain the ability to propagate images between nodes.
Related
I deploy a docker swarm with 6 nodes. I built some images and I am trying to add them as services to the swarm. I have 5 microservices. When I run the on one host with docker-compose everything works fine. I run this command docker service create rate --with-registry-auth and I get the following message.
image rate:latest could not be accessed on a registry to record
its digest. Each node will access rate:latest independently,
possibly leading to different nodes running different
versions of the image.
yyf9m49xw3enwano1scr55ufc
overall progress: 0 out of 1 tasks
1/1: No such image: rate:latest
I run docker images and the rate image is appeared. rate is the repository name. I also tried with the image id but didn't worked. The only images that I can add to swarm is images that is public.
There is an issue (https://github.com/moby/moby/issues/35187) on the Moby project about that.
If you tried already to set --with-registry-auth but didn't solved it, you should manually login to each cluster worker node and pull the Docker images.
image rate:latest could not be accessed on a registry to record
its digest. Each node will access rate:latest independently,
possibly leading to different nodes running different
versions of the image.
This error indicates you are trying to run an image that was never pushed to a registry. Push your images to a registry first. And then you can run them on any node in the cluster (which will pull any missing images from that registry). If the registry requires authentication to pull the image, then run docker stack deploy --with-registry-auth ..., but you must first push the image, and specific the pushed image name (which will not be rate:latest since you do not have access to push to the official library on Docker Hub).
I am deploying 4 containers using docker stack deploy as below:
docker stack deploy --compose-file compose.yml --with-registry-auth myapp
For the first time, the containers are built using the latest image on the registry, no problem.
But when I push new images to the registry and run the commands again, the containers are not rebuilt using the latest images.
I am using the latest tag in my images. I know it is not the recommended way to do things, but for what I have read in the documentation, docker stack deploy if using the latest tag, will check for image sha with the registry, if it is different the containers will rebuild using latest images, but In my case, it's not happening. Am I missing something here?
I also get an error/warning when I run docker stack deploy once the stack is already up:
Updating service service_name (id: some_hash_value)
image docker.pkg.github.com/username/repository/image-name:latest could not be accessed on a registry to record
its digest. Each node will access docker.pkg.github.com/username/repository/image-name:latest independently,
possibly leading to different nodes running different
versions of the image.
I encoutered the same error message when I started using a new docker registry. The new registry's SSL certificate was not considered secured by docker.
So I got this error until I added my new registry to the insecure-registries section of the /etc/docker/daemon.json
I've seen nobody mentionning this solution on this question or other similar ones, so I hoped this could help.
I created two new images anubh_custom_build_image/ubuntu_bionic:version1 & ubuntu_bionic_mldev:version1 from the base image ubuntu:bionic. The purpose of creating the custom-built ubuntu-docker images was to use Linux system on windows platform. I have faced many issues in past one such is installing a new version of tensorflow library ! pip install -q tf-nightly, I can't find a substitute of ! to run this command on windows cmd-prompt/PowerShell. Moreover, I want to invest more time on my codebase rather than fixing the issues on different OS. So, I pull the latest Ubuntu image from docker, installed a bunch of libraries for my usage and committed using docker commit command:
docker commit 503130713dff ubuntu_bionic_MLdev:version1
I can see the images using :
PS C:\Users\anubh> docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
ubuntu_bionic_mldev version1 e7d1b154b69f 21 hours ago 9.33GB
anubh_custom_build_image/ubuntu_bionic version1 3c98f8954731 22 hours ago 9.33GB
tensorflow/tensorflow latest 2c8d1fd8bde4 2 days ago 1.25GB
ubuntu bionic 735f80812f90 2 weeks ago 83.5MB
ubuntu latest 735f80812f90 2 weeks ago 83.5MB
floydhub/dl-docker cpu 0b9fc622f1b7 2 years ago 2.87GB
When I tried to spin-up the containers using these images, The following command ran without any error.
PS C:\Users\anubh> docker run anubh_custom_build_image/ubuntu_bionic:version1
PS C:\Users\anubh> docker run ubuntu_bionic_mldev:version1
EDIT:
The issue is that run command is executing but the containers aren't spinning up for the above two images. I apologize for attaching the wrong error message in the first post, I edited it now. The below two containers were spined-up using docker run -it -p 8888:8888 tensorflow/tensorflow & docker run ubuntu:bionic commands.
PS C:\Users\anubh> docker container ls
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
94d59b217b70 tensorflow/tensorflow "/run_jupyter.sh --a…" 21 hours ago Up 21 hours 6006/tcp, 8888/tcp boring_clarke
503130713dff ubuntu:bionic "bash" 38 hours ago Up 38 hours awesome_bardeen
Could anyone suggest what I am missing for running these images anubh_custom_build_image/ubuntu_bionic:version1 & ubuntu_bionic_mldev:version1 from the base image ubuntu:bionic as
containers?
Also, I can't find the location of any of these images on my disk.
Could anyone also suggest where to look for inside Windows OS?
NOTE: I will write a dockerfile in future to build a custom image, but for now, I want to use the commit command to create new image & use them.
Your docker run command doesn't work because you don't have the :version1 tag at the end of it. (Your question claims you do, but the actual errors you quote don't have it.)
However: if this was anything more than a simple typo, the community would probably be unable to help you, because you have no documentation about what went into your image. (And if not "the community", then "you, six months later, when you discover your image has a critical security vulnerability".) Learn how the Dockerfile system works and use docker build to build your custom images. Best-practice workflows tend to not use docker commit at all.
On a coreOS (899.15.0) machine, when I execute docker search and images I get the following results :
docker search private-registry:5000/
NAME DESCRIPTION STARS OFFICIAL AUTOMATED
docker images on the private registry machine:
REPOSITORY TAG IMAGE ID CREATED VIRTUAL SIZE
nginx latest e32087da8ee6dfa45221c48670fa9475f3d8a53a0e9ccabef4f741c62c77d49b 2 weeks ago 182.6 MB
registry 0.9.1 facc02b3acf6f811e8eace6d07b34cd5ab687e926ac5b5231da93264b259f1a4 12 weeks ago 422.8 MB
<none> <none> db81ebdc7ebd3d7aec05d4faa6f4c9c2e35954896e968bce2f90a9736485aa06 3 months ago 422.8 MB
...and a few more images
The reference for docker search mentions that it looks up on docker hub, but since I am specifying a specific registry here, I suppose it is looking up in the private repository.
I am not sure what is the difference between these two commands here and why the difference in results.
You are supposing wrongly. docker search searches only Docker Hub, not private registries. docker images command lists images on the machine (locally built or pulled from registries).
If you want to search or list images in the private registry, you need to use registry API to do so: https://docs.docker.com/registry/spec/api/
I'm very new to docker, I made a simple django app with docker-compose.
How do I post it to docker hub so someone can run docker run against it?
Docker hub is a repository for Docker images (make with a Dockerfile). When you use docker-compose your are simply connecting together one or more images on docker hub using your composition (the yaml that describes the images and how to connect them). You aren't making an Image with docker-compose. I don't think there is a place to store/share compositions (yet) at Docker. However, you might take a look at tutum.co. There you can save your docker-compose (they call them stacks) and deploy them from tutum. Full disclosure, I have nothing to do with tutum.co.
As the last answer said, Docker hub is only for Docker images (i.e. dockerfiles).
Tutum was bought by docker and recently shut down:
Tutum has shut down as of May 31st 2016. An evolution of the service
is now offered in Docker Cloud
It looks like there is an attempt (end of 2015) to create a kind of hub for docker-compose.yml files, using github: composehub.com
It doesn't seem very dynamic ( 55 stars on github, last update a year ago, on October 2015)