Does anyone know if there is a way to have docker swarm restart one service that is part of a stack without restarting the whole stack?
Doing docker stack deploy again for me is the way to go to update services. As Francois' Answer, and also in my own experience, doing so updates only services that need to be updated.
But sometimes, it seems easier when testing stuff to only restart a single service. In my case, I had to clear the volume and update the service to start it like it was fresh. I'm not sure if there is downside to the method I will describe. I tested it on my development stack and it worked great for me.
Get the service id you want to tear down then use docker service update --force <id> to force the update of the service which effectively re-deploy it
$ docker stack services <stack_name>
ID NAME ...
3xrdy2c7pfm3 stack-name_api ...
$ docker service update --force 3xrdy2c7pfm3
The --force flag will force the service to update causing it to restart.
Scale to 0 and back up:
docker service scale myservice=0
docker service scale myservice=10
Looking at the docker stack documentation:
Extended description
Create and update a stack from a compose or a dab file on the swarm
From this blog article: docker stack works in a similar way as docker compose. It’s idempotent. If the stack is already deployed, docker stack deploy will restart only those services which has the digest or tag that is updated:
From my experience, when I deploy the same stack again with one service changing, only the updated service will be restarted.
BUT... there seems to be some limitations to changes that are taken into account (some report bugs with image tags), so give it a try and see if works as expected.
You can also use service update if you want to be sure that only targeted service if updated with your changes.
You can also refer to this similar SO QA.
As per the example in the documentation for rolling updates:
$ docker service update --image redis:3.0.7 redis
However, that only works if your image is already on the local machines. If not then you need to use --with-registry-auth to send registry authentication details to the swarm agents. See details in the docker service update documentation.
$ docker service update --with-registry-auth --image redis:3.0.7 redis
To restart a single service (with rolling restart to avoid downtime in case the service has multiple replicas) in already configured, existing stack, you can do:
docker service update --force stack_service_name
I don't recommend running the same command again (in another shell) until this one completes (because otherwise the rolling restart isn't guaranteed; it might restart all replicas of that service).
The docker service update command also checks for newer version of the image:tag you are trying to use.
If your registry requires auth, also pass --with-registry-auth argument, like so:
docker service update --force --with-registry-auth stack_service_name
If you don't pass this argument, the service will still be restarted, but the check won't be made and the service will still use the old container image without pulling the new one first. Which might be what you want.
In case you want to also switch to different image tag (or completely different image), you can do it from here too, but remember to also change the tag in your docker-stack.yml, or your next docker stack deploy will revert it back to the verison defined there:
docker service update --with-registry-auth --force --image nginx:edge stack_service_name
remove it:
docker stack rm stack_name
redeploy it:
docker stack deploy -c docker-compose.yml stack_name
Related
I'm using docker swarm deploy -c docker-compose.yml somestack to deploy to a docker swarm. However, I can later scale it with docker service scale somestack_someservice=5 (or whatever). So now docker-compose.yml no longer reflects the system. My question is, is there any way to save off the current configuration of the stack, and then later reapply it, similar to how I originally created it (with docker-compose.yml)?
There is no direct way to generate docker-compose.yml file, although you can use
docker service inspect --pretty <service-name> command to obtain all configuration in text/json format.
There are some chances we can try to create docker-compose.yml from it.
Also, think about lack of this feature as advantage. If you want to make some adjustments, make them in docker-comose.yml first and then call docker stack deploy to apply it.
I have a swarm stack deployed and I removed couple services from the stack and tried to deploy them again. these services are showing with desired state remove and current state preparing .. also their name got changed from the custom service name to a random docker name. swarm also trying to start these services which are also stuck in preparing. I ran docker system prune on all nodes and them removed the stack. all the services in the stack are not existent anymore except for the random ones. now I cant delete them and they still in preparing state. the services are not running anywhere in the swarm but I want to know if there is a way to remove them.
I had the same problem. Later I found that the current state, 'Preparing' indicates that docker is trying to pull images from docker hub. But there is no clear indicator in docker service logs <serviceName> available in the docker-compose-version above '3.1'.
But it sometimes imposes the latency due to n\w bandwidth or other docker internal reasons.
Hope it helps! I will update the answer if I find more relevant information.
P.S. I identified that docker stack deploy -c <your-compose-file> <appGroupName> is not stuck when switching the command to docker-compose up. For me, it took 20+ minutes to download my image for some reasons.
So, it proves that there is no open issues with docker stack deploy,
Adding reference from Christian to club and complete this answer.
Use docker-machine ssh to connect to a particular machine:
docker-machine ssh <nameOfNode/Machine>
Your prompt will change. You are now inside another machine. Inside this other machine do this:
tail -f /var/log/docker.log
You'll see the "daemon" log for that machine. There you'll see if that particular daemon is doing the "pull" or what's is doing as part of the service preparation. In my case, I found something like this:
time="2016-09-05T19:04:07.881790998Z" level=debug msg="pull progress map[progress:[===========================================> ] 112.4 MB/130.2 MB status:Downloading
Which made me realise that it was just downloading some images from my docker account.
I have a swarm cluster wherein different technology dockers are deployed. (Zookeeper, Kafka, Elastic, Storm and custom web application)
Web application goes under tremendous changes and have to update the stack everytime web docker changes. Once in a while there will be updates to Elasticsearch image.
When i run docker stack deploy, it goes and restarts all existing docker services which are not even changed. This hampers whole stack and there is a downtime for whole application. Docker stack does not have option of update.
Someone has solution for this?
docker service update --image does the trick.
Check the docker service update docs.
Redeploying the stack with changed configuration(docker-compose.yml file) solves the problem see https://docs.docker.com/engine/reference/commandline/stack_deploy/#extended-description.
There they stated "Create and update a stack from a compose or a dab file on the swarm." Also i dont see any command like 'docker stack update
'. So this can solve the problem.
If you have docker stack created from compose.yml and you need to re-deploy only one service from stack, just do this:
docker service rm <your-service>
and then:
docker stack deploy -c compose.yml <stack-name>
And you just will update your stack, not recreate all services.
I have a doubt in using docker swarm mode commands to update existing services after having deployed a set of services using docker stack deploy.
As far I understood every service is pinned to the SHA256 digest of the image at the time of creation, so if you rebuild and push an image (with same tag) and you try to run a docker service update, service image is not updated (even if SHA256 is different). On the contrary, if you run a docker stack deploy again, all the services are updated with the new images.
I managed to update the service image also by using docker service update --image repository/image:tag <service>. Is this the normal behavior of these commands or is there something I didn't understood?
I'm using Docker 17.03.1-ce
Docker stack deploy documentation says:
"Create and update a stack from a compose or a dab file on the swarm. This command has to be run targeting a manager node."
So the behaviour you described is as expected.
Docker service update documentation is not so clear but you yourself said it only runs with --image repository/image:tag <service> so the flag is necessary to update the image.
You have two ways to accomplish what you want.
It is normal and expected behavior for docker stack deploy to update images of existing services to whatever hash the specified tag is linked.
If no tag is present, latest is assumed - which can be problematic at times, since the latest tag is not well understood by most persons, and thus lead to some unexpected results.
I enjoy a lot using docker-compose.
Eg. on my server, when I want to update my app with minor changes, I only need to git pull origin master && docker-compose restart, works perfectly.
But sometimes, I need to rebuild (eg. I added an npm dependency, need to run npm install again).
In this case, I do docker-compose build --no-cache && docker-compose restart.
I would expect this to :
create a new instance of my container
stop the existing container (after the newer has finished building)
start the new one
optionally remove the old one, but this could be done manually
But in practice it seems to restart the former one again.
Is it the expected behavior?
How can I handle a rebuild and start the new one after it is built?
Maybe I missed a specific command? Or would it make sense to have it?
from the manual docker-compose restart
If you make changes to your docker-compose.yml configuration these
changes will not be reflected after running this command.
you should be able to do
$docker-compose up -d --no-deps --build <service_name>
The --no-deps will not start linked services.
The problem is that restart will restart your current containers, which is not what you want.
As an example, I just did this
change the docker file for one of the images
call docker-compose build to build the images
call docker-compose down1 and docker-compose up
docker-compose restart will NOT work here
using docker-compose start instead also does not work
To be honest, i'm not completly sure you need to do a down first, but that should be easy to check.1 The bottomline is that you need to call up. You will see the containers of unchanged images restarting, but for the changed image you'll see recreating.
The advantage of this over just calling up --build is that you can see the building-process first before you restart.
1: from the comments; down is not needed, you can just call up --build. Down has some "down"-sides, including possible being destructive to your (volume-)data.
Use the --build flag to the up command, along with the -d flag to run your containers in the background:
docker-compose up -d --build
This will rebuild all images defined in your compose file, then restart any containers whose images have changed.
-d assumes that you don't want to keep everything running in your shell foreground. This makes it act more like restart, but it's not required.
Don't manage your application environment directly. Use deployment tool like Rancher / Kubernetes. Using one you will be able to upgrade your dockerized application without any downtime and even downgrade it should you need to.
Running Rancher is as easy as running another docker container as this tool is available in the Docker Hub.
You can use Swarm. Init swarm first by docker swarm init command and use healthcheck in docker-compose.yml.
Then run below command:
docker stack deploy -c docker-compose.yml project_name
instead of
docker-compose up -d.
When docker-compose.yml file is updated only run this command again:
docker stack deploy -c docker-compose.yml project_name
Docker Swarm will create new version of services and stop old version after that.
Though the accepted answer shall work to rebuild the container before starting the new one as a replacement, it is ok for simple use case, but the container will still be down during new container initialization process. If this is quite long, it can be an issue.
I managed to achieve rolling updates with docker-compose (along with a nginx reverse proxy), and detailed how I built that in this github issue: https://github.com/docker/compose/issues/1786#issuecomment-579794865
Hope it can help!
Run the following commands:
docker-compose pull
docker-compose up -d --no-deps --build <service_name>
As the top rated answer mentioned
docker-compose up -d --no-deps --build <service_name>
will restart a single service without taking down the whole compose.
I just wanted to add to the top answer in case anyone is unsure how to update an image without restarting the container.
Another way:
docker-compose restart in your case could be replaced with docker-compose up -d --force-recreate, see https://docs.docker.com/compose/reference/up/
Running docker-compose up while docker-compose is in the running state, will recreate container that got their configuration changed.
Thats the easiest way, and it will only affect containers that got their configuration changed.
root#docker:~# docker-compose up
traefik is up-to-date
nginx is up-to-date
Recreating php ... done