If I have a Docker Swarm cluster consisting of 5 nodes, I'm aware that I can assign labels to particular nodes to guide which VMs services get deployed to. I.e. in the Docker Stack configuration:
...
deploy:
replicas: 1
placement:
constraints:
- node.labels.service == postgres
Is it possible to specify a preferred placement node? Something like this:
...
deploy:
replicas: 1
placement:
constraints:
- node.labels.service == postgres # This is always the first choice
- node.labels.service == postgres_2 # This is the second choice
i.e. the logic I'm looking for is something along the lines of:
Unless something is wrong with the node labeled postgres, deploy there. If something is wrong with that node (for example; a corrupted file system), then deploy to the node labeled postgres_2. When a node labeled postgres exists again as part of the swarm, redeploy to that node and delete the postgres service on postgres_2
Related
I have a Docker swarm Environment with 7 nodes(3 master and 4 Workers) I am trying to Deploy a Container and buy requirement is that at any point of time I need 2 instance of this container running but when I scale this the Container should be deployed to a different node than it is currently running.
Ex: say one instance of the container is running in Node 4 and I scale to scale=2 it should run in any other node except for Node 4.
tried this but no luck:
deploy:
mode: global
placement:
constraints:
- node.labels.cloud.type == nodesforservice
We solved this issue with deployment preferences configuration (under Placement section). We set up node.labels.worker, on all our worker nodes. We have 3 workers they have node.labels.worker = worker1, node.labels.worker = worker2 and node.labels.worker = worker3 labels set to each of them. On the docker compose side then we configure it:
placement:
max_replicas_per_node: 2
constraints:
- node.role==worker
preferences:
- spread: node.labels.worker
Note this will not FORCE it always on the separate node, but if it is possible it will do so. So it is not hard limit. Beware of that.
Is there a possibility to start a service on a specific node, based on another running service? (using Docker Swarm)
To make myself a little more clear:
I want to run Nextcloud on a different node than for example, a Typo3, to spare some resources on my Nextcloud node.
How would I write that in a compose?
Look into deploy and using labels:
Example:
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.labels.NextcloudDaemon == true
restart_policy:
condition: any
The above example will run exactly 1 container, and only on the node you've already given the label of "NextcloudDaemon".
I have a jenkins pipeline where it build and runs various containers on a NUC server.
This NUC is on a cluster (Swarm) with another NUC.
I recently added a couple of raspberry Pis on my setup and on that cluster, so now I wonder if there is a way to command Jenknis to deploy on x86_x64 or armhf devices.
I tried the -e constraint:node==<node_name> solution I found on other questions, but i had no luck.
I tried the above command from one x86_x64 node pointing to another and from a x86_x64 node pointing to a armhf node
I dont want to run those containers as a service and i dont care of any load balancer, I just want to run a container on a specific architecture (x86_x64 or armhf depending on what i want to deploy)
You cannot use constraints on containers, only on services.
That being said, using constraints on services seems a good way to go.
You can learn more about services constraints in the official documentation.
Here are a few example on how constraints can match node or Docker Engine labels:
# Node ID
node.id==2ivku8v2gvtg4
# Node hostname
node.hostname!=node-2
# Node role
node.role==manager
# Node labels
node.labels.security==high
# Docker Engine's labels
engine.labels.operatingsystem==ubuntu 14.04
If you want to match a specific hostname you need to use node.hostname==<hostname> and not node==<hostname>
You will also want to update the restart_policy key from your service definition deploy policy to prevent from starting a new container once the first one terminates its process successfully.
Wrapping it all together you need something like that:
version: "3.7"
services:
myapp:
image: <my_image>
deploy:
placement:
constraints:
- node.labels.arch == x86_64
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
Of course, it is up to you to add the labels to each Swarm node. Ansible and its module docker-node are really well suited for this purpose. Here is an example playbook:
- hosts: swarm
tasks:
- name: Add label to node specifying architecture
docker_node:
hostname: "{{ inventory_hostname }}"
labels:
arch: "{{ ansible_architecture }}"
Your docker nodes would be labelled with the key arch and one of the following values:
armhf
armv7l
i386
x86_64
Why don't you want to use a service? Services is how we orchestrate things in Swarm. docker container ... will only start/stop/inspect/manipulate things in on a single host. As you already have a swarm cluster set up, why not use a service?
I would like to deploy a stack to a docker swarm where I want each node to run a given service.
I looked at deploy.placement configuration and the closest option I found is the placement preference spread=node.label.abc which will equally distribute the services on nodes matching the label. However this requires updating the replicas count all the time to match the number of nodes.
Is there a way to automatically deploy a service on all the nodes without manually updating the replica count?
Is there a way to automatically deploy a service on all the nodes without manually updating the replica count?
Yes, deploy your service in global mode instead of replicated. Example from the link:
version: '3'
services:
worker:
image: dockersamples/examplevotingapp_worker
deploy:
mode: global
This will run a single instance of the container on every node matching your constraints.
In Docker Swarm you can set maximum system requirements like so:
my-service
image: hello-world
deploy:
resources:
limits:
cpus: '2'
memory: 4GB
I have a container that has minimum system requirements of 2 CPU cores and 4GB of RAM which is the exact size of the nodes in my Docker Swarm. This means that when this container is running, it needs to be the only container running on that node.
However, when I run the container alongside others, other containers get placed on the same node. How can I ensure that Docker gives this container a minimum level of CPU and RAM?
Update
I added reservations as suggested by #yamenk, however I still get other containers starting on the same node which causes performance problems for the container I am trying to protect:
my-service
image: hello-world
deploy:
resources:
reservations:
cpus: '2'
memory: 4GB
Update
Apparently the effect of memory reservations in docker swarm are not very well documented and they work as a best effort. To understand the effect of memory reservation flag, check the documentation:
When memory reservation is set, Docker detects memory contention or
low memory and forces containers to restrict their consumption to a
reservation limit.
...
Memory reservation is a soft-limit feature and does not guarantee
the limit won’t be exceeded. Instead, the feature attempts to ensure
that, when memory is heavily contended for, memory is allocated based
on the reservation hints/setup.
To enforce that no other container runs on the same node, you need to set service constraints. What you can do is give nodes in the swarm specific labels and use these labels to scheduel services to run only on nodes that have those specific labels.
As decribed here, node labels can be added to a node using the command:
docker node update --label-add hello-world=yes <node-name>
Then inside your stack file, you can restrict the container to run on nodes only having the specified label, and other container to avoid nodes labeled with hello-world=yes.
my-service:
image: hello-world
deploy:
placement:
constraints:
- node.labels.hello-world == yes
other-service:
...
deploy:
placement:
constraints:
- node.labels.hello-world == no
If you want to start replicas of my-service on multiple nodes, and still have one container running on each node, you need to set the global mode of my-service, and add the same label to nodes where you want a container to run.
The global mode ensures that exactly one container will run each node that satisfies the service constraints:
my-service:
image: hello-world
deploy:
mode: global
placement:
constraints:
- node.labels.hello-world == yes
Old Answer:
You can set resource reservations as such:
version: '3'
services:
redis:
image: redis:alpine
deploy:
resources:
reservations:
cpus: '1'
memory: 20M