i’m currently in the process of setting up a swarm with 5 machines. i’m just wondering if i can and should limit the swarm to only allow one active instance of a service? and all others just wait till they should jump in when the service fail.
This is to prevent potential concurrency problems with maria-db (as the nodes sill write to a nas), or connection limit to an external service (like node red with telegram)
If you're deploying with stack files you can set "replicas: 1" in the deploy section to make sure only one instance runs at a time.
If that instance fails (crashes or exits) docker will start another one.
https://docs.docker.com/compose/compose-file/deploy/#replicas
If the service is replicated (which is the default), replicas
specifies the number of containers that SHOULD be running at any given
time.
services: frontend:
image: awesome/webapp
deploy:
mode: replicated
replicas: 6
If you want multiple instances running and only one "active" hitting the database you'll have to coordinate that some other way.
Related
I have a Docker swarm Environment with 7 nodes(3 master and 4 Workers) I am trying to Deploy a Container and buy requirement is that at any point of time I need 2 instance of this container running but when I scale this the Container should be deployed to a different node than it is currently running.
Ex: say one instance of the container is running in Node 4 and I scale to scale=2 it should run in any other node except for Node 4.
tried this but no luck:
deploy:
mode: global
placement:
constraints:
- node.labels.cloud.type == nodesforservice
We solved this issue with deployment preferences configuration (under Placement section). We set up node.labels.worker, on all our worker nodes. We have 3 workers they have node.labels.worker = worker1, node.labels.worker = worker2 and node.labels.worker = worker3 labels set to each of them. On the docker compose side then we configure it:
placement:
max_replicas_per_node: 2
constraints:
- node.role==worker
preferences:
- spread: node.labels.worker
Note this will not FORCE it always on the separate node, but if it is possible it will do so. So it is not hard limit. Beware of that.
I'm using docker swarm to deploy an application as single node only, but for having zero downtime deployment.
Question: how could I tell docker swarm to wait for X seconds before switching the started container, and before taking the old container down?
I know I could add a custom healthcheck, but I'd like to to simply define a time interval that should block the container and give it some warmup time before it is taken live.
Or maybe some kind of initial sleep?
deploy:
replicas: 1
update_config:
order: start-first
failure_action: rollback
#this did not help. new container is going live instant!
delay: 10s
Based on this picture in this document https://docs.docker.com/engine/swarm/how-swarm-mode-works/services/:
How should the output of the command #docker service ls look like for the two services depicted? My understanding is:
grey global "1/1" consul:latest
yellow replicated "1/3" consul:latest
I am not sure about the numbers between ""
I need support to understand the output
Correct output based on answer and the picture would be:
grey global "5/5" consul:latest
yellow replicated "3/3" consul:latest
Without placement constraint, a service in global mode will be deployed with one replica on each cluster node. As a result you will have 5/5 replicas.
You can use placement constraints to restrict deployment to specific nodes, for instance to worker nodes or nodes having a specific node label:
You could use a placement constraint to restrict deployment to your
worker nodes, which will result in 4 replicas.
You could add a node label to n of your nodes and use it as placement
constraint, resulting in n replicas.
While global mode services guaranty that exactly one replica of a service is running on each node that fulfills the placement constraints, the same is not necessarily true for replicate mode services. Replicated mode services are usualy fanned out accross the nodes, but could also be placed on a single node...
Why do you list the replicas to 1/3 for the (yellow) replicated mode service? If all replicas are deployed successfully it should be 3/3.
The numbers indicates a summary of the total deployment for the service. It does not indicate how the replicas are spread accross the cluster, nor where the replicas are running.where the replicas are running.
(Query section below towards middle)
Cross posted at https://developer.jboss.org/message/982355
Environment:
Infinispan 9.13,
Embedded cache in a cluster with jgroups,
Single file store,
Using JGroups inside Docker services in a single docker host/daemon (Not in AWS yet).
Infinispan.xml below:
<jgroups><stack-file name="external-file" path="${path.to.jgroups.xml}"/></jgroups>
Application = 2 webapps + database
Issue:
When I deploy the 2 webapps in separate tomcats directly on a machine (not docker yet), the Infinispan manager initializing the cache (in each webapp) forms a cluster using jgroups (i.e it works). But with the exact same configuration (and same channel name in jgroups), when deploying the webapps as services in docker, they don't join the same cluster (rather they are separate and have just one member in view - logs below).
The services are docker containers from images = linux + tomcat + webapp and are launched using docker compose v3.
I have tried the instructions at https://github.com/belaban/jgroups-docker for a container containing JGroups and a couple of demos where it suggests either using --network=host mode for the docker services (this does work but we cannot do this as the config files would need to have separate ports if we scale), or passing the external_addr=docker_host_IP_address field in jgroups.xml (this is NOT working and the query is how to make this work).
Its not a timing issue as I also tried putting a significant delay in starting the 2nd service deployed in the stack, but still the 2 apps's Infinispan cluster have just one member in its view (that container itself). Calling the cacheManager.getMembers() also shows just one entry inside each app (should show 2).
Log showing just one member in first app:
org.infinispan.remoting.transport.jgroups.JGroupsTransport.receiveClusterView ISPN000094: Received new cluster view for channel CHANNEL_NAME: [FirstContainerId-6292|0] (1) [FirstContainerId-6292].
org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded ISPN000079: Channel CHANNEL_NAME local address is FirstContainerId-6292, physical addresses are [10.xx.yy.zz:7800]
Log showing just one member in second app:
org.infinispan.remoting.transport.jgroups.JGroupsTransport.receiveClusterView ISPN000094: Received new cluster view for channel CHANNEL_NAME: [SecondContainerId-3502|0] (1) [SecondContainerId-3502]
29-Apr-2018 11:47:42.357 INFO [localhost-startStop-1] org.infinispan.remoting.transport.jgroups.JGroupsTransport.startJGroupsChannelIfNeeded ISPN000079: Channel CHANNEL_NAME local address is 58cfa4b95c16-3502, physical addresses are [10.xx.yy.zz:7800]
The docker compose V3 is below and shows the overlay network:
version: "3"
services:
app1:
image: app1:version
ports:
- "fooPort1:barPort"
volumes:
- "foo:bar"
networks:
- webnet
app2:
image: app2:version
ports:
- "fooPort2:barPort"
volumes:
- "foo:bar"
networks:
- webnet
volumes:
dbdata:
networks:
webnet:
Deployed using $docker stack deploy --compose-file docker-compose.yml OurStack
The JGroups.xml has the relevant config part below:
<TCP
external_addr="${ext-addr:docker.host.ip.address}"
bind_addr="${jgroups.tcp.address:127.0.0.1}"
bind_port="${jgroups.tcp.port:7800}"
enable_diagnostics="false"
thread_naming_pattern="pl"
send_buf_size="640k"
sock_conn_timeout="300"
bundler_type="sender-sends-with-timer"
thread_pool.min_threads="${jgroups.thread_pool.min_threads:1}"
thread_pool.max_threads="${jgroups.thread_pool.max_threads:10}"
thread_pool.keep_alive_time="60000"/>
<MPING bind_addr="${jgroups.tcp.address:127.0.0.1}"
mcast_addr="${jgroups.mping.mcast_addr:228.2.4.6}"
mcast_port="${jgroups.mping.mcast_port:43366}"
ip_ttl="${jgroups.udp.ip_ttl:2}"/>
The code is similar to:
DefaultCacheManager manager = new DefaultCacheManager(jgroupsConfigFile.getAbsolutePath());
Cache someCache = new Cache(manager.getCache("SOME_CACHE").getAdvancedCache().withFlags(Flag.IGNORE_RETURN_VALUES));
Query:
How do we deploy with docker-compose (as two services in docker containers) and jgroups.xml above so that the Infinispan cache in each of the two webapps join and form a cluster - so both apps can access the same data each other read/write in the cache. Right now they connect to the same channel name and each becomes a cluster with one member, even if we point jgroups to external_addr.
Tried so far:
Putting delay in second service's startup so first has enough time to advertise.
JGroups - Running JGroups in Docker can deploy the belaban/jgroups containers as two services in a stack using docker compose and they are able to form a cluster (chat.sh inside container shows 2 member view).
Tried --net=host which works but is infeasible. Tried external_addr=docker.host.ip in jgroups.xml which is the ideal solution but its not working (the log above is from that).
Thanks! Will try to provide any specific info if required.
Apparently external_addr="${ext-addr:docker.host.ip.address}" does not resolve (or reseolves to null), so bind_addr of 127.0.0.1 is used. Is docker.host.ip.address set by you (e.g. as an env variable)?
The external_addr attribute should point to a valid IP address.
I would like to deploy a stack to a docker swarm where I want each node to run a given service.
I looked at deploy.placement configuration and the closest option I found is the placement preference spread=node.label.abc which will equally distribute the services on nodes matching the label. However this requires updating the replicas count all the time to match the number of nodes.
Is there a way to automatically deploy a service on all the nodes without manually updating the replica count?
Is there a way to automatically deploy a service on all the nodes without manually updating the replica count?
Yes, deploy your service in global mode instead of replicated. Example from the link:
version: '3'
services:
worker:
image: dockersamples/examplevotingapp_worker
deploy:
mode: global
This will run a single instance of the container on every node matching your constraints.