How can I scale a service but apply a different cpuset on each instance with docker-compose ?
For example: I have 4 cpus, I want 4 instances, each using 1 unique cpu.
What version of docker-compose are you using? I'm asking because accomplish what you desire is only possible with docker-compose v2.x or docker-swarm as you can see below.
you can check more info here in the docker doc.
supposing that you are using docker-compose 2.4, you can define a service like this in your `docker-compose.yaml
version: '2.4'
services:
redis:
image: redis:1.0.0
restart: always
environment:
- REDIS_PASSWORD=1234
cpu_count: 1
mem_limit: 200m
Where cpu_count is number o cpu cores you want to use in the service, and mem_limit is the limit of memory that your service can consume.
To define the number of replicas you must run:
docker-compose up --scale redis=2
Where redis is name of the service in the docker-compose and 2 is the number of replicas that you desire. So both the two containers will spin up with 1 core of CPU and 200m of memory.
To check the container resources consumption you can run docker stats
Source:
https://docs.docker.com/engine/reference/run/#runtime-constraints-on-resources
https://docs.docker.com/compose/compose-file/compose-file-v2/#cpu-and-other-resources
Related
I have a docker-compose file that looks like the following:
version: "3.9"
services:
api:
build: .
ports:
- "5000"
deploy:
resources:
reservations:
devices:
- capabilities: [gpu]
count: 1
When I run docker-compose up, this runs as intended, using the first GPU on the machine.
However, if I run docker-compose up --scale api=2, I would expect each docker container to reserve one GPU on the host.
The actual behaviour is that both containers receive the same GPU, meaning that they compete for resources. Additionally, I also get this behaviour if I have two containers specified in the docker-compose.yml, both with count: 1. If I manually specify device_ids for each container, it works.
How can I make it so that each docker container reserves exclusive access to 1 GPU? Is this a bug or intended behaviour?
The behavior of docker-compose when a scale is requested is to create additional containers as per the exact specification provided by the service.
There are very few specification parameters that will vary during the creation of the additional containers and the devices which are part of the host_config set of parameters are copied without modifications.
docker-compose is python project, so if this is important feature for you, you can try to implement it. The logic that drives the lifecycle of the services (creation, scaling, etc.) reside in compose/services.py.
After fiddling around for a couple days with what was new to me a week ago, I'm kind of stuck and would like your help. I've created a docker swarm with some Pi's running Ubuntu server 20.04 LTS and when I use the command:
$ docker stack deploy --compose-file docker-compose.visualizer.yml visualizer
The terminal feedback is:
Creating network visualizer_default
Creating service visualizer_visualizersvc
Practically the same output when I run:
$ docker stack deploy --compose-file docker-compose.home-assistant.yml home-assistant
Checking the stacks:
$ docker stack ls
NAME SERVICES ORCHESTRATOR
home-assistant 1 Swarm
visualizer 1 Swarm
Checking services in stacks:
$ docker stack services visualizer
ID NAME MODE REPLICAS IMAGE PORTS
t5nz28hzbzma visualizer_visualizersvc replicated 0/1 dockersamples/visualizer:latest *:8000->8080/tcp
$ docker stack services home-assistant
ID NAME MODE REPLICAS IMAGE PORTS
olj1nbx5vj40 home-assistant_homeassistant replicated 0/1 homeassistant/home-assistant:stable *:8123->8123/tcp
When I then browse to the ports specified in docker-compose.visualizer.yml or docker-compose.home-assistant.yml there is no response on the server side ("can't connect"). Identical for both the manager and worker IP. This is inside a home network, in a single subnet with no traffic rules set for LAN traffic.
EDIT: a portscan reveals no open ports in the specified range on either host.
Any comments on my work are welcome as I'm learning, but I would very much like to see some containers 'operational'.
As a reference I included the docker-compose files:
docker-compose.home-assistant.yml
version: "3"
services:
homeassistant:
image: homeassistant/home-assistant:stable
ports:
- "8123:8123"
volumes:
- './home-assistant:/config'
environment:
TZ: 'Madrid'
restart: unless-stopped
network_mode: host
docker-compose.visualizer.yml
version: "3"
services:
visualizersvc:
image: alexellis2/visualizer-arm:latest
deploy:
placement:
constraints:
- 'node.role==manager'
ports:
- '8000:8080'
volumes:
- '/var/run/docker.sock:/var/run/docker.sock'
Bonus points for telling me if I should always approach the manager through the specified ports or if I have to approach the machine running the service (or any good documentation on the subject.)
Not long after you post a question you happen to find the answer yourself of course:
I never scaled the services (to 1 in my case)
docker service scale [SERVICE_ID]=1
EDIT: The services were not scaling to 1 because of another error, I think in the visualizer, but this brought me to the final answer.
Now I'm getting a mountain of new error messages, but at least those are verbose :)
Any feedback is still welcome.
On Docker for Windows, I have a simple SQL Server container based on microsoft/mssql-server-windows-developer that is launched with docker-compose up via a simple docker-compose.yaml file.
Is there a way to allocate more than 1GB of memory to this container? I can do it when running the image directly or when I build my image with -m 4GB, but I can't figure out how to do this when using Docker Compose. This container needs more than 1GB of RAM to run properly and all of my research has revealed nothing helpful thus far.
I've looked into the resources configuration option, but that only applies when running under Docker Swarm, which I don't need.
In docker compose version 2.* you could use the mem_limit option as below
version: '2.4'
services:
my-svc:
image: microsoft/mssql-server-windows-developer
mem_limit: 4G
In docker compose version 3 it is replaced by the resources options which requires docker swarm.
version: '3'
services:
my-svc:
image: microsoft/mssql-server-windows-developer
deploy:
resources:
limits:
memory: 4G
There is a compatibility flag that can be used to translate the deploy section to equivalent version 2 parameters when running docker-compose --compatibility up. However this is not recommended for production deployments
From documentation
docker-compose 1.20.0 introduces a new --compatibility flag designed
to help developers transition to version 3 more easily. When enabled,
docker-compose reads the deploy section of each service’s definition
and attempts to translate it into the equivalent version 2 parameter.
Currently, the following deploy keys are translated:
resources
limits and memory reservations
replicas
restart_policy
condition and max_attempts All other keys are ignored and produce a
warning if present. You can review the configuration that will be used
to deploy by using the --compatibility flag with the config command.
We recommend against using --compatibility mode in production. Because the resulting configuration is only an approximate using non-Swarm mode properties, it may produce unexpected results.
Looking for options to set resources on non swarm mode containers?
The options described here are specific to the deploy key and swarm mode. If you want to set resource constraints on non swarm deployments, use Compose file format version 2 CPU, memory, and other resource options. If you have further questions, refer to the discussion on the GitHub issue docker/compose/4513.
You can use the docker-compose file on version 2 instead of version 3. You can use mem_limit (available on version 2) to set the memory limit. So you can use a docker-compose file like this:
version: "2.4"
services:
sql-server:
image: microsoft/mssql-server-windows-developer
environment:
- ACCEPT_EULA=Y
- SA_PASSWORD=t3st&Pa55word
mem_limit: 4GB
You can check the memory limit using docker stats.
Was also out for setting this up via docker-compose. Had a hard time figuring out why sql server worked on a new machine but not any longer on my older one. Finally recalled I had tuned the size down able to allocate in Docker Desktop. Utilizing this you find it through the settings button, Resources/Advanced. Setting Memory to 2GB resolved the issue for me.
Here is part of my docker-compose.yaml file
version: '3.4'
services:
app:
build:
context: .
dockerfile: Dockerfile
working_dir: /app
deploy:
resources:
limits:
cpus: '0.50'
memory: 23M
Starting it docker-compose up -d
When I do docker stats it says that limit is still 1.9GiB. What am I doing wrong?
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM %
13b6588evc1e app_1 1.86% 20.45MiB / 1.952GiB 1.02%
deploy key only works in swarm mode and with docker-compose file version 3 and above.
In your case, use docker-compose file version 2 and define resource limits:
version: "2.2"
services:
app:
image: foo
cpus: "0.5"
mem_limit: 23m
See official docs here
Are you running the docker-compose in swarm mode ? If not Recommended to run 2.x version of compose file format.
3.X require docker-compose to be run in swarm mode for new set of resource directives to take effect.
Alternatives in 2.X are cpu_shares, cpu_quota, cpuset, mem_limit, memswap_limit, mem_swappiness
If you not intend to use docker swarm stack deployments, always stick to the latest 2.x version supported by the docker engine version you operate. Docker versions 17.12 and later support compose file version 2.4. Docker-Compose has all features the cli provides, while swarm still lacks some of those: see https://github.com/moby/moby/issues/25303.
If you use docker-compose, all swarm related elements in a 3.x file will be ignorend, except secrets (or was it configs?!). If you start to mix 3.x only elements with 2.x only elements, your configuration will become invalid.
I'm aware that docker-compose with docker-swarm (which is now legacy) is able to co-schedule some services on one node (using dependency filters such as link)
I was wondering if this kind of co-scheduling is possible using modern docker engine swarm mode and the new stack deployment introduced in Docker 1.13
In docker-compose file version 3, links are said to be ignored while deploying a stack in a swarm, so obviously links aren't the solution.
We have a bunch of servers to run batch short-running jobs and the network between them is not very high speed. We want to run each batch job (which consists of multiple containers) on one server to avoid networking overhead. Is this feature implemented in docker stack or docker swarm mode or we should use the legacy docker-swarm?
Also, I couldn't find co-scheduling with another container in the placement policies.
#Roman: You are right.
To deploy to a specific node you need to use placement policy:
version: '3'
services:
job1:
image: example/job1
deploy:
placement:
node.hostname: node-1
networks:
- example
job2:
image: example/job2
deploy:
placement:
node.hostname: node-1
networks:
- example
networks:
example:
driver: overlay
You can still use depends_on
It worth having a look at dockerize too.