I am trying use memory and CPU in docker-compose file.
I get below error code:
The Compose file './docker-compose.yml' is invalid because:
Unsupported config option for services.web: 'resources'
My docker-compose.yml file is below
version: '3'
services:
web:
build: .
volumes:
- "./app:/home"
ports:
- "8080:8080"
resources:
limits:
cpus: '0.001'
memory: 512M
How can I use CPU, memory with in docker-compose?
Docker compose resources were introduced in file format version 3, which needs docker-compose 1.13 or newer. Chances are you are using an older version:. Check the output of:
docker-compose version
See the upgrade guide.
The OP uses docker-compose 1.12, which does not yet support version 3.
solved: I use version: '2' instead of version: '3' in docker-composer file and ı use mem_limit instead of resources
If you are using Docker Compose v.3 configs and running docker-compose up mode, then this can be useful.
This is not documented anywhere in docker-compose, but you can pass any valid system call setrlimit option in ulimits.
So, you can specify in docker-compose.yaml
ulimits:
as:
hard: 130000000
soft: 100000000
memory size is in bytes. After going over this limit your process will get memory allocation exceptions, which you may or may not trap.
Related
Running container in non-swarm mode. docker-compose.yml is configured this way:
version: '3.9'
services:
backend:
container_name: 'w_server'
restart: always
build: .
mem_reservation: '30G'
mem_limit: '40G'
environment:
NODE_USER: '[...]'
However, after successful building and starting the container, stats look like this:
docker stats --all --no-stream
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
d4426dd4e34d w_server 0.00% 573.7MiB / 23.55GiB 2.38% 225MB / 8.47MB 0B / 16MB 18
I learned about deploy.resources.reservations and deploy.resources.limits, but it only works in swarm mode and displays warning when building such configuration (and of course, the settings aren't taken into consideration when the building gets processed).
Is there any other way to assign memory resources?
Docker and docker-compose versions are:
docker-compose version 1.28.5, build c4eb3a1f
Docker version 18.09.7, build 2d0083d
Edit:
Found out this question, and answers suggest that mem_reservation and mem_limit are available in docker-compose.yml in version 2.x; version 3.x doesn't support it.
However, changing just the version to 2.4 gave exactly the same results: limit reported by docker stats was the same, not read from configuration file.
You should define your limits for v3 like that:
version: '3.9'
services:
backend:
container_name: 'w_server'
restart: always
build: .
deploy:
resources:
limits:
memory: 40g
environment:
NODE_USER: '[...]'
But, you need use --compatibility flag for it to work like:
docker compose --compatibility up
I am kind of new to docker and docker compose. I am using 20.10.12 version of docker and 2.9.0 of portainer. My aim is to do the docker compose for elasticSearch to deploy it in portainer but I get a problem that the memory given is not enough. After looking through other questions I found that I could execute the following bash command to assign a bigger limit to the VM memory
sysctl -w vm.max_map_count=262144
So my .yml is like this:
version: "3.8"
services:
command: >
bash -c ' sysctl -w vm.max_map_count=262144'
master1:
image: docker.elastic.co/elasticsearch/elasticsearch:8.0.0
environment:
node.name: "master1"
ulimits:
memlock:
soft: -1
hard: -1
deploy:
endpoint_mode: dnsrr
mode: "replicated"
replicas: 1
resources:
limits:
memory: 4G
The problem is that when I try to deploy this compose, it says "services.command must be a mapping".
I think that problem is raised when then indentation is not correct but I think in my case is indented correctly.
vm.max_map_count must be set on the host, and not in the docker container.
Set it as described in this official doc.
I’m running Sonarqube on Docker compose and my file looks like this:
version: "3"
services:
sonarqube:
image: sonarqube
ports:
- "9000:9000"
- "5432:5432"
links:
- db:db
environment:
- SONARQUBE_JDBC_URL=jdbc:postgresql://db:5432/sonar
- SONARQUBE_JDBC_USERNAME=postgres
- SONARQUBE_JDBC_PASSWORD=sonar
volumes:
- ..../Work/tools/_SonarQube_home/conf:/opt/sonarqube/conf
# - sonarqube_data:/opt/sonarqube_new/data
- ...../Work/tools/_SonarQube_home/data:/opt/sonarqube/data
- ....../Work/tools/_SonarQube_home/extensions:/opt/sonarqube/extensions
- ..../Work/tools/_SonarQube_home/bundled-plugins:/opt/sonarqube/lib/bundled-plugins
db:
image: postgres
environment:
- POSTGRES_USER=postgres
- POSTGRES_PASSWORD=sonar
- POSTGRES_DB=sonar
volumes:
- .../Work/tools/_PostgreSQL_data:/var/lib/postgresql
# This needs explicit mapping due to https://github.com/docker-library/postgres/blob/4e48e3228a30763913ece952c611e5e9b95c8759/Dockerfile.template#L52
- ..../Work/tools/_PostgreSQL_data/data:/var/lib/postgresql/data
Everything works and that’s great. One moment I saw that Sonarqube instance started to act slowly, therefore I checked docker stats. It looks like this:
| CPU | Mem Usage/ Limit |
|-------| --------------------
| 5.39% | 1.6GiB / 1.952GiB |
How do I define more RAM resources for the server, let’s say 4 GB? Previously it was mem_limit but now on version 3 it doesn’t exist.
What would be a good solution for that?
Thanks!
If you are deploying to Swarm, then you can use the resources keyword in your Compose file. (it's described under Resources in the file reference https://docs.docker.com/compose/compose-file/)
So you can do something like this is Swarm:
version: "3.7"
services:
redis:
image: redis:alpine
deploy:
resources:
limits:
cpus: '0.50'
memory: 50M
reservations:
cpus: '0.25'
memory: 20M
If you are using Compose, then you have the option to go back to Compose file version 2.0, as described in the Compose file reference by Docker.
Looking for options to set resources on non swarm mode containers?
The options described here are specific to the deploy key and swarm mode. If you want to set resource constraints on non swarm
deployments, use Compose file format version 2 CPU, memory, and other
resource options. If you have further questions, refer to the
discussion on the GitHub issue docker/compose/4513.
I'm not familiar with Sonarqube memory issue, but you may want to have a look at this https://docs.sonarqube.org/display/SONARqube71/Java+Process+Memory.
In Compose file version 3, resource limits moved to under a deploy: {resources: ...} key, but are also only documented to work in Swarm mode. So to actually set them you need to switch to a mostly-compatible version 2 Compose file.
version: '2'
services:
sonarqube:
mem_limit: 4g
The default should be for the container to be able to use an unlimited amount of memory. If you're running in an environment where Docker is inside a Linux VM (anything based on Docker Toolbox or Docker Machine, Docker for Mac) it's limited by the memory size of the VM.
I'm new to docker and trying to understand what docker stack does. Currently trying out this container https://hub.docker.com/r/instapy/instapy
this is the docker-compose file
services:
web:
image: instapy/instapy:latest
container_name: "${COMPOSE_PROJECT_NAME}_web"
env_file: .env
environment:
- PYTHONUNBUFFERED=0
- INSTAPY_WORKSPACE=/code/InstaPy
volumes:
- ./:/code
The errors I'm getting seem to indicate quite a few issues
Ignoring deprecated options:
container_name: Setting the container name is not supported.
service "web": container_name is deprecated
service "web": env_file are ignored
Stack.compose.docker.com "test" is invalid: test: Invalid value: "null": conversion to kube entities failed: C:\Users\roole\instapy-docker\docker-compose: only absolute paths can be specified in mount source
docker compose version info
docker-compose version 1.24.1, build 4667896b
docker-py version: 3.7.3
CPython version: 3.6.8
OpenSSL version: OpenSSL 1.0.2q 20 Nov 2018
Content asked for from ' docker-compose config'
services:
web:
container_name: instapy_web
environment:
COMPOSE_PROJECT_NAME: instapy
INSTAPY_WORKSPACE: /code/InstaPy
PYTHONUNBUFFERED: '0'
image: instapy/instapy:latest
volumes:
- C:\Users\roole\instapy-docker\docker-compose:/code:rw
version: '3.0'
Any help in understanding what the hell I'm supposed to be doing would be mega.
At the beginning of each docker-compose.yml file you need to specify the version. Each version of docker-compose supports certain versions of the yml file specification.
This should work for you:
version: "3.3"
services:
web:
image: instapy/instapy:latest
container_name: "${COMPOSE_PROJECT_NAME}_web"
env_file: .env
environment:
- PYTHONUNBUFFERED=0
- INSTAPY_WORKSPACE=/code/InstaPy
volumes:
- ./:/code
When deploying a stack the container name is not relevant (in fact after version "3" is not supported). The reason for that is that docker needs to be able to change the container name in case you scale your service (multiple versions of the same container might end up running on the same docker instance and then they need to have different container names).
Also when you specify a volume you need to specify full, absolute paths. You can simply replace your volume declaration with what you got from running docker-compose config (C:\Users\roole\instapy-docker\docker-compose:/code:rw) or you can use $PWD or the equivalent for your OS to refer to your current directory
I want to have a policy in my docker-compose which a docker container memory usage will be higher than the amount of a certain memory limitation it restarts.
This is what I have done so far:
version: '3'
services:
modbus_collector:
build: .
image: modbus_collector:2.0.0
container_name: modbus_collector
restart: unless-stopped
deploy:
resources:
limits:
memory: 28M
I was expecting that to be restarted when the container memory usage exceeds 28M, but when I monitor docker containers by docker stats I see that this container memory usage grow up and not happens about restart!
I also tried by restart: always but the result was the same.
[UPDATE]:
With version 2 it works fine with mem_limit:. But it fails while using version 3, putting them under deploy section doesn't seem worthy unless I am using swarm mode.
Even on version 2.1, I have a problem in restarting the container: limitation applied correctly, but when the container memory usage grows up, this limitation prevents of that but I expected instead of decrease memory it will restart that container.
version: '2.1'
services:
modbus_collector:
build: .
image: modbus_collector:2.0.0
container_name: modbus_collector
restart: unless-stopped
mem_limit: 28m