I'm currently having some issues with the shared memory in one of my containers.
I have a docker-compose file in where I expect to be able to set the size. I basically converted an old docker run that had a --shm-size 16gb. I would guess it's as easy as adding shm_size:16gb to my service in the compose file.
Adding it just gives me the info: Ignoring unsupported options: shm_size.
I did check the docs, but it didn't really help me.
Just to clarify, it's not in the build but really for the "running" state.
Does one of you ever had this issue/know how to solve it?
Setup:
docker swarm with 7 nodes
Service should run on just a single node
Only running stacks
64 GB RAM host
32 GB shm (host)
Docker version 18.09.7, build 2d0083d
Using v 3.7 in my compose file
Compose file:
version: "3.7"
services:
server:
shm_size: 16GB # <<<<<<< This fails
image: local_repo/my_app:v1-dev
command: run
environment:
- UPDATES=enabled
volumes:
- type: volume
source: data
target: /var/lib/my_app/
- type: volume
source: db
target: /var/lib/postgresql/10/main
networks:
- xxx_traefik
deploy:
mode: replicated
labels:
- traefik.docker.network=xxx_traefik
- traefik.enable=true
- traefik.port=80
- traefik.frontend.rule=Host:my_container.xxx.com
- traefik.backend.loadbalancer.stickiness=true
- traefik.protocol=http
replicas: 1
placement:
constraints:
- node.hostname==node2
volumes:
db:
external: true
data:
external: true
networks:
xxx_traefik:
external: true
# shm_size: 16GB <<<<<<< Also tried to put it here since documentation doesn't show indents
Any help is appreciated:)
It should be below service, I can verify it, but here is what offical documentation said
SHM_SIZE
Added in version 3.5 file format
Set the size of the /dev/shm partition for this build’s containers.
Specify as an integer value representing the number of bytes or as a
string expressing a byte value.
build:
context: .
shm_size: '2gb'
compose-file-SHM_SIZE
Here is test
version: '3.7'
services:
your_service:
image: alpine
command: ash -c "sleep 2 && df /dev/shm"
shm_size: 2gb
Related
Hi guys and excuse me for my English. I'm using docker swarm, when I attempt to deploy docker application with this command
docker stack deploy -c docker-compose.yml -c docker-compose.prod.yml chatappapi
it shows the next error : services.chat-app-api Additional property pull_policy is not allowed
why this happens?
how do I solve this?
docker-compose.yml
version: "3.9"
services:
nginx:
image: nginx:stable-alpine
ports:
- "5000:80"
volumes:
- ./nginx/default.conf:/etc/nginx/conf.d/default.conf:ro
chat-app-api:
build: .
image: username/myapp
pull_policy: always
volumes:
- ./:/app
- /app/node_modules
environment:
- PORT= 5000
- MAIL_USERNAME=${MAIL_USERNAME}
- MAIL_PASSWORD=${MAIL_PASSWORD}
- CLIENT_ID=${CLIENT_ID}
- CLIENT_SECRET=${CLIENT_SECRET}
- REDIRECT_URI=${REDIRECT_URI}
- REFRESH_TOKEN=${REFRESH_TOKEN}
depends_on:
- mongo-db
mongo-db:
image: mongo
environment:
MONGO_INITDB_ROOT_USERNAME: 'username'
MONGO_INITDB_ROOT_PASSWORD: 'password'
ports:
- "27017:27017"
volumes:
- mongo-db:/data/db
volumes:
mongo-db:
docker-compose.prod.yml
version: "3.9"
services:
nginx:
ports:
- "80:80"
chat-app-api:
deploy:
mode: replicated
replicas: 8
restart_policy:
condition: any
update_config:
parallelism: 2
delay: 15s
build:
context: .
args:
NODE_ENV: production
environment:
- NODE_ENV=production
- MONGO_USER=${MONGO_USER}
- MONGO_PASSWORD=${MONGO_PASSWORD}
- MONGO_IP=${MONGO_IP}
command: node index.js
mongo-db:
environment:
MONGO_INITDB_ROOT_USERNAME: ${MONGO_USER}
MONGO_INITDB_ROOT_PASSWORD: ${MONGO_PASSWORD}
Information
docker-compose version 1.29.2
Docker version 20.10.8
Ubuntu 20.04.2 LTS
Thanks in advance.
Your problem line is in docker-compose.yml
chat-app-api:
build: .
image: username/myapp
pull_policy: always # <== this is the bad line, delete it
The docker compose file reference doesn't have any pull_policy in the api because
If the image does not exist, Compose attempts to pull it, unless you have also specified build, in which case it builds it using the specified options and tags it with the specified tag.
I think pull_policy used to be a thing for compose? Maybe keep the latest api documentation open to refer to/search through whilst you're developing (things can and do change fairly frequently with compose).
If you want to ensure that the most recent version of an image is pulled onto all servers in a swarm then run docker compose -f ./docker-compose.yml pull on each server in turn (docker stack doesn't have functionality to run this over an entire swarm yet).
As an aside: I wouldn't combine two .yml files with a single docker stack command without a very good reason to do so.
You are mixing docker-compose and docker swarm ideas up in the same files:
It is probably worth breaking your project up into 3 files:
docker-compose.yml
This would contain just the basic service definitions common to both compose and swarm.
docker-compose.override.yml
Conveniently, docker-compose and docker compose both should read this file automatically. This file should contain any "port:", "depends_on:", "build:" directives, and any convenience volumes use for development.
stack.production.yml
The override file to be used in stack deployments should contain everything understood by swarm and not compose, and b. everything required for production.
Here you would use configs: or even secrets: rather than volume mappings to local folders to inject content into containers. Rather than relying on ports: directives, you would install an ingress router on the swarm such as traefik. and so on.
With this arrangement, docker compose can be used to develop and build your compose stack locally, and docker stack deploy won't have to be exposed to compose syntax it doesn't understand.
pull_policy is in the latest version of docker-compose.
To upgrade your docker-compose refer to:
https://docs.docker.com/compose/install/
The spec for more info:
https://github.com/compose-spec/compose-spec/blob/master/spec.md#pull_policy
I am using docker-compose.yml to deploy services in a docker swarm which has cluster of raspberry pis. My services require access to the raspberry pi GPIO and needs privileged mode. I am using docker version 18.02 with docker-compose version 3.6. When I deploy the stack, I receive the following message and the services do not get deployed: "Ignoring unsupported options: privileged". Any tips? Below is my docker-compose.yml file
version: '3.6'
networks:
swarm_network:
driver: overlay
services:
service1:
image: localrepo/img1:v0.1
privileged: true
deploy:
mode: replicated
replicas: 1
placement:
constraints:
- node.hostname == home-desktop
ports:
- published: 8000
target: 8000
mode: host
networks:
swarm_network:
service2:
image: localrepo/img1:v0.1
privileged: true
deploy:
mode: replicated
replicas: 1
ports:
- published: 7000
target: 7000
mode: host
networks:
swarm_network:
nodeViewer:
image: alexellis2/visualizer-arm:latest
ports:
- "8080:8080"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
deploy:
placement:
constraints: [node.role == manager]
networks:
- swarm_network
Thats because privileged is not supported in docker swarm. I had a similar docker compose running in privileged mode but while using it to docker swarm I removed them and was working well.
That not exactly an error .For example if you use something like links or depends_on . You get similar warning message. These are just the warnings not errors.
This is how you actually check the error logs if there is any
docker service ls (to check running service)
docker service logs servicename
Whole feature is implemented and works as far I can see so who ever want to test it can do it by downloading latest nightly build of Docker engine (dockerd) from https://master.dockerproject.org and the custom build version of Docker CLI from https://github.com/olljanat/cli/releases/tag/beta1
You can also find usage examples for CLI from docker/cli#2199 and for Stack from docker/cli#1940 If you find bugs from those please leave comment to correct PR. Also notice that syntax might still change during review.
Source: https://github.com/moby/moby/issues/25885#issuecomment-557790402
I've personally tested it and it works like a charm. thanks to the author.
I have a situation with cassandra container.
I have 2 docker-compse.yaml files in different folders.
docker-compose.yaml in folder 1
version: "3"
services:
cassandra-cluster-node-1:
image: cassandra:3.0
container_name: cassandra-cluster-node-1
hostname: cassandra-cluster-node-1
ports:
- '9142:9042'
- '7199:7199'
- '9160:9160'
docker-compose.yaml in folder 2
version: "3"
services:
cassandra-cluster-node-2:
image: cassandra:3.0
container_name: cassandra-cluster-node-2
hostname: cassandra-cluster-node-2
ports:
- '9242:9042'
- '7299:7199'
- '9260:9160'
I tried to up cassandra on folder 1, the system work well, after that I up cassandra on folder 2. But at this time, service cassandra on folder 1 is killed automatically. So I didn't understand with them, could you guys please, who have experiences with Docker can help me to explain this situation?
The error in cassandra_1 after I run cassandra_2
cassandra-cluster-node-1 exited with code 137
Thank you, I'm going to appreciate your help.
137 is out of memory error. Cassandra uses a lot of memory if started with default settings. By default it takes 1/4 of the system memory. For each instans. You can restrict the memory usage using environment variables (see my example further down)
Docker compose creates a network for each directory it runs under. With your setup the two nodes will never be able to find each other. This is the output from my test, your files are put into two directories; cass1 and cass1
$ docker network ls
NETWORK ID NAME DRIVER SCOPE
dbe9cafe0af3 bridge bridge local
70cf3d77a7fc cass1_default bridge local
41af3e02e247 cass2_default bridge local
21ac366b7a31 host host local
0787afb9aeeb none null local
You can see the two networks cass1_default and cass2_default. So the two nodes will not find each other.
If you want them to find each other you have to give the first one as a seed to second one, and they have to be in the same network (same docker-compose file)
version: "3"
services:
cassandra-cluster-node-1:
image: cassandra:3.0
container_name: cassandra-cluster-node-1
hostname: cassandra-cluster-node-1
environment:
- "MAX_HEAP_SIZE=1G"
- "HEAP_NEWSIZE=256M"
ports:
- '9142:9042'
- '7199:7199'
- '9160:9160'
cassandra-cluster-node-2:
image: cassandra:3.0
container_name: cassandra-cluster-node-2
hostname: cassandra-cluster-node-2
environment:
- "MAX_HEAP_SIZE=1G"
- "HEAP_NEWSIZE=256M"
- "CASSANDRA_SEEDS=cassandra-cluster-node-1"
ports:
- '9242:9042'
- '7299:7199'
- '9260:9160'
depends_on:
- cassandra-cluster-node-1
I have part of my current config like this
mymicroservice:
image: service_img
networks: myoverlay
volumes:
- /Users/abcdUser/mountme:/opt/company/
This does the job as my machine's directory gets mounted to /opt/company when I deploy the docker swarm service stack.
However, I want to specify the source directory under a separate volumes: and then specify that name over there. I think this is possible but I am not able to find the syntax.
So I want something along the following lines but not able to do so:
mymicroservice:
image: service_img
networks: myoverlay
volumes:
- myownvolume:/opt/company/
volumes:
- myownvolume: /Users/abcdUser/mountme
I want to clarify that myownvolume here is just pointing to the directory /Users/abcdUser/mountme and I am not intending to create a docker volume. Or there is any other better way to do this?
It is possible to do so but not with the standard setup. The default volume driver doesn't allow the format you are looking for. You need to use docker plugins which requires external installation. Consider the below yaml
version: '2'
services:
one:
image: alpine
working_dir: /one/
command: sleep 600
volumes:
- data:/one/
two:
image: alpine
working_dir: /two/
command: sleep 600
volumes:
- data:/two/
volumes:
data:
driver: local-persist
driver_opts:
mountpoint: /data/local-persist/data
Above would work when you have the local-persist plugin installed. https://github.com/CWSpear/local-persist
You can find about other plugins available on
https://docs.docker.com/engine/extend/legacy_plugins/#volume-plugins
Also if repetition of volumes entries is an issues for you then you can use anchors in YAML
version: '3'
services:
alpines:
image: alpine
command: sleep 200
volumes: &common_volumes
- ./data:/data
- ./config:/config
alpine2:
image: alpine
command: sleep 200
volumes: *common_volumes
$ docker-compose config
services:
alpine2:
command: sleep 200
image: alpine
volumes:
- /home/vagrant/so/volumes2/data:/data:rw
- /home/vagrant/so/volumes2/config:/config:rw
alpines:
command: sleep 200
image: alpine
volumes:
- /home/vagrant/so/volumes2/data:/data:rw
- /home/vagrant/so/volumes2/config:/config:rw
version: '3.0'
That's not possible. You either have to use the bind mount syntax or the volume syntax. Volumes at the compose config top level won't allow you to mix both. See how to define a general mount point in docker compose for a similar question.
I am unable to specify CPU and memory limitation for services specified in version 3.
With version 2 it works fine with mem_limit & cpu_shares parameters under the services. But it fails while using version 3, putting them under deploy section doesn't seem worthy unless I am using swarm mode.
Can somebody help?
version: "3"
services:
node:
build:
context: .
dockerfile: ./docker-build/Dockerfile.node
restart: always
environment:
- VIRTUAL_HOST=localhost
volumes:
- logs:/app/out/
expose:
- 8083
command: ["npm","start"]
cap_drop:
- NET_ADMIN
- SYS_ADMIN
I know the topic is a bit old and seems stale, but anyway I was able to use these options:
deploy:
resources:
limits:
cpus: '0.001'
memory: 50M
when using 3.7 version of docker-compose
What helped in my case, was using this command:
docker-compose --compatibility up
--compatibility flag stands for (taken from the documentation):
If set, Compose will attempt to convert deploy keys in v3 files to
their non-Swarm equivalent
Think it's great, that I don't have to revert my docker-compose file back to v2.
deploy:
resources:
limits:
cpus: '0.001'
memory: 50M
reservations:
cpus: '0.0001'
memory: 20M
More: https://docs.docker.com/compose/compose-file/compose-file-v3/#resources
In you specific case:
version: "3"
services:
node:
image: USER/Your-Pre-Built-Image
environment:
- VIRTUAL_HOST=localhost
volumes:
- logs:/app/out/
command: ["npm","start"]
cap_drop:
- NET_ADMIN
- SYS_ADMIN
deploy:
resources:
limits:
cpus: '0.001'
memory: 50M
reservations:
cpus: '0.0001'
memory: 20M
volumes:
- logs
networks:
default:
driver: overlay
Note:
Expose is not necessary, it will be exposed per default on your stack network.
Images have to be pre-built. Build within v3 is not possible
"Restart" is also deprecated. You can use restart under deploy with on-failure action
You can use a standalone one node "swarm", v3 most improvements (if not all) are for swarm
Also Note:
Networks in Swarm mode do not bridge. If you would like to connect internally only, you have to attach to the network. You can 1) specify an external network within an other compose file, or have to create the network with --attachable parameter (docker network create -d overlay My-Network --attachable)
Otherwise you have to publish the port like this:
ports:
- 80:80
Docker Compose v1 does not support the deploy key. It's only respected when you use your version 3 YAML file in a Docker Stack.
This message is printed when you add the deploy key to you docker-compose.yml file and then run docker-compose up -d
WARNING: Some services (database) use the 'deploy' key, which will be
ignored. Compose does not support 'deploy' configuration - use docker stack deploy to deploy to a swarm.
The documentation (https://docs.docker.com/compose/compose-file/#deploy) says:
Specify configuration related to the deployment and running of
services. This only takes effect when deploying to a swarm with docker
stack deploy, and is ignored by docker-compose up and docker-compose
run.
Nevertheless you can use Docker Compose v2. Given the following Docker composition you can use the deploy key to limit your containers resources.
version: "3.9"
services:
database:
image: mariadb:10.10.2-jammy
container_name: mydb
environment:
MYSQL_ROOT_PASSWORD: root_secret
MYSQL_DATABASE: mydb
MYSQL_USER: myuser
MYSQL_PASSWORD: secret
TZ: "Europe/Zurich"
MARIADB_AUTO_UPGRADE: "true"
tmpfs:
- /var/lib/mysql:rw
ports:
- "127.0.0.1:3306:3306"
deploy:
resources:
limits:
cpus: "4.0"
memory: 200M
networks:
- mynetwork
When you run docker compose up -d (Note: in version 2 of Docker Compose you call the docker binary at not the docker-compose python application) and then inspect the resources you see that the memory is limited to 200 MB. The CPU limit is not exposed by docker stats.
❯ docker stats --no-stream
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
2c71fb8de607 mydb 0.04% 198MiB / 200MiB 99.02% 2.67MB / 3.77MB 70.6MB / 156MB 18
This is possible with version >= 3.8. Here is an example using docker-compose >= 1.28.x :
version: '3.9'
services:
app:
image: nginx
cpus: "0.5"
mem_reservation: "10M"
mem_limit: "250M"
Proof of it working (see the MEM USAGE) column :
The expected behavior when reaching memory limit is the container getting killed. In this case, whether set restart: always or adjust your app code.
Limits and restarts settings in Docker compose v3 should now be set using (restart: always is also deprecated):
deploy:
restart_policy:
condition: on-failure
delay: 5s
max_attempts: 3
window: 120s
resources:
limits:
cpus: '0.50'
memory: 50M
reservations:
cpus: '0.25'
memory: 20M
I have other experiences, maybe somebody can explain this.
Maybe this is bug(i think this is a feature), but, I am able to use deployments limits (memory limits) in docker-compose without swarm, hovever CPU limits doesn't work but replication does.
$> docker-compose --version
docker-compose version 1.29.2
$> docker --version
Docker version 20.10.12
version: '3.2'
services:
limits-test:
image: alexeiled/stress-ng
command: [
'--vm', '1', '--vm-bytes', '20%', '--vm-method', 'all', '--verify', '-t', ' 10m', '-v'
]
deploy:
resources:
limits:
cpus: '0.50'
memory: 1024M
Docker stats
b647e0dad247 dc-limits_limits-test_1 0.01% 547.1MiB / 1GiB 53.43% 942B / 0B 0B / 0B 3
Edited, thx #Jimmix
I think there is confusion here over using docker-compose and docker compose (with a space). You can install the compose plugin using https://docs.docker.com/compose/install if you don't already have it.
Here is an example compose file just running Elasticsearch
version: "3.7"
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:7.15.2
restart: always
ports:
- "9222:9200"
deploy:
resources:
limits:
cpus: "4"
memory: "2g"
environment:
- "node.name=elasticsearch"
- "bootstrap.memory_lock=true"
- "discovery.type=single-node"
- "xpack.security.enabled=false"
- "ingest.geoip.downloader.enabled=false"
I have it in a directory called estest the file is called es-compose.yaml. The file sets CPU and memory limits.
If you launch using docker-compose e.g.
docker-compose -f es-compose.yaml up
Then look at docker stats you see
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
e3b6253ee730 estest_elasticsearch_1 342.13% 32.39GiB / 62.49GiB 51.83% 7.7kB / 0B 27.3MB / 381kB 46
so the cpu and memory resource limits are ignored. During the launch you see the warning
WARNING: Some services (elasticsearch) use the 'deploy' key, which will be ignored. Compose does not support 'deploy' configuration - use `docker stack deploy` to deploy to a swarm.
Which I think is what leads people to look at Docker stack/swarm. However if you just switch to using the newer docker compose now built in to the docker CLI https://docs.docker.com/engine/reference/commandline/compose/ e.g.
docker compose -f es-compose.yaml up
And look again at docker stats you see
CONTAINER ID NAME CPU % MEM USAGE / LIMIT MEM % NET I/O BLOCK I/O PIDS
d062eda10ffe estest-elasticsearch-1 0.41% 1.383GiB / 2GiB 69.17% 8.6kB / 0B 369MB / 44MB 6
Therefore the limits have been applied.
This is better in my opinion than swarm as it still allows you to build containers as part of the compose project and pass environment easily via a file. I would recommend removing docker-compose and switching over to use the newer docker compose wherever possible.