<EDIT: Docker compose now recognizes and applies the the limits set for cpu and memory in the deploy key. so the below issue no longer applies.>
I'm fairly new to docker and I'm trying to run some minecraft containers, however they don't seem to be honoring the cpu limits I'm setting. As you can see in the commented out portions, i'v also tried using v3. No matter what limit I set, the container shows up to 400% cpu usage in docker stats. It is a 4cpu virtual machine hosted with Oracle, running Ubuntu.
Is there any other way to enforce this limit?
---
version: "2.4"
#version: "3.9"
services:
mc-cl:
container_name: mc-crimson
image: itzg/minecraft-server:java11
ports:
- 25566:25565
volumes:
- ./data:/data
environment:
- TYPE=PAPER
- VERSION=1.15.2
- EULA=TRUE
- MEMORY=1536M
- GID=1001
- UID=1001
- USE_AIKAR_FLAGS=true
cpus: 1
mem_limit: 3072M
mem_reservation: 3072M
# deploy:
# resources:
# limits:
# cpus: '1'
# memory: 3072M
# reservations:
# cpus: '0.5'
# memory: 3072M
restart: unless-stopped
If it's helpful, here's my docker version and docker compose version.
Client: Docker Engine - Community
Version: 20.10.12
API version: 1.41
Go version: go1.16.12
Git commit: e91ed57
Built: Mon Dec 13 11:44:28 2021
OS/Arch: linux/arm64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.12
API version: 1.41 (minimum version 1.12)
Go version: go1.16.12
Git commit: 459d0df
Built: Mon Dec 13 11:43:05 2021
OS/Arch: linux/arm64
Experimental: false
containerd:
Version: 1.4.12
GitCommit: 7b11cfaabd73bb80907dd23182b9347b4245eb5d
runc:
Version: 1.0.2
GitCommit: v1.0.2-0-g52b36a2
docker-init:
Version: 0.19.0
GitCommit: de40ad0
Docker Compose version v2.2.2
So I've managed to fix my own issue. I had a thought, and so I did this by removing docker compose v2 (yaml v3) with
rm ~/.docker/cli-plugins/docker-compose
and installed docker-compose v1.25.0 from the repository with
apt install docker-compose -y
The limits are now being honored when running the container.
Related
I am using Maven to interpolate a docker compose file, in order to map the working directory in either Linux and Windows. Interpolation works as intended on both OSs.
In my local Windows environment, when running "docker compose up" I get both containers with the mapped volume (which already exists on the host machine), without specifying "volumes: " at top-level, only at service-level.
However, if I try to run the same setup in linux-based TeamCity, I get the following message "service "job_controller" refers to undefined volume path/to/target/classes: invalid compose project"
After checking others' answers from here, I've understood that I also have to specify "volumes:" at top-level, which I did at the bottom of the compose file.
Now, I am prompted with "volumes Additional property /opt/buildagent/work/9857567c5e342350/path/to/target/classes is not allowed"
name: Distributed
services:
create_database:
container_name: create_database
command:
- ./script.sh
- deployer
- -f
- ../config/product-mssql-v11.manifest.yaml
- drop-create-database-properties
image: alpine-3-corretto-11-wildfly-11.11.0-SNAPSHOT
networks:
- deploy
volumes:
- C:\\SourceCode\\Path\\to\\target/classes:/opt/product/config
healthcheck:
test: ["CMD", "/opt/product/script.sh", "deployer", "-f", "/opt/product/config/product-mssql-v11.manifest.yaml", "healthy"]
interval: 20s
timeout: 60s
retries: 5
job_controller:
container_name: job_controller
environment:
DEPLOYMENT_MANIFEST: /opt/product/config/main.manifest.yaml
PROPERTIES_FILE_NAME: /opt/product/config/risk-wildfly.properties
JAVA_OPTS: "-Xms1g -Xmx4g -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=1g -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true"
ports:
- 8080:8080
image: alpine-3-corretto-11-wildfly-11.11.0-SNAPSHOT
volumes:
- C:\\SourceCode\\Path\\to\\target/classes:/opt/product/config
networks:
- deploy
depends_on:
create_database:
condition: service_completed_successfully
restart: on-failure
healthcheck:
test: ["CMD", "/opt/product/script.sh", "health-check", "--context-path","product"]
interval: 20s
timeout: 60s
retries: 5
networks:
deploy:
name: deploy
external: true
volumes:
C:\\SourceCode\\Path\\to\\target/classes:
external: true
Now, locally, if I try to run "docker compose up" with the "volumes: " specified at the bottom I get as well the same "volumes Additional property C:\SourceCode\Path\to\target/classes is not allowed"
If, instead of
volumes:
C:\\SourceCode\\Path\\to\\target/classes:
external: true
I use
volumes:
I get the "volumes: " must be a mapping.
So neither of this works.
C:\>docker compose version
Docker Compose version v2.10.2
C:\>docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
C:\>docker version
Client:
Cloud integration: v1.0.29
Version: 20.10.17
API version: 1.41
Go version: go1.17.11
Git commit: 100c701
Built: Mon Jun 6 23:09:02 2022
OS/Arch: windows/amd64
Context: default
Experimental: true
Server: Docker Desktop 4.12.0 (85629)
Engine:
Version: 20.10.17
API version: 1.41 (minimum version 1.12)
Go version: go1.17.11
Git commit: a89b842
Built: Mon Jun 6 23:01:23 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.8
GitCommit: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
How can I run this successfully in both OSs considering the volume mapping?
If you want to map specific folder from host to docker container you don't need root section
volumes:
at all
It's used to create named volumes somewhere inside docker and reference them by name in volumes section of service definition (and across multiple docker-compose files if external flag is set)
How do I "FLUSHALL" in redis in this situation?
Running redis via docker on PopOs 21.0.4 as shown in the following docker-compose.yml
version: "2.4"
services:
redis:
image: redis:5-alpine
command: redis-server --save "" --appendonly yes
restart: always
volumes:
- "${PWD}//redis/data:/data"
ports:
- "6379:6379"
Connecting to redis-cli and issuing a FLUSHALL (or FLUSHDB) command and I get the error:
127.0.0.1:6379[1]> FLUSHALL
(error) LOADING Redis is loading the dataset in memory
Here is docker version:
Client: Docker Engine - Community
Version: 20.10.10
API version: 1.41
Go version: go1.16.9
Git commit: b485636
Built: Mon Oct 25 07:43:13 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.10
API version: 1.41 (minimum version 1.12)
Go version: go1.16.9
Git commit: e2f740d
Built: Mon Oct 25 07:41:20 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.11
GitCommit: 5b46e404f6b9f661a205e28d59c982d3634148f8
runc:
Version: 1.0.2
GitCommit: v1.0.2-0-g52b36a2
docker-init:
Version: 0.19.0
GitCommit: de40ad0
The error message means that Redis is still loading data, i.e. in your case, the AOF file. You cannot run FLUSHALL until the loading finishes.
If you don't need the data to be loaded, you can delete the AOF file before starting Redis.
The docker-compose file is like this:
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
ports:
- "8080:8080"
volumes:
-$PWD/jenkins_home: /var/jenkins_home
networks:
- net
networks:
net:
The docker version information:
Client: Docker Engine - Community
Version: 20.10.5
API version: 1.41
Go version: go1.13.15
Git commit: 55c4c88
Built: Tue Mar 2 20:33:55 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.5
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 363e9a8
Built: Tue Mar 2 20:32:17 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.4
GitCommit: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
runc:
Version: 1.0.0-rc93
GitCommit: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
docker-init:
Version: 0.19.0
GitCommit: de40ad0
The error is:
The Compose file './docker-compose.yml' is invalid because: services.jenkins.volumes contains an invalid type, it should be an array
There should be a space between - and the value, and the value should be a string. See the docker compose volumes documentation.
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
ports:
- "8080:8080"
volumes:
- "${PWD}/jenkins_home:/var/jenkins_home"
networks:
- net
networks:
net:
You should add a space after - in docker-compose file. I also recommend using relative path instead of using PWD variable in docker-compose, so your file in volumes part would be like this:
volumes:
- /path/to/jenkins_home:/var/jenkins_home
Or:
volumes:
- jenkins_home:/var/jenkins_home
Also remember do not add space before or after :
I can't mount a file if I put it in a subdirectory (myconfig) relative to the docker-compose up command.
$PWD is /home/username/temp/ and docker-compose up run from there.
This is not working:
version: '3'
services:
proxy:
container_name: proxy
image: nginx
ports:
- "80:80"
volumes:
- $PWD/myconfig/nginx.conf:/etc/nginx/conf.d/default.conf
But this works if I copy nginx.conf to the $PWD directory:
version: '3'
services:
proxy:
container_name: proxy
image: nginx
ports:
- "80:80"
volumes:
- $PWD/nginx.conf:/etc/nginx/conf.d/default.conf
So none of these works:
$PWD/myconfig/nginx.conf
./myconfig/nginx.conf
/home/username/temp/myconfig/nginx.conf
But these work, if I copy $PWD/myconfig/nginx.conf to $PWD/config.conf:
./nginx.conf
$PWD/nginx.conf
log:
Creating network "temp_default" with the default driver
Creating proxy ... error
ERROR: for proxy Cannot create container for service proxy: not a directory
ERROR: for proxy Cannot create container for service proxy: not a directory
ERROR: Encountered errors while bringing up the project.
However when I use the long syntax, it is working with subdirectory:
version: '3'
services:
proxy:
container_name: proxy
image: nginx
ports:
- "80:80"
volumes:
- type: bind
source: $PWD/myconfig/nginx.conf
target: /etc/nginx/conf.d/default.conf
Is here something that I miss, or is this a bug or lack of documentation?
Docker and docker-compose version:
➜ ~ docker version
Client: Docker Engine - Community
Cloud integration: 1.0.7
Version: 20.10.2
API version: 1.41
Go version: go1.13.15
Git commit: 2291f61
Built: Mon Dec 28 16:17:34 2020
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.2
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 8891c58
Built: Mon Dec 28 16:15:28 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.19.0
GitCommit: de40ad0
➜ ~ docker-compose version
docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019
Your example works fine for me.
# docker-compose --version
docker-compose version 1.25.0, build unknown
Since you mentioned that even with explicit path it was not working, while it was working inside your main folder, I would suggest to check the permissions for the myconfig folder. Make sure all users are allowed access and read ('x' and 'r')
# ls -ltr
total 8
drwxr-xr-x 2 root root 4096 Feb 23 06:03 myconfig
-rw-r--r-- 1 root root 182 Feb 23 06:05 docker-compose.yml
if the the permissions are not correct, set them with chmod a+xr myconfig/ and chmod a+r myconfig/nginx.conf
I have a single node docker swarm which has just been created with docker swarm init.
I have a docker-stack.yml which contains the following deployment:
version: '3.3'
services:
website:
image: docker.pkg.github.com/foo/bar/bar:latest
ports:
- "80:80"
- "443:443"
environment:
HOST: bar.com
secrets:
- site.key
- site.cert
networks:
- foo-net
networks:
foo-net:
driver: overlay
external: true
secrets:
site.key:
external: true
site.cert:
external: true
First, I login into the registry and it tells me that the login was successful.
Second, I deploy the stack
$ docker stack deploy --with-registry-auth --compose-file docker-stack.yml foo
Creating service foo_bar
After that, the service is running and everything works as expected.
However, when I try to deploy a stack with the same command a second time (a few seconds later), it fails with the following message.
$ docker stack deploy --with-registry-auth --compose-file docker-stack.yml foo
Updating service foo_website (id: uvtqpdx74784dy9egj5i1c7sb)
image docker.pkg.github.com/foo/bar/bar:latest could not be accessed on a registry to record
its digest. Each node will access docker.pkg.github.com/foo/bar/bar:latest independently,
possibly leading to different nodes running different
versions of the image
Why does this happen?
All networks and secrets are created beforehand and my docker version is:
$ docker version
Client:
Version: 19.03.6
API version: 1.40
Go version: go1.12.17
Git commit: 369ce74a3c
Built: Fri Feb 28 23:45:43 2020
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 19.03.6
API version: 1.40 (minimum version 1.12)
Go version: go1.12.17
Git commit: 369ce74a3c
Built: Wed Feb 19 01:06:16 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.3.3-0ubuntu1~18.04.2
GitCommit:
runc:
Version: spec: 1.0.1-dev
GitCommit:
docker-init:
Version: 0.18.0
GitCommit:
I appreciate any help!
Thanks