docker-compose mount single file with directory - docker

I can't mount a file if I put it in a subdirectory (myconfig) relative to the docker-compose up command.
$PWD is /home/username/temp/ and docker-compose up run from there.
This is not working:
version: '3'
services:
proxy:
container_name: proxy
image: nginx
ports:
- "80:80"
volumes:
- $PWD/myconfig/nginx.conf:/etc/nginx/conf.d/default.conf
But this works if I copy nginx.conf to the $PWD directory:
version: '3'
services:
proxy:
container_name: proxy
image: nginx
ports:
- "80:80"
volumes:
- $PWD/nginx.conf:/etc/nginx/conf.d/default.conf
So none of these works:
$PWD/myconfig/nginx.conf
./myconfig/nginx.conf
/home/username/temp/myconfig/nginx.conf
But these work, if I copy $PWD/myconfig/nginx.conf to $PWD/config.conf:
./nginx.conf
$PWD/nginx.conf
log:
Creating network "temp_default" with the default driver
Creating proxy ... error
ERROR: for proxy Cannot create container for service proxy: not a directory
ERROR: for proxy Cannot create container for service proxy: not a directory
ERROR: Encountered errors while bringing up the project.
However when I use the long syntax, it is working with subdirectory:
version: '3'
services:
proxy:
container_name: proxy
image: nginx
ports:
- "80:80"
volumes:
- type: bind
source: $PWD/myconfig/nginx.conf
target: /etc/nginx/conf.d/default.conf
Is here something that I miss, or is this a bug or lack of documentation?
Docker and docker-compose version:
➜ ~ docker version
Client: Docker Engine - Community
Cloud integration: 1.0.7
Version: 20.10.2
API version: 1.41
Go version: go1.13.15
Git commit: 2291f61
Built: Mon Dec 28 16:17:34 2020
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.2
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 8891c58
Built: Mon Dec 28 16:15:28 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.3
GitCommit: 269548fa27e0089a8b8278fc4fc781d7f65a939b
runc:
Version: 1.0.0-rc92
GitCommit: ff819c7e9184c13b7c2607fe6c30ae19403a7aff
docker-init:
Version: 0.19.0
GitCommit: de40ad0
➜ ~ docker-compose version
docker-compose version 1.27.4, build 40524192
docker-py version: 4.3.1
CPython version: 3.7.7
OpenSSL version: OpenSSL 1.1.0l 10 Sep 2019

Your example works fine for me.
# docker-compose --version
docker-compose version 1.25.0, build unknown
Since you mentioned that even with explicit path it was not working, while it was working inside your main folder, I would suggest to check the permissions for the myconfig folder. Make sure all users are allowed access and read ('x' and 'r')
# ls -ltr
total 8
drwxr-xr-x 2 root root 4096 Feb 23 06:03 myconfig
-rw-r--r-- 1 root root 182 Feb 23 06:05 docker-compose.yml
if the the permissions are not correct, set them with chmod a+xr myconfig/ and chmod a+r myconfig/nginx.conf

Related

Docker Compose "volumes Additional property is not allowed" or "volumes must be a mapping"

I am using Maven to interpolate a docker compose file, in order to map the working directory in either Linux and Windows. Interpolation works as intended on both OSs.
In my local Windows environment, when running "docker compose up" I get both containers with the mapped volume (which already exists on the host machine), without specifying "volumes: " at top-level, only at service-level.
However, if I try to run the same setup in linux-based TeamCity, I get the following message "service "job_controller" refers to undefined volume path/to/target/classes: invalid compose project"
After checking others' answers from here, I've understood that I also have to specify "volumes:" at top-level, which I did at the bottom of the compose file.
Now, I am prompted with "volumes Additional property /opt/buildagent/work/9857567c5e342350/path/to/target/classes is not allowed"
name: Distributed
services:
create_database:
container_name: create_database
command:
- ./script.sh
- deployer
- -f
- ../config/product-mssql-v11.manifest.yaml
- drop-create-database-properties
image: alpine-3-corretto-11-wildfly-11.11.0-SNAPSHOT
networks:
- deploy
volumes:
- C:\\SourceCode\\Path\\to\\target/classes:/opt/product/config
healthcheck:
test: ["CMD", "/opt/product/script.sh", "deployer", "-f", "/opt/product/config/product-mssql-v11.manifest.yaml", "healthy"]
interval: 20s
timeout: 60s
retries: 5
job_controller:
container_name: job_controller
environment:
DEPLOYMENT_MANIFEST: /opt/product/config/main.manifest.yaml
PROPERTIES_FILE_NAME: /opt/product/config/risk-wildfly.properties
JAVA_OPTS: "-Xms1g -Xmx4g -XX:MetaspaceSize=96M -XX:MaxMetaspaceSize=1g -Djava.net.preferIPv4Stack=true -Djava.net.preferIPv4Addresses=true"
ports:
- 8080:8080
image: alpine-3-corretto-11-wildfly-11.11.0-SNAPSHOT
volumes:
- C:\\SourceCode\\Path\\to\\target/classes:/opt/product/config
networks:
- deploy
depends_on:
create_database:
condition: service_completed_successfully
restart: on-failure
healthcheck:
test: ["CMD", "/opt/product/script.sh", "health-check", "--context-path","product"]
interval: 20s
timeout: 60s
retries: 5
networks:
deploy:
name: deploy
external: true
volumes:
C:\\SourceCode\\Path\\to\\target/classes:
external: true
Now, locally, if I try to run "docker compose up" with the "volumes: " specified at the bottom I get as well the same "volumes Additional property C:\SourceCode\Path\to\target/classes is not allowed"
If, instead of
volumes:
C:\\SourceCode\\Path\\to\\target/classes:
external: true
I use
volumes:
I get the "volumes: " must be a mapping.
So neither of this works.
C:\>docker compose version
Docker Compose version v2.10.2
C:\>docker-compose version
docker-compose version 1.29.2, build 5becea4c
docker-py version: 5.0.0
CPython version: 3.9.0
OpenSSL version: OpenSSL 1.1.1g 21 Apr 2020
C:\>docker version
Client:
Cloud integration: v1.0.29
Version: 20.10.17
API version: 1.41
Go version: go1.17.11
Git commit: 100c701
Built: Mon Jun 6 23:09:02 2022
OS/Arch: windows/amd64
Context: default
Experimental: true
Server: Docker Desktop 4.12.0 (85629)
Engine:
Version: 20.10.17
API version: 1.41 (minimum version 1.12)
Go version: go1.17.11
Git commit: a89b842
Built: Mon Jun 6 23:01:23 2022
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.6.8
GitCommit: 9cd3357b7fd7218e4aec3eae239db1f68a5a6ec6
runc:
Version: 1.1.4
GitCommit: v1.1.4-0-g5fd4c4d
docker-init:
Version: 0.19.0
GitCommit: de40ad0
How can I run this successfully in both OSs considering the volume mapping?
If you want to map specific folder from host to docker container you don't need root section
volumes:
at all
It's used to create named volumes somewhere inside docker and reference them by name in volumes section of service definition (and across multiple docker-compose files if external flag is set)

do not add `name` key value when generating docker compose file using `docker compose config` command

There seems to be a discrepancy in the implementation between docker compose CLI command and docker-compose CLI. Although they follow the same specification for Compose.
Services
As a simple example, I have two compose files
docker-compose.hello-world.yml
version: '3.7'
services:
hello:
image: hello-world:nano-server
container_name: hello_world
docker-compose.base.yml
very simple base file
version: '3.7'
volumes:
dummy-vol:
I can override them using the -f flag as follows:
docker compose -f docker-compose.hello-world.yml -f docker-compose.base.yml config > docker-compose.yml
Discrepancy
If one uses the above command using the docker compose CLI in built in the Docker Engine:
docker compose -f docker-compose.hello-world.yml -f docker-compose.base.yml config
The resultant configuration generated is:
here test-con is just the name of the directory I have the files in
name: test-con
services:
hello:
container_name: hello_world
image: hello-world:nano-server
networks:
default: null
networks:
default:
name: test-con_default
However when using the docker-compose CLI,
docker-compose -f docker-compose.hello-world.yml -f docker-compose.base.yml config
generates the following output:
services:
hello:
container_name: hello_world
image: hello-world:nano-server
version: '3.7'
volumes:
test: {}
As you can see name key value is not generated with docker-compose.
Is there a way to suppress generating the name key-value from docker compose CLI?
System Specs
docker version
Client:
Version: 20.10.12
API version: 1.41
Go version: go1.17.5
Git commit: e91ed5707e
Built: Mon Dec 13 22:31:40 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server:
Engine:
Version: 20.10.12
API version: 1.41 (minimum version 1.12)
Go version: go1.17.5
Git commit: 459d0dfbbb
Built: Mon Dec 13 22:30:43 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: v1.6.1
GitCommit: 10f428dac7cec44c864e1b830a4623af27a9fc70.m
runc:
Version: 1.1.0
GitCommit: v1.1.0-0-g067aaf85
docker-init:
Version: 0.19.0
GitCommit: de40ad0
docker compose version
Docker Compose version 2.3.3
docker-compose version
docker-compose version 1.29.2, build unknown
docker-py version: 5.0.3
CPython version: 3.10.2
OpenSSL version: OpenSSL 1.1.1m 14 Dec 2021
This discrepancy is resolved when upgraded to v2.4.1 for docker.
Upon conducting the same test:
docker-compose -f docker-compose.hello-world.yml config > std.yml
as well as
docker compose -f docker-compose.hello-world.yml config > cli.yml
and checking to see if there is any difference using diff
diff std.yml cli.yml
now provides the same configuration in both tools
name: test-com
services:
hello:
container_name: hello_world
image: hello-world:nano-server
networks:
default: null
networks:
default:
name: test-com_default

ERROR: The Compose file './docker-compose.yml' is invalid because: services.jenkins.volumes contains an invalid type, it should be an array

The docker-compose file is like this:
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
ports:
- "8080:8080"
volumes:
-$PWD/jenkins_home: /var/jenkins_home
networks:
- net
networks:
net:
The docker version information:
Client: Docker Engine - Community
Version: 20.10.5
API version: 1.41
Go version: go1.13.15
Git commit: 55c4c88
Built: Tue Mar 2 20:33:55 2021
OS/Arch: linux/amd64
Context: default
Experimental: true
Server: Docker Engine - Community
Engine:
Version: 20.10.5
API version: 1.41 (minimum version 1.12)
Go version: go1.13.15
Git commit: 363e9a8
Built: Tue Mar 2 20:32:17 2021
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.4.4
GitCommit: 05f951a3781f4f2c1911b05e61c160e9c30eaa8e
runc:
Version: 1.0.0-rc93
GitCommit: 12644e614e25b05da6fd08a38ffa0cfe1903fdec
docker-init:
Version: 0.19.0
GitCommit: de40ad0
The error is:
The Compose file './docker-compose.yml' is invalid because: services.jenkins.volumes contains an invalid type, it should be an array
There should be a space between - and the value, and the value should be a string. See the docker compose volumes documentation.
version: '3'
services:
jenkins:
container_name: jenkins
image: jenkins/jenkins
ports:
- "8080:8080"
volumes:
- "${PWD}/jenkins_home:/var/jenkins_home"
networks:
- net
networks:
net:
You should add a space after - in docker-compose file. I also recommend using relative path instead of using PWD variable in docker-compose, so your file in volumes part would be like this:
volumes:
- /path/to/jenkins_home:/var/jenkins_home
Or:
volumes:
- jenkins_home:/var/jenkins_home
Also remember do not add space before or after :

Docker Swarm: docker stack deploy fails when executed second time

I have a single node docker swarm which has just been created with docker swarm init.
I have a docker-stack.yml which contains the following deployment:
version: '3.3'
services:
website:
image: docker.pkg.github.com/foo/bar/bar:latest
ports:
- "80:80"
- "443:443"
environment:
HOST: bar.com
secrets:
- site.key
- site.cert
networks:
- foo-net
networks:
foo-net:
driver: overlay
external: true
secrets:
site.key:
external: true
site.cert:
external: true
First, I login into the registry and it tells me that the login was successful.
Second, I deploy the stack
$ docker stack deploy --with-registry-auth --compose-file docker-stack.yml foo
Creating service foo_bar
After that, the service is running and everything works as expected.
However, when I try to deploy a stack with the same command a second time (a few seconds later), it fails with the following message.
$ docker stack deploy --with-registry-auth --compose-file docker-stack.yml foo
Updating service foo_website (id: uvtqpdx74784dy9egj5i1c7sb)
image docker.pkg.github.com/foo/bar/bar:latest could not be accessed on a registry to record
its digest. Each node will access docker.pkg.github.com/foo/bar/bar:latest independently,
possibly leading to different nodes running different
versions of the image
Why does this happen?
All networks and secrets are created beforehand and my docker version is:
$ docker version
Client:
Version: 19.03.6
API version: 1.40
Go version: go1.12.17
Git commit: 369ce74a3c
Built: Fri Feb 28 23:45:43 2020
OS/Arch: linux/amd64
Experimental: false
Server:
Engine:
Version: 19.03.6
API version: 1.40 (minimum version 1.12)
Go version: go1.12.17
Git commit: 369ce74a3c
Built: Wed Feb 19 01:06:16 2020
OS/Arch: linux/amd64
Experimental: false
containerd:
Version: 1.3.3-0ubuntu1~18.04.2
GitCommit:
runc:
Version: spec: 1.0.1-dev
GitCommit:
docker-init:
Version: 0.18.0
GitCommit:
I appreciate any help!
Thanks

Deploy container to docker swarm with compose: publish Additional property publish is not allowed

The following is not working, do you know why?
This is the error: publish Additional property publish is not allowed
version: "3"
services:
app:
image: golang:1.9
working_dir: /app/src
command: go run main.go
volumes:
- ./app:/app/src/
publish:
- "8080:8080"
docker version
Client:
Version: 17.06.1-ce
API version: 1.30
Go version: go1.8.3
Git commit: 874a737
Built: Thu Aug 17 22:53:38 2017
OS/Arch: darwin/amd64
Server:
Version: 17.07.0-ce
API version: 1.31 (minimum version 1.12)
Go version: go1.8.3
Git commit: 8784753
Built: Tue Aug 29 17:46:50 2017
OS/Arch: linux/amd64
Experimental: false
You should be using ports and not publish
version: "3"
services:
app:
image: golang:1.9
working_dir: /app/src
command: go run main.go
volumes:
- ./app:/app/src/
ports:
- "8080:8080"

Resources