My Scenario
I am trying to build docker swarm with docker-compose.yml file
version: '3'
services:
myapp-nginx:
image: nginx:1.10
volumes:
- ./nginx/certs:/etc/nginx/certs
ports:
- 80:80
- 443:443
using build command as below
$ docker stack deploy --compose-file docker-compose.yml myapp
Creating network myapp_default
Creating service myapp_myapp-nginx
My system have Docker Version 17.03.1-ce-mac5 (16048)
Problem
Volumes mounts are not working, If I comment out volumes: - ./nginx/certs:/etc/nginx/certs then everything works fine.
Otherwise nginx doesn't work at all, I see
CAGTAM1059934:docker iskumar$ docker stack ps myapp
ID NAME IMAGE NODE DESIRED STATE CURRENT STATE ERROR PORTS
efjia8b7y6jd myapp_myapp-nginx.1 nginx:1.10 manager1 Ready Rejected 5 seconds ago "invalid mount config for type…"
c9c6ogh11osi \_ myapp_myapp-nginx.1 nginx:1.10 worker2 Shutdown Rejected 10 seconds ago "invalid mount config for type…"
iveikxpnbq6x \_ myapp_myapp-nginx.1 nginx:1.10 worker1 Shutdown Rejected 15 seconds ago "invalid mount config for type…"
Is there any special way to mount volumes in docker-compose v3?
You must use absolute paths when mounting volumes, also with docker run.
$ docker run --rm -it -v ./eos:/eos:ro ubuntu
docker: Error response from daemon: create ./eos: "./eos" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intented to pass a host directory, use absolute path.
See 'docker run --help'
I was running docker stack deploy from my macbook instead on boot2docker VMs. Which is why system couldn't find those network mount source, even though I had manager1 activated using eval $(docker-machine env manager1)
First I used --no-trunc argument to view full error message.
$> docker node ps --no-trunc
---
Showed "invalid mount config for type "bind": bind source path does not exist"
---
After that copies the files from host system (My MacBook) to VM and launching stack worked fine.
Credit to https://github.com/docker/docker/issues/31202#issuecomment-281312704
Related
I'm having trouble demonstrating that data I generate on a shared volume is persistent, and I can't figure out why. I have a very simple docker-compose file:
version: "3.9"
# Define network
networks:
sorcernet:
name: sorcer_net
# Define services
services:
preclean:
container_name: cleaner
build:
context: .
dockerfile: DEESfile
image: dees
networks:
- sorcernet
volumes:
- pgdata:/usr/share/appdata
#command: python run dees.py
process:
container_name: processor
build:
context: .
dockerfile: OASISfile
image: oasis
networks:
- sorcernet
volumes:
- pgdata:/usr/share/appdata
volumes:
pgdata:
name: pgdata
Running the docker-compose file to keep the containers running in the background:
vscode ➜ /com.docker.devenvironments.code $ docker compose up -d
[+] Running 4/4
⠿ Network sorcer_net Created
⠿ Volume "pgdata" Created
⠿ Container processor Started
⠿ Container cleaner Started
Both images are running:
vscode ➜ /com.docker.devenvironments.code $ docker image ls
REPOSITORY TAG IMAGE ID CREATED SIZE
oasis latest e2399b9954c8 9 seconds ago 1.09GB
dees latest af09040befd5 31 seconds ago 1.08GB
and the volume shows up as expected:
vscode ➜ /com.docker.devenvironments.code $ docker volume ls
DRIVER VOLUME NAME
local pgdata```
Running the docker container, I navigate to the volume folder. There's nothing in the folder -- this is expected.
vscode ➜ /com.docker.devenvironments.code $ docker run -it oasis
[root#049dac037802 opt]# cd /usr/share/appdata/
[root#049dac037802 appdata]# ls
[root#049dac037802 appdata]#
Since there's nothing in the folder, I create a file in called "dog.txt" and recheck the folder contents. The file is there. I exit the container.
[root#049dac037802 appdata]# touch dog.txt
[root#049dac037802 appdata]# ls
dog.txt
[root#049dac037802 appdata]# exit
exit
To check the persistence of the data, I re-run the container, but nothing is written to the volume.
vscode ➜ /com.docker.devenvironments.code $ docker run -it oasis
[root#1787d76a54b9 opt]# cd /usr/share/appdata/
[root#1787d76a54b9 appdata]# ls
[root#1787d76a54b9 appdata]#
What gives? I've tried defining the volume as persistent, and I know each of the images have a folder location at /usr/share/appdata.
If you want to check the persistence of the data in the containers defined in your docker compose, the --volumes-from flag is the way to go
When you run
docker run -it oasis
This newly created container shares the same image, but it doesn't know anything about the volumes defined.
In order to link the volume to the new container run this
docker run -it --volumes-from $CONTAINER_NAME_CREATED_FROM_COMPOSE oasis
Now this container shares the volume pgdata.
You can go ahead and create files at /usr/share/appdata and validate their persistence
I'm deploying few Docker services via docker-compose with remote context. I configured it to use SSH:
docker context create remote --docker "host=ssh://user#my.remote.host"
docker context use remote
On the remote host I have multiple configuration files which I want to mount into the Docker. It's working fine when I'm trying with docker CLI:
docker run -v /home/user/run:/test -it alpine:3.11
# ls -la /test
-> shows remote files correctly here
But when I'm starting it using docker-compose with config file:
version: "3.3"
services:
nginx:
image: nginx:1.17.10-alpine
container_name: nginx
restart: unless-stopped
volumes:
- ${HOME}/run/nginx.conf:/etc/nginx/nginx.conf
ports:
- "80:80"
- "443:443"
It's trying to mount local files instead of remote for some reason and fails with error:
ERROR: for nginx Cannot start service nginx: OCI runtime create failed: container_linux.go:296: starting container process caused "process_linux.go:398: container init caused \"rootfs_linux.go:58: mounting \\\"/home/local-user/run/nginx.conf\\\" to rootfs \\\"/hdd/docker/overlay2/c869ef9f2c983d33245fe1b4360eb602d718786ba7d0245d36c40385f7afde65/merged\\\" at \\\"/hdd/docker/overlay2/c869ef9f2c983d33245fe1b4360eb602d718786ba7d0245d36c40385f7afde65/merged/etc/nginx/nginx.conf\\\" caused \\\"not a directory\\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
Is it possible to mount remote resources via docker-compose similar to standard Docker CLI?
You need to explicitly set DOCKER_HOST to access your remote docker host from docker-compose.
From the compose documentation
Compose CLI environment variables
DOCKER_HOST
Sets the URL of the docker daemon. As with the Docker client, defaults
to unix:///var/run/docker.sock.
In your given case, docker context use remote sets current context to remote
only for your docker command. docker-compose still uses your default (local) context. In order for docker-compose to detect it, you must pass it via the DOCKER_HOST environment variable.
Example:
$ export DOCKER_HOST=ssh://user#my.remote.host
$ docker-compose up
I'm trying to create a custom Nextcloud config locally, then have the ability to mount it to the appropriate folder using volumes as defined here: https://github.com/nextcloud/docker#persistent-data. All the volume mounts work except for the config mount... Why is that being treated differently here?
Steps to reproduce
0) Enter a new/emptry directory (containing no sub-directories or additional files).
1) Create a docker-compose.yml file containing only the below contents:
version: "3.4"
services:
nextcloud:
image: nextcloud:latest
volumes:
- "./nextcloud/custom_apps:/var/www/html/custom_apps"
- "./nextcloud/config:/var/www/html/config"
- "/data/nextcloud:/var/www/html/data"
- "./themes:/var/www/html/themes"
2) docker-compose up -d
Expected behavior
Work. I should be able to see the /var/www/html/config contents locally at ./nextcloud/config, and then insert a customer config.php, which is then updated within the container.
Actual behavior
An ERROR when bringing up the container, specific to the config directory. If I remove the ./nextcloud/config:/var/www/html/config volume mount above, then the container will start without error.
ERROR message
ERROR: for nextcloud Cannot start service nextcloud: OCI runtime create failed: container_linux.go:349: starting container process caused "process_linux.go:449: container init caused \"rootfs_linux.go:58: mounting \\"/home/user/Nextcloud-test/nextcloud/config\\" to rootfs \\"/var/lib/docker/overlay2/41b567141e23b16cf5e4f99f4c33703fc9a533aa5a4bef68fbba70a74842ca88/merged\\" at \\"/var/lib/docker/overlay2/41b567141e23b16cf5e4f99f4c33703fc9a533aa5a4bef68fbba70a74842ca88/merged/var/www/html/config\\" caused \\"not a directory\\"\"": unknown: Are you trying to mount a directory onto a file (or vice-versa)? Check if the specified host path exists and is the expected type
ERROR: Encountered errors while bringing up the project.
Server configuration
Operating system:
Operating System: Debian GNU/Linux 10 (buster)
Kernel: Linux 4.19.0-8-cloud-amd64
Architecture: x86-64
Image
nextcloud:latest (apache)
I could not reproduce using your steps (Ubuntu 18.04).
From here, running docker-compose up -d then docker-compose logs, I see no errors. Then, when running docker inspect on the container, I see the following:
...
"Volumes": {
"/var/www/html": {},
"/var/www/html/config": {},
"/var/www/html/custom_apps": {},
"/var/www/html/data": {},
"/var/www/html/themes": {}
},
...
Which suggests the mount has worked without problem.
What I suggest you do:
Check the directory ./nextcloud/config exists and is not a file
Check your Docker and Docker Compose installation is up-to-date
Try running the Docker container with docker run -it -v ./nextcloud/config:/var/www/html/config <containername> /bin/bash to explore if the mount works manually
Try to do the same on a minimal example such as the Getting started example
I would like to use a standard way of running my docker containers. I have have been keeping a docker_run.sh file, but docker-compose.yml looks like a better choice. This seems to work great until I try to access my website running in the container. The ports don't seem to be set up correctly.
Using the following docker_run.sh, I can access the website at localhost. I expected the following docker-compose.yml file to have the same results when I use the docker-compose run web command.
docker_run.sh
docker build -t web .
docker run -it -v /home/<user>/git/www:/var/www -p 80:80/tcp -p 443:443/tcp -p 3316:3306/tcp web
docker-compose.yml
version: '3'
services:
web:
image: web
build: .
ports:
- "80:80"
- "443:443"
- "3316:3306"
volumes:
- "../www:/var/www"
Further analysis
The ports are reported as the same in docker ps and docker-compose ps. Note: these were not up at the same time.
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
<id> web "/usr/local/scripts/…" About an hour ago Up About an hour 0.0.0.0:80->80/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:3307->3306/tcp <name>
$ docker-compose ps
Name Command State Ports
---------------------------------------------------------------------------------------------------------------
web /usr/local/scripts/start_s ... Up 0.0.0.0:3316->3306/tcp, 0.0.0.0:443->443/tcp, 0.0.0.0:80->80/tcp
What am I missing?
As #richyen suggests in a comment, you want docker-compose up instead of docker-compose run.
docker-compose run...
Runs a one-time command against a service.
That is, it's intended to run something like a debugging shell or a migration script, in the overall environment specified by the docker-compose.yml file, but not the standard command specified in the Dockerfile (or the override in the YAML file).
Critically to your question,
...docker-compose run [...] does not create any of the ports specified in the service configuration. This prevents port collisions with already-open ports. If you do want the service’s ports to be created and mapped to the host, specify the --service-ports flag.
Beyond that, the docker run command you show and the docker-compose.yml file should be essentially equivalent.
You don't run docker-compose.yamls the same way that you would run a local docker image that you have either installed or created on your machine. docker-compose files are typically launched running the command docker-compose up -d to run in detached mode. Then when you run docker ps you should see it running. You can also run docker-compose ps as you did above.
I am trying to run bamboo on server using docker containers. When i running on local machine work normally and volume save datas successfully. But when i run same docker compose file on server, volume data not save my datas.
docker-compose.yml
version: '3.2'
services:
bamboo:
container_name: bamboo-server_test
image: atlassian/bamboo-server
volumes:
- ./volumes/bamboo_test_vol:/var/atlassian/application-data/bamboo
ports:
- 8085:8085
volumes:
bamboo_test_vol:
Run this compose file on local machine
$ docker-compose up -d
Creating network "test_default" with the default driver
Creating volume "test_bamboo_test_vol" with default driver
Creating bamboo-server_test ... done
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
916c98ca1a9d atlassian/bamboo-server "/entrypoint.sh" 24 minutes ago Up 24 minutes 0.0.0.0:8085->8085/tcp, 54663/tcp bamboo-server_test
$ ls
docker-compose.yml volumes
$ cd volumes/bamboo_test_vol/
$ ls
bamboo.cfg.xml logs
localhost:8085
Run this compose file on server
$ ssh <name>#<ip_address>
password for <name>:
$ docker-compose up -d
Creating network "test_default" with the default driver
Creating volume "test_bamboo_test_vol" with default driver
Creating bamboo-server_test ... done
$ docker ps
CONTAINER ID IMAGE COMMAND CREATED STATUS PORTS NAMES
38b77e1b736f atlassian/bamboo-server "/entrypoint.sh" 12 seconds ago Up 11 seconds 0.0.0.0:8085->8085/tcp, 54663/tcp bamboo-server_test
$ ls
docker-compose.yml volumes
$ cd volumes/
$ cd bamboo_test_vol/
$ ls
$ # VOLUME PATH IS EMPTY
server_ip:8085
I didn't have this problem when I tried the same process for jira-software. Why can't it work through the bamboo server even though I use the exact same compose file?
I had the same problem when I wanted to upgrade my Bamboo server instance with my mounted host volume for the bamboo-home directory.
The following was in my docker-compose file:
version: '2.2'
bamboo-server:
image: atlassian/bamboo-server:${BAMBOO_VERSION}
container_name: bamboo-server
environment:
TZ: 'Europe/Berlin'
restart: always
init: true
volumes:
- ./bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo
ports:
- "8085:8085"
- "54663:54663"
When i started with docker-compose up -d bamboo-server, the container never took the files from the host system. So I tried it first without docker-compose, following the instructions of Atlassian Bamboo with the following command:
docker run -v ./bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo --name="bamboo-server" --init -d -p 54663:54663 -p 8085:8085 atlassian/bamboo-server:${BAMBOO_VERSION}
The following error message was displayed:
docker: Error response from daemon: create ./bamboo/bamboo-server/data: "./bamboo/bamboo-server/data" includes invalid characters for a local volume name, only "[a-zA-Z0-9][a-zA-Z0-9_.-]" are allowed. If you intended to pass a host directory, use absolute path.
So I converted the error message and took the absolute path:
docker run -v /var/project/bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo --name="bamboo-server" --init -d -p 54663:54663 -p 8085:8085 atlassian/bamboo-server:${BAMBOO_VERSION}
After the successful start, I switched to the docker container via SSH and all files were as usual in the docker directory.
I transferred the whole thing to the docker-compose file and took the absolute path in the volumes section. Subsequently it also worked with the docker-compose file.
My docker-compose file then looked like this:
[...]
init: true
volumes:
- /var/project/bamboo/bamboo-server/data:/var/atlassian/application-data/bamboo
ports:
[...]
Setting up a containerized Bamboo Server is not supported for these reasons;
Repository-stored Specs (RSS) are no longer processed in Docker by default. Running RSS in Docker was not possible because;
there is no Docker capability added on the Bamboo server by default,
the setup would require running Docker in Docker.