How do I create an external volume using docker compose? - docker

Basing on this Node-RED tutorial, I'm trying to mount an external volume with the Node-RED files outside the docker machine. I'm using the following docker-compose file:
version: "3.7"
services:
node-red:
image: nodered/node-red:latest
environment:
- TZ=Europe/Amsterdam
ports:
- "2000:1880"
networks:
- node-red-net
volumes:
- node-red-data:/home/user/node-red1
volumes:
node-red-data:
networks:
node-red-net:
However, even though this file works fine when I run docker-compose up, the volume exists only inside the docker machine. I've tried adding the line external: true in volumes but I get the following error:
ERROR: In file './docker-compose.yml', volume 'external' must be a mapping not a boolean.
What am I missing? How do I mount an external volume using docker-compose files?

I ended up finding a related question with this answer. There are multiple answers that didn't work for me there (also there's no accepted answer). The syntax that worked was:
node-red-data:
driver: local
driver_opts:
o: bind
type: none
device: /path/external/folder
So the final dockerfile that works after running docker-compose up is:
version: "3.7"
services:
node-red:
image: nodered/node-red:latest
environment:
- TZ=Europe/Amsterdam
ports:
- "2000:1880"
networks:
- node-red-net
volumes:
- node-red-data:/data
container_name: node-red
volumes:
node-red-data:
driver: local
driver_opts:
o: bind
type: none
device: "/home/user/node-red1"
networks:
node-red-net:
Update
If we don't mind having a random name for the volume, the following solution also works fine:
version: "3.7"
services:
node-red:
image: nodered/node-red:latest
environment:
- TZ=Europe/Amsterdam
ports:
- "2000:1880"
volumes:
- /home/user/node-red1:/data
container_name: node-red

Related

docker-compose: service "gateway" refers to undefined volume ${PWD}/config/gateway/gateway-configuration.ini: invalid compose project

My goal: generate docker-compose.yaml from docker-compose.yaml and docker-compose.override.yaml and keep the variables as they are now = without interpolate
I've tried to run
docker compose -f docker-compose.yaml -f docker-compose.override.yaml convert --no-interpolate > new-docker-compose.yaml
Here is my docker-compose.yaml:
version: "3.5"
services:
redis-db:
image: redislabs/rejson:2.0.11
container_name: redis-db
restart: unless-stopped
volumes:
- redis-storage-vol:/data
- ${PWD}/config/redis/redis.conf:/usr/local/etc/redis/redis.conf:ro
ports:
- 6379:6379
runner:
image: "${REPO}/runner/${RUNNER_CPU_IMAGE}:${RUNNER_CPU_TAG}"
container_name: runner
restart: unless-stopped
volumes:
- data-storage-vol:/data
- ${PWD}/config/runner/runner-configuration.ini:/configuration.ini:ro
- "${PWD}/solutions/${ALGO}:/home/scripts/algorithmic_solutions_list.txt:ro"
depends_on:
- "redis-db"
gateway:
image: "${REPO}/gateway/gateway-server:${GATEWAY_TAG}"
container_name: gateway
restart: unless-stopped
volumes:
- data-storage-vol:/data
- ${PWD}/config/gateway/gateway-configuration.ini:/configuration.ini:ro
ports:
- 8000:8000
depends_on:
- "redis-db"
volumes:
data-storage-vol:
driver_opts:
type: "tmpfs"
device: "tmpfs"
o: "size=5g,uid=1000"
redis-storage-vol:
driver: local
docker-compose.override.yaml
version: "3.5"
services:
runner:
image: "${REPO}/runner/${RUNNER_GPU_IMAGE}:${RUNNER_GPU_TAG}"
deploy:
resources:
reservations:
devices:
- driver: nvidia
capabilities: [ GPU ]
What I've tried:
Run docker compose convert without flag --no-interpolate, it worked well but the variables was populated.
Run like the example - but got this error:
service "gateway" refers to undefined volume ${PWD}/config/gateway/gateway-configuration.ini: invalid compose project
I want to keep using docker compose commands and not edit files after its created.

Two containers one compose file network not working

This should be something simple. I have a docker compose file that starts a DB2 container and another Java application container that does work against the database. The purpose of this stack is for solely to verify testing, the values in the database should not persist. But, I need the Java container to be able to make a JDBC connection to DB2. Right now, it's refusing, but I'm not sure why. I created a common network for both of them (I thought).
My docker-compose.yml
version: "3.2"
services:
ssc-file-generator-db2-test:
container_name: "ssc-file-generator-db2-test"
image: ibmcom/db2:latest
hostname: db2server
privileged: true
ports:
- 50100:50000
- 55100:55000
networks:
- back-tier
restart: "no"
volumes:
- setup-sql:/setup-sql
- db2-shell-scripts:/var/custom
- host-dirs:/host-dirs
env_file:
- acceptance-run.environment
ssc-file-generator:
container_name: "ssc-file-generator_testing"
image: "ssc-file-generator:latest"
depends_on: ["ssc-file-generator-db2-test"]
entrypoint: ["sh", "/ssc-file-generator/bin/wait-for-db2.sh"]
env_file: ["acceptance-run.environment"]
networks:
- back-tier
restart: "no"
volumes:
- setup-sql:/setup-sql
- db2-shell-scripts:/var/custom
- host-dirs:/host-dirs
networks:
back-tier: {}
volumes:
setup-sql:
driver: local
driver_opts:
o: bind
type: none
device: ./setup-sql
db2-shell-scripts:
driver: local
driver_opts:
o: bind
type: none
device: ./db2-shell-scripts
host-dirs:
driver: local
driver_opts:
o: bind
type: none
device: ./host-dirs
The db in docker compose context should be available at the host:
ssc-file-generator-db2-test:50000
ssc-file-generator-db2-test:55000
try to add healthcheck to db service. To start ssc-file-generator after ssc-file-generator-db2-test successfully started and ready for connections.
https://docs.docker.com/compose/compose-file/compose-file-v2/#depends_on
ssc-file-generator-db2-test:
...
healthcheck:
test: some command that checks db2 is working
ssc-file-generator:
...
depends_on:
ssc-file-generator-db2-test:
condition: service_healthy

docker volume mapped with filesystem (windows)

I have this lighted docker-compose file :
version: "2"
services:
orthanc:
build: orthanc
restart: unless-stopped
ports: ["${PORT}:8042"]
volumes: ["orthanc-storage:/var/lib/orthanc/db:Z"]
[...]
volumes:
orthanc-storage:
I don't understand how I finally define the volume orthanc-storage to link C:/tmp for example.
Someone could explain it to me ? thank you.
By default the driver used is local to assign volumes and creates the volume on host at /var/lib/docker/volumes/<project_name>_dbdata in Linux. Not confirm where does it points in windows.
You can define the volumes to use different driver and give the path to the volume like this
version: "2"
services:
orthanc:
build: orthanc
restart: unless-stopped
ports: ["${PORT}:8042"]
volumes: ["orthanc-storage:/var/lib/orthanc/db:Z"]
[...]
volumes:
orthanc-storage:
volumes:
orthanc-storage:
driver_opts:
type: 'none'
o: 'bind'
device: "C:/tmp"

Docker does not support storing secrets on Windows home system using Docker toolbox

Using Docker toolbox on Windows 10 Home, Docker version 19.03, we have created a docker-compose.yml and added a secrets file as JSON, it runs fine on a Mac system, but it is unable to run the same in Windows 10 Home.
Error after running docker-compose up:
ERROR: for orthancserver Cannot create container for service orthanc: invalid mount config for type
"bind": invalid mount path: 'C:/Users/ABC/Desktop/Project/orthanc.json' mount path must be absolute
docker-compose.yml:
version: "3.7"
services:
orthanc:
image: jodogne/orthanc-plugins:1.6.1
command: /run/secrets/
container_name: orthancserver
restart: always
ports:
- "4242:4242"
- "8042:8042"
networks:
- mynetwork
volumes:
- /tmp/orthanc-db/:/var/lib/orthanc/db/
secrets:
- orthanc.json
dcserver:
build: ./dc_node_server
depends_on:
- orthanc
container_name: dcserver
restart: always
ports:
- "5001:5001"
networks:
- mynetwork
volumes:
- localdb:/database
volumes:
localdb:
external: true
networks:
mynetwork:
external: true
secrets:
orthanc.json:
file: orthanc.json
orthanc.json file kept next to docker-compose.yml
Found an alternative solution for windows 10 home, with docker toolbox. as commented by #Schwarz54, the file-sharing works well with docker volume for Dockerized Orthanc server.
Add shared folder:
Open Oracle VM manager
Go to setting of default VM
Click Shared Folders
Add C:\ drive to the list
Edit docker-compose.yml to transfer the config file to Orthanc via volume
version: "3.7"
services:
orthanc:
image: jodogne/orthanc-plugins:1.6.1
command: /run/secrets/
container_name: orthancserver
restart: always
ports:
- "4242:4242"
- "8042:8042"
networks:
- mynetwork
volumes:
- /tmp/orthanc-db/:/var/lib/orthanc/db/
- /c/Users/ABCUser/Desktop/Project/orthanc.json:/etc/orthanc/orthanc.json:ro
dcserver:
build: ./dc_node_server
depends_on:
- orthanc
container_name: dcserver
restart: always
ports:
- "5001:5001"
networks:
- mynetwork
volumes:
- localdb:/database
volumes:
localdb:
external: true
networks:
mynetwork:
external: true

docker compose issue on windows 10

I have an opensource project cloned, and it has docker-compose.yml.
I execute
docker-compose up
But I see error:
ERROR: could not find plugin bridge in v1 plugin registry: plugin not found
I even tried a commonly mentioned solution on SO:
docker network create --driver nat network-name
But issue still persists.
I understand that docker-compose is part of docker desktop install on windows.
How to solve it?
Content of the file:
version: "3"
services:
elasticsearch:
image: docker.elastic.co/elasticsearch/elasticsearch:6.7.0
ports: ["9200:9200"]
networks: ["sandbox"]
environment: ["discovery.type=single-node"]
kibana:
image: docker.elastic.co/kibana/kibana:6.7.0
ports: ["5601:5601"]
networks: ["sandbox"]
depends_on: ["elasticsearch"]
logstash:
image: docker.elastic.co/logstash/logstash:6.7.0
volumes:
- ./config/logstash.conf:/usr/share/logstash/pipeline/logstash.conf
networks: ["sandbox"]
ports: ["5000:5000/udp"]
depends_on: ["elasticsearch"]
grafana:
image: grafana/grafana:6.0.2
volumes: ["./grafana/plugins/cinnamon-elasticsearch-app:/var/lib/grafana/plugins/cinnamon-elasticsearch-app"]
ports: ["3000:3000"]
networks: ["sandbox"]
depends_on: ["elasticsearch"]
networks:
sandbox:
driver: bridge

Resources