I would like to run two docker containers via docker-compose.
The following project structure is present:
|- service
|- docker-compose
|- DockerFile
|- config.yaml
|- client
|- docker-compose
|- DockerFile
I try to run two containers via the following command:
$ docker-compose -f ./client/docker-compose.yml -f ./service/docker-compose.yml up
Everything seems to be working just fine, except for one error message:
{"action":"startup","error":"config file 'config.yaml' not found.","level":"error","msg":"could not load config","time":"2020-03-17T15:17:19Z"}
But when I navigate into each directory and run them separately everything works just fine. So it seems that the config file which is stated in the volume is somehow not found.
The docker-compose file is:
./service/docker-compose.yml
services:
thing:
command:
- --host
- 0.0.0.0
- --port
- '8080'
- --scheme
- http
- --config-file
- config.yaml
ports:
- 8080:8080
volumes:
- ./config.yaml:/config.yaml
web:
build: .
environment:
- web_host=http://my-site.com:8080
depends_on:
- anotherThing
links: thing:thing.com
version: '3.4'
The configuration file itself has some arbitrary info:
---
authentication:
my_arbitrary_key:
enabled: true
Any idea how to make sure the config file is found when running docker-compose from a sub-directory? Or am I misusing the docker-compose command...
Update
Interestingly enough if I swap the files in the docker-compose command I get it doesn't run.
So when I use:
$ docker-compose -f ./service/docker-compose.yml -f ./client/docker-compose.yml up
Docker doesn't run and the error is:
Cannot create container for service thing: invalid volume specification: '/Users/user/Site/service/config.yaml:config.yaml:rw': invalid mount config for type "bind": invalid mount path: 'config.yaml' mount path must be absolute
It's looking for config file in outer directory, you could use absolute path or put it in some .env
Best practice would be to use single docker-compose file which could have relative path of other folder
Brief background : Docker-compose is utility to run multiple docker containers - called as services, - Compose is a tool for defining and running multi-container Docker applications. With Compose, you use a YAML file to configure your application’s services. Then, with a single command, you create and start all the services from your configuration. To learn more about all the features of Compose, see the list of features.
Hence it's not necessary to have two docker-compose files over here , we should have one which will start both services also you can user docker-compose to run one service if required
Find more details at - https://docs.docker.com/compose/
Related
I wanted to translate this docker CLI command (from smallstep/step-ca) into a docker-compose.yml file to run with docker compose (version 2):
docker run -d -v step:/home/step \
-p 9000:9000 \
-e "DOCKER_STEPCA_INIT_NAME=Smallstep" \
-e "DOCKER_STEPCA_INIT_DNS_NAMES=localhost,$(hostname -f)" \
smallstep/step-ca
This command successfully starts the container.
Here is the compose file I "composed":
version: "3.9"
services:
ca:
image: smallstep/step-ca
volumes:
- "step:/home/step"
environment:
- DOCKER_STEPCA_INIT_NAME=Smallstep
- DOCKER_STEPCA_INIT_DNS_NAMES=localhost,ubuntu
ports:
- "9000:9000"
When I run docker compose up (again, using v2 here), I get this error:
service "ca" refers to undefined volume step: invalid compose project
Is this the right way to go about this? I'm thinking I missed an extra step with volume creation in docker compose projects, but I am not sure what that would be, or if this is even a valid use case.
The Compose file also has a top-level volumes: block and you need to declare volumes there.
version: '3.9'
services:
ca:
volumes:
- "step:/home/step"
et: cetera
volumes: # add this section
step: # does not need anything underneath this
There are additional options possible, but you do not usually need to specify these unless you need to reuse a preexisting Docker named volume or you need non-standard Linux mount options (the linked documentation gives an example of an NFS-mount volume, for example).
Citing the Compose specification:
To avoid ambiguities with named volumes, relative paths SHOULD always begin with . or ...
So it should be enough to make your VOLUME's host path relative:
services:
ca:
volumes:
- ./step:/home/step
If you don't intend to share the step volume with other containers, you don't need to define it in the top-level volumes key:
If the mount is a host path and only used by a single service, it MAY be declared as part of the service definition instead of the top-level volumes key.
it seems that docker-compose don't know the "volume" you created via command: sudo docker volume create my_xx_volume
so ,just manually mkdir to create a folder and chmod 777 <my_folder>, then your mysql docker will use it very well.
( in production env, don't use chmod but chown to change the file permission )
I am very (read very) new to Docker so experimenting. I have created a very basic Dockerfile to pull in Laravel:
FROM composer:latest
RUN composer_version="$(composer --version)" && echo $composer_version
RUN composer global require laravel/installer
WORKDIR /var/www
RUN composer create-project --prefer-dist laravel/laravel site
My docker-compose.yml file looks like:
version: '3.7'
services:
laravel:
build:
context: .
dockerfile: laravel.dockerfile
container_name: my_laravel
network_mode: host
restart: on-failure
volumes:
- ./site:/var/www/site
When I run docker-compose up, the ./site directory is created but the contents are empty. I've put this in docker-compose as I plan on on including other things like nginx, mysql, php etc
The command:
docker run -v "/where/i/want/data/site:/var/www/site" my_laravel
Results in the same behaviour.
I know the install is successful as I modified my dockerfile with the follwing two lines appended to it:
WORKDIR /var/www/site
RUN ls -la
Which gives me the correct listing.
Clearly misunderstanding something here. Any help appreciated.
EDIT: So, I was able to get this to work... although, it slightly more difficult than just specifying a path..
You can accomplish this by specifying a volume in docker-compose.yml.. The path to the directory (on the host) is labeled as device in the compose file.. It appears that the root of the path has to be an actual volume (possibly a share would work) but the 'destination' of the path can be a directory on the specified volume..
I created a new volume called docker on my machine but I suppose you could do this with your existing disk/volume..
I am on a Mac and this docker-compose.yml file worked for me:
version: '3.7'
services:
nodemon-test:
container_name: my-nodemon-test
image: oze4/nodemon-docker-test
ports:
- "1337:1337"
volumes:
- docker_test_app:/app # see comment below on which name to use here
volumes:
docker_test_app: # use this name under `volumes:` for the service
name: docker_test_app
driver: local
driver_opts:
o: bind
type: none
device: /Volumes/docker/docker_test_app
The container specified exists in my DockerHub.. this is the source code for it, just in case you are worried about anything malicious. I created it like two weeks ago to help someone else on StackOverflow.
Shows files from the container on my machine (the host)..
You can read more about Docker Volume configs here if you would like.
ORIGINAL ANSWER:
It looks like you are trying to share the build directory with your host machine.. After some testing, it appears Docker will overwrite the specified path on the container with the contents of the path on the host.
If you run docker logs my_laravel you should see an error about missing files at /var/www/site.. So, even though the build is successful - once Docker mounts the directory from your machine (./site) onto the container (/var/www/site) it overwrites the path within the container (/var/www/site) with the contents of the path on your host (./site) - which is empty.
To test and make sure the contents of /var/www/site are in fact being overwritten, you can run docker exec -it /bin/bash (you may need to replace /bin/bash with /bash).. This will give you command line access inside of the container. From there you can do ls -a /var/www/site..
Furthermore, you can also pre-stage ./site to have a random test file in it (test.txt or whatever), then docker-compose up -d, then run the same commands from the step above docker exec -it ... and see if the staged test.txt file is now inside the container - this gives you definitive evidence that when you run volumes, the data on your host overwrites data in the container.
With that being said, doing something like this and sharing a log directory will work... the volume path specified on the container is still overwritten, the difference is the container is writing to that path.. it doesn't rely on it for config files/app files.
Hope this helps.
I am following lynda Docker tutorials and performing stuff related to docker compose file.
This is my docker-compose.yml file.
more docker-compose.yml
version: '3'
services:
web:
image: jboss/wildfly
volumes:
- ~/deployments:/opt/jboss/wildfly/standalone/deployments
ports:
- 8080:8080
As per authors, I am trying to copy webapp.war file to deployments/ folder giving me error. It look like volume mapping for the docker file is not working.
cp /home/user/Demos/docker-for-java/chapter2/webapp.war deployments/
cp: cannot create regular file ‘deployments/’: Not a directory
docker-compose ps
Name Command State Ports
--------------------------------------------------------------------------------
helloweb_web_1 /opt/jboss/wildfly/bin/sta ... Up 0.0.0.0:8080->8080/tcp
I think you might be misinterpreting tutorial. I haven't seen the tutorial itself, but checking the documentation for the WildFly Docker image here there's a mention that you need to extend base image and add your war file inside:
To do this you just need to extend the jboss/wildfly image by creating a new one. Place your application inside the deployments/ directory with the ADD command (but make sure to include the trailing slash on the deployment folder path, more info). You can also do the changes to the configuration (if any) as additional steps (RUN command).
This means that you need to create a Dockerfile with approximately this contents (change your-awesome-app.war with the path to your war file):
FROM jboss/wildfly
ADD your-awesome-app.war /opt/jboss/wildfly/standalone/deployments/
After that you need to change you docker-compose.yml to build from your Dockerfile instead of using jboss/wildfly (note the use of build: . instead of image: jboss/wildfly):
version: '3'
services:
web:
build: .
ports:
- 8080:8080
Try that and comment if you run into any issues
I am new to Docker and have docker-compose.yml which is containing many services and iI need to start one particular service. I have docker-compose.yml file with information:
version: '2'
services:
postgres:
image: ${ARTIFACTORY_URL}/datahub/postgres:${BUILD_NUMBER}
restart: "no"
volumes:
- /etc/passwd:/etc/passwd
volumes_from:
- libs
depends_on:
- libs
setup:
image: ${ARTIFACTORY_URL}/setup:${B_N}
restart: "no"
volumes:
- ${HOME}:/usr/local/
I am able to call docker-compose.yml file using command:
docker-compose -f docker-compose.yml up -d --no-build
But I need to start "setup service" in docker-compose file:
How can I do this?
It's very easy:
docker compose up <service-name>
In your case:
docker compose -f docker-compose.yml up setup -d
To stop the service, then you don't need to specify the service name:
docker compose down
will do.
Little side note: if you are in the directory where the docker-compose.yml file is located, then docker-compose will use it implicitly, there's no need to add it as a parameter.
You need to provide it in the following situations:
the file is not in your current directory
the file name is different from the default one, eg. myconfig.yml
As far as I understand your question, you have multiple services in docker-compose but want to deploy only one.
docker-compose should be used for multi-container Docker applications. From official docs :
Compose is a tool for defining and running multi-container Docker
applications.
IMHO, you should run your service image separately with docker run command.
PS: If you are asking about recreating only the container whose image is changed among the multiple services in your docker-compose file, then docker-compose handles that for you.
My problem is that I have a docker-compose.yml file and an haproxy.cfg file and I want docker-compose to copy the haproxy.cfg file to the docker container. As per the post Docker composer copy files I can use volumes to do it but in my case I'm getting the below error. Can anybody help me achieve this.
Below is the code and everything
docker-compose.yml
version: "3.3"
services:
###After all services are up, we are initializing the gateway
gateway:
container_name: gateway-haproxy
image: haproxy
volumes:
- .:/usr/local/etc/haproxy
ports:
- 80:80
network_mode: "host"
Folder Structure
Command output
root#ubuntu:/home/karunesh/Desktop/Stuff/SelfStudy/DevOps/docker# docker-compose up
Creating gateway-haproxy ...
Creating gateway-haproxy ... done
Attaching to gateway-haproxy
gateway-haproxy | <7>haproxy-systemd-wrapper: executing /usr/local/sbin/haproxy -p /run/haproxy.pid -f /usr/local/etc/haproxy/haproxy.cfg -Ds
gateway-haproxy | [ALERT] 219/163305 (6) : [/usr/local/sbin/haproxy.main()] No enabled listener found (check for 'bind' directives) ! Exiting.
gateway-haproxy | <5>haproxy-systemd-wrapper: exit, haproxy RC=1
gateway-haproxy exited with code 1
Try this:
volumes:
- ./haproxy.cfg:/usr/local/etc/haproxy/haproxy.cfg:ro
Instead of mounting the whole directory, this will only mount haproxy.cfg. The ro is an abbreviation for read-only, and its usage guarantees the container won't modify it after it gets mounted.
In order to add additional files to the container, you have to build on top of the existing image from haproxy.
For example, your Dockerfile should look like this:
FROM haproxy:latest
COPY haproxy.cfg /usr/local/etc/haproxy/haproxy.cfg
Then you can update your docker compose file accordingly.
If you plan on using this for local development, just mount the file(s), see #MatTheWhale's answer
See more at the official haproxy Docker page