I am having problems with writing files out from inside a docker container to my host computer. I believe this is a privilege issue and prefer not to set privileged: True. A work around for writing out files is by pre-pending ../ to a volume in my docker-compose.yml file. For example,
version: '3'
services:
example:
volumes:
- ../:/example
What exactly is ../ doing here? Is it taking from the container's privileges and "going up" a directory to the host machine? Without ../, I am unable to write out files to my host machine.
Specifying a path as the source, as opposed to a volume name, bind mounts a host path to a path inside the container. In your example, ../ will be visible inside the container at /example on a recent version of docker.
Older versions of docker can only access the directory it is in and lower, not higher, unless you specify the higher directory as the context.
To run the docker build from the parent directory:
docker build -f /home/me myapp/Dockerfile
As opposed to
docker build -f /home/me/myapp Dockerfile
Doing the same in composer:
#docker-compose.yml
version: '3.3'
services:
yourservice:
build:
context: /home/me
dockerfile: myapp/Dockerfile
Or with your example:
version: '3'
services:
build:
context: /home/me/app
dockerfile: docker/Dockerfile
example:
volumes:
- /home/me/app:/example
Additionally you have to supply full paths, not relative paths. Ie.
- /home/me/myapp/files/example:/example
If you have a script that is generating the Dockerfile from an unknown path, you can use:
CWD=`pwd`; echo $CWD
To refer to the current working directory. From there you can append /..
Alternately you can build the image from a directory one up, or use a volume which you can share with an image that is run from a higher directory, or you need to output your file to stdout and redirect the output of the command to the file you need from the script that runs it.
See also: Docker: adding a file from a parent directory
The statement volumes: ['../:/example'] makes the parent directory of the directory containing docker-compose.yml on the host (../) visible inside the container at /example. Host directory bind-mounts like this, plus some equivalent constructs using a named volume attached to a specific host directory, are the only way a container can write out to the host filesystem.
Related
How do you specify a mount volume in docker-compose, so your Dockerfile can access files from it?
I have a docker-compose.yml like:
version: "3.6"
services:
app_test:
build:
context: ..
dockerfile: Dockerfile
volumes:
- /tmp/cache:/tmp/cache
And in my Dockerfile, I want to access files from /tmp/cache via RUN like:
RUN cat /tmp/cache/somebinary.tar.gz | processor.sh
However, running docker-compose gives me the error:
/tmp/cache/somebinary.tar.gz does not exist
Even though on the host, ls /tmp/cache/somebinary.tar.gz confirms it does exist.
Why is docker-compose/Docker unable to mount or access my host directory?
Dockerfile RUN commands are executed at build time of the image.
The volume is mounted at run time once the image is run as a container. So the mounted files will not be available until you spawn a container based on your image.
To define the commands to use at run time, use CMD, or depending on how you intend your image to be used ENTRYPOINT.
You would need to add this at the end of your Dockerfile:
CMD cat /tmp/cache/somebinary.tar.gz | processor.sh
I have a few repos that I want to deploy using docker-compose within the same project, for each I have a docker-compose.yaml in the root of the directory that defines the list of services. The docker files are located in docker/{service}/Dockerfile relative to the repo root, and hence the docker-compose.yaml looks something like
version: "3"
services:
service1:
build:
context: .
dockerfile: ./docker/service1/Dockerfile
networks:
- default
service2:
build:
context: .
dockerfile: ./docker/service2/Dockerfile
networks:
- default
service3:
build:
context: .
dockerfile: ./docker/service3/Dockerfile
networks:
- default
networks:
default:
However when I run the command from the directory above the repo roots
docker-compose -p project -f repo1/docker-compose.yaml -f repo2/docker-compose.yaml build
I get the error ERROR: Cannot locate specified Dockerfile: ./docker/service1/Dockerfile
Which is a service from repo2
These commands run fine with just one file specified and only one set of services are "not found" (repo2's). I assume that the first file is therefore setting the context of compose, is there a way I can tell compose not to do this?
The documentation for the docker-compose -f option notes:
When you use multiple Compose files, all paths in the files are relative to the first configuration file specified with -f.
Practically, this probably means all of the Compose files need to be in the same directory. If you have a file that really has no host paths at all (neither a build: context directory nor volumes: bind mounts) it could in principle be somewhere else, but IME this is pretty unusual.
For the setup you describe, it might be more practical to launch the separate projects separately:
(cd repo1 && docker-compose up --build -d)
(cd repo2 && docker-compose up --build -d)
A more typical use of multiple Compose files is to provide options split across several files; for example, a base file that specifies image names and an override file that provides developer-oriented features like publishing database ports and building images from source. Share Compose configurations between files and projects describes this use case a little more. That page similarly comments:
Tracking which fragment of a service is relative to which path is difficult and confusing, so to keep paths easier to understand, all paths must be defined relative to the base file.
In short: it's an intentional feature of Compose that all path references in all Compose files are relative to the location of the first file, regardless of the location of the file that contains the path reference, and there's not an override option for it.
I am very (read very) new to Docker so experimenting. I have created a very basic Dockerfile to pull in Laravel:
FROM composer:latest
RUN composer_version="$(composer --version)" && echo $composer_version
RUN composer global require laravel/installer
WORKDIR /var/www
RUN composer create-project --prefer-dist laravel/laravel site
My docker-compose.yml file looks like:
version: '3.7'
services:
laravel:
build:
context: .
dockerfile: laravel.dockerfile
container_name: my_laravel
network_mode: host
restart: on-failure
volumes:
- ./site:/var/www/site
When I run docker-compose up, the ./site directory is created but the contents are empty. I've put this in docker-compose as I plan on on including other things like nginx, mysql, php etc
The command:
docker run -v "/where/i/want/data/site:/var/www/site" my_laravel
Results in the same behaviour.
I know the install is successful as I modified my dockerfile with the follwing two lines appended to it:
WORKDIR /var/www/site
RUN ls -la
Which gives me the correct listing.
Clearly misunderstanding something here. Any help appreciated.
EDIT: So, I was able to get this to work... although, it slightly more difficult than just specifying a path..
You can accomplish this by specifying a volume in docker-compose.yml.. The path to the directory (on the host) is labeled as device in the compose file.. It appears that the root of the path has to be an actual volume (possibly a share would work) but the 'destination' of the path can be a directory on the specified volume..
I created a new volume called docker on my machine but I suppose you could do this with your existing disk/volume..
I am on a Mac and this docker-compose.yml file worked for me:
version: '3.7'
services:
nodemon-test:
container_name: my-nodemon-test
image: oze4/nodemon-docker-test
ports:
- "1337:1337"
volumes:
- docker_test_app:/app # see comment below on which name to use here
volumes:
docker_test_app: # use this name under `volumes:` for the service
name: docker_test_app
driver: local
driver_opts:
o: bind
type: none
device: /Volumes/docker/docker_test_app
The container specified exists in my DockerHub.. this is the source code for it, just in case you are worried about anything malicious. I created it like two weeks ago to help someone else on StackOverflow.
Shows files from the container on my machine (the host)..
You can read more about Docker Volume configs here if you would like.
ORIGINAL ANSWER:
It looks like you are trying to share the build directory with your host machine.. After some testing, it appears Docker will overwrite the specified path on the container with the contents of the path on the host.
If you run docker logs my_laravel you should see an error about missing files at /var/www/site.. So, even though the build is successful - once Docker mounts the directory from your machine (./site) onto the container (/var/www/site) it overwrites the path within the container (/var/www/site) with the contents of the path on your host (./site) - which is empty.
To test and make sure the contents of /var/www/site are in fact being overwritten, you can run docker exec -it /bin/bash (you may need to replace /bin/bash with /bash).. This will give you command line access inside of the container. From there you can do ls -a /var/www/site..
Furthermore, you can also pre-stage ./site to have a random test file in it (test.txt or whatever), then docker-compose up -d, then run the same commands from the step above docker exec -it ... and see if the staged test.txt file is now inside the container - this gives you definitive evidence that when you run volumes, the data on your host overwrites data in the container.
With that being said, doing something like this and sharing a log directory will work... the volume path specified on the container is still overwritten, the difference is the container is writing to that path.. it doesn't rely on it for config files/app files.
Hope this helps.
I've nignx container and one asset container which have all my assets build from grunt or some other tools.
Now in docker compose file, i want to mount asset container's 's folder path into nginx container so nginx can serve that files.
How can we do that? i don't remember but i think there is a option where we can share path of one container with another.
Suppose if i scale up nginx to 2 container then will that mount works for all instance of nginx?
if i scale up asset container then what will happen?
i also want to mount that with my host so development can be done be easily.
What you want to do is use a volume, and then mount that volume into whatever containers you want it to appear in.
Completely within Docker
You can do this completely inside of Docker.
Here is an example (stripped-down - your real file would have much more than this in it, of course).
version: '3'
services:
nginx:
volumes:
- asset-volume:/var/lib/assets
asset:
volumes:
- asset-volume:/var/lib/assets
volumes:
asset-volume:
At the bottom is a single volume defined, named "asset-volume".
Then in each of your services, you tell Docker to mount that volume at a certain path. I show example paths inside the container, just adjust these to be whatever path you wish them to be in the container.
The volume is an independent entity not owned by any particular container. It is just mounted into each of them, and is shared. If one container modifies the contents, then they all see the changes.
Note that if you prefer only one can make changes, you can always mount the volume as read-only in some services, by adding :ro to the end of the volume string.
services:
servicename:
volumes:
- asset-volume:/var/lib/assets:ro
Using a host directory
Alternately you can use a directory on the host and mount that into the containers. This has the advantage of you being able to work directly on the files using your tools outside of Docker (such as your GUI text editor and other tools).
It's the same, except you don't define a volume in Docker, instead mounting the external directory.
version: '3'
services:
nginx:
volumes:
- ./assets:/var/lib/assets
asset:
volumes:
- ./assets:/var/lib/assets
In this example, the local directory "assets" is mounted into both containers using the relative path ./assets.
Using both depending on environment
You can also set it up for a different dev and production environment. Put everything in docker-compose.yml except the volume mounts. Then make two more files.
docker-compose.dev.yml
docker-compose.prod.yml
In these files put only the minimum config to define the volume mount. We'll mix this with the docker-compose.yml to get a final config.
Then use this. It will use the config from docker-compose.yml, and use anything in the second file as an override or supplemental config.
docker-compose -f docker-compose.yml \
-f docker-compose.dev.yml \
up -d
And for production, just use the prod file instead of the dev file.
The idea here is to keep most of the config in docker-compose.yml, and only the minimum set of differences in the alternative files.
Example:
docker-compose.prod.yml
version: '3'
services:
nginx:
volumes:
- asset-volume:/var/lib/assets
docker-compose.dev.yml
version: '3'
services:
nginx:
volumes:
- ./assets:/var/lib/assets
I'm using the Docker and docker-compose to run my app inside a container. Here is my docker-compose.yml:
version: '2'
services:
omsevents:
build:
context: ./omsevents
volumes:
- "../oms-events:/usr/app/oms-events"
- "../oms-events/assets:/usr/app/oms-events/assets"
The thing is, my app is a server and one of the things it does is file upload. When I upload a file it is created inside a container (I can bash inside the container and check and it's there, in /usr/app/oms-events/assets), bit it's not created on the host filesystem (the ../oms-events/assets/ folder is empty).
What can be the issue?
According to the documentation of Docker Compose:
You can mount a relative path on the host, which will expand relative to the directory of the Compose configuration file being used. Relative paths should always begin with . or ..
So you can try moving the docker-compose file into the context directory and remove the context clause from the build section, leaving it just build: . (or wherever your Dokerfile is). Then just fix the relative paths of the volumes or try with absolute ones.