i am trying to create a volume of a directory in a docker container (confluence).
https://hub.docker.com/r/atlassian/confluence-server/
To fix a bug with postgres, i have to add driver files manually in the container. The location inside the container is:
/opt/atlassian/confluence/confluence/WEB-INF/lib
After creating the volume i wanted to add the newer driver into the directory. So, inside my docker-compose.yaml i mapped a volume to the directory.
- ./data/driverfiles:/opt/atlassian/confluence/confluence/WEB-INF/lib
Volume and directory get created after calling docker-compose up and everything seems fine.
Problem is, that the volume remains empty and when starting an interactive shell into the container, the once filles with thousands of files directory is empty, too. When removing the volume from the docker-compose.yaml, the directory is full of files again.
Objectively, it looks like mapping the volume to this directory somehow prohibits the container of enriching it with files, what is going on here?
If you're mounting a host directory over a container directory, at container startup time, this is always a one-way operation: whatever is in the host directory (if anything) completely replaces whatever might have been in the image. Content from a container will never get copied into a host directory unless the image startup code explicitly does this for you.
If you need to modify a configuration file in the container, you need to first copy it out of the image; for example
# with the volumes: mount deleted
docker-compose run confluence \
sh -c 'cd /opt/atlassian/confluence/confluence/WEB-INF/lib && tar cf - .' \
| tar xf - .
That particular invocation will copy the entire directory out to the host, where you can mount it in again.
Note that if there's an updated image that changes the contents of this lib directory, the content you have on the host will always take precedence; this hides any changes that might be made in the image.
You might find it more reliable to build a custom image that adds the driver files you need
FROM ...conflunce:...
COPY ... /opt/atlassian/confluence/confluence/WEB-INF/lib
# Use CMD and all other options from the original image
Specify build: . (and no image:) in the docker-compose.yml file to use this Dockerfile.
Related
I made a docker image called myImage, there is a folder: /data I want to let the user edit it by themselves. I read that -v flag can mount the volume, so I used it like following:
I run the container with this command:
docker run -v /my_local_path:/data -it myImage /bin/bash
But surprisingly, docker cleared all the files in /data in the container. But this is not I want... I want actually the host can get all the files from /data... :(
How can I do that?
When you share a volume like this, the volume on the host overwrites the volume in the container, so the files in the container's folder will be removed.
What you need to do is put the files in the container in folder A (a folder in the container). Mount folder B (another folder in the container). Then AFTER the volume is mounted, move the files from folder A to folder B. Then these files will be both available to the host and inside the container.
You can achieve this 'move files' operation using a RUN or an ENTRYPOINT script in your Dockerfile.
See Run a script in Dockerfile
Sorry, I forget if you need RUN or ENTRYPOINT (or if either will work) but one of these will definitely do it.
I think you want ENTRYPOINT because an ENTRYPOINT script runs AFTER the container is created. Thus it will run after the volume is mounted.
I have a Dockerfile which copies some files into the container and after that creates a VOLUME.
...
ADD src/ /var/www/html/
VOLUME /var/www/html/files
...
In the src folder is an files folder and in this files folder are some files I need to have copied to the VOLUME the first time the container gets started.
I thought the first time the container gets created it uses the content of the original dir specified in the volume but this is not the case.
So how can I get the files into this folder?
Do I need to create an extra folder and copy it with a runscript (I hope not)?
Whatever you put in your Dockerfile is just evaluated at build time (and not when you are creating a new container).
If you want to make file from the host available in your container use a data volume:
docker run -v /host_dir:/container_dir ...
In case you just want to copy files from the host to a container as a one-off operation you can use:
docker cp /host_dir mycontainer:/container_dir
The issue is with your ADD statement. Also you might not understand how volumes are accessed. Compare your efforts with the demo below:
FROM alpine #, or your favorite tiny image
ADD src/files /var/www/html/files
VOLUME /var/www/html/files
Build an image called 'dataimg':
docker build -t dataimg .
Use the dataimg image to create a data container named 'datacon':
docker run --name datacon dataimg /bin/cat
Mount the volume from datacon in your nginx container:
docker run --volumes-from datacon nginx ls -la /var/www/html/files
And you'll see the listing of /var/www/html/files reflects the contents of src/files
I've searched the docs but nothing came up so time to test it. But for a quick future reference...
Is the host folder populated with the container folder contents?
Is it the opposite?
Are both folder contents merged? (In that case: What happens when a file with the same name is in both folders?)
Or does it produce an error? Is the error thrown on launch or is it thrown when you try to build an image with a VOLUME pointing to an existing populated folder on the container?
Also, another thing that isn't in the docs: Do I have to define the container path as a VOLUME in the Dockerfile in order to use -v against it when launching the container or can I create volumes on the fly?
When you run a container and mount a volume from the host, all you see in the container is what is on the host - the volume mount points at the host directory, so if there was anything in the directory in the image it gets bypassed.
With an image from this Dockerfile:
FROM ubuntu
WORKDIR /vol
RUN touch /vol/from-container
VOLUME /vol
When you run it without a host mount, the image contents get copied into the volume:
> docker run vol-test ls /vol
from-container
But mount the volume from the host and you only see the host's content:
> ls $(pwd)/host
from-host
> docker run -v $(pwd)/host:/vol vol-test ls /vol
from-host
And no, you don't need the VOLUME instruction. The behaviour is the same without it.
Whenever a Docker container is created with a volume mounted on the host, e.g.:
docker run -v /path/on/host:/data container-image
Any contents that are already in /data due to the image build process are always completely discarded, and whatever is currently at /path/on/host is used in its place. (If /path/on/host does not exist, it is created as an empty directory, though I think some aspect of that behavior may currently be deprecated.)
Pre-defining a volume in the Dockerfile with VOLUME is not necessary; all VOLUME does is cause any containers run from the image to have an implicit -v /volume/path (Note lack of host mount path) argument added to their docker run command which is ignored if an explicit -v /host/path:/volume/path is used.
I have a docker-compose dev stack. When I run, docker-compose up --build, the container will be built and it will execute
Dockerfile:
RUN composer install --quiet
That command will write a bunch of files inside the ./vendor/ directory, which is then only available inside the container, as expected. The also existing vendor/ on the host is not touched and, hence, out of date.
Since I use that container for development and want my changes to be available, I mount the current directory inside the container as a volume:
docker-compose.yml:
my-app:
volumes:
- ./:/var/www/myapp/
This loads an outdated vendor directory into my container; forcing me to rerun composer install either on the host or inside the container in order to have the up to date version.
I wonder how I could manage my docker-compose stack differently, so that the changes during the docker build on the current folder are also persisted on the host directory and I don't have to run the command twice.
I do want to keep the vendor folder mounted, as some vendors are my own and I like being able to modifiy them in my current project. So only mounting the folders I need to run my application would not be the best solution.
I am looking for a way to tell docker-compose: Write all the stuff inside the container back to the host before adding the volume.
You can run a short side container after docker-compose build:
docker run --rm -v /vendor:/target my-app cp -a vendor/. /target/.
The cp could also be something more efficient like an rsync. Then after that container exits, you do your docker-compose up which mounts /vendor from the host.
Write all the stuff inside the container back to the host before adding the volume.
There isn't any way to do this directly, but there are a few options to do it as a second command.
as already suggested you can run a container and copy or rsync the files
use docker cp to copy the files out of a container (without using a volume)
use a tool like dobi (disclaimer: dobi is my own project) to automate these tasks. You can use one image to update vendor, and another image to run the application. That way updates are done on the host, but can be built into the final image. dobi takes care of skipping unnecessary operations when the artifact is still fresh (based on modified time of files or resources), so you never run unnecessary operations.
So currently I can do: docker -v .:/usr/src/app or even specify it in my docker-compose.yml:
web:
volumes:
- .:/usr/src/app
But when I attempt to define this in my Dockerfile:
VOLUME .:/usr/src/app
It doesn't mount anything.
Now I understand the complexities in that I'm using OSX and so I have to virtualize the environment to run Docker via boot2docker, and that boot2docker solves the copy issue by mounting /User to the linux machine running Docker.
The documentation wants me to be explicit, but since my explicitness would require me to name my user (in this case /User/krainboltgreene/code/krainboltgreene/blankrails) it seems non-idiomatic, as that obviously doesn't work on other people's environments.
What's the solution for this? I mean, I can technically get this all working without (as noted above the CLI and compose works fine), but it means not being able to do project specific provisioning (bower install, npm install, vulcanize, etc).
You can't specify a host directory for a volume inside a Dockerfile, because of the portability reasons you mention (not everyone will have the same directories and there are security issues regarding mounting sensitive files).
If you instead do:
VOLUME /usr/src/app
Docker will automatically set up a volume at run-time for the folder, which will be mapped to a directory under /var/lib/docker/volumes.
If you want to be able to quickly make changes during development, I would suggest using COPY in the Dockerfile, but mounting local changes over the top with a volume at run-time. This has the disadvantage that if you volume mount a folder, all the contents of that directory in the container will be hidden (rather than merged).
The docker run -v .:/usr/src/app ... command as well as the docker-compose definitions are executing during runtime. Whereas the Dockerfile instructions are executed during build time.
By the way the instruction in your Dockerfile is syntactically incorrect. It should be VOLUME /usr/src/app instead.
That VOLUME keyword only defines that later during runtime this location will be stored on a volume. So all files that you add by further Dockerfile instructions or manual commits to that location are ignored and not added to the resulting image.
Now during runtime when you did not specify a volume it Docker will generate a volume for you which is empty by default.
To have your docker-compose setup working for other colleagues you could simply make the docker-compose configuration file being part of your blankrails project folder. Everybody then runs docker-compose from within that directory and your provided configuration will work.
EDIT:
I do not know exactly what you mean with project specific provisioning. But if your aim is to provide default contents for the defined volume you could do something like the following:
Add all required project files during the Dockerfile build to a /bootstrap folder on the image.
Instead of executing your app directly use a start shell script for CMD.
In that start script you can check whether the volume mounted to /usr/src/app is empty or not. When it is empty copy all the /bootstrap contents into it.
Afterwards start your app from within that script in foreground.
With that approach you can easily provide a default file set for mounted volumes. And when you re-use that volume e.g. after a container restart the container just works with the files that are on the volume without touching them again during startup. So modified files will be persisted.