If I do something like
docker run -v /opt/datadir:/var/lib/mysql image
I am mapping some location inside the container to a location in the host.
How is this different to the command COPY used when writing a Dockerfile?
The major difference is seen in case we edit any of the file present inside that location.
Suppose the directory /opt/datadir contains a file temp.txt
In case of bind mount, if you try to edit the file temp.txt from the host machine, the changes will be reflected inside the container and vice-versa.
When we create COPY command in Dockerfile, it copies the content to the filesystem of the container. Hence any changes done inside the container are DOES NOT affect the files present on the host machine.
In this case, if you want changes done on the host machine to be reflected inside the container, then you need to build a docker image and run a new container using the updated image.
When to use what?
For scenarios where the resource needs frequent updates, use bind mounts.
Eg: We want to provide our web server a configuration file that might change frequently.
In case the resource is independent of host filesystem, use COPY command inside dockerfile.
Eg: .tar, .zip, .war files or any file that requires no or very few updates inside the container.
Related
I am finding many answers on how to develop inside a container in Visual Studio Code with the Remote Containers extension, but surprisingly none address my use case.
I can add to the repo only from the host, but can run the code only from the container. If I edit a file in the container, I have to manually copy it to the host so I can commit it, but if I edit the file on the host, I have to manually copy it into the container so I can test it.
I would like to set up the IDE to automatically copy files from the host into the container whenever I save or change a file in a particular directory. This way I can both commit files on the host, and run them in the container, without having to manually run docker cp each time I change a file. Files should not automatically be copied from the container to the host, since the container will contain built files which should not be added to the repo.
It seems highly unlikely that this is impossible; but how?
This can be configured using the Run on Save extension.
Set it to run docker cp on save.
I have a container that runs a Python script in order to download a few big files from Amazon S3. The purpose of this container is just to download the files so I have them on my host machine. Because these files are needed by my app (which is running in a separate container with a different image), I bind mount from my host to the app's container the directory downloaded from the first container.
Note 1: I don't want to run the script directly from my host as it has various dependencies that I don't want to install on my host machine.
Note 2: I don't want to download the files while the app's image is being built as it takes too much time to rebuild the image when needed. I want to pass these files from outside and update them when needed.
Is there a way to make the first container to download those files directly to my host machine without downloading them first in the container and then copying them to the host as they take 2x the space needed before cleaning up the container?
Currently, the process is the following:
Build the temporary container image and run it in order to download
the models
Copy the files from the container to the host
Cleanup unneeded container and image
Note: If there is a way to download the files from the first container directly to the second and override them if they exist, it may work too.
Thanks!
You would use a host volume for this. E.g.
docker run -v "$(pwd)/download:/data" your_image
Would run your_image and anything written to /data inside the container would actually write to the host in the ./download directory.
I am trying to setup a windows nanoserver container as a sidecar container holding the certs that I use for SSL. Because the SSL cert that I need changes in each environment, I need to be able to change the sidecar container (i.e. dev-cert container, prod-cert container, etc) at startup time. I have worked out the configuration problems, but am having trouble using the same pattern that I use for Linux containers.
On linux containers, I simply copy my files into a container and use the VOLUMES step to export my volume. Then, on my main application container, I can use volumes_from to import the volume from the sidecar.
I have tried to follow that same pattern with nanoserver and cannot get working. Here is my dockerfile:
# Building stage
FROM microsoft/nanoserver
RUN mkdir c:\\certs
COPY . .
VOLUME c:/certs
The container builds just fine, but I get the following error when I try and run it. The dockerfile documentation says the following:
Volumes on Windows-based containers: When using Windows-based
containers, the destination of a volume inside the container must be
one of:
a non-existing or empty directory
a drive other than C:
so I thought, easy, I will just switch to the D drive (because I don't want to export an empty directory like #1 requires). I made the following changes:
# Building stage
FROM microsoft/windowservercore as build
VOLUME ["d:"]
WORKDIR c:/certs
COPY . .
RUN copy c:\certs d:
and this container actually started properly. However, I missed in the docs where is says:
Changing the volume from within the Dockerfile: If any build steps
change the data within the volume after it has been declared, those
changes will be discarded.
so, when I checked, I didn't have any files in the d:\certs directory.
So how can you mount a drive for external use in a windows container if, #1 the directory must be empty to make a VOLUME on the c drive in the container, and use must use VOLUME to create a d drive, which is pointless because anything put in there will not be in the final container?
Unfortunately you cannot use Windows containers volumes in this way. Also this limitation is the reason why using database containers (like microsoft/mssql-server-windows-developer) is a real pain. You cannot create volume on non-empty database folder and as a result you cannot restore databases after container re-creation.
As for your use case, I would suggest you to utilize a reverse proxy (like Nginx for example).
You create another container with Nginx server and certificates inside. Then you let it handle all incoming HTTPS requests, terminate SSL/TLS and then pass request to inner application container using plain HTTP protocol.
With such deployment you don't have to copy and install HTTPS certificates to all application containers. There is only one place where you store certificates and you can change dev/test/etc certificates just by using different Nginx image versions (or by binding certificate folder using volume).
UPDATE:
Also if you still want to use sidecar container you can try one small hack. So basically you will move this operation
COPY . .
from build time to runtime (after container starts).
Something like this:
FROM microsoft/nanoserver
RUN mkdir c:\\certs_in
RUN mkdir c:\\certs_out
VOLUME c:/certs_out
CMD copy "C:\certs_in" *.* "D:\certs_out"
I want to achieve the following with Docker: I want to give a container access to a host directory, such that the container can make changes, but the changes are discarded once the container is exiting/removed (pretty much like an overlayfs).
Simply mounting the directory as a volume for the docker container seems like the wrong way to me, since changes made to a volume persist and I don't want that.
How do I tackle this problem?
The only way for a container to modify the host is to mount a directory between the host and the container. But the changes made by host or container will persist.
You could try the other way: COPY the files you want from host to container using a Dockerfile. The files will be only on the container. When you remove and launch another one, the new container will start with the original files.
This experiment tries to build a container using this Docker file:
FROM lambdalinux/baseimage-amzn:2016.09-000
COPY ./bundle /opt/bundle/
VOLUME /bundle
Then inside the container, create a /opt/bundle/file.txt and put some text in it. But that file did not show up in the bundle directory on the host as I expected after reading Should I include my code with COPY/ADD or a volume last paragraph:
There may be cases where you’ll want to use both. You can have the image include the code using a COPY, and use a volume in your Compose file to include the code from the host during development. The volume overrides the directory contents of the image.
Doesn't Dockerfile VOLUME do the same as docker-compose.yml VOLUME? If so, how can this be done so that changes in the host directory is reflected inside the container directory in this case?
I also created a file on the host bundle/play.txt but that did not show up inside the container /opt/bundle/...
A VOLUME instruction in the dockerfile creates a mount point but initially only maps it to Docker's internal data directory.
In order to map the volume to the host filesystem, you need to specify which path on the host should be mapped to the volume. You can do this in the docker-compose file using the volumes parameter. (Note: you can create volumes using docker-compose without declaring them in the Dockerfile.)
Note that when mapping a directory from the host to a container, the directory contents on the host will replace the contents in the container, not vice versa.