docker-compose is not picking up file system changes in shared volume - docker

Question

Good day everyone.
My task here is to use some volume which will be shared among several services. This volume will be populated by ADD or COPY command in certain service Dockerfile. Problem I encountered is that volume is not getting updated when services are started via docker-compose up.
Consider following setup:
# docker-compose.yml
version: "3"
services:
service-a:
build:
context: service-a
dockerfile: Dockerfile
volumes:
- shared:/test/dir
service-b:
build:
context: service-b
dockerfile: Dockerfile
volumes:
- shared:/another-test/dir
volumes:
shared:
# ./service-a/Dockerfile
FROM alpine:3.8
COPY test-dir /test/dir
CMD [ "tail", "-f", "/dev/null" ]
# ./service-b/Dockerfile
FROM alpine:3.8
CMD [ "tail", "-f", "/dev/null" ]
And we will create ./service-a/test-dir folder. Now let's build it:
> docker-compose build
Building service-a
Step 1/3 : FROM alpine:3.8
---> 196d12cf6ab1
Step 2/3 : COPY test-dir /test/dir
---> ac66ed92b442
Step 3/3 : CMD [ "tail", "-f", "/dev/null" ]
---> Running in 932eb32b6184
Removing intermediate container 932eb32b6184
---> 7e0385d17f96
Successfully built 7e0385d17f96
Successfully tagged docker-compose-test_service-a:latest
Building service-b
Step 1/2 : FROM alpine:3.8
---> 196d12cf6ab1
Step 2/2 : CMD [ "tail", "-f", "/dev/null" ]
---> Running in 59a8b91c6b2d
Removing intermediate container 59a8b91c6b2d
---> 4e2c16ea5a80
Successfully built 4e2c16ea5a80
Successfully tagged docker-compose-test_service-b:latest
And start services:
> docker-compose up --no-build -d
Creating network "docker-compose-test_default" with the default driver
Creating volume "docker-compose-test_shared" with default driver
Creating docker-compose-test_service-a_1 ... done
Creating docker-compose-test_service-b_1 ... done
Let's check mapped directories:
> docker-compose exec service-a ls -lah /test/dir
total 8
drwxr-xr-x 2 root root 4.0K Dec 12 06:14 .
drwxr-xr-x 3 root root 4.0K Dec 12 06:14 ..
> docker-compose exec service-b ls -lah /another-test/dir
total 8
drwxr-xr-x 2 root root 4.0K Dec 12 06:14 .
drwxr-xr-x 3 root root 4.0K Dec 12 06:14 ..
Now let's put few text files in ./service-a/test-dir on host machine and build again:
> docker-compose build
Building service-a
Step 1/3 : FROM alpine:3.8
---> 196d12cf6ab1
Step 2/3 : COPY test-dir /test/dir
---> bd168b0fc8cc
Step 3/3 : CMD [ "tail", "-f", "/dev/null" ]
---> Running in 6e81b32243e1
Removing intermediate container 6e81b32243e1
---> cc28fc6de9ac
Successfully built cc28fc6de9ac
Successfully tagged docker-compose-test_service-a:latest
Building service-b
Step 1/2 : FROM alpine:3.8
---> 196d12cf6ab1
Step 2/2 : CMD [ "tail", "-f", "/dev/null" ]
---> Using cache
---> 4e2c16ea5a80
Successfully built 4e2c16ea5a80
Successfully tagged docker-compose-test_service-b:latest
As you can see cache is not used on COPY step in service-a, meaning changes are baked into the image. Now let's start services:
> docker-compose up --no-build -d
Recreating docker-compose-test_service-a_1 ...
Recreating docker-compose-test_service-a_1 ... done
Once again service-b remains untouched, only service-a gets recreated. Let's check actual services (this is where problem happens):
> docker-compose exec service-a ls -lah /test/dir
total 8
drwxr-xr-x 2 root root 4.0K Dec 12 06:17 .
drwxr-xr-x 3 root root 4.0K Dec 12 06:20 ..
> docker-compose exec service-b ls -lah /another-test/dir
total 8
drwxr-xr-x 2 root root 4.0K Dec 12 06:17 .
drwxr-xr-x 3 root root 4.0K Dec 12 06:14 ..
So files are not reflected... However if we launch temporary container based on service-a image it will show proper list:
> docker run --rm docker-compose-test_service-a ls -lah /test/dir
total 8
drwxr-xr-x 2 root root 4.0K Dec 12 06:20 .
drwxr-xr-x 3 root root 4.0K Dec 12 06:20 ..
-rwxr-xr-x 1 root root 0 Dec 12 06:20 rrrrrr.txt
-rwxr-xr-x 1 root root 0 Dec 12 06:16 test.txt
Any idea for workaround on this? So far it seems like only complete shutdown via docker-compose down with volume destruction helps. Not the best solution though as with real project it will cause serious downtime.
I hope configuration is readable, but I can put it into small git repo maybe if needed.
Thanks in advance.

Answer 1

Docker will only auto-populate a volume when it is first created. In the workflow you describe, when you delete and recreate the "first" container with its new image, you need to delete and recreate the volume too, which probably means deleting and recreating the whole stack.
There are a couple of ways around this issue:
Rearchitect your application to not need to share files. (Obviously the most work.) If there's some sort of semi-static content, you might need to "bake them in" to the consuming image, or set up the two parts of your system to communicate via HTTP. (This sort of data sharing across containers is not something Docker is great at, and it gets worse when you start looking at multi-host solutions like Swarm or Kubernetes.)
If there are only two parts involved, build a single image that runs the two processes. I've seen other SO questions that do this for a PHP-FPM component that serves dynamic content plus an nginx server that serves static content and forwards some requests to PHP-FPM, but where all of the static and dynamic content is "part of the application" and the HTTP-via-nginx entry point is "the single entrypoint into the container". supervisord is the de facto default control process when this is necessary.
Inside the "producing" container, write some startup-time code that copies data from something in the image into a shared-data location
#!/bin/sh
# Run me as the image's ENTRYPOINT.
if [ -d /data ]; then
cp -r /app/static /data
done
exec "$#"
This will repopulate the data volume on every startup.