I'm running a docker-compose setup, and when I want to update files in my image I create a new docker image. Though the problem is; the file I'm editing is located in the persistent volume, meaning the Docker image itself will get the changes, but since I'm not deleting docker-compose volumes the volume will be used by the new image, hence the old file will be used by new image.
Running docker-compose down -v is not an options because I want to keep other existing files in the volume (logs etc.).
I want to know if it possible to do this without too much hacks, since I'm looking to automate this.
Example docker-compose.yml
version: '3.3'
services:
myService:
image: myImage
container_name: myContainer
volumes:
- data_volume:/var/data
volumes:
data_volume
NOTE: The process of doing change in my case:
docker-compose down
docker build -t myImage:t1 .
docker compose up -d
You could start a container, mount the volume and execute a command to delete single files. Something like
docker run -d --rm -v data_volume:/var/data myImage rm /var/data/[file to delete]
Related
Here is a simplified version of my docker-compose.yml (it's the volume in buggy-service that does not behave as I expect):
version: '3.4'
services:
local-db:
image: postgres:9.6
environment:
- DB_NAME=${DB_NAME}
# other env vars (not important)
ports:
- 5432:5432
volumes:
- ~/.docker-volumes/${DB_NAME}/postgresql/data:/var/lib/postgresql/data
- postgresql:/docker-entrypoint-initdb.d
buggy-service:
build:
context: .
dockerfile: Dockerfile.test
target: buggy-image
args:
# bunch of args (not important)
volumes:
- /Users/me/temp:/temp
volumes:
postgresql:
driver_opts:
type: none
device: /Users/me/postgresql
o: bind
If I do docker-compose -f docker-compose.yml up -d local-db, a container for it starts up automatically and I find that /Users/me/postgresql on the host machine (Mac OSX) binds correctly to /docker-entrypoint-initdb.d with content synced.
However, if I do docker-compose -f docker-compose.yml up --build -d buggy-service, a container does not start up automatically.
Question: How do I get buggy-service to behave like local-db, i.e., start up automatically with the required volume mounted?
Here's the stripped down version of Dockerfile.test referenced by buggy-service:
FROM microsoft/dotnet:2.1-sdk-alpine AS buggy-image
# Bunch of ARG definitions (not important)
VOLUME /temp
# other stuff (not important)
ENTRYPOINT ["/bin/bash"]
# Other FROMs
Edit 1
A bit more info about what I’m trying to achieve...
The buggy-container I’m trying to get working runs .Net Core as the base image. Its purpose is to run dotnet test and generate coverage reports, which can then be consumed in the host, which may either be a local dev machine or a build server (in this case, BitBucket pipelines).
... followed by docker run -dit --name buggy-container buggy-image
This command creates a new container, not based on anything in the compose yml file. Without a volume specification, it will only get an anonymous volume since you've defined the volume in the Dockerfile (I tend to recommend against defining a volume there). You can see the anonymous volumes with a docker volume ls command, they'll be the ones with a long unique id and no reference to what they belong to.
To define a host volume from docker run, you need the -v flag:
docker run -dit -v /Users/me/temp:/temp --name buggy-container buggy-image
From your now changed question, you have a new issue. Your container specifies a single command to run in the entrypoint:
ENTRYPOINT ["/bin/bash"]
When bash runs, it reads input from stdin. When that input ends, like when you run a container with no input attached, bash will exit. When the process your container runs exits, the container exits. From the details available, I can't tell you what that command should be, but a good starting point is to look at other images on docker hub that perform a similar task that you're trying to run, and look at the Dockerfile they use (many hub images point back to a GitHub repo with the full source).
I'm using docker compose to build my application using docker.
Version of docker-compose is 2.2
I have all the containers running well at the moment where one of the container has nginx running.
I need to change some configuration on this container.
The way I need to do (because of special scenario) is, to update the config inside the container.
Then I commit the container to build a new image.
docker commit <container> <image-name>
Now I have new image with tag latest.
What I want is to use this image when I run, docker-compose down && docker-compose up --build next time.
docker-compose down && docker-compose up --build -d
With --build option, docker-compose will go through the steps in Dockerfile and run those and all my changes will be reverted.
Question:
Is there anyway that I can tell docker-compose to use the newly created image as cache and ignore Dockerfile for this one container?
Solution Tried:
I have tried with docker-compose-override and using option cache-from and it's not working.
docker-compose.override.yml
container:
build:
cache_from:
- new-image:latest
Thanks in advance.
I don't understand why you would want to build an image from docker-compose even though you have already built it by docker-commit.
Now I have new image with tag latest.
What I want is to use this image when I run, docker-compose down && docker-compose up
If you have already built image, skip the build phase in docker-compose. Just specify which image should be used like so:
container:
image: new-image:latest
container_name: "Foo bar"
.....(other options)
Image
Specify the image to start the container from. Can either be a
repository/tag or a partial image ID.
image: redis
image: ubuntu:14.04
image: tutum/influxdb
image: example-registry.com:4000/postgresql
image: a4bc65fd
If the image does
not exist, Compose attempts to pull it, unless you have also specified
build, in which case it builds it using the specified options and tags
it with the specified tag.
If you have any other images that you build from inside docker-compose run:
docker-compose build && docker-compose up
If not simple docker-compose up will suffice.
Short version:
I want to add files in a docker container in docker-compose or Dockerfile and I want to make it accessible from other containers that I made in docker-compose file. How can I do that?
Long version:
I have a Python app in a container that uses a .csv file to generate a POJO machine learning model.
I also have a Java app in a container that uses the POJO machine learning model and appends the .csv file. The java app has a fileWatcher() method implemented.
The containers are made from the docker-compose file that calls Dockerfiles for each one of them. So I want to add them this way and not with CMD docker commands.
You can add the same named volume to different containers:
docker volume create --name volume_data
docker run -t -i -v volume_data:/public debian:jessie /bin/bash
docker run -t -i -v volume_data:/public2 debian:jessie /bin/bash
or as docker-compose.yml
services:
assets:
image: any_asset_image
volumes:
- assets:"/public/assets"
proxy:
image: nginx
volumes:
- assets
volumes:
- assets
Up to recent version of Docker (v1.10), we were thought that we can use DOC: data-only containers. So I would create such DOC (based on e.g. busybox) and use --volumes-from to link it to my container. You can still read about this in Docker documentation.
With new version of docker, it is said that instead of DOC we should use named volumes. Here is an example of docker-compose.yml:
version: '2'
services:
elasticsearch:
image: elasticsearch:2.2.0
command: elasticsearch -Des.network.host=0.0.0.0
ports:
- "9201:9200"
volumes:
- "es-data:/usr/share/elasticsearch/data"
volumes:
es-data:
Here we created and use named volume es-data.
There is still not much documentation on this new feature. I am asking:
Can we replace DOC with named containers? How long volume is persisted? What if I remove the container that is using it?
How can we e.g. backup now? Previously, I could docker run --rm --volumes-from es-data ... and then tar it.
Can we replace DOC with named containers?
In many cases, yes, named containers will be a better option.
How long volume is persisted? What if I remove the container that is using it?
If you remove the container, the volume will still be there. The only way to remove the volume is to use docker-compose down -v or docker volume rm <volume name>.
How can we e.g. backup now? Previously, I could docker run --rm --volumes-from es-data ... and then tar it.
Instead of --volumes-from, you can use --volume=<volume name>.
Note that volumes created by docker-compose are always prefixed with the project name, so if you use it with a docker command the full name is actually <project_name>_es-data.
I would like to make a docker crash test on my server, to see how many containers based on the same image my server will support. (Because I've installed jupyterhub and I want to see how many containers can run in good condition.)
So how can I copy an existing container?
No need to copy an existing container, just create new ones of the same image. For your purposes I would recommend using the scale feature of docker-compose.
docker-compose.yml:
web:
image: <someimage>
db:
image: <someotherimage>
Then simply specify the amount of containers you would like to start:
$ docker-compose up -d
$ docker-compose ps
$ docker-compose scale web=15 db=3
$ docker-compose ps