I built image using
docker build -t my-image
docker-compose.yml has
django:
build:
context: .
dockerfile: ./compose/django/Dockerfile-dev
image: my-image
Then I run docker-compose build
I see my-image is being built again even though I built it previously.
Can an image built by docker build be used by docker-compose ?
What you have written
As the docs say:
If you specify image as well as build, then Compose names the built image with the webapp and optional tag specified in image:
How to avoid this
If you want to rebuild each time
Build as you are, and the build artifact will be saved with the name my-image
If you want to reuse the build
Change to just specify the image to use
If you only want to build if the image doesn't exist
Run compose with --no-build as this describes
Related
I have a docker-compose script that brings up a service
version : '2.0'
services:
orig-db:
image: web-url:{image_tag}
custom-db:
image: local_image: latest
Where image used in custom DB is the the result of bringing up a container with orig-db, performing some basic bash commands, and doing a docker commit. I want the custom-db image to always be the original image + these commands, even if the original image is updated. Is there a way to "rebase" off the original image?
You can think of a Dockerfile as a simple form of a "rebase".
# Content of subdir/Dockerfile
FROM orig_image:latest
RUN some.sh
RUN basic.sh
RUN bash_commands.sh
When you build an image based on this file, it will always run the bash commands on top of the base image. Inside the compose file you can use the build property to instruct docker-compose to build the image instead of using a pre-made image.
version : '2.0'
services:
orig-db:
image: web-url:{image_tag}
custom-db:
build: somedir
If the base image changes, you need to tell docker-compose to rebuild the custom-db image again, running the bash commands again on top of the updated original image.
docker-compose up -d --build custom-db
I am trying to come up with a CI system where I validate the Dockerfile and docker-compose.yaml files that are used to build our images.
I found Google containter-structure-tests
that can be used to verify the structre of Docker images that are built. This works if the docker images are build from Dockerfile.
Is there a way that I can verify the docker images with all the configurations that are added to the images by Docker-compose?
EDIT:
Maybe I didn't all put all my details into the questions.
Lets say I have docker-compose file with the following structure:
version: "3"
services:
image-a:
build:
context: .
dockerfile: Dockerfile-a
image-b:
build:
context: .
dockerfile: Dockerfile-b
ports:
- '8983:8983'
volumes:
- '${DEV_ENV_ROOT}/solr/cores:/var/data/solr'
- '${DEV_ENV_SOLR_ROOT}/nginx:/var/lib/nginx'
Now that the images would be built from Dockerfile-a and Dockerfile-b, there would be configurations made on top of image foo-b. How can I validate those configurations without building the container from image foo-b? Would that even be possible?
Assuming you have the following docker-compose.yml file:
version: "3"
services:
image-a:
build:
context: .
dockerfile: Dockerfile-a
image-b:
build:
context: .
dockerfile: Dockerfile-b
Build your images running the command docker-compose --project-name foo build. This will make all images' name start with the prefix foo_. So you would end up with the following image names:
foo_image-a
foo_image-b
The trick is to use a unique id (such as your CI job id) instead of foo so you can identify the very images that were just built.
Now that you know the names of your images, you can use:
container-structure-test test --image foo_image-a --config config.yaml
container-structure-test test --image foo_image-b --config config.yaml
If you are to make some kind of generic job which does not know the docker compose service names, you can use the following command to get the list of images starting with that foo_ prefix:
docker image list --filter "reference=foo_*"
REPOSITORY TAG IMAGE ID CREATED SIZE
foo_image-a latest 0c5e1cf8c1dc 16 minutes ago 4.15MB
foo_image-b latest d4e384157afb 16 minutes ago 4.15MB
and if you want a script to iterate over this result, add the --quiet option to obtain just the images' id:
docker image list --filter "reference=foo_*" --quiet
0c5e1cf8c1dc
d4e384157afb
I'm using docker compose to build my application using docker.
Version of docker-compose is 2.2
I have all the containers running well at the moment where one of the container has nginx running.
I need to change some configuration on this container.
The way I need to do (because of special scenario) is, to update the config inside the container.
Then I commit the container to build a new image.
docker commit <container> <image-name>
Now I have new image with tag latest.
What I want is to use this image when I run, docker-compose down && docker-compose up --build next time.
docker-compose down && docker-compose up --build -d
With --build option, docker-compose will go through the steps in Dockerfile and run those and all my changes will be reverted.
Question:
Is there anyway that I can tell docker-compose to use the newly created image as cache and ignore Dockerfile for this one container?
Solution Tried:
I have tried with docker-compose-override and using option cache-from and it's not working.
docker-compose.override.yml
container:
build:
cache_from:
- new-image:latest
Thanks in advance.
I don't understand why you would want to build an image from docker-compose even though you have already built it by docker-commit.
Now I have new image with tag latest.
What I want is to use this image when I run, docker-compose down && docker-compose up
If you have already built image, skip the build phase in docker-compose. Just specify which image should be used like so:
container:
image: new-image:latest
container_name: "Foo bar"
.....(other options)
Image
Specify the image to start the container from. Can either be a
repository/tag or a partial image ID.
image: redis
image: ubuntu:14.04
image: tutum/influxdb
image: example-registry.com:4000/postgresql
image: a4bc65fd
If the image does
not exist, Compose attempts to pull it, unless you have also specified
build, in which case it builds it using the specified options and tags
it with the specified tag.
If you have any other images that you build from inside docker-compose run:
docker-compose build && docker-compose up
If not simple docker-compose up will suffice.
What is the difference between docker-compose build and docker build?
Suppose in a dockerized project path there is a docker-compose.yml file:
docker-compose build
And
docker build
docker-compose can be considered a wrapper around the docker CLI (in fact it is another implementation in python as said in the comments) in order to gain time and avoid 500 characters-long lines (and also start multiple containers at the same time). It uses a file called docker-compose.yml in order to retrieve parameters.
You can find the reference for the docker-compose file format here.
So basically docker-compose build will read your docker-compose.yml, look for all services containing the build: statement and run a docker build for each one.
Each build: can specify a Dockerfile, a context and args to pass to docker.
To conclude with an example docker-compose.yml file :
version: '3.2'
services:
database:
image: mariadb
restart: always
volumes:
- ./.data/sql:/var/lib/mysql
web:
build:
dockerfile: Dockerfile-alpine
context: ./web
ports:
- 8099:80
depends_on:
- database
When calling docker-compose build, only the web target will need an image to be built. The docker build command would look like :
docker build -t web_myproject -f Dockerfile-alpine ./web
docker-compose build will build the services in the docker-compose.yml file.
https://docs.docker.com/compose/reference/build/
docker build will build the image defined by Dockerfile.
https://docs.docker.com/engine/reference/commandline/build/
Basically, docker-compose is a better way to use docker than just a docker command.
If the question here is if docker-compose build command, will build a zip kind of thing containing multiple images, which otherwise would have been built separately with usual Dockerfile, then the thinking is wrong.
Docker-compose build, will build individual images, by going into individual service entry in docker-compose.yml.
With docker images, command, we can see all the individual images being saved as well.
The real magic is docker-compose up.
This one will basically create a network of interconnected containers, that can talk to each other with name of container similar to a hostname.
Adding to the first answer...
You can give the image name and container name under the service definition.
e.g. for the service called 'web' in the below docker-compose example, you can give the image name and container name explicitly, so that docker does not have to use the defaults.
Otherwise the image name that docker will use will be the concatenation of the folder (Directory) and the service name. e.g. myprojectdir_web
So it is better to explicitly put the desired image name that will be generated when docker build command is executed.
e.g.
image: mywebserviceImage
container_name: my-webServiceImage-Container
example docker-compose.yml file :
version: '3.2'
services:
web:
build:
dockerfile: Dockerfile-alpine
context: ./web
ports:
- 8099:80
image: mywebserviceImage
container_name: my-webServiceImage-Container
depends_on:
- database
Few additional words about the difference between docker build and docker-compose build.
Both have an option for building images using an existing image as a cache of layers.
with docker build, the option is --cache-from <image>
with docker-composer, there is a tag cache_from in the build section.
Unfortunately, up until now, at this level, images made by one are not compatible with the other as a cache of layers (Ids are not compatible).
However, docker-compose v1.25.0 (2019-11-18), introduces an experimental feature COMPOSE_DOCKER_CLI_BUILD so that docker-compose uses native docker builder (therefore, images made by docker build can be used as a cache of layers for docker-compose build)
I am building a lamp with docker-compose.
In my docker-compose.yml i have the following:
ubuntu-base:
build: ./ubuntu-base
webserver-base:
build: ./webserver-base
webserver-base is derived from the ubuntu-base image.
In webserver-base Dockerfile:
FROM docker_ubuntu-base
ubuntu-base is built
FROM ubuntu:14.04
Now, if i execute the docker-compose.yml, it does not build the ubuntu-base image, but its trying to build the webserver-base image and fails, because it does not find the ubuntu-base image.
Output:
$ docker-compose up -d
Building webserver-base
Step 1 : FROM docker_ubuntu-base
Pulling repository docker.io/library/docker_ubuntu-base
ERROR: Service 'webserver-base' failed to build: Error: image library/docker_ubuntu-base:latest not found
It all works if i build the ubuntu-base image manually first.
why does it not build the ubuntu-base image?
Sadly, build ordering is a missing feature in docker-compose, that is requested for many month now.
As workaround you can link the containers like this:
ubuntu-base:
build: ./ubuntu-base
webserver-base:
build: ./webserver-base
links:
- ubuntu-base
this way ubuntu-base gets built before webserver-base.
First do a
docker-compose build ubuntu-base
But this will not create the image docker_ubuntu-base locally because you do not have any build steps. Only docker.io/ubuntu:14.04 will be downloaded.
If you add a build step like:
FROM ubuntu:14.04
RUN date
A docker_ubuntu-base image will be created.
So first do a:
docker-compose build ubuntu-base
This will create the image docker_ubuntu-base. Then you can do a docker-compose build.
But I would advise against this nested-docker image construction. This is cumbersome because as #kev indicated you have no control over the order of the builds. Why don't you create two independent docker files? Let docker derive webserver-base from ubuntu-base by keeping the Dockerfile instructions as identical as possible and reusing the layers.