I have a setup that I am migrating to docker-compose from a set of shell scripts. The container I want to build uses a parent container that is created by a custom dockerfile. But I can't see how to get docker-compose to (re)build the requisite parent container.
There are three files as per:
/code/containers/parent/DockerFile
FROM centos:7
RUN... (rest of file to create common stuff used by multiple child images)
/code/containers/child-one/Dockerfile
FROM parent
RUN...
/code/docker-compose.yml
version: '3.3'
services:
my-service:
image: child-one
build:
dockerfile: containers/child-one/Dockerfile
context: .
As expected it will fail with:
Service 'my-service' failed to build: repository parent not found: does not exist or no pull access
Can't find any solution to this other than manually running docker to build the parent image first.
Any ideas much appreciated.
edit: base on VonCs idea:
version: '3.3'
services:
parent-notservice;
image: parent
build:
dockerfile: containers/parent/Dockerfile
context: .
my-service:
image: child-one
depends_on:
parent
build:
dockerfile: containers/child-one/Dockerfile
context: .
However I had to use depends_on, which was a hack, I am worried about effects of the parent starting (when child is run). This is not my intent.
As of January 2021, there is another way to do this by using the profiles key. Support for this key was added in Docker Compose 1.28.0.
version: '3.9'
services:
parent-notservice:
build:
dockerfile: containers/parent/Dockerfile
context: .
profiles:
- donotstart
my-service:
image: parent-notservice
another-service:
build:
context: .
dockerfile: ./containers/service/Dockerfile
my-service will be based directly on parent-notservice. Then you can add whatever ports and volumes are needed for that container in the docker-compose.yml file.
another-service will be built from the Dockerfile which could also be based on parent-notservice using the FROM command. Other packages can then be installed on that container that are not part of the parent.
FROM parent-notservice
RUN...
And the best part is that when you use docker-compose up the parent-notservice will not start.
Further documentation on the profiles key can be found here: https://docs.docker.com/compose/profiles/
"parent" would exist in your local docker registry if your build was setting the name "parent" (docker build -t parent)
With a docker-compose file, you need to build parent/DockerFile first, with an image: parent under your build directive.
However, you should only build service you intent to run, which is not the case for "parent": parent should be built before docker-compose is involved.
The proper solution is to have a wrapper script which would:
docker build -t parent ...
then call docker-compose
No more depends_on hack between imaginary services (there is no "service" for parent: nothing runs there).
Related
My microservices project structure is like this:
my-service-one/
- Dockerfile
- ...
my-service-two/
- Dockerfile
- ...
docker-compose.yml
As you can see, each service directory contains a Dockerfile. There is a docker-compose.yml in the root level.
The docker-compose.yml :
version: "3"
services:
service-one:
container_name: service-one
build:
dockerfile: ./my-service-one/Dockerfile
ports:
- "8081:8081"
service-two:
container_name: service-two
build:
dockerfile: ./my-service-two/Dockerfile
ports:
- "8082:8082"
Now, I run docker-compose up -d from the root. I end up with error:
$ docker-compose up -d
ERROR: The Compose file is invalid because:
Service service-one has neither an image nor a build context specified. At least one must be provided.
My question is why does docker-compose think my service-one doesn't have a build context specified? Didn't I specify it already with:
build:
dockerfile: ./my-service-one/Dockerfile
Why this error?
why does docker-compose think my service-one doesn't have a build context specified?
Weeeell, because you did not specified the build context.
Didn't I specify it already with:
No, you specified the dockerfile. No the context.
Why this error?
You have to specify the context so that docker knows what to build.
If you want to build with the context of current directory, you would do:
build:
context: .
dockerfile: ./my-service-two/Dockerfile
Maybe the context is inside my-service-two, I suspect youw antto write:
build:
context: ./my-service-two
dockerfile: ./Dockerfile
or really just:
build: ./my-service-two
Peovide a context property below both services in build section like that:
build:
context: YOUR_DIRECTORY
dockerfile: ./my-service-one/Dockerfile
YOUR_DIRECTORY is the place where the files for your project are listed.
Most probably YOUR_DIRECTORY is already written i the child .yml files.
You have a couple of main approaches:
To copy paste the context from the child .yml
To produce the docker build using the child .yml with a command like:
docker-compose -f docker-compose.yml -f docker-compose-dev.yml up
--build
I have a simple frontend and backend app. And I have a docker-compose file but it's inside frontend folder. So when I run it both frontend and backend containers are under frontend container (it takes name of the folder) how can I rename this main container? I am using version 3.9
version: "3.9"
services:
be_service:
container_name: backend
build:
context: ../backend
dockerfile: ./Dockerfile
ports:
- "8089:8080"
fe_service:
container_name: frontend
build:
context: ./
dockerfile: ./Dockerfile
ports:
- "8088:80"
depends_on:
- be_service
When refering to your main container, you are probably refering to the project name, which you could usually set via the -p flag. (See other answers)
For docker-compose, you can set the top level variable name to your desired project name.
docker-compose.yml file:
version: "3.9"
name: my-project-name
services:
myService:
...
If you are using Docker Desktop, make sure Use Docker Compose V2 is enabled there.
Related to Docker Compose docs you can set your project name with:
docker-compose -p app up --build
with -p app to set your compose container name to app.
I think that your docker compose file is right and to change the co you can use the containe_name instruction but I think you should run this command when you want to run your application :
docker-compose up --build
Use -p to specify a project name
Each configuration has a project name. If you supply a -p flag, you can specify a project name. If you don’t specify the flag, Compose uses the current directory name.
Calling docker-compose --profile frontend up will start the services with the profile frontend and services without specified profiles. You can also enable multiple profiles, e.g. with docker-compose --profile frontend --profile debug up the profiles frontend and debug will be enabled
Also refer https://docs.docker.com/compose/profiles/
I see lots of questions around setting/changing the COMPOSE_PROJECT_NAME or PROJECT_NAME using ENV variables.
I'm fine with the default project name, but I would like to reference it in my compose file.
version: "3.7"
services:
app:
build: DockerFile
container_name: app
volumes:
- ./:/var/app
networks:
- the-net
npm:
image: ${project_name}_app
volumes:
- ./:/var/app
depends_on:
- app
entrypoint: [ 'npm' ]
networks:
- the-net
npm here is arbitrary , hopefully the fact that could be run as its own container or in other ways does not distract from the questions.
is it possible to reference the project name with out setting it manually or first?
Unfortunately it is not possible.
As alluded to, you can create a .env file and populate it with COMPOSE_PROJECT_NAME=my_name, but the config option does not present itself in your environment by default.
Unfortunately the env substitution in docker-compose is fairly limited, meaning we cannot use the available PWD env variable and greedy match it at all
$ cd ~
$ pwd
/home/tqid
$ echo "Base Dir: ${PWD##*/}"
Base Dir: tqid
When we use this reference, compose has issues:
$ docker-compose up -d
ERROR: Invalid interpolation format for "image" option in service "demo": "${PWD##*/}"
It's probably better to be explicit anyway, the COMPOSE_PROJECT_NAME is based on your dir, and if someone clones to a new folder then it gets out of whack, including the .env file in source control would provide a re-usable and consistent place to reference the name
https://docs.docker.com/compose/reference/envvars/#compose_project_name
using the same image as another container was what I was after ... reuse the image and change the entry point.
Specify the same build: options for both containers.
This seems inefficient, in that it will trigger the build sequence twice and docker images will list both of them. However, the way Docker's layer caching works, if identical RUN commands are run on identical input images, the resulting layer will simply be reused, and the two final images will have the same image ID; they will literally be the same image with two names.
The context I've run into this the most is with a Python application where the same code base is used for a Django or Flask Web server, plus a Celery worker. The Docker-level setup is fairly language-independent, though: specify the same build: for both containers, and override the command: for the container(s) that need to do a non-default task.
version: '3.8'
services:
app:
build: .
ports: ['3000:3000']
environment:
REDIS_HOST: redis
worker:
build: . # <-- same as app
command: npm run worker # <-- overrides Dockerfile CMD
environment:
REDIS_HOST: redis
redis:
image: redis
It is also valid to specify build: and image: together in the docker-compose.yml file; this specifies the name of the image that will be built. It's frequently useful to explicitly specify this because you will need to point at a specific Docker Hub or other registry location to push the built image. If you do this, then you'll know the image name and don't need to depend on the context name.
version: '3.8'
services:
app:
build: .
image: registry.example.com/my/app:${TAG:-latest}
worker:
image: registry.example.com/my/app:${TAG:-latest}
command: npm run worker
You will need to manually docker-compose build in this setup. Compose's workflow doesn't have a way to specify that one container's build must run before a different container can start.
I am trying to use docker and want to create an Ubuntu base with three containers that do the following:
Container: Install Wildfly
Container: Install MySQL
Container: Other Required Packages
Does that mean, I have to create three Dockerfiles in three different directories containing each the following top line?:
FROM ubuntu:18.04
Orchestrate the containers with docker-compose
1- Create docker-compose.yml
2- Inside define:
version: '3'
services:
wildly:
build:
context: .
dockerfile: Dockerfile_Wildfly
mysql:
build:
context: .
dockerfile: Dockerfile_Mysql
anotherpackages:
build:
context: .
dockerfile: Dockerfile_AnotherPackages
it is not always the case that you need to write a docker file, for example for the database service you can simply pull the image from the docker hub and use it directly.
something like below
db:
image: mysql
3- Create files and both define commands you prefer:
Dockerfile_Wildfly
FROM wildfly
Dockerfile_Mysql
FROM mariadb
Dockerfile_AnotherPackages
FROM node
FROM nginx
You can create many Dockerfile and precise their name in the build command as suggested in another answer (#Krumelur's answer), but you can also use docker compose by calling directly the image from docker.io (if the base image for those dependecies in the hub match your neeed)
In this way you dont need any Dockerfile at all.
It should looks like this :
version: '3.3'
services:
wildfly:
#this image will be automatically downloaded from your registry (by default Docker hub)
image: jboss/wildfly
ports:
- '8080:8080'
- '9990:9990'
volumes:
- 'wildfly_data:/wildfly_data'
environment:
- WILDFLY_PASSWORD=password
mysql:
#this image will be automatically downloaded from your registry (by default Docker hub)
image: mysql:5.7
restart: always
environment:
MYSQL_DATABASE: 'db'
# So you don't have to use root, but you can if you like
MYSQL_USER: 'user'
# You can use whatever password you like
MYSQL_PASSWORD: 'password'
# Password for root access
MYSQL_ROOT_PASSWORD: 'password'
ports:
# <Port exposed> : < MySQL Port running inside container>
- '3306:3306'
expose:
# Opens port 3306 on the container
- '3306'
# Where our data will be persisted
volumes:
- my-db:/var/lib/mysql
otherService :
image: busybox
volumes:
my-db:
wildfly_data:
Then you just need to call the command : docker-compose up
You can have more than one Dockerfile in the same directory if desired. To specify the Dockerfile to use, use the -f argument, e.g
docker build -f wildfly.Dockerfile ./wildfly
docker build -f mysql.Dockerfile ./mysql
docker build -f other.Dockerfile ./other
In Compose, these arguments correspond to the dockerfile and context properties.
It is not always the case that you need to write a docker file, for example for the database service you can simply pull the image from the docker hub and use it directly.
something like below
db:
image: mysql
You can, of course, have them share the same context, e.g.
docker build -f wildfly.Dockerfile .
docker build -f mysql.Dockerfile .
docker build -f other.Dockerfile .
Just be aware that the context is sent in full to the daemon (respecting .dockerignore) so this might lead to longer build times if there is a lot of redundant data.
If there is a lot of reuse between the Dockerfiles, you can even have all of them in one file, e.g.
FROM ubuntu:20.04 as base
...
FROM base AS wildfly
(install wildfly)
FROM base AS mysql
(install mysql)
...
Then you can build the specific image with e.g.
docker build --target wildfly .
In Compose, these arguments correspond to the target and context properties.
This is called multi-stage builds, and is not always a good idea but is sometimes helpful to mitigate Docker's lack of support for #include.
Please help me understand the difference between 'image' and 'build' within docker compose
image means docker compose will run a container based on that image
build means docker compose will first build an image based on the Dockerfile found in the path associated with build (and then run a container based on that image).
PR 2458 was eventually merged to allow both (and use image as the image name when building, if it exists).
therobyouknow mentions in the comments:
dockerfile: as a sub-statement beneath build: can be used to specify the filename/path of the Dockerfile.
version: '3'
services:
webapp:
build:
context: ./dir
dockerfile: Dockerfile-alternate
args:
buildno: 1
build: expects dockerfile path as an argument, it will build an image first and then use the image to create a container.
image: expects existing image name as argument , it will launch container using this image.
Example:docker-compose.yaml
version: '3'
services:
service1:
build: .
ports:
- "5000:5000"
service2:
image: "redis:alpine"
service1 will build an image first based on Dockerfile of current path and run container based on this image.
service2 will download "redis:alpine" image from docker hub and run container on downloaded image.