I have two services in my docker-compose.yaml that use the same build context:
service_1:
image: 'service_1:latest'
container_name: service_1
env_file:
- ./service_dir/.env
ports:
- "8000:80"
build:
context: ./service_dir/
dockerfile: Dockerfile
volumes:
- ./service_dir/app:/app
command: /start-reload.sh
service_2:
container_name: service_2
build:
context: ./service_dir/
dockerfile: Dockerfile
command: python app/my_script.py
Building this takes forever. I assume it has something to do with the fact that the build context is taken twice to the docker daemon? I'm not sure if these commands could be run in the same service but I'd prefer to have two separate containers so I can e.g. follow service_2 logs more easily. Any suggestions for a good solution to my problem?
Related
I have a docker-compose.yml
services:
nextjs:
container_name: next_app
build:
context: ./
restart: on-failure
command: npm run dev
volumes:
- ./:/app
- /app/node_modules
- /app/.next
ports:
- "3000:3000"
cypress:
image: "cypress/included:9.4.1"
depends_on:
- next_app
environment:
- CYPRESS_baseUrl=http://nextjs:3000
working_dir: /e2e
volumes:
- ./e2e:/e2e
I want to change env_file for next_app from cypress service. I found solution like this
cypress:
image: "cypress/included:9.4.1"
depends_on:
- next_app
environment:
- CYPRESS_baseUrl=http://nextjs:3000
working_dir: /e2e
volumes:
- ./e2e:/e2e
next_app:
env_file: .env.test
But this solution does not work. Is it even possible ?
Try something like cp .env #docker/.env
No. In Compose (or Docker, or even more generally in Linux/Unix) there is no way for one container (process) to specify environment variables for another.
You can think of a docker-compose.yml file as a set of instructions only for running containers. If you need a specific set of containers for a specific context – you don't normally need to run Cypress in production, but this is an integration-test setup – it's fine to write a separate Compose file just for that setup.
# docker-compose.cypress.yml
# Used only for integration testing
version: '3.8'
services:
nextjs:
build: .
restart: on-failure
ports:
- "3000:3000"
env_file: .env.test # <-- specific to this test-oriented Compose file
cypress:
build: ./e2e
depends_on:
- nextjs
environment:
- CYPRESS_baseUrl=http://nextjs:3000
docker-compose -f docker-compose.cypress.yml up --build
This can also be a case where using multiple Compose files together can be a reasonable option. You can define a "standard" Compose setup that only defines the main service, and then an e2e-test Compose file that adds the Cypress container and the environment settings.
# docker-compose.yml
version: '3.8'
services:
nextjs:
image: registry.example.com/nextjs:${NEXTJS_TAG:-latest}
restart: on-failure
ports:
- '3000:3000'
# docker-compose.e2e.yaml
version: '3.8'
services:
nextjs:
# These add to the definitions in the base `docker-compose.yml`
build: .
env_file: .env.test
cypress:
# This is a brand new container for this specific setup
depends_on: [nextjs]
et: cetera # copy from question or previous Compose setup
docker-compose \
-f docker-compose.yml \
-f docker-compose.e2e.yml \
up --build
I was using docker-compose, but when I tried to build it again, this error shows, I have build this docker-compose multiple times:
ERROR: Service 'api' failed to build: max depth exceeded
I tried to execute docker system prune to clean my containers, but it didn't work.
docker-compose.yml
version: "3"
services:
client:
container_name: my_client
image: mhart/alpine-node:12
build: ./client
restart: always
ports:
- "3000:3000"
working_dir: /client
volumes:
- ./client:/client
entrypoint: ["npm", "start"]
links:
- api
networks:
- my_network
api:
container_name: my_api
build: ./api
restart: always
ports:
- "9000:9000"
environment:
DB_HOSTNAME: mysql
working_dir: /api
volumes:
- ./api:/api
depends_on:
- mysql
networks:
- my_network
mysql:
container_name: my_mysql
build: ./db
restart: always
volumes:
- /var/lib/mysql
- ./db:/db
ports:
- "3307:3306"
environment:
- MYSQL_ROOT_PASSWORD=n
- MYSQL_USER=n
- MYSQL_PASSWORD=n
- MYSQL_DATABASE=n
networks:
- my_network
command: '--default-authentication-plugin=mysql_native_password'
networks:
my_network:
driver: bridge
this is the Dockerfile:
FROM mhart/alpine-node:12
WORKDIR /api
COPY package*.json /api/
RUN npm i -G nodemon
RUN npm install
COPY . /api/
EXPOSE 9000
CMD ["npm", "run", "dev"]
any help is appreciated.
So, I figure out, I just needed to execute docker system prune -a to remove any stopped container. Now --build is working again.
This command deleted all my local docker images related to my dockerfile. After building it so many times my local storage has reached a limited, thus the error max depth exceeded.
Max depth doesn't indicate an out-of-storage-capacity error (though a prune could accidentally fix it).
Rather it indicates that the api image that you were building had too many layers.
A plausible theory is that you have a recursion caused by having this in your compose file:
image: mhart/alpine-node:12
build: ./client
and this in a Dockerfile
FROM mhart/alpine-node:12
(I'm assuming the Dockerfile in ./client is also FROM the same image).
Your build is essentially adding a few layers onto your local mhart/alpine-node:12 image every time you run it (you can confirm by running docker history mhart/alpine-node:12).
If so, you should probably rename the image in your compose file.
Docker doesn't use the latest code after running git checkout <non_master_branch>, while I can see it in the vscode.
I am using the following docker-compose file:
version: '2'
volumes:
pgdata:
backend_app:
services:
nginx:
container_name: nginx-angular-dev
image: nginx-angular-dev
build:
context: ./frontend
dockerfile: /.docker/nginx.dockerfile
ports:
- "80:80"
- "443:443"
depends_on:
- web
web:
container_name: django-app-dev
image: django-app-dev
build:
context: ./backend
dockerfile: /django.dockerfile
command: ["./wait-for-postgres.sh", "db", "./django-entrypoint.sh"]
volumes:
- backend_app:/code
ports:
- "8000:8000"
depends_on:
- db
env_file: .env
environment:
FRONTEND_BASE_URL: http://192.168.99.100/
BACKEND_BASE_URL: http://192.168.99.100/api/
MODE_ENV: DOCKER_DEV
db:
container_name: django-db
image: postgres:10
env_file: .env
volumes:
- pgdata:/var/lib/postgresql/data
I have tried docker-compose build --no-cache, followed by docker-compose up --force-recreate but it didn't solve the problem.
What is the root of my problem?
Your volumes: are causing problems. Docker volumes aren't intended to hold code, and you should delete the volume declarations that mention backend_app:.
Your docker-compose.yml file says in part:
volumes:
backend_app:
services:
web:
volumes:
- backend_app:/code
backend_app is a named volume: it keeps data that must be persisted across container runs. If the volume doesn't exist yet the first time then data will be copied into it from the image, but after that, Docker considers it to contain critical user data that must not be updated.
If you keep code or libraries in a Docker volume, Docker will never update it, even if the underlying image changes. This is a common problem in JavaScript applications that mount an anonymous volume on their node_modules directory.
As a temporary workaround, if you docker-compose down -v, it will delete all of the volumes, including the one with your code in it, and the next time you start it will get recreated from the image.
The best solution is to simply not use a volume here at all. Delete the lines above from your docker-compose.yml file. Develop and test your application in a non-Docker environment, and when you're ready to do integration testing, run docker-compose up --build. Your code will live in the image, and an ordinary docker build will produce a new image with new code.
Trying to use docker-compose for the first time, but not having much luck. I have the following setup:
docker-compose version 1.8.0, build f3628c7
/home/GabeThermComposer contains the docker-compose.yml
/home/GabeThermComposer/GabeThermApache contains Dockerfile
/home/GabeThermComposer/GabeThermPHPMyAdmin contains Dockerfile
/home/GabeThermComposer/GabeThermDB contains Dockerfile and nest-init.sql
When I create docker images using the Dockerfile in each subdir, it all works without issues. I was hoping with the docker-compose.yml to do all the seperate building of images at once.
The docker-compose.yml looks like this:
version: '2'
services:
GabeThermDB:
build:
context: ./GabeThermDB
dockerfile: Dockerfile
GabeThermApache:
build:
context: ./GabeThermApache
dockerfile: Dockerfile
ports:
- "80:80"
GabeThermPHPMyAdmin:
build:
context: ./GabeThermPHPMyAdmin
dockerfile: Dockerfile
ports:
- "8080:80"
When trying to run "docker-compose up", I get the following error:
ERROR: The Compose file './docker-compose.yml' is invalid because:
services.GabeThermPHPMyAdmin.build contains unsupported option: 'ports'
services.GabeThermApache.build contains unsupported option: 'ports'
I have no clue on what is wrong with this. I think I did exactly as other examples have shown. Btw, I do know that the "context:" and "dockerfile:" is overdone, but since I'm new, I wanted to be sure to what files I'm pointing in case I forget it automatically dives into the subdir and runs the Dockerfile.
Any help is appreciated.
You have to move the ports out of the build block.
version: '2'
services:
GabeThermDB:
build:
context: ./GabeThermDB
dockerfile: Dockerfile
GabeThermApache:
build:
context: ./GabeThermApache
dockerfile: Dockerfile
ports:
- "80:80"
GabeThermPHPMyAdmin:
build:
context: ./GabeThermPHPMyAdmin
dockerfile: Dockerfile
ports:
- "8080:80"
I have two docker-compose.yml files in separate folders.
I'd like to run the two of them in the same command, in order for the services from both to be able to talk to each other.
However, when I go to the lowest common path ancestor and try to run docker-compose with both files, here is what happens:
$ docker-compose -f ./api-folder/docker-compose.yml -f ./front-folder/docker-compose.yml up -d
ERROR: build path /projects/front-folder/api either does not exist, is not accessible, or is not a valid URL.
$ docker-compose -f ./front-folder/docker-compose.yml -f ./api-folder/docker-compose.yml up -d
ERROR: build path /projects/api-folder/app either does not exist, is not accessible, or is not a valid URL.
Here are the two docker-compose.yml files:
/projects/front-folder/docker-compose.yml
version: '2'
services:
app:
restart: always
build: ./app
environment:
NODE_ENV: 'dev'
ports:
- "4400:4400"
volumes:
- ./app:/usr/src/app
nginx:
restart: always
build: ./nginx
volumes:
- ./logs:/usr/local/var/log/nginx
links:
- app
ports:
- "80:80"
/projects/api-folder/docker-compose.yml
version: '2'
services:
api:
restart: always
build: ./api
expose:
- "4600"
volumes:
- ./api:/usr/src/app
- ./logs:/logs
nginx:
restart: always
build: ./nginx
volumes:
- ./logs:/usr/local/var/log/nginx
links:
- api
ports:
- "81:80"
networks:
- hackerz
And the directory structure:
- /projects
- /front-folder
- /app
Dockerfile
- /nginx
Dockerfile
docker-compose.yml
- /api-folder
- /api
Dockerfile
- /nginx
Dockerfile
docker-compose.yml
I'm guessing the problem is with the build paths, but what I don't understand is:
Why Docker insists on searching build: ./api in /front-folder or the other way around?
How to circumvent this problem and be able to run both files together?
DOCKERFILE
Alternate Dockerfile.
Compose uses an alternate file to build with. A build path must also be specified.
service3:
build:
context: .
dockerfile: Dockerfile-alternate
docker compose build giving custom file
This isn't how compose works (by design). See my comment here: https://github.com/docker/compose/issues/3530#issuecomment-222490501.