Docker-compose expected container is up to date - docker

I created docker-compose yaml file to run a docker image after it has been pulled locally and this yaml file contain another services (mysql and phpmyadmin) so when I run a command docker-composer up -d I found a conflict in creating a container as it been used by another container with the same name and I expected to show me that the container is already run and up to date, I know that the container must be removed or renamed before creating a new one but I aiming to get the newer version of image and check if mysql and phpmyadmin services is up or not if up gives me those container is up to date and if not create it as another environment bellow.
docker-compose.yml
version: '3.3'
services:
app-prod:
image: prod:1.0
container_name: app-prod
ports:
- "81:80"
links:
- db-prod
depends_on:
- db-prod
- phpmyadmin-prod
db-prod:
image: mysql:8
container_name: db-prod
environment:
- MYSQL_ROOT_PASSWORD=secret
- MYSQL_DATABASE=laravel
- MYSQL_USER=user
- MYSQL_PASSWORD=secret
volumes:
- db-prod:/var/lib/mysql
phpmyadmin-prod:
image: phpmyadmin/phpmyadmin:5.0.1
restart: always
environment:
PMA_HOST: db-prod
container_name: phpmyadmin-prod
ports:
- "5001:80"
volumes:
db-prod:
Error
Creating phpmyadmin-prod ... error
Creating db-prod ...
ERROR: for phpmyadmin-prod Cannot create container for service phpmyadmin: Conflict. The container name "/phpmyadmin-prod" is already in use by container "5a52b27b64f7302bccb1c3a0eaeca8a33b3dfee5f1a279f6a809695Creating db-prod ... error
ERROR: for db-prod Cannot create container for service db: Conflict. The container name "/db-prod" is already in use by container "5d01c21efafa757008d1b4dbcc8d09b4341ac1457a0ca526ee57873cd028cf2b". You have to remove (or rename) that container to be able to reuse that name.
ERROR: for phpmyadmin Cannot create container for service phpmyadmin: Conflict. The container name "/phpmyadmin-prod" is already in use by container "5a52b27b64f7302bccb1c3a0eaeca8a33b3dfee5f1a279f6a809695f482500a9". You have to remove (or rename) that container to be able to reuse that name.
ERROR: for db Cannot create container for service db: Conflict. The container name "/db-prod" is already in use by container "5d01c21efafa757008d1b4dbcc8d09b4341ac1457a0ca526ee57873cd028cf2b". You have to remove (or rename) that container to be able to reuse that name.
ERROR: Encountered errors while bringing up the project.
Error: No such container: app-prod
Error: No such container: app-prod
While using another docker-compose file for test environment I got this which I expected
db-stage is up-to-date
phpmyadmin-stage is up-to-date
Creating app-stage ... done

docker-compose run command will create new containers. But in your case, 2 of them are already running, hence, you can use
docker-compose up -d

That specific error is easy to fix. You're trying to manually specify container_name: for every container, but the error message says you have existing containers with those same names already. Left to its own, Compose will assign non-conflicting names, and you can almost always just delete container_name: from the Compose file.
version: '3.8'
services:
app:
image: prod:1.0
ports:
- "81:80"
depends_on: [db, phpmyadmin]
# no container_name: or links:
db: { ... }
phpmyadmin: { ... }
The other obvious point of conflict for running the same Compose file in multiple places is the ports:; only one container or host process can listen on a given (first) port number. If you're trying to run the same Compose file multiple times on the same system you might need some way to replace the port numbers. This could be a place where using multiple Compose files fits in well: define a base docker-compose.yml that defines the services but not any of the ports, and an override file that assigns specific host ports.

As I have several docker-compose files and I run this command docker-composer -f <compose_file_path> -p <project_name> -d up and then I try to run docker-composer up -d in the same location without specify the project name -p <project_name> which gives me the conflict of the container as this make the compose-file up with a different project name and with the same container name.

Related

How to reuse existing docker container after rename the directory

I have an existing docker container with the name foo_db_1.
The docker-compose file is located in the foo directory.
The container contains a huge DB.
I want to rename the directory from foo to bar
So I stopped the running container and renamed the directory and the container name
mv foo bar
docker rename foo_db_1 bar_db_1
Now when I try to start the container again I get a Conflict error:
docker-compose up -d
ERROR: for bar_db_1 Cannot create container for service db: Conflict. The container name "/bar_db_1" is already in use by container "4a9204e8031d72327be283967d2b9ec3268450140f00254eb2fc51ba8af0eb58". You have to remove (or rename) that container to be able to reuse that name.
When I rename the container, then a completely new container is created and I have to import all DB data again to that container.
When I remove the container, then I will loose the DB data and also have to import all db data again.
When I run docker inspect bar_db_1 I can see, that there are several entries pointing to the foo directory.
So how can I move an existing docker setup to another directory without loosing my existing data?
My docker-compose.yml looks like this:
version: "3.8"
volumes:
db:
services:
db:
image: mariadb:10.4
environment:
MYSQL_DATABASE: 'app'
MYSQL_USER: 'dbuser'
MYSQL_PASSWORD: '1234567890'
MYSQL_ROOT_PASSWORD: '1234567890'
volumes:
- db:/var/lib/mysql
- ./db-dumps:/var/db-dumps
command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
ports:
- "3306:3306"

Create multiple containers of the same service with Docker Compose

I am currently starting with Docker Compose, and I wanted to know how to create multiple containers of the same service 'Redis'. I've already tried with docker-compose up --scale redis=3, but it gives an error, I've been searching through Google the possible solution, but I could not find one.
Thank you in advance.
Here is my docker-compose.yml
version: '3'
services:
redis:
container_name: redis
hostname: redis
image: redis
redis-commander:
container_name: redis-dbms
hostname: redis-commander
image: rediscommander/redis-commander:latest
restart: always
environment:
- REDIS_HOSTS=local:redis:6379
ports:
- "8081:8081"
And here is the error it gives me.
docker-compose up --scale redis=3
Creating network "ex2_default" with the default driver
WARNING: The "redis" service is using the custom container name "redis". Docker requires each container to have a unique name. Remove the custom name to scale the service.
Creating redis-dbms ...
Creating redis ... done
Creating redis ...
Creating redis ...
Creating redis-dbms ... done
ERROR: for redis Cannot create container for service redis: Conflict. The container name "/redis" is already in use by container "d4c93ae4ca68da0b6430e5eddc657d9dda0f40002c7a81c89368535df05eae24". You have to remove (or rename) that container to be able to reuse that name.
ERROR: for redis Cannot create container for service redis: Conflict. The container name "/redis" is already in use by container "d4c93ae4ca68da0b6430e5eddc657d9dda0f40002c7a81c89368535df05eae24". You have to remove (or rename) that container to be able to reuse that name.
The error raise because the reason below:
Link: https://docs.docker.com/compose/compose-file/compose-file-v3/#container_name
Because Docker container names must be unique, you cannot scale a service beyond 1 container if you have specified a custom name. Attempting to do so results in an error.
You can read more add here: https://github.com/docker/compose/issues/3722

docker-compose how to create multiple containers with using the same image

for some reason, I need to create the container with the same image, But when I started the second one, It just restarted the fist one's container
the first yml file:
version: "3.1"
services:
php:
image:php:php73-fpm
restart: always
ports:
- "9000:9000"
- "9501:9501"
volumes:
- $PWD/../:/var/www/html/
networks:
- app_net
container_name: php
networks:
app_net:
driver: bridge
the second yml file:
version: "3.1"
services:
php:
image:php:php73-fpm
restart: always
ports:
- "19000:19000"
- "19501:19501"
volumes:
- $PWD/../:/var/www/html/
networks:
- app_net2
container_name: php73
networks:
app_net2:
driver: bridge
when I run docker-compose up -d to start the first one:
$ cd ~/Document/php/work/docker/
$ docker-compose up -d
Creating network "docker_app_net" with driver "bridge"
Creating php ... done
then I switch the directory to the second yml file
$ cd ../../private/docker/
$ docker-compose up -d
Recreating php ... done
Compose has a notion of a project name. By default the project name is the basename of the directory containing the docker-compose.yml file. In your example both directories are named docker (even if they're in different parent directories) so Compose looks for a project named docker and a container named php, and finds a match.
There are four ways to override this:
Rename one of the directories.
Set the COMPOSE_PROJECT_NAME environment variable.
Create a .env file in the current directory, and set COMPOSE_PROJECT_NAME there.
Use the docker-compose -p option (on every docker-compose command).
Within your docker-compose.yml file, the second part of ports: needs to match what the container is listening on; this is allowed to be different from the first part. So use the same 9500/9501 in both files.
Another consequence of the Compose project naming is that the standard names of containers, volumes, and networks that Compose creates will be prefixed with the project name. If the project name (current directory name) is docker2, and you reduce the Compose file to
version: "3.1"
services:
php:
build: .
restart: always
ports:
- "19000:9000"
- "19501:9501"
# no manual container_name: or networks:
The container will be named docker2_php_1, and it will be attached to a network named docker2_default; these will be different from the container/network created in the docker1 project/directory.
You can't have two containers with the same name. Since both names are just php, Docker thought they were settings that were supposed to be merged for the same container. Rename one of them.

Two docker services depends_on the same service

I am having two docker-compose files
/cfacing/docker-compose.yml
app-customer-facing:
build: .
depends_on:
- mysql-db
mysql-db:
container_name: staging-mysql-db
image: mysql:5.6
/afacing/docker-compose.yml
app-admin-facing:
build: .
depends_on:
- mysql-db
mysql-db:
container_name: staging-mysql-db
image: mysql:5.6
I want both customer-facing container and admin-facing container to depend on the same mysql-db container. This is currently not working, the app-customer-facing will start with mysql-db but app-admin-facing will not start throwing:
ERROR: for mysql-db Cannot create container for service mysql-db: Conflict. The container name "/staging-mysql-db" is already in use by container "fe63e1ab0c1fd19236551bfc5930544cb31e649a4c18421c05959dc1274eb600". You have to remove (or rename) that container to be able to reuse that name.
The error is that you are duplicating your mysql-db service, you are not re-using it. That's why you get an error telling that there is already a container named staging-mysql-db.
To resolve your use case, you'll have to extend your docker-compose file.
You can see an exemple here. See the block ADMINISTRATIVE TASKS which is basically what you are trying to do.

docker compose orphan containers warning

How to be with orphan images when you have 2 independent projects and you want them to work at the same time or at least to build running docker-compose up -d without --remove-orphans flag when images are already built for another project.
docker compose file1:
version: '2'
services:
applications:
image: tianon/true
volumes:
- ../../:/var/www/vhosts/project1
nginx:
build: ./images/nginx
image: project1/nginx:latest
ports:
- "80:80"
volumes_from:
- applications
networks:
appnet:
aliases:
- project1.app
- admin.project1.app
php:
image: project1/php:latest
ports:
- "7778:7778"
build:
context: ./images/php
dockerfile: Dockerfile
volumes_from:
- applications
networks:
- appnet
mysql:
image: project1/mysql:latest
build: ./images/mysql
environment:
MYSQL_ROOT_PASSWORD: secret
volumes:
- mysqldata:/var/lib/mysql
networks:
- appnet
ports:
- "33066:3306"
workspace:
image: project1/workspace:latest
build:
context: ./images/workspace
volumes_from:
- applications
working_dir: /var/www/vhosts/project1
networks:
- appnet
networks:
appnet:
driver: "bridge"
volumes:
mysqldata:
driver: "local"
the second docker compose file:
version: '2'
services:
project2_applications:
image: tianon/true
volumes:
- ../../:/var/www/vhosts/project2
project2_nginx:
build: ./images/nginx
image: project2/nginx:latest
ports:
- "8080:80"
volumes_from:
- project2_applications
networks:
project2_appnet:
aliases:
- project2.app
- admin.project2.app
project2_php:
image: project2/php:latest
ports:
- "7777:7777"
build:
context: ./images/php
dockerfile: Dockerfile
volumes_from:
- project2_applications
networks:
- project2_appnet
project2_mysql:
image: project2/mysql:latest
build: ./images/mysql
environment:
MYSQL_ROOT_PASSWORD: secret
volumes:
- project2_mysqldata:/var/lib/mysql
networks:
- project2_appnet
ports:
- "33067:3306"
project2_workspace:
image: project2/workspace:latest
build:
context: ./images/workspace
volumes_from:
- project2_applications
working_dir: /var/www/vhosts/videosite
networks:
- project2_appnet
networks:
project2_appnet:
driver: "bridge"
volumes:
project2_mysqldata:
driver: "local"
And now when I have already built project1 and trying to run docker-compose up -d for the second project I see warning:
WARNING: Found orphan containers (docker_workspace_1, docker_nginx_1, docker_php_1, docker_mysql_1, docker_memcached_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
I have a supposition that it's because container names for project1 should be more specific and I need to add some prefixes like I'm doing for project2, but project1 is in use by many other developers and I do not want to change it.
Is there any way to turn off orphan check?
And the second thing: is just a warning message but for some reason, after it appearing compose is failing with error:
ERROR: Encountered errors while bringing up the project.
And to make it work I need to run docker-compose up -d --remove-orphans
Compose uses the project name (which defaults to the basename of the project directory) internally to isolate projects from each other. The project name is used to create unique identifiers for all of the project's containers and other resources. For example, if your project name is myapp and it includes two services db and web, then Compose starts containers named myapp_db_1 and myapp_web_1 respectively.
You get the "Found orphan containers" warning because docker-compose detects some containers which belong to another project with the same name.
To prevent different projects from interfering with each other (and suppress the warning) you can set a custom project name by using any of the following options:
The -p command line option.
COMPOSE_PROJECT_NAME environment variable. This environment variable can also be set via an environment file (.env in the current working directory by default).
Top-level name element in the Compose file. Note: if you pass multiple files to docker-compose via the -f option, then the value from the last file will be used.
docker-compose takes the name of the directory it is in as the default project name.
You can set a different project name by using -p or --project-name.
https://docs.docker.com/compose/reference/#use--p-to-specify-a-project-name
I had a similar problem because my projects all had the docker/docker-compose.yml structure.
To build on other answers, I create a .env file with my docker compose projects. I have a number of projects that all use the docker directory but are different projects.
To use docker-compose -p is a bit error prone, so creating .env file in the same directory as the docker-compose.yml:
-rw-rw-r-- 1 auser auser 1692 Aug 22 20:34 docker-compose.yml
-rw-rw-r-- 1 auser auser 31 Aug 22 20:44 .env
alleviates the necessary overhead of remembering -p.
In the .env file, I can now set the COMPOSE_PROJECT_NAME variable:
COMPOSE_PROJECT_NAME=myproject
On running:
docker-compose up -d
the COMPOSE_PROJECT_NAME is substituted without the use of -p.
Reference:
https://docs.docker.com/compose/env-file/
docker-compose up --remove-orphans
you can run this command to clean orphan containers. As specified in the warning
If the orphaned containers are expected and not intended to remove, you can set COMPOSE_IGNORE_ORPHANS variable to true.
Consise but just right away working source is here.
One option is to put it as a line into .env file next to docker-compose.yml like this:
COMPOSE_IGNORE_ORPHANS=True
Another option is pass or set it as an environment variable.
sh:
COMPOSE_IGNORE_ORPHANS=True docker-compose up -d
or
export COMPOSE_IGNORE_ORPHANS=True
docker-compose up -d
cmd:
SET COMPOSE_IGNORE_ORPHANS=True&& docker-compose up -d
powershell:
$env:COMPOSE_IGNORE_ORPHANS = 'True'; & docker-compose up -d
TL;DR
You can also add a unique name: myproject to each of your compose files.
My journey
In case this helps anybody else scrounging around to find help for the above issue (This is in support of the already good comments here):
I have several config files in the same directory
redis.yml
mariadb.yml
...
and I kept getting the same error about orphan containers when I ran
docker-compose -f <one of my configs>.yml up
as of now you can simply put each yml file into a separate project. This is simply done using the command like parameter "-p my_project_name" as has already been mentioned before. BUT the name must be in all lowercase!
This got me a little closer but I also kept forgetting that to bring the docker container down using docker-compose I needed to include that parameter as well.
For example to start the container:
docker-compose -p myproject-d redis.yml up -d
and to destroy the container
docker-compose -p myproject-d redis.yml down
Today I found that I can simply add the name: bit into the yml config. Here is an example for redis:
version: '3.9'
name: redis
services:
redis_0:
...
Now I can simply start the container with the following and don't have to worry about project names again:
docker-compose -f redis.yml <up/down>
This happens when your docker-compose file has got updated. I received similar error on Docker startup and found out that another team member updated the docker-compose.yml as part of cleanup.
To fix this, I deleted the docker group using the Delete button in Docker Desktop and started it again. This fixed the error for me.
As a complement for the existing answers, if you're using docker-compose with the -f option, to my surprise docker-compose will use the name of the parent folder of the first file passed via -f as the project name.
For example, assuming the following folder structure:
/
└── Users/
└── papb/
├── a.yml
└── foo/
└── b.yml
If you're in /Users and run docker-compose -f papb/a.yml -f papb/foo/b.yml:
The project name will be inferred as papb
Any relative paths you have in both files will be resolved against /Users/papb
If you're in /Users and run docker-compose -f papb/foo/b.yml -f papb/a.yml:
The project name will be inferred as foo
Any relative paths you have in both files will be resolved against /Users/papb/foo
If you're in /Users/papb and run docker-compose -f foo/b.yml -f a.yml:
The project name will be inferred as foo
Any relative paths you have in both files will be resolved against /Users/papb/foo

Resources