I have an existing docker container with the name foo_db_1.
The docker-compose file is located in the foo directory.
The container contains a huge DB.
I want to rename the directory from foo to bar
So I stopped the running container and renamed the directory and the container name
mv foo bar
docker rename foo_db_1 bar_db_1
Now when I try to start the container again I get a Conflict error:
docker-compose up -d
ERROR: for bar_db_1 Cannot create container for service db: Conflict. The container name "/bar_db_1" is already in use by container "4a9204e8031d72327be283967d2b9ec3268450140f00254eb2fc51ba8af0eb58". You have to remove (or rename) that container to be able to reuse that name.
When I rename the container, then a completely new container is created and I have to import all DB data again to that container.
When I remove the container, then I will loose the DB data and also have to import all db data again.
When I run docker inspect bar_db_1 I can see, that there are several entries pointing to the foo directory.
So how can I move an existing docker setup to another directory without loosing my existing data?
My docker-compose.yml looks like this:
version: "3.8"
volumes:
db:
services:
db:
image: mariadb:10.4
environment:
MYSQL_DATABASE: 'app'
MYSQL_USER: 'dbuser'
MYSQL_PASSWORD: '1234567890'
MYSQL_ROOT_PASSWORD: '1234567890'
volumes:
- db:/var/lib/mysql
- ./db-dumps:/var/db-dumps
command: ['mysqld', '--character-set-server=utf8mb4', '--collation-server=utf8mb4_unicode_ci']
ports:
- "3306:3306"
I am currently starting with Docker Compose, and I wanted to know how to create multiple containers of the same service 'Redis'. I've already tried with docker-compose up --scale redis=3, but it gives an error, I've been searching through Google the possible solution, but I could not find one.
Thank you in advance.
Here is my docker-compose.yml
version: '3'
services:
redis:
container_name: redis
hostname: redis
image: redis
redis-commander:
container_name: redis-dbms
hostname: redis-commander
image: rediscommander/redis-commander:latest
restart: always
environment:
- REDIS_HOSTS=local:redis:6379
ports:
- "8081:8081"
And here is the error it gives me.
docker-compose up --scale redis=3
Creating network "ex2_default" with the default driver
WARNING: The "redis" service is using the custom container name "redis". Docker requires each container to have a unique name. Remove the custom name to scale the service.
Creating redis-dbms ...
Creating redis ... done
Creating redis ...
Creating redis ...
Creating redis-dbms ... done
ERROR: for redis Cannot create container for service redis: Conflict. The container name "/redis" is already in use by container "d4c93ae4ca68da0b6430e5eddc657d9dda0f40002c7a81c89368535df05eae24". You have to remove (or rename) that container to be able to reuse that name.
ERROR: for redis Cannot create container for service redis: Conflict. The container name "/redis" is already in use by container "d4c93ae4ca68da0b6430e5eddc657d9dda0f40002c7a81c89368535df05eae24". You have to remove (or rename) that container to be able to reuse that name.
The error raise because the reason below:
Link: https://docs.docker.com/compose/compose-file/compose-file-v3/#container_name
Because Docker container names must be unique, you cannot scale a service beyond 1 container if you have specified a custom name. Attempting to do so results in an error.
You can read more add here: https://github.com/docker/compose/issues/3722
for some reason, I need to create the container with the same image, But when I started the second one, It just restarted the fist one's container
the first yml file:
version: "3.1"
services:
php:
image:php:php73-fpm
restart: always
ports:
- "9000:9000"
- "9501:9501"
volumes:
- $PWD/../:/var/www/html/
networks:
- app_net
container_name: php
networks:
app_net:
driver: bridge
the second yml file:
version: "3.1"
services:
php:
image:php:php73-fpm
restart: always
ports:
- "19000:19000"
- "19501:19501"
volumes:
- $PWD/../:/var/www/html/
networks:
- app_net2
container_name: php73
networks:
app_net2:
driver: bridge
when I run docker-compose up -d to start the first one:
$ cd ~/Document/php/work/docker/
$ docker-compose up -d
Creating network "docker_app_net" with driver "bridge"
Creating php ... done
then I switch the directory to the second yml file
$ cd ../../private/docker/
$ docker-compose up -d
Recreating php ... done
Compose has a notion of a project name. By default the project name is the basename of the directory containing the docker-compose.yml file. In your example both directories are named docker (even if they're in different parent directories) so Compose looks for a project named docker and a container named php, and finds a match.
There are four ways to override this:
Rename one of the directories.
Set the COMPOSE_PROJECT_NAME environment variable.
Create a .env file in the current directory, and set COMPOSE_PROJECT_NAME there.
Use the docker-compose -p option (on every docker-compose command).
Within your docker-compose.yml file, the second part of ports: needs to match what the container is listening on; this is allowed to be different from the first part. So use the same 9500/9501 in both files.
Another consequence of the Compose project naming is that the standard names of containers, volumes, and networks that Compose creates will be prefixed with the project name. If the project name (current directory name) is docker2, and you reduce the Compose file to
version: "3.1"
services:
php:
build: .
restart: always
ports:
- "19000:9000"
- "19501:9501"
# no manual container_name: or networks:
The container will be named docker2_php_1, and it will be attached to a network named docker2_default; these will be different from the container/network created in the docker1 project/directory.
You can't have two containers with the same name. Since both names are just php, Docker thought they were settings that were supposed to be merged for the same container. Rename one of them.
I am having two docker-compose files
/cfacing/docker-compose.yml
app-customer-facing:
build: .
depends_on:
- mysql-db
mysql-db:
container_name: staging-mysql-db
image: mysql:5.6
/afacing/docker-compose.yml
app-admin-facing:
build: .
depends_on:
- mysql-db
mysql-db:
container_name: staging-mysql-db
image: mysql:5.6
I want both customer-facing container and admin-facing container to depend on the same mysql-db container. This is currently not working, the app-customer-facing will start with mysql-db but app-admin-facing will not start throwing:
ERROR: for mysql-db Cannot create container for service mysql-db: Conflict. The container name "/staging-mysql-db" is already in use by container "fe63e1ab0c1fd19236551bfc5930544cb31e649a4c18421c05959dc1274eb600". You have to remove (or rename) that container to be able to reuse that name.
The error is that you are duplicating your mysql-db service, you are not re-using it. That's why you get an error telling that there is already a container named staging-mysql-db.
To resolve your use case, you'll have to extend your docker-compose file.
You can see an exemple here. See the block ADMINISTRATIVE TASKS which is basically what you are trying to do.
How to be with orphan images when you have 2 independent projects and you want them to work at the same time or at least to build running docker-compose up -d without --remove-orphans flag when images are already built for another project.
docker compose file1:
version: '2'
services:
applications:
image: tianon/true
volumes:
- ../../:/var/www/vhosts/project1
nginx:
build: ./images/nginx
image: project1/nginx:latest
ports:
- "80:80"
volumes_from:
- applications
networks:
appnet:
aliases:
- project1.app
- admin.project1.app
php:
image: project1/php:latest
ports:
- "7778:7778"
build:
context: ./images/php
dockerfile: Dockerfile
volumes_from:
- applications
networks:
- appnet
mysql:
image: project1/mysql:latest
build: ./images/mysql
environment:
MYSQL_ROOT_PASSWORD: secret
volumes:
- mysqldata:/var/lib/mysql
networks:
- appnet
ports:
- "33066:3306"
workspace:
image: project1/workspace:latest
build:
context: ./images/workspace
volumes_from:
- applications
working_dir: /var/www/vhosts/project1
networks:
- appnet
networks:
appnet:
driver: "bridge"
volumes:
mysqldata:
driver: "local"
the second docker compose file:
version: '2'
services:
project2_applications:
image: tianon/true
volumes:
- ../../:/var/www/vhosts/project2
project2_nginx:
build: ./images/nginx
image: project2/nginx:latest
ports:
- "8080:80"
volumes_from:
- project2_applications
networks:
project2_appnet:
aliases:
- project2.app
- admin.project2.app
project2_php:
image: project2/php:latest
ports:
- "7777:7777"
build:
context: ./images/php
dockerfile: Dockerfile
volumes_from:
- project2_applications
networks:
- project2_appnet
project2_mysql:
image: project2/mysql:latest
build: ./images/mysql
environment:
MYSQL_ROOT_PASSWORD: secret
volumes:
- project2_mysqldata:/var/lib/mysql
networks:
- project2_appnet
ports:
- "33067:3306"
project2_workspace:
image: project2/workspace:latest
build:
context: ./images/workspace
volumes_from:
- project2_applications
working_dir: /var/www/vhosts/videosite
networks:
- project2_appnet
networks:
project2_appnet:
driver: "bridge"
volumes:
project2_mysqldata:
driver: "local"
And now when I have already built project1 and trying to run docker-compose up -d for the second project I see warning:
WARNING: Found orphan containers (docker_workspace_1, docker_nginx_1, docker_php_1, docker_mysql_1, docker_memcached_1) for this project. If you removed or renamed this service in your compose file, you can run this command with the --remove-orphans flag to clean it up.
I have a supposition that it's because container names for project1 should be more specific and I need to add some prefixes like I'm doing for project2, but project1 is in use by many other developers and I do not want to change it.
Is there any way to turn off orphan check?
And the second thing: is just a warning message but for some reason, after it appearing compose is failing with error:
ERROR: Encountered errors while bringing up the project.
And to make it work I need to run docker-compose up -d --remove-orphans
Compose uses the project name (which defaults to the basename of the project directory) internally to isolate projects from each other. The project name is used to create unique identifiers for all of the project's containers and other resources. For example, if your project name is myapp and it includes two services db and web, then Compose starts containers named myapp_db_1 and myapp_web_1 respectively.
You get the "Found orphan containers" warning because docker-compose detects some containers which belong to another project with the same name.
To prevent different projects from interfering with each other (and suppress the warning) you can set a custom project name by using any of the following options:
The -p command line option.
COMPOSE_PROJECT_NAME environment variable. This environment variable can also be set via an environment file (.env in the current working directory by default).
Top-level name element in the Compose file. Note: if you pass multiple files to docker-compose via the -f option, then the value from the last file will be used.
docker-compose takes the name of the directory it is in as the default project name.
You can set a different project name by using -p or --project-name.
https://docs.docker.com/compose/reference/#use--p-to-specify-a-project-name
I had a similar problem because my projects all had the docker/docker-compose.yml structure.
To build on other answers, I create a .env file with my docker compose projects. I have a number of projects that all use the docker directory but are different projects.
To use docker-compose -p is a bit error prone, so creating .env file in the same directory as the docker-compose.yml:
-rw-rw-r-- 1 auser auser 1692 Aug 22 20:34 docker-compose.yml
-rw-rw-r-- 1 auser auser 31 Aug 22 20:44 .env
alleviates the necessary overhead of remembering -p.
In the .env file, I can now set the COMPOSE_PROJECT_NAME variable:
COMPOSE_PROJECT_NAME=myproject
On running:
docker-compose up -d
the COMPOSE_PROJECT_NAME is substituted without the use of -p.
Reference:
https://docs.docker.com/compose/env-file/
docker-compose up --remove-orphans
you can run this command to clean orphan containers. As specified in the warning
If the orphaned containers are expected and not intended to remove, you can set COMPOSE_IGNORE_ORPHANS variable to true.
Consise but just right away working source is here.
One option is to put it as a line into .env file next to docker-compose.yml like this:
COMPOSE_IGNORE_ORPHANS=True
Another option is pass or set it as an environment variable.
sh:
COMPOSE_IGNORE_ORPHANS=True docker-compose up -d
or
export COMPOSE_IGNORE_ORPHANS=True
docker-compose up -d
cmd:
SET COMPOSE_IGNORE_ORPHANS=True&& docker-compose up -d
powershell:
$env:COMPOSE_IGNORE_ORPHANS = 'True'; & docker-compose up -d
TL;DR
You can also add a unique name: myproject to each of your compose files.
My journey
In case this helps anybody else scrounging around to find help for the above issue (This is in support of the already good comments here):
I have several config files in the same directory
redis.yml
mariadb.yml
...
and I kept getting the same error about orphan containers when I ran
docker-compose -f <one of my configs>.yml up
as of now you can simply put each yml file into a separate project. This is simply done using the command like parameter "-p my_project_name" as has already been mentioned before. BUT the name must be in all lowercase!
This got me a little closer but I also kept forgetting that to bring the docker container down using docker-compose I needed to include that parameter as well.
For example to start the container:
docker-compose -p myproject-d redis.yml up -d
and to destroy the container
docker-compose -p myproject-d redis.yml down
Today I found that I can simply add the name: bit into the yml config. Here is an example for redis:
version: '3.9'
name: redis
services:
redis_0:
...
Now I can simply start the container with the following and don't have to worry about project names again:
docker-compose -f redis.yml <up/down>
This happens when your docker-compose file has got updated. I received similar error on Docker startup and found out that another team member updated the docker-compose.yml as part of cleanup.
To fix this, I deleted the docker group using the Delete button in Docker Desktop and started it again. This fixed the error for me.
As a complement for the existing answers, if you're using docker-compose with the -f option, to my surprise docker-compose will use the name of the parent folder of the first file passed via -f as the project name.
For example, assuming the following folder structure:
/
└── Users/
└── papb/
├── a.yml
└── foo/
└── b.yml
If you're in /Users and run docker-compose -f papb/a.yml -f papb/foo/b.yml:
The project name will be inferred as papb
Any relative paths you have in both files will be resolved against /Users/papb
If you're in /Users and run docker-compose -f papb/foo/b.yml -f papb/a.yml:
The project name will be inferred as foo
Any relative paths you have in both files will be resolved against /Users/papb/foo
If you're in /Users/papb and run docker-compose -f foo/b.yml -f a.yml:
The project name will be inferred as foo
Any relative paths you have in both files will be resolved against /Users/papb/foo