When I run docker-compose up -d, docker always creates container name and network name prepended with the folder name that contains docker-compose.ymlfile.
I can specify the container name as follows:
nginx:
container_name: nginx
build:
context: .
dockerfile: .docker/docker-nginx.dockerfile
But how can I specify the network name so that it doesn't prepend folder name to it?
Thanks.
Docker Prepends the current folder name with all the components name created using docker compose file.
Eg : If the current folder name containing the docker-compose.yml file is test, all the volumes,network and container names will get test appended to it. In order to solve the problem people earlier proposed the idea of using -p flag with docker-compose command but the solution is not the most feasible one as a project name is required just after the -p attribute. The project name then gets appended to all the components created using docker compose file.
The Solution to the above problem is using the name property as in below.
volumes:
data:
driver: local
name: mongodata
networks:
internal-network:
driver: bridge
name: frontend-network
This volume can be referred in the service section as
services:
mongo-database:
volumes:
- data:/data/db
networks:
- internal-network
The above name attribute will prevent docker-compose to prepend folder name.
Note : For the container name one could use the property container_name
services:
mongo-database:
container_name: mongo
You have to place a ".env" file in the root of your docker-compose project directory.
COMPOSE_PROJECT_NAME=MyFancyProject
See Docker docs for further information: https://docs.docker.com/compose/reference/envvars/
If you don't use docker-compose, you can use the "-p" parameter to set this on docker run.
Related
I am writing a docker-compose.yaml file for my project. I have checked the volumes documentation here .
I also understand the concept of volume in docker that I can mount a volume e.g. -v my-data/:/var/lib/db where my-data/ is a directory on my host machine while /var/lib/db is the path inside database container.
My confuse is with the link I put above. There it has the following sample:
version: "3.9"
services:
db:
image: db
volumes:
- data-volume:/var/lib/db
backup:
image: backup-service
volumes:
- data-volume:/var/lib/backup/data
volumes:
data-volume:
I wonder does it mean that I have to create a directory named data-volume on my host machine? What if I have a directory on my machine with path temp/my-data/ and I want to mount that path to the database container /var/lib/db ? Should I do something like below?
version: "3.9"
services:
db:
image: db
volumes:
- temp/my-data/:/var/lib/db
volumes:
temp/my-data/:
My main confusion is the volumes: section at the bottom, I am not sure whether the volume name should be the path of my directory or should be just literally a name I give & if it is the latter case then how could the given name be mapped with temp/my-data/ on my machine? The sample doesn't indicate that & is ambiguous to clarify that.
Could someone please clarify it for me?
P.S. I tried with above docker-compose I guessed, ended up with the error:
ERROR: The Compose file './docker-compose.yaml' is invalid because:
volumes value 'temp/my-data/' does not match any of the regexes: '^[a-zA-Z0-9._-]+$'
Mapped volumes can either be files/directories on the host machine (sometimes called bind mounts in the documentation) or they can be docker volumes that can be managed using docker volume commands.
The volumes: section in a docker-compose file specify docker volumes, i.e. not files/directories. The first docker-compose in your post uses such a volume.
If you want to map a file or directory (like in your last docker-compose file), you don't need to specify anything in the volumes: section.
Docker volumes (the ones specified in the volumes: section or created using docker volume create) are of course also stored somewhere on your host computer, but docker manages that and you shouldn't normally need to know where or what the format is.
This part of the documentation is pretty good about explaining it, I think https://docs.docker.com/storage/volumes/
As #HansKilian mentions, you don't need both volumes and services.volumes. To use services.volumes, map the host directory to the container directory like this:
services:
db:
image: db
volumes:
- /host/path/lib/db:/container/path/lib/db
With that, the directory /host/path/lib/db on the host machine will be used by the container and available at /container/path/lib/db.
Now, if you're like me, I get really confused with fake examples, so let's say the real directory on your host machine is /var/lib/db and you just want to see it at /db when you run a shell in Docker (i.e., docker exec -it /bin/bash container-id).
docker-compose.yaml would look like this:
services:
db:
image: db
volumes:
- /var/lib/db:/db
Now when you run the shell, cd /logs and ls, you'll see the same results as if you'd cd /var/lib/db on the host.
If you want to use the volumes section to indicate a global volume to use, you first have to create that volume using docker volume create. The documentation Hans linked includes steps to do this. The syntax of /host/path:/container/path is replaced by volume-name:/container/path. Then, once defined, you'd alter your docker-compose.yaml to be more like this:
services:
db:
image: db
volumes:
- your-global-volume-name:/db
volumes:
your-global-volume-name:
external: true
Note that I have not tested or used the this configuration. I'm assuming it's correct based on the other method working and the few changes I can identify in the docs.
Here is compose file with config of container that I wish to connect to from external container (defined in another compose file):
version: '3.5'
services:
service-to-connect-to:
build: .
networks:
- my-external-network
networks:
my-external-network:
external: true
and another compose file that contains config for container from which I wish to connect to service-to-connect-to:
version: "3.5"
services:
service-to-connect-from:
build: .
I tried to connect to service-to-connect-to via this domains:
service-to-connect-to
service-to-connect-to.my-external-network
my_external_network.service-to-connect-to
but nothing of them worked.
Where I'm wrong?
Thanks
First, you have to add both services to same network in order to connect them. So, the latter compose file should be something like
version: "3.5"
services:
service-to-connect-from:
build .
networks:
- my-external-network
networks:
my-external-network:
external: true
Now that both services are on the same network they can find each other using container's name. Container name is by default same as the service name BUT docker compose also prefixes it by project name which is by default the directory name where the compose file exists. You can see this if you first start the services by docker-compose up -d and then see how the containers get named by running docker ps. The container name could be for example project1_service-to-connect-to. With this name you can connect from another service.
If you like to, you can also set the container's name explicitly using container_name option for service. When used, compose doesn't prefix the container name anymore.
This is not possible with compose, because the second docker-compose file will create a different network with the current directory name and then your both the services will be in two different docker network and will not be able talk to each other.
So you can do two things:
1. either use same network for both the services (then its better to have one combined compose file).
2. use docker swarm stacks, where you can connect to other services with <service_name>.<stack_name>
References:
https://docs.docker.com/docker-cloud/apps/service-links/#using-service-and-container-names-as-hostnames
https://docs.docker.com/v17.09/engine/userguide/networking/configure-dns/
Note: Even service links does not help, because compose creates different network while running the second file.
I am new to Docker and have docker-compose.yml which is containing many services and iI need to start one particular service. I have docker-compose.yml file with information:
version: '2'
services:
postgres:
image: ${ARTIFACTORY_URL}/datahub/postgres:${BUILD_NUMBER}
restart: "no"
volumes:
- /etc/passwd:/etc/passwd
volumes_from:
- libs
depends_on:
- libs
setup:
image: ${ARTIFACTORY_URL}/setup:${B_N}
restart: "no"
volumes:
- ${HOME}:/usr/local/
I am able to call docker-compose.yml file using command:
docker-compose -f docker-compose.yml up -d --no-build
But I need to start "setup service" in docker-compose file:
How can I do this?
It's very easy:
docker compose up <service-name>
In your case:
docker compose -f docker-compose.yml up setup -d
To stop the service, then you don't need to specify the service name:
docker compose down
will do.
Little side note: if you are in the directory where the docker-compose.yml file is located, then docker-compose will use it implicitly, there's no need to add it as a parameter.
You need to provide it in the following situations:
the file is not in your current directory
the file name is different from the default one, eg. myconfig.yml
As far as I understand your question, you have multiple services in docker-compose but want to deploy only one.
docker-compose should be used for multi-container Docker applications. From official docs :
Compose is a tool for defining and running multi-container Docker
applications.
IMHO, you should run your service image separately with docker run command.
PS: If you are asking about recreating only the container whose image is changed among the multiple services in your docker-compose file, then docker-compose handles that for you.
I'm using Docker Toolbox on Windows 10
I can access the php part succesfully via http://192.168.99.100:8000, I have been working around with the mariadb part but still having several problems
I have an sql file as /mariadb/initdb/abc.sql so I should be copied into /docker-entrypoint-initdb.d, after the container is created I use docker-compose exec mariadb to access the container, there is the file as /docker-entrypoint-initdb.d/abc.sql but the file never get executed, I also have tested to import the sql file to the container manually, it was succesful so the sql file is valid
I don't quite understand about the data folder mapping, and what to do to get the folder sync with the container, I always get the warning when recreate the container using docker-compose up -d
WARNING: Service "mariadb" is using volume "/var/lib/mysql" from the previous container. Host mapping "/.../mariadb/data" has no effect. Remove the existing containers (with docker-compose rm mariadb) to use the Recreating db ... done
Questions
How to get the sql file in /docker-entrypoint-initdb.d to be executed ?
What is the right way to map the data folder with the mariadb container ?
Please guide
Thanks
This is my docker-compose.yml
version: "3.2"
services:
php:
image: php:7.1-apache
container_name: web
restart: always
volumes:
- /.../php:/var/www/html
ports:
- "8000:80"
mariadb:
image: mariadb:latest
container_name: db
restart: always
environment:
- MYSQL_ROOT_PASSWORD=12345
volumes:
- /.../mariadb/initdb:/docker-entrypoint-initdb.d
- /.../mariadb/data:/var/lib/mysql
ports:
- "3306:3306"
For me the issue was the fact that Docker didn't clean up my mounted volumes from previous runs.
Doing a:
docker volume ls
Will list any volumes, and if previous exist, then run 'rm' command on the volume to remove it.
As stated on docker mysql docks, scripts in the '/docker-entrypoint-initdb.d' folder is only evalutated the first time the container runs, and if a previous volume remains, it won't run the scripts.
As for the mapping, you simply need to mount your script folder to the '/docker-entrypoint-initdb.d' folder in the image:
volumes:
- ./db/:/docker-entrypoint-initdb.d
I have a single script file in a folder named db, relative to my docker-compose file.
In your Docker file for creating mariaDB, at the end add the abc.sql file to your docker entry point like so:
COPY abc.sql /docker-entrypoint-initdb.d/
Remove the - /.../mariadb/initdb:/docker-entrypoint-initdb.d mapping as any file copied into the entry point will be executed.
Note: Windows containers do not execute anything in docker-entrypoint-initdb.d/
I have Docker commands to create a container and then use that container's name with --volumes-from to run another container and it works fine -
docker create -v /home/dev/docker/my/config:/home/myuser/4.0/config --name shared-config my/configurator:4.0.0
The above would create a new container by name shared-config from image my/configurator:4.0.0
and when trying to run any other container (say my/oms:4.0.0) I can simply use volume from container named shared-config using --volumes-from
docker run --volumes-from shared-config -p 8083:8080 -d my/oms:4.0.0
using --volumes-from we can use the volume multiple times in which ever container it is required.
Till here everything seems fine.
Now, I am trying to do the above in docker-compose using file-format version "3" and not able to understand how will I be able to re-use data-volume once it is created. Since as per docker-compose in version 3 they have discontinued use of --volumes-from.
They say -
To share a volume between services, define it using the top-level volumes option and reference it from each service that shares it using the service-level volumes option.
In above statement they are referring to named volumes, please refer from here.
But I just want to mount a host directory as a data volume and re-use that data volume. My question is how do I reuse this data-volume through docker-compose file version "3".
To the simplest, for each service I want to run through docker-compose I can use volume key at service level
version: "3"
services:
my-oms:
image: my/oms:4.0.0
ports:
- "8083:8080"
volumes:
- /home/dev/docker/my/config:/home/myuser/4.0/config
But what if I want to use my host's directory (/home/dev/docker/my/config) as a data volume in different services. Should I have the volume key for each service or actually there is a better way in docker-compose version "3" where I can re-use the data-volume in other services (how we did using --volumes-from).
Any pointers or suggestions or something that I missed?
The best option to avoid repeating syntax is to extend your docker-compose.yml using the extends option:
So you can have a common-services.yml that looks like:
version: "3"
services:
generic-vol:
volumes:
- /home/dev/docker/my/config:/home/myuser/4.0/config
And then your docker-compose.yml gets updated to look like:
version: "3"
services:
my-oms:
extends:
file: common-services.yml
service: generic-vol
image: my/oms:4.0.0
ports:
- "8083:8080"
Note that docker stack deploy -c docker-compose.yml may not support all these options, I've encountered issues using variables and multiple docker-compose files for my project. The solution to that is to use docker-compose to parse the file into something the stack deploy can use with docker-compose config >docker-compose.stack.yml and then pass that yml file to your stack deploy.
A second option is to utilize the features of the yml syntax itself. It allows anchors and references to those anchors. That syntax looks like:
version: "3"
services:
my-oms:
image: my/oms:4.0.0
ports:
- "8083:8080"
volumes: &common-vol
- /home/dev/docker/my/config:/home/myuser/4.0/config
my-xyz:
image: my/xyz:4.0.0
ports:
- "8888:8080"
volumes: *common-vol
The first &common-vol creates an anchor, and the later *common-vol is a reference to that same part of yml data.