I'm learning Docker, and I have question about environments.
Project structure:
nginx (Dockerfile)
nuxt.js + express.js (Dockerfile)
laravel (Dockerfile)
mysql (Dockerfile)
docker compose
I need to save all settings variables in root folder with docker-compose. But laravel needs .env file inside their root dir.
So my question is: Is there some path to store all settings in one .env file in the root directory, and when docker-compose run, it give that setting to all services?
In your docker-compose.yml, you can either declare a volume mount for a directory containing your .env file. Then read from that directory in your laravel application for example.
Or you can choose to use the env_file configuration option to give the running process access to those variables.
services:
foo:
image: foo
volumes:
- ./env:/app/env
bar:
image: bar
env_file: ./env/.env
Read more here
Related
If I want to provide custom value for ASPNETCORE_ENVIRONMENT to docker-compose up cli command how would I do that?
Is it possible to provide this through cli or I need to use docker-compose.yml file (this solution I would like to skip).
Unlike docker-compose run, docker-compose up does not provide this - unfortunately. Alternatively, you can make use of env_file for this:
services:
webapi:
image: aspnetcore-image
env_file:
- .env
By creating a file named .env (it can be named as desired) and placing it in the appropriate directory (here it is placed in the same directory as docker-compose.yml), docker-compose up will then export the respective env vars to the docker container.
I am trying to use the same docker-compose.yml and .env files for both docker-compose and swarm. The variables from the .env file should get parsed, via sed, into a config file by running a run.sh script at boot. This setup works fine when using the docker-compose up command, but they're not getting passed when I use the docker stack deploy command.
How can I pass the variables into the container so that the run.sh script will parse them at boot?
Loading the .env file is a feature of docker-compose that is not part of the docker CLI. You can manually load the contents of this file in your shell before performing the deploy:
set -a; . ./.env; set +a
docker stack deploy -c docker-compose.yml stack_name
Other options include using docker-compose to pre process the compose file:
docker-compose config >docker-compose.processed.yml
Or you could use envsubst to replace the variables to make a compose file with the variables already expanded:
set -a; . ./.env; set +a
envsubst <docker-compose.yml >docker-compose.processed.yml
To pass shell environment variables through to containers use env_file syntax:
web:
env_file:
- web-variables.env
As docs state:
You can pass multiple environment variables from an external file through to a service’s containers with the ‘env_file’ option
However, using .env as external filename may cause unexpected results and is semantically problematic.
Placing .env in the folder where the docker-compose command serves different purpose:
As Docs, Docs2, Docs3 state:
The environment variables you define here are used for variable
substitution in your Compose file
You can set default values for environment variables using a .env
file, which Compose automatically looks for
So if compose file contains:
db:
image: "postgres:${POSTGRES_VERSION}"
You .env would contain:
POSTGRES_VERSION=4.0
This feature indeed works only in compose:
The .env file feature only works when you use the docker-compose up
command and does not work with docker stack deploy
Actually I found the best/easiest way is to just add this argument to the docker-compose.yml file:
env_file:
- .env
I have a docker-compose.yaml file and then dockerfiles. Now, I don't want to write all the host, port, password in this docker-compose/dockerfile. Rather, I would like to keep all these configurations in a different file. I know, half of the requirement could be achieved by using .env file, where I can keep my environment variables, but then it will be good to keep all configurations in a single file (main.cfg), which could used by other components.
You can use volumes to share a file or a entire folder to a docker service. Here is an example of a mysql which shares a database to docker:
version: '3'
services:
mysql-database:
image: mysql
container_name: mysql-database
volumes:
- ./mysql/docker-entrypoint-initdb.d/:/docker-entrypoint-initdb.d
env_file:
- .env
ports:
- 3306:3306
The directory structure is as follows:
BaseDirectory
|____docker-compose.yml
| mysql
| |____docker-entrypoint-initdb.d
| |____sqldb1.gz
| |____sqldb2.gz
In above example, all files inside docker-entrypoint-initdb.d from host, are shared to the docker container, and can be accessed from inside the container also at /docker-entrypoint-initdb.d. You can mount individual files also by giving exact filename. For example:
volumes:
- ./mysql/docker-entrypoint-initdb.d/sqldb1.gz:/docker-entrypoint-initdb.d/sqldb1.gz
As I have shared a database, similarly you can share a config file and place it at the exact location you want inside the docker.
I've nignx container and one asset container which have all my assets build from grunt or some other tools.
Now in docker compose file, i want to mount asset container's 's folder path into nginx container so nginx can serve that files.
How can we do that? i don't remember but i think there is a option where we can share path of one container with another.
Suppose if i scale up nginx to 2 container then will that mount works for all instance of nginx?
if i scale up asset container then what will happen?
i also want to mount that with my host so development can be done be easily.
What you want to do is use a volume, and then mount that volume into whatever containers you want it to appear in.
Completely within Docker
You can do this completely inside of Docker.
Here is an example (stripped-down - your real file would have much more than this in it, of course).
version: '3'
services:
nginx:
volumes:
- asset-volume:/var/lib/assets
asset:
volumes:
- asset-volume:/var/lib/assets
volumes:
asset-volume:
At the bottom is a single volume defined, named "asset-volume".
Then in each of your services, you tell Docker to mount that volume at a certain path. I show example paths inside the container, just adjust these to be whatever path you wish them to be in the container.
The volume is an independent entity not owned by any particular container. It is just mounted into each of them, and is shared. If one container modifies the contents, then they all see the changes.
Note that if you prefer only one can make changes, you can always mount the volume as read-only in some services, by adding :ro to the end of the volume string.
services:
servicename:
volumes:
- asset-volume:/var/lib/assets:ro
Using a host directory
Alternately you can use a directory on the host and mount that into the containers. This has the advantage of you being able to work directly on the files using your tools outside of Docker (such as your GUI text editor and other tools).
It's the same, except you don't define a volume in Docker, instead mounting the external directory.
version: '3'
services:
nginx:
volumes:
- ./assets:/var/lib/assets
asset:
volumes:
- ./assets:/var/lib/assets
In this example, the local directory "assets" is mounted into both containers using the relative path ./assets.
Using both depending on environment
You can also set it up for a different dev and production environment. Put everything in docker-compose.yml except the volume mounts. Then make two more files.
docker-compose.dev.yml
docker-compose.prod.yml
In these files put only the minimum config to define the volume mount. We'll mix this with the docker-compose.yml to get a final config.
Then use this. It will use the config from docker-compose.yml, and use anything in the second file as an override or supplemental config.
docker-compose -f docker-compose.yml \
-f docker-compose.dev.yml \
up -d
And for production, just use the prod file instead of the dev file.
The idea here is to keep most of the config in docker-compose.yml, and only the minimum set of differences in the alternative files.
Example:
docker-compose.prod.yml
version: '3'
services:
nginx:
volumes:
- asset-volume:/var/lib/assets
docker-compose.dev.yml
version: '3'
services:
nginx:
volumes:
- ./assets:/var/lib/assets
We have a requirement of creating multiple isolated envs of our app in a single host, using a single compose file.
I realized that by specifying the project name using -p option we can create multiple isolated envs using docker compose in a single host.
However is it possible to override the ports: and volumes: in the compose file for different environment, without having 2 seperate docker compose files?
For instance, I would like to override the following properties, preferably through command-line args.
For dev environment
ports:
8081:8080
volumes:
/etc/myapp/dev/properties/:/etc/myapp/properties
For QA environment
ports:
8082:8080
volumes:
/etc/myapp/qa/properties/:/etc/myapp/properties
You can use a template.yml and pass the variables you want to generate docker-compose.yml
First, create a template.yml with the following content:
version: "2"
...
ports:
"$HOST_PORT":8080
volumes:
"$HOST_VOLUME":/etc/myapp/properties
Now, you can create a script with the variables you want by environment. For dev environment it would look like this:
#!/bin/bash
# Variables to use in template.yml
export HOST_PORT="8081"
export HOST_VOLUME="/etc/myapp/dev/properties/"
# build docker-compose.yml from the template
source env.sh; rm -rf docker-compose.yml; envsubst < "template.yml" > "docker-compose.yml";
This will generate a docker-compose.yml with the concrete values.
Here's an usage example: https://github.com/bsferreira/mysql-fabric
Needed this as well and stumbled upon this question. There is built in support for isolating environments:
Multiple isolated environments on a single host
Compose uses a project name to isolate environments from each other. You can make use of this project name in several different contexts:
* on a dev host, to create multiple copies of a single environment, such as when you want to run a stable copy for each feature branch of a project
* on a CI server, to keep builds from interfering with each other, you can set the project name to a unique build number
* on a shared host or dev host, to prevent different projects, which may use the same service names, from interfering with each other
The default project name is the basename of the project directory. You can set a custom project name by using the -p command line option or the COMPOSE_PROJECT_NAME environment variable.
The default project directory is the base directory of the Compose file. A custom value for it can be defined with the --project-directory command line option.
https://docs.docker.com/compose/#multiple-isolated-environments-on-a-single-host
You can use Variable substitution with Declare default environment variables in file.
For example, my compose project structure is
C:\docker-compose
└───multiple-envs
│ .env
│ .env-dev
│ docker-compose.yml
│
├───dev-files
└───files
docker-compose.yml, file content below, you can set ports and other values using ${ENV_VAR} syntax. It will automatically substitute by docker compose cli.
You can instruct docker compose to show error message when the ${ENV_VAR} not set or empty using syntax ${HOST_MYSQL_PORT:?HOST_MYSQL_PORT is not set}, after :? is the error message.
services:
mysqldb:
image: mysql
restart: always
ports:
- ${HOST_MYSQL_PORT:?HOST_MYSQL_PORT is not set}:3306
volumes:
- type: bind
source: ${VOLUMES_SOURCE:?VOLUMES_SOURCE is not set}
target: /mnt
read_only: true
environment:
MYSQL_ROOT_PASSWORD: ${MYSQL_ROOT_PASSWORD:?MYSQL_ROOT_PASSWORD is not set}
MYSQL_USER: ${MYSQL_USER:?MYSQL_USER is not set}
MYSQL_PASSWORD: ${MYSQL_PASSWORD:?MYSQL_PASSWORD is not set}
.env file, set ENV_VAR=VAL in file called .env, let's say for production.
HOST_MYSQL_PORT=13306
VOLUMES_SOURCE=./files
MYSQL_ROOT_PASSWORD=p419460507
MYSQL_USER=u130085860
MYSQL_PASSWORD=p689273542
.env-dev file, set ENV_VAR=VAL in file called .env-dev, let's say for development.
HOST_MYSQL_PORT=23306
VOLUMES_SOURCE=./dev-files
MYSQL_ROOT_PASSWORD=dev419460507
MYSQL_USER=dev130085860
MYSQL_PASSWORD=dev689273542
To compose in different config, specify environment file when invoke docker compose cli, --env-file multiple-envs\.env for production or --env-file multiple-envs\.env-dev for development.
C:\docker-compose> docker-compose --project-directory multiple-envs --env-file multiple-envs\.env up --detach
[+] Running 2/2
- Network multiple-envs_default Created 0.6s
- Container multiple-envs-mysqldb-1 Started 1.8s
C:\docker-compose> docker-compose --project-directory multiple-envs --env-file multiple-envs\.env-dev up --detach
[+] Running 1/1
- Container multiple-envs-mysqldb-1 Started 11.7s