How to manage environment variables in Rancher - docker

I want manage environment variables in a stack, then the service can use it. For example:
I definition a evn tracker_ip=192.168.0.101, then I want use it in service create.
what should I do

There can be several answers depending on what you are trying to do and how you are deploying your stack.
Using the CLI / Rancher Compose
If you are using the command line, you can just use variable interpolation. Instructions on how to do so can be found in the official documentation:
https://docs.rancher.com/rancher/v1.5/en/cli/variable-interpolation/
Using the Rancher UI / Catalogs
If you want to do it through the Rancher UI, you can do it by creating a template in a catalog and having questions to input your environment variables. More details on how to do so here:
https://docs.rancher.com/rancher/v1.5/en/catalog/private-catalog/
You can define questions in the rancher-compose.yml file like this:
version: '2'
catalog:
name: My Application
version: v0.0.1
questions:
- variable: TRACKER_IP
label: Tracker IP address
required: true
default: 192.168.0.101
type: string
You can then push the answers to the environment section of your docker-compose.yml template for use within your image:
version: '2'
services:
web:
image: myimage
ports:
- 8000
environment:
TRACKER_IP: ${TRACKER_IP}

There is no way to do exactly what you're asking, because that would allow editing of the variables of running containers and containers are immutable. Environment variables can be defined on services but not defined once on stacks and made available to all services.
Secrets are somewhat like this and can be shared across services, but not edited.

Depending on where the tracker_ip is associated, you could also create an external service as part of the stack. The external service essentially just creates a DNS entry in Rancher. So you could then just link your service to the external_tracker service in compose and refer to tracker.
version: '2'
services:
myservice:
...
link:
- tracker_service:tracker
...

Related

docker-compose interpolate environment variables: use default variables provided by docker-compose

I need some help with the following template:
services:
nginx:
image: nginx
restart: unless-stopped
labels:
- "traefik.enable=true"
- "traefik.http.routers.nginx-${COMPOSE_PROJECT_NAME}.rule=Host(`fuu.bar`)"
networks:
- treafik
My goal is to create a template which I can use e. g. in portainer with almost zero configuration.
I thought that the following variables are available in docker-compose config but the expression ${COMPOSE_PROJECT_NAME} results in an empty string: docker-compose config
services:
nginx:
image: nginx
restart: unless-stopped
labels:
- "traefik.enable=true"
- "traefik.http.routers.nginx-.rule=Host(`fuu.bar`)"
networks:
- treafik
Are there any default environment variables provided by docker-compose which I can use for environment interpolation?
---- Update
I use traefik (v2) as a reverse proxy. To make the containers available through treafik, you need to define routers on every service. The router name has to be unique. Lets imagine you deploy 2 or more stacks of the above template. The router name has to be unique for all services across all stacks. Because Im a lazy guy, I tried to simply integrate the environment variable COMPOSE_PROJECT_NAME (which I know is already unique in my setup because every stack must have a unique name). But the variable is not available when deploying the stack.
Of course, I could simply define the variable COMPOSE_PROJECT_NAME by myself in a .env-file, but i hoped that there are any default environment variables provided by docker.
You can use environment variables to passing strings to your docker file.
There are many ways through docker documentation. For example:
You can set default values for any environment variables referenced in the Compose file, or used to configure Compose, in an environment file named .env. The .env file path is as follows:
Starting with +v1.28, .env file is placed at the base of the project
directory
Project directory can be explicitly defined with the --file option or
COMPOSE_FILE environment variable. Otherwise, it is the current
working directory where the docker compose command is executed
(+1.28).
For previous versions, it might have trouble resolving .env file with
--file or COMPOSE_FILE. To work around it, it is recommended to use --project-directory, which overrides the path for the .env file. This inconsistency is addressed in +v1.28 by limiting the filepath to the
project directory.

Docker-compose: Replacement for the extends keyword

From the docker docs:
Docker Compose’s extends keyword enables sharing of common
configurations among different files, or even different projects
entirely. Extending services is useful if you have several services
that reuse a common set of configuration options. Using extends you
can define a common set of service options in one place and refer to
it from anywhere.
For some reason this feature was removed in version 3.
Found also this thread, but it is inactive for 2 years.
I'm trying to find a replacement for this feature in the newer versions.
Would like to hear if somebody found a replacement for extends.
Thanks.
There are 2 ways to achieve what you need, you can decide to use one of them or both at the same time as they work slightly differently:
Multiple compose files
You can specify multiple compose files when running a docker compose command, you could for instance set up your project with:
docker-compose -f config1.yml -f config2.yml up
You could also use an environment variable to specify your files:
COMPOSE_FILE=config1.yml:config2.yml docker-compose up
What happens is that docker compose creates a single config merging what you defined in each of them.
Here the documentation showing how to merge multiple compose files.
You can also generate your final config file running the config command.
YAML Anchors
Since docker compose files are basically YAML files, you can take advantage of YAML Anchors to define a block of properties and reuse them in multiple parts of your config.
For example:
version: '3'
common: &common
image: "myrepo/myimage"
restart: "unless-stopped"
volumes:
- "volume:/mnt/myvolume"
services:
service1:
<<: *common
ports:
- "5000:5000"
service2:
<<: *common
environment:
- MYENV: value

How to override docker-compose values in multiple combined files?

Letzt imagine i have 3 compose files (only focus on the mysql service)
docker-compose.yml
docker-compose.staging.yml
docker-compose.prod.yml
In my docker compose.yml i have my basic mysql stuff with dev als build target
version: "3.4"
services:
mysql:
build:
target: dev
...
And start it with
docker-compose up -d
In my staging environment i would like to expose port 3306, but also want another build target so i would create the docker-compose.staging.yml with the following content.
version: "3.4"
services:
mysql:
build
target: prod
ports:
- 3306:3306
And combine it with
docker-compose -f docker-compose.yml -f docker-compose.staging.yml up -d
So the build target is overwritten and the port 3306 is now exposed to the outside.
Now i want the same in the docker-compose.prod.yml, just without having the port 3306 exposed to the outside ... How can i override the ports directive to not having ports exposed?
I tried to put an empty array in the prod.yml without success (port is still exposed):
version: "3.4"
services:
mysql:
ports: []
In the end i would like to stack the up command like this:
docker-compose -f docker-compose.yml -f docker-compose.staging.yml -f docker-compose.prod.yml up -d
I also know the docs says
For the multi-value options ports, expose, external_links, dns, dns_search, and tmpfs, Compose concatenates both sets of values
But how can i reach my goal anyway without duplicating configuration?
Yes for sure, i could omit the docker-compose.staging.yml but in the staging.yml are build steps defined, which should also be used for the prod stage to not have any differences between the built container.
So duplicating things isn't really an option.
Thanks
I would actually strongly suggest just not using the "target" command in your compose files. I find it to be extremely beneficial to build a single image for local/staging/production - build once, test it, and deploy it in each environment. In this case, you change things using environment variables or mounted secrets/config files.
Further, using compose to build the images is... fragile. I would recommend building the images in a CI system, pushing them to a registry, and then using the image version tags in your compose file- it is a much more reproducible system.
You might consider using extends key in your compose files like this:
mysql:
extends:
file: docker-compose.yml
service: mysql
ports:
- 3306:3306
# other definitions
Although you'd have to change your compose version from 3.4 to < 3 ( like 2.3 ) because v3 doesn't support this feature ref as there is a open feature request hanging for a long time now.
Important note here is that you shouldn't expose any ports in your base docker-compose.yml file, only on the specific composes.
Oficial docs ref for extends
edit
target clause is not supported in v2.0 so I've adjusted the answer to match the extends and target requirement. That's compose v2.3.
edit from comments
As there is a deploy keyword requirement, then there is compose v3 requirement. And as for now, there is no possibility to extend composes. I've read in some official doc (can't find it now for ref) that they encourage us to use flat composes specific for environment so that it's always clear. Also Docker states that's hard to implement in v3 (ref in the above issue) and it's not going to be implemented anywhere soon. You have to use separate compose files per environment.

How to set environment variable into docker container using docker-compose

I want to set credentials to use Google Translate Api Client so I have to set environment variable GOOGLE_APPLICATION_CREDENTIALS that value is path to credential file (from Google Cloud).
When I have been used docker build and docker run it was pretty easy.
I have been used docker run
--env GOOGLE_APPLICATION_CREDENTIALS=/usr/src/app/CryptoTraderBot-901d31d199ce.json and environment variable has been set.
More difficult things come when I tried to set it in docker-compose. I have to use docker-compose because I need few containers so it is only way to achieve this.
Based on Docker compose environment variables documentation I created my docker-compose.yml file that looks like this:
version: "3"
services:
redis:
image: redis:4-alpine
crypto-bot:
build: .
depends_on:
- redis
environment:
- GOOGLE_APPLICATION_CREDENTIALS = /usr/src/app/CryptoTraderBot-901d31d199ce.json
I also have been tried multiple combination of path to .json file but none of this has been worked properly.
Have you got any idea how can I set it properly ?
While creating this question I have been resolve this problem in a funny and easy way but I have been thought that I post answer to help someone in the future with similiar problem.
All you have to do is remove " " (space) next = sign so two last lines of docker-compose.yml should looks like this:
environment:
- GOOGLE_APPLICATION_CREDENTIALS=/usr/src/app/CryptoTraderBot-901d31d199ce.json
Docker Compose has a newer feature called secrets. You can bind the credentials like this:
services:
secret-service:
build:
context: secret-service
environment:
- GOOGLE_APPLICATION_CREDENTIALS=/run/secrets/gcp-credentials
secrets:
- gcp-credentials
secrets:
gcp-credentials:
file: ./gcp-credentials.json
Reference: https://docs.docker.com/compose/compose-file/#secrets

Stack service containers dynamic hostname

I am migrating away from Docker Cloud to pure Docker Swarm setup. One thing that I am missing is the nice way of how the containers got the hostname set as $SERVICE_NAME-$SLOT_NUMBER.
Is it possible to dynamically set the hostname/container name in a swarm stack service?
In your stack/compose file, use this format:
services:
thaservice:
...
hostname: "{{.Service.Name}}-{{.Task.Slot}}"
Link to documentation for available template variables: https://docs.docker.com/engine/swarm/services/#create-services-using-templates

Resources