It seems that I am not understanding something about variable substitution in the following page (my variable NUM is not registering): https://github.com/compose-spec/compose-spec/blob/master/spec.md#Interpolation
See screenshot below. Running this on mac OSX.
Regarding docker-compose variable substitution, it can depend on how NUM has been set.
set NUM=5 would only set it in the current shell, not for another process.
Make sure to type:
export NUM=5
It is mentioned in the docs:
You can use a $$ (double-dollar sign) when your configuration needs a
literal dollar sign. This also prevents Compose from interpolating a
value, so a $$ allows you to refer to environment variables that you
don’t want processed by Compose.
web:
build: .
command: "$$VAR_NOT_INTERPOLATED_BY_COMPOSE"
If you forget and use a single dollar sign ($), Compose interprets the
value as an environment variable and will warn you:
The VAR_NOT_INTERPOLATED_BY_COMPOSE is not set. Substituting an empty
string.
According to that, line 03 of your compose file should be:
command: echo $$NUM
In addition to $$ solution provided by #ayman-nedjmeddine above you also need to do following, to make shell variables available in compose you have two options
Option 1
log in as root , set your variable and execute docker-compose
root>export NUM=5
root>docker-compose up
Option 2
use sudo -E from user shell, -E will propagate user shell env to sudo,
provide sudo access to docker/docker-compose
add :SETENV: to the command in sudoer file to use -E option in sudo
eg:
sudo visudo -f /etc/sudoers.d/docker-compose
ALL ALL=(ALL:ALL) NOPASSWD:SETENV: /usr/local/bin/docker-compose
sudo visudo -f /etc/sudoers.d/docker
ALL ALL=(ALL:ALL) NOPASSWD:SETENV: /usr/bin/docker
finally use
user1>export NUM=5
user1>sudo -E docker-compose up
Related
I am having an issue passing (or perhaps understanding?) environment variables via the "service -> command" vs using them inside the container's CLI.
This is a pseudo docker-compose.yml
version: "3"
services:
service:
restart: unless-stopped
build:
context: .
dockerfile: Dockerfile
volumes:
- ./config:/config
command: "printenv"
environment:
- REDIS_HOST=some.host
If I run the above with docker-compose up --build, I get the following output as a result of the printenv command for the ENVIRONMENT vars:
service_1 | SHLVL=1
service_1 | OLDPWD=/
service_1 | PATH=/command:/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
service_1 | PWD=/app/src
However, if I remove the command key on the service description and go into the container cli and run
$ printenv
I get:
SHLVL=1
HOME=/root
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
REDIS_HOST=some.host
PWD=/app/src
As you can see, there is an inconsistency between the two.
My "real" objective is to use this ENV vars inside Python scripts (which are also not seeing the ENV vars), so getting the actual var value into the printenv result is the objective, not really "printing" the value.
EDIT:
As requested, the pseudo-Dockerfile is this:
FROM lsiobase/rdesktop-web:alpine AS base
FROM base AS service
CMD printenv
I believe this has to do with lsiobase image more than anything else. If I change the image to something like FROM python:3.9-slim-bullseye AS base things mostly work, but then I cannot add Libreoffice, which is the root need where everything started.
I hope this is clear, thanks for any help.
Best regards,
Rafa.
go into the container cli and run $ printenv
Yes, and when you "go into the container cli", what you are really doing is you are executing a shell, and in that shell you are typing printenv, which makes that shell executes a program printenv with whatever the shell decides that the environment should be. And that shell decided to add HOME variable to the environment of the program. Shell.
Note that there is a difference in startup files between interactive and non-interactive shell. Typically https://www.gnu.org/software/bash/manual/html_node/Bash-Startup-Files.html . If you want to source login interactive startup files, source them yourself command: bash -lc '. ~/.bashrc && printfenv' or similar.
I am trying to pass via the command parameter the value of an environment variable, this does work as I have tested it.
The problem is that the environment variable I really need is something like this
HOST_IP=$(ip route|awk '/default/ { print $3 }')
If I now "docker compose up", I get an error saying " Invalid Template" in the .env file.
I know about how you can map a DNS name to the IP of the host but this doesn't work for me, I need the real IP address so I am passing it in
command: []
As I say, I have done a test via passing the env variable and works great BUT I can't have the variable be the result of running a $() command - this does work in the bash shell directly.
The problem is I need the HOST IP but this is NOT guaranteed to be a specific address.
Can anyone help ?
Maybe the .env needs a different format ?
Thanks in advance
It's not possible.
Maybe the .env needs a different format ?
No.
Use a script to export the variable and wrap docker-compose in it.
#!/bin/sh
# run-docker-compose.sh
export HOST_IP=$(ip route | awk '/default/ { print $3 }')
docker-compose "$#"
For example, I can quote ABC as part of BCD.
ABC='value_abc'
BCD=${ABC}:9876
The .env file is not parsed by a shell, it's simply a key=value mapping defined in the compose-go spec. It will not handle any nested variables, quotes, escape characters, etc., they are simply passed directly through as the value of the variable.
To do anything more complex, you'll need to set your environment variables yourself before calling compose. This could be with a script that sources your env file, e.g. you could make a docker-compose-expanded script that contains:
set -a
[ -f ./.env-expanded ] && . ./.env-expanded
set +a
docker-compose "$#"
I am involved in a Docker Compose project and we take advantage of the .env file possibility. However, I discovered that I can not reuse one environment variable while constructing another one, or reuse existing OS-level environment variables.
For example, this doesn't work:
VIRTUAL_HOST=domain.com
LETSENCRYPT_HOST=${VIRTUAL_HOST}
LETSENCRYPT_EMAIL=contact#${VIRTUAL_HOST}
Any ways around it?
Create a entry script similar to this:
#!/usr/bin/env bash
set -e
# Run a substitution because docker don't support nesting variables
export LETSENCRYPT_HOST=$(echo ${LETSENCRYPT_HOST} | envsubst)
export LETSENCRYPT_EMAIL=$(echo ${LETSENCRYPT_EMAIL} | envsubst)
exec "$#"
I really don't know how to get supervisor to work with environment variables.
Below is a configuration snippet.
[program:htNotificationService]
priority=2
#autostart=true
#autorestart=true
directory=/home/ubuntu/workspace/htFrontEnd/heythat/htsite
command = /usr/bin/python htNotificationService.py -service
stdout_logfile=/var/log/heythat/htNotificationService.log
redirect_stderr=true
environment=PATH=/home/ubuntu/workspace/htFrontEnd/heythat
stopsignal=QUIT
I have tried the following:
environment=PATH=/home/ubuntu/workspace/htFrontEnd/heythat
environment=PYTHONPATH=$PYTHONPATH:/home/ubuntu/workspace/htFrontEnd/heythat
environment=PATH=/home/ubuntu/workspace/htFrontEnd/heythat,PYTHONPATH=$PYTHONPATH:/home/ubuntu/workspace/htFrontEnd/heythat
When I start supervisor I get
htNotificationService: ERROR (abnormal termination)
I can start from the shell by setting the python path, but not from supervisor. In the logs I get an error that says that an import can't be found. Well, that would be solved if supervisor would work. I even have the path in /etc/environments?
Why will supervisor not work?
Referencing existing env vars is done with %(ENV_VARNAME)s
See: https://github.com/Supervisor/supervisor/blob/master/supervisor/skel/sample.conf
Setting multiple environment variables is done by separating them with commas
See: http://supervisord.org/subprocess.html#subprocess-environment
Try:
environment=PYTHONPATH=/opt/mypypath:%(ENV_PYTHONPATH)s,PATH=/opt/mypath:%(ENV_PATH)s
In your .conf file under the supervisord block, you can add all the environment key=value pairs as such
[supervisord]
environment=CELERY_BROKER_URL="amqp://guest:guest#127.0.0.1:5672//",FLASK_CONFIG="TESTING"
[program:celeryd]
command=celery worker -A celery --loglevel=info -P gevent -c 1000
If you dont want to hardcode the variables but want to pull it in from the os environment, step 1 on your bash
Export env var
>> sudo export CELERY_BROKER_URL="amqp://guest:guest#127.0.0.1:5672//"
Reload Bash
>> . ~/.bashrc
Check if env vars are set properly
>> env
Now modify the conf file to read - Note: prepend your env variables with ENV_
[supervisord]
environment=CELERY_BROKER_URL="%(ENV_CELERY_BROKER_URL)s",FLASK_CONFIG="%(ENV_FLASK_CONFIG)s"
[program:celeryd]
command=celery worker -A celery --loglevel=info -P gevent -c 1000
this works for me. note the tabs before each line:
environment=
CLOUD_INSTANCE_NAME=media-server-xx-xx-xx-xx,
CLOUD_APPLICATION=media-server,
CLOUD_APP_COMPONENT=none,
CLOUD_ZONE=a,
CLOUD_REGION=b,
CLOUD_PRIVATE_IP=none,
CLOUD_PUBLIC_IP=xx.xx.xx.xx,
CLOUD_PUBLIC_IPV6=xx.xx.xx.xx.xx.xx,
CLOUD_PROVIDER=c
I know this is old but I just struggled with this for hours and wanted to maybe help out the next guy.
Don't forget to reload your config files after making updates
supervisorctl reread
supervisorctl update
If you install supervisor from a package installer, check which Supervisor version you are using.
As of August 2016 you will get 3.0b2. If this is the case you will need a newer version of supervisor. You can get it by installing supervisor manually or by using Python's pip. Make sure all the dependencies are met, along with the upstart setup so that supervisord works as a service and starts on system boot.