Read ENV variable value within the ENV File only [duplicate] - docker

I am using a base.env as an env_file for several of my docker services.In this base.env I have several parts of the environment variable that repeat throughout the file. For example, port and ip are the same for three different environment variables.
I would like to specify these in an environment variable and reuse those variables to fill out the other environment variables.
Here is base.env:
### Kafka
# kafka's port is 9092 by default in the docker-compose file
KAFKA_PORT_NUMBER=9092
KAFKA_TOPIC=some-topic
KAFKA_IP=kafka
KAFKA_CONN: //$KAFKA_IP:$KAFKA_PORT_NUMBER/$KAFKA_TOPIC
# kafka topic that is to be created. Note that ':1:3' should remain the same.
KAFKA_CREATE_TOPICS=$KAFKA_TOPIC:1:3
# the url for connecting to kafka
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://$KAFKA_IP:$KAFKA_PORT_NUMBER
I have tried writing
KAFKA_CONN: //$${KAFKA_IP}:$${KAFKA_PORT_NUMBER}/$${KAFKA_TOPIC}
in the environment section of the appropriate service in the docker-compose.yml, but this gets interpreted as a literal string in the container.
Is there a way to do what I want in the base.env file?
Thank you for your help!

You can actually do it like this (at least in vlucas/dotenv package (php), not sure about others, please check it yourself)
MAIL_NAME=${MAIL_FROM}
Read more about it here

There is no way to do this in an env_file since it is not run as a bash command. This means that the variable is not created and then concatenated into the next variable it appears in. The values are just read in as they are in the env_file.

I used $ in Node.js and React.js , and both worked
POSTGRES_PORT=5432
DATABASE_URL="postgresql://root#localhost:${POSTGRES_PORT}/dbname"
and in react
REACT_APP_DOMAIN=domain.com
#API Configurations
REACT_APP_API_DOMAIN=$REACT_APP_DOMAIN

I know that I am a little late to the party, but I had the same question and I found a way to do it. There is a package called env-cmd, which allows you to use a .js file as an .env file. The file simply needs to export an object with the keys being your environment variable names and the values being, well, the values. This now allows you to run javascript before the environment variables are exported and thus use environment variables to set others.

I temporarly managed to deal with this where I create a script to replace env file vars from another env file vars like so:
.env.baseurl:
BASEURL1=http://127.0.0.1
BASEURL2=http://192.168.1.10
.env.uris.default:
URI1=${BASEURL1}/uri1
URI2=${BASEURL2}/uri2
URI3=${BASEURL2}/uri3
convert-env.sh:
#!/bin/bash
# To allow using sed correctly from same file multiple times
cp ./.env.uris.default ./.env.uris
# Go through each variable in .env.baseurl and store them as key value
for VAR in $(cat ./.env.baseurl); do
key=$(echo $VAR | cut -d "=" -f1)
value=$(echo $VAR | cut -d "=" -f2)
# Replace env vars by values in ./.env.uris
sed -i "s/\${$key}/$value/g" ./.env.uris
done
then you can run docker run command to start the container and load it with your env vars (from .env.baseurl to .env.uris) :
docker run -d --env-file "./.env.uris" <image>
This is not the best solution but helped me for now.

Using Nextjs, in the .env.local file I have the following variables:
NEXT_PUBLIC_BASE_URL = http://localhost:5000
NEXT_PUBLIC_API_USERS_URL_REGISTER = ${NEXT_PUBLIC_BASE_URL}/api/users/register
it works well, I used the variable NEXT_PUBLIC_BASE_URL in the variable NEXT_PUBLIC_API_USERS_URL_REGISTER.

There is a simple way to do this you will just need to run:
env >>/root/.bashrc && source /root/.bashrc
this will append all environment variables inside /root/.bashrc then convert those if they have not been converted while passing the env-file

you can use something like this ${yourVar}
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://${KAFKA_IP}:${$KAFKA_PORT_NUMBER}
I test this on PHP / Laravel .env it's working fine

Related

How to make environment variable visible to non root users inside the container?

I am trying to pass on environment variables to be read from an XML file inside a docker container running wildly app service and hosted inside REHL 7 image.
What I've done so far:
I've created an environment file as key value pair, for example: FILESERVICE_MAX_POOL_SIZE=5
I am running docker by referencing the environment file: docker run -d --env-file ./ENV_VARIABLES <myImage>
In the Dockerfile I copy the xml template I need: COPY dockerfiles/standalone.xml /opt/wildfly/standalone/configuration/standalone.xml
Inside the XML template I'm trying to reference the environment variable: <max-pool-size>${env.FILESERVICE_MAX_POOL_SIZE}</max-pool-size>
I can see those environment variables inside the running container as root but not as the wildly user which needs them. How can I make an attribute visible to a specific user other than root ?
Clearly I'm doing something fundamentally wrong here just not sure what ?
Thanks in advance for your help.
Problem solved: wildfly couldn't see the attributes because in my startup script I didn't add the -E flag for sudo to preserve environment variables.

Replace string placeholder with value in sh file

I have to say that in Windows environment /Powershell I would have done it immediately, but since I have to execute this shell script inside a docker Linux image, I need your help.
I have a node.js env file where I store my environment variables, so the nodejs app can use them later.I've set some placeholders and I need to replace them substituting from the event args parameter I got from docker run command.
The content of the .env file is
NodePort={NodePort}
DBServer={DBServer}
DBDatabaseName={DBDatabaseName}
DBUser={DBUser}
DBPassword={DBPassword}
DBEncrypt= {DBEncrypt}
RFIDNodeUrlRoot={RFIDNodeUrlRoot}
RFIDStartMethod={RFIDStartMethod}
RFIDStopMethod={RFIDStopMethod}
RFIDGetTagsMethod={RFIDGetTagsMethod}
I don't know which is the best approach to open the file, replace the values from env variables, and then save it.
Anyone can please help me?
Thanks
You can use envsubst which ist part of gettext-base package
see:
https://stackoverflow.com/a/14157575/2087704
https://unix.stackexchange.com/a/294400/193945
.env.temp
NodePort=${NodePort} # notice the `$` before `{}`
DBServer=${DBServer}
..
Assuming you are setting environment variables with
docker run -e "NodePort=8080" -e "DBServer=foo"
Inside that container you will have to use some entrypoint.sh script to run:
envsubst \$NodePort,$\DBServer,.. < .env.temp > .env
then start your app passing .env to your nodejs app.
As an alternative you can also use sed to edit .env, which might be hard to understand.
subst_env() {
eval val="\$$1" # expands environment variable to val
sed -i "s%\$$1%${val}%g" $2 # using % as sed-delimiter to avoid escaping slashes in urls
}
subst_env 'ENV_DOCKER_DOMAIN' .env

Dockerfile single line `ENV` composing variables not working

I want to compose two environment variables: first define a "root" and in the same line use that to create a composed one. In example, filename and append extension.
Doing this container,
FROM centos:7
ENV ROOT_VAR=stringy ROOT_VAR_TGZ=${ROOT_VAR}.tar.gz
RUN echo ${ROOT_VAR} $ ${ROOT_VAR_TGZ}
The output for echo is
stringy $ .tar.gz
But when splitting each variable in an individual ENV command is composed correctly.
Is this the expected behaviour?
The behaviour is clearly explained in the docker reference document:
Environment variable substitution will use the same value for each variable throughout the entire instruction. In other words, in this example:
ENV abc=hello
ENV abc=bye def=$abc
ENV ghi=$abc
will result in def having a value of hello, not bye. However, ghi will have a value of bye because it is not part of the same instruction that set abc to bye.

How to Use Environment Variables in a .env file to fill out other environment variable in the same .env file

I am using a base.env as an env_file for several of my docker services.In this base.env I have several parts of the environment variable that repeat throughout the file. For example, port and ip are the same for three different environment variables.
I would like to specify these in an environment variable and reuse those variables to fill out the other environment variables.
Here is base.env:
### Kafka
# kafka's port is 9092 by default in the docker-compose file
KAFKA_PORT_NUMBER=9092
KAFKA_TOPIC=some-topic
KAFKA_IP=kafka
KAFKA_CONN: //$KAFKA_IP:$KAFKA_PORT_NUMBER/$KAFKA_TOPIC
# kafka topic that is to be created. Note that ':1:3' should remain the same.
KAFKA_CREATE_TOPICS=$KAFKA_TOPIC:1:3
# the url for connecting to kafka
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://$KAFKA_IP:$KAFKA_PORT_NUMBER
I have tried writing
KAFKA_CONN: //$${KAFKA_IP}:$${KAFKA_PORT_NUMBER}/$${KAFKA_TOPIC}
in the environment section of the appropriate service in the docker-compose.yml, but this gets interpreted as a literal string in the container.
Is there a way to do what I want in the base.env file?
Thank you for your help!
You can actually do it like this (at least in vlucas/dotenv package (php), not sure about others, please check it yourself)
MAIL_NAME=${MAIL_FROM}
Read more about it here
There is no way to do this in an env_file since it is not run as a bash command. This means that the variable is not created and then concatenated into the next variable it appears in. The values are just read in as they are in the env_file.
I used $ in Node.js and React.js , and both worked
POSTGRES_PORT=5432
DATABASE_URL="postgresql://root#localhost:${POSTGRES_PORT}/dbname"
and in react
REACT_APP_DOMAIN=domain.com
#API Configurations
REACT_APP_API_DOMAIN=$REACT_APP_DOMAIN
I know that I am a little late to the party, but I had the same question and I found a way to do it. There is a package called env-cmd, which allows you to use a .js file as an .env file. The file simply needs to export an object with the keys being your environment variable names and the values being, well, the values. This now allows you to run javascript before the environment variables are exported and thus use environment variables to set others.
I temporarly managed to deal with this where I create a script to replace env file vars from another env file vars like so:
.env.baseurl:
BASEURL1=http://127.0.0.1
BASEURL2=http://192.168.1.10
.env.uris.default:
URI1=${BASEURL1}/uri1
URI2=${BASEURL2}/uri2
URI3=${BASEURL2}/uri3
convert-env.sh:
#!/bin/bash
# To allow using sed correctly from same file multiple times
cp ./.env.uris.default ./.env.uris
# Go through each variable in .env.baseurl and store them as key value
for VAR in $(cat ./.env.baseurl); do
key=$(echo $VAR | cut -d "=" -f1)
value=$(echo $VAR | cut -d "=" -f2)
# Replace env vars by values in ./.env.uris
sed -i "s/\${$key}/$value/g" ./.env.uris
done
then you can run docker run command to start the container and load it with your env vars (from .env.baseurl to .env.uris) :
docker run -d --env-file "./.env.uris" <image>
This is not the best solution but helped me for now.
Using Nextjs, in the .env.local file I have the following variables:
NEXT_PUBLIC_BASE_URL = http://localhost:5000
NEXT_PUBLIC_API_USERS_URL_REGISTER = ${NEXT_PUBLIC_BASE_URL}/api/users/register
it works well, I used the variable NEXT_PUBLIC_BASE_URL in the variable NEXT_PUBLIC_API_USERS_URL_REGISTER.
There is a simple way to do this you will just need to run:
env >>/root/.bashrc && source /root/.bashrc
this will append all environment variables inside /root/.bashrc then convert those if they have not been converted while passing the env-file
you can use something like this ${yourVar}
KAFKA_ADVERTISED_LISTENERS=PLAINTEXT://${KAFKA_IP}:${$KAFKA_PORT_NUMBER}
I test this on PHP / Laravel .env it's working fine

Passing many environment variables to Rails application ran by Unicorn

I need to pass a large number of environment variables to Rails application ran by Unicorn web server. The sample unicorn init script has the following lines:
APP_ROOT=/home/x/my_app/current
<...>
INIT_CONF=$APP_ROOT/config/init.conf
<...>
test -f "$INIT_CONF" && . $INIT_CONF
So I created a $APP_ROOT/config/init.conf, put all my variables there like this:
VAR1=value1
VAR2=value2
I even made this file executable (not sure if it is necessary)
And restarted Unicorn. But ENV["VAR1"] returns nothing in my application...
Is it supposed to work this way? If yes, what am I doing wrong? If no, then how can I pass many env vars into Rails app in a clean way? (without polluting global environment or putting all of them in the command line)
Update My investigation showed that shell file like this:
. init.conf
echo $VAR1
works as expected. But this one:
. init.conf
ruby -e "puts ENV['VAR1']"
does not. So . imports code into the script but env vars set this way are not transferred further.
You probably have to "export" the variables from within the config file. Does it work if you put
export VAR1=value1
export VAR2=value2
into the config file?
I would consider using foreman, specifically for its use of .env files as defined here.

Resources