Docker env files look similar to shell scripts defining variables.
Unfortunately in env files, values cannot be quoted and so simply sourcing them only works if there are no "special" characters in the values.
VAR1=This_works
VAR2=This will not work
VAR3=This won't either
Is there any way you can use these files in a shell script?
My current approach is this:
eval $( perl -ne '
s/\x27/\x27\\\x27\x27/g;
s/^(\w+)=(.+)$/export $1=\x27$2\x27/ and print
' "path/to/env_file" )
So I'm searching for any quote in each line of the env file and replace it by '\''.
Then I'm modifying each line which starts with an identifier (\w+) followed by a = and any text (.+). So the VAR3 will become: export VAR3='This won'\''t either.
The modified line is then printed.
Everything which was printed is eval-ed and so the variables will be available in my shell's environment.
Are there other proposals how to achieve this?
Related
I have to say that in Windows environment /Powershell I would have done it immediately, but since I have to execute this shell script inside a docker Linux image, I need your help.
I have a node.js env file where I store my environment variables, so the nodejs app can use them later.I've set some placeholders and I need to replace them substituting from the event args parameter I got from docker run command.
The content of the .env file is
NodePort={NodePort}
DBServer={DBServer}
DBDatabaseName={DBDatabaseName}
DBUser={DBUser}
DBPassword={DBPassword}
DBEncrypt= {DBEncrypt}
RFIDNodeUrlRoot={RFIDNodeUrlRoot}
RFIDStartMethod={RFIDStartMethod}
RFIDStopMethod={RFIDStopMethod}
RFIDGetTagsMethod={RFIDGetTagsMethod}
I don't know which is the best approach to open the file, replace the values from env variables, and then save it.
Anyone can please help me?
Thanks
You can use envsubst which ist part of gettext-base package
see:
https://stackoverflow.com/a/14157575/2087704
https://unix.stackexchange.com/a/294400/193945
.env.temp
NodePort=${NodePort} # notice the `$` before `{}`
DBServer=${DBServer}
..
Assuming you are setting environment variables with
docker run -e "NodePort=8080" -e "DBServer=foo"
Inside that container you will have to use some entrypoint.sh script to run:
envsubst \$NodePort,$\DBServer,.. < .env.temp > .env
then start your app passing .env to your nodejs app.
As an alternative you can also use sed to edit .env, which might be hard to understand.
subst_env() {
eval val="\$$1" # expands environment variable to val
sed -i "s%\$$1%${val}%g" $2 # using % as sed-delimiter to avoid escaping slashes in urls
}
subst_env 'ENV_DOCKER_DOMAIN' .env
I want to compose two environment variables: first define a "root" and in the same line use that to create a composed one. In example, filename and append extension.
Doing this container,
FROM centos:7
ENV ROOT_VAR=stringy ROOT_VAR_TGZ=${ROOT_VAR}.tar.gz
RUN echo ${ROOT_VAR} $ ${ROOT_VAR_TGZ}
The output for echo is
stringy $ .tar.gz
But when splitting each variable in an individual ENV command is composed correctly.
Is this the expected behaviour?
The behaviour is clearly explained in the docker reference document:
Environment variable substitution will use the same value for each variable throughout the entire instruction. In other words, in this example:
ENV abc=hello
ENV abc=bye def=$abc
ENV ghi=$abc
will result in def having a value of hello, not bye. However, ghi will have a value of bye because it is not part of the same instruction that set abc to bye.
As part of a Dockerfile, I am attempting to modify a text file (ssh_config) to contain a variable passed through by the user (as an ARG) to the docker container at build time.
In my Dockerfile I have (this is not the entire file):
ARG key_name
RUN echo 'Host geoserver\n\
User user\n\
HostName 38.191.191.111\n\
IdentityFile /root/$key_name' >> /etc/ssh/ssh_config
This collects the argument key_name, and then appends some text to the ssh_config text file.
This is run as follows:
docker build --build-arg key_name=GC -t pyramid .
When I check to see what has been written, the key_name variable hasn't been parsed, and instead has been written as text (so literally as $key_name). Obviously I want it to be replaced with the variable passed through ARG.
I have tried using ${key_file} instead of just $key_file, I just get the same text in the text file but with curly braces included.
So my question is, how can I use the ARG variable correctly within the RUN echo statement?
First: Make sure, your ARG comes after your FROM. See: https://docs.docker.com/engine/reference/builder/#understand-how-arg-and-from-interact
Second: As you can see here, variables won't be interpretad inside '', so use "" instead.
When you surround the variable with single quotes it doesn't get replaced.
If you need the single qoutes in the file just surround everything with double quotes, otherwise just remove the single quotes all together.
ARG key_name
RUN echo "'Host geoserver\n\
User user\n\
HostName 38.191.191.111\n\
IdentityFile /root/$key_name'" >> /etc/ssh/ssh_config
I'm storing environment variables in /etc/environment like:
FACEBOOK_API_KEY=XXXXXXXXXXX
FACEBOOK_API_SECRET=XXXXXXXXXX
But when i access the ENV variable through Rails i get this:
ENV['FACEBOOK_API_KEY']
=> XXXXXXXXX\r
Notice the \r, How to get rid of that without cleaning up each and every call to ENV vars?
My guess is that you're getting a \r because you're editing /etc/environment using a Windows text editor and installing it on a Unix system (or possibly using Cygwin in Windows, but the same applies). Or you copied/pasted it from a source that did that. In any case, something has introduced a CRLF into a Unix file that only wants LF line-endings.
If it is an editor, you'll want to fix it to stop using Windows CRLF (\r\n) line endings, and use Unix LF (\n) line endings. Notepad++ has an option for this, as do many other editors and IDEs. Google around for yours and find out how to use Unix line endings. You'll run into a ton of problems like this otherwise.
You can fix the existing file by running it through a program like dos2unix (on a Unix system; you may have to install the package), or using a simple tr command like this:
Edit: fixed the filename order in the mv command below.
tr -d '\r' </etc/environment >/tmp/environment
# <verify new file looks good>
mv /tmp/environment /etc/environment
Please be careful, make backups, check the file, etc.
You can make sure there aren't any rogue \r characters in your file by looking at an octal dump:
od -c /tmp/environment
Look for any \r in the output.
You can use figaro to manage your ENV variables.
It creates a config/application.yml file for you that should not be checked into version control.
# config/application.yml
FACEBOOK_API_KEY: XXXXXXXXXXX
FACEBOOK_API_SECRET: XXXXXXXXXX
Variables will be available at ENV['FACEBOOK_API_KEY'] as you are used to.
An alternative would be configatron.
I have a shell script I wish to read parameters from an external file, to get files via FTP:
parameters.txt:
FTP_SERVER=ftpserer.foo.org
FTP_USER_NAME=user
FTP_USER_PASSWORD=pass
FTP_SOURCE_DIRECTORY="/data/secondary/"
FTP_FILE_NAME="core.lst"
I cannot find how to read these variables into my FTP_GET.sh script, I have tried using read but it just echoed the vars and doesn't store them as required.
Assuming that 'K Shell' is Korn Shell, and that you are willing to trust the contents of the file, then you can use the dot command '.':
. parameters.txt
This will read and interpret the file in the current shell. The feature has been in Bourne shell since it was first released, and is in the Korn Shell and Bash too. The C Shell equivalent is source, which Bash also treats as a synonym for dot.
If you don't trust the file then you can read the values with read, validate the values, and then use eval to set the variables:
while read line
do
# Check - which is HARD!
eval $line
done