If I have docker run ... -e VAR=x ... -e VAR=y ..., what is the value of VAR in the container? From a quick test it looks like the last one is used, but is this guaranteed anywhere in the documentation?
No, this is not guaranteed as far as I can tell. Here are the relevant locations in the documentation:
The docker run reference
The docker run command line reference
Also the docker daemon API does not specify what happens on duplicate entries in the Env array.
There is also the documentation on Environment variables precedence for docker-compose. But this also does not mention duplicate keys in one of the layers.
You are probably best advised to not rely on the fact that this is implemented as it is.
Related
Using the docker run command, I'm trying to pass my NEPTUNE_API_TOKEN to my container.
My understanding is that I should use the -e flag as follows: -e ENV_VAR='env_var_value' and that might work.
I wish, however, to use the value existing in the already-running session, as follows:
docker run -e NEPTUNE_API_TOKEN=$(NEPTUNE_API_TOKEN) <my_image>
However, after doing so, NEPTUNE_API_TOKEN is set to empty when checking the value inside the container.
My question is whether I'm doing something wrong or if this is not possible and I must provide an explicit Neptune API token as a string.
$(NEPTUNE_API_TOKEN) is the syntax for running a command and grabbing the output. Use $NEPTUNE_API_TOKEN.
You can set up and pass NEPTUNE_API_TOKEN as a:
Docker run command options environment variable
Example: docker run -e NEPTUNE_API_TOKEN="<YOUR_API_TOKEN>" <image-name>
Dockerfile environment variable
Docker secret
Neptune will work with any of the methods described above.
For your case, I believe using method 2 and 3 will work best as you will set the API token only once and all containers can reuse it. Additionally, they are more secure methods.
You can read this guide on how to use Neptune with Docker I created last year.
Docs: https://docs.neptune.ai/how-to-guides/automation-pipelines/how-to-use-neptune-with-docker
Currently my run command looks like this...
docker run -e DB_URL=$DB_URL -e DB_PORT=$DB_PORT ... <image name>
This works but not very scalable. Is there a way to just pass all configured env vars to the container without declaring each one?
I am using OSX and these are set in a .bash_profile.
env > envFile && docker run --env-file=envFile alpine env
However I would not recommend doing this as this will pass even un-necessary info to docker container.
And you should rather use a compose file or maybe even a simple script to only pass in variables that are actually needed.
This might even mess with the shell inside of a container for things like prompts and locales etc.
Ultimately, the best option is to explicitly list the vars you want to pass. There should only be a finite number of them, they shouldnt change often, and this solution is more robust and more secure.
Note that you can simplify your example. If you want to pass in a defined var with its current value, just name the var instead of setting it:
docker run -e DB_URL -e DB_PORT ... <image name>
I am trying to use one Dockerfile for both my production and development. The only difference between the production and development are the environment variables I set. Therefore I would like someway import the environment variables from a file. Before using Docker I would simply do the following
. ./setvars
./main.py
However if change ./main.py with the Docker equivalent
. ./setvars
docker run .... ./main.py
then the variables will be on the host and not accessible from the Docker instance. Of course a quick and dirty hack would be to make a file with
#!/bin/bash
. ./setvars
./main.py
and run that in the instance. That would however be really annoying, since I got lots of scripts I would like to run (with the same environment variables), and would then have to create a extra script for everyone of those.
Are there any other solution to get my environment variables inside docker without using a different Dockerfile and the method I described above?
Your best options is to use either the -e flag, or the --env-file of the docker run command.
The -e flag allows you to specify key/value pairs of env variable,
for example:
docker run -e ENVIRONMENT=PROD
You can use several time the -e flag to define multiple env
variables. For example, the docker registry itself is configurable
with -e flags, see:
https://docs.docker.com/registry/deploying/#running-a-domain-registry
The --env-file allow you to specify a file. But each line of the file
must be of type VAR=VAL
Full documentation:
https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables-e-env-env-file
I've created a docker image for my database server and one for the web application. Using the documentation I'm able to link between both container using the environment variables as follow:
value="jdbc:postgresql://${DB_PORT_5432_TCP_ADDR}:${DB_PORT_5432_TCP_PORT}/db_name"
It works fine now but it would be better that the environment variables are more general and without containing a static port number. Something like:
value="jdbc:postgresql://${DB_URL}:${DB_PORT}/db_name"
Is there anyway to link between the environment variables? for example by using the ENV command in the dockerfile ENV DB_URL=$DB_PORT_5432_TCP_ADDR or by using the argument --env by running the image docker run ... -e DB_URL=$DB_PORT_5432_TCP_ADDR docker_image ?
Without building this kind of functionality into your docker startup shell scripts or other orchestration mechanism, this is not possible at the moment to create environment variables like you are describing here. You do mention a couple of workarounds. However, the problem at least with using the -e DB_URL=... in your docker run command is that your $DB_PORT_5432_TCP_ADDR environment variable is not known at runtime, and so you will not be able to set this value when you run it. Typically, this is what your orchestration layer is used for, service discovery and passing this kind of data among your containers. There is at least one workaround mentioned here on SO that involves constructing a special shell script that you put in your CMD or ENTRYPOINT directives that passes the environment variable to the container.
I have two containers webinterface and db, while webinterface is started using the --link option (for db) which generates the environment variables
DB_PORT_1111_TCP=tcp://172.17.0.5:5432
DB_PORT_1111_TCP_PROTO=tcp
DB_PORT_1111_TCP_PORT=1111
DB_PORT_1111_TCP_ADDR=172.17.0.5
...
Now my webinterface container uses a Dockerfile where some static environment variables are defined to define the connection:
ENV DB_HOST localhost
ENV DB_PORT 2222
Knowing that there is also an -e option for docker run, the problem is that I want to use those variables in the Dockerfile (used in some scripts) but overwrite them with the values generated with the --link option, i.e. something like:
docker run -d -e DB_HOST=$DB_PORT_1111_TCP_ADDR
This would use the host's defined environment variable which doesn't work here.
Is there a way to handle this?
This is a variable expansion issue so to resolve try the following:
docker run -d -e DB_HOST="$DB_PORT"_1111_TCP_ADDR
With a Unix process that is already running, its environment variables can only be changed from inside the process, not from the outside, so their are somewhat non-dynamic by nature.
If you find Docker links limiting, you are not the only person out there. One simple solution to this would be using WeaveDNS. With WeaveDNS you can simply use default ports (as with Weave overlay network there is no need to expose/publish/remap any internal ports) and resolve each component by via DNS (i.e. your app would just need to look for db.weave.local, and doesn't need to be aware of clunky environment variable scheme that Docker links present). To get a better idea of how WeaveDNS works, checkout one of the official getting started guides. WeaveDNS effectively gives you service discovery without having to modify the application you have.