I need to set today's date as environment variable in my docker container.
What I am trying to do is to get today's date so I can use it to call some API with Logstash's http_poller plugin.
The solution I am thinking about is to get today's date using the command date +%Y%m%d
How can I set the result of this command as an environment variable at run time?
ARG should be what you are looking for:
FROM base
# to be able to use in Dockerfile
ARG now
# to store the value as environment variable in the image
ENV build_date=$now
Now you can build this with
# pass value explicitly
docker build --build-arg now="$(date +%Y%m%d)" .
# pass value from environment
export now="$(date +%Y%m%d)"
docker build --build-arg now .
This still requires to run date on the host since doing this inside the Dockerfile is not possible unfortunately:
The only way to execute arbitrary commands in the build is within a RUN statement; but
The only way to persist a variable into the environment of an image/container is with an ENV statement which can only use environment variables from outside the build
You could use a custom ENTRYPOINT tough and inject the date to the environment from a file:
FROM base
RUN date +%Y%m%d > /build-timestamp
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT /entrypoint.sh
entrypoint.sh:
#!/bin/bash
export BUILD_TIMESTAMP="$(cat /build-timestamp)"
exec "$#"
So according to your answer to my comment, if you want to have the variable set at runtime you need to inject it when you start your container
✗ docker run -e TS=$(date +%Y%m%d) python:3.7-alpine env
TS=20201230
PYTHON_VERSION=3.7.9
HOME=/root
Here is the link to the docker documentation : https://docs.docker.com/engine/reference/run/#env-environment-variables
But you are confusing the Dockerfile with the container. A Dockerfile is a definition for building an image. This image is static. Then you start multiples containers from this base image.
So if your start 3 containers with the same image, according to your question you expect 3 different dates in your containers. So you see why you can't set a date or a command in a Dockerfile. The previous snippet does what you ask.
According to the documentation ENV only supports <key>=<value> entries, I see no way to put it there with a command. I suggest you drop the value into some file then read it when needed. Or, if you have a launch script, using RUN you can add export VAR_NAME=<value> to the launch script so that the variable is automatically added to the environment on launch.
My solution was to first do export command and then running service startup like this
CMD export INSTALL_DATE=$(date +%s) && python3 /product/server.py
Related
Objective
I have an env variable script file that looks like:
#!/bin/sh
export FOO="public"
export BAR="private"
I would like to source the env variables to be available when a docker image is being built. I am aware that I can use ARG and ENV with build args, but I have too many Env Variables, and I am afraid that will be a lengthy list.
It's worth mentioning that I only need the env variables to install a specific step in my docker file (will highlight in the Dockerfile below), and do not necessarily want them to be available in the built image after that.
What I have tried so far
I have tried having a script (envs.sh) that export env vars like:
#!/bin/sh
export DOG="woof"
export CAT="meow"
My Docker file looks like:
FROM fishtownanalytics/dbt:0.18.1
# Define working directory
# Load ENV Vars
COPY envs.sh envs.sh
CMD ["sh", "envs.sh"]
# Install packages required
CMD ["sh", "-c", "envs.sh"]
RUN dbt deps # I need to env variables to be available for this step
# Exposing DBT Port
EXPOSE 8081
But that did not seem to work. How can I export env variables as a script to the docker file?
In the general case, you can't set environment variables in a RUN command: each RUN command runs a new shell in a new container, and any environment variables you set there will get lost at the end of that RUN step.
However, you say you only need the variables at one specific step in your Dockerfile. In that special case, you can run the setup script and the actual command in the same RUN step:
FROM fishtownanalytics/dbt:0.18.1
COPY envs.sh envs.sh
RUN . ./envs.sh \
&& dbt deps
# Anything that envs.sh `export`ed is lost _after_ the RUN step
(CMD is irrelevant here: it only provides the default command that gets run when you launch a container from the built image, and doesn't have any effect on RUN steps. It also looks like the image declares an ENTRYPOINT so that you can only run dbt subcommands as CMD, not normal shell commands. I also use the standard . to read in a script file instead of source, since not every container has a shell that provides that non-standard extension.)
Your CMD call runs a new shell (sh) that defines those variables and then dies, leaving the current process unchanged. If you want those environment variables to apply to the current process, you could source it:
CMD ["source", "envs.sh"]
I have a dockerfile that has a entrypoint.sh file which exports some Postgres variable.
Then I want to start the parent docker container which is referenced in "FROM pactfoundation/pact-broker" image. Looking at github for it's Dockerfile github pact broker it has CMD ["config.ru"] at the end. So I did similar to that in my Dockerfile:
FROM pactfoundation/pact-broker
COPY entrypoint.sh .
CMD ["config.ru"]
When I execute my docker run command:
docker run --rm -e POSTGRES_PORT=5433 -e POSTGRES_DBNAME=pactsd -e POSTGRES_URL=localhost -e POSTGRES_PASSWORD=1234 -e POSTGRES_USERNAME=postgres --name pact sonamsamdupkhangsar/pact:test -d
I see my entrypoint.sh echo statement and the container is dead.
setting pact broker database variables
How do I start the parent container after setting my envrionment variables in my entrypoint.sh file?
I also tried with the following:
FROM pactfoundation/pact-broker
ENV PACT_BROKER_DATABASE_NAME=${POSTGRES_DBNAME}
ENV PACT_BROKER_DATABASE_USERNAME=${POSTGRES_USERNAME}
ENV PACT_BROKER_DATABASE_PASSWORD=${POSTGRES_PASSWORD}
ENV PACT_BROKER_DATABASE_HOST=${POSTGRES_URL}
ENV PACT_BROKER_DATABASE_NAME=${POSTGRES_DBNAME}
ENV PACT_BROKER_DATABASE_PORT=$POSTGRES_PORT
RUN echo "PACT_BROKER_DATABASE_PORT: $PACT_BROKER_DATABASE_PORT"
Yet, when I run my built docker image I still don't see the variables being set. I tried both approaches for "${}" and "$" for env var setting.
You've to set your environment variables using the ENV in your docker file.
As each step executed at different containers which altogether builds the image if you set via shell scripts it won't work. Consider using the ENV command to set it
Ref: DOCKERFILE ENV
What is happening is that those environment variables that you are passing in at run-time with '-e' parameter are not yet defined at build-time as the ENV instructions are executed at build-time only.
E.g. at build-time this line you have:
ENV PACT_BROKER_DATABASE_NAME=${POSTGRES_DBNAME}
becomes this line:
ENV PACT_BROKER_DATABASE_NAME=
as '${POSTGRES_DBNAME}' evaluates to empty at build-time. Then, when run-time happens, you are defining all your POSTGRES_ environment variables as parameters so they will indeed exist in the container, BUT no further instructions will be executed to set the PACT_BROKER_ environment variables to any other values.
Proposed solution: I would recommend the simplest approach if you can make it work to just use the environment variables 'directly' however you define them as parameters. I.e. either change the names of your '-e' parameters to PACT_BROKER_'s or use the POSTGRES_ environment variables in your container. Either way you would remove the ENV lines from the Dockerfile.
If you really-really need to set the environment variables to other names at run-time, then you should be able to do this by writing to the appropriate 'startup' file in the Dockerfile (making sure to literally write the '$'s to the file so they could be dereferenced at run-time).
The ENV instruction sets the environment variable to the value . This value will be in the environment for all subsequent instructions in the build stage
Below instruction:
ENV PATH=$PATH:$HOME/go/bin
does not append PATH variable
$HOME/go/bin is /root/go/bin
How to append $HOME/go/bin to $PATH? in below docker file
FROM golang:1.14.10
MAINTAINER xyz
ENV GOPATH=
ENV PATH=$PATH:$HOME/go/bin
RUN echo $PATH
Apparently Docker doesn't let you use environment variables defined outside of your Dockerfile within an ENV or ARG declaration.
As a workaround, you can pass the names/directories to your Dockerfile explicitly using ARG:
FROM golang:1.14.10
# set default to `root`
ARG USERNAME=root
ENV PATH=$PATH:/$USERNAME/go/bin
RUN echo $PATH
You can then pass the USERNAME via docker build --build-arg USERNAME=myuser
Depending on your usecase you can also do this using a RUN or ENTRYPOINT.
I think the confusion is this: when you say $HOME in ENV, home isn't defined yet. But when you say RUN echo $HOME, home is defined by the shell in the base image.
PATH is working and causing confusion, because it's defined by the base image you're using with FROM.
ENV is used to define default variables for the image that will be built, and that will be accessible in RUN statements. Think of it this way: the Dockerfile can provide variables to the container, but the container cannot provide variables to the Dockerfile.
Really, I would just hardcode in /root if root is the user you want to run from. The variables provided by the build are meant to be defaults if you want to do something fancy and dynamic, you probably are better off injecting a script into your image, and running that.
As per mentioned here in this link
I tried using this format and it worked with me:
ENV gradle=/opt/gradle/gradle-6.6.1/bin
ENV PATH=${gradle}:${PATH}
Say I have a Dockerfile:
.
.
RUN echo 'source /root/script.sh' >> /etc/bash.bashrc
(The script adds some env variables)
If I:
1) Do this:
docker run -it -v /home/user/script.sh:/root/script.sh image
It takes me to shell where if I call "env" I see the variable set by the script
But if I:
2) Do this:
docker run -it -v /home/user/script.sh:/root/script.sh image env
It prints out env and exits and my variable is missing
What am I missing? I need the variable to exists even if I specify a command/script like "env" at the end of the docker run command
When you run a command like
docker run ... image command
Docker directly runs the command you give; it doesn’t launch any kind of shell, and there’s no opportunity for a .bashrc or similar file to be read.
I’d suggest two things here:
If your program does need environment variables set in some form, set them directly using Dockerfile ENV directives. Don’t try to edit .bashrc or /etc/profile or any other shell dotfile; they won’t reliably get run.
As much as you can install things in places so that you don’t need to change environment variables. For instance, Python supports a “virtual environment” concept that allows an isolated library environment, which requires changing $PATH and similar things; but Docker provides the same isolation on its own, so just install things into the “global” package space.
If you really can’t manage either of these things, then you can write an entrypoint script that sets environment variables and then launches the container’s command. This might look like
#!/bin/sh
. /root/script.sh
exec "$#"
And then you could include this in your Dockerfile like
...
COPY entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
CMD ["/app/myapp"]
(If you need to use docker exec to get a debugging shell in the container, that won’t be a child process of the entrypoint and won’t get its environment variables.)
My specific use case is that I want to organize some data about the EC2 instance a container is running on and make i available as an environment variable. I'd like to do this when the container is built.
I was hoping to be able to do something like ENV VAR_NAME $(./script/that/gets/var) in my Dockerfile, but unsurprisingly that does not work (you just get the string $(./script...).
I should mention that I know the docker run --env... will do this, but I specifically want it to be built into the container.
Am I missing something obvious? Is this even possible?
Docker v1.9 or newer
If you are using Docker v1.9 or newer, this is possible via support for build time arguments. Arguments are declared in the Dockerfile by using the ARG statement.
ARG REQUIRED_ARGUMENT
ARG OPTIONAL_ARGUMENT=default_value
When you later actually build your image using docker build you can pass arguments via the flag --build-arg as described in the docker docs.
$ docker build --build-arg REQUIRED_ARGUMENT=this-is-required .
Please note that it is not recommended to use build-time variables for passwords or secrets such as keys or credentials.
Furthermore, build-time variables may have great impact on caching. Therefore the Dockerfile should be constructed with great care to be able to utilize caching as much as possible and therein speed up the building process.
Edit: the "docker newer than v1.9"-part was added after input from leedm777:s answer.
Docker before v1.9
If you are using a Docker-version before 1.9, the ARG/--build-arg approach was not possible. You couldn't resolve this kind of info during the build so you had to pass them as parameters to the docker run command.
Docker images are to be consistent over time whereas containers can be tweaked and considered as "throw away processes".
More info about ENV
A docker discussion about dynamic builds
The old solution to this problem was to use templating. This is not a neat solution but was one of very few viable options at the time. (Inspiration from this discussion).
save all your dynamic data in a json or yaml file
create a docker file "template" where the dynamic can later be expanded
write a script that creates a Dockerfile from the config data using some templating library that you are familiar with
Docker 1.9 has added support for build time arguments.
In your Dockerfile, you add an ARG statement, which has a similar syntax to ENV.
ARG FOO_REQUIRED
ARG BAR_OPTIONAL=something
At build time, you can pass pass a --build-arg argument to set the argument for that build. Any ARG that was not given a default value in the Dockerfile must be specified.
$ docker build --build-arg FOO_REQUIRED=best-foo-ever .
To build ENV VAR_NAME $(./script/that/gets/var) into the container, create a dynamic Dockerfile at build time:
$ docker build -t awesome -f Dockerfile .
$ # get VAR_NAME value:
$ VAR_VALUE=`docker run --rm awesome \
bash -c 'echo $(./script/that/gets/var)'`
$ # use dynamic Dockerfile:
$ {
echo "FROM awesome"
echo "ENV VAR_NAME $VAR_VALUE"
} | docker build -t awesome -
https://github.com/42ua/docker-autobuild/blob/master/emscripten-sdk/README.md#build-docker-image