I have a dockerfile that has a entrypoint.sh file which exports some Postgres variable.
Then I want to start the parent docker container which is referenced in "FROM pactfoundation/pact-broker" image. Looking at github for it's Dockerfile github pact broker it has CMD ["config.ru"] at the end. So I did similar to that in my Dockerfile:
FROM pactfoundation/pact-broker
COPY entrypoint.sh .
CMD ["config.ru"]
When I execute my docker run command:
docker run --rm -e POSTGRES_PORT=5433 -e POSTGRES_DBNAME=pactsd -e POSTGRES_URL=localhost -e POSTGRES_PASSWORD=1234 -e POSTGRES_USERNAME=postgres --name pact sonamsamdupkhangsar/pact:test -d
I see my entrypoint.sh echo statement and the container is dead.
setting pact broker database variables
How do I start the parent container after setting my envrionment variables in my entrypoint.sh file?
I also tried with the following:
FROM pactfoundation/pact-broker
ENV PACT_BROKER_DATABASE_NAME=${POSTGRES_DBNAME}
ENV PACT_BROKER_DATABASE_USERNAME=${POSTGRES_USERNAME}
ENV PACT_BROKER_DATABASE_PASSWORD=${POSTGRES_PASSWORD}
ENV PACT_BROKER_DATABASE_HOST=${POSTGRES_URL}
ENV PACT_BROKER_DATABASE_NAME=${POSTGRES_DBNAME}
ENV PACT_BROKER_DATABASE_PORT=$POSTGRES_PORT
RUN echo "PACT_BROKER_DATABASE_PORT: $PACT_BROKER_DATABASE_PORT"
Yet, when I run my built docker image I still don't see the variables being set. I tried both approaches for "${}" and "$" for env var setting.
You've to set your environment variables using the ENV in your docker file.
As each step executed at different containers which altogether builds the image if you set via shell scripts it won't work. Consider using the ENV command to set it
Ref: DOCKERFILE ENV
What is happening is that those environment variables that you are passing in at run-time with '-e' parameter are not yet defined at build-time as the ENV instructions are executed at build-time only.
E.g. at build-time this line you have:
ENV PACT_BROKER_DATABASE_NAME=${POSTGRES_DBNAME}
becomes this line:
ENV PACT_BROKER_DATABASE_NAME=
as '${POSTGRES_DBNAME}' evaluates to empty at build-time. Then, when run-time happens, you are defining all your POSTGRES_ environment variables as parameters so they will indeed exist in the container, BUT no further instructions will be executed to set the PACT_BROKER_ environment variables to any other values.
Proposed solution: I would recommend the simplest approach if you can make it work to just use the environment variables 'directly' however you define them as parameters. I.e. either change the names of your '-e' parameters to PACT_BROKER_'s or use the POSTGRES_ environment variables in your container. Either way you would remove the ENV lines from the Dockerfile.
If you really-really need to set the environment variables to other names at run-time, then you should be able to do this by writing to the appropriate 'startup' file in the Dockerfile (making sure to literally write the '$'s to the file so they could be dereferenced at run-time).
Related
I have a Dockerfile ,to deploy node js app with some secret api keys ,that I want to hide from the docker file.
Currently I am using ENV keyword to define env variables like below
FROM node:17
WORKDIR /usr/app
COPY package.json /usr/app/
RUN npm install
COPY . /usr/app
ENV TWILIO_ACCOUNT_SID=""
ENV TWILIO_AUTH_TOKEN=""
ENV OTP_TEXT="This is your Otp"
ENV TWILLIO_SENDER=99999
ENV PORT=8080
ENV DB_URL=""
ENV JWT_SECRET="Some Secrete"
ENV JWT_EXPIRES_IN=30min
ENV OTP_EXPIRE_TIME_SECONDS=150000
ENV AWS_S3_REGION = us-east-2
ENV AWS_S3_BUCKET = gos32
ENV AWS_ACCESS_KEY_ID =""
ENV AWS_SECRET_ACCESS_KEY =""
CMD ["npm", "start"]
Any better way to do that ?
Edit :
Just adding What works me from the answer given by #blami
docker build -t app .
then I ran
docker run --env-file env.txt -d -p 8080:8080 app
docker run with the file option after putting all the env variables in env.txt file
You should not put sensitive data in Dockerfile at all. If your application is configured via environment, you should only provide these variables to container when it is started e.g. manually (using docker -e , --env , and --env-file flags directly on command line) or via your container runtime (which you do not specify in your question):
Kubernetes can manage secrets and expose them via files or environment variables - see documentation with examples
Docker Swarm supports managing secrets out of the box via docker secret command and can expose such secrets as environment variables too - see documentation with examples
Managed cloud providers usually have option to manage secrets and somehow inject them to containers too (or directly expose features of runtime they use).
In any of cases above secrets usually live in secure storage from where they are retrieved only when container starts and injected into it. That way you don't need to have them in Dockerfile. Note that if someone gains access to your running container with application or higher privileges they will be able to retrieve secrets from the environment (as that is how environment variables work).
I would like to set the LD_LIBRARY_PATH variable based on my working directory using my Dockerfile. I have tried using ENV $PWD/some/subpath but when I inspect the container later using docker exec mycontainer bash -c "env" it shows up as /some/subpath rather than /my/working/dir/some/subpath however I also see that PWD is defined as /my/working/dir/ as I would expect it to be. so why is using $PWD in my Dockerfile not substituting the way I am expecting it to?
From this answer, $PWD is a special environment variable set when running a shell. Unlike RUN commands, ENV commands do not create a shell so PWD is never set.
To get the value of PWD at build time, you could instead use a build-arg and pass in $PWD in the build command.
You'd do this in your Dockerfile like this:
# dockerfile
ARG working_directory
ENV $working_directory/some/subpath
and build like this:
docker build --build-arg "working_directory=$PWD" .
Objective
I have an env variable script file that looks like:
#!/bin/sh
export FOO="public"
export BAR="private"
I would like to source the env variables to be available when a docker image is being built. I am aware that I can use ARG and ENV with build args, but I have too many Env Variables, and I am afraid that will be a lengthy list.
It's worth mentioning that I only need the env variables to install a specific step in my docker file (will highlight in the Dockerfile below), and do not necessarily want them to be available in the built image after that.
What I have tried so far
I have tried having a script (envs.sh) that export env vars like:
#!/bin/sh
export DOG="woof"
export CAT="meow"
My Docker file looks like:
FROM fishtownanalytics/dbt:0.18.1
# Define working directory
# Load ENV Vars
COPY envs.sh envs.sh
CMD ["sh", "envs.sh"]
# Install packages required
CMD ["sh", "-c", "envs.sh"]
RUN dbt deps # I need to env variables to be available for this step
# Exposing DBT Port
EXPOSE 8081
But that did not seem to work. How can I export env variables as a script to the docker file?
In the general case, you can't set environment variables in a RUN command: each RUN command runs a new shell in a new container, and any environment variables you set there will get lost at the end of that RUN step.
However, you say you only need the variables at one specific step in your Dockerfile. In that special case, you can run the setup script and the actual command in the same RUN step:
FROM fishtownanalytics/dbt:0.18.1
COPY envs.sh envs.sh
RUN . ./envs.sh \
&& dbt deps
# Anything that envs.sh `export`ed is lost _after_ the RUN step
(CMD is irrelevant here: it only provides the default command that gets run when you launch a container from the built image, and doesn't have any effect on RUN steps. It also looks like the image declares an ENTRYPOINT so that you can only run dbt subcommands as CMD, not normal shell commands. I also use the standard . to read in a script file instead of source, since not every container has a shell that provides that non-standard extension.)
Your CMD call runs a new shell (sh) that defines those variables and then dies, leaving the current process unchanged. If you want those environment variables to apply to the current process, you could source it:
CMD ["source", "envs.sh"]
The ENV instruction sets the environment variable to the value . This value will be in the environment for all subsequent instructions in the build stage
Below instruction:
ENV PATH=$PATH:$HOME/go/bin
does not append PATH variable
$HOME/go/bin is /root/go/bin
How to append $HOME/go/bin to $PATH? in below docker file
FROM golang:1.14.10
MAINTAINER xyz
ENV GOPATH=
ENV PATH=$PATH:$HOME/go/bin
RUN echo $PATH
Apparently Docker doesn't let you use environment variables defined outside of your Dockerfile within an ENV or ARG declaration.
As a workaround, you can pass the names/directories to your Dockerfile explicitly using ARG:
FROM golang:1.14.10
# set default to `root`
ARG USERNAME=root
ENV PATH=$PATH:/$USERNAME/go/bin
RUN echo $PATH
You can then pass the USERNAME via docker build --build-arg USERNAME=myuser
Depending on your usecase you can also do this using a RUN or ENTRYPOINT.
I think the confusion is this: when you say $HOME in ENV, home isn't defined yet. But when you say RUN echo $HOME, home is defined by the shell in the base image.
PATH is working and causing confusion, because it's defined by the base image you're using with FROM.
ENV is used to define default variables for the image that will be built, and that will be accessible in RUN statements. Think of it this way: the Dockerfile can provide variables to the container, but the container cannot provide variables to the Dockerfile.
Really, I would just hardcode in /root if root is the user you want to run from. The variables provided by the build are meant to be defaults if you want to do something fancy and dynamic, you probably are better off injecting a script into your image, and running that.
As per mentioned here in this link
I tried using this format and it worked with me:
ENV gradle=/opt/gradle/gradle-6.6.1/bin
ENV PATH=${gradle}:${PATH}
Say I have a Dockerfile:
.
.
RUN echo 'source /root/script.sh' >> /etc/bash.bashrc
(The script adds some env variables)
If I:
1) Do this:
docker run -it -v /home/user/script.sh:/root/script.sh image
It takes me to shell where if I call "env" I see the variable set by the script
But if I:
2) Do this:
docker run -it -v /home/user/script.sh:/root/script.sh image env
It prints out env and exits and my variable is missing
What am I missing? I need the variable to exists even if I specify a command/script like "env" at the end of the docker run command
When you run a command like
docker run ... image command
Docker directly runs the command you give; it doesn’t launch any kind of shell, and there’s no opportunity for a .bashrc or similar file to be read.
I’d suggest two things here:
If your program does need environment variables set in some form, set them directly using Dockerfile ENV directives. Don’t try to edit .bashrc or /etc/profile or any other shell dotfile; they won’t reliably get run.
As much as you can install things in places so that you don’t need to change environment variables. For instance, Python supports a “virtual environment” concept that allows an isolated library environment, which requires changing $PATH and similar things; but Docker provides the same isolation on its own, so just install things into the “global” package space.
If you really can’t manage either of these things, then you can write an entrypoint script that sets environment variables and then launches the container’s command. This might look like
#!/bin/sh
. /root/script.sh
exec "$#"
And then you could include this in your Dockerfile like
...
COPY entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
CMD ["/app/myapp"]
(If you need to use docker exec to get a debugging shell in the container, that won’t be a child process of the entrypoint and won’t get its environment variables.)