I have built an image from this Dockerfile:
FROM jenkins
RUN export http_proxy=http://user:pass#x.x.x.x:port
#more things
But when I enter in the container (via exec /bin/bash), http_proxy is not exported.
What can I do?
Thanks
Unless you have a specific reason for using export, Docker has built-in support for using environment variables in a Dockerfile:
ENV http_proxy='http://user:pass#x.x.x.x:port'
Related
I have a Dockerfile ,to deploy node js app with some secret api keys ,that I want to hide from the docker file.
Currently I am using ENV keyword to define env variables like below
FROM node:17
WORKDIR /usr/app
COPY package.json /usr/app/
RUN npm install
COPY . /usr/app
ENV TWILIO_ACCOUNT_SID=""
ENV TWILIO_AUTH_TOKEN=""
ENV OTP_TEXT="This is your Otp"
ENV TWILLIO_SENDER=99999
ENV PORT=8080
ENV DB_URL=""
ENV JWT_SECRET="Some Secrete"
ENV JWT_EXPIRES_IN=30min
ENV OTP_EXPIRE_TIME_SECONDS=150000
ENV AWS_S3_REGION = us-east-2
ENV AWS_S3_BUCKET = gos32
ENV AWS_ACCESS_KEY_ID =""
ENV AWS_SECRET_ACCESS_KEY =""
CMD ["npm", "start"]
Any better way to do that ?
Edit :
Just adding What works me from the answer given by #blami
docker build -t app .
then I ran
docker run --env-file env.txt -d -p 8080:8080 app
docker run with the file option after putting all the env variables in env.txt file
You should not put sensitive data in Dockerfile at all. If your application is configured via environment, you should only provide these variables to container when it is started e.g. manually (using docker -e , --env , and --env-file flags directly on command line) or via your container runtime (which you do not specify in your question):
Kubernetes can manage secrets and expose them via files or environment variables - see documentation with examples
Docker Swarm supports managing secrets out of the box via docker secret command and can expose such secrets as environment variables too - see documentation with examples
Managed cloud providers usually have option to manage secrets and somehow inject them to containers too (or directly expose features of runtime they use).
In any of cases above secrets usually live in secure storage from where they are retrieved only when container starts and injected into it. That way you don't need to have them in Dockerfile. Note that if someone gains access to your running container with application or higher privileges they will be able to retrieve secrets from the environment (as that is how environment variables work).
I need to set today's date as environment variable in my docker container.
What I am trying to do is to get today's date so I can use it to call some API with Logstash's http_poller plugin.
The solution I am thinking about is to get today's date using the command date +%Y%m%d
How can I set the result of this command as an environment variable at run time?
ARG should be what you are looking for:
FROM base
# to be able to use in Dockerfile
ARG now
# to store the value as environment variable in the image
ENV build_date=$now
Now you can build this with
# pass value explicitly
docker build --build-arg now="$(date +%Y%m%d)" .
# pass value from environment
export now="$(date +%Y%m%d)"
docker build --build-arg now .
This still requires to run date on the host since doing this inside the Dockerfile is not possible unfortunately:
The only way to execute arbitrary commands in the build is within a RUN statement; but
The only way to persist a variable into the environment of an image/container is with an ENV statement which can only use environment variables from outside the build
You could use a custom ENTRYPOINT tough and inject the date to the environment from a file:
FROM base
RUN date +%Y%m%d > /build-timestamp
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT /entrypoint.sh
entrypoint.sh:
#!/bin/bash
export BUILD_TIMESTAMP="$(cat /build-timestamp)"
exec "$#"
So according to your answer to my comment, if you want to have the variable set at runtime you need to inject it when you start your container
✗ docker run -e TS=$(date +%Y%m%d) python:3.7-alpine env
TS=20201230
PYTHON_VERSION=3.7.9
HOME=/root
Here is the link to the docker documentation : https://docs.docker.com/engine/reference/run/#env-environment-variables
But you are confusing the Dockerfile with the container. A Dockerfile is a definition for building an image. This image is static. Then you start multiples containers from this base image.
So if your start 3 containers with the same image, according to your question you expect 3 different dates in your containers. So you see why you can't set a date or a command in a Dockerfile. The previous snippet does what you ask.
According to the documentation ENV only supports <key>=<value> entries, I see no way to put it there with a command. I suggest you drop the value into some file then read it when needed. Or, if you have a launch script, using RUN you can add export VAR_NAME=<value> to the launch script so that the variable is automatically added to the environment on launch.
My solution was to first do export command and then running service startup like this
CMD export INSTALL_DATE=$(date +%s) && python3 /product/server.py
Objective
I have an env variable script file that looks like:
#!/bin/sh
export FOO="public"
export BAR="private"
I would like to source the env variables to be available when a docker image is being built. I am aware that I can use ARG and ENV with build args, but I have too many Env Variables, and I am afraid that will be a lengthy list.
It's worth mentioning that I only need the env variables to install a specific step in my docker file (will highlight in the Dockerfile below), and do not necessarily want them to be available in the built image after that.
What I have tried so far
I have tried having a script (envs.sh) that export env vars like:
#!/bin/sh
export DOG="woof"
export CAT="meow"
My Docker file looks like:
FROM fishtownanalytics/dbt:0.18.1
# Define working directory
# Load ENV Vars
COPY envs.sh envs.sh
CMD ["sh", "envs.sh"]
# Install packages required
CMD ["sh", "-c", "envs.sh"]
RUN dbt deps # I need to env variables to be available for this step
# Exposing DBT Port
EXPOSE 8081
But that did not seem to work. How can I export env variables as a script to the docker file?
In the general case, you can't set environment variables in a RUN command: each RUN command runs a new shell in a new container, and any environment variables you set there will get lost at the end of that RUN step.
However, you say you only need the variables at one specific step in your Dockerfile. In that special case, you can run the setup script and the actual command in the same RUN step:
FROM fishtownanalytics/dbt:0.18.1
COPY envs.sh envs.sh
RUN . ./envs.sh \
&& dbt deps
# Anything that envs.sh `export`ed is lost _after_ the RUN step
(CMD is irrelevant here: it only provides the default command that gets run when you launch a container from the built image, and doesn't have any effect on RUN steps. It also looks like the image declares an ENTRYPOINT so that you can only run dbt subcommands as CMD, not normal shell commands. I also use the standard . to read in a script file instead of source, since not every container has a shell that provides that non-standard extension.)
Your CMD call runs a new shell (sh) that defines those variables and then dies, leaving the current process unchanged. If you want those environment variables to apply to the current process, you could source it:
CMD ["source", "envs.sh"]
It is possible to use an environment variable to set the container version inside the Dockerfile?
e.g.:
.env
NODE_VERSION=12.14.0-alpine
Dockerfile
ARG NODE_VERSION
FROM node:${NODE_VERSION}
[...]
And if it is possible, is ok to set the container version like that?
I'm using macOS and Docker v19.03.5
Yes, you can get an access to env variables by using --build-arg docker build command argument.
$ docker build --build-arg NODE_VERSION=12.14.0-alpine
More tutorial information is documented here: https://vsupalov.com/docker-arg-env-variable-guide/
You can also just use ENV inside your Dockerfile to setup ENVs. For example
#Declare Variables
ENV SERVICE_NAME services/clicks
ENV PKG_PATH /root/home/package
ENV GO_VERSION 1.12.0
ENV ALPINE_VERSION 3.9
FROM golang:GO_VERSION-alpineALPINE_VERSION
I have a Python app that uses environment variables and I want make dev\prod setup with one Dockerfile and one docker-compose.yml file (only change env file with environment variables).
Here are the files I use to start the application:
Dockerfile:
FROM python:3.7-slim-buster
RUN apt-get update
WORKDIR /usr/src/app
RUN mkdir /usr/src/app/excel_users_dump
COPY ./requirements.txt .
RUN pip install -r requirements.txt
COPY . .
RUN python /usr/src/app/myblumbot/main.py
docker-compose.yml:
version: '3'
services:
bot:
build:
context: .
environment:
ENV: ${ENV}
PYTHONPATH: ${PYTHONPATH}
PYTHONBUFFERED: ${PYTHONBUFFERED}
volumes:
- states:/var/myblumbot_states
volumes:
states:
.env (in the same directory as docker-compose.yml)
PYTHONBUFFERED=1
PYTHONPATH=/usr/src/app
ENV=DEV
When I'm running docker-compose up
command, it builds and tells me that I didn't have some environment variables so application can't start.
env = os.environ['ENV']
KeyError: 'ENV'
But if I add ENV VAR value in Dockerfile, everything works good.
How can I pass variables from docker-compose and .env file?
When you have a setup with both a Dockerfile and a docker-compose.yml file like you show, things run in two phases. During the first phase the image is built, and during the second the container actually gets run. Most of the settings in docker-compose.yml don't have an effect during the build stage; that includes network settings, environment variables, and published ports.
In your Dockerfile you're running your application in a RUN step. That happens as part of the build, not the execution phase; the image that finally gets generated is the filesystem that results after your application exits. Since it's during the build phase, environment variable settings don't take effect.
If you change RUN to CMD, then this will get recorded in the image, and after the build completes, it will run as the main container process with environment variable and other settings.
(In comments you suggest ENTRYPOINT. This will work too, for the same reasons, but it makes a couple of tasks like getting a debug shell harder, and there's a standard Docker first-time setup pattern that needs ENTRYPOINT for its own purposes. I'd prefer CMD here.)
Try to follow the docs:
Compose supports declaring default environment variables in an
environment file named .env placed in the folder where the
docker-compose command is executed (current working directory)
Try to use ENTRYPOINT python /usr/src/app/myblumbot/main.py instead of RUN...