How to using env variables in Dockerfile?
not args
This is my Dockerfile now:
FROM node:12
I want Dockerfile:
FROM node:$version
How can I do it? If it is not possible, then I will create a feature request for Docker.
You need to add as ARGs under a different place from the ENV variables. You can use them in the compose, just a little differently.
ARGs will be used during the build only, if you need them to available for the build & again while running the container for some reason you will have to set both an ARG as well as an ENV variable. {{{During the Build stage they will be called in the exact same manner}}}. ENV are set to replace defaults when running a container, so the build's ENV TZ=Europe/London will be replaced by the Compose's Environment: TZ=America/New_York
services:
my_node_version:
image: my/node
build:
context: .
dockerfile: ../node.Dockerfile
args:
- VERSION=12
- NEEDED_IN_BUILD_AND_RUN=SomeStrangeUseCase
environment:
- ONLY_USED_IN_IMAGE=MyImageWorks
- NEEDED_IN_BUILD_AND_RUN=SomeStrangeUseCase
- OVERRIDE_BUILD_ENV="Will replace the ENV from the Build"
I'm pretty sure that fulfills all the intent from the question (you asked to not use args I assume because you wanted them set in the docker-compose.yaml rather than the dockerfile). You can do this, you just have to set them in a different way.
ENV cannot be used in the build from the Compose (ENV variables set in the Dockerfile just set the defaults that the container will use if none are declared in the OS, the Compose, or anywhere Docker gets them from), so that simply isn't a possibility, so if you need it to be ENV specifically for some reason then the answer is simply "You can't" end of discussion. But I don't see anything that could possibly accomplish, & no reason using ARGs wouldn't accomplish exactly what you want. You just need to place them in a different section, but they should function as needed
Related
UPDATE
It appears the problem is specifically related to the RUN command in the Dockerfile. If I remove it, the build works fine and the environment variables are clearly being picked up since the password gets applied and I can connect using it. Not sure why the login fails in the RUN command, I've seen many examples using similar code.
I'm working on a very basic docker compose file to setup a dev environment for an app and I started with the database server, which is MS SQL. Here's what the docker-compose.yml file looks like:
version: '3.8'
services:
mssql:
build:
context: .
dockerfile: docker/mssql/Dockerfile
ports:
- '1434:1433'
environment:
ACCEPT_EULA: "Y"
SA_PASSWORD: "YourStrong!Passw0rd"
volumes:
- mssql-data:/var/opt/mssql
As you can see from my dockerfile path, that's in a sub-path and looks like this:
FROM mcr.microsoft.com/mssql/server:2019-latest
COPY ./docker/mssql/TESTDB.bak /var/opt/mssql/backup/TESTDB.bak
RUN ( /opt/mssql/bin/sqlservr --accept-eula & ) | grep -q "Service Broker manager has started" && /opt/mssql-tools/bin/sqlcmd -S localhost,1433 -U SA -P "YourStrong!Passw0rd" -Q 'RESTORE DATABASE TESTDB FROM DISK = "/var/opt/mssql/backup/TESTDB.bak" WITH MOVE "TESTDB_Data" to "/var/opt/mssql/data/TESTDB.mdf", MOVE "TESTDB_Log" to "/var/opt/mssql/data/TESTDB_log.ldf"'
(Yes, I realize that the password in the RUN command is redundant, I had tried to use a variable there earlier and since it wasn't working I hard coded it.)
When I run docker-compose up -d, I always get this error: Login failed for user 'SA'
I wasted way too much time thinking there was actually something wrong with the password until I realized that if I add the environment variables directly in the Dockerfile, it works. So in my Dockerfile, above the RUN command, I can just do this:
ENV ACCEPT_EULA=Y
ENV SA_PASSWORD=YourStrong!Passw0rd
So I concluded that my environment variables simply aren't being read. I tried with quotes, without quotes, using env_file instead, nothing seems to work. I also tried the following format, no luck:
environment
- ACCEPT_EULA=Y
- SA_PASSWORD=YourStrong!Passw0rd
I also tried using MSSQL_SA_PASSWORD instead of SA_PASSWORD, as well as having both in there. I assumed that was unlikely to be the problem though given SA_PASSWORD works fine. Lastly, I tried using a 2017 image in case it was image specific, that didn't work either.
I'm assuming it must be something silly I'm missing. I saw a lot of talk of .env in the root being different, but if I understood correctly people go wrong with that when they try to use environment values in their docker-compose.yml file, which is not what I'm doing here. So I'm about ready to lose my mind on this as it seems like such a simple, basic thing.
I think you're confusing the ENV statement in Dockerfile with the environment variables set when running an image. The key is still in the details of the docs. It notes that they are the same as saying docker run -e, not docker build.
What's causing more confusion, when you use ENV, you are setting defaults for when the image runs later:
https://docs.docker.com/engine/reference/builder/#env
If you haven't yet, I very much recommend getting familiar with building and running your image with docker run and docker build before moving on to compose, it's much less confusing that way.
The issue with your build here stems from a confusion between the build-time and run-time environment variables: with the environment or env_file properties you specify the environment variables to be set for the service container.
But the RUN command in your Dockerfile is executed at the build-time of the image! To pass variables when building a new image you should use build args instead, as you already mentioned in your comment:
services:
mssql:
build:
context: .
dockerfile: docker/mssql/Dockerfile
args:
SA_PASSWORD: "YourStrong!Passw0rd"
# ...
With this you can use the SA_PASSWORD as a build ARG:
FROM mcr.microsoft.com/mssql/server:2019-latest
COPY ./docker/mssql/TESTDB.bak /var/opt/mssql/backup/TESTDB.bak
ARG SA_PASSWORD
RUN ( /opt/mssql/bin/sqlservr --accept-eula & ) | grep -q "Service Broker manager has started" && /opt/mssql-tools/bin/sqlcmd -S localhost,1433 -U SA -P "$SA_PASSWORD" -Q 'RESTORE DATABASE TESTDB FROM DISK = "/var/opt/mssql/backup/TESTDB.bak" WITH MOVE "TESTDB_Data" to "/var/opt/mssql/data/TESTDB.mdf", MOVE "TESTDB_Log" to "/var/opt/mssql/data/TESTDB_log.ldf"'
If you want to move the actual password to a .env file you can use variable substitution in the compose.yml:
services:
mssql:
build:
# ...
args:
SA_PASSWORD: "$SA_PASSWORD"
# ...
In your docker-compose.yml, have you tried:
- ACCEPT_EULA=Y
- SA_PASSWORD=YourStrong!Passw0rd
Both responses above are fine, just a few more things:
SA_PASSWORD is deprecated instead use MSSQL_SA_PASSWORD
It is always nice to define .env files with the variables for instance:
sapassword.env
MSSQL_SA_PASSWORD=YourStrong!Passw0rd
sqlserver.env
ACCEPT_EULA=Y
MSSQL_DATA_DIR=/var/opt/sqlserver/data
MSSQL_LOG_DIR=/var/opt/sqlserver/log
MSSQL_BACKUP_DIR=/var/opt/sqlserver/backup
And in docker-compose.yml instance the env files the following way:
environment:
- sqlserver.env
- sapassword.env
Is there any way to create local variables in Dockerfile that only available during the build process? What I can see is if I define a variable with the ENV keyword, then it will be available later in the image as an exported environment variable. But I would like to have a "technical" variable with build scope only.
I would like to avoid repetition in my Doclerfile so I would like to have a variable available only from the Dockerfile:
ENV MY_JAR=myJar.jar
COPY bin/$MY_JAR $ORACLE_HOME/user_projects/domains/$DOMAIN_NAME/lib/
COPY bin/$MY_JAR $ORACLE_HOME/wlserver/server/lib/mbeantypes/
But the MY_JAR variable appears in the container. I do not need it there. It just confuses users. Can I do this somehow?
Use ARG instead of ENV
ARG MY_JAR=myJar.jar # ARG is only available during the build of a Docker image
COPY bin/$MY_JAR $ORACLE_HOME/user_projects/domains/$DOMAIN_NAME/lib/
COPY bin/$MY_JAR $ORACLE_HOME/wlserver/server/lib/mbeantypes/
see also ARG or ENV, which one to use in this case?
You can use the --build-arg parameter to pass environment variables that lives just during docker building process.
So your docker build command will look something like this
docker build --build-arg HTTP_PROXY=http://10.20.30.2:1234 -t sample:v1 .
Where HTTP_PROXY is just available during the build process.
I'm trying to construct a Docker container in which I have two environment variables set to the same thing. I have
version: "3.2"
services:
sql-server-db:
image: mcr.microsoft.com/mssql/server:latest
ports:
- 3900:1433
env_file: ./tests/.test_env
...
command: /bin/bash /my-app/my-script.sh
and then in my tests/.test_env file I have
MY_DB_PASSWORD=reallylongpassword
SA_PASSWORD=${MY_DB_PASSWORD}
I would like to set the "MY_DB_PASSWORD" and "SA_PASSWORD" env vars to the same thing, however, the above doesn't do it, because "SA_PASSWORD" seems to be set to the literal string "${MY_DB_PASSWORD}". How do I set my two variables to the same thing without hard-coding the "reallylongpassword" string twice?
This is not possible using env file, simply do in your script like this. This way is more flexible as it will take the value if not set.
SA_PASSWORD="${SA_PASSWORD:-$MY_DB_PASSWORD}"
If you are worry about password security, you should use docker secret but you need to run in Swarm mode.
Here is the dockerfile used by the image. You can define your own CMD to override the behavior and take your env or use the trick to take default value.
CMD .\start -sa_password $env:sa_password -ACCEPT_EULA $env:ACCEPT_EULA -attach_dbs \"$env:attach_dbs\" -Verbose
//using you MY_DB_PASSWORD instead of SA_PASSWORD
CMD .\start -sa_password $env:my_db_password -ACCEPT_EULA $env:ACCEPT_EULA -attach_dbs \"$env:attach_dbs\" -Verbose
After reading the config point of the 12 factor app I decided to override my config file containing default value with environment variable.
I have 3 Dockerfiles, one for an API, one for a front-end and one for a worker. I have one docker-compose.yml to run those 3 services plus a database.
Now I'm wondering if I should define the environment variables in Dockerfiles or docker-compose.yml ? What's the difference between using one rather than another ?
See this:
You can set environment variables in a service’s containers with the 'environment' key, just like with docker run -e VARIABLE=VALUE ...
Also, you can use ENV in dockerfile to define a environment variable.
The difference is:
Environment variable define in Dockerfile will not only used in docker build, it will also persist into container. This means if you did not set -e when docker run, it will still have environment variable same as defined in Dockerfile.
While environment variable define in docker-compose.yaml just used for docker run.
Maybe next example could make you understand more clear:
Dockerfile:
FROM alpine
ENV http_proxy http://123
docker-compose.yaml:
app:
environment:
- http_proxy=http://123
If you define environment variable in Dockerfile, all containers used this image will also has the http_proxy as http://123. But the real situation maybe when you build the image, you need this proxy. But, the container maybe run by other people maybe not need this proxy or just have another http_proxy, so they had to remove the http_proxy in entrypoint or just change to another value in docker-compose.yaml.
If you define environment variable in docker-compose.yaml, then user could just choose his own http_proxy when do docker-compose up, http_proxy will not be set if user did not configure it docker-compose.yaml.
I have a docker-compose file that allows me to pass the environment variables as a file (.env file). As I have multiple ENV variables, Is there any option in Dockerfile like env_file in docker-compose for passing multiple environment variables during docker build?
This is the docker-compose.yml
services:
web:
image: "node"
links:
- "db"
env_file: "env.app"
AFAIK, there is no such way to inject environment variables using a file during the build step using Dockerfile. However, in most cases, people do end up using an entrypoint script & injecting variables during the docker run or docker-compose up.
In case it's a necessity you might need to write a shell wrapper which will change the values in the Dockerfile dynamically by taking a key-value pair text file as an input or make it something as below but the ENV file name need to be included in Dockerfile.
COPY my-env-vars /
RUN export $(cat my-env-vars | xargs)
It's an open issue - https://github.com/moby/moby/issues/28617
PS - You need to be extra careful while using this approach because the secrets are baked into the image itself.