Given I have a dockerfile like:
ARG MAX_MEMORY_PER_NODE="10GB"
ENV P_MAX_MEMORY_PER_NODE="${MAX_MEMORY_PER_NODE}"
ENTRYPOINT ["/var/p/entrypoint.sh"]
And the entrypoint.sh does something like:
echo "Max memory ${P_MAX_MEMORY_PER_NODE}"
If I were to run the container using the defaults, I would expect
Max Memory 10GB
And that works, but if I run
docker run me/mycontainer:latest -e P_MAX_MEMORY_PER_NODE=1GB
The script still uses the default value (does not print 1GB instead). In fact if I ran:
docker run me/mycontainer:latest -e A_TEST=Hello
And the script had
echo "My test: ${A_TEST}"
It would output
My test:
What am I doing wrong here? What can't I override (or even set) the environment variables being used in the entrypoint script from docker run?
Set the environment variable before the image:
docker run -e "A_TEST=hello" alpine env
For docker-compose
Similar to the this answer: https://stackoverflow.com/a/48915478/11406645
when using docker-compose, and you are passing docker-compose.yaml file an environment variable, or overriding one in env_file; you should pass your environment variable like so: DEBUG=1 docker-compose up
Another problem I faced is that docker commands require sudo permissions:
If you are using sudo before the docker-compose command, add the environment variable after the sudo like so: sudo DEBUG=1 docker-compose up.
The wrong way:
DEBUG=1 sudo docker-compose up
The right way:
sudo DEBUG=1 docker-compose up
Related
When I call...
docker run myImage -e ASPNETCORE_ENVIRONMENT=Development
... the environment in the app is Production.
This however results in the environment set to Development.
docker run myImage -e ENVIRONMENT=Development
Why is this? It's the same with the Compose file that Visual Studio generates. Even when you run it from Visual Studio, the app ignores the value of ASPNETCORE_ENVIRONMENT.
There's an option within ASP.net to remove the prefix when loading env variables but it defaults to true and I can't find anywhere where it's set.
https://github.com/dotnet/aspnetcore/blob/a450cb69b5e4549f5515cdb057a68771f56cefd7/src/Hosting/Hosting/src/WebHostBuilderOptions.cs
if (!options.SuppressEnvironmentConfiguration)
{
configBuilder.AddEnvironmentVariables(prefix: "ASPNETCORE_");
}
https://github.com/dotnet/aspnetcore/blob/259ff381eb80b197eb9d9d2421251e3e1edd40ae/src/Hosting/Hosting/src/GenericHost/GenericWebHostBuilder.cs
Discussed here...
https://github.com/dotnet/aspnetcore/pull/25136
I am not sure what is wrong with your sample - you did not attach your Dockerfile.
Dockerfile itself should declare its environment variables via ENV directive.
Let's see empty dotnet6-based image:
FROM mcr.microsoft.com/dotnet/sdk:6.0
#dev as default value
ENV ASPNETCORE_ENVIRONMENT=Development
ENTRYPOINT ["bash", "-c", "tail -f /dev/null"]
Build image:
docker build --no-cache -t aspnet_env_test:1.0 .
Run container with default environment:
docker run -d aspnet_env_test:1.0
Check environment variable using command:
docker exec <containerId> env
Personally i see this output:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=0af57424ed63
ASPNETCORE_URLS=
DOTNET_RUNNING_IN_CONTAINER=true
DOTNET_VERSION=6.0.1
ASPNET_VERSION=6.0.1
Logging__Console__FormatterName=
DOTNET_GENERATE_ASPNET_CERTIFICATE=false
DOTNET_NOLOGO=true
DOTNET_SDK_VERSION=6.0.101
DOTNET_USE_POLLING_FILE_WATCHER=true
NUGET_XMLDOC_MODE=skip
POWERSHELL_DISTRIBUTION_CHANNEL=PSDocker-DotnetSDK-Debian-11
ASPNETCORE_ENVIRONMENT=Development
HOME=/root
Now stop, remove container and run new one based on the same image, but with the Production environment:
docker rm -f <containerId> && docker run --rm -d -e ASPNETCORE_ENVIRONMENT=Production aspnet_env_test:1.0
Check again the environment of newly created container:
docker exec <containerId> env
Mine output is this one:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=ebdaeb19795a
ASPNETCORE_ENVIRONMENT=Production
ASPNETCORE_URLS=
DOTNET_RUNNING_IN_CONTAINER=true
DOTNET_VERSION=6.0.1
ASPNET_VERSION=6.0.1
Logging__Console__FormatterName=
DOTNET_GENERATE_ASPNET_CERTIFICATE=false
DOTNET_NOLOGO=true
DOTNET_SDK_VERSION=6.0.101
DOTNET_USE_POLLING_FILE_WATCHER=true
NUGET_XMLDOC_MODE=skip
POWERSHELL_DISTRIBUTION_CHANNEL=PSDocker-DotnetSDK-Debian-11
HOME=/root
So, we see here that environment is successfully passed to container file system.
The only thing to be done in your app - just call aforementioned command for your env variables to be caught (with or without prefix parameter overload):
builder.Configuration.AddEnvironmentVariables();
BTW: afaik you might or might not use prefix in environment variables. But prefix itself is removed from env variable names, so you get access to them from your app without prefix if you use it (did not check it).
Guess your error was in using ENV directives without prefix in Dockerfile.
Names in Dockerfile should match exactly.
That works fine.
I have seen the following links to execute multiple commands in docker-compose file:
Docker-Compose + Command
Using Docker-Compose, how to execute multiple commands
docker-compose run multiple commands for a service
which tell us how to execute multiple commands in docker-compose file (also in the docker container).
In order to run sburn/apache-atlas image properly, I have to set some environment variables which exists in /opt/apache-atlas-2.1.0/conf/atlas-env.sh directory.
I have tried the following docker-compose.yml file:
version: "3.3"
services:
atlas:
image: sburn/apache-atlas
container_name: atlas
ports:
- "21000:21000"
volumes:
- "./bash_script:/app"
command: bash -c "
source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
&& chmod 777 /app/import-hive.sh
&& /opt/apache-atlas-2.1.0/bin/atlas_start.py
"
Unfortunately, the first command (I mean source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh) doesn't work. It doesn't have any error but the environment variables such as JAVA_HOME aren't set.
How are you checking that the variables are not set?
Run Docker exec -it atlas bash in the terminal.
Run set in the terminal. It shows all the environment variables.
Check whether the environment variables are set or not.
Your question involves a lot of stuff, if you can narrow it down people can help better. Here are my suggestions to debug it:
bash -exc "
echo home1=$JAVA_HOME
source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
echo home2=$JAVA_HOME
chmod 777 /app/import-hive.sh
echo home3=$JAVA_HOME
/opt/apache-atlas-2.1.0/bin/atlas_start.py
"
If JAVA_HOME is never set, there's something wrong with .sh file, either you fix that file or manually set it with
export JAVA_ENV=/aaa/bbb/ccc
Or defining it in your compose yaml file.
Also the way you're checking for env vars is wrong, running Docker exec -it atlas bash won't run in the same bash as bash -c "source ./opt/apache-a..."
to set enviroment variables you must set this:
environment:
- JAVA_HOME=/usr/bin/java
- OTHER_VARIABLE=example
Or you can set your variables on Dockerfile with:
ENV JAVA_HOME="Your variable"
ENV OTHER_VARIABLE="example"
If you want execute ./opt/apache-atlas-2.1.0/conf/atlas-env.sh script at the container start because this script have all environments that you need, you can include it on entrypoint or Dockerfile with CMD exec
Example:
FROM: source_image
RUN source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
ENTRYPOINT []
To execute commands from your docker-compose try this:
command: sh -c "source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh"
Regards
Sources: docker-compose, run a script after container has started?
I would like to set the LD_LIBRARY_PATH variable based on my working directory using my Dockerfile. I have tried using ENV $PWD/some/subpath but when I inspect the container later using docker exec mycontainer bash -c "env" it shows up as /some/subpath rather than /my/working/dir/some/subpath however I also see that PWD is defined as /my/working/dir/ as I would expect it to be. so why is using $PWD in my Dockerfile not substituting the way I am expecting it to?
From this answer, $PWD is a special environment variable set when running a shell. Unlike RUN commands, ENV commands do not create a shell so PWD is never set.
To get the value of PWD at build time, you could instead use a build-arg and pass in $PWD in the build command.
You'd do this in your Dockerfile like this:
# dockerfile
ARG working_directory
ENV $working_directory/some/subpath
and build like this:
docker build --build-arg "working_directory=$PWD" .
I have a Dockerfile as follows.
ENV SPRING_ENV="local"
ENV APP_OPTS "-Xmx8144m"
RUN echo "/usr/lib/jvm/java-1.8-openjdk/bin/java ${APP_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /apps/demo/demo-fe.jar --spring.config.location=file:///apps/demo/conf/ump.properties -Dspring.profiles.active=${SPRING_ENV} &" > /apps/demo/entrypoint.sh
RUN chmod +x /apps/demo/entrypoint.sh
When I run the dockerfile, I see a file 'entrypoint.sh' with the java command that I specified in the Dockerfile.
But I want to change the java max memory depending on the environment. So I am running like this.
docker run -it <image_id> sh -e "APP_OPTS=-Xmx9144m" -e "SPRING_ENV=dev"
But when I run it, i check the entrypoint.sh, i don't see the environment variables replaced. Am I missing something?
Does it replace only on the fly when I actually run the container?
You need to escape the $ in ${APP_OPTS} (i.e., change it to \${APP_OPTS}) -- during docker build, the variable is getting replaced with the "current" environment variable, which would be whatever is in your env output (otherwise null). Calling docker run ... -e "APP_OPTS=-Xmx9144m" won't do anything at this point because ${APP_OPTS} has been replaced after the image was created.
Otherwise, you could try saving the entrypoint.sh file and put it in the same folder as your Dockerfile instead of having your Dockerfile create it (and use COPY instead to put it where you want it). That way, the ${APP_OPTS} environment variable won't get replaced during docker build
The Dockerfile (and the RUN command) are only executed when you build the image. SPRING_ENV and APP_UMPFE_OPTS are being evaluated only once and during the build.
When you run the image, the --env=KEY=VALUE are passed to the shell (!) running the process defined in the ENTRYPOINT or CMD (which you need but do not have).
You're missing a FROM ... statement near the top of the Dockerfile too.
You will need to define (recommend the shell-form of) ENTRYPOINT that invokes the java runtime, passes the environment variables and runs your code, perhaps (have not tried this):
FROM ???
ENV SPRING_ENV="local"
ENV APP_OPTS "-Xmx8144m"
ENTRYPOINT /usr/lib/jvm/java-1.8-openjdk/bin/java ${APP_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /apps/demo/demo-fe.jar --spring.config.location=file:///apps/demo/conf/ump.properties -Dspring.profiles.active=${SPRING_ENV}
Example:
FROM busybox
ENV DOG=Freddie
ENTRYPOINT echo ${DOG}
Then:
docker build --tag=58208029 --file=./Dockerfile .
docker run -it 58208029:latest
Freddie
docker run -it --env=DOG=Henry 58208029:latest
Henry
HTH!
The entrypoint.sh is being written when you build the image, so that RUN statement won't be executed again when you run the container. So the entrypoint.sh file itself will not be updated.
Another issue is that when you do the docker run, the -e options need to be before the image name and command:
docker run -it -e "APP_OPTS=-Xmx9144m" -e "SPRING_ENV=dev" <image_id> sh
Otherwise those are just being passed as arguments to the entrypoint/command
Also, in your Dockerfile, you probably want single quotes around your entrypoint script so that it doesn't interpolate the values at build time.
RUN echo '/usr/lib/jvm/java-1.8-openjdk/bin/java ${APP_OPTS} -Djava.security.egd=file:/dev/./urandom -jar /apps/demo/demo-fe.jar --spring.config.location=file:///apps/demo/conf/ump.properties -Dspring.profiles.active=${SPRING_ENV} &' > /apps/demo/entrypoint.sh
Then when you run the container, the entrypoint script should read the variable values at run time from the environment.
I have the following entryfile
FROM <image-of-nodejs>
COPY docker/node/entry.sh /var/entries/entry.sh
RUN apt-get update
RUN apt-get install ant -y
CMD ["/var/entries/entry.sh"]
the image is used by a docker-compose file:
version: "3.3"
services:
my_node:
build:
context: ./
dockerfile: docker/node/Dockerfile-build-dev
volumes:
- type: bind
source: ./
target: /var/proj
and the entry.sh file is the following:
#!/bin/bash
export QNAMAKER_SUB_KEY=b13615t
If I then start the image and I enter the docker, I won't find my env variable set:
docker-compose up --force-recreate -d
docker-compose run my_node bash
root#9c081bedde65:/# echo ${QNAMAKER_SUB_KEY}
<empty>
I would prefer to set my variables throug my script in place of the ENV Dockerfile command. What's wrong?
There are a couple of things going on here.
First, docker-compose run doesn't run a command inside the container you started with docker-compose up. It starts a new container to run a one-off command. You probably want docker-compose exec.
The reason you don't see the variable when using docker-compose run is that you are overriding your CMD by providing a new command (bash) on the docker-compose run command line.
You could consider:
Using ENV statements in your Dockerfile.
Using the environment key in your docker-compose.yml
The former will embed the information into your image, while the latter would mean that the variable would be unset if you didn't explicitly set it in your docker-compose.yaml file (or using -e on the docker run command line).
You may be able to accomplish your goal using an ENTRYPOINT script and setting the value there, but that won't impact the environment visible to you when using docker exec (or docker-compose exec).