I have seen the following links to execute multiple commands in docker-compose file:
Docker-Compose + Command
Using Docker-Compose, how to execute multiple commands
docker-compose run multiple commands for a service
which tell us how to execute multiple commands in docker-compose file (also in the docker container).
In order to run sburn/apache-atlas image properly, I have to set some environment variables which exists in /opt/apache-atlas-2.1.0/conf/atlas-env.sh directory.
I have tried the following docker-compose.yml file:
version: "3.3"
services:
atlas:
image: sburn/apache-atlas
container_name: atlas
ports:
- "21000:21000"
volumes:
- "./bash_script:/app"
command: bash -c "
source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
&& chmod 777 /app/import-hive.sh
&& /opt/apache-atlas-2.1.0/bin/atlas_start.py
"
Unfortunately, the first command (I mean source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh) doesn't work. It doesn't have any error but the environment variables such as JAVA_HOME aren't set.
How are you checking that the variables are not set?
Run Docker exec -it atlas bash in the terminal.
Run set in the terminal. It shows all the environment variables.
Check whether the environment variables are set or not.
Your question involves a lot of stuff, if you can narrow it down people can help better. Here are my suggestions to debug it:
bash -exc "
echo home1=$JAVA_HOME
source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
echo home2=$JAVA_HOME
chmod 777 /app/import-hive.sh
echo home3=$JAVA_HOME
/opt/apache-atlas-2.1.0/bin/atlas_start.py
"
If JAVA_HOME is never set, there's something wrong with .sh file, either you fix that file or manually set it with
export JAVA_ENV=/aaa/bbb/ccc
Or defining it in your compose yaml file.
Also the way you're checking for env vars is wrong, running Docker exec -it atlas bash won't run in the same bash as bash -c "source ./opt/apache-a..."
to set enviroment variables you must set this:
environment:
- JAVA_HOME=/usr/bin/java
- OTHER_VARIABLE=example
Or you can set your variables on Dockerfile with:
ENV JAVA_HOME="Your variable"
ENV OTHER_VARIABLE="example"
If you want execute ./opt/apache-atlas-2.1.0/conf/atlas-env.sh script at the container start because this script have all environments that you need, you can include it on entrypoint or Dockerfile with CMD exec
Example:
FROM: source_image
RUN source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
ENTRYPOINT []
To execute commands from your docker-compose try this:
command: sh -c "source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh"
Regards
Sources: docker-compose, run a script after container has started?
Related
When I call...
docker run myImage -e ASPNETCORE_ENVIRONMENT=Development
... the environment in the app is Production.
This however results in the environment set to Development.
docker run myImage -e ENVIRONMENT=Development
Why is this? It's the same with the Compose file that Visual Studio generates. Even when you run it from Visual Studio, the app ignores the value of ASPNETCORE_ENVIRONMENT.
There's an option within ASP.net to remove the prefix when loading env variables but it defaults to true and I can't find anywhere where it's set.
https://github.com/dotnet/aspnetcore/blob/a450cb69b5e4549f5515cdb057a68771f56cefd7/src/Hosting/Hosting/src/WebHostBuilderOptions.cs
if (!options.SuppressEnvironmentConfiguration)
{
configBuilder.AddEnvironmentVariables(prefix: "ASPNETCORE_");
}
https://github.com/dotnet/aspnetcore/blob/259ff381eb80b197eb9d9d2421251e3e1edd40ae/src/Hosting/Hosting/src/GenericHost/GenericWebHostBuilder.cs
Discussed here...
https://github.com/dotnet/aspnetcore/pull/25136
I am not sure what is wrong with your sample - you did not attach your Dockerfile.
Dockerfile itself should declare its environment variables via ENV directive.
Let's see empty dotnet6-based image:
FROM mcr.microsoft.com/dotnet/sdk:6.0
#dev as default value
ENV ASPNETCORE_ENVIRONMENT=Development
ENTRYPOINT ["bash", "-c", "tail -f /dev/null"]
Build image:
docker build --no-cache -t aspnet_env_test:1.0 .
Run container with default environment:
docker run -d aspnet_env_test:1.0
Check environment variable using command:
docker exec <containerId> env
Personally i see this output:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=0af57424ed63
ASPNETCORE_URLS=
DOTNET_RUNNING_IN_CONTAINER=true
DOTNET_VERSION=6.0.1
ASPNET_VERSION=6.0.1
Logging__Console__FormatterName=
DOTNET_GENERATE_ASPNET_CERTIFICATE=false
DOTNET_NOLOGO=true
DOTNET_SDK_VERSION=6.0.101
DOTNET_USE_POLLING_FILE_WATCHER=true
NUGET_XMLDOC_MODE=skip
POWERSHELL_DISTRIBUTION_CHANNEL=PSDocker-DotnetSDK-Debian-11
ASPNETCORE_ENVIRONMENT=Development
HOME=/root
Now stop, remove container and run new one based on the same image, but with the Production environment:
docker rm -f <containerId> && docker run --rm -d -e ASPNETCORE_ENVIRONMENT=Production aspnet_env_test:1.0
Check again the environment of newly created container:
docker exec <containerId> env
Mine output is this one:
PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin
HOSTNAME=ebdaeb19795a
ASPNETCORE_ENVIRONMENT=Production
ASPNETCORE_URLS=
DOTNET_RUNNING_IN_CONTAINER=true
DOTNET_VERSION=6.0.1
ASPNET_VERSION=6.0.1
Logging__Console__FormatterName=
DOTNET_GENERATE_ASPNET_CERTIFICATE=false
DOTNET_NOLOGO=true
DOTNET_SDK_VERSION=6.0.101
DOTNET_USE_POLLING_FILE_WATCHER=true
NUGET_XMLDOC_MODE=skip
POWERSHELL_DISTRIBUTION_CHANNEL=PSDocker-DotnetSDK-Debian-11
HOME=/root
So, we see here that environment is successfully passed to container file system.
The only thing to be done in your app - just call aforementioned command for your env variables to be caught (with or without prefix parameter overload):
builder.Configuration.AddEnvironmentVariables();
BTW: afaik you might or might not use prefix in environment variables. But prefix itself is removed from env variable names, so you get access to them from your app without prefix if you use it (did not check it).
Guess your error was in using ENV directives without prefix in Dockerfile.
Names in Dockerfile should match exactly.
That works fine.
I have a docker file
FROM tomcat:9.0.45-jdk8-adoptopenjdk-hotspot
RUN mkdir -p /opt/main
WORKDIR /opt/main
COPY run.sh test.sh runmain.sh /opt/main
RUN chmod +x /opt/main/run.sh && bash /opt/main/run.sh
ENTRYPOINT bash /usr/local/tomcat/bin/runmain.sh && /usr/local/tomcat/bin/catalina.sh run
An env file
ENV_MQ_DETAILS=tcp://10.222.12.12:61616
ENV_DB_HOST=10.222.12.12
runmain.sh file has the following code
#!/bin/bash
echo ${ENV_MQ_DETAILS}
echo ${ENV_DB_HOST}
when I run the docker run command
docker run --env-file .env bootstrap -d
The docker logs shows both env variable values printed.
when I use the docker-compose file
version: "3"
services:
bootstrap:
image: bootstrap
container_name: bootstrap
hostname: bootstrap
ports:
- 8080:8080
and run the command
docker-compose -f docker-compose-bootstrap.yaml --env-file .env bootstrap -d
I get two issues
While running the docker-compose-bootstrap.yaml the environment variables aren't shown in the logs hence can use them in the latest part of the code, why is it so and please help to resolve this (highest priority).
2 In both the cases (docker run and docker-compose run) , it keeps echoing the files in /opt/main/ folder
but nothing to bother though, but why ?
Please help in resolving the above issues.
I was trying to use variable substitution for a volume declaration. This led to the fallowing error:
ERROR: for php Cannot create container for service php: create .: volume name is too short, names should be at least two alphanumeric characters
Docker's Compose file:
services:
php:
image: php5-apache-composer
volumes:
- ${DIR}:/var/www/html
Is variable substitution supported for the volume declaration? I can use environment variables for any other declaration except this one.
You need to use .env file in folder where docker-compose.yaml is in order to declaring default environment variables for docker-compose.yaml file.
Just create .env file with the following content:
DIR=/var/www/html
Of course you can do that dinamically on each build like:
echo "DIR=/var/www/html" > .env && docker-compose up
I've run export DIR="/var/www/html" and sudo docker-compose run php -e DIR="/var/www/html"
The sudo creates a different environment for the command it's running, the DIR variable doesn't exist there. You can check that with a simple env command:
$ export USER_VAR=test
$ sudo env | grep USER_VAR
$
To get this to work, you'll need to either:
Create a .env and let docker-compose source the variable from that.
Run a shell with sudo -s and run both commands as root from there.
Run a shell as the sudo command, e.g.: sudo /bin/sh -c "DIR=/var/www/html docker-compose run php -e DIR=/var/www/html"
Given I have a dockerfile like:
ARG MAX_MEMORY_PER_NODE="10GB"
ENV P_MAX_MEMORY_PER_NODE="${MAX_MEMORY_PER_NODE}"
ENTRYPOINT ["/var/p/entrypoint.sh"]
And the entrypoint.sh does something like:
echo "Max memory ${P_MAX_MEMORY_PER_NODE}"
If I were to run the container using the defaults, I would expect
Max Memory 10GB
And that works, but if I run
docker run me/mycontainer:latest -e P_MAX_MEMORY_PER_NODE=1GB
The script still uses the default value (does not print 1GB instead). In fact if I ran:
docker run me/mycontainer:latest -e A_TEST=Hello
And the script had
echo "My test: ${A_TEST}"
It would output
My test:
What am I doing wrong here? What can't I override (or even set) the environment variables being used in the entrypoint script from docker run?
Set the environment variable before the image:
docker run -e "A_TEST=hello" alpine env
For docker-compose
Similar to the this answer: https://stackoverflow.com/a/48915478/11406645
when using docker-compose, and you are passing docker-compose.yaml file an environment variable, or overriding one in env_file; you should pass your environment variable like so: DEBUG=1 docker-compose up
Another problem I faced is that docker commands require sudo permissions:
If you are using sudo before the docker-compose command, add the environment variable after the sudo like so: sudo DEBUG=1 docker-compose up.
The wrong way:
DEBUG=1 sudo docker-compose up
The right way:
sudo DEBUG=1 docker-compose up
I have the following entryfile
FROM <image-of-nodejs>
COPY docker/node/entry.sh /var/entries/entry.sh
RUN apt-get update
RUN apt-get install ant -y
CMD ["/var/entries/entry.sh"]
the image is used by a docker-compose file:
version: "3.3"
services:
my_node:
build:
context: ./
dockerfile: docker/node/Dockerfile-build-dev
volumes:
- type: bind
source: ./
target: /var/proj
and the entry.sh file is the following:
#!/bin/bash
export QNAMAKER_SUB_KEY=b13615t
If I then start the image and I enter the docker, I won't find my env variable set:
docker-compose up --force-recreate -d
docker-compose run my_node bash
root#9c081bedde65:/# echo ${QNAMAKER_SUB_KEY}
<empty>
I would prefer to set my variables throug my script in place of the ENV Dockerfile command. What's wrong?
There are a couple of things going on here.
First, docker-compose run doesn't run a command inside the container you started with docker-compose up. It starts a new container to run a one-off command. You probably want docker-compose exec.
The reason you don't see the variable when using docker-compose run is that you are overriding your CMD by providing a new command (bash) on the docker-compose run command line.
You could consider:
Using ENV statements in your Dockerfile.
Using the environment key in your docker-compose.yml
The former will embed the information into your image, while the latter would mean that the variable would be unset if you didn't explicitly set it in your docker-compose.yaml file (or using -e on the docker run command line).
You may be able to accomplish your goal using an ENTRYPOINT script and setting the value there, but that won't impact the environment visible to you when using docker exec (or docker-compose exec).