I have the following entryfile
FROM <image-of-nodejs>
COPY docker/node/entry.sh /var/entries/entry.sh
RUN apt-get update
RUN apt-get install ant -y
CMD ["/var/entries/entry.sh"]
the image is used by a docker-compose file:
version: "3.3"
services:
my_node:
build:
context: ./
dockerfile: docker/node/Dockerfile-build-dev
volumes:
- type: bind
source: ./
target: /var/proj
and the entry.sh file is the following:
#!/bin/bash
export QNAMAKER_SUB_KEY=b13615t
If I then start the image and I enter the docker, I won't find my env variable set:
docker-compose up --force-recreate -d
docker-compose run my_node bash
root#9c081bedde65:/# echo ${QNAMAKER_SUB_KEY}
<empty>
I would prefer to set my variables throug my script in place of the ENV Dockerfile command. What's wrong?
There are a couple of things going on here.
First, docker-compose run doesn't run a command inside the container you started with docker-compose up. It starts a new container to run a one-off command. You probably want docker-compose exec.
The reason you don't see the variable when using docker-compose run is that you are overriding your CMD by providing a new command (bash) on the docker-compose run command line.
You could consider:
Using ENV statements in your Dockerfile.
Using the environment key in your docker-compose.yml
The former will embed the information into your image, while the latter would mean that the variable would be unset if you didn't explicitly set it in your docker-compose.yaml file (or using -e on the docker run command line).
You may be able to accomplish your goal using an ENTRYPOINT script and setting the value there, but that won't impact the environment visible to you when using docker exec (or docker-compose exec).
Related
What I am trying to achieve:
copy a redis.config template to my docker image
read .env variables content and replace the template variables references (such as passwords, ports etc.) with values from .env
start the redis-server with the prepared config file
This way, I can have multiple redis instances setup for local dev, staging and production environments.
I have the following folder structure:
/redis
--.env
--Dockerfile
--redis.conf
This is the Dockerfile:
FROM redis:latest
COPY redis.conf ./
RUN apt-get update
RUN apt-get -y install gettext
RUN envsubst < redis.conf > redisconf
EXPOSE $REDIS_PORT
CMD ["redis-server redis.conf"]
When I go to the redis folder and run docker build -t redis-test . everything builds as expected, but when I do docker run -dp 6379:6379 redis-test afterwards the container crashes with the following error:
Fatal error, can't open config file '/data/redis-server redis.conf': No such file or directory
It seems that the redis.conf file from my folder is not getting correctly copied to my image? But the envsubst runs as expected so it seems that the file is there and the .env variables get overwriten as expected?
What am I doing wrong?
The immediate error is that you've explicitly put the CMD as a single word, so it is interpreted as an executable filename containing a space rather than an executable and a parameter. Split this into two words:
CMD ["redis-server", "redis.conf"]
There's a larger and more complex problem around when envsubst gets run. You're RUNning it as part of the image build, but that means it happens before the container is run and the environment variables are known.
I'd generally address this by writing a simple entrypoint wrapper script. This runs as the main container process, so after the Docker-level container setup happens, and it can see all of the container environment variables. It can run envsubst or whatever other first-time setup is required, and then run exec "$#" to invoke the normal container command.
#!/bin/sh
envsubst < redis.conf.tmpl > redis.conf
exec "$#"
Make this script executable on the host (chmod +x entrypoint.sh), COPY it into your image, and make that the ENTRYPOINT.
ROM redis:latest
COPY redis.conf.tmpl entrypoint.sh ./
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get -y install gettext
ENTRYPOINT ["./entrypoint.sh"]
CMD ["redis-server", "redis.conf"]
I have a docker file
FROM tomcat:9.0.45-jdk8-adoptopenjdk-hotspot
RUN mkdir -p /opt/main
WORKDIR /opt/main
COPY run.sh test.sh runmain.sh /opt/main
RUN chmod +x /opt/main/run.sh && bash /opt/main/run.sh
ENTRYPOINT bash /usr/local/tomcat/bin/runmain.sh && /usr/local/tomcat/bin/catalina.sh run
An env file
ENV_MQ_DETAILS=tcp://10.222.12.12:61616
ENV_DB_HOST=10.222.12.12
runmain.sh file has the following code
#!/bin/bash
echo ${ENV_MQ_DETAILS}
echo ${ENV_DB_HOST}
when I run the docker run command
docker run --env-file .env bootstrap -d
The docker logs shows both env variable values printed.
when I use the docker-compose file
version: "3"
services:
bootstrap:
image: bootstrap
container_name: bootstrap
hostname: bootstrap
ports:
- 8080:8080
and run the command
docker-compose -f docker-compose-bootstrap.yaml --env-file .env bootstrap -d
I get two issues
While running the docker-compose-bootstrap.yaml the environment variables aren't shown in the logs hence can use them in the latest part of the code, why is it so and please help to resolve this (highest priority).
2 In both the cases (docker run and docker-compose run) , it keeps echoing the files in /opt/main/ folder
but nothing to bother though, but why ?
Please help in resolving the above issues.
I have seen the following links to execute multiple commands in docker-compose file:
Docker-Compose + Command
Using Docker-Compose, how to execute multiple commands
docker-compose run multiple commands for a service
which tell us how to execute multiple commands in docker-compose file (also in the docker container).
In order to run sburn/apache-atlas image properly, I have to set some environment variables which exists in /opt/apache-atlas-2.1.0/conf/atlas-env.sh directory.
I have tried the following docker-compose.yml file:
version: "3.3"
services:
atlas:
image: sburn/apache-atlas
container_name: atlas
ports:
- "21000:21000"
volumes:
- "./bash_script:/app"
command: bash -c "
source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
&& chmod 777 /app/import-hive.sh
&& /opt/apache-atlas-2.1.0/bin/atlas_start.py
"
Unfortunately, the first command (I mean source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh) doesn't work. It doesn't have any error but the environment variables such as JAVA_HOME aren't set.
How are you checking that the variables are not set?
Run Docker exec -it atlas bash in the terminal.
Run set in the terminal. It shows all the environment variables.
Check whether the environment variables are set or not.
Your question involves a lot of stuff, if you can narrow it down people can help better. Here are my suggestions to debug it:
bash -exc "
echo home1=$JAVA_HOME
source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
echo home2=$JAVA_HOME
chmod 777 /app/import-hive.sh
echo home3=$JAVA_HOME
/opt/apache-atlas-2.1.0/bin/atlas_start.py
"
If JAVA_HOME is never set, there's something wrong with .sh file, either you fix that file or manually set it with
export JAVA_ENV=/aaa/bbb/ccc
Or defining it in your compose yaml file.
Also the way you're checking for env vars is wrong, running Docker exec -it atlas bash won't run in the same bash as bash -c "source ./opt/apache-a..."
to set enviroment variables you must set this:
environment:
- JAVA_HOME=/usr/bin/java
- OTHER_VARIABLE=example
Or you can set your variables on Dockerfile with:
ENV JAVA_HOME="Your variable"
ENV OTHER_VARIABLE="example"
If you want execute ./opt/apache-atlas-2.1.0/conf/atlas-env.sh script at the container start because this script have all environments that you need, you can include it on entrypoint or Dockerfile with CMD exec
Example:
FROM: source_image
RUN source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh
ENTRYPOINT []
To execute commands from your docker-compose try this:
command: sh -c "source ./opt/apache-atlas-2.1.0/conf/atlas-env.sh"
Regards
Sources: docker-compose, run a script after container has started?
I have a Python app that uses environment variables and I want make dev\prod setup with one Dockerfile and one docker-compose.yml file (only change env file with environment variables).
Here are the files I use to start the application:
Dockerfile:
FROM python:3.7-slim-buster
RUN apt-get update
WORKDIR /usr/src/app
RUN mkdir /usr/src/app/excel_users_dump
COPY ./requirements.txt .
RUN pip install -r requirements.txt
COPY . .
RUN python /usr/src/app/myblumbot/main.py
docker-compose.yml:
version: '3'
services:
bot:
build:
context: .
environment:
ENV: ${ENV}
PYTHONPATH: ${PYTHONPATH}
PYTHONBUFFERED: ${PYTHONBUFFERED}
volumes:
- states:/var/myblumbot_states
volumes:
states:
.env (in the same directory as docker-compose.yml)
PYTHONBUFFERED=1
PYTHONPATH=/usr/src/app
ENV=DEV
When I'm running docker-compose up
command, it builds and tells me that I didn't have some environment variables so application can't start.
env = os.environ['ENV']
KeyError: 'ENV'
But if I add ENV VAR value in Dockerfile, everything works good.
How can I pass variables from docker-compose and .env file?
When you have a setup with both a Dockerfile and a docker-compose.yml file like you show, things run in two phases. During the first phase the image is built, and during the second the container actually gets run. Most of the settings in docker-compose.yml don't have an effect during the build stage; that includes network settings, environment variables, and published ports.
In your Dockerfile you're running your application in a RUN step. That happens as part of the build, not the execution phase; the image that finally gets generated is the filesystem that results after your application exits. Since it's during the build phase, environment variable settings don't take effect.
If you change RUN to CMD, then this will get recorded in the image, and after the build completes, it will run as the main container process with environment variable and other settings.
(In comments you suggest ENTRYPOINT. This will work too, for the same reasons, but it makes a couple of tasks like getting a debug shell harder, and there's a standard Docker first-time setup pattern that needs ENTRYPOINT for its own purposes. I'd prefer CMD here.)
Try to follow the docs:
Compose supports declaring default environment variables in an
environment file named .env placed in the folder where the
docker-compose command is executed (current working directory)
Try to use ENTRYPOINT python /usr/src/app/myblumbot/main.py instead of RUN...
I'm trying to get the variable from the command line using:
sudo docker-compose -f docker-compose-fooname.yml run -e BLABLA=hello someservicename
My file looks like this:
version: '3'
services:
someservicename:
environment:
- BLABLA
image: docker.websitename.com/image-name:latest
volumes:
- /var/www/image-name
command: ["npm", "run", BLABLA]
All of this is so that I can run a script defined by what I use as BLABLA in the command line, I've tried going with official documentation.
Tried several options including:
sudo COMPOSE_OPTIONS="-e BLABLA=hello" docker-compose -f docker-compose-fooname.yml run someservicename
UPDATE:
I have to mention that as it is, I always get:
WARNING: The FAKE_SERVER_MODE variable is not set. Defaulting to a blank string.
Even when I just run the following command (be it remove, stop..):
sudo docker-compose -f docker-compose-fooname.yml stop someservicename
For the record: I'm pulling the image first, I never build it but my CI/CD tool does (gitlab), does this affect it?
I'm using docker-compose version 1.18, docker version 18.06.1-ce, Ubuntu 16.04
That docker-compose.yml syntax doesn't work the way you expect. If you write:
command: ["npm", "run", BLABLA]
A YAML parser will turn that into a list of three strings npm, run, and BLABLA, and when Docker Compose sees that list it will try to run literally that exact command, without running a shell to try to interpret anything.
If you set it to a string, Docker will run a shell over it, and that shell will expand the environment variable; try
command: "npm run $BLABLA"
That having been said, this is a little bit odd use of Docker Compose. As the services: key implies the more usual use case is to launch some set of long-running services with docker-compose up; you might npm run start or some such as a service but you wouldn't typically have a totally parametrizable block with no default.
I might make the docker-compose.yml just say
version: '3'
services:
someservicename:
image: docker.websitename.com/image-name:latest
command: ["npm", "run", "start"]
and if I did actually need to run something else, run
docker-compose run --rm someservicename npm run somethingelse
(or just use my local ./node_modules/.bin/somethingelse and not involve Docker at all)