I want to set the environment variables from .env file to the container.
But host environment variables are set to the container instead of .env file.
Setting files as follows.
host # ls -la:
docker-compose.yml
Dockerfile
.env
host environment variable:
JAVA_HOME="/usr/lib/jvm/java-8-oracle/jre"
LANG="en_US.UTF-8"
.env file:
TZ=Asia/Tokyo
LANG=ja_JP.utf8
ORACLE_URL=http://download.oracle.com/otn-pub/java/jdk/8u112-b15/jdk-8u112-linux-x64.rpm
ORACLE_VERSION=8u112
JAVA_HOME=/usr/java/jdk1.8.0_112/jre
docker-compose.yml:
version: '2'
services:
oracle8:
build:
context: ./
args:
ORACLE_URL: ${ORACLE_URL}
ORACLE_VERSION: ${ORACLE_VERSION}
environment:
- TZ=$TZ
- LANG=$LANG
- JAVA_HOME=$JAVA_HOME
image: oracle8
privileged: true
tty: true
container_name: oracle8
hostname: oracle8
Dockerfile:
FROM centos:latest
ARG ORACLE_URL
ARG ORACLE_VERSION
ARG JAVA_HOME
ENV JAVA_HOME ${JAVA_HOME}
RUN set -x \
&& yum update -y \
&& yum install -y wget tar \
&& yum clean all \
&& wget --no-check-certificate --no-cookies --header "Cookie: oraclelicense=accept-securebackup-cookie" ${ORACLE_URL} \
&& rpm -ivh jdk-"${ORACLE_VERSION}"-linux-x64.rpm \
&& rm -f jdk-"${ORACLE_VERSION}"-x64.rpm \
&& echo ${JAVA_HOME}
CMD ["tail", "-f", "/dev/null"]
and then, docker-compose build & up
host # docker-compose build
host # docker-compose up -d
host # docker exec -it oracle8 bash
oracle8 # export
JAVA_HOME="/usr/lib/jvm/java-8-oracle/jre"
LANG="en_US.UTF-8"
TZ="Asia/Tokyo"
$TZ is set from .env file because host is not set $TZ.
$LANG and $JAVA_HOME are not set "ja_JP.utf8" and "/usr/java/jdk1.8.0_112" of .env file but "en_US.UTF-8" and "/usr/lib/jvm/java-8-oracle/jre" are set of host environment variables.
If I write it directly in a docker-compose.yml, it's ok.
but I want to handle it in .env file.
The variables are prioritized in the following order:
compose file itself
shell variables
.env file
Dockerfile
This can be found: here
Environment variables set in the shell take precedence of those set in the .env file: https://docs.docker.com/compose/environment-variables/#/the-env-file
What you could do is to rename the conflicting variable names in your .env file and rename them again in your Dockerfile:
.env:
DOCKER_JAVA_HOME="/usr/lib/jvm/java-8-oracle/jre"
Dockerfile:
ARG DOCKER_JAVA_HOME
ENV JAVA_HOME=${DOCKER_JAVA_HOME}
Related
As mentioned above, I have a file owner problem with Laravel setup.
https://www.fullstaq.com/knowledge-hub/blogs/docker-and-the-host-filesystem-owner-matching-problem
This site has the solution.but the example of dockerfile in this site is for debian.
I want to create apache dockerfile.So Not suitable for what I want to do.
below is my dockerfile and docker-compose.yml.
# Dockerfile
FROM php:7.3-apache
COPY ./000-default.conf /etc/apache2/sites-available/000-default.conf
COPY --from=composer:2.0 /usr/bin/composer /usr/bin/composer
RUN apt-get update \
&& apt-get -y install \
git \
zip \
unzip \
vim \
&& docker-php-ext-install pdo_mysql bcmath mbstring \
&& a2enmod rewrite
WORKDIR /var/www/html
EXPOSE 80
# I added below the code,but An error occurred with the code.
ARG uid
RUN useradd -G www-data,root -u $uid -d /home/devuser devuser
RUN mkdir -p /home/devuser/.composer && \
chown -R devuser:devuser /home/devuser
# docker-compose.yml
version: '3'
services:
backend:
build:
context: ./docker/php
dockerfile: Dockerfile
ports:
- "80:80"
volumes:
- ./src/backend:/var/www/html
# - /etc/group:/etc/group:ro
# - /etc/user:/etc/user:ro
depends_on:
- db
db:
build:
context: ./docker/mysql
dockerfile: Dockerfile
command: --default-authentication-plugin=mysql_native_password
ports:
- "3306:3306"
volumes:
- ./docker/mysql/data:/var/lib/mysql
environment:
- MYSQL_ROOT_PASSWORD=pass
What is the wrong in my dockerfile?
Thank you for your help.
The error your getting here is
useradd: invalid user ID '-d'
This is because with
ARG uid
RUN useradd -G www-data,root -u $uid -d /home/devuser devuser
you specified a build ARG which is used in the following RUN directive. But when building the image with docker-compose there is no actual value specified for that ARG.
So when building it is substituted with an empty string and the executed command becomes
useradd -G www-data,root -u -d /home/devuser devuser
You need to specify a value for uid (i.e. the uid of your user on the host system) with the args attribute in the docker-compose.yml:
services:
backend:
build:
context: ./docker/php
dockerfile: Dockerfile
args:
uid: 1000
# or using env variables:
# uid: $HOST_UID
I want to Running docker-compose inside a dockerized jenkins container.
but my docker-compose file is work on my local but when I try to CD in jenkins it doesn't work
with this error
[0m[32mgunicorn-backend |[0m sh: 0: Can't open /code/gunicorn/gunicorn_start.sh
jenkinsfile
#!groovy
node {
environment {
Django_secret_key = credentials('Django_secret_key')
}
stage("Checkout") {
checkout scm
}
stage('Stop previous containers') {
dir('backend') {
withEnv(["PATH=$PATH:/usr/local/bin"]){
sh """
docker-compose -p LBS_Platform down
"""
}
}
}
stage('Run current containers') {
dir('backend') {
withEnv(["PATH=$PATH:/usr/local/bin"]){
sh """
docker-compose -p LBS_Platform up --build
"""
}
}
}
}
jenkins docker, docker-compose
# dockerfile
FROM jenkins/jenkins:lts
ARG HOST_UID=1004
ARG HOST_GID=999
USER root
RUN curl -fsSL https://get.docker.com -o get-docker.sh && sh get-docker.sh
RUN curl -L "https://github.com/docker/compose/releases/download/1.28.6/docker-compose-$(uname -s)-$(uname -m)" -o /usr/local/bin/docker-compose ; chmod +x /usr/local/bin/docker-compose
RUN usermod -u $HOST_UID jenkins
RUN groupmod -g $HOST_GID docker
RUN usermod -aG docker jenkins
USER jenkins
# docker-compose file
version: "3"
services:
jenkins:
privileged: true
build:
context: ./
container_name: jenkins
restart: always
user: root
ports:
- 8083:8080
- 50003:50000
expose:
- "8080"
- "50000"
volumes:
- "/var/run/docker.sock:/var/run/docker.sock"
- "./jenkins_home:/var/jenkins_home"
environment:
TZ: "Asia/Seoul"
volumes:
jenkins_home:
driver: local
docker-compose what i want to run on jenkins coniatiner
# dockerfile
FROM python:3.9
ENV PYTHONUNBUFFERED 1
RUN apt-get -y update
ARG Django_secret_key
ENV Django_secret_key $Django_secret_key
ENV BOARD_DEBUG 1
# 유저, 그룹 나중에 수정 TODO
# the user to run as
ENV USER root
# how many worker processes should Gunicorn spawn
ENV NUM_WORKERS 3
# which settings file should Django use
ENV DJANGO_SETTINGS_MODULE backend.settings
# WSGI module name
ENV DJANGO_WSGI_MODULE backend.wsgi
ENV PORT 8000
RUN echo "Starting $NAME as $(whoami)"
RUN mkdir /code
WORKDIR /code
COPY . /code/
RUN apt-get update \
&& DEBIAN_FRONTEND=noninteractive apt-get install -y netcat
RUN chmod 755 /code/gunicorn/gunicorn_start.sh
ENTRYPOINT ["sh", "/code/gunicorn/gunicorn_start.sh"]
# docker-compose file
networks:
app-tier:
driver: bridge
services:
gunicorn-backend:
restart: always
container_name: gunicorn-backend
build:
context: .
args:
Django_secret_key: "${Django_secret_key}"
command: bash -c "pipenv run python manage.py runserver 0.0.0.0:8000"
volumes:
- .:/code
networks:
- app-tier
ports:
- "8000:8000"
nginx-backend:
restart: always
container_name: nginx-backend
image: nginx:latest
volumes:
- ./nginx/config:/etc/nginx/conf.d
- ./nginx/logs:/var/backend-logs
expose:
- "80"
- "443"
ports:
- "80:80"
- "443:443"
networks:
- app-tier
depends_on:
- gunicorn-backend
environment:
- NGINX_HOST=0.0.0.0
- NGINX_PORT=80
# gunicorn/gunicorn_start.sh
#!/bin/bash
# Name of the application
NAME="backend"
# https://stackoverflow.com/questions/4774054/reliable-way-for-a-bash-script-to-get-the-full-path-to-itself
SCRIPT_PATH=$(dirname `which $0`)
# Django project directory
# . 경로
DJANGODIR=$SCRIPT_PATH
# /Users/Han/programming/DjangoCRUDBoard
PARENT_PATH=$(cd $SCRIPT_PATH ; cd .. ; pwd)
# we will communicte using this unix socket
SOCKFILE=$PARENT_PATH/run/gunicorn.sock
echo $PARENT_PATH
# Activate the virtual environment
export DJANGO_SETTINGS_MODULE=$DJANGO_SETTINGS_MODULE
export PYTHONPATH=$DJANGODIR:$PYTHONPATH
# Create the run directory if it doesn't exist
RUNDIR=$(dirname $SOCKFILE)
test -d $RUNDIR || mkdir -p $RUNDIR
cd $PARENT_PATH
# # DB 연결될때까지 블로킹 (미그레이션은 DB가 연결되어야 가능하다)
# while ! nc -z database 5432; do sleep 1; done;
pip install --upgrade pip
pip install pipenv
pipenv install
pipenv run python manage.py makemigrations
pipenv run python manage.py migrate
# Start your Django Unicorn
# Programs meant to be run under supervisor should not daemonize themselves (do not use --daemon)
# pipenv 사용
exec pipenv run gunicorn ${DJANGO_WSGI_MODULE}:application \
-b 0.0.0.0:$PORT \
--name $NAME \
--workers $NUM_WORKERS \
--user=$USER \
--bind=unix:$SOCKFILE \
--log-level=debug \
--log-file=-
docker & docker-compose work on jenkins container but I don't understand why
[0m[32mgunicorn-backend |[0m sh: 0: Can't open /code/gunicorn/gunicorn_start.sh
this error showed.....
Is there any solution to solve this problem?!
I have the following docker file
FROM alpine:3.10 as builder
# CAMUNDA VARIABLES
ARG VERSION=7.12.0
ARG DISTRO=tomcat
ARG SNAPSHOT=false
ARG EE=true
#Understand difference between ARG and ENV:
#https://vsupalov.com/docker-arg-vs-env/
#AZURE SQL VARIABLES, DEFAULT SETTINS ARE DEV DB Environment
#Be sure database and tables are created upfront
#Use docker build . -t xx/camundawithbasicdev
#Use docker build . -t xx/camundawithbasicauthqa --build-arg SQLSERVER=a_value DBPASSWORD=a_value DBUSERNAME=a_value
#Use docker build . -t xx/camundawithbasicauthprod --build-arg some_variable_name=a_value
#And replace variables as needed.
ARG SQLSERVER=xx
ARG DBPASSWORD=Hxx
ARG DBUSERNAME=xx
ARG DATABASENAME=xx
RUN apk add --no-cache \
ca-certificates \
maven \
tar \
wget \
xmlstarlet
COPY settings.xml download.sh camunda-tomcat.sh camunda-wildfly.sh /tmp/
RUN /tmp/download.sh
#Enable Basic AUTH
COPY web.xml /camunda/webapps/engine-rest/WEB-INF/web.xml
##### FINAL IMAGE #####
FROM alpine:3.10
ARG VERSION=7.12.0
ENV CAMUNDA_VERSION=${VERSION}
ENV DB_DRIVER=com.microsoft.sqlserver.jdbc.SQLServerDriver
ENV DB_URL=jdbc:sqlserver://${SQLSERVER}.database.windows.net:1433;database=${DATABASENAME};user=${DBUSERNAME};password={DBPASSWORD};encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=30;
ENV DB_USERNAME=${DBUSERNAME}
ENV DB_PASSWORD=${DBPASSWORD}
ENV DB_CONN_MAXACTIVE=20
ENV DB_CONN_MINIDLE=5
ENV DB_CONN_MAXIDLE=20
ENV DB_VALIDATE_ON_BORROW=true
ENV DB_VALIDATION_QUERY="SELECT 1"
ENV SKIP_DB_CONFIG=
ENV WAIT_FOR=
ENV WAIT_FOR_TIMEOUT=120
ENV TZ=UTC
ENV DEBUG=TRUE
ENV JAVA_OPTS="-Xmx768m -XX:MaxMetaspaceSize=256m"
ENV JMX_PROMETHEUS=false
EXPOSE 8080 8000
# Downgrading wait-for-it is necessary until this PR is merged
# https://github.com/vishnubob/wait-for-it/pull/68
RUN apk add --no-cache \
bash \
ca-certificates \
openjdk11-jre-headless \
tzdata \
tini \
xmlstarlet \
&& wget -O /usr/local/bin/wait-for-it.sh \
"https://raw.githubusercontent.com/vishnubob/wait-for-it/a454892f3c2ebbc22bd15e446415b8fcb7c1cfa4/wait-for-it.sh" --no-check-certificate \
&& chmod +x /usr/local/bin/wait-for-it.sh
RUN addgroup -g 1000 -S camunda && \
adduser -u 1000 -S camunda -G camunda -h /camunda -s /bin/bash -D camunda
# end ssh config
WORKDIR /camunda
USER camunda
#MSSQL SERVER JDBC DRIVER INSTALL
COPY mssql-jdbc-7.2.2.jre11.jar /camunda/lib/
ENTRYPOINT ["/sbin/tini", "--"]
CMD "./camunda.sh"
COPY --chown=camunda:camunda --from=builder /camunda .
However when I run the image, it looks like the variables are not replaced on the connection string part
what am I missing?
You're setting the arguments on a different stage. As far as I know, it's not possible to pass them between stages. For example:
This won't work
FROM alpine:3.10 as builder
ARG TEST1=1234
FROM alpine:3.10
ENV TEST2=blablah-${TEST1}
if you print TEST2 you'll get blablah-
This will:
FROM alpine:3.10 as builder
ARG TEST1=1234
FROM alpine:3.10
ARG TEST1=98765
ENV TEST2=blablah-${TEST1}
if you print TEST2 you'll get `blahblah-98765
However, I would strongly advice against this way as it'll store your passwords and so on as default env vars in the Docker image you're building. I would pass the connection details when you run docker either with docker run -e TEST1=1234 or even better, with a docker-compose file.
I would have Dockerfile like this one (very simplified, only with the env vars) Normally, when I need to construct variables like that I do so in the entrypoint.sh
So my entrypoint looks like:
#!/bin/bash
export DB_URL="jdbc:sqlserver://${SQLSERVER}.database.windows.net:1433;database=${DB_NAME};user=${DB_USERNAME};password=${DB_PASSWORD};encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=0;"
echo $DB_URL
echo "Running: '$#'"
exec $#
My Dockerfile looks like:
FROM alpine:3.10 as builder
ARG VERSION
ARG DISTRO
ARG SNAPSHOT
ARG EE
FROM alpine:3.10
ARG VERSION
ENV CAMUNDA_VERSION=${VERSION}
ENV DB_DRIVER=com.microsoft.sqlserver.jdbc.SQLServerDriver
ENV DB_CONN_MAXACTIVE=20
ENV DB_CONN_MINIDLE=5
ENV DB_CONN_MAXIDLE=20
ENV DB_VALIDATE_ON_BORROW=true
ENV DB_VALIDATION_QUERY="SELECT 1"
ENV SKIP_DB_CONFIG=
ENV WAIT_FOR=
ENV WAIT_FOR_TIMEOUT=120
ENV TZ=UTC
ENV DEBUG=TRUE
ENV JAVA_OPTS="-Xmx768m -XX:MaxMetaspaceSize=256m"
ENV JMX_PROMETHEUS=false
RUN apk update && apk add bash
ADD entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
And my docker-compose.yml
version: '2.3'
services:
so-example:
image: stackoverflow-example
build:
context: .
args:
- VERSION
- DISTRO
- SNAPSHOT
environment:
- SQLSERVER
- DB_USERNAME
- DB_PASSWORD
- DB_NAME
Then, to build the new image I run:
docker-compose build --build-arg VERSION=7.12.0 --build-arg DISTRO=tomcat --build-arg SNAPSHOT=false --build-arg EE=true
When I need to run it I export the values I need. For instance:
export SQLSERVER=myhost
export DB_USERNAME=db_user
export DB_PASSWORD=mysecretpassword
export DB_NAME=mydb
And then run it with compose. For example, if I run
docker-compose run so-example env
you should see that DB_URL is
jdbc:sqlserver://myhost.database.windows.net:1433;database=mydb;user=db_user;password=mysecretpassword;encrypt=true;trustServerCertificate=false;hostNameInCertificate=*.database.windows.net;loginTimeout=0;
I hope this helps
In the WordPress Dockerfile, there's a VOLUME /var/www/html statement. If I understand correctly, this means that the WordPress files (in /var/www/html) should be mapped to the directory on my host containing the docker-compose.yml BUT this is not happening. Do you know why?
I created my own WordPress Dockerfile that extends the original WordPress Dockerfile where you'll find said VOLUME /var/www/html statement on line 44 (https://github.com/docker-library/wordpress/blob/b3739870faafe1886544ddda7d2f2a88882eeb31/php7.2/apache/Dockerfile).
I even tried to add the VOLUME /var/www/html statement at the bottom of my Dockerfile as you can see in my Dockerfile below. I added it just in case but I don't think anything is going wrong in there.
FROM wordpress:4.9.8-php7.2-apache
##########
# XDebug #
##########
# Install
RUN pecl install xdebug-2.6.1; \
docker-php-ext-enable xdebug
# Configure
RUN echo "error_reporting = E_ALL" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini; \
echo "display_startup_errors = On" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini; \
echo "display_errors = On" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini; \
echo "xdebug.idekey=\"PHPSTORM\"" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini; \
echo "xdebug.remote_port=9000" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini; \
echo "xdebug.remote_enable=1" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini; \
echo "xdebug.remote_host=docker.for.win.localhost" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini
#RUN echo "xdebug.remote_autostart=1" >> /usr/local/etc/php/conf.d/docker-php-ext-xdebug.ini ##
###########
# PHPUnit #
###########
RUN apt-get update; \
apt-get install wget
RUN wget https://phar.phpunit.de/phpunit-7.4.phar; \
chmod +x phpunit-7.4.phar; \
mv phpunit-7.4.phar /usr/local/bin/phpunit
RUN phpunit --version
###################
# PHP Codesniffer #
###################
RUN curl -OL https://squizlabs.github.io/PHP_CodeSniffer/phpcs.phar; \
mv phpcs.phar /usr/local/bin/phpcs; \
chmod +x /usr/local/bin/phpcs
############
# Composer #
############
RUN php -r "copy('https://getcomposer.org/installer', 'composer-setup.php');"; \
php -r "if (hash_file('sha384', 'composer-setup.php') === '93b54496392c062774670ac18b134c3b3a95e5a5e5c8f1a9f115f203b75bf9a129d5daa8ba6a13e2cc8a1da0806388a8') { echo 'Installer verified'; } else { echo 'Installer corrupt'; unlink('composer-setup.php'); } echo PHP_EOL;"; \
php composer-setup.php; \
php -r "unlink('composer-setup.php');"; \
mv composer.phar /usr/local/bin/composer
##################
# Install Nodejs #
##################
RUN apt-get install -y gnupg2; \
curl -sL https://deb.nodesource.com/setup_11.x | bash -; \
apt-get install -y nodejs
##################
# Install Grunt #
##################
RUN npm install -g grunt-cli
#####################
# BASH customization#
#####################
RUN echo "alias ll='ls --color=auto -lA'" >> ~/.bashrc
VOLUME /var/www/html
docker-compose.yml
version: '3'
services:
db:
image: mysql:5.7
volumes:
- ./docker-mysql/db_data:/var/lib/mysql
restart: always
environment:
MYSQL_ROOT_PASSWORD: wordpress
MYSQL_DATABASE: wordpress
MYSQL_USER: wordpress
MYSQL_PASSWORD: wordpress
wordpress:
depends_on:
- db
image: progonkpa/wordpress:1.0
restart: always
ports:
- "80:80"
environment:
WORDPRESS_DB_HOST: db:3306
WORDPRESS_DB_USER: wordpress
WORDPRESS_DB_PASSWORD: wordpress
restart: always
volumes:
db_data:
The volume is being created, it just isn't being created in the execution context where you docker-compose.yml file lives. I assume you are running the ls -lah command and expecting something to be created in the directory where your docker-compose.yml file is. That is why you say, "BUT this is not happening"
The VOLUME command in the Dockerfile is limited. The host is unknown when you build an image from the Dockerfile. It is not until the docker run is executed using your built image that the Docker host is known.
And so, when using the VOLUME command in a Dockerfile and then using docker run with that image, the volume is created in a location configured by the Docker installation. To confirm that a volume has indeed been created for your container use this command:
docker inspect -f '{{ .Mounts }}' [container_name]
To have better control and specify where you VOLUME is created on your Docker host, you need to use the -v option with docker run or configure in your docker-compose.yml file, like is being done for your MySQL persistence container.
You can remove VOLUME /var/www/html from your Dockerfile, and you should. Because your FROM wordpress image creates the VOLUME, as you already know.
Here is my Docker file that is using by docker-compose build command:
FROM python:3.5
ENV PYTHONUNBUFFERED 1
RUN apt-get update \
&& apt-get install -y --no-install-recommends libnewlib-arm-none-eabi avr-libc git \
&& rm -rf /var/lib/apt/lists/* \
&& pip install cryptography
RUN mkdir /code
COPY . /code/
WORKDIR /code
RUN pip install -r /code/misc/requirements.txt
RUN mkdir /static
RUN python /code/manage.py migrate
RUN python /code/manage.py collectstatic --noinput
Here is the service in docker-compose.yml that uses this Dockerfile:
version: "3.5"
services:
...
django:
build:
context: .
command: bash -c "/code/run/gunicorn.sh"
volumes:
- ./static:/data/django/static
- ./media:/data/django/media
depends_on:
- db
env_file:
- misc/envs/.env
So, when I try to build this compose with docker-compose build --no-cache command it says that this environment variable is still undefined even if I try to explicityly add RUN set -a && source /code/misc/envs/.env && set +a step into Dockerfile
UPD
As a result my code running on last Dockerfile step
from django.core.exceptions import ImproperlyConfigured
import os
def get_env_setting(setting, default=None):
try:
return os.environ[setting]
except KeyError:
if default is not None:
return default
error_msg = "Set the %s env variable" % setting
raise ImproperlyConfigured(error_msg)
SECRET_KEY = get_env_setting('SECRET_KEY')
gives me an error:
File "/code/project/settings.py", line 34, in <module>
SECRET_KEY = get_env_setting('SECRET_KEY')
File "/code/project/settings.py", line 22, in get_env_setting
raise ImproperlyConfigured(error_msg)
django.core.exceptions.ImproperlyConfigured: Set the SECRET_KEY env variable