Not able to substitute nginx environment variable in Dockerfile - docker

I am passing endpoint of REST APIs as env variable during docker build of nginx and having spent 6 hours trying most of the suggestions, I have run out of things/patience to try.
nginx conf that I am substituting:
location /my-api/ {
proxy_pass ${api_url}/;
}
I am passing this value during docker build:
#base_url comes from system env
docker build --build-arg base_api_url=$base_url
I am getting this value in my Dockerfile:
ARG base_api_url
ENV api_url=$base_api_url
# This prints the value
RUN echo "api_url= ${api_url}" .
COPY package.json /usr/src/app/package.json
RUN npm install
COPY . /usr/src/app
RUN npm run build
FROM nginx:1.15.8-alpine
COPY --from=builder /usr/src/app/build /usr/share/nginx/html
EXPOSE 80
# this works
COPY nginx.conf /etc/nginx/nginx.conf.template
# Initially following code was building and deploying docker image and url was hard coded. it was working
# COPY nginx.conf /etc/nginx/conf.d/default.conf
# Below will start the image but no REST endpoint configured
# CMD ["nginx", "-g", "daemon off;"]
# To substitute the api url few of the many things I have tried.
# Non of the below, have been able to replace the env api_url with its value
# Actually I don't know -- since /etc/nginx/conf.d/default.conf is not replaced at all
# CMD /bin/bash -c "envsubst < nginx.conf > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
# CMD /bin/sh -c "envsubst < nginx.conf > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;' || cat /etc/nginx/nginx.conf"
# Last status
CMD envsubst < /etc/nginx/nginx.conf.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'
/etc/nginx/conf.d/default.conf should have api_url replaced:
location /my-api/ {
proxy_pass http://theApiURL;
}
I have also tried specifically passing env variable like this:
CMD envsubst ${api_url} < /etc/nginx/nginx.conf.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'
and variations like using tee.
Any help/direction to get this resolved is appreciated.

I typically do this outside of a Dockerfile, using sed, in a deployment script.
Here's an example:
#!/usr/bin/env bash
# Deploys locally for dev work
set -e
export API_URL=www.theapiurl.com
sed "s/api_url/${API_URL}/" nginx.conf.template > nginx.conf
...
# run docker
docker-compose build --no-cache
docker-compose up -d
Of course you can set the environment variable however makes sense for your use case. I find this approach much more flexible than anything Docker provides.

Related

Docker - Nginx autoreload on config file changes

I have an nginx container, with the following Dockerfile:
FROM nginx:1.19.2
COPY ./nginx.conf /etc/nginx/nginx.conf
COPY ./conf.d /etc/nginx/conf.d
WORKDIR /etc/nginx/conf.d
RUN ln -s /etc/nginx/conf.d/my-site/my-domain.generic.conf \
&& ln -s /etc/nginx/conf.d/my-site/my-domain.conf
COPY ./certs/* /etc/ssl/
and I have the following docker-compose file:
version: '3.5'
services:
my-site_nginx:
container_name: my-site_nginx
build:
context: ./nginx
network: host
image: my-site_nginx
ports:
- '80:80'
- '443:443' # SSL
volumes:
- ./nginx/conf.d:/etc/nginx/conf.d:ro
I am looking for a way to have the nginx service inside the container to autoreload (nginx -s reload) when I change anything inside the conf.d folder as well as in the nginx.conf file that's located at the same level with the conf.d folder.
The closest thing I've found was this tutorial here: https://cyral.com/blog/how-to-auto-reload-nginx/
But I had to adapt the paths a bit, I don't know what openresty is, I suppose it's a custom image or something? (Docker noob here)... Anyway, I've tried the following from that link:
Created the docker-entrypoint.sh and nginxReloader.sh files:
docker-entrypoint.sh:
#!/bin/bash
###########
sh -c "nginxReloader.sh &"
exec "$#"
nginxReloader.sh:
#!/bin/bash
###########
while true
do
inotifywait --exclude .swp -e create -e modify -e delete -e move /etc/nginx/conf.d
nginx -t
if [ $? -eq 0 ]
then
echo "Detected Nginx Configuration Change"
echo "Executing: nginx -s reload"
nginx -s reload
fi
done
And added this to Dockerfile:
# [...]
COPY ./nginxReloader.sh /usr/local/bin/nginxReloader.sh
COPY ./docker-entrypoint.sh /usr/local/bin/docker-entrypoint.sh
RUN chmod +x /usr/local/bin/nginxReloader.sh
RUN chmod +x /usr/local/bin/docker-entrypoint.sh
RUN apt-get update && apt-get install -y --no-install-recommends apt-utils
RUN apt-get install inotify-tools -y
ENTRYPOINT [ "/usr/local/bin/docker-entrypoint.sh" ]
# CMD ["/usr/local/openresty/bin/openresty", "-g", "daemon off;"] (don't know what this would do, but I wouldn't know what to replace `openresty` with in my case, so I omitted this line from the tutorial at the link I provided)
But when trying to docker-compose up --build it either errored with No such file or directory for line exec "$#" in the nginxReloader.sh file OR I got nginx exited with code 0 when doing docker-compose up (of course, I tried different things between those errors, but can't remember exactly what).
Also, I've tried to point the ENTRYPOINT in the Dockerfile directly to nginxReloader.sh (ENTRYPOINT [ "/usr/local/bin/nginxReloader.sh" ]) but then when trying docker-compose up I only get 2 lines of output:
Setting up watches.
Watches established.
and nginx never starts (I suppose it's because of that while true loop).
Also, if I completely remove the ENTRPOINT line in Dockerfile, when running docker-compose up I still get the following output:
my-site_nginx | /docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
my-site_nginx | /docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
my-site_nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
my-site_nginx | 10-listen-on-ipv6-by-default.sh: error: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
my-site_nginx | /docker-entrypoint.sh: Launching /docker-entrypoint.d/20-envsubst-on-templates.sh
my-site_nginx | /docker-entrypoint.sh: Configuration complete; ready for start up
like Docker is somehow aware of that file being in the folder, on the same level with the Dockerfile... No errors, but changing a config still doesn't trigger nginx -s reload
Your issue is that you are not running the original entrypoint when you override it with your new entrypoint, so nginx will not start. Change
docker-entrypoint.sh:
#!/bin/bash
###########
sh -c "nginxReloader.sh &"
exec "$#"
to
docker-entrypoint.sh:
#!/bin/bash
###########
sh -c "nginxReloader.sh &"
exec /docker-entrypoint.sh "$#"
I had the same problem and the accepted answer by #TarunLalwani is missing changes to the Dockerfile that are also required.
I wasn't able to edit their answer, so here is the updated solution that is verified to work, including some additional clarifications.
..................
There are 2 issues:
First, you are not running the original entrypoint when you override it with your new entrypoint. Change
docker-entrypoint.sh:
#!/bin/bash
###########
sh -c "nginxReloader.sh &"
exec "$#"
to
docker-entrypoint.sh:
#!/bin/bash
###########
sh -c "nginxReloader.sh &"
# See changes to Dockerfile next for more details, but
# you aren't including `CMD` in your Dockerfile. So
# forwarding the arguments ("$#") won't include "nginx",
# which is what the default nginx /docker-entrypoint.sh
# will look for before setting up the service.
exec /docker-entrypoint.sh "$#"
Second, as stated above, you also need to include CMD in your Dockerfile, or you'll still get nginx exited with code 0. Add:
Dockerfile
...
# same as before
ENTRYPOINT [ "/usr/local/bin/docker-entrypoint.sh" ]
# But include this, which is the same command as in the
# default nginx Dockerfile:
CMD [ "nginx", "-g", "daemon off;" ]
The line in the tutorial that you linked to but commented out in your Dockerfile version was:
CMD ["/usr/local/openresty/bin/openresty", "-g", "daemon off;"]
You're not using the openresty version of nginx, so you're just using "nginx". And it's a registered command so you don't need to use the path to the bin file like he's doing for openresty.

How to extend nginx docker image without getting error systemctl: command not found?

I want to build my own custom docker image from nginx image.
I override the ENTRYPOINT of nginx with my own ENTERYPOINT file.
Which bring me to ask two questions:
I think I lose some commands from nginx by doing so. am I right? (like expose the port.. )
If I want to restart the nginx I run this commands: nginx -t && systemctl reload nginx. but the output is:
nginx: configuration file /etc/nginx/nginx.conf test is successful
/entrypoint.sh: line 5: systemctl: command not found
How to fix that?
FROM nginx:latest
WORKDIR /
RUN echo "deb http://ftp.debian.org/debian stretch-backports main" >> /etc/apt/sources.list
RUN apt-get -y update && \
apt-get -y install apt-utils && \
apt-get -y upgrade && \
apt-get -y clean
# I ALSO WANT TO INSTALL CERBOT FOR LATER USE (in my entrypoint file)
RUN apt-get -y install python-certbot-nginx -t stretch-backports
# COPY ./something ./tothisimage
# COPY ./something ./tothisimage
# COPY ./something ./tothisimage
# COPY ./something ./tothisimage
COPY entrypoint.sh /entrypoint.sh
ENTRYPOINT ["bash", "/entrypoint.sh"]
entrypoint.sh
echo "in entrypoint"
# I want to run some commands here...
# After I want to run nginx normally....
nginx -t && systemctl reload nginx
echo "after reload"
this will work using service command:
echo "in entrypoint"
# I want to run some commands here...
# After I want to run nginx normally....
nginx -t && service nginx reload
echo "after reload"
output:
in entrypoint
nginx: the configuration file /etc/nginx/nginx.conf syntax is ok
nginx: configuration file /etc/nginx/nginx.conf test is successful
Restarting nginx: nginx.
after reload
Commands like service and systemctl mostly just don't work in Docker, and you should totally ignore them.
At the point where your entrypoint script is running, it is literally the only thing that is running. That means you don't need to restart nginx, because it hasn't started the first time yet. The standard pattern here is to use the entrypoint script to do some first-time setup; it will be passed the actual command to run as arguments, so you need to tell it to run them.
#!/bin/sh
echo "in entrypoint"
# ... do first-time setup ...
# ...then run the command, nginx or otherwise
exec "$#"
(Try running docker run --rm -it myimage /bin/sh. You will get an interactive shell in a new container, but after this first-time setup has happened.)
The one thing you do lose in your Dockerfile is the default CMD from the base image (setting an ENTRYPOINT resets that). You need to add back that CMD:
COPY entrypoint.sh /entrypoint.sh
RUN chmod +x /entrypoint.sh
ENTRYPOINT ["/entrypoint.sh"]
CMD ["nginx", "-g", "daemon off;"]
You should keep the other settings from the base image, like ENV definitions and EXPOSEd ports.
The "systemctl" command is specific to some SystemD based operating system. But you do not have such a SystemD daemon running on PID 1 - so even if you install those packages it wont work.
You can only check in the nginx.service file which command the "reload" would execute for real. Or have something like the docker-systemctl-replacement script do it for you.

Docker nginx not starting after running custom script

We have an angular app we are trying to run in docker nginx.
I have to run a script on startup that uses an environment variableto replace the app url for each stage. This is needed to connect to the backend.
We do not wish to build the container for each stage. The container will run in Azure docker.
For now i am running it locally. It executes my script and then the app shuts down.
The Docker File:
FROM node:8.11.2-alpine as node
LABEL author="My Online Presence"
WORKDIR /usr/src/app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
FROM nginx:1.13.12-alpine
COPY /certificates /etc/nginx/
COPY --from=node /usr/src/app/dist /usr/share/nginx/html
COPY ./nginx.conf /etc/nginx/conf.d/default.conf
ADD run.sh /usr/share/nginx/html/run.sh
RUN apk add --update bash && rm -rf /var/cache/apk/*
RUN chmod +x /usr/share/nginx/html/run.sh
ENTRYPOINT ["/usr/share/nginx/html/run.sh"]
CMD ["nginx", "-g", "daemon off;"]
The run.sh file
#!/bin/sh
FILE_NAME=$(find . -name "main*.js")
sed -i "s/localhost\:4200/${DIGITISE_URL}/g" $FILE_NAME
echo 'File updated with correct url'
i use
docker build -t digitise .
and then
docker run -p 80:80 -p 443:443 -e DIGITISE_URL=digitise.co.za digitise
I got it working simply by adding
exec "$#"
to the end of my script being run
You can also start nginx service in your script.

Error: Starting container process caused "exec: \"/docker-entrypoint.sh\": permission denied"

I'm trying to build docker-compose, but I'm getting this error:
ERROR: for indicaaquicombrold_mysqld_1 Cannot start service mysqld:
oci runtime error: container_linux.go:247: starting container process
caused "exec: \"/docker-entrypoint.sh\": permission denied"
ERROR: for mysqld Cannot start service mysqld: oci runtime error:
container_linux.go:247: starting container process caused "exec:
\"/docker-entrypoint.sh\": permission denied"
ERROR: Encountered errors while bringing up the project.
docker-compose.yml
version: '3'
services:
php:
build:
context: ./docker/php
image: indicaaqui.com.br:tag
volumes:
- ./src:/var/www/html/
- ./config/apache-config.conf:/etc/apache2/sites-enabled/000-default.conf
ports:
- "80:80"
- "443:443"
mysqld:
build:
context: ./docker/mysql
environment:
- MYSQL_DATABASE=db_indicaaqui
- MYSQL_USER=indicaqui
- MYSQL_PASSWORD=secret
- MYSQL_ROOT_PASSWORD=docker
volumes:
- ./config/docker-entrypoint.sh:/docker-entrypoint.sh
- ./database/db_indicaaqui.sql:/docker-entrypoint-initdb.d/db_indicaaqui.sql
Dockerfile (php)
FROM php:5.6-apache
MAINTAINER Limup <limup#outlook.com>
CMD [ "php" ]
RUN docker-php-ext-install pdo_mysql
# Enable apache mods.
# RUN a2enmod php5.6
RUN a2enmod rewrite
# Expose apache.
EXPOSE 80
EXPOSE 443
# Use the default production configuration
# RUN mv "$PHP_INI_DIR/php.ini-production" "$PHP_INI_DIR/php.ini"
RUN mv "$PHP_INI_DIR/php.ini-development" "$PHP_INI_DIR/php.ini"
# Override with custom opcache settings
# COPY ./../../config/php.ini $PHP_INI_DIR/conf.d/
# Manually set up the apache environment variables
ENV APACHE_RUN_USER www-data
ENV APACHE_RUN_GROUP www-data
ENV APACHE_LOG_DIR /var/log/apache2
ENV APACHE_LOCK_DIR /var/lock/apache2
ENV APACHE_PID_FILE /var/run/apache2.pid
# Update the PHP.ini file, enable <? ?> tags and quieten logging.
RUN sed -i "s/short_open_tag = Off/short_open_tag = On/" "$PHP_INI_DIR/php.ini"
RUN sed -i "s/error_reporting = .*$/error_reporting = E_ERROR | E_WARNING | E_PARSE/" "$PHP_INI_DIR/php.ini"
RUN a2dissite 000-default.conf
RUN chmod -R 777 /etc/apache2/sites-enabled/
WORKDIR /var/www/html/
# By default start up apache in the foreground, override with /bin/bash for interative.
CMD ["/usr/sbin/apache2ctl", "-D", "FOREGROUND"]
Dockerfile (Mysql)
FROM mariadb:latest
RUN chmod -R 777 /docker-entrypoint.sh
ENTRYPOINT ["/docker-entrypoint.sh"]
EXPOSE 3306
CMD ["mysqld"]
Please, help me solve this problem!
Any ideas?
That is most likely a Linux file permission issue on config/docker-entrypoint.sh. If your host is Linux/Mac, you can run:
chmod 755 config/docker-entrypoint.sh
For more on linux permissions, here's a helpful article: https://www.linux.com/learn/understanding-linux-file-permissions
First, you need to copy entrypoint.sh file into other directory instead of same your source code (Eg. /home/entrypoint.sh), then grant permission to execute entrypoint script:
RUN ["chmod", "+x", "/home/entrypoint.sh"]
Solution
ENV USER root
ENV WORK_DIR_PATH /home
RUN mkdir -p $WORK_DIR_PATH && chown -R $USER:$USER $WORK_DIR_PATH
WORKDIR $WORK_DIR_PATH
Info
The USER instruction sets the user name (or UID) and optionally the user group (or GID) to use when running the image and for any RUN, CMD and ENTRYPOINT instructions that follow it in the Dockerfile.
The WORKDIR instruction sets the working directory for any RUN, CMD, ENTRYPOINT, COPY and ADD instructions that follow it in the Dockerfile. If the WORKDIR doesn’t exist, it will be created even if it’s not used in any subsequent Dockerfile instruction.
Links
chown command
docker builder reference
A pretty common solution if nothing works is to re-install Docker.. That's what ended up working for me after trying for like 5 hours everything under the sun in terms of permissions etc.

Heroku (Docker) PORT environment variable in nginx

I just take a few modifications on a Dockerfile to run it with nginx on Heroku. Something special about Heroku is, that everything is running as non-root. Second certain behaviour is the use of a random Port which comes from Heroku itself and you can't be modified. They provide the env $PORT which you should bind to nginx. If Heroku recognizes that something isn't bind to that port it stops the entire container. Question is:
How can I bind nginx to a given env variable in order to have a dynamic port in the nginx-site.conf?
I tried to use things like follows in the Dockefile:
env PORT; in nginx.conf and listen PORT_from_env; in the nginx-site.conf
Also tried listen 80; in the nginx-site.conf and RUN /bin/sed -i "s/listen 80/listen ${PORT}/" /etc/nginx/sites-available/default.conf in the Dockerfile
I am absolutely above my capabilities. Someone has an idea or can help? It be very helpful!
Further information:
https://devcenter.heroku.com/articles/dynos#web-dynos
I got it working for my app by following this example :
Step 1: listen to $PORT in default.conf.template
server {
listen $PORT default_server;
location / {
root /usr/share/nginx/html;
index index.html;
}
}
Step 2: add this directive to your Dockerfile
COPY default.conf.template /etc/nginx/conf.d/default.conf.template
Step 3: add this at the end of your Dockerfile
CMD /bin/bash -c "envsubst '\$PORT' < /etc/nginx/conf.d/default.conf.template > /etc/nginx/conf.d/default.conf" && nginx -g 'daemon off;'
I'll show a solution which doesn't require to write a new Dockerfile, you can use the offical nginx images.
Like #Jimmy's solution we'll use the envsubst command which substitutes environment variables in shell format strings.
This command is available with the offical nginx image and also with the alpine version.
Step #1
write your nginx configuration in a template file - let's call it: site.template:
server {
listen ${PORT};
server_name localhost;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
}
}
Notice the PORT placeholder.
Step #2 - with docker compose
Mount that inside the /etc/nginx/conf.d directory and then execute the envsubst command to use the template as a reference for default.conf:
web:
image: nginx:alpine
volumes:
- ./site.template:/etc/nginx/conf.d/site.template
ports:
- "3000:8080"
environment:
- PORT=8080
command: /bin/sh -c "envsubst < /etc/nginx/conf.d/site.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
Notice that:
1. You need to execute the nginx daemon after that.
2. I used /bin/sh and not /bin/bash because my base image is alpine.
Step #2 (Another option) - inline docker run
If, for some reason you don't want to work with docker-compose you can use the following bash script:
#!/usr/bin/env bash
##### Variables #####
PORT=8080 #Or $1 if you pass it from command line
TEMPLATE_DIR=$(pwd)/site.template
TEMPLATE_REMOTE_DIR=/etc/nginx/conf.d/site.template
IMAGE_NAME=nginx:alpine
echo "Starting nginx on port: $PORT ..."
##### The docker command #####
docker run -p 3000:$PORT -v $TEMPLATE_DIR:$TEMPLATE_REMOTE_DIR $IMAGE_NAME \
/bin/sh -c "envsubst < $TEMPLATE_REMOTE_DIR > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"
You need to update your nginx configuration at boot time.
See this buildpack as an example (I haven't tested that it does work).

Resources