Docker compose environment variables not picking up in nginx - docker

I have a vue app running on the front-end with spring boot backend both on different containers.
I want to dockerize my vuejs app to pass environment variables from the docker-compose file to nginx.
My problem is that my nginx conf file is not picking up environment variables from docker-compose.
Docker Compose File
backend-service:
container_name: backend-service
image: backend-service-local
networks:
- app-network
ports:
- 8081:8080
restart: on-failure
depends_on:
postgresdb:
condition: service_healthy
vue-app:
container_name: vue-app
image: vue-app-local
networks:
- app-network
ports:
- 8080:80
environment:
VUE_APP_BASE_URL: http://backend-service:8080
restart: on-failure
depends_on:
backend-service:
condition: service_started
DOCKER FILE
# build stage
FROM node:lts-alpine as build-stage
WORKDIR /app
COPY package*.json ./
RUN npm install
COPY . .
RUN npm run build
# production stage
FROM nginx:stable-alpine as production-stage
COPY --from=build-stage /app/dist /usr/share/nginx/html
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
NGINX CONF
# Run as a less privileged user for security reasons.
user nginx;
# #worker_threads to run;
# "auto" sets it to the #CPU_cores available in the system, and
# offers the best performance.
worker_processes auto;
events { worker_connections 1024; }
http {
sendfile on;
upstream docker-backend {
server ${VUE_APP_BASE_URL};
}
server {
# Hide nginx version information.
server_tokens off;
listen 80;
root /usr/share/nginx/html;
include /etc/nginx/mime.types;
location / {
try_files $uri $uri/ /index.html;
}
location /api/ {
proxy_pass http://docker-backend;
}
}
}

Please advise nginx docker image docs in the Using environment variables in nginx configuration section of the page.
The way the nginx docker image deals with environment variables is injecting them in runtime using the configs in the linked page

Related

docker nginx does't serve static files through upstream Next.JS

I couldn't find an answer to my question from other similar questions.
So, I have two docker containers:
Next.JS web-app
nginx reverse proxy
NextJS container without nginx reverse proxy worked as expected.
Even more, I can log in to the nginx container with docker exec -it nginx sh and with curl read whose static files on Next.JS container. I also see static files in the folder from a shared volume.
I run them with docker-compose:
volumes:
nextjs-build:
version: '3.9'
services:
nginx:
image: arm64v8/nginx:alpine
container_name: nginx
ports:
- "80:80"
- "443:443"
networks:
- blog
restart: unless-stopped
depends_on:
- website-front
volumes:
- type: volume
source: nextjs-build
target: /nextjs
read_only: true
- type: bind
source: /etc/ssl/private/blog-ssl
target: /etc/ssl/private/
read_only: true
- type: bind
source: ./nginx/includes
target: /etc/nginx/includes
read_only: true
- type: bind
source: ./nginx/conf.d
target: /etc/nginx/conf.d
read_only: true
- type: bind
source: ./nginx/dhparam.pem
target: /etc/nginx/dhparam.pem
read_only: true
- type: bind
source: ./nginx/nginx.conf
target: /etc/nginx/nginx.conf
read_only: true
website-front:
build: ./website
container_name: website-front
ports:
- "3000"
networks:
- blog
restart: unless-stopped
volumes:
- nextjs-build:/app/.next
networks:
blog:
external:
name: nat
my nginx configs:
upstream nextjs_upstream {
server website-front:3000;
}
server {
listen 443 http2 ssl;
listen [::]:443 http2 ssl;
server_name website_url;
ssl_certificate /etc/ssl/private/chain.crt;
ssl_certificate_key /etc/ssl/private/server.key;
ssl_trusted_certificate /etc/ssl/private/ca.ca-bundle;
# access_log /var/log/nginx/host.access.log main;
# security
include includes/security.conf;
include includes/general.conf;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
location /_next {
proxy_pass http://nextjs_upstream/_next/;
}
location / {
proxy_pass http://nextjs_upstream;
}
}
Tried multiple nginx configurations for static route:
localhost /_next {
root /nextjs;
}
NextJS dockerfile:
FROM node:alpine AS builder
# this ensures we fix simlinks for npx, Yarn, and PnPm
RUN apk add --no-cache libc6-compat
RUN corepack disable && corepack enable
WORKDIR /app
COPY ./ ./
RUN yarn install --frozen-lockfile
RUN yarn build
ENV NODE_ENV production
CMD chown -R node:node /app/.next
EXPOSE 3000
USER node
CMD [ "yarn", "start" ]
With that config I can see my website, but for static files I got 404 through upstream.
So, the problem was in a wrong path for mime.types file in nginx.conf and default location paths in includes/general.conf file.
After I changed it from: mime.types to /etc/nginx/mime.types it started working again.

nginx reverse proxy with docker-compose config to serve traffic to multiple domains

I am trying to use nginx with docker-compose to route traffic for two different apps with different domain names. I want to be able to go to publisher.dev but I can only access that app from localhost:3000 (this is a react app) and I have another app which I want to access from widget.dev but I can only access from localhost:8080 (this is a Preact app). This is my folder structure and configs:
|-docker-compose.yml
|-nginx
|--default.conf
|--Dockerfile.dev
|-publisher
|--// react app
|--Dockerfile.dev
|-widget
|--// preact app (widget)
|--Dockerfile.dev
# default.conf
upstream publisher {
server localhost:3000;
}
upstream widget {
server localhost:8080;
}
server {
listen 80;
server_name publisher.dev;
location / {
proxy_pass http://publisher/;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
server {
listen 80;
server_name widget.dev;
location / {
proxy_pass http://widget/;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
nginx Dockerfile.dev
FROM nginx:stable-alpine
COPY ./default.conf /etc/nginx/conf.d/default.conf
publisher Dockerfile.dev (same as widget Dockerfile.dev)
# Specify the base image
FROM node:16-alpine
# Specify the working directory inside the container
WORKDIR /app
# copy the package json from your local hard drive to the container
COPY ./package.json ./
# install dependencies
RUN npm install
# copy files from local hard drive into container
# by copying the package.json and running npm install before copy files,
# this insures that a change to a file does not cause a re-run of npm-install
COPY ./ ./
# command to run when the container starts up
CMD ["npm", "run", "start"]
# build this docker container with:
# docker build -f Dockerfile.dev .
# run this container with:
# docker run <container id>
docker-compose.yml
version: '3'
services:
nginx:
build:
dockerfile: Dockerfile.dev
context: ./nginx
ports:
- 3050:80
restart: always
depends_on:
- publisher
- widget
publisher:
stdin_open: true
build:
dockerfile: Dockerfile.dev
context: ./publisher
volumes:
- /app/node_modules
- ./publisher:/app
ports:
- 3000:3000
environment:
VIRTUAL_HOST: publisher.dev
widget:
stdin_open: true
build:
dockerfile: Dockerfile.dev
context: ./widget
volumes:
- /app/node_modules
- ./widget:/app
ports:
- 8080:8080
environment:
VIRTUAL_HOST: widget.dev
hosts file
127.0.0.1 publisher.dev
127.0.0.1 widget.dev
why is your upstream trying to connect with
publisher and widget, shouldn't they connect to localhost:3000 and localhost:8080, let upstream server name be publisher and widget but connect them to localhost.
upstream publisher {
#server publisher:3000;
server localhost:3000;
}

Traefik: Level=error msg=“field not found, node: mywebsite” providerName=docker

I am building a static website using Gatsby, and I am using Nginx to serve the static files.
I am also setting up Docker for the application deployment to production and also using Traefik as the reverse proxy in the Docker container.
Traefik runs on a different container while the Gatsby application runs on a different container with Nginx together.
However, when I run the application in production I get this error:
level=error msg="field not found, node: mywebsite" providerName=docker container=web-my-website
Here's my code:
Nginx's defualt.conf
server {
listen 3008;
add_header Cache-Control no-cache;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri $uri/ /index.html;
expires -1;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
Dockerfile
# Set base image
FROM node:latest AS builder
# Set working directory
WORKDIR /app
# Copy package.json and install packages
COPY package.json .
RUN npm install
# Copy other project files and build
COPY . ./
RUN npm run build
# Set nginx image
FROM nginx:latest
# Nginx config
RUN rm -rf /etc/nginx/conf.d/default.conf
COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
# Static build
COPY --from=builder /app/public /usr/share/nginx/html
# Set working directory
WORKDIR /usr/share/nginx/html
# Start Nginx server
CMD ["/bin/bash", "-c", "nginx -g \"daemon off;\""]
Gatsby application's docker-compose.yml
version: "3"
services:
web:
image: my-website
build:
context: .
dockerfile: Dockerfile
expose:
- "3004"
labels:
- traefik.enable=true
- traefik.http.routers.mywebsite.rule=Host(`mywebsite.com`)
- traefik.http.services.educollectwebsite.loadbalancer.server.port=3004
restart: always
volumes:
- .:/app
networks:
default:
external:
name: traefik-proxy
Traefik's docker-compose.yml
version: "3"
services:
reverse-proxy:
# The official v2 Traefik docker image
image: traefik:v2.2
# Enables the web UI and tells Traefik to listen to docker
command:
- --api.insecure=true
- --entrypoints.web.address=:80
- --providers.docker=true
- --providers.docker.exposedbydefault=false
ports:
# The HTTP port
- "88:80"
# The Web UI (enabled by --api.insecure=true)
- "8088:8080"
restart: always
volumes:
# So that Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
networks:
default:
external:
name: traefik-proxy
I can't seem to figure out what the issue is here. Any form of help will be appreciated.
I was finally able to resolve it after some hours of working with my Line Manager.
The issue was that I defined port 3008 in the Nginx default.conf file and then defined port 3004 in the Gatsby application's docker-compose.yml file. This did not allow traffic into the application from Traefik reverse proxy. since both ports were different.
Solution 1:
Simply defining the same port of 3008 in the Nginx default.conf and in the Gatsby application's docker-compose.yml file fixed it:
Nginx's defualt.conf
server {
listen 3008;
add_header Cache-Control no-cache;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri $uri/ /index.html;
expires -1;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
Gatsby application's docker-compose.yml
version: "3"
services:
web:
image: my-website
build:
context: .
dockerfile: Dockerfile
expose:
- "3004"
labels:
- traefik.enable=true
- traefik.http.routers.mywebsite.rule=Host(`mywebsite.com`)
- traefik.http.services.educollectwebsite.loadbalancer.server.port=3008
restart: always
volumes:
- .:/app
networks:
default:
external:
name: traefik-proxy
Solution 2:
Defining the default port in Traefik which is port 80 in the Nginx default.conf and in the Gatsby application's docker-compose.yml file fixed it. This is more preferable when deploying static applications since it helps me to assume a reasonable default for the application.
Nginx's defualt.conf
server {
listen 80;
add_header Cache-Control no-cache;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri $uri/ /index.html;
expires -1;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
Gatsby application's docker-compose.yml
version: "3"
services:
web:
image: my-website
build:
context: .
dockerfile: Dockerfile
expose:
- "80"
labels:
- traefik.enable=true
- traefik.http.routers.mywebsite.rule=Host(`mywebsite.com`)
restart: always
volumes:
- .:/app
networks:
default:
external:
name: traefik-proxy
Note: Using the same port with Traefik which is port 80 in the application, invalidates the need for a Traefik loadbalancer service.
- traefik.http.services.educollectwebsite.loadbalancer.server.port=80
That's all.
I hope this helps

Use docker-compose service names in Nginx config

I have an application with 4 services. One of them is Nginx which will act as a proxy.
I use docker compose to run the services. In nginx when I specify a path and where to proxy I want to be able to use the service name. This is what I have done so far.
version: '3'
services:
go_app:
image: go_app
depends_on:
- mysql
ports:
- "8000:8000"
mysql:
image: mysql_db
ports:
- "3306:3306"
flask_app:
image: flask_app
ports:
- "8080:8080"
nginx:
image: nginx_app
ports:
- "80:80"
depends_on:
- mysql
- flask_app
- go_app
With the above I create all services. They all work on their respective ports. I want Nginx to listen on port 80 and proxy as defined in the config:
server {
listen 0.0.0.0:80;
server_name localhost;
location / {
proxy_pass http://${FLASK_APP}:8080/;
}
}
You may ask where does FLASK_APP come from. I specify it inside nginx docker image:
FROM nginx
ENV FLASK_APP=flask_app
RUN rm /etc/nginx/conf.d/default.conf
COPY config/default.conf /etc/nginx/conf.d/default.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
Nginx container keeps failing with the following error:
[emerg] 1#1: unknown "flask_app" variable
nginx: [emerg] unknown "flask_app" variable
The way I understand docker compose, flask_app should resolve as the flask_app service.
What am I doing wrong/misunderstanding?
The issue is that nginx does not read ENV variables .
see https://github.com/docker-library/docs/tree/master/nginx#using-environment-variables-in-nginx-configuration
a solution :
you can modify you dockerfile for nginx with this
FROM nginx
ENV FLASK_APP=flask_app
RUN rm /etc/nginx/conf.d/default.conf
COPY default.conf /etc/nginx/conf.d/default.template
EXPOSE 80
CMD ["/bin/bash","-c","envsubst < /etc/nginx/conf.d/default.template > /etc/nginx/conf.d/default.conf && exec nginx -g 'daemon off;'"]
the COPY command is modified to copy you configuration as a template.
the last line is modified to do a substitution using your ENV variables.

NGINX + Docker + Google Cloud Compute Engine error - [emerg] 1#1: host not found in upstream

I am trying to deploy my docker container running Next.js and NGINX to Google Compute Engine. It works perfectly when deployed on local with docker-compose up but when deployed to CE I get the error
[emerg] 1#1: host not found in upstream "nextjs:3000" in /etc/nginx/conf.d/default.conf:4
The logs on local are
Attaching to nextjs, profiles-fe_nginx_1
nextjs | 2019-05-02T03:51:26: PM2 log: Launching in no daemon mode
nextjs | 2019-05-02T03:51:26: PM2 log: App [npm:0] starting in -fork mode-
nextjs | 2019-05-02T03:51:26: PM2 log: App [npm:0] online
nextjs | > create-next-example-app# start /usr/app
nextjs | > NODE_ENV=production node server.js
nextjs | > Ready on http://localhost:3000
Deployment to CE commands
docker-compose build
docker tag <local_container> <remote_container>
docker push <remove_container
gcloud beta compute --project=<project_name> instances create-with-container <instance_name> --zone=europe-west2-c --machine-type=g1-small --container-image=<remove_container> --tags http-server,https-server
default.conf
upstream app {
server nextjs:3000;
}
server {
listen 80;
gzip on;
gzip_proxied any;
gzip_comp_level 4;
gzip_types text/css application/javascript image/svg+xml;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
# BUILT ASSETS (E.G. JS BUNDLES)
# Browser cache - max cache headers from Next.js as build id in url
# Server cache - valid forever (cleared after cache "inactive" period)
location /_next/static {
# proxy_cache STATIC;
proxy_pass http://app;
}
# STATIC ASSETS (E.G. IMAGES)
# Browser cache - "no-cache" headers from Next.js as no build id in url
# Server cache - refresh regularly in case of changes
location /static {
# proxy_cache STATIC;
proxy_ignore_headers Cache-Control;
proxy_cache_valid 60m;
proxy_pass http://app;
}
# DYNAMIC ASSETS - NO CACHE
location / {
proxy_pass http://app;
}
}
docker-compose file
version: '3'
services:
nextjs:
build: ./
ports:
- "3000:3000"
networks:
- app-network
container_name: nextjs
restart: unless-stopped
nginx:
build: ./nginx
depends_on:
- nextjs
ports:
- "80:80"
links:
- nextjs:nextjs
networks:
- app-network
networks:
app-network:
driver: bridge
./nginx/Dockerfile
FROM nginx
# Remove any existing config files
RUN rm /etc/nginx/conf.d/*
# Copy config files
# *.conf files in "conf.d/" dir get included in main config
COPY ./default.conf /etc/nginx/conf.d/
EXPOSE 80
./Dockerfile
FROM node:11.14.0-alpine
# Set working directory
WORKDIR /usr/app
# Install PM2 globally
RUN npm install --global pm2
# Copy "package.json" and "package-lock.json" before other files
# Utilise Docker cache to save re-installing dependencies if unchanged
COPY ./package*.json ./
# Install dependencies
RUN npm install --save
# Copy all files
COPY ./ ./
# Build app
RUN npm run build
EXPOSE 3000
# Launch app with PM2
CMD [ "pm2-runtime", "start", "npm", "--", "start" ]

Resources