I am really stumped and can use help figuring out why my environment variables aren't transferring from Docker to nginx config files.
I have a docker-compose.yml
nginx:
image: nginx
container_name: proxier
volumes:
- ./conf/nginx.conf:/etc/nginx/nginx.conf
- ./conf/server.nginx.conf.tpl:/etc/nginx/server.nginx.conf.tpl
- ./build/web:/srv/static:ro
- ./docker/proxier:/tmp/docker
ports:
- "80:80"
- "443:443"
environment:
- HOST_EXTERNAL_IP=localhost
- DEVSERVER_PORT=8000
- DEVSERVICE_PORT=5000
command: /bin/bash -c "env && envsubst '$$HOST_EXTERNAL_IP $$DEVSERVER_PORT $$DEVSERVICE_PORT' < /etc/nginx/server.nginx.conf.tpl > /etc/nginx/conf.d/default.conf && nginx -g 'daemon off;'"
I have an nginx.conf file
user nginx;
worker_processes 1;
error_log /dev/stdout warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
client_max_body_size 100g;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /dev/stdout main;
sendfile off;
tcp_nopush on;
keepalive_timeout 65;
gzip on;
server_tokens off;
upstream app {
server myapp:8000 fail_timeout=0;
}
include /etc/nginx/server.nginx.conf.tpl;
}
I have a server.nginx.conf.tpl file
server {
listen 80;
listen 443 ssl http2 default_server;
server_name localhost;
index index.html;
location ^~ /services/ {
proxy_pass https://myurl.com;
proxy_set_header USER_DN $ssl_client_s_dn;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
}
location / {
proxy_http_version 1.1;
proxy_set_header Connection "keep-alive";
proxy_pass http://${HOST_EXTERNAL_IP}:${DEVSERVER_PORT}; # Won't read environment variables here
}
}
When I run this however, I get the error
nginx: [emerg] unknown "host_external_ip" variable I am using envsubst correctly to pass the environment variable from docker per the docs
Do not copy nginx.conf directly. Instead create a shell file to generate the nginx file e.g.
echo 'you nginx conf goes here with $envVariable' > location/to/conf/folder/nginx.conf
and run that file inside the container. So when that shell file will run. It will replace the environment variables that you set with it's actual value in the nginx.conf.
Do not forget to skip $ of nginx variables.
Related
I'm writing apk uploading functionality in my React App. Every time I try to upload an apk file bigger than 10 mb, I get back an error: 413 Request Entity Too Large. I've already used client_max_body_size 888M; directive. Can anybody explain me, what I do wrong?
Here is my nginx.conf:
user nginx;
worker_processes auto;
error_log /var/log/nginx/errors.log notice;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/accesses.log main;
sendfile on;
keepalive_timeout 65;
server {
listen 80;
listen [::]:80;
server_name up-app.unpluggedsystems.app;
# return 301 http://$server_name$request_uri;
root /usr/share/nginx/html;
index index.html;
client_max_body_size 888M;
location /api/7/apps/search {
proxy_pass http://ws75.aptoide.com/api/7/apps/search;
proxy_set_header X-Forwarded-For $remote_addr;
}
location ~ \.(apk)$ {
proxy_pass https://pool.apk.aptoide.com;
proxy_set_header X-Forwarded-For $remote_addr;
}
location /api {
proxy_pass https://up-app.unpluggedsystems.app;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
}
And Dockerfile that I use (maybe something wrong here?):
# build environment
FROM node:13.12.0-alpine as build
WORKDIR /app
ENV PATH /app/node_modules/.bin:$PATH
COPY package.json ./
COPY package-lock.json ./
COPY create-env-file.sh ./create-env-file.sh
RUN npm install
RUN npm install react-scripts#3.4.1 -g
COPY . ./
RUN npm run build
# production environment
FROM nginx:stable-alpine
COPY --from=build /app/build /usr/share/nginx/html
COPY fullchain.crt /etc/ssl/fullchain.crt
COPY unpluggedapp.key.pem /etc/ssl/unpluggedapp.key.pem
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx", "-g", "daemon off;"]
I FE Junior, and I so weak in such things. Maybe I put client_max_body_size in wrong place?
Please move client_max_body_size under http{} section
http {
# some code here
sendfile on;
client_max_body_size 888M;
#...
}
Make sure to restart nginx After modifying the configuration file.
You can try
client_max_body_size 0;
(set to no limit, but this is not recommended for production env.)
More over, Remember that if you have SSL, that will require you to set the above limitation for the SSL server and location{} in nginx.conf too. If your client (browser) tries to upload on http, and you expect them to get 301'd to https, nginx server will actually drop the connection before the redirect due to the file being too large for the http server, so the limitation has to be set in both http and https.
The trick is to put in both http{} and server{} (for vhost)
http {
# some code
client_max_body_size 0;
}
server {
# some code
client_max_body_size 0;
}
I am trying to get the front end and backend working together for the spring boot pet clinic app. I have already done ng --prod on a windows pc and then used github to transfer my code to a VM. I had it working once but only on IE but it doesn't again I don't know what's wrong. Please help it's done my head in for a few weeks.
nginx.conf file:
worker_processes 1;
pid /run/nginx.pid;
events {
worker_connections 768;
# multi_accept on;
}
http {
sendfile off;
tcp_nopush on;
tcp_nodelay on;
keepalive_timeout 65;
types_hash_max_size 2048;
include /etc/nginx/mime.types;
default_type application/octet-stream;
#access_log /var/log/nginx/access.log;
#error_log /var/log/nginx/error.log;
gzip on;
gzip_disable "msie6";
server {
#listen 8443 ssl;
listen 4200;
#server_name localhost;
#ssl_certificate localhost.crt;
#ssl_certificate_key localhost.key;
location / {
root /AngularApp/dist;
index index.html;
}
location /api/ {
proxy_pass_header Server;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
proxy_connect_timeout 20;
proxy_read_timeout 20;
proxy_pass http://springcommunity:9966/petclinic;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
}
Dockefile for front end.
FROM nginx
RUN rm /etc/nginx/conf.d/default.conf
RUN rm /etc/nginx/nginx.conf
COPY /nginx.conf /etc/nginx/nginx.conf
COPY /AngularApp /AngularApp
WORKDIR /AngularApp
docker-compose file:
version: '3.7'
services:
springcommunity:
image: springcommunity/spring-petclinic-rest
ports:
- "9966:9966"
web:
container_name: nginx
build: .
ports:
- "4200:4200"
depends_on:
- springcommunity
links:
- springcommunity
restart: always
environment.prod.ts and environment.ts file before ng --prod (production)
export const environment = {
production: true,
REST_API_URL:'http://localhost:9966/petclinic/'
};
Things I have tried and failed:
export const environment = {
production: true,
REST_API_URL:'http://springcommunity:9966/petclinic'
};
Exposing 4200 in the Dockerfile for the front end.
I have tried port mapping in docker compose:
example:
4200:9966
9966:4200
Exposing 9966 as well in the compose file.
The front end and backend work but just not together, only individually I have a feeling that one container needs to be delayed the front end I have done some google searching but can't find a viable option. I have no idea how to do it, please help.
Update 5/06/2020
I am currently running a wait-for.sh so the backend runs before the the web container but now nginx exits with a error code 0. I am also trying to see the nginx error logs but I can't get to this could someone please shed some light on this?
If your frontend can't reach the backend on that VM may be that your docker containers are not o the same network.
You can try REST_API_URL:'http://0.0.0.0:9966/petclinic'
Or can specify a custom networks in docker-compose file and use REST_API_URL:'http://springcommunity:9966/petclinic'
https://docs.docker.com/compose/networking/#specify-custom-networks
I'm using Nginx as a proxy for Jenkins server. Both in Docker container.
The idea is Jenkins running on port 8080, with port 8080 exposed. Nginx listening on port 80 and redirecting traffic to Jenkins on port 8080. If you try to access port 8080 directly it will refuse the connection.
Please see docker-compose.yml file:
version: '3.7'
services:
master:
build: ./jenkins-master
networks:
- jenkins-net
volumes:
- jenkins-log:/var/log/jenkins
- jenkins-data:/var/jenkins_home
nginx:
build: ./jenkins-nginx
ports:
- "80:80"
networks:
- jenkins-net
networks:
jenkins-net:
volumes:
jenkins-log:
jenkins-data:
Jenkins-master Dockerfile:
FROM jenkins/jenkins:alpine
LABEL maintainer=''
USER root
RUN mkdir /var/log/jenkins
RUN mkdir /var/cache/jenkins
RUN chown -R jenkins:jenkins /var/log/jenkins
RUN chown -R jenkins:jenkins /var/cache/jenkins
USER jenkins
ENV JAVA_OPTS='-Xmx8192m'
ENV JENKINS_OPTS=' --handlerCountMax=300 -- logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war'
This is the nginx.conf file:
server {
listen 80;
server_name localhost;
access_log off;
location / {
proxy_pass http://master:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto http;
proxy_max_temp_file_size 0;
proxy_connect_timeout 150;
proxy_send_timeout 100;
proxy_read_timeout 100;
proxy_buffer_size 8k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
}
}
this is my jenkins-nginx Dockerfile:
FROM nginx:mainline-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY jenkins.conf /etc/nginx/conf.d/jenkins.conf
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx"]
Adding Nginx Dockerfile for completeness:
FROM nginx:mainline-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY jenkins.conf /etc/nginx/conf.d/jenkins.conf
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx"]
Jenkins.conf file:
daemon off;
user nginx;
worker_processes 2;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
use epoll;
accept_mutex off;
}
http {
include /etc/nginx/mime.types;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request"'
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" ';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
client_max_body_size 300m;
client_body_buffer_size 128k;
gzip on;
gzip_http_version 1.0;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_proxied any;
gzip_types text/plain text/css text/xml text/javascript application/xml application/xml+rss application/javascript application/json;
gzip_disable 'MSIE [1-6]\.';
gzip_vary on;
include /etc/nginx/conf.d/*.conf;
}
The problem is both works independently but as soon as I try to connect them on one network they crash.
Error throws localhost refused to connect on both services
You need to expose port 8080 on your docker-compose
ports:
- 8080
- 50000:50000
This may help in nginx (jenkins.conf)
proxy_redirect http://master:8080/;
First of all you publish the master and nginx on port 80. This is way too much. Just publish port 80 on nginx. The other ports on master are not needed, except you want to bind this port 50000 on your local address and port.
Container in the same network can resolve the names and reaches their ports without being published. Keep in mind Container cannot call localhost to reach your host. It would just solve the container itself. Use the container names inside the configurations and container itself.
UPDATE:
I've setup my configuration like the following. This worked for me.
docker-compose.yaml:
version: '3.7'
services:
master:
image: jenkins/jenkins:alpine
networks:
- jenkins-net
volumes:
- jenkins-log:/var/log/jenkins
- jenkins-data:/var/jenkins_home
nginx:
image: nginx:alpine
ports:
- "80:80"
volumes:
- type: bind
source: ./nginx.conf
target: /etc/nginx/conf.d/default.conf
networks:
- jenkins-net
networks:
jenkins-net:
volumes:
jenkins-log:
jenkins-data:
nginx.conf:
server {
listen 80;
server_name localhost;
access_log off;
location / {
proxy_pass http://master:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto http;
proxy_max_temp_file_size 0;
proxy_connect_timeout 150;
proxy_send_timeout 100;
proxy_read_timeout 100;
proxy_buffer_size 8k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
}
}
Then i was able to call http://localhost and worked as expected. Hope it works as expected and you can adapt it to your personal needs.
This is the configuration that worked in my case:
docker-compose.yml:
version: '3.7'
services:
master:
build: ./jenkins-master
networks:
- jenkins-net
volumes:
- jenkins-log:/var/log/jenkins
- jenkins-data:/var/jenkins_home
nginx:
build: ./jenkins-nginx
ports:
- "80:80"
volumes:
- type: bind
source: ./jenkins-nginx/nginx.conf
target: /etc/nginx/conf.d/default.conf
networks:
- jenkins-net
networks:
jenkins-net:
volumes:
jenkins-log:
jenkins-data:
Nginx-Dockerfile:
FROM nginx:mainline-alpine
COPY ./jenkins.conf /etc/nginx/conf.d/jenkins.conf
COPY ./nginx.conf /etc/nginx/nginx.conf
EXPOSE 80
CMD ["nginx"]
Jenkins-Dockerfile:
FROM jenkins/jenkins:alpine
LABEL maintainer=''
USER root
RUN mkdir /var/log/jenkins
RUN mkdir /var/cache/jenkins
RUN chown -R jenkins:jenkins /var/log/jenkins
RUN chown -R jenkins:jenkins /var/cache/jenkins
USER jenkins
ENV JAVA_OPTS='-Xmx8192m'
ENV JENKINS_OPTS=' --handlerCountMax=300 --logfile=/var/log/jenkins/jenkins.log --webroot=/var/cache/jenkins/war'
nginx.conf:
server {
listen 80;
server_name localhost;
access_log off;
location / {
proxy_pass http://master:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto http;
proxy_max_temp_file_size 0;
proxy_connect_timeout 150;
proxy_send_timeout 100;
proxy_read_timeout 100;
proxy_buffer_size 8k;
proxy_buffers 4 32k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
}
}
jenkins.conf:
daemon off;
user nginx;
worker_processes 2;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
use epoll;
accept_mutex off;
}
http {
include /etc/nginx/mime.types;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request"'
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for" ';
access_log /var/log/nginx/access.log main;
sendfile on;
keepalive_timeout 65;
client_max_body_size 300m;
client_body_buffer_size 128k;
gzip on;
gzip_http_version 1.0;
gzip_comp_level 6;
gzip_buffers 16 8k;
gzip_proxied any;
gzip_types text/plain text/css text/xml text/javascript application/xml application/xml+rss application/javascript application/json;
gzip_disable 'MSIE [1-6]\.';
gzip_vary on;
include /etc/nginx/conf.d/*.conf;
}
For me it worked after I changed the name for jenkins.conf with nginx.conf. Also I have used this git repo: https://github.com/lucasp90/jenkins-nginx and it worked fine.
I have a environment where I have 2 tomcat containers are exposed say dev and test on ports 8080 and 8081 respectively. I am able to access the tomcat instances with host and port combinations as below.
http://<ip>:8080
http://<ip>:8081
Now I am trying to setup an nginx container as a proxy to send all /dev requests to dev(8080) container and all /test requests to test(8081) container.
Below is my docker-compose.yml
version: "3.5"
services:
web1:
image: "tomcat:latest"
container_name: "web1"
ports:
- "8080:8080"
web2:
image: "tomcat:latest"
container_name: "web2"
ports:
- "8081:8080"
nginx:
image: "nginx:latest"
container_name: "nginx"
ports:
- "8000:80"
volumes:
- "./nginx.conf:/etc/nginx/nginx.conf"
#- "./default.conf:/etc/nginx/conf.d/default.conf"
Below is my nginx.conf file
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
include /etc/nginx/conf.d/*.conf;
server {
listen 80 default_server;
listen [::]:80 default_server;
server_name _;
proxy_redirect off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Scheme $scheme;
client_max_body_size 0;
location / {
}
location /dev {
proxy_pass http://35.239.73.252:8080/;
}
location /test {
proxy_pass http://35.239.73.252:8081/;
}
}
}
Now the problem is when i try to load my tomcat containers directly they work fine. But when they are accessed through nginx with uri paths /dev and /test the pages are broken and images and css are not loaded.
What could the issue and how to fix it.
I believe you need a closing "/" after both your location path and your target. Here's a working example from a project of mine:
location ^~ /ll/ {
proxy_pass http://werther:8080/;
}
I am attempting to use an NGINX container to host a static web application. This container should also redirect certain requests (i.e. www.example.com/api/) to another container on the same network.
I am getting the "host not found in upstream" issue when calling docker-compose build, even though I am enforcing that the NGINX container is the last to be built.
I have tried the following solutions:
Enforcing a network name and aliases (as per Docker: proxy_pass to another container - nginx: host not found in upstream)
Adding a "resolver" directive (as per Docker Networking - nginx: [emerg] host not found in upstream and others), both for 8.8.8.8 and 127.0.0.11.
Rewriting the nginx.conf file to have the upstream definition before the location that will redirect to it, or after it.
I am running on a Docker for Windows machine that is using a mobylinux VM to run the relevant container(s). Is there something I am missing? It isn't obvious to me that the "http://webapi" address should resolve correctly, as the images are built but not running when you are calling docker-compose.
nginx.conf:
user nginx;
worker_processes 1;
error_log /var/log/nginx/error.log warn;
pid /var/run/nginx.pid;
events {
worker_connections 1024;
}
http {
include /etc/nginx/mime.types;
default_type application/octet-stream;
log_format main '$remote_addr - $remote_user [$time_local] "$request" '
'$status $body_bytes_sent "$http_referer" '
'"$http_user_agent" "$http_x_forwarded_for"';
access_log /var/log/nginx/access.log main;
sendfile on;
#tcp_nopush on;
keepalive_timeout 65;
#gzip on;
upstream docker-webapi {
server webapi:80;
}
server {
listen 80;
server_name localhost;
location / {
root /wwwroot/;
try_files $uri $uri/ /index.html;
}
location /api {
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://docker-webapi;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
}
error_page 500 502 503 504 /50x.html;
location = /50x.html {
root /usr/share/nginx/html;
}
}
}
docker-compose:
version: '3'
services:
webapi:
image: webapi
build:
context: ./src/Api/WebApi
dockerfile: Dockerfile
volumes:
- /etc/example/secrets/:/app/secrets/
ports:
- "61219:80"
model.api:
image: model.api
build:
context: ./src/Services/Model/Model.API
dockerfile: Dockerfile
volumes:
- /etc/example/secrets/:/app/secrets/
ports:
- "61218:80"
webapp:
image: webapp
build:
context: ./src/Web/WebApp/
dockerfile: Dockerfile
ports:
- "80:80"
depends_on:
- webapi
Dockerfile:
FROM nginx
RUN mkdir /wwwroot
COPY nginx.conf /etc/nginx/nginx.conf
COPY wwwroot ./wwwroot/
EXPOSE 80
RUN service nginx start
You issue is below
RUN service nginx start
You never run service command inside docker, because there is no init system. Also RUN commands are run during build time.
nginx original image has everything you need for nginx to start fine. So just remove the line and it would work.
By default nginx image has below CMD instruction
CMD ["nginx" "-g" "daemon off;"]
You can easily find that out by running the below command
docker history --no-trunc nginx | grep CMD