To get the hang of nginx with docker, I have a very simple nginx.conf file + docker-compose, running 2 containers for 1 service (service itself+db).
What I want:
localhost --> show static page
localhost/pics --> show another static page
localhost/wekan --> redirect to my container, which is running on port 3001.
the last part (redirect to docker-container) does not work. The app can be reached under localhost:3001, tho.
My nginx.conf:
user www-data;
worker_processes auto;
pid /run/nginx.pid;
include /etc/nginx/modules-enabled/*.conf;
events {
worker_connections 768;
# multi_accept on;
}
http {
server{
listen 80;
location / {
root /home/user/serverTest/up1; #index.html is here
}
location /wekan {
proxy_pass http://localhost:3001;
rewrite ^/wekan(.*)$ $1 break; # this didnt help either
}
location /pics {
proxy_pass http://localhost/example.jpg;
}
location ~ \.(gif|jpg|png)$ {
root /home/user/serverTest/data/images;
}
}
docker-compose.yml:
version: '2'
services:
wekandb:
image: mongo:3.2.21
container_name: wekan-db
restart: always
command: mongod --smallfiles --oplogSize 128
networks:
- wekan-tier
expose:
- 27017
volumes:
- /home/user/wekan/wekan-db:/data/db
- /home/user/wekan/wekan-db-dump:/dump
wekan:
image: quay.io/wekan/wekan
container_name: wekan-app
restart: always
networks:
- wekan-tier
ports:
# Docker outsideport:insideport
- 127.0.0.1:3001:8080
environment:
- MONGO_URL=mongodb://wekandb:27017/wekan
- ROOT_URL=http://localhost
Looking at the nginx-error logs, I get this:
2018/12/17 11:57:16 [error] 9060#9060: *124 open() "/home/user/serverTest/up1/31fb090e9e6464a4d62d3588afc742d2e11dc1f6.js" failed (2: No such file or directory),
client: 127.0.0.1, server: ,
request: "GET /31fb090e9e6464a4d62d3588afc742d2e11dc1f6.js?meteor_js_resource=true HTTP/1.1", host: "localhost",
referrer: "http://localhost/wekan"
So I guess this makes sense because in my understanding, nginx is now adding the redirect to the root given # /, but clearly this is not where the container is running.
How do I prevent that?
Your nginx cannot access the local network interface of your docker composition.
Try to bind wekan's port like this:
wekan:
ports:
- 127.0.0.1:3001:8080
Mind the 127.0.0.1
See https://docs.docker.com/compose/compose-file/#ports
the problem was within the docker-compose configuration.
For anyone wondering, all you need is a proxy pass addr:port or addr:port/ whereas the 2nd option does the same as the rewrite part, so this can be skipped.
Apart from that, i had to add the /wekan into the ROOT_URL inside my docker-compose
Related
I'm using "linuxserver"'s swag image to reverse-proxy my docker-compose images. I want to serve one of my images as my domain root and the other one as a subdomain (e.g. root-image # mysite.com and subdomain-image # staging.mysite.com). Here are the steps I went through:
redirected my domain and subdomain names to my server in Cloudflare (pinging them shows my server's IP and this is step OK! )
Configured Cloudflare DNS config for swag (this working OK!)
Configured docker-compose file:
swag:
image: linuxserver/swag:version-1.14.0
container_name: swag
cap_add:
- NET_ADMIN
environment:
- PUID=1000
- PGID=1000
- URL=mysite.com
- SUBDOMAINS=www,staging
- VALIDATION=dns
- DNSPLUGIN=cloudflare #optional
- EMAIL=me#mail.com #optional
- ONLY_SUBDOMAINS=false #optional
- STAGING=false #optional
volumes:
- /docker-confs/swag/config:/config
ports:
- 443:443
- 80:80 #optional
restart: unless-stopped
root-image:
image: ghcr.io/me/root-image
container_name: root-image
restart: unless-stopped
subdomain-image:
image: ghcr.io/me/subdomain-image
container_name: subdomain-image
restart: unless-stopped
Defined a proxy conf for my root-image (at swag/config/nginx/proxy-confs/root-image.subfolder.conf)
location / {
include /config/nginx/proxy.conf;
resolver 127.0.0.11 valid=30s;
set $upstream_app root-image;
set $upstream_port 3000;
set $upstream_proto http;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
Commented out the nginx's default location / {} block (at swag/config/nginx/site-confs/default)
Defined a proxy conf for my subdomain-image (at swag/config/nginx/proxy-confs/subdomain-image.subdomain.conf)
server {
listen 443 ssl;
listen [::]:443 ssl;
server_name staging.*;
include /config/nginx/ssl.conf;
client_max_body_size 0;
location / {
include /config/nginx/proxy.conf;
resolver 127.0.0.11 valid=30s;
set $upstream_app subdomain-image;
set $upstream_port 443;
set $upstream_proto https;
proxy_pass $upstream_proto://$upstream_app:$upstream_port;
}
}
Both images expose port 3000. Now my root image is working fine (OK!), but I'm getting 502 error for my subdomain image. I checked the nginx error log, but it doesn't show anything meaningful for me:
*35 connect() failed (111: Connection refused) while connecting to upstream, client: xxx.xxx.xxxx.xxx, server: staging.*, request: "GET /favicon.ico HTTP/2.0", upstream: "https://xxx.xxx.xxx:443/favicon.ico", host: "staging.mysite.com", referrer: "https://staging.mysite.com"
Docker logs for all the 3 containers are also fine (without showing any warnings or errors).
Which step I'm going wrong or is there anything I missed? Thanks for helping
I have a website host on a server in Digital Ocean that is behaving weirdly.
The website is written in Flask which is deployed in Docker and using reverse proxy with a combination of Let's Encrypt to host on the web.
The website's domain is mes.th3pl4gu3.com.
If I go on mes.th3pl4gu3.com/web/ the website appears and works normal.
If I go on mes.th3pl4gu3.com/web it gives me http://localhost/web/ in the URl instead and conenction fails.
However, when I run it locally, it works fine.
I've checked my nginx logs, when i browse mes.th3pl4gu3.com/web/ the access_logs returns success but when i use mes.th3pl4gu3.com/web nothing comes to the log.
Does anyone have any idea what might be causing this ?
Below are some codes that might help in troubleshooting.
server {
server_name mes.th3pl4gu3.com;
location / {
access_log /var/log/nginx/mes/access_mes.log;
error_log /var/log/nginx/mes/error_mes.log;
proxy_pass http://localhost:9003; # The mes_pool nginx vip
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/..........
ssl_certificate_key /etc/letsencrypt/........
include /etc/letsencrypt/.........
ssl_dhparam /etc/letsencrypt/......
}
server {
if ($host = mes.th3pl4gu3.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name mes.th3pl4gu3.com;
return 404; # managed by Certbot
}
Docker Instances:
7121492ad994 docker.pkg.github.com/mervin16/mauritius-emergency-services-api/mes:1.2.5 "uwsgi
app.ini" 4 weeks ago Up 4 weeks 0.0.0.0:9002->5000/tcp mes-instace-2
f4dc063e33b8 docker.pkg.github.com/mervin16/mauritius-emergency-services-api/mes:1.2.5 "uwsgi app.ini" 4 weeks ago Up 4 weeks 0.0.0.0:9001->5000/tcp mes-instace-1
fb269ed2229a nginx "/docker-entrypoint.…" 4 weeks ago Up 4 weeks 0.0.0.0:9003->80/tcp nginx_mes
2ad5afe0afd1 docker.pkg.github.com/mervin16/mauritius-emergency-services-api/mes:1.2.5 "uwsgi app.ini" 4 weeks ago Up 4 weeks 0.0.0.0:9000->5000/tcp mes-backup
docker-compose-instance.yml
version: "3.8"
# Contains all Production instances
# Should always stay up
# In case both instances fails, backup instance will takeover
services:
mes-instace-1:
container_name: mes-instace-1
image: "docker.pkg.github.com/mervin16/mauritius-emergency-services-api/mes:${MES_VERSION}"
networks:
- mes_net
volumes:
- ./data:/app/data
env_file:
- secret.env
ports:
- "9001:5000"
restart: always
environment:
- MES_VERSION=${MES_VERSION}
mes-instace-2:
container_name: mes-instace-2
image: "docker.pkg.github.com/mervin16/mauritius-emergency-services-api/mes:${MES_VERSION}"
networks:
- mes_net
volumes:
- ./data:/app/data
env_file:
- secret.env
ports:
- "9002:5000"
restart: always
environment:
- MES_VERSION=${MES_VERSION}
networks:
mes_net:
name: mes_network
driver: bridge
docker-compose.yml
version: "3.8"
# Contains the backup instance and the nginx server
# This should ALWAYS stay up
services:
mes-backup:
container_name: mes-backup
image: "docker.pkg.github.com/mervin16/mauritius-emergency-services-api/mes:${MES_VERSION}"
networks:
- mes_net
volumes:
- ./data:/app/data
env_file:
- secret.env
ports:
- "9000:5000"
restart: always
environment:
- MES_VERSION=${MES_VERSION}
nginx_mes:
image: nginx
container_name: nginx_mes
ports:
- "9003:80"
networks:
- mes_net
volumes:
- ./nginx/nginx.conf:/etc/nginx/conf.d/default.conf
- ./log/nginx:/var/log/nginx
depends_on:
- mes-backup
restart: always
networks:
mes_net:
name: mes_network
driver: bridge
I have multiple instances for load balancing across apps.
Can someone please help or if anyone has any clue why this might be happening ?
As long as I tested the page https://mes.th3pl4gu3.com/web with or without / trailing at the end it worked fine. (Firefox version 87 on Ubuntu)
Maybe there is a bug / problem with your web browser or any kind of VPN / Proxy you are running. Make sure all of them are off.
Plus on Nginx you can get rid of trailing / using rewrite rule
e.g.
location = /stream {
rewrite ^/stream /stream/;
}
which tells Nginx parse stream as it is stream/
and for making sure you are not facing any issue because of cached data, disable and clear all the cache. On your web browser hit F12 -> go to console tab , hit F1 and there disable cache. On Nginx set "no cache" for header, e.g.
add_header Last-Modified $date_gmt;
add_header Cache-Control 'no-store, no-cache, must-revalidate, proxy-revalidate, max-age=0';
if_modified_since off;
expires off;
etag off;
I tested your site with Chrome, Safari, and Curl, and I can't see that issue.
Try clear your cache.
Method 1: Ctrl-Shift-R
Method 2: DevTool -> Application/Storage -> Clear site data
My guess is it is related to your Flask SERVER_NAME
As you said, locally your SERVER_NAME might be set to localhost:8000.
However, on production, it would need to be something like
SERVER_NAME = "th3pl4gu3.com"
Your issue is that you are pulling the SERVER_NAME from the Flask SERVER_NAME variable, so it ends up as https://localhost/web instead of your desired URL.
I have a simple flask app running on port 5000 inside the container , and i'm trying to add nginx load balance to scale the app(3 instances)
Here is my docker-compose file :
version: "3.7"
services:
chat-server:
image: chat-server
build:
context: .
dockerfile: Dockerfile
volumes:
- './chat_history:/src/app/chat_history'
networks:
- "chat_net"
ngnix-server:
image: nginx:1.13
ports:
- "8080:80"
volumes:
- './ngnix.conf:/etc/ngnix/nginx.conf'
networks:
- "chat_net"
depends_on:
- chat-server
networks:
chat_net:
And here is my nginx.conf file :
events { worker_connections 1024;}
http {
upstream app {
server chat-server_1:5000;
server chat-server_2:5000;
server chat-server_3:5000;
}
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}
both services are on the same chat_net network , but when i hit localhost:8080 on my browser im getting the nginx default page , why is that? what am i missing ?
You have a typo and are not mounting in your nginx.conf file correctly.
You spell it ngnix in a couple of places in your volumes section and the container runs with the default config (hence default home page).
Once you fix that, you will probably hit the error mentioned by #Federkun (nginx won't be able to resolve the 3 domain names you're proxying).
You also have your server directive in the wrong place (it needs to be within the http section).
This should be the modified version of your file:
events { worker_connections 1024;}
http {
upstream app {
server chat-server:5000;
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}
}
Notice this is better than needing nginx to be aware of the replica count. You can run docker-compose up with --scale chat-server=N and resize at anytime by running the same command with a different N without downtime.
#Jedi
shouldn't be at least below?
events { worker_connections 1024;}
http {
upstream app {
least_conn;
server chat-server_1:5000;
server chat-server_2:5000;
}
server {
listen 80;
location / {
proxy_pass http://app;
}
}
}
I have a docker-compose.yaml similar to this (shortened for simplicity):
# ...
services:
my-ui:
# ...
ports:
- 5402:8080
networks:
- my-net
networks:
my-net:
external:
name: my-net
and I'm trying to set up nginx as a reverse proxy with this configuration:
upstream client {
server my-ui:5402;
}
server {
listen 80;
location / {
proxy_pass http://client;
}
}
and this is the docker-compose.yaml I have for nginx:
# ...
services:
reverse-proxy:
# ...
ports:
- 5500:80
networks:
- my-net
networks:
my-net:
external:
name: my-net
What happens now is that when I run my-ui and reverse-proxy (each using its own docker-compose up), and I go to http://localhost:5500, I get a Bad Gateway message, and my nginx logs says this:
connect() failed (111: Connection refused) while connecting to
upstream, client: 172.19.0.1, server: , request: "GET / HTTP/1.1",
upstream: "http://172.19.0.5:5402/", host: "localhost:5500"
If I exec into my nginx container and use ping:
ping my-ui
ping 172.19.0.5
Both are successful, but if I want to, for example, curl:
curl -L http://my-ui
curl -L http://my-ui:5402
curl -L http://172.19.0.1
All of them fail with connection refused message. What am I missing here?
PS: I'm not sure, but it might be useful to add that my-ui is a basic vuejs application, running on Webpack dev server.
PS2: I also tried passing host headers etc. but same result
The name of the container (my-ui) resolves to the IP of the container. Therefor you have to provide in upstream the port of the container and not the port you have mapped to the host.
upstream client {
server my-ui:8080;
}
server {
listen 80;
location / {
proxy_pass http://client;
}
}
You could also configure your upstream with the name of your host machine and use the mapped port. (server <name of host>:5402) But this could get quite messy and you would lose the advantage of isolating services with docker networks.
Furthermore you could also remove the port mapping unless you need to access the webservice without reverse proxy:
# ...
services:
reverse-proxy:
# ...
# ports:
# - 5500:80
My nginx.conf file currently has the routes defined directly:
worker_processes auto;
events { worker_connections 1024; }
http {
upstream wordSearcherApi {
least_conn;
server api1:61370 max_fails=3 fail_timeout=30s;
server api2:61370 max_fails=3 fail_timeout=30s;
server api3:61370 max_fails=3 fail_timeout=30s;
}
server {
listen 80;
server_name server_name 0.0.0.0;
location / {
proxy_pass http://wordSearcherApi;
}
}
}
Is there any way to create just one service in docker-compose.yml and when docker-compose up --scale api=3, does nginx do automatic load balance?
Nginx
Dynamic upstreams are possible in Nginx (normal, sans Plus) but with tricks and limitations.
You give up on upstream directive and use plain proxy_pass.
It gives round robin load balancing and failover, but no extra feature of the directive like weights, failure modes, timeout, etc.
Your upstream hostname must be passed to proxy_pass by a variable and you must provide a resolver.
It forces Nginx to re-resolve the hostname (against Docker networks' DNS).
You lose location/proxy_pass behaviour related to trailing slash.
In the case of reverse-proxying to bare / like in the question, it does not matter. Otherwise you have to manually rewrite the path (see the references below).
Let's see how it works.
docker-compose.yml
version: '2.2'
services:
reverse-proxy:
image: nginx:1.15-alpine
volumes:
- ./nginx.conf:/etc/nginx/nginx.conf
ports:
- 8080:8080
app:
# A container that exposes an API to show its IP address
image: containous/whoami
scale: 4
nginx.conf
worker_processes 1;
events {
worker_connections 1024;
}
http {
access_log /dev/stdout;
error_log /dev/stderr;
server {
listen 8080;
server_name localhost;
resolver 127.0.0.11 valid=5s;
set $upstream app;
location / {
proxy_pass http://$upstream:80;
}
}
}
Then...
docker-compose up -d
seq 10 | xargs -I -- curl -s localhost:8080 | grep "IP: 172"
...produces something like the following which indicates the requests are distributed across 4 app containers:
IP: 172.30.0.2
IP: 172.30.0.2
IP: 172.30.0.3
IP: 172.30.0.3
IP: 172.30.0.6
IP: 172.30.0.5
IP: 172.30.0.3
IP: 172.30.0.6
IP: 172.30.0.5
IP: 172.30.0.5
References:
Nginx with dynamic upstreams
Using Containers to Learn Nginx Reverse Proxy
Dynamic Nginx configuration for Docker with Python
Traefik
Traefik relies on Docker API directly and may be a simpler and more configurable option. Let's see it in action.
docker-compose.yml
version: '2.2'
services:
reverse-proxy:
image: traefik
# Enables the web UI and tells Traefik to listen to docker
command: --api --docker
ports:
- 8080:80
- 8081:8080 # Traefik's web UI, enabled by --api
volumes:
# So that Traefik can listen to the Docker events
- /var/run/docker.sock:/var/run/docker.sock
app:
image: containous/whoami
scale: 4
labels:
- "traefik.frontend.rule=Host:localhost"
Then...
docker-compose up -d
seq 10 | xargs -I -- curl -s localhost:8080 | grep "IP: 172"
...also produces something the output that indicates the requests are distributed across 4 app containers:
IP: 172.31.0.2
IP: 172.31.0.5
IP: 172.31.0.6
IP: 172.31.0.4
IP: 172.31.0.2
IP: 172.31.0.5
IP: 172.31.0.6
IP: 172.31.0.4
IP: 172.31.0.2
IP: 172.31.0.5
In the Traefik UI (http://localhost:8081/dashboard/ in the example) you can see it recognised the 4 app containers:
References:
The Traefik Quickstart (Using Docker)
It's not possible with your current config since it's static. You have two options -
1. Use docker engine swarm mode - You can define replicas & swarm internal DNS will automatically balance the load across those replicas.
Ref - https://docs.docker.com/engine/swarm/
2. Use famous Jwilder nginx proxy - This image listens to the docker sockets, uses templates in GO to dynamically change your nginx configs when you scale your containers up or down.
Ref - https://github.com/jwilder/nginx-proxy