Reverse Proxy with Hostnames with Docker compose - docker

I have 2 APIs and a front end that work in docker containers. They can all be created via docker-compose and I have an Nginx reverse proxy that should route to them using their hostnames i.e. api-1.my-project.localhost. In fact, I have had this process working, but every so often, like this morning, I get the "Non-existent domain" or "We can't find this site" when visiting it using these hostnames.
I don't believe the docker-compose.yml is all that important but I have included it here:
version: "3.2"
services:
api-1:
# Runs on port 5000
image: my-companies-docker-registry.api-1
api-2:
# Runs on port 5000
image: my-companies-docker-registry.api-2
main-front-end:
# Runs on port 3000
image: my-companies-docker-registry.main-front-end
my-project:
image: nginx
depends_on:
- api-1
- api-2
- main-front-end
ports:
- 80:80
volumes:
- type: bind
read_only: true
source: ./nginx
target: /etc/nginx
My nginx config looks like so:
events {}
http {
# Api 1
upstream api-1{
server api-1:5000;
}
server {
listen 80;
server_name api-1.my-project.localhost;
location / {
proxy_pass http://api-1;
}
}
# Api 2
upstream api-2 {
server api-2:5000;
}
server {
listen 80;
server_name api-2.my-project.localhost;
location / {
proxy_pass http://api-2;
}
}
# Main Front end
upstream main-front-end {
server main-front-end:3000;
}
server {
listen 80 default_server deferred;
server_name my-project.localhost www.my-project.localhost;
location / {
proxy_pass http://main-front-end;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
}
}
}
The aim of this is so that I can access the site using http://my-project.localhost and the APIs using api-1.my-project.localhost and api-2.my-project.localhost. As I have listen 80 default_server deferred by visiting http://localhost I can hit nginx, which lets me see my front end site. So docker-compose has bound the ports correctly.
At some point in the past, I have been able to access the sites using the my-project.localhost suffix but now this is no longer the case. This site suggests adding a host entry for every site, I don't remember doing this before, as there are a lot of sites, this would have taken a while but it is possible I did and now they have been removed. I am also aware that Nginx is not docker, so I have no idea how those hostnames would have been extracted and added to my machines host resolution process.
How could this have worked before and not now? And how can I get my setup to work without making manual host file changes?

Related

One nginx config for multiple dockerized ShinyR apps under different locations

we're trying to host a bunch of dockerized shinyR applications on a server. Each shinyR app is provided via docker-compose under a specific port. Each app should be available under its own location, e.g. https://example.com/app1 and https://example.com/app2
Requests under location app1 shall be directed to http://localhost:.
This is an excerpt of the nginx config file we're using:
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name example.com;
root /var/www/html;
location /app1 {
proxy_pass http://127.0.0.1:3040/;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
}
...
}
What I receive is the bare shinyR application which is good but the client requests the proper assets (various JS libraries, images, etc) under the wrong location (/ and not /app1/) as can be seen in my browsers dev console:
GET https://example.com/shiny-javascript-1.7.4/shiny.min.js net::ERR_ABORTED 404
I can request this asset via https://example.com/app1/shiny-javascript-1.7.4/shiny.min.js, so it's served properly somehow.
My first question is, whether this is rather a shinyR or an NGINX problem? My guess is that shiny is telling the browser to request the assets from a wrong location and that we should start digging there but since I'm neither an expert on shinyR nor nginx it's a bit hard to tell where to start debugging.
For the sake of completeness here's the compose file running the shinyR apps
version: '3'
services:
app1:
image: "registry.gitlab.com/dezim/somefoo/bar:latest"
container_name: "app1"
command: R -e "shiny::runApp(appDir='/home/app', port=3040, host='0.0.0.0')"
restart: unless-stopped
ports:
- '127.0.0.1:3040:3040'

Running Container Registry in Gitlab with relative url

I'm running gitlab and nginx in docker. I had to use an unbound version of nginx because there's other applications running and I can only publish ports 80 and 443. So far I've managed to access gitlab with my.domain.com/gitlab/ and can also login, upload projects, etc...
I've wanted to use the container registry for uploading and storing images from my projects, which is why I've uncommented gitlab_rails['registry_enabled'] = true in gitlab.rb. Now the container registry is visible for my projects, however I get a Docker Connection Error when trying to access the page.
My question is, are there any other settings I have to tweak to get the inbuilt container registry running or did I already mess things up in the way I've set this up? Especially since I need gitlab to run on a relative url and my projects url for cloning/pulling is something like https://<server.ip.address>/gitlab/group-name/test-project, even though browser tabs show https://my.domain.com/gitlab/group-name/test-project
So far, this is my setup:
nginx
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name my.domain.com www.my.domain.com;
server_tokens off;
ssl_certificate /etc/nginx/ssl/live/my.domain.com/fullchain.pem;
ssl_certificate_key /etc/nginx/ssl/live/my.domain.com/privkey.pem;
##############
## GITLAB ##
##############
location /gitlab/ {
root /home/git/gitlab/public;
proxy_http_version 1.1;
proxy_pass http://<server.ip.address>:8929/gitlab/;
gzip off;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
gitlab docker-compose.yml
services:
git:
image: gitlab/gitlab-ce:14.10.0-ce.0
restart: unless-stopped
container_name: gitlab
hostname: 'my.domain.com'
environment:
GITLAB_OMNIBUS_CONFIG: |
gitlab_rails['gitlab_shell_ssh_port'] = 2224
external_url 'http://<server.ip.address>:8929/gitlab/'
gitlab_rails['registry_enabled'] = true
ports:
- "8929:8929"
- "2224:2224"

How do I reverse-proxy with Nginx to a container inside Docker network made by docker-compose?

I have a few web apps running on my VPS. I have one domain, and I run the apps on subdomains using Nginx and server_name directives.
I decided to containerize my newest one, using Docker and docker-compose.
However, I can't reach this app with Nginx.
On my VPS, I configured app-client-proxy.nginx so that it tries to redirect to Docker IP with port that client-app (name changed) listens to:
server {
root /home/dartungar/projects/app-client;
server_name app.dartungar.com www.app.dartungar.com;
location / {
proxy_pass http://172.17.0.1:8043;
}
# omitted certbot things
}
server { # server that listens to :80 and redirects to :443 }
Here's docker-compose.yml:
version: "3"
services:
client:
image: dartungar/app-client
restart: unless-stopped
ports:
- 8043:443
Inside app-client container I also have Nginx that listens on 443 and serves html file:
server {
listen 443 ssl;
location / {
root /usr/share/nginx/html;
index index.html index.htm;
try_files $uri $uri/ /index.html =404;
}
}
When I try to open URL app.dartungar.com, I get 502 Bad Gateway.
I know it has to do with docker networking - my guess is that I have to reverse-proxy to network IP, not Docker IP.
How do I do that? Perhaps I missed it in the docks - if so, just send me a link.
Cheers!

How to configure nginx with multiple docker-compose.yml files?

If I want to setup nginx with my docker containers, one option is to setup the nginx instance in my docker-compose.yml, and link the nginx container to all application containers.
The drawback of this approach, however, is that the docker-compose.yml becomes server-level, since only one nginx container can expose ports 80/443 to the internet.
I'm interested in being able to define several docker-compose.yml files on the same server, but still easily expose the public-facing containers in each compose file via a single server-specific nginx container.
I feel this should be pretty easy, but I haven't been able to find a good resource or example for this.
First, you need to create a network for nginx and the proxied containers:
docker network create nginx_network
Next, configure the nginx container in a compose file like this:
services:
nginx:
image: your_nginx_image
ports:
- "80:80"
- "443:443"
networks:
- nginx_network
networks:
nginx_network:
external: true
After that you can run proxied containers:
services:
webapp1:
image: ...
container_name: mywebapp1
networks:
- nginx_network # proxy and app must be in same network
- webapp1_db_network # you can use additional networks for some stuff
database:
image: ...
networks:
- webapp1_db_network
networks:
nginx_network:
external: true
webapp1_db_network: ~ # this network won't be accessible from outside
Also, to make this work you need to configure your nginx properly:
server {
listen 80;
server_name your_app.example.com;
# Docker DNS
resolver 127.0.0.11;
location / {
# hack to prevent nginx to resolve container's host on start up
set $docker_host "mywebapp1";
proxy_pass http://$docker_host:8080;
}
}
You need to tell nginx to use Docker's DNS, so it will be able to access containers by their names.
But note that if you run the nginx container before the others, then nginx will try to resolve another container's host and fail, because the other containers are not running yet. You can use a hack with placing the host into a variable. With this hack, nginx won't try to resolve host until receiving a request.
With this combination you can have nginx always up, while starting and stopping proxied applications independently.
Update:
If you want a more dynamic solution, you can modify the nginx config as follows:
server {
listen 80;
resolver 127.0.0.11;
# define server_name with regexp which will read subdomain into variable
server_name ~^(?<webapp>.+)\.example\.com;
location / {
# use variable from regexp to pass request to desired container
proxy_pass http://$webapp:8080;
}
}
With this configuration, a request to webapp1.example.com will be passed to container "webapp1", webapp2.example.com to "webapp2" etc. You only need to add DNS records and run app containers with right name.
The accepted answer is great, but since I am in the trenches with this right now I'm going to expand upon it with my debugging steps in hopes it helps someone (myself in the future?)
Docker-compose projects often use nginx as reverse-proxy to route http traffic to the other docker services. nginx was a service in my projectfolder/docker-compose.yml which was connected to two docker networks.
One was the default network created when I used docker-compose up on projectfolder/docker-compose.yml (It is named projectfolder_default and services connect to it by default UNLESS you have a networks property for your service with another network, then make sure you add - default to the list). When I ran docker network ls I saw projectfolder_default in the list and when I ran docker network inspect projectfolder_default I saw the nginx container, so everything was good.
The other was a network called my_custom_network that I set up myself. I had a startup script that created it if it did not exist using https://stackoverflow.com/a/53052379/13815107 I needed it in order to talk to the web service in otherproject/docker-compose.yml. I had correctly added my_custom_network to:
nginx service's networks list of projectfolder/docker-compose.yml
bottom of projectfolder/docker-compose.yml
web service's networks in otherproject/docker-compose.yml
bottom of otherproject/docker-compose.yml
The network showed up and had the right containers using docker network ls and docker network inspect my_custom_network
However, I assumed that proxy_pass to http://web in my server.conf would map to the docker service web.projectfolder_default. I was mistaken. To test this I opened shell on the nginx container (docker exec -it nginx sh). When I used ping web (may need to apt-get update, apt-get install iputils-ping) it succeeded, but it printed a url with my_custom_network which is how I figured out the mistake.
Update: I tried using http://web.my_custom_network in server.conf.template and it routed great, but my webapp (Django-based) choked on underscores in the url. I renamed web to web2 in otherproject/docker-compose.yml and then used something like docker stop otherproject_web and docker rm otherproject_web to get rid of the bad one.
projectfolder/docker-compose.yml
services:
# http://web did NOT map to this service!! Use http://web.main_default or change the names
web:
...
nginx:
...
links:
- web
networks:
- default
- my_custom_network
...
networks:
- my_custom_network
external: true
otherproject/docker-compose.yml
services:
# http://web connected to this service instead. You could use http://web.my_custom_network to call it out instead
web:
...
networks:
- default
- my_custom_network
...
networks:
- my_custom_network
external: true
projectfolder/.../nginx/server.conf.template (next to Dockerfile)
...
server {
...
location /auth {
internal;
# This routed to wrong 'web'
proxy_pass http://web:9001;
proxy_pass_request_body off;
proxy_set_header Content-Length "";
}
location / {
alias /data/dist/;
}
location /robots.txt {
alias /robots.txt;
}
# Project Folder backend
location ~ ^/(api|login|logout)/ {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300s;
proxy_read_timeout 300s;
# This routed to wrong 'web'
proxy_pass http://web:9001;
}
# Other project UI
location /other-project {
alias /data/other-project-client/dist/;
}
# Other project Django server
location ~ ^/other-project/(rest)/ {
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_connect_timeout 300s;
proxy_read_timeout 300s;
# nginx will route, but won't work some frameworks like django (it doesn't like underscores)
# I would rename this web2 in yml and use http://web2
proxy_pass http://web.my_custom_network:8000;
}
}

Subdomains, Nginx-proxy and Docker-compose

I'm looking for a way to configure Nginx to access hosted services through a subdomain of my server. Those services and Nginx are instantiated with Docker-compose.
In short, when typing jenkins.192.168.1.2, I should access to Jenkins hosted on 192.168.1.2 redirected with Nginx proxy.
A quick look of what I currently have. It doesn't work without a top domain name, so it works fine on play-with-docker.com, but not locally with for example 192.168.1.2.
server {
server_name jenkins.REVERSE_PROXY_DOMAIN_NAME;
location / {
proxy_pass http://jenkins:8080;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-Forwarded-Host $host:$server_port;
proxy_set_header X-Forwarded-Server $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
To have a look of what I want: https://github.com/Ivaprag/devtools-compose
My overall goal is to access remote docker containers without modifying clients' DNS service.
Unfortunately nginx doesn't support sub-domains on IP addresses like that.
You would either have to modify the clients hosts file (which you said you didn't want to do)...
Or you can just set your nginx to redirect like so:
location /jenkins {
proxy_pass http://jenkins:8080;
...
}
location /other-container {
proxy_pass http://other-container:8080;
}
which would allow you to access jenkins at 192.168.1.2/jenkins
Or you can try and serve your different containers through different ports. E.g:
server {
listen 8081;
location / {
proxy_pass http://jenkins:8080;
...
}
}
server {
listen 8082;
location / {
proxy_pass http://other-container:8080;
...
}
}
And then access jenkins from 192.168.1.2:8081/
If you are already using docker-compose I recommend using the jwilder nginx-proxy container.
https://github.com/jwilder/nginx-proxy
This allows you to add unlimited number of web service containers to the backend of the defined nginx proxy, for example:
nginx-proxy:
image: jwilder/nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- "/etc/nginx/vhost.d"
- "/usr/share/nginx/html"
- "/var/run/docker.sock:/tmp/docker.sock:ro"
- "nginx_certs:/etc/nginx/certs:rw"
nginx:
build:
context: ./docker/nginx/
dockerfile: Dockerfile
volumes_from:
- data
environment:
VIRTUAL_HOST: www.host1.com
nginx_2:
build:
context: ./docker/nginx_2/
dockerfile: Dockerfile
volumes_from:
- data
environment:
VIRTUAL_HOST: www.host2.com
apache_1:
build:
context: ./docker/apache_1/
dockerfile: Dockerfile
volumes_from:
- data
environment:
VIRTUAL_HOST: www.host3.com
The nginx-proxy mount the host docker sock file in order to get information about the other containers running, if any of them have the env variable VIRTUAL_HOST set then it will add it to its configuration.
I was trying to configure subdomains in nginx (host), for two virtualhosts in one LXC container.
The way it worked for me:
For apache (in the container), I created two virtual hosts: one in port 80 and the other one in port 90.
For enabling port 90 in apache2 (container), it was necessary to add the line "Listen 90" below "Listen 80" in /etc/apache2/ports.conf
For NGINX (host machine), configured two DOMAINS, both in port 80 creating independent .conf files in /etc/nginx/sites-available. Created symbolic link for each file to /etc/nginx/sites-enabled.
In the first NGINX myfirstdomain.conf file, redirect to http://my.contai.ner.ip:80.
In the second NGINX myseconddomain.conf file, redirect to http://my.contai.ner.ip:90
That was it for me !

Resources