Docker compose & nginx network hostnames - docker

I have the following configuration:
nginx started by a docker compose file acting as reverse proxy // ssl endpoint
an application service started by a different docker compose (for modularity reasons) file serving a application
reverse_proxy and app_service are connected via the app_network
Since they are connected via the app_network, I am able to address the app_service in the nginx configuration like http://app_service:8080, which is nice because I don't need to expose ports on the app service itself.
Unfortunately, nginx won't start until I brought up the app service container because it checks the existance of the hostname app_service upon startup.
How can I prevent nginx from checking the hostname on startup, maybe causing a Bad Gateway error when trying to connect while app_service is not running yet?
Configuration files for reference:
# reverse proxy docker-compose.yml
version: '3'
services:
reverse_proxy:
restart: always
container_name: reverse_proxy
image: nginx
ports:
- 80:80
- 443:443
volumes:
- /srv/docker/nginx/config:/etc/nginx
- /srv/ssl:/srv/ssl
networks:
default:
external:
name: app_network
# application service docker-compose.yml
version: '3'
services:
app_service:
restart: always
container_name: app_service
image: <my_app_image>
networks:
default:
external:
name: app_network
# nginx config
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name app.example.com;
location / {
proxy_pass http://app_service:8080;
include proxy.conf;
}
ssl_certificate /srv/ssl/<mycert>.crt;
ssl_certificate_key /srv/ssl/<mykey>.key;
}

Here is a way that worked for me:
# nginx config
server {
listen 443 ssl http2;
listen [::]:443 ssl http2;
server_name app.example.com;
location / {
set $upstream app_service:8080;
proxy_pass $upstream;
include proxy.conf;
}
ssl_certificate /srv/ssl/.crt;
ssl_certificate_key /srv/ssl/.key;
}
This results in a 502 Bad Gateway message if the host is unavailable and does not check availability at startup.

Related

Deploying a Docker container with Nginx and FastAPI on Google Cloud Platform from SSH-term

I have a some issue when trying to deploy a simple FastAPI application with Nginx on Google Cloud Platform. In my case I should use SSH-terminal to run Docker container with Nginx and FastAPI. My nginx.conf configuration looks like:
access_log /var/log/nginx/app.log;
error_log /var/log/nginx/app.log;
server {
server_name example.com;
listen 80;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name example.com;
ssl_certificate /root/ssl/cert.pem;
ssl_certificate_key /root/ssl/key.pem;
location / {
proxy_pass "http://example.com:8004/";
}
}
And my docker-compose.yml looks like:
version: '3.8'
services:
nginx-proxy:
image: nginx
container_name: nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- /var/run/docker.sock:/tmp/docker.sock:ro
- ./nginx:/etc/nginx/conf.d
- ./ssl/cert1.pem:/root/ssl/cert.pem
- ./ssl/privkey1.pem:/root/ssl/key.pem
- ./ssl/dhparam.pem:/etc/nginx/dhparam/dhparam.pem
web:
environment: [.env]
build: ./project
ports:
- 8004:8000
command: gunicorn main:app -k uvicorn.workers.UvicornWorker -w 2 -b 0.0.0.0:8000
volumes:
- ./project:/usr/src/app
networks:
default:
external:
name: nginx-proxy
Also, I have a Google Cloud VM instance with Firewall HTTP, HTTPS traffic On option, and additionally configured Firewall with rules allowed TCP connections over 443 and 80 ports (Domain name is provided by Google Cloud also, and redirects to VM's external IP address when I put it in my browser address field).
I run my docker-image from SSH-terminal with docker-compose up --build, then I get 502 Bad Gateway Nginx error in my browser (after going to example.com). I would like to know whether it is possible to run the docker image this way from inside SSH-terminal, as well as which steps did I miss to do it the right way?

Docker Nginx on a non-docker host application

I have many docker containers which pass through docker nginx combo (docker-compose.yml outlined below) and they work very well. I want docker nginx to do the same to a non-docker app thats running on localhost:8080, that is I want docker nginx container to run connections to example.com to 127.0.0.1:8080 where 127.0.0.1:8080 is ran by a non-docker app (code-server do be specific but that shouldn't matter)
version: '3'
services:
nginx-proxy:
image: jwilder/nginx-proxy
container_name: nginx-proxy
ports:
- "80:80"
- "443:443"
volumes:
- /apps/proxy/conf:/etc/nginx/conf.d
- /apps/proxy/vhost:/etc/nginx/vhost.d
- /apps/proxy/html:/usr/share/nginx/html
- /apps/proxy/dhparam:/etc/nginx/dhparam
- /apps/proxy/certs:/etc/nginx/certs:ro
- /var/run/docker.sock:/tmp/docker.sock:ro
restart: always
letsencrypt:
image: jrcs/letsencrypt-nginx-proxy-companion
container_name: nginx-proxy-le
depends_on:
- nginx-proxy
volumes:
- /apps/proxy/vhost:/etc/nginx/vhost.d
- /apps/proxy/html:/usr/share/nginx/html
- /apps/proxy/dhparam:/etc/nginx/dhparam:ro
- /apps/proxy/certs:/etc/nginx/certs
- /var/run/docker.sock:/var/run/docker.sock:ro
environment:
- NGINX_PROXY_CONTAINER=nginx-proxy
restart: always
networks:
default:
external:
name: nginx-proxy
and its running well on docker containers, the moment i include "nginx-proxy" so that it can detect them, fantastic tool. I cant simply paste something like this into default.conf (conf of nginx )
server {
listen 80;
listen [::]:80;
server_name example.com;
location / {
proxy_pass http://localhost:8080/;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection upgrade;
proxy_set_header Accept-Encoding gzip;
}
}
My guess is that what is "localhost" for me is not "localhost" for nginx given that it is inside a docker (im new to this docker stuff, so I might be talking shit). I saw the issue where they mentioned many approaches, none of them worked for me. In particular I tried running a dummy docker as CWempe suggested
docker run -d \
-e VIRTUAL_HOST=foo.bar.com \
-e VIRTUAL_PORT=8080 \
-e UPSTREAM_NAME=webserver.local \
--rm \
cwempe/docker-dummy:latest
This didnt't work, nginx didnt even detect it which made me think its probably because its not on the nginx-network so i added that (and turned it into docker-compose for convenience)
version: '3.3'
services:
docker-dummy:
environment:
- VIRTUAL_HOST=example.com
- VIRTUAL_PORT=8080
- UPSTREAM_NAME=127.0.0.1
image: 'cwempe/docker-dummy:latest'
networks:
default:
external:
name: nginx-proxy
Then looking at default.conf i get
# mydomain.com
upstream mydomain.com {
## Can be connected with "nginx-proxy" network
# code-server_docker-dummy_1
server 172.25.0.7 down;
}
server {
server_name mydomain.com;
listen 80 ;
access_log /var/log/nginx/access.log vhost;
include /etc/nginx/vhost.d/default;
location / {
proxy_pass http://example.com;
}
}
server {
server_name example.com;
listen 443 ssl http2 ;
access_log /var/log/nginx/access.log vhost;
return 500;
ssl_certificate /etc/nginx/certs/default.crt;
ssl_certificate_key /etc/nginx/certs/default.key;
}
So sure it has seen it but it also believes it is down and doesn't include the VIRTUAL_PORT at all and obviously the 127.25.0.7 IP doesn't make sense to me either. Changing 127.25.0.7 -> 127.0.0.1:8080 does nothing. Any idea how I can remedy this ? Thank you for your input in advance.

Nginx reverse proxy (in Docker) to a web app (also in Docker)

I have set up a web application in docker which is currently running internal to the host at 172.19.0.3:8888. I want this web application accessible over the internet on port 443 (https), with requests to port 80 (HTTP) redirected to 443.
I plan to use an Nginx reverse proxy in a docker container to achieve this, but I do not know how to properly configure it to point at the docker container 172.19.0.3:8888. Accessing http://172.19.0.3:8888 from the host works.
Here is the guide I tried to follow, but it just didn't show how to point at a docker container specifically.
https://medium.com/#pentacent/nginx-and-lets-encrypt-with-docker-in-less-than-5-minutes-b4b8a60d3a71
Note
If I set the port 443 proxy_pass to http://example.org, it works. So Cert configurations are working correctly.
Web application
Running on 172.19.0.3:8888 internal to the host
docker-compose for Nginx and Certbot
My certs are coming back clean.
version: '3'
services:
nginx:
image: nginx:1.15-alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./Volumes/nginx:/etc/nginx/conf.d
- ./Volumes/certbot/conf:/etc/letsencrypt
- ./Volumes/certbot/www:/var/www/certbot
certbot:
image: certbot/certbot
volumes:
- ./Volumes/certbot/conf:/etc/letsencrypt
- ./Volumes/certbot/www:/var/www/certbot
Nginx app.conf
server {
listen 80;
server_name forums.example.com;
location /.well-known/acme-challenge/ {
root /var/www/certbot;
}
location / {
return 301 https://$host$request_uri;
}
}
server {
listen 443 ssl;
server_name forums.example.com;
ssl_certificate /etc/letsencrypt/live/forums.example.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/forums.example.com/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
location / {
proxy_pass http://172.19.0.3:8888/;
}
}
Web Application
flarum:
image: mondedie/docker-flarum:0.1.0-beta.8.1-stable
container_name: flarum
env_file:
- ./flarum.env
volumes:
- ./Volumes/assets:/flarum/app/public/assets
- ./Volumes/extensions:/flarum/app/extensions
- ./Volumes/nginx:/etc/nginx/conf.d
depends_on:
- mariadb
mariadb:
image: mariadb:10.2
container_name: mariadb
environment:
- MYSQL_ROOT_PASSWORD=password
- MYSQL_DATABASE=flarum
- MYSQL_USER=flarum
- MYSQL_PASSWORD=password
volumes:
- ./Volumes/mysql/db:/var/lib/mysql
Docker Compose creates a seprate network for docker-compose.yaml file.
So you can add your web application as service (eg: webapp) in current compose file. And in nginx.conf directly point to your service. Rather than using IP you can use the service name as DNS which will resolve by Docker for the same network.
location / {
proxy_pass http://webapp:8888/;
}

Nginx Proxy Upstream Server Docker Compose - 502 Bad Gateway Connection Refused

I am trying to containerize all things related to my web app using Docker Compose, including Nginx & SSL Certificates. To do this I am using the Nginx Proxy image from JWilder and the LetsEncrypt Companion, but am having trouble getting the nginx proxy to work, result ends up being:
Nginx 502 Bad Gateway
[error] 31160#0: *35 connect() failed (111: Connection refused) while connecting to upstream, client: xx.xx.xx.xx, server: domain.com, request: "GET /dev/ HTTP/1.1", upstream: "webapp://127.0.0.1:8080", host: "domain.com"
This only happened when trying to set up the Nginx proxy and SSL certificates so I know it's a configuration issue for either or both of these containers. If anyone can spot where I am going wrong, I would be extremely grateful!
Here are the containers in question:
version: '3.1'
networks:
mywebapp:
services:
nginx-proxy:
container_name: nginx-proxy
build: ./env/nginx-proxy
networks:
- mywebapp
ports:
- 80:80
- 443:443
env_file:
- ./env/nginx-proxy/.env
depends_on:
- webapp
tty: true
volumes:
- ./src:/home/www/mywebapp/src
- ./storage:/home/www/storage/mywebapp
- ./data/nginx-proxy/logs:/var/log/nginx
- ./env/nginx-proxy/webserver/nginx.conf:/etc/nginx/nginx.conf
- ./env/nginx-proxy/webserver/conf.d:/etc/nginx/conf.d
- ./env/nginx-proxy/webserver/vhost.d:/etc/nginx/vhost.d
- ./env/nginx-proxy/webserver/defaults:/etc/nginx/defaults
- ./env/nginx-proxy/webserver/global:/etc/nginx/global
- ./env/nginx-proxy/ssl/certs:/etc/nginx/certs
- ./env/nginx-proxy/share:/usr/share/nginx/html
- /var/run/docker.sock:/tmp/docker.sock:ro
labels:
com.github.jrcs.letsencrypt_nginx_proxy_companion.nginx_proxy: "true"
letsencrypt:
restart: always
container_name: letsencrypt
image: jrcs/letsencrypt-nginx-proxy-companion
env_file:
- ./env/letsencrypt/.env
volumes:
- ./data/letsencrypt/logs:/var/log
- ./env/nginx-proxy/webserver/nginx.conf:/etc/nginx/nginx.conf
- ./env/nginx-proxy/webserver/conf.d:/etc/nginx/conf.d
- ./env/nginx-proxy/webserver/vhost.d:/etc/nginx/vhost.d
- ./env/nginx-proxy/webserver/defaults:/etc/nginx/defaults
- ./env/nginx-proxy/webserver/global:/etc/nginx/global
- ./env/nginx-proxy/ssl/certs:/etc/nginx/certs
- ./env/nginx-proxy/share:/usr/share/nginx/html
- /var/run/docker.sock:/var/run/docker.sock:ro
depends_on:
- nginx-proxy
webapp:
container_name: webapp
build: ./env/webapp
hostname: webapp
networks:
- mywebapp
ports:
- 9000:9000
- 8080:8080
volumes:
- ./env/composer:/home/www/.composer
- ./env/global/bashrc:/home/www/.bashrc
- ./data/bash/.bash_history:/home/www/.bash_history
- ~/.ssh:/home/www/.ssh:ro
- ~/.gitconfig:/home/www/.gitconfig:ro
- ./storage:/home/www/storage/mywebapp
- ./src:/home/www/mywebapp/src
Key points being:
Webapp is the source of my web application which is running PHP, MySQL and Nginx webserver. The webapp webserver exposes and listens on port 8080 to serve the PHP files.
Nginx proxy exposes standard ports 443 and 80 and proxy passes to webapp on port 8080
LetsEncrypt Companion generates the certs and renews.
Nginx Proxy server configuration:
upstream webapp {
server 127.0.0.1:8080;
}
server {
listen 80;
listen [::]:80;
server_name webapp.localhost;
location / {
proxy_pass http://webapp;
}
}
server {
# SSL configuration
listen 443 ssl http2;
listen [::]:443 ssl http2;
ssl_certificate certs/default.crt;
ssl_certificate_key certs/default.key;
include /etc/nginx/global/ssl-params.conf;
server_name webapp.localhost;
location / {
proxy_pass http://webapp;
}
}
Webapp webserver configuration:
server {
listen 8080;
listen [::]:8080;
server_name webapp.localhost;
root /home/www/webapp/src;
index index.php;
include /etc/nginx/defaults/php.conf;
location / {
try_files $uri $uri/ /index.php$is_args$args;
}
}
When visiting http://webapp.localhost:8080 I can see the webapp webserver serves the page no trouble, so I suspect it's something wrong with my Nginx Proxy server configuration.
Thanks for reading.
Since nginx and webapp are on two separate containers, nginx can't reach webapp on localhost(127.0.0.1) as you've configured for upstream:
upstream webapp {
server 127.0.0.1:8080;
}
Change it to webapp:8080.

Run Docker image on a different host

Is it possible to run docker image on other host than localhost?
Ports 80 and 443 are always occupied by other applications, so I was wondering if it's possible to run it on, e.g. 192.168.0.100, and then set up the /etc/hosts file to assign a name for that IP address.
I was trying to set up extra_hosts option, but I'm not sure if it's meant for that. Either way, I did not succeed with setting it up as there was some issue with Value should be a mapping, not an array.
I guess it's worth mentioning that I'm on macOS using docker-compose to run everything. Thank you in advance.
docker-compose.yaml
version: "2"
services:
php:
build: ./php
volumes:
- ../develog.org:/usr/share/nginx/html
depends_on:
- memcached
nginx:
build: ./nginx
ports:
- 4001:80
- 4002:443
volumes_from:
- php:ro
depends_on:
- php
memcached:
image: memcached:alpine
networks:
default:
driver: bridge
ipam:
driver: default
config:
- subnet: 172.1.0.0/16
nginx configuration
map $scheme $ssl_request {
https "https";
}
server {
# support http and ipv6
listen 80 default_server;
listen [::]:80 default_server ipv6only=on;
# support https and ipv6
listen 443 default_server ssl;
listen [::]:443 ipv6only=on default_server ssl;
# path to web directory
root /web/;
index index.html index.htm;
# domain or subdomain
server_name localhost;
include self-signed.conf;
include ssl-params.conf;
}
Why not use a different port when running your image? (unless you have to use port 80/443)
So, for example, if you are running a webserver image like Apache httpd, which normally runs on port 80 use:
docker run -p 127.0.0.1:8080:80/tcp httpd
Then you can just use port 8080 on localhost.
For docker-compose add a ports section to the docker-compose.yaml file. E.g.
image: <image>
ports:
- "8080:80"
The actual ports/image would depend on which image you are using.
See https://docs.docker.com/compose/compose-file/#ports for details.

Resources