Docker-compose + Nginx + Certbot + Simple Django Rest Framework app - docker

Question

I am trying to deploy a simple Django Rest Framework app to the production server using Docker. My aim is to install Nginx with a proxy and Certbot for a regular Let'sEncrypt SSL at the same time. I manage my dependencies in DockerFiles and docker-compose.
So the folder structure has the following view:
app
DockerFile
nginx
DockerFile
init-letsencrypt.sh
nginx.conf
docker-compose.yml
My idea is to hold all the configs in app/docker-compose.yml and start many different instances from the same source. But I do not have any nginx or certbot config in app/DockerFile - that's only for Django Rest Framework and that works well. But in docker-compose.yml I have the following code:
version: '3'
'services':
app:
container_name: djangoserver
command: gunicorn prototyp.wsgi:application --env DJANGO_SETTINGS_MODULE=prototyp.prod_settings --bind 0.0.0.0:8000 --workers=2 --threads=4 --worker-class=gthread
build:
context: ./api
dockerfile: Dockerfile
restart: always
ports:
- "8000:8000"
depends_on:
- otherserver
otherserver:
container_name: otherserver
build:
context: ./otherserver
dockerfile: Dockerfile
restart: always
nginx:
build: ./nginx
ports:
- 80:80
depends_on:
- app
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
certbot:
image: certbot/certbot
entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
This makes me to build "app", "otherserver", "nginx" and "certbot".
The most important parts are in "nginx" folder.
I used this manual and cloned file "init-letsencrypt.sh" from the source just the way it was described. Then I tried to bash it:
nginx/DockerFile:
FROM nginx:1.19.0-alpine
RUN rm /etc/nginx/conf.d/default.conf
COPY nginx.conf /etc/nginx/conf.d
RUN mkdir -p /usr/src/app
COPY init-letsencrypt.sh /usr/src/app
WORKDIR /usr/src/app
RUN chmod +x init-letsencrypt.sh
ENTRYPOINT ["/usr/src/app/init-letsencrypt.sh"]
In nginx/nginx.conf I have the following code:
upstream django {
server app:8000;
}
server {
listen 80;
server_name app.com www.app.com;
location / {
return 301 https://$host$request_uri;
}
}
server {
listen 443 ssl;
server_name app.com www.app.com;
access_log /var/log/nginx-access.log;
error_log /var/log/nginx-error.log;
ssl_certificate /etc/letsencrypt/live/app.com/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/app.com/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
location ^/static/rest_framework/((img/|css/|js/|fonts).*)$ {
autoindex on;
access_log off;
alias /usr/src/app/static/rest_framework/$1;
}
location / {
proxy_pass http://django;
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
client_body_buffer_size 256k;
proxy_connect_timeout 120;
proxy_send_timeout 120;
proxy_read_timeout 120;
proxy_buffer_size 64k;
proxy_buffers 4 64k;
proxy_busy_buffers_size 64k;
proxy_temp_file_write_size 64k;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
client_max_body_size 100M;
}
}
So, with this configuration when I do "docker-compose build", the build works without any errors and everything is successfully built. But as soon as I do "docker-compose up" I have the problem that certbot and nginx are not connect and the app is working only when I use http://app.com:8000 instead of https://app.com.
In console I do not have any errors.
What do I do wrong? What have I missed? Any help will be appreciated.

Answer 1

I see in your setup you try to run let's encrypt from within the nginx container. But I believe there are two better way that I describe in details here and here.
The idea behind the first method is to have a docker-compose file to initiate the letsencrypt certificate, and another docker-compose file to run the system and renew the certificate.
So without further ado, here is the file structure and content that is working really well for me (you still need to adapt the files to suit your needs)
./setup.sh
./docker-compose-initiate.yaml
./docker-compose.yaml
./etc/nginx/templpates/default.conf.template
./etc/nginx/templpates-initiation/default.conf.template
The setup in 2 phases:
In the first phase "the initiation phase" we will run an nginx container, and a certbot container just to obtain the ssl certificate for the first time and store it on the host ./etc/letsencrypt folder
I the second phase "the operation phase" we run all necessary services for the app including nginx that will use the letsencrypt folder this time to serve https on port 443, a certbot container will also run (on demand) to renew the certificate. We can add a cron job for that. So the setup.sh script is a simple convenience script that runs the commands one after another:
#!/bin/bash
# the script expects two arguments:
# - the domain name for which we are obtaining the ssl certificatee
# - the Email address associated with the ssl certificate
echo DOMAIN=$1 >> .env
echo EMAIL=$2 >> .env
# Phase 1 "Initiation"
docker-compose -f ./docker-compose-first.yaml up -d nginx
docker-compose -f ./docker-compose-first.yaml up certbot
docker-compose -f ./docker-compose-first.yaml down
# Phase 2 "Operation"
crontab ./etc/crontab
docker-compose -f ./docker-compose.yaml up -d
Phase 1: The ssl certificate initiation phase:
./docker-compose-initiate.yaml
version: "3"
services:
nginx:
container_name: nginx
image: nginx:latest
environment:
- DOMAIN
ports:
- 80:80
volumes:
- ./etc/nginx/templates-initiate:/etc/nginx/templates:ro
- ./etc/letsencrypt:/etc/letsencrypt:ro
- ./certbot/data:/var/www/certbot
certbot:
container_name: certbot
image: certbot/certbot:latest
depends_on:
- nginx
command: >-
certonly --reinstall --webroot --webroot-path=/var/www/certbot
--email ${EMAIL} --agree-tos --no-eff-email
-d ${DOMAIN}
volumes:
- ./etc/letsencrypt:/etc/letsencrypt
- ./certbot/data:/var/www/certbot
./etc/nginx/templates-initiate/default.conf.template
server {
listen [::]:80;
listen 80;
server_name $DOMAIN;
location ~/.well-known/acme-challenge {
allow all;
root /var/www/certbot;
}
}
Phase 2: The operation phase
./docker-compose.yaml
services:
app:
{{your_configurations_here}}
{{other_services...}}:
{{other_services_configuraitons}}
nginx:
container_name: nginx
image: nginx:latest
restart: always
environment:
- DOMAIN
depends_on:
- app
ports:
- 80:80
- 443:443
volumes:
- ./etc/nginx/templates:/etc/nginx/templates:ro
- ./etc/letsencrypt:/etc/letsencrypt
- ./certbot/data:/var/www/certbot
- /var/log/nginx:/var/log/nginx
certbot:
container_name: certbot
image: certbot/certbot:latest
depends_on:
- nginx
command: >-
certonly --reinstall --webroot --webroot-path=/var/www/certbot
--email ${EMAIL} --agree-tos --no-eff-email
-d ${DOMAIN}
volumes:
- ./etc/letsencrypt:/etc/letsencrypt
- ./certbot/data:/var/www/certbot
./etc/nginx/templates/default.conf.template
server {
listen [::]:80;
listen 80;
server_name $DOMAIN;
return 301 https://$host$request_uri;
}
server {
listen [::]:443 ssl http2;
listen 443 ssl http2;
server_name $DOMAIN;
ssl_certificate /etc/letsencrypt/live/$DOMAIN/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/$DOMAIN/privkey.pem;
location ~ /.well-known/acme-challenge {
allow all;
root /var/www/html;
}
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_pass http://app:80;
}
}
The second method uses two docker images: http-proxy and http-proxy-acme-companion that were developed specifically for this reason. I suggest looking at the blog post for further details.

Answer 2

As I see, you havenot exposed port 443 for nginx container:
nginx:
build: ./nginx
ports:
- 80:80
- 443:443
depends_on:
Add more 443 port.