Forcing https in elasticbeanstalk with certificate from ACM - ruby-on-rails

I have provisioned a scalable EB(Elasticbeanstalk) rails(puma) instance. I have applied for https through ACM(Amazon Certificate Manager) and applied it to my load balancer. HTTPS is enabled for my website now. But how do I force redirect to https? I have tried a number of solutions online where it was suggested to make a nginx configuration setting manually through .ebextensions and I am not sure where to get the certificate from ACM for this?(I am assuming that is not possible with ACM right now?). How do I force HTTPS?

The current AWS EB Rails and Node.js setups both use nginx (if your web server is apache see this answer), so the following should work (adapted from this question):
Create the file .ebextensions/01-force-https.config (the .config is important, not .conf) with the following content.
If your environment is a single instance:
files:
"/etc/nginx/conf.d/01-force-https.conf":
owner: root
group: root
mode: "000644"
content: |
server {
listen 8080;
return 301 https://$host$request_uri;
}
If your environment is load balanced, you unfortunately cannot simply add to the existing config but need to modify it with sed:
files:
"/tmp/45_nginx_https_rw.sh":
owner: root
group: root
mode: "000644"
content: |
#! /bin/bash
CONFIGURED=`grep -c "return 301 https" /opt/elasticbeanstalk/support/conf/webapp_healthd.conf`
if [ $CONFIGURED = 0 ]
then
sed -i '/listen 80;/a \ if ($http_x_forwarded_proto = "http") { return 301 https://$host$request_uri; }\n' /opt/elasticbeanstalk/support/conf/webapp_healthd.conf
logger -t nginx_rw "https rewrite rules added"
exit 0
else
logger -t nginx_rw "https rewrite rules already set"
exit 0
fi
container_commands:
00_appdeploy_rewrite_hook:
command: cp -v /tmp/45_nginx_https_rw.sh /opt/elasticbeanstalk/hooks/appdeploy/enact
01_configdeploy_rewrite_hook:
command: cp -v /tmp/45_nginx_https_rw.sh /opt/elasticbeanstalk/hooks/configdeploy/enact
02_rewrite_hook_perms:
command: chmod 755 /opt/elasticbeanstalk/hooks/appdeploy/enact/45_nginx_https_rw.sh /opt/elasticbeanstalk/hooks/configdeploy/enact/45_nginx_https_rw.sh
03_rewrite_hook_ownership:
command: chown root:users /opt/elasticbeanstalk/hooks/appdeploy/enact/45_nginx_https_rw.sh /opt/elasticbeanstalk/hooks/configdeploy/enact/45_nginx_https_rw.sh
Then add it to your git repo or app bundle and eb deploy. This creates /etc/nginx/conf.d/01-force-https.conf which is automatically included from /etc/nginx/nginx.conf. Note that eb deploy won't delete the file on the server if you later remove the corresponding file from .ebextensions. Also, I found the following helpful in debugging through eb ssh:
sudo service nginx configtest
sudo service nginx restart

AWS has a help article for HTTP to HTTPS redirection here:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/configuring-https-httpredirect.html
It covers 2 main approaches, and has links to the relevant scripts you can use to do it all for you (which they maintain as they update the Elastic Beanstalk platform).

Related

Certbot failing acme-challenge (connection refused)

I'm trying to set up a Django project with docker + nginx following the tutorial Nginx and Let's Encrypt with Docker in Less Than 5 Minutes.
The issue is when I run the script init-letsencrypt.sh I end up with failed challenges.
Here is the content of my script:
#!/bin/bash
if ! [ -x "$(command -v docker-compose)" ]; then
echo 'Error: docker-compose is not installed.' >&2
exit 1
fi
domains=(xxxx.yyyy.net www.xxxx.yyyy.net)
rsa_key_size=4096
data_path="./data/certbot"
email="myemail#example.com" # Adding a valid address is strongly recommended
staging=1 # Set to 1 if you're testing your setup to avoid hitting request limits
if [ -d "$data_path" ]; then
read -p "Existing data found for $domains. Continue and replace existing certificate? (y/N) " decision
if [ "$decision" != "Y" ] && [ "$decision" != "y" ]; then
exit
fi
fi
if [ ! -e "$data_path/conf/options-ssl-nginx.conf" ] || [ ! -e "$data_path/conf/ssl-dhparams.pem" ]; then
echo "### Downloading recommended TLS parameters ..."
mkdir -p "$data_path/conf/"
curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot-nginx/certbot_nginx/_internal/tls_configs/options-ssl-nginx.conf > "$data_path/conf/options-ssl-nginx.conf"
curl -s https://raw.githubusercontent.com/certbot/certbot/master/certbot/certbot/ssl-dhparams.pem > "$data_path/conf/ssl-dhparams.pem"
echo
fi
echo "### Creating dummy certificate for $domains ..."
path="/etc/letsencrypt/live/$domains"
mkdir -p "$data_path/conf/live/$domains"
docker-compose -f docker-compose-deploy.yml run --rm --entrypoint "\
openssl req -x509 -nodes -newkey rsa:$rsa_key_size -days 1\
-keyout '$path/privkey.pem' \
-out '$path/fullchain.pem' \
-subj '/CN=localhost'" certbot
echo
echo "### Starting nginx ..."
docker-compose -f docker-compose-deploy.yml up --force-recreate -d proxy
echo
echo "### Deleting dummy certificate for $domains ..."
docker-compose -f docker-compose-deploy.yml run --rm --entrypoint "\
rm -Rf /etc/letsencrypt/live/$domains && \
rm -Rf /etc/letsencrypt/archive/$domains && \
rm -Rf /etc/letsencrypt/renewal/$domains.conf" certbot
echo
echo "### Requesting Let's Encrypt certificate for $domains ..."
#Join $domains to -d args
domain_args=""
for domain in "${domains[#]}"; do
domain_args="$domain_args -d $domain"
done
# Select appropriate email arg
case "$email" in
"") email_arg="--register-unsafely-without-email" ;;
*) email_arg="--email $email" ;;
esac
# Enable staging mode if needed
if [ $staging != "0" ]; then staging_arg="--staging"; fi
docker-compose -f docker-compose-deploy.yml run --rm --entrypoint "\
certbot -v certonly --webroot -w /var/www/certbot \
$staging_arg \
$email_arg \
$domain_args \
--rsa-key-size $rsa_key_size \
--agree-tos \
--force-renewal" certbot
echo
echo "### Reloading nginx ..."
docker-compose -f docker-compose-deploy.yml exec proxy nginx -s reload
And my nginx configuration file:
server {
listen 80;
server_name xxxx.yyyy.net;
location ^~ /.well-known/acme-challenge/ {
root /var/www/certbot;
}
location / {
return 301 https://$server_name$request_uri;
}
}
server {
listen 443 ssl;
server_name xxxx.yyyy.net;
ssl_certificate /etc/letsencrypt/live/xxxx.yyyy.net/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/xxxx.yyyy.net/privkey.pem;
include /etc/letsencrypt/options-ssl-nginx.conf;
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem;
location /static {
alias /vol/static;
}
location / {
uwsgi_pass web:8000;
include /etc/nginx/uwsgi_params;
}
}
The output of the part that fails:
Requesting a certificate for xxxx.yyyy.net and www.xxxx.yyyy.net
Performing the following challenges:
http-01 challenge for xxxx.yyyy.net
http-01 challenge for www.xxxx.yyyy.net
Using the webroot path /var/www/certbot for all unmatched domains.
Waiting for verification...
Challenge failed for domain xxxx.yyyy.net
Challenge failed for domain www.xxxx.yyyy.net
http-01 challenge for xxxx.yyyy.net
http-01 challenge for www.xxxx.yyyy.net
Certbot failed to authenticate some domains (authenticator: webroot). The Certificate Authority reported these problems:
Domain: xxxx.yyyy.net
Type: connection
Detail: Fetching http://xxxx.yyyy.net/.well-known/acme-challenge/XJw9w39lRSSbPf-4tb45RLtTnSbjlUEi1f0Cqwsmt-8: Connection refused
Domain: www.xxxx.yyyy.net
Type: connection
Detail: Fetching http://www.xxxx.yyyy.net/.well-known/acme-challenge/b47s4WJARyOTS63oFkaji2nP7oOhiLx5hHp4kO9dCGI: Connection refused
Hint: The Certificate Authority failed to download the temporary challenge files created by Certbot. Ensure that the listed domains serve their content from the provided --webroot-path/-w and that files created there can be downloaded from the internet.
Cleaning up challenges
Some challenges have failed.
Ask for help or search for solutions at https://community.letsencrypt.org. See the logfile /var/log/letsencrypt/letsencrypt.log or re-run Certbot with -v for more details.
ERROR: 1
One of the comments said:
But there's no further explanation as to how to solve it.
Check the certbot commit
Problem is nginx configuration file. The container fails to start up correctly because of missing certification files. I commented out the ssl server portion, rebuilt the image and executed the script again. Everything worked out just fine. After certificates were generated I just uncommented the ssl configuration, rebuilt the image and composed up the services.
Had the same issue;
The solution was ensuring I defined the volume blocks in both the nginx and certbot services correctly.
//other services
nginx:
container_name: nginx
image: nginx:1.13
ports:
- "80:80"
- "443:443"
volumes:
- ./config/nginx/conf.d:/etc/nginx/conf.d
- ./data/certbot/conf:/etc/letsencrypt
- ./data/certbot/www:/var/www/certbot
certbot:
container_name: certbot
image: certbot/certbot
volumes:
- ./data/certbot/conf:/etc/letsencrypt
- ./data/certbot/www:/var/www/certbot
Also if you are using EC2 as your cloud server don't forget to add inbound rules for ports 80 and 443.
A More Beginner-friendly Version!
I can confirm that the first answer that was posted (remove all lines regarding SSL certificate registration/HTTPS redirection when first running the init-letsencrypt.sh) works perfectly!
The lack of documentation is really annoying on this one, and i had to find the answer deep in the community section. Even for someone whose first language isn't English this answer would be really difficult to find. I wish they documented more on this matter. :(
So here are some of the steps that you have to follow to resolve this issue...
Basically gotta remove all the HTTPS SSL-related stuff from both the docker-compose.yml and the nginx.conf / nginx/app.conf file.
Then run the init-letsencrypt.sh script.
Then add the HTTPS SSL-related stuff back to both the docker-compose.yml and the nginx.conf / nginx/app.conf file. (If you're on Git, just revert your commits)
Then run docker-compose up -d --build. Then run the init-letsencrypt.sh script again.
Hope this helps, and wish y'all the best of luck!!
P/S: The back-end stack I used was Flask + Celery (Allows Flask to Run Heavy Tasks Asyncronously) + Redis (A Bridge/Middleman Between Flask and Celery) + NGINX + Certbot all running inside individual docker containers, chained using docker-compose. I deployed it on a DigitalOcean Droplet VPS. (VPS is essentially a computer OS that runs on the internet, 24/7)
For newbies, Docker: Think of Python's virtualenv or Node.js's localized node_modules but for OS-level/C-based dependencies. Like those that can be only installed through package managers such as Linux's apt-get install, macOS's brew install, or Windows's choco install.
Docker Compose: e.g. The client and the server may have different OS-level dependencies and you want to separate them so they don't conflict with each other. You can only allow certain communications between by "chaining" them through docker-compose.
What's NGINX? It's a reverse-proxy solution; TLDR: you can connect the domain/URL you purchased and direct it to your web app. Let's Encrypt allows the server to have that green chain lock thing next to your address for secure communication.
Also important thing to note: Do NOT install NGINX or Redis OUTSIDE of the Docker container on the Linux terminal! That will cause conflicts (ports 443 and 80 already being occupied). 443 is for HTTPS, 80 is for HTTP.
These are the tutorial I used for setting up my tech stack:
https://testdriven.io/blog/dockerizing-flask-with-postgres-gunicorn-and-nginx/
https://pentacent.medium.com/nginx-and-lets-encrypt-with-docker-in-less-than-5-minutes-b4b8a60d3a71
I can also share my docker-compose.yml file below for your reference:
version: '3.8'
services:
web:
build: .
image: web
container_name: web
command: gunicorn --worker-class=gevent --worker-connections=1000 --workers=5 api:app --bind 0.0.0.0:5000
volumes:
- .:/usr/src/app
environment:
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
depends_on:
- redis
expose:
- 5000
worker:
build: .
command: celery --app tasks.celery worker --loglevel=info
volumes:
- .:/usr/src/app
environment:
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
depends_on:
- web
- redis
nginx:
image: nginx:1.15-alpine
ports:
- "80:80"
- "443:443"
volumes:
- ./server/nginx:/etc/nginx/conf.d
- ./server/certbot/conf:/etc/letsencrypt
- ./server/certbot/www:/var/www/certbot
command: "/bin/sh -c 'while :; do sleep 6h & wait $${!}; nginx -s reload; done & nginx -g \"daemon off;\"'"
depends_on:
- web
certbot:
image: certbot/certbot
volumes:
- ./server/certbot/conf:/etc/letsencrypt
- ./server/certbot/www:/var/www/certbot
entrypoint: "/bin/sh -c 'trap exit TERM; while :; do certbot renew; sleep 12h & wait $${!}; done;'"
redis:
image: redis:6-alpine
restart: always
ports:
- 6379:6379
# HOW TO SET REDIS PASSWORD VIA ENVIRONMENT VARIABLE
# https://stackoverflow.com/questions/68461172/docker-compose-redis-password-via-environment-variable
dashboard:
build: .
command: celery --app tasks.celery flower --port=5555 --broker=redis://redis:6379/0
ports:
- 5556:5555
environment:
- CELERY_BROKER_URL=redis://redis:6379/0
- CELERY_RESULT_BACKEND=redis://redis:6379/0
depends_on:
- web
- redis
- worker
Also sharing my Dockerfile JUST IN CASE,
# FOR FRONT-END DEPLOYMENT... (REACT)
FROM node:16-alpine as build-step
WORKDIR /app
ENV PATH /app/web/node_modules/.bin:$PATH
COPY web ./web
WORKDIR /app/web
RUN yarn install
RUN yarn build
# FOR BACK-END DEPLOYMENT... (FLASK)
FROM python:3.10.4-slim
WORKDIR /
# Don't forget "--from"! It acts as a bridge that connects two seperate stages
COPY --from=build-step app ./app
WORKDIR /app
RUN apt-get update && apt-get install -y python3-pip python3-dev mesa-utils libgl1-mesa-glx libglib2.0-0 build-essential libssl-dev libffi-dev redis-server
COPY server ./server
WORKDIR /app/server
RUN pip3 install -r ./requirements.txt
# Pretty much pass everything in the root folder except for the client folder, as we do NOT want to overwrite the pre-generated client folder that is already in the ./app folder
# THIS IS CALLED MULTI-STAGE BUILDING IN DOCKER
EXPOSE 5000
All the notes I made while resolving this problem:
'''
TIPS & TRICKS
-------------
UPDATED ON: 2023-02-11
LAST EDITED BY:
WONMO "JOHN" SEONG,
LEAD DEV. AND THE CEO OF HAVIT
----------------------------------------------
HOW TO INSTALL DOCKER-COMPOSE ON DIGITALOCEAN VPS:
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-ubuntu-22-04
DOCKERIZE FLASK + CELERY + REDIS APPLICATION WITH DOCKER-COMPOSE:
https://nickjanetakis.com/blog/dockerize-a-flask-celery-and-redis-application-with-docker-compose
https://testdriven.io/blog/flask-and-celery/ <-- PRIMARILY USED THIS TUTORIAL
CELERY VS. GUNICORN WORKERS:
https://stackoverflow.com/questions/24317917/difference-between-celery-and-gunicorn-workers
1. Gunicorn solves concurrency of serving HTTP requests - this is "online" code where each request triggers a Django view, which returns a response. Any code that runs in a view will increase the time it takes to get a response to the user, making the website seem slow. So long running tasks should not go in Django views for that reason.
2. Celery is for running code "offline", where you don't need to return an HTTP response to a user. A Celery task might be triggered by some code inside a Django view, but it could also be triggered by another Celery task, or run on a schedule. Celery uses the model of a worker pulling tasks off of a queue, there are a few Django compatible task frameworks that do this. I give a write up of this architecture here.
CELERY, GUNICORN, AND SUPERVISOR:
https://medium.com/sightwave-software/setting-up-nginx-gunicorn-celery-redis-supervisor-and-postgres-with-django-to-run-your-python-73c8a1c8c1ba
DEPLOY GITHUB REPO ON DIGITALOCEAN VPS USING SSH KEYS:
https://medium.com/swlh/how-to-deploy-your-application-to-digital-ocean-using-github-actions-and-save-up-on-ci-cd-costs-74b7315facc2
COMANDS TO RUN ON VPS TO CLONE GITHUB REPO (WORKS ON BOTH PRIVATE AND PUBLIC REPOS):
1. Login as root
2. Set up your credentials (GitHub SSH-related) and run the following commands:
- apt-get update
- apt-get install git
- mkdir ~/github && cd ~/github
- git clone git#github.com:wonmor/HAVIT-Central.git
3. To get the latest changes, run git fetch origin
HOW TO RUN DOCKER-COMPOSE ON VPS:
https://www.digitalocean.com/community/tutorials/how-to-install-and-use-docker-compose-on-ubuntu-22-04
1. Login as root
2. Run the following commands:
- cd ~/github/HAVIT-Central
- docker compose up --build -d // builds and runs the containers in detached mode
OR docker compose up --build -d --remove-orphans // builds and runs the containers in detached mode and removes orphan containers
- docker compose ps // lists all running containers in Docker engine.
3. To stop the containers, run:
- docker-compose down
HOW TO SET UP NGINX ON UBUNTU VPS TO PROXY PASS TO GUNICORN ON DIGITALOCEAN:
https://www.datanovia.com/en/lessons/digitalocean-initial-ubuntu-server-setup/
https://www.digitalocean.com/community/tutorials/how-to-secure-nginx-with-let-s-encrypt-on-ubuntu-22-04
https://www.datanovia.com/en/lessons/digitalocean-how-to-install-nginx-and-ssl/
CAPROVER CLEAN/REMOVE ALL PREVIOUS DEPLOYMENTS:
docker container prune --force
docker image prune --all
FORCE MERGE USING GIT:
git reset --hard origin/main
NGINX - REDIRECT TO DOCKER CONTAINER:
https://gilyes.com/docker-nginx-letsencrypt/
https://github.com/nginx-proxy/acme-companion
https://github.com/nginx-proxy/acme-companion/wiki/Docker-Compose
https://github.com/evertramos/nginx-proxy-automation
https://github.com/buchdag/letsencrypt-nginx-proxy-companion-compose
https://testdriven.io/blog/dockerizing-flask-with-postgres-gunicorn-and-nginx/
https://pentacent.medium.com/nginx-and-lets-encrypt-with-docker-in-less-than-5-minutes-b4b8a60d3a71 <--- THIS IS THE BEST TUTORIAL
Simply run docker-compose up and enjoy your HTTPS-secured website or app.
Then run chmod +x init-letsencrypt.sh and sudo ./init-letsencrypt.sh.
VVIP: HOW TO RUN THIS APP ON VPS:
1. Login as root, run sudo chmod +x init_letsencrypt.sh
2. Now for the bit… that tends to go wrong. Navigate into your remote project folder, and run the initialization script (Run ./<Script-Name>.sh on Terminal). First, docker will build the images, and then run through the script step-by-step as described above. Now, this worked first time for me while putting together the tutorial, but in the past it has taken me hours to get everything set up correctly. The main problem was usually the locations of files: the script would save it to some directory, which was mapped to a volume that nginx was incorrectly mapped to, and so on. If you end up needing to debug, you can run the commands in the script yourself, substituting variables as you go. Pay close attention to the logs — nginx is often quite good at telling you what it’s missing.
3. If all goes to plan, you’ll see a nice little printout from Lets Encrypt and Certbot saying “Congratulations” and your script will exit successfully.
HOW TO OPEN/ALLOW PORTS ON DIGITALOCEAN:
https://www.digitalocean.com/community/tutorials/opening-a-port-on-linux
sudo ufw allow <PORT_NUMBER>
WHAT ARE DNS RECORDS?
https://docs.digitalocean.com/products/networking/dns/how-to/manage-records/
PS: Highers the TTL, the longer it takes for the DNS record to update.
But it will be cached for longer, which means that there will be less load on the DNS server.
TIP: MAKE SURE YOU SET UP THE CUSTOM NAMESPACES FOR DIGITALOCEAN ON GOOGLE DOMAINS:
https://docs.digitalocean.com/tutorials/dns-registrars/
DOCKER SWARM VS. DOCKER COMPOSE:
The difference between Docker Swarm and Docker Compose is that Compose is used for configuring multiple containers in the same host. Docker Swarm is different in that it is a container orchestration tool. This means that Docker Swarm lets you connect containers to multiple hosts similar to Kubernetes.
Cannot load certificate /etc/letsencrypt/live/havit.space/fullchain.pem: BIO_new_file() failed (SSL: error:02001002:system library:fopen:No such file or directory FIX:
https://community.letsencrypt.org/t/lets-encrypt-with-nginx-i-got-error-ssl-error-02001002-system-library-fopen-no-such-file-or-directory-fopen-etc-letsencrypt-live-xxx-com-fullchain-pem-r/20990/5
RUNNING MULTIPLE DOCKER COMPOSE FILES:
https://stackoverflow.com/questions/43957259/run-multiple-docker-compose
nginx: [emerg] open() "/etc/letsencrypt/options-ssl-nginx.conf" failed (2: No such file or directory) in /etc/nginx/conf.d/app.conf:20 FIX:
https://stackoverflow.com/questions/64940480/nginx-letsencrypt-error-etc-letsencrypt-options-ssl-nginx-conf-no-such-file-o
VVVIP: RESOLVE NGINX + DOCKER + LETSENCRYPT ISSUES!
https://stackoverflow.com/questions/68449947/certbot-failing-acme-challenge-connection-refused
Basically gotta remove all the HTTPS SSL-related stuff from both the docker-compose.yml and the nginx.conf file.
Then run the init-letsencrypt.sh script. Then add the HTTPS SSL-related stuff back to both the docker-compose.yml and the nginx.conf file.
Then run docker-compose up -d --build. Then run the init-letsencrypt.sh script again.
'''

Docker mount volume on SELinux enabled server

I'm trying to run Nginx as the docker container on SELinux enabled server with mounted configuration as
-v /host/path/nginx.conf:/etc/nginx/nginx.conf:Z
That works fine till I will deploy updated configuration using ansible playbook (user/group root and mode 0644) as:
tasks:
- name: remove NGINX conf
file:
path: "/host/path/nginx.conf"
state: absent
become: true
- name: copy NGINX conf
become: true
copy:
src: "../nginx-conf/nginx.conf"
dest: "/host/path/nginx.conf"
mode: 0640
owner: root
group: root
and then reload nginx in the container as
docker container exec nginx-service nginx -s reload
that will error out with Permission Denied (13) to deployed configuration.
When I will check permissions to /host/path/nginx.conf those are correct (user/group:root and mode 0644) but for some reason nginx is not able to read that file.
When I will get inside the container and check /etc/nginx/nginx.conf permissions there are not carried over - they show up as question marks.
Any clue how to get it to work without reloading whole container just nginx service?

Hugo server in Docker container not reachable in Windows 10

A few days ago I started a little side project: Dockerizing my Hugo build on my Windows 10 machine. The Hugo container itself, which runs as a Linux container, was the easy part and seems to work (at least by looking at the console output
$ docker run --rm -it -p 1313:1313/tcp hugo:latest
Building sites …
Replace Autoprefixer browsers option to Browserslist config.
Use browserslist key in package.json or .browserslistrc file.
Using browsers option cause some error. Browserslist config
can be used for Babel, Autoprefixer, postcss-normalize and other tools.
If you really need to use option, rename it to overrideBrowserslist.
Learn more at:
https://github.com/browserslist/browserslist#readme
https://twitter.com/browserslist
WARN 2019/11/23 14:05:35 found no layout file for "HTML" for "section": You should create a template file which matches Hugo Layouts Lookup Rules for this combination.
| DE | EN
+------------------+----+----+
Pages | 9 | 7
Paginator pages | 0 | 0
Non-page files | 0 | 0
Static files | 25 | 25
Processed images | 0 | 0
Aliases | 1 | 0
Sitemaps | 2 | 1
Cleaned | 0 | 0
Total in 680 ms
Watching for changes in /app/{assets,content,i18n,layouts,static}
Watching for config changes in /app/config.yaml
Environment: "development"
Serving pages from memory
Running in Fast Render Mode. For full rebuilds on change: hugo server --disableFastRender
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Press Ctrl+C to stop
My Dockerfile the I run looks like this
FROM node:13-alpine
ENV VERSION 0.59.1
EXPOSE 1313
RUN apk add --no-cache git openssl py-pygments libc6-compat g++ curl
RUN curl -L https://github.com/gohugoio/hugo/releases/download/v${VERSION}/hugo_extended_${VERSION}_Linux-64bit.tar.gz | tar -xz \
&& cp hugo /usr/bin/hugo \
&& apk del curl \
&& hugo version
WORKDIR /app
COPY assets assets
COPY content content
COPY i18n i18n
COPY layouts layouts
COPY static static
COPY package.json package.json
COPY postcss.config.js postcss.config.js
COPY config.yaml config.yaml
RUN yarn
CMD [ "hugo", "server", "--buildDrafts","--watch" ]
The hard part for me now is to connect to the running Hugo server on my host's systems (Windows 10 Pro) browser.
I basically tried everything: localhost:1313 & http://172.17.0.2:1313/ (the container IP I get by running docker inspect <container ID>), with firewall enabled and disabled, but nothing seems to work.
To verify that it should work I ran hugo server --buildDrafts --watch directly on my host system and can access the server just fine. I also invested several hours in reading up on the issue, but none of the solutions seem to work in my case.
How can I solve this issue?
Here's your problem:
Web Server is available at http://localhost:1313/ (bind address 127.0.0.1)
Hugo is binding to the loopback address (127.0.0.1) inside the container. It does this by default because hugo serve is meant strictly as a development tool, not for actually serving pages in production. In order to avoid any security issues, it defaults to binding to the loopback interface so that you can only connect to it from the local machine.
Unfortunately, in the context of a container, localhost means "this container". So with Hugo bound to 127.0.0.1 inside a container you'll never be able to connect to it.
The solution is to provide a different bind address using the --bind option. You probably want to modify your Dockerfile so that it looks like:
CMD [ "hugo", "server", "--buildDrafts", "--watch", "--bind", "0.0.0.0" ]
This will cause hugo to bind to "all interfaces" inside the container, which should result in it working as you expect.

Customizing Nginx Configuration in AWS Elastic Beanstalk

I'm running a rails application on Ruby 2.0/Puma instances and am trying to customize the nginx configuration. I need to increase the permitted request size to allow file uploads. I've found some other posts that have lead me to add this to my .ebextensions:
files:
"/etc/nginx/conf.d/proxy.conf" :
mode: "000755"
owner: root
group: root
content: |
client_max_body_size 70M;
That does create the file as expected, but it doesn't seem to work until I manually restart nginx. Because of that, I've tried to figure out a way to restart nginx with .ebextensions commands, but haven't had any success. Does anyone know of a way to restart nginx with .ebextensions or know of a better approach to solving this problem?
I found a way to restart nginx after deployment using an undocumented technique for running post-deployment scripts. I added this to my .ebextensions:
files:
"/opt/elasticbeanstalk/hooks/appdeploy/post/03_restart_nginx.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
service nginx restart
To reload the nginx config, you can use container_commands
From http://www.infoq.com/news/2012/11/elastic-beanstalk-config-files:
The container_commands key allows you to execute commands for your container. They are run after the application and web server have been set up and the application has been extracted, but before the application is deployed. container_commands are processed in lexicographical order by name.
container_commands:
01_reload_nginx:
command: "service nginx reload"
I might be a little late with the response here, but I've discovered another, less intrusive way to configure nginx on Elastic Beanstalk.
You can specify configuration files for nginx directly by creating an .ebextensions/nginx/conf.d directory. Any config files found inside are automatically copied to your /etc/nginx/conf.d/ directory during the EB deployment.
This seems to be a more robust solution.
Documentation available here:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/java-se-nginx.html
EDIT: As pointed out in comments, Elastic Beanstalk has inconsistent implementations between platforms. The documentation here is for Java SE, and it appears this documentation is not relevant for all platforms.
this is my configuration and worked for me. You have to include it inside of the http block.
files:
"/etc/nginx/conf.d/proxy.conf" :
mode: "000755"
owner: root
group: root
content: |
http {
client_max_body_size 20M;
}
The following worked for me (I increased my HTTP payload to 100M - please adjust if you'd like to increase to another size):
files:
"/etc/nginx/conf.d/proxy.conf":
mode: "000755"
owner: root
group: root
content: |
client_max_body_size 100M;
"/opt/elasticbeanstalk/hooks/appdeploy/post/03_restart_nginx.sh":
mode: "000755"
owner: root
group: root
content: |
#!/usr/bin/env bash
service nginx restart
I got it working like this. No need to restart or reload nginx since the commands (and not container_commands) runes BEFORE the application deploy.
commands:
01-get-nginx-conf-file:
command: "aws s3 cp s3://somepath/nginx.conf /home/ec2-user"
02-replace-default-nginx-config:
command: "cp /home/ec2-user/nginx.conf /etc/nginx/nginx.conf"
I had a similar situation with a Docker deployment into Elastic Beanstalk. I was able to solve the change as well as the nginx reload with a single config file here: <app>/.ebextensions/increase_upload_size.config including the following code:
container_commands:
01_reload_nginx:
command: "sudo service nginx reload"
files:
"/etc/nginx/conf.d/proxy.conf" :
mode: "000644"
owner: root
group: root
content: |
client_max_body_size 20M;
The change was implemented when I did an "Upload and Deploy" within EB.
I was able to get it working by adding the configuration files under the .platform directory as noted in https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/platforms-linux-extend.html. I created a file .platform/nginx/conf.d/increase_upload_size.conf with the following code and just did a standard deploy
client_max_body_size 10M;

Apache + passenger - /tmp permission denied

I am trying to run ruby on rails under passenger with apache2 under fedora 19 and I got this error in log:
[Tue Feb 25 09:37:52.367683 2014] [passenger:error] [pid 2779] ***
Passenger could not be initialized because of this error: Unable to
start the Phusion Passenger watchdog because it encountered the
following error during startup: Cannot change the directory
'/tmp/passenger.1.0.2779/generation-1/buffered_uploads' its UID to 48
and GID to 48: Operation not permitted (errno=1)
That directory (/tmp/passenger.1.0.2779) doesn't even exist. I think that problem is with selinux. I tried to solve it about 4 hours. Httpd is running under user apache and group apache, I tried:
cat /var/log/audit/audit.log | grep passenger | audit2allow -M
passenger semodule -i passenger.pp
but still nothing.
In your case, you should switch SELinux into Permissive mode at first, then try to capture the audit log from starting Apache to run your application.1
Once you got the home page of your application, you can build your custom policy with the logs.
Switch SELinux into Permissive mode and clean audit.log
]# setenforce 0
]# rm /var/log/audit/audit.log
]# service auditd restart
Restart Apache
]# service httpd restart
Try to open your application with a web browser
It might give more information about what is happenning when you application is running.
Make a custom policy module to allow these actions
]# mkdir work
]# cd work
]# grep httpd /var/log/audit/audit.log | audit2allow -M passenger
]# ls
passenger.pp passenger.te
Load postgrey policy module using the 'semodule' command into the current SELinux policy:
]# semodule -i passenger.pp
]# setenforce 1
Restart Apache
]# service httpd restart
References:
http://wiki.centos.org/HowTos/SELinux#head-faa96b3fdd922004cdb988c1989e56191c257c01
I ran into a similar error, with a startup error about being unable to create a directory that did not exist. (logs, not tmp, but same sort of thing) I, too, battled with it for an hour and couldn't make sense of it. I created/deleted/chmod the directory many ways without success.
The fix for me was to change the parameters to passenger-start. Initially, my Docker container started passenger with:
exec bundle exec passenger start --auto --disable-security-update-check --min-instances 20 --max-pool-size 20 --max-request-queue-size 500
I removed all parameters, leaving just this:
exec bundle exec passenger start
At this point, passenger could create the log folder and file, and all was well. I could have restored the params at this point, but we decided they were not needed for the development environment so left them out moving ahead.
In hindsight, I have a hunch that I deleted the log directory while a file in it was still open, and the file system persisted that condition in some way. But that's just a hunch. Perhaps simply rebooting my Mac would have fixed it...

Resources