Docker with Nginx: host not found in upstream - docker

I'm trying to follow this guide to setting up a reverse proxy for a docker container (serving a static file), using another container with an instance of nginx as a reverse proxy.
I expect to see my page served on /, but I am blocked in the build with the error message:
container_nginx_1 | 2020/05/10 16:54:12 [emerg] 1#1: host not found in upstream "container1:8001" in /etc/nginx/conf.d/sites-enabled/virtual.conf:2
container_nginx_1 | nginx: [emerg] host not found in upstream "container1:8001" in /etc/nginx/conf.d/sites-enabled/virtual.conf:2
nginx_docker_test_container_nginx_1 exited with code 1
I have tried many variations on the following virtual.conf file, and this is the current, based on the example given and various other pages:
upstream cont {
server container1:8001;
}
server {
listen 80;
location / {
proxy_pass http://cont/;
}
}
If you are willing to look at a 3rd party site, I've made a minimal repo here, otherwise the most relevant files are below.
My docker-compose file looks like this:
version: '3'
services:
container1:
hostname: container1
restart: always
image: danjellz/http-server
ports:
- "8001:8001"
volumes:
- ./proj1:/public
command: "http-server . -p 8001"
depends_on:
- container_nginx
networks:
- app-network
container_nginx:
build:
context: .
dockerfile: docker/Dockerfile_nginx
ports:
- 8080:8080
networks:
- app-network
networks:
app-network:
driver: bridge
and the Dockerfile
# docker/Dockerfile_nginx
FROM nginx:latest
# add nginx config files to sites-available & sites-enabled
RUN mkdir /etc/nginx/conf.d/sites-available
RUN mkdir /etc/nginx/conf.d/sites-enabled
ADD projnginx/conf.d/sites-available/virtual.conf /etc/nginx/conf.d/sites-available/virtual.conf
RUN cp /etc/nginx/conf.d/sites-available/virtual.conf /etc/nginx/conf.d/sites-enabled/virtual.conf
# Replace the standard nginx conf
RUN sed -i 's|include /etc/nginx/conf.d/\*.conf;|include /etc/nginx/conf.d/sites-enabled/*.conf;|' /etc/nginx/nginx.conf
WORKDIR /
I'm running this using docker-compose up.
Similar: react - docker host not found in upstream

The problem is if the hostname can not be resolved in upstream blocks, nginx will not start. Here you have defined service container1 to be dependent on container_nginx . But nginx container is never up due to the fact the container1 hostname is not resolved (because container1 is not yet started) Don't you think it should be reverse? Nginx container should be dependent on the app container.
Additionally in your nginx port binding you have mapped 8080:8080 while in nginx conf you have 80 listening.

Related

How to change php-fpm default port?

I'm using php-fpm which runs for default on the port 9000. The problem's that I have other docker container based on php-fpm, so I need to change the default port to another one, in order to not confuse nginx.
This is my Dockerfile:
FROM php:8.0.2-fpm-alpine
RUN sed -i 's/9000/9001/' /usr/local/etc/php-fpm.d/zz-docker.conf
WORKDIR /var/www/html
CMD ["php-fpm"]
EXPOSE 9001
I tried to use the sed command to replace the port 9000 with 9001.
Inside my docker-compose file I have this configuration:
version: '3.9'
services:
php-fpm:
container_name: app
restart: always
build:
context: .
dockerfile: ./docker/php-fpm/Dockerfile
ports:
- "9001:9000"
volumes:
- ./src:/var/www/html
- ./docker/php-fpm/config/www.conf:/usr/local/etc/php-fpm.d/www.conf
- ./src/public:/app/public
- ./src/writable:/app/writable
nginx:
image: nginx:stable-alpine
container_name: nginx
restart: always
volumes:
- ./src:/var/www/html
- ./docker/nginx/nginx.conf:/etc/nginx/nginx.conf
- ./docker/nginx/sites/:/etc/nginx/sites-available
- ./docker/nginx/conf.d/:/etc/nginx/conf.d
depends_on:
- php-fpm
environment:
VIRTUAL_HOST: ${HOST}
LETSENCRYPT_HOST: ${HOST}
LETSENCRYPT_EMAIL: ${EMAIL}
as you can see I have exposed the port 9001 also in the docker-compose file.
The file default.conf available within conf.d folder contains this:
upstream php-upstream {
server php-fpm:9001;
}
the problem's that for some reason, when I load my site I get the error 500. So this means that the stream doesn't send any signal. If I change to port 9000 everything works, but the stream is wrong 'cause it's the content of another application.
How can I correctly change the default port?
I think the problem is not the sed command itself, it's related to the wrong file you mentioned for it.
/usr/local/etc/php-fpm.d/zz-docker.conf
this is the file you are trying to change the port in it but inside your docker-compose file you are mapping something else
./docker/php-fpm/config/www.conf:/usr/local/etc/php-fpm.d/www.conf
Keep in mind that your nginx docker connects via the internal docker net work, your docker host port mapping 9001:9000 is not taken into account.
If your php-fpm is really listening on port 9001, then that is the port you must use in your nginx config.
Also EXPOSE is primarily declarative and does not expose anything by itself (https://docs.docker.com/engine/reference/builder/#expose) the corresponding service must still be configured correctly to use that port.
for this line in Dockerfile is not working(?):
RUN sed -i 's/9000/9001/' /usr/local/etc/php-fpm.d/zz-docker.conf
so I decided to make this change after the container is up with:
docker exec -i your_container bash -c 'sed -i 's/9000/9001/' /usr/local/etc/php-fpm.d/zz-docker.conf'
and then restert the container to make changes happened
hope help you

How can I connect the Nginx container to my React container?

I have tried reading through the other stackoverflow questions here but I am either missing something or none of them are working for me.
Context
I have two docker containers setup on a DigitalOcean server running Ubuntu.
root_frontend_1 running on ports 0.0.0.0:3000->3000/tcp
root_nginxcustom_1 running on ports 0.0.0.0:80->80/tcp
If I connect to http://127.0.0.1, I get the default Nginx index.html homepage. If I http://127.0.0.1:3000 I am getting my react app.
What I am trying to accomplish is to get my react app when I visit http://127.0.0.1. Following the documentation and suggestions here on StackOverflow, I have the following:
docker-compose.yml in root of my DigitalOcean server.
version: "3"
services:
nginxcustom:
image: nginx
hostname: nginxcustom
ports:
- "80:80"
volumes:
- ./nginx.conf:/root/nginxcustom/conf/custom.conf
tty: true
backend:
build: https://github.com/Twitter-Clone/twitter-clone-api.git
ports:
- "8000:8000"
tty: true
frontend:
build: https://github.com/dougmellon/react-api.git
ports:
- "3000:3000"
stdin_open: true
tty: true
nginxcustom/conf/custom.conf :
server {
listen 80;
server_name http://127.0.0.1;
location / {
proxy_pass http://root_frontend_1:3000; # this one here
proxy_redirect off;
}
}
When I run docker-compose up, it builds but when I visit the ip of my server, it's still showing the default nginx html file.
Question
What am I doing wrong here and how can I get it so the main URL points to my react container?
Thank you for your time, and if there is anything I can add for clarity, please don't hesitate to ask.
TL;DR;
The nginx service should proxy_pass to the service name (customnginx), not the container name (root_frontend_1) and the nginx config should be mounted to the correct location inside the container.
Tip: the container name can be set in the docker-compose.yml for services setting the container_name however beware you can not --scale services with a fixed container_name.
Tip: the container name (root_frontend_1) is generated using the compose project name which defaults to using the current directory name if not set.
Tip: the nginx images are packaged with a default /etc/nginx/nginx.conf that will include the default server config from /etc/nginx/conf.d/default.conf. You can docker cp the default configuration files out of a container if you'd like to inspect them or use them as a base for your own configuration:
docker create --name nginx nginx
docker cp nginx:/etc/nginx/conf.d/default.conf default.conf
docker cp nginx:/etc/nginx/nginx.conf nginx.conf
docker container rm nginx
With nginx proxying connections for the frontend service we don't need to bind the hosts port to the container, the services ports definition can be replaced with an expose definition to prevent direct connections to http://159.89.135.61:3000 (depending on the backend you might want prevent direct connections as well):
version: "3"
services:
...
frontend:
build: https://github.com/dougmellon/react-api.git
expose:
- "3000"
stdin_open: true
tty: true
Taking it a step further we can configure an upstream for the
frontend service then configure the proxy_pass for the upstream:
upstream frontend {
server frontend:3000 max_fails=3;
}
server {
listen 80;
server_name http://159.89.135.61;
location / {
proxy_pass http://frontend/;
}
}
... then bind-mount the custom default.conf on top of the default.conf inside the container:
version: "3"
services:
nginxcustom:
image: nginx
hostname: nginxcustom
ports:
- "80:80"
volumes:
- ./default.conf:/etc/nginx/conf.d/default.conf
tty: true
... and finally --scale our frontend service (bounce the services removing the containers to make sure changes to the config take effect):
docker-compose stop nginxcustom \
&& docker-compose rm -f \
&& docker-compose up -d --scale frontend=3
docker will resolve the service name to the IP's of the 3 frontend containers which nginx will proxy the connections for in a (by default) round robin manner.
Tip: you can not --scale a service that has ports mappings, only a single container can bind to the port.
Tip: if you've updated the config and can connect to your load balanced service then you're all set to create a DNS record to resolve a hostname to your public IP address then update your default.conf's server_name.
Tip: for security I maintain specs for building a nginx docker image with Modsecurity and Modsecurity-nginx pre-baked with the OWASP Core Rule Set.
In Docker when multiple services needs to communicate with each other, you can use the service name in the url (set in the docker-composer.yml instead of the ip (which is attributed from the available pool of the network, default by default), it will automatically be resolve to the right container ip due to network management by docker.
For you it would be http://frontend:3000

How do I configure nginx to just deliver whatever it finds at the configured url

I start a nginx reverse proxy in docker-compose.
The first docker compose file looks like this:
version: "3.5"
services:
rproxy:
build:
context: ./nginx
dockerfile: Dockerfile
ports:
- 80:80
- 443:443
volumes:
- '/etc/letsencrypt:/etc/letsencrypt'
networks:
- main
networks:
main:
name: main_network
The dockerfile just makes sure the nginx server has the following configuration:
server {
listen 443 ssl;
server_name website.dev;
ssl_certificate /etc/letsencrypt/live/www.website.dev/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/www.website.dev/privkey.pem;
location / {
resolver 127.0.0.11;
set $frontend http://website;
proxy_pass $frontend;
}
}
First I run this following docker-compose file. Then when I try to access www.website.dev i get a 502 error as expected.
Then I run this other docker-compose file defined below:
version: '3.5'
services:
website:
image: registry.website.dev/frontendcontainer:latest
command: npm run deploy
networks:
main:
aliases:
- website
networks:
main:
external:
name: main_network
This should start the website container on the same network as the nginx container.
"docker ps" shows that the docker container is running.
going to website.dev gives a 502 error. This is unexpected. I expect Nginx to now be able to connect to the now running docker container.
I reset the nginx server by running the following on the first docker-compose file:
docker-compose up -d
Going to website.dev now displays the contents of the website container.
I make changes to the website container upload the new docker container to the private container.
I use the following commands on the second docker-compose file:
docker-compose down
The old website container is no longer in existence.
docker-compose pull
The new website container is pulled.
docker-compose up
The new website container is now online.
Going to website.dev now displays the contents of the old (confirmed to be non-existent) container instead of the new container. This is unexpected
Reseting the nginx server will cause it to now deliver the correct website.
My question is, How do I configure nginx to just deliver whatever it finds at the configured url without having to reset the nginx server?
dockerfile as requested:
FROM nginx:alpine
RUN rm /etc/nginx/conf.d/*
COPY proxy.conf /etc/nginx/conf.d
we got all the parameter.
Using angular-cli and compile the code with
ng-build, this result in a static files, you don't need to serve
them with proxy path. You only need to set the location to the folder
with index.html and everything will work alone without http-server
NGinx:
server {
listen 443 ssl default_server;
server_name website.dev _ default_server;
ssl_certificate /etc/letsencrypt/live/www.website.dev/fullchain.pem;
ssl_certificate_key /etc/letsencrypt/live/www.website.dev/privkey.pem;
location / {
root /path/to/dist/website; # where ng build put index.html with all .js and assets
index index.htm index.html;
}
}
docker-compose NGinx:
version: "3.5"
services:
rproxy:
build:
context: ./nginx
dockerfile: Dockerfile
ports:
- 80:80
- 443:443
volumes:
- '/etc/letsencrypt:/etc/letsencrypt'
- '/host/path/shared:/path/to' # <-- Add this line. (host: /host/path/shared)
networks:
- main
networks:
main:
name: main_network
docker-compose website:
version: '3.5'
services:
website:
image: registry.website.dev/frontendcontainer:latest
command: npm run deploy
volumes:
- '/host/path/shared:/path/to' # <-- Add this line. (host: /host/path/shared)
networks:
main:
aliases:
- website
networks:
main:
external:
name: main_network
Now ng build --prod will create index.html and assets into
/host/path/shared/dist/website (internally: /path/to/dist/website).
Then NGinx will have access to those file internally on
/path/to/dist/website, without using http-server. Angular is
frontend client it don't need to be start in production mode

Nginx revers proxy can't reach docker container by host name

Nginx reverse proxy can't reach docker host. Hosting on amazon (EC2)
I want to load different apps depends on location.
nginx.conf
server {
listen 80 ;
server_name localhost;
location /web {
proxy_pass http://web:4000/;
}}
Location works and it means that nginx image builded correct either.
docker-compose file
services:
web:
image: web
container_name: web
ports:
- 4000:4000
hostname: web
networks:
- default
nginx:
image: nginx
container_name: nginx
ports:
- 80:80
depends_on:
- web
networks:
- default
networks:
default:
external:
name: my-network
I expect
- when I type in url /web it should show app from docker container
I've tried
Run single container - works fine (web or nginx)
Added 127.0.0.1 web in /etc/hosts (I can do 'curl web' but it shows localhost response)
Added index index.html in location section
Added resolver in the location section
Use links instead of network
When "docker-compose up" I can inspect docker container (web) and see IP - 192.168.10.2 . Then curl 192.168.10.2 shows me index.html. But I can't make curl http://web:4000 seems that hostname in unreachable, but I think that using IP in proxy_pass is a bad decision.
I wasn't able to handle those issues, so I've chose another approach.
Create network ipam
docker network create --gateway 172.20.0.1 --subnet 172.20.0.0/24 ipam
Assigned for each service ipv4address in docker-compose file
networks:
default:
ipv4_address: 172.20.0.5 for web
where
networks:
default:
external:
name: ipam
Add chmod for directory /var/www/html in my web docker image
chmod -R 755 /var/www/html
(seems this additional step required if you build LINUX container under windows docker)

Nginx reverse proxy and path location

Hello I'm new to the world of Docker, so I tried an installation with NGINX reverse proxy (jwilder image) and a Docker app.
I have installed both without SSL to make it easy. Since the Docker app seems to be installed in the root path I want to separate the NGINX web server and the docker app.
upstream example.com {
server 172.29.12.2:4040;
}
server {
server_name example.com;
listen 80 ;
access_log /var/log/nginx/access.log vhost;
location / {
proxy_pass http://example.com;
root /usr/share/nginx/html;
index index.html index.htm;
}
location /app {
proxy_pass http://example.com:4040;
}
}
So I want with http://example.com be redirected to the index.html
and with http://example.com/app be redirected to the docker app.
Furthermore, when I build the installation, I use in docker-compose expose: "4040" so when I reload NGINX configuration file with nginx -s reload, it warns me that I have not the port 4040 open.
With the configuration file I posted above any path lead me to the docker app.
I can't find a simple solution to my question.
As I far I understood your logic is right, docker is designed to run a single service to a single container; to reach your goal you still have a couple of thing to look after, if the EXPOSE 4040 was declared in you Docker file, that is not enough to let service reachable. in the docker-compose file you have to declare also the ports, I.E. for nginx you let host system to listen on all interface by adding
...
ports:
- 80:80
...
and this is the first thing, also you have to think on which way you want your proxy reach the "app", from the container network on the same node? If yes you can add in the composer file:
...
depends_on:
- app
...
where app is the declared name of your service in the docker-compose file like this nginx are able to reach your app with name app, so redirect will point to app:
location /app {
proxy_pass http://app:4040;
}
In case you want to reach the "app" via host network, may because one day will run in another host, you can add entry in the hosts file of the container run nginx with:
...
extra_hosts:
- "app:10.10.10.10"
- "appb:10.10.10.11"
...
and so on
Reference: https://docs.docker.com/compose/compose-file/
edit 01/01/2019!!!!! happy new year!!
an example using an "huge" docker compose file:
version: '3'
services:
app:
build: "./app" # in case you docker file is in a app dir
image: "some image name"
restart: always
command: "command to start your app"
nginx:
build: "./nginx" # in case you docker file is in a nginx dir
image: "some image name"
restart: always
ports:
- "80:80"
- "443:443"
depends_on:
- app
In the above example nginx can reach yuor app just with the "app" name so redirect will point to http://app:4040
systemctl (start directly with docker - no compose)
[Unit]
Description=app dockerized service
Requires=docker.service
After=docker.service
[Service]
ExecStartPre=/usr/bin/sleep 1
ExecStartPre=/usr/bin/docker pull mariadb:10.4
ExecStart=/usr/bin/docker run --restart=always --name=app -p 4040:4040 python:3.6-alpine # or your own builded image
ExecStop=/usr/bin/docker stop app
ExecStopPost=/usr/bin/docker rm -f app
ExecReload=/usr/bin/docker restart app
[Install]
WantedBy=multi-user.target
like the above example you can reach the app at port 4040 on the system host (which is in listen for connection on port 4040 by all interfaces) to give a specific interface: -p 10.10.10.10:4040:4040 like this will listen to port 4040 on address 10.10.10.10 (host machine)
docker-compose with extra_host:
version: '3'
services:
app:
build: "./app" # in case you docker file is in a app dir
image: "some image name"
restart: always
command: "command to start your app"
nginx:
build: "./nginx" # in case you docker file is in a nginx dir
image: "some image name"
restart: always
ports:
- "80:80"
- "443:443"
extra_hosts:
- "app:10.10.10.10"
like the above example nginx defined service can reach the name app at 10.10.10.10
least but not last extends service on compose file:
docker-compose.yml:
version: '2.1'
services:
app:
extends:
file: /path/to/app-service.yml
service: app
nginx:
extends: /path/to/nginx-service.yml
service: nginx
app-service.yml:
version: "2.1"
service:
app:
build: "./app" # in case you docker file is in a app dir
image: "some image name"
restart: always
command: "command to start your app"
nginx-service.yml
version: "2.1"
service:
nginx:
build: "./nginx" # in case you docker file is in a nginx dir
image: "some image name"
restart: always
ports:
- "80:80"
- "443:443"
extra_hosts:
- "app:10.10.10.10"
really hope the above posted are enough examples.

Resources