Connection refused: when uwsgi and nginx in different containers - docker

I am trying to setup two docker containers(yes separate without docker-compose): one with nginx and one with uwsgi with basic flask app.
I run containers in same network within docker
My nginx config for site added/linked to sites-enabled(everything else is default):
server {
listen 80;
server_name 127.0.0.1;
location / {
include uwsgi_params;
uwsgi_pass 0.0.0.0:8080;
}
}
My uwsgi.ini
[uwsgi]
module = app:app
master = true
processes = 2
socket = 0.0.0.0:8080
uwsgi entry point in docker looks like
.local/bin/uwsgi --ini uwsgi.ini
Containers run fine on their own - uwsgi receives request on 8080 and nginx receives expected requests. How ever when I try to access 127.0.0.1 i get 502 status code and nginx logs error:
1 connect() failed (111: Connection refused) while connecting to
upstream, client: 192.168.4.1, server: 127.0.0.1, request: "GET /
HTTP/1.1", upstream: "uwsgi://0.0.0.0:8080", host: "127.0.0.1"
By googling i find solution that rather use one container and some_socket.sock as file or use docker compose. Apparently problem with permissions, but I do not know how to solve them or diagnose.
I launch containers with these commands:
docker run --network app_network --name nginx --rm -p 80:80 my_nginx
docker run --network app_network --name flaskapp --rm -p 8080:8080 my_uwsgi

EDIT
You can simply use the hostname of the docker container in the uwsgi_pass directive as both docker containers are on the same subnet.
location / {
include uwsgi_params;
uwsgi_pass flaskapp:8080;
}
0.0.0.0 isn't the IP address of the server, it essentially tells the server to be hosted on every IP that the device has allocated.
To connect to it from nginx, you will need to use the IP address of the container instead.
You can find the IP address of the container running uWsgi with the following command:
docker inspect CONTAINER_ID
Where CONTAINER_ID is the ID of the container you started uwsgi in.
From here you can update the nginx config as follows:
uwsgi_pass IP_ADDRESS:8080;
Where IP_ADDRESS is the one you found from the command above
You can also set the ip address of the container when you start it with the following option
--ip <ip>
Be careful, however, to ensure that the IP address you set is in the same subnet as the standard IP's assigned.

Related

why Connection Refused error on simple Docker + nginx configuration?

I have this Dockerfile:
FROM nginx:latest
COPY devops/nginx_proxy.conf /etc/nginx/conf.d/default.conf
EXPOSE 8080
and a devops/nginx_proxy.conf:
server {
listen 8080;
client_max_body_size 32M;
underscores_in_headers on;
}
Running the Dockerfile with docker run -p 8080:80 test and then testing with curl http://localhost/, I see this error:
curl: (7) Failed to connect to localhost port 80: Connection refused
Even more curious, curl http://localhost:8080/ returns this:
curl: (52) Empty reply from server
Why am I getting these errors?
With Docker you can bind containers ports to host ports using the -p option.
General rule
docker run -p HOST_PORT:CONTAINER_PORT
Bind container 8080 port to the 80 of the host
docker run -p 80:8080 test
Ports which are not bound to the host (i.e., -p 80:80 instead of -p 127.0.0.1:80:80) are accessible from the outside
Bind the port limiting the access to localhost
docker run -p 127.0.0.1:80:8080 test

Nginx reverse proxy not finding other internal Docker container using hostname

I have two docker containers. One runs Kestrel (172.17.0.3), The other runs Nginx (172.17.0.4) using a reverse proxy to connect to Kestrel. Nginx connects fine when I use internal Docker ip of Kestrel container but when I try to connect to Kestrel using container's hostname in nginx.conf (kestral) I get following error:
2020/06/30 00:23:03 [emerg] 58#58: host not found in upstream "kestrel" in /etc/nginx/nginx.conf:7
nginx: [emerg] host not found in upstream "kestrel" in /etc/nginx/nginx.conf:7
I launched containers with these two lines
docker run -d --name kestrel --restart always -h kestrel mykestrelimage
docker run -d --name nginx --restart always -p 80:80 -h nginx mynginximage
My nginx.conf file below.
http {
# I've tried with and without line below that I found on Stackoverflow
resolver 127.0.0.11 ipv6=off;
server {
listen 80;
location / {
# lines below don't work
# proxy_pass http//kestrel:80;
# proxy_pass http//kestrel
# proxy_pass http//kestrel:80/;
# proxy_pass http//kestrel/;
# when I put internal docker ip of Kestrel server works fine
proxy_pass http://172.17.0.3:80/;
}
}
}
events {
}
I figured out solution to my problem. There were two issues.
First problem: By default Docker uses default bridge network when creating containers. The default Docker bridge network does not resolve DNS though. You have to create a custom bridge network and then specify network when creating docker containers. The below allowed me to ping between containers using hostname
docker network create --driver=bridge mycustomnetwork
docker run -d --name=kestrel --restart=always -h kestrel.local --network=mycustomnetwork mykestrelimage
docker run -d --name=nginx --restart always -p 80:80 -h nginx.local --network=mycustomnetwork mynginximage
Second problem: Even though it was only one kestrel server for some reason Nginx required that I setup an upstream section in /etc/nginx/nginx.conf
http {
upstream backendservers {
server kestrel;
}
server {
listen 80;
location / {
proxy_pass http://backendservers/;
}
}
}
events {
}

docker nginx proxy nginx connect() failed (111: Connection refused) while connecting to upstream

I'm trying to run a nginx container as the main entry point for all of my websites and web services. I managed to run a portainer as a container, and I'm able to reach it from the internet. Right now I'm trying to reach a static website hosted by another nginx container, but I fail doing so - when I go to the URL, I get
502 Bad Gateway
I've tried adding the upstream section to my main nginx's config, but nothing changed (after every config change, I reload my main nginx service inside the container).
On the other hand, adding upstream is something I'd like to avoid if it's possible because spawning multiple different applications would require adding an upstream for each application - and that's much more work than I'd expect.
Here is my main nginx's configuration file:
events {
worker_connections 1024;
}
http {
server {
listen 80;
location /portainer/ {
proxy_pass http://portainer:9000/;
}
location /helicon/ {
proxy_pass http://helicon:8001/;
}
}
}
Here is how I start my main nginx container:
docker run -p 80:80 --name nginx -v /var/nginx/conf:/etc/nginx:ro --net=internal-net -d nginx
Here is my static website's nginx configuration file:
events {
worker_connections 1024;
}
http {
server {
listen 80;
server_name helicon;
root /var/www/html/helicon;
error_log /var/log/nginx/localhost.error.log;
access_log /var/log/nginx/localhost.access.log;
}
}
Here is how the docker-compose file to create and start that container:
version: '3.5'
services:
helicon:
build: .
image: helicon
ports:
- "127.0.0.1:8001:80"
container_name: helicon
networks:
- internal-net
networks:
internal-net:
external: true
I'm using internal-net network to keep all apps in the same network instead of deprecated --link option for docker run
When I go to http://my.server.ip.address/helicon I get 502. Then I check logs with docker logs nginx and there is an information
2018/06/24 11:15:28 [error] 848#848: *466 connect() failed (111: Connection refused) while connecting to upstream, client: Y.Y.Y.Y, server: , request: "GET /helicon/ HTTP/1.1", upstream: "http://172.18.0.2:8001/", host: "X.X.X.X"
The helicon container indeed has an IP address of 172.18.0.2.
What am I missing? Maybe my approach should be completely different from using networks?
Kind regards,
Daniel
To anyone coming across this page here is my little contribution to your understanding of docker networking.
I would like to illustrate with an example scenario.
We are running several contianers with docker-compose such as the following:
Docker container client
Docker container nginx reverse proxy
Docker container service1
Docker container service2
etc ...
to make sure you are setup correctly, make sure of the following:
All containers are on same network!
first run: "docker network ls" to find your network name for your stack
secondly run: "docker network inspect [your_stack_network_name]"
note that the ports you expose in docker-compose have nothing to do with nginx reverse proxying!
that means that any ports you exposed in your docker-compose file are available on your actual host machine i.e your latptop or pc via your browser BUT for proxying purposes you must point nginx to actual ports of your services.
A walkthrough:
lets say service1 runs inside container 1 on port 3000, you mapped port 8080 in your docker-compose file like so: "8080:3000", with this configuration on your local machine you can access the container via your browser on port 8080 (localhost:8080) BUT for nginx reverse proxy container, when trying to proxy to service1, port 8080 is not relevant! the dockerized nginx reverse-proxy will use docker DNS to map service1 to its ip within then docker network and look at it entirely independently from your local host.
to nginx reverse proxy inside docker network, service1 only listens on port 3000!!!
so make sure to point nginx to the correct port!
Solved. I was working on this for hours thinking it was a nginx config issue. I modified nginx.conf endlessly but couldn't fix it. I was getting 502 Bad Gateway and the error description was:
failed (111: Connection refused) while connecting to upstream
I was looking in the wrong place. It turns out that the http server in my index.js file was listening on the url 'localhost'.
httpServer.listen(PORT, 'localhost', async err => {
This works fine on your development machine, but when running inside a container it must be named the url of the container itself. My containers are networked, and in my case the container is named 'backend'
I changed the url from 'localhost' to 'backend' and everything works fine.
httpServer.listen(PORT, 'backend', async err => {
I was getting the same error. In docker-compose.yml my service port was mapped to a different port (e.g. 1234:8080) and I was using the mapped port number (1234) inside nginx.conf.
However, inside the docker network, the containers do not use their mapped port numbers. In order to solve this problem, I changed the proxy_pass statement to use the correct port number (8080).
To make it clear the working configuration is like this: (Check the used port number in nginx.conf!)
docker-compose.yml
version: '3.8'
services:
...
web1:
ports:
- 1234:8080
networks:
- net1
...
proxy:
image: nginx:latest
networks:
- net1
...
networks:
net1:
driver: bridge
nginx.conf
...
location /api
...
proxy_pass http://web1:8080/;
I must thank user8458126 for pointing me in the right direction.
For me, I had overwrote my default.conf nginx file and mistyped the destination for it in my docker file which told nginx not to be listening on the correct port, and instead defaulting to port 80.
Long story short, make sure you're overwriting to the correct path.
What I had:
COPY ./default.conf ./etc/nginx/default.conf
Correct:
COPY ./default.conf ./etc/nginx/conf.d/default.conf
Hope this saves someone a few hours of racking their brain.

Nginx reverse proxy to an app in host

I have an app that is running outside Docker on port 5000. I am trying to run a reverse proxy in nginx via Docker compose but am unable to communicate with the host's port 5000. In my docker-compose.yml file I have:
ports:
- 80:80
- 443:443
- 5000:5000
When I try to run this I get:
ERROR: for nginx Cannot start service nginx: driver failed programming external connectivity on endpoint nginx (374026a0d34c8b6b789dcd82d6aee6c4684b3201258cfbd3fb18623c4101): Error starting userland proxy: listen tcp 0.0.0.0:5000: bind: address already in use
If I comment out - 5000:5000 I get:
[error] 6#6: *1 connect() failed (111: Connection refused) while connecting to upstream
How do I connect to an already running app in the Host from a Docker nginx container?
EDIT:
My nginx.conf file
user www-data;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 768;
}
http {
upstream mysite {
server 0.0.0.0:5000;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://mysite;
}
}
}
The response when I try to curl localhost is 502 Bad Gateway. The app itself and curl 127.0.0.1:5000 responds fine from the host.
EDIT 2:
I have also tried the solution found here but I get nginx: [emerg] host not found in upstream "docker". Docker is my host's hostname.
EDIT 3:
My docker-compose.yml
version: '3'
services:
simple:
build: ./simple
container_name: simple
ports:
- 80:80
- 443:443
My Dockerfile:
FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80 443
CMD ["nginx", "-g", "daemon off;", "-c", "/etc/nginx/nginx.conf"]
EDIT:
I am getting the computer host via the "hostname" command in linux.
The problem lies with 0.0.0.0:5000. Since Nginx is running inside the docker, it tries to find this address inside docker machine but fails since there is nothing running on 0.0.0.0:5000 inside docker.
So in order to resolve this
You need to give it an address that it can reach. Solving it requires that you
first run your application at 0.0.0.0:5000 on your host machine i.e you should be able to open your application at 0.0.0.0:5000 from your browser.
Find your IP address. once you get your IP address you should be able to
open you application through ip_address:5000. since your docker and host share the same network this address can be reached from docker also
Now, replace the 0.0.0.0:5000 in your Nginx conf file with this ip_address:5000. you would be able to serve your application
172.17.0.1 is the default host ip available to docker container running on host.
Just use 172.17.0.1:5000 in your nginx conf file and you should be able to connect to your application running on host outside the container.
My docker version is 19.03.12 where I tested the same.
I need to use a different variable to access the host container: http://host.docker.internal.
Note: I'm running on a Windows host. Not sure if that matters.

Dynamic DNS Resolution with HAProxy and Docker

I'm trying to setup HAProxy inside a Docker host.
Using HAProxy 1.7 and Docker 1.12
My haproxy.cfg looks like:
# Simple configuration for an HTTP proxy listening on port 81 on all
# interfaces and forwarding requests to a single backend "servers" with a
# single server "server1" listening on 127.0.0.1:8000
global
daemon
maxconn 256
resolvers docker
# nameserver dnsmasq 127.0.0.1:53
nameserver dns 127.0.0.1:53
defaults
mode http
timeout connect 5000ms
timeout client 50000ms
timeout server 50000ms
default-server init-addr none
frontend http-in
bind *:80
default_backend www_somedomain1_com
# Define hosts
acl host_www_somedomain1_com hdr(host) -i somedomain1.com
acl host_www_somedomain1_com hdr(host) -i www.somedomain1.com
acl host_www_somedomain2_com hdr(host) -i www.somedomain2.com
## figure out which one to use
use_backend www_somedomain1_com if host_www_somedomain1_com
use_backend www_somedomain2_com if host_www_somedomain2_com
backend www_somedomain1_com
# Utilizing the Docker DNS to resolve below host
# server server1 www-somedomain1-com maxconn 32 check port 80
server server1 www-somedomain1-com resolvers docker check maxconn 32
backend www_somedomain2_com
# Utilizing the Docker DNS to resolve below host
# server server1 www-somedomain2-com maxconn 32 check resolvers docker resolve-prefer ipv4
server server1 www-somedomain2-com maxconn 32 check port 80
I want to use Docker's embedded DNS system - which, in my understanding, is only enabled when using a user defined network.
So I create a network (using the default bridge driver)
docker network create mynetwork
When I run my two named docker containers, (my-haproxy and www-somedomain1-com) I add them to that network with the --net flag.
Docker run commands:
docker run --name myhaproxy --net mynetwork -p 80:80 -d haproxy
docker run --name www-somedomain1-com --net mynetwork -d nginx
I know the Docker dns is functional because I can resolve from one container to the other when I hop on them in a bash shell. I can't get the right combo/config in HAProxy to enable the dynamic DNS resolution.
HAProxy stats page always shows the downstream backends as brown/resolution issue....
Some things that have helped:
- the "default-server init-addr none" helps pass the haproxy config check on startup.
Any guidance is greatly appreciated!
I think your issue is that you are using 127.0.0.1:53 for your resolver dns, when it needs to be 127.0.0.11:53 for the docker bridge network.
Here is my haproxy setup for dev docker stuff:
global
quiet
defaults
log global
mode http
option forwardfor
timeout connect 60s
timeout client 60s
timeout server 60s
default-server init-addr none
resolvers docker_resolver
nameserver dns 127.0.0.11:53
frontend https-proxy
bind 0.0.0.0:80
bind 0.0.0.0:443 ssl crt /usr/local/etc/haproxy/dev_server.pem
redirect scheme https if !{ ssl_fc }
acl is_api_server hdr(host) -i mywebsite
use_backend api_server if is_api_server
backend api_server
server haproxyapi api-server-dev:80 check inter 10s resolvers docker_resolver resolve-prefer ipv4

Resources