I have an app that is running outside Docker on port 5000. I am trying to run a reverse proxy in nginx via Docker compose but am unable to communicate with the host's port 5000. In my docker-compose.yml file I have:
ports:
- 80:80
- 443:443
- 5000:5000
When I try to run this I get:
ERROR: for nginx Cannot start service nginx: driver failed programming external connectivity on endpoint nginx (374026a0d34c8b6b789dcd82d6aee6c4684b3201258cfbd3fb18623c4101): Error starting userland proxy: listen tcp 0.0.0.0:5000: bind: address already in use
If I comment out - 5000:5000 I get:
[error] 6#6: *1 connect() failed (111: Connection refused) while connecting to upstream
How do I connect to an already running app in the Host from a Docker nginx container?
EDIT:
My nginx.conf file
user www-data;
worker_processes auto;
pid /run/nginx.pid;
events {
worker_connections 768;
}
http {
upstream mysite {
server 0.0.0.0:5000;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://mysite;
}
}
}
The response when I try to curl localhost is 502 Bad Gateway. The app itself and curl 127.0.0.1:5000 responds fine from the host.
EDIT 2:
I have also tried the solution found here but I get nginx: [emerg] host not found in upstream "docker". Docker is my host's hostname.
EDIT 3:
My docker-compose.yml
version: '3'
services:
simple:
build: ./simple
container_name: simple
ports:
- 80:80
- 443:443
My Dockerfile:
FROM nginx
COPY nginx.conf /etc/nginx/nginx.conf
EXPOSE 80 443
CMD ["nginx", "-g", "daemon off;", "-c", "/etc/nginx/nginx.conf"]
EDIT:
I am getting the computer host via the "hostname" command in linux.
The problem lies with 0.0.0.0:5000. Since Nginx is running inside the docker, it tries to find this address inside docker machine but fails since there is nothing running on 0.0.0.0:5000 inside docker.
So in order to resolve this
You need to give it an address that it can reach. Solving it requires that you
first run your application at 0.0.0.0:5000 on your host machine i.e you should be able to open your application at 0.0.0.0:5000 from your browser.
Find your IP address. once you get your IP address you should be able to
open you application through ip_address:5000. since your docker and host share the same network this address can be reached from docker also
Now, replace the 0.0.0.0:5000 in your Nginx conf file with this ip_address:5000. you would be able to serve your application
172.17.0.1 is the default host ip available to docker container running on host.
Just use 172.17.0.1:5000 in your nginx conf file and you should be able to connect to your application running on host outside the container.
My docker version is 19.03.12 where I tested the same.
I need to use a different variable to access the host container: http://host.docker.internal.
Note: I'm running on a Windows host. Not sure if that matters.
Related
I am trying to setup two docker containers(yes separate without docker-compose): one with nginx and one with uwsgi with basic flask app.
I run containers in same network within docker
My nginx config for site added/linked to sites-enabled(everything else is default):
server {
listen 80;
server_name 127.0.0.1;
location / {
include uwsgi_params;
uwsgi_pass 0.0.0.0:8080;
}
}
My uwsgi.ini
[uwsgi]
module = app:app
master = true
processes = 2
socket = 0.0.0.0:8080
uwsgi entry point in docker looks like
.local/bin/uwsgi --ini uwsgi.ini
Containers run fine on their own - uwsgi receives request on 8080 and nginx receives expected requests. How ever when I try to access 127.0.0.1 i get 502 status code and nginx logs error:
1 connect() failed (111: Connection refused) while connecting to
upstream, client: 192.168.4.1, server: 127.0.0.1, request: "GET /
HTTP/1.1", upstream: "uwsgi://0.0.0.0:8080", host: "127.0.0.1"
By googling i find solution that rather use one container and some_socket.sock as file or use docker compose. Apparently problem with permissions, but I do not know how to solve them or diagnose.
I launch containers with these commands:
docker run --network app_network --name nginx --rm -p 80:80 my_nginx
docker run --network app_network --name flaskapp --rm -p 8080:8080 my_uwsgi
EDIT
You can simply use the hostname of the docker container in the uwsgi_pass directive as both docker containers are on the same subnet.
location / {
include uwsgi_params;
uwsgi_pass flaskapp:8080;
}
0.0.0.0 isn't the IP address of the server, it essentially tells the server to be hosted on every IP that the device has allocated.
To connect to it from nginx, you will need to use the IP address of the container instead.
You can find the IP address of the container running uWsgi with the following command:
docker inspect CONTAINER_ID
Where CONTAINER_ID is the ID of the container you started uwsgi in.
From here you can update the nginx config as follows:
uwsgi_pass IP_ADDRESS:8080;
Where IP_ADDRESS is the one you found from the command above
You can also set the ip address of the container when you start it with the following option
--ip <ip>
Be careful, however, to ensure that the IP address you set is in the same subnet as the standard IP's assigned.
I have two docker containers. One runs Kestrel (172.17.0.3), The other runs Nginx (172.17.0.4) using a reverse proxy to connect to Kestrel. Nginx connects fine when I use internal Docker ip of Kestrel container but when I try to connect to Kestrel using container's hostname in nginx.conf (kestral) I get following error:
2020/06/30 00:23:03 [emerg] 58#58: host not found in upstream "kestrel" in /etc/nginx/nginx.conf:7
nginx: [emerg] host not found in upstream "kestrel" in /etc/nginx/nginx.conf:7
I launched containers with these two lines
docker run -d --name kestrel --restart always -h kestrel mykestrelimage
docker run -d --name nginx --restart always -p 80:80 -h nginx mynginximage
My nginx.conf file below.
http {
# I've tried with and without line below that I found on Stackoverflow
resolver 127.0.0.11 ipv6=off;
server {
listen 80;
location / {
# lines below don't work
# proxy_pass http//kestrel:80;
# proxy_pass http//kestrel
# proxy_pass http//kestrel:80/;
# proxy_pass http//kestrel/;
# when I put internal docker ip of Kestrel server works fine
proxy_pass http://172.17.0.3:80/;
}
}
}
events {
}
I figured out solution to my problem. There were two issues.
First problem: By default Docker uses default bridge network when creating containers. The default Docker bridge network does not resolve DNS though. You have to create a custom bridge network and then specify network when creating docker containers. The below allowed me to ping between containers using hostname
docker network create --driver=bridge mycustomnetwork
docker run -d --name=kestrel --restart=always -h kestrel.local --network=mycustomnetwork mykestrelimage
docker run -d --name=nginx --restart always -p 80:80 -h nginx.local --network=mycustomnetwork mynginximage
Second problem: Even though it was only one kestrel server for some reason Nginx required that I setup an upstream section in /etc/nginx/nginx.conf
http {
upstream backendservers {
server kestrel;
}
server {
listen 80;
location / {
proxy_pass http://backendservers/;
}
}
}
events {
}
I'm setting up my AWS EC2 instance. I wanted to let that instance access via https but I get a
This is what I tried
run docker pull abiosoft/caddy
Put Caddyfile in home folder
Run mkdir -p $HOME/caddycerts; chmod ugo+rwx $HOME/caddycerts;
Run docker run -d -e "CADDYPATH=/etc/caddycerts" -v $HOME/Caddyfile:/etc/Caddyfile -v $HOME/caddycerts:/etc/caddycerts -p 443:443 abiosoft/caddy
Run docker restart *dockerName*
My Caddyfile looks like this:
some-domain-name.com {
tls myemail
proxy / 172.17.0.1:9001 {
header_upstream Host {host}
header_upstream X-Real-IP {remote}
header_upstream X-Forwarded-Proto {scheme}
}
}
Error: curl: (7) Failed to connect to some-domain-name.com port 443: Connection refused
EC2 instance's security group has https enabled for port 443
when you use AWS make sure that the port you are using is allowed and you have the right to use it
AWS Security group and ACL doesn't give connection refused, they silently drops the packet. From the message connection refused it seems the service isn't running or server isn't listening on port 443.
Have you tried to telnet it locally ? Does it work ?
telnet localhost 443
Error: curl: (7) Failed to connect to some-domain-name.com port 443: Connection refused
The above error message means that your web server is not running on the specified port of 443. You can simply validate via a telnet (which I see in James's answer above).
From your caddyfile it points to port 9001. The first line of the Caddyfile is always the address of the site to serve.
Without seeing the dockerfile it's hard to pinpoint, but I'd say there's nothing configured to run on 443 in your application
I'm trying to run a nginx container as the main entry point for all of my websites and web services. I managed to run a portainer as a container, and I'm able to reach it from the internet. Right now I'm trying to reach a static website hosted by another nginx container, but I fail doing so - when I go to the URL, I get
502 Bad Gateway
I've tried adding the upstream section to my main nginx's config, but nothing changed (after every config change, I reload my main nginx service inside the container).
On the other hand, adding upstream is something I'd like to avoid if it's possible because spawning multiple different applications would require adding an upstream for each application - and that's much more work than I'd expect.
Here is my main nginx's configuration file:
events {
worker_connections 1024;
}
http {
server {
listen 80;
location /portainer/ {
proxy_pass http://portainer:9000/;
}
location /helicon/ {
proxy_pass http://helicon:8001/;
}
}
}
Here is how I start my main nginx container:
docker run -p 80:80 --name nginx -v /var/nginx/conf:/etc/nginx:ro --net=internal-net -d nginx
Here is my static website's nginx configuration file:
events {
worker_connections 1024;
}
http {
server {
listen 80;
server_name helicon;
root /var/www/html/helicon;
error_log /var/log/nginx/localhost.error.log;
access_log /var/log/nginx/localhost.access.log;
}
}
Here is how the docker-compose file to create and start that container:
version: '3.5'
services:
helicon:
build: .
image: helicon
ports:
- "127.0.0.1:8001:80"
container_name: helicon
networks:
- internal-net
networks:
internal-net:
external: true
I'm using internal-net network to keep all apps in the same network instead of deprecated --link option for docker run
When I go to http://my.server.ip.address/helicon I get 502. Then I check logs with docker logs nginx and there is an information
2018/06/24 11:15:28 [error] 848#848: *466 connect() failed (111: Connection refused) while connecting to upstream, client: Y.Y.Y.Y, server: , request: "GET /helicon/ HTTP/1.1", upstream: "http://172.18.0.2:8001/", host: "X.X.X.X"
The helicon container indeed has an IP address of 172.18.0.2.
What am I missing? Maybe my approach should be completely different from using networks?
Kind regards,
Daniel
To anyone coming across this page here is my little contribution to your understanding of docker networking.
I would like to illustrate with an example scenario.
We are running several contianers with docker-compose such as the following:
Docker container client
Docker container nginx reverse proxy
Docker container service1
Docker container service2
etc ...
to make sure you are setup correctly, make sure of the following:
All containers are on same network!
first run: "docker network ls" to find your network name for your stack
secondly run: "docker network inspect [your_stack_network_name]"
note that the ports you expose in docker-compose have nothing to do with nginx reverse proxying!
that means that any ports you exposed in your docker-compose file are available on your actual host machine i.e your latptop or pc via your browser BUT for proxying purposes you must point nginx to actual ports of your services.
A walkthrough:
lets say service1 runs inside container 1 on port 3000, you mapped port 8080 in your docker-compose file like so: "8080:3000", with this configuration on your local machine you can access the container via your browser on port 8080 (localhost:8080) BUT for nginx reverse proxy container, when trying to proxy to service1, port 8080 is not relevant! the dockerized nginx reverse-proxy will use docker DNS to map service1 to its ip within then docker network and look at it entirely independently from your local host.
to nginx reverse proxy inside docker network, service1 only listens on port 3000!!!
so make sure to point nginx to the correct port!
Solved. I was working on this for hours thinking it was a nginx config issue. I modified nginx.conf endlessly but couldn't fix it. I was getting 502 Bad Gateway and the error description was:
failed (111: Connection refused) while connecting to upstream
I was looking in the wrong place. It turns out that the http server in my index.js file was listening on the url 'localhost'.
httpServer.listen(PORT, 'localhost', async err => {
This works fine on your development machine, but when running inside a container it must be named the url of the container itself. My containers are networked, and in my case the container is named 'backend'
I changed the url from 'localhost' to 'backend' and everything works fine.
httpServer.listen(PORT, 'backend', async err => {
I was getting the same error. In docker-compose.yml my service port was mapped to a different port (e.g. 1234:8080) and I was using the mapped port number (1234) inside nginx.conf.
However, inside the docker network, the containers do not use their mapped port numbers. In order to solve this problem, I changed the proxy_pass statement to use the correct port number (8080).
To make it clear the working configuration is like this: (Check the used port number in nginx.conf!)
docker-compose.yml
version: '3.8'
services:
...
web1:
ports:
- 1234:8080
networks:
- net1
...
proxy:
image: nginx:latest
networks:
- net1
...
networks:
net1:
driver: bridge
nginx.conf
...
location /api
...
proxy_pass http://web1:8080/;
I must thank user8458126 for pointing me in the right direction.
For me, I had overwrote my default.conf nginx file and mistyped the destination for it in my docker file which told nginx not to be listening on the correct port, and instead defaulting to port 80.
Long story short, make sure you're overwriting to the correct path.
What I had:
COPY ./default.conf ./etc/nginx/default.conf
Correct:
COPY ./default.conf ./etc/nginx/conf.d/default.conf
Hope this saves someone a few hours of racking their brain.
After setup of enketo express, When i run enketo-express using command "docker-compose up -d" on an ubuntu server, it shows an below error related to nginx
ERROR: for docker_nginx_1 Cannot start service nginx: b'driver failed programming external connectivity on endpoint docker_nginx_1 (7c414e255d50f42a0fa14d07c0b0d29125f666d77e55e5eb4437e43e3e4d9454): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use'
ERROR: for nginx Cannot start service nginx: b'driver failed programming external connectivity on endpoint docker_nginx_1 (7c414e255d50f42a0fa14d07c0b0d29125f666d77e55e5eb4437e43e3e4d9454): Error starting userland proxy: listen tcp 0.0.0.0:80: bind: address already in use'
By default 80 port is used by the apache service on ubuntu server. This means you need to change your port.
Just try to Bind port 8081 of your ubuntu with the port 80 of the container. For this use this command,
$ run docker run -d -p 8081:80 --name webserver nginx
This creates the link you need to access it at http://localhost:8081/
Note- Change port in "setup/docker/docker-compose.yml" file. eg: 80:80 to 8081:80
It's because something is already running on port 80 on the host machine.
View which process it is, get the PID -
$ lsof -i:80
$ netstat -tulnap | grep :80
Kill the process -
$ kill -9 $PID
Try now & you will be good -
$ docker-compose up -d