I want to serve a flask app that uses embedded bokeh serve from a server on my local network. To illustrate I made an example using the bokeh serve example and a docker image to replicate the server. The docker image runs Nginx and Gunicorn. I think there is a problem with my nginx configuration routing the requests to the /bkapp uri.
I have detailed the problem and provided all source code in the following git repo
I have started a discussion on bokeh google group
Single Container
In order to reduce the complexity of running nginx in its own container I built this image that runs nginx in same container as the web app
Installation
NOTE: I am using Docker version 17.09.0-ce
Download or clone repo and navigate to this directory (single_container).
# as root
docker build -f Dockerfile -t single_container .
build
start a terminal session in new container
# as root
docker run -ti single_container:latest
In new container start nginx
nginx
now start gunicorn
gunicorn -w 1 -b :8000 flask_gunicorn_embed:app
start
in a separate terminal (on host machine) find the IP address of the single_container container you are running
#as root
docker ps
# then do copy CONTAINER ID and inspect it
docker inspect [CONTIANER ID] | grep IPAddress
find
PROBLEM
Using IP found above (with container running) check out in firefox with inspector.
As you can in screenshot above (see screenshots folder "single_container_broken.png" for raw the get request just hangs
broke_1
I can verify that nginx is serving the static files though by navigating to /bkapp/static/ (see bokeh_recipe/single_container/nginx/bokeh_app.conf for config)
static
Another oddidy is that I try to hit the embedded bokeh server directly (with /bkapp/) but i end up with a 400 (denied?)
bkapp
Note about app
to reduce complexity of dynamically assigning available ports to tornado workers I hard coded in 46518 for port to talk to bokeh serve
nginx config
I know you could just look at bokeh_recipe/single_container/nginx/bokeh_app.conf but I want to show it here.
I think I need to config nginx to make explicit that the "request" to bkapp to the 127.0.0.1:46518 is originating FROM the server not the client.
## Define the parameters for a specific virtual host/server
server {
# define what port to listen on
listen 80;
# Define the specified charset to the “Content-Type” response header field
charset utf-8;
# Configure NGINX to deliver static content from the specified folder
# NOTE this should be a docker volume shared from the bokehrecipe_web container (css, js for bokeh serve)
location /bkapp/static/ {
alias /home/flask/app/web/static/;
autoindex on;
}
# Configure NGINX to reverse proxy HTTP requests to the upstream server (Gunicorn (WSGI server))
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_http_version 1.1;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host:$server_port;
proxy_buffering off;
}
# deal with the http://127.0.0.1/bkapp/autoload.js (note hard coded port for now)
location /bkapp/ {
proxy_pass http://127.0.0.1:46518;
}
}
Related
I deployed an nginx:1.22.1 instance alongside a static react app server on a worker node in a docker swarm. This is docker swarm mode, not classic swarm.
The advertise address that I listed when joining the swarm is internal to the data center, I do not know if that matters because I can still access these services with the public addresses.
Both containers are pinned to the same worker node and communicate over a user-created overlay network.
I can retrieve the full bundle directly from the react app server over the public network.
I cannot retrieve the full bundle through the nginx reverse-proxy server over the public network.
When I attempt to fetch the bundle using chrome browser as the user-agent I get 2 errors:
net::ERR_INCOMPLETE_CHUNKED_ENCODING 200 (OK).
The app bundle is cutoff mid js function as if a chunk of data was not transmitted.
Rarely, the upstream server will send html and not a js bundle. But I receive that whole response body and it is not truncated like the js bundle.
I have played with all kinds of configuration and cannot get it to work.
(most relevant)
This is my configuration under /etc/nginx/conf.d/default.conf
resolver 127.0.0.11 valid=10s;
error_log /dev/stdout info;
map $http_upgrade $connection_upgrade {
default upgrade;
'' close;
}
server {
listen 80;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
ssl_certificate /etc/nginx/certs/nginx.pem;
ssl_certificate_key /etc/nginx/certs/key.pem;
client_max_body_size 100M;
proxy_buffers 8 1024k;
proxy_buffer_size 1024k;
proxy_max_temp_file_size 1024m;
location / {
set $reactapp reacthost;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_set_header X-Forwarded-For $remote_addr;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_pass http://$reactapp:3000/;
proxy_redirect off;
}
root /usr/share/nginx/html;
}
error_page 500 502 503 504 /50x.html;
}
I use the variable $reactapp for service discovery after nginx server start. See NGINX blog here.
Note that the nginx:1.22.1 instance runs with user nginx after it is deployed to the stack. I only see this below message when I deploy via docker stack. If I start the container directly using docker engine, I do not see it.
/docker-entrypoint.sh: /docker-entrypoint.d/ is not empty, will attempt to perform configuration
/docker-entrypoint.sh: Looking for shell scripts in /docker-entrypoint.d/
/docker-entrypoint.sh: Launching /docker-entrypoint.d/10-listen-on-ipv6-by-default.sh
10-listen-on-ipv6-by-default.sh: info: can not modify /etc/nginx/conf.d/default.conf (read-only file system?)
However, I can exec into the container as the nginx user, access /var/cache/nginx/, and create a directory.
I do not know if:
my server / location configuration is plain bad.
The NGINX server cannot write a part of the container it needs to write when the service is deployed in stack mode.
If I cannot access the server properly over the public network via the overlay network.
Prior to using docker stack I was able to use this reverse proxy.
The two containers were on the same host without swarm mode running.
The containers communicated over a bridge network.
The reverse proxy server port was published on the public interface of the server it was deployed on.
The NGINX server started after the upstream server.
Because there is no depends_on key honored in stack mode I have to allow DNS service discovery after the NGINX server starts up. Placing them on an overlay gives me more flexibility in how I do my deployments, but this has become a bit muddled. There are enough differences between the two environments that it has become difficult to get the stack to behave as I expect.
I'm having an issue configuring my nginx.conf for the container (server) container.
Without nginx: I can properly access the app through gunicorn.
With nginx: I get 502 Bad Gateway
My first question would be, should I even have nginx on the container(server) if I have ingress nginx?
My second question is, why is this configuration not working. Here is my docker file for container (server)
Dockerfile
FROM python:3 as builder
WORKDIR '/app'
COPY requirements.txt ./
RUN pip install -r requirements.txt
COPY . .
EXPOSE 8000
ENTRYPOINT [ "gunicorn", "-b", "0.0.0.0:8000", "run:app" ]
FROM nginx
EXPOSE 80
COPY ./nginx/default.conf /etc/nginx/conf.d/default.conf
nginx.conf
upstream flask_server {
server localhost:8000;
}
server {
listen 80;
server_name localhost;
location / {
proxy_pass http://flask_server;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
Command
# build
docker build -t test/server .
# run
docker run -p 80:80 test/server
I suppose my issue is the upstream localhost is not working. When I'm developing locally is there no way for me to test this container specifically through docker? Or do I have to test locally with docker-compose and put nginx in a separate container?
I know, that post could be outdated, but it is still searching by Google :)
My second question is, why is this configuration not working. Here is my docker file for container (server)
So, as I can see, your specified proxy_pass as http://flask_server.
That means, flask_server hostname should be the resolvable name.
This name could be just the name of your Docker container, but your Dockerfile or docker run does not contain such options.
This name could be group of servers, but in this case, you have to define this directive
upstream flask_server {
server <hostname1>;
server <hostname2>;
}
(c) https://nginx.org/en/docs/http/ngx_http_proxy_module.html#proxy_pass
If a domain name resolves to several addresses, all of them will be used in a round-robin fashion. In addition, an address can be specified as a server group.
To sum up: to fix 502 error (which means connection timeout to the upstream) you can use 2 ways:
change proxy_pass http://flask_server; to proxy_pass http://0.0.0.0:8000;
OR
define upstream directive according manual
My first question would be, should I even have nginx on the container(server) if I have ingress nginx?
In my opinion - I should not.
You can deploy Nginx on the container(server) just in case you want (in the future) load-balance traffic between multiply gunicorn instances or want to provide some HTTP authentication or access restrictions.
I have 2 dockers containers running on my EC2 instance:
Docker1: Wordpress website running with PHP server mapped to port 8081 of EC2 instance.
Docker2: Portal created on Angular running with NGINX mapped to port 8082 of EC2 instance.
I want to use the same EC2 instance for my domain and subdomain xyz.com and portal.xyz.com on the same port 80.
Ideally, if the request comes from xyz.com, it should redirect to Docker1 running on 8081 and if it is from portal.xyz.com, it should be redirected to Docker2 running on 8082.
Is it feasible and if yes, how? I do not want to spawn 2 EC2 instances for this and both have to be mapped to HTTP on port 80.
Using multiple load balancers and target groups can solve your problem. https://aws.amazon.com/about-aws/whats-new/2019/07/amazon-ecs-services-now-support-multiple-load-balancer-target-groups/
You can set up both load balancers to listen on HTTP and target your one ECS instance on different ports. After that, setting up the routes in Route53 will be straight forward.
I had done something similar on a VPS server, technically it should work on an ec2 instance as well.
I created a new docker network 'proxy-network' (Note: You can do without creating a network and just proxy to localhost:8081 and localhost:8082. This is just cleaner)
Launch all the application servers in that network with proper names (eg: wordpress, angular). Use --name in run command or conatiner_name in docker-compose.
Launch a new nginx server map host port 80 and 443(if you need https to work). I used nginx:latest image. Created a new default.conf and replace /etc/nginx/conf.d/default.conf in container.
Sample proxy.conf should like this:
server {
listen 80;
server_name domain1.com www.domain1.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://wordpress;
}
}
server {
listen 80 default_server;
server_name domain2.com www.domain2.com;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://angular;
}
}
Once you update the alias records in domain registrar works like a charm. Hope it helps. Good luck.
I'm trying to keep a jenkins container(docker) behind nginx reverse proxy. It works fine with this path, https://example.com/ but it returns 502 Bad Gateway when I add parameter to the path, https://example.com/jenkins.
The docker container for jenkins is run like this
docker container run -d -p 127.0.0.1:8080:8080 jenkins/jenkins
Here is my code,
server {
listen 80;
root /var/www/html;
server_name schoolcloudy.com www.schoolcloudy.com;
location / {
proxy_pass http://localhost:8000;
}
}
# Virtual Host configuration for example.com
upstream jenkins {
server 127.0.0.1:8080;
}
server {
listen 80;
server_name jenkins;
location /jenkins {
proxy_pass http://jenkins;
proxy_redirect 127.0.0.1:8080 https://schoolcloudy.com/jenkins;
}
}
Specify the Jenkins container's network with --network=host flag when you run the container. This way the container will be able to interact with host network or use the container's IP explicitly in the Nginx conf.
good practice in such questions is official documentation usage:
wiki.jenkins.io
I've configured Jenkins behind Nginx reverse proxy several time, wiki works fine for me each time.
P.S.: look like proxy_pass option value in your config should be changed to http://127.0.0.1:8080
I am reading a lot these days about how to setup and run a docker stack. But one of the things I am always missing out on is how to setup that particular containers respond to access through their domain name and not just their container name using docker dns.
What I mean is, that say I have a microservice which is accessible externally, for example: users.mycompany.com, it will go through to the microservice container which is handling the users api
Then when I try to access the customer-list.mycompany.com, it will go through to the microservice container which is handling the customer lists
Of course, using docker dns I can access them and link them into a docker network, but this only really works for wanting to access container to container, but not internet to container.
Does anybody know how I should do that? Or the best way to set that up.
So, you need to use the concept of port publishing, so that a port from your container is accessible via a port from your host. Using this, you can can setup a simple proxy_pass from an Nginx that will redirect users.mycompany.com to myhost:1337 (assuming that you published your port to 1337)
So, if you want to do this, you'll need to setup your container to expose a certain port using:
docker run -d -p 5000:5000 training/webapp # publish image port 5000 to host port 5000
You can then from your host curl your localhost:5000 to access the container.
curl -X GET localhost:5000
If you want to setup a domain name in front, you'll need to have a webserver instance that allows you to proxy_pass your hostname to your container.
i.e. in Nginx:
server {
listen 80;
server_name users.mycompany.com;
location / {
proxy_pass http://localhost:5000;
}
}
I would advise you to follow this tutorial, and maybe check the docker run reference.
For all I know, Docker doesn't provide this feature out of the box. But surely there are several workarounds here. In fact you need to deploy a DNS on your host that will distinguish the containers and resolve their domain names in dynamical IPs. So you could give a try to:
Deploy some of Docker-aware DNS solutions (I suggest you to use SkyDNSv1 / SkyDock);
Configure your host to work with this DNS (by default SkyDNS makes the containers know each other by name, but the host is not aware of it);
Run your containers with explicit --hostname (you will probably use scheme container_name.image_name.dev.skydns.local)
You can skip step #2 and run your browser inside container too. It will discover the web application container by hostname.
Here is a one solution with the nginx and docker-compose:
users.mycompany.com is in nginx container on port 8097
customer-list.mycompany.com is in nginx container on port 8098
Nginx configuration:
server {
listen 0.0.0.0:8097;
root /root/for/users.mycompany.com
...
}
server {
listen 0.0.0.0:8098;
root /root/for/customer-list.mycompany.com
...
}
server {
listen 0.0.0.0:80;
server_name users.mycompany.com;
location / {
proxy_pass http://0.0.0.0:8097;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
server {
listen 0.0.0.0:80;
server_name customer-list.mycompany.com;
location / {
proxy_pass http://0.0.0.0:8098;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $remote_addr;
}
}
Docker compose configuration :
services:
nginx:
container_name: MY_nginx
build:
context: .docker/nginx
ports:
- '80:80'
...