Map http requests from nginx to docker - docker

I have different versions of my web application running in Docker containers. And nginx is running on my host machine.
Is it possible to access the desired deployed version of my web application with the help of sub-domain such as v1.myapp.io, v2.myapp.io without reconfiguring and restarting the nginx?
I also want to access future versions in the same way?
Could anyone tell me if there is any way to achieve it?
Please consider me a newbie to Docker/nginx world.
Thanks in Advance.

Yes, although it can be done but its very difficult to achieve with docker only. kubernetes will make this very easy and everything like dns, service mapping is provided out of the box. I will include both docker and kubernetes approach:
Docker approach:
A first draft will look like this, use regex in nginx server_name and set the docker container names with a pattern. Create a /etc/hosts entry for different containers like:
172.16.0.1 v1.docker.container
172.16.0.2 v2.docker.container
And nginx server conf look like:
server {
listen 80;
server_name "~^(?<ns>[a-z]+.+)\.myapp\.io";
resolver 127.0.0.1:53 valid=30s;
# make sure $ns.docker.container is resolved to container IP
set $proxyserver "$ns.docker.container";
location / {
try_files $uri #clusterproxy;
}
location #clusterproxy {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-IP $clientip;
proxy_set_header X-Forwarded-For $clientip;
proxy_set_header X-Real-IP $clientip;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-FORWARDED-PROTO 80;
proxy_pass http://$proxyserver:80;
}
}
Kubernetes approach:
Create different service and deployment for different versions in a namespace. Lets say namespace is 'app-namespace'. Service names are self explanatory:
APP version v1: v1-app-service
APP version v2: v2-app-service
To make nginx more flexible you can add the service name as namespace to $proxyserver
Nginx rule:
server {
listen 80;
server_name "~^(?<version>[a-z]+.+)\.myapp\.io";
# you can replace this with kubernetes dns server IP
resolver 127.0.0.1:53 valid=30s;
# make sure $ns.docker.container is resolved to container IP
set $proxyserver "$version.app-namespace.svc.kubernetes";
location / {
try_files $uri #clusterproxy;
}
location #clusterproxy {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-IP $clientip;
proxy_set_header X-Forwarded-For $clientip;
proxy_set_header X-Real-IP $clientip;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-FORWARDED-PROTO 80;
proxy_pass http://$proxyserver:80;
}
}

I found another solution to this problem after digging a lot. It can be easily be done with Automated Nginx Reverse Proxy for Docker.
Once docker container for ngninx was up and running on my system.
I spun two docker containers of my webapp(diff versions) with the following command:
docker run -e VIRTUAL_HOST=v1.myapp.io --name versionOne -d myapp.io:v1
docker run -e VIRTUAL_HOST=v2.myapp.io --name versionTwo -d myapp.io:v2
and it worked out for me.
Additional notes:
1. I am using dnsmasq for handling all dns queries

Related

How to update the NGINX timeout setting for a Magento 2 Cloud Docker setup

I am working with Magento 2.4.2 (Adobe Commerce Enterprise Edition) and have a local site set up using the Magento Cloud Docker setup. I would like to change the nginx timeout setting to be long enough to let a page I'm testing run for as long as it needs to but still render the page on the browser in the frontend.
Is there a specific environment variable that I can set in my docker-compose.yml file to accomplish this? I'm not seeing anything that would make this update in the docker-environment or Dockerfile files. Do I just have to add my own custom lines to either of these files to update the timeout setting?
if you use magento cloud docker development, no you can't without overriding the docker image.
if you want to set nginx timeout, you need to override the nginx docker image and include it in docker-compose.override.yml. here are the step :
copy vendor/magento/magento-cloud-docker/images/nginx to .docker/images/nginx, i.e like this
edit .docker/images/nginx/1.19/etc/nginx.conf and .docker/images/nginx/1.19/etc/vhost.conf
create docker-compose.override.yml , like this
and run docker-compose up --build --force-recreate --no-deps --remove-orphans -d
check this link for the full example.
Note : the .docker/config.env file will be overwritten when you run ./vendor/bin/ece-docker 'build:compose'
According to Adobe Commerce support, this isn't possible on their Cloud platform which is very unfortunate
In terms of a local environment for testing, a method which is quicker and hackier than the one presented by Deki above is below:
ssh into your tls docker container
edit the /etc/nginx/conf.d/default.conf file as per below:
server {
listen 80;
listen 443 ssl;
server_name _;
ssl_certificate /etc/nginx/ssl/magento.crt;
ssl_certificate_key /etc/nginx/ssl/magento.key;
**# Add the 3 lines below**
proxy_read_timeout NEW_TIMEOUT_VALUE;
proxy_connect_timeout NEW_TIMEOUT_VALUE;
keepalive_timeout NEW_TIMEOUT_VALUE;
location / {
proxy_pass http://varnish:80;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
}
}

.Net core POST API in NGINX reverse proxy throws error 404

This is my first time posting a question. Kindly let me know if I am missing something that needs to be shared.
I am trying to POST some data into database through my C# APP and API (separately build), but it throws error 404 only for the POST API. All other pages work fine and so does the GET request. The app and API have been deployed on a LINUX machine through NGINX reverse proxy server. Both of them work on HTTP protocol. The feature works for localhost, but not for IP dependent URL.
Here is the content of service file for the app, I do not know what is missing in it. Please take care of the "/" as well where ever it is needed. While performing RnD, I found that the POST request in NGINX gets redirected to GET, I don't know if this will be helpful or not, but felt like sharing.
server {
listen myIP:6002;
server_name attendancepp;
root /home/user/net-core/Publish/AttendanceModule/AttendanceApp;
location /AttendanceApp/{
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_pass http://myIP:6002/;
proxy_set_header Accept-Encoding "";
proxy_cache_bypass $http_upgrade;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection 'upgrade';
proxy_redirect off;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
}
The app works on the URL http://myIP:6002/attendance/allPages . All the pages are accessible without any issue. Just the POST part is not working.
Thank you, in advance.
working fine after commenting
try_files $uri $uri/ =404
cd /etc/nginx/sites-available/
sudo nano default << nginx
edit file then CTR+X and 'y' for yes
sudo systemctl restart nginx

NGinx and multiple docker containers but one subdomain

My Goal
I want to use a single NGinx docker container as a proxy.
I want it to respond to traffic on my domain: "sub.domain.com" and listen on port 80 and 443.
When traffic comes in on /admin I want it to direct all traffic to one docker container (say... admin_container:6000).
When traffic comes in on /api I want it to direct all traffic to another docker container (say... api_container:5500).
When traffic comes in on any other path (/anything_else) I want it to direct all traffic to another docker container (say... website_container:5000).
Some Helpful Context
Real quick, let me provide some context in case it's helpful. I have a NodeJS website running in a docker container. I'd like to also have an Admin section, created with ASP.NET Core that runs in a second docker container. I'd like both of these websites to share and make use of a single ASP.NET Core Web Api project, running in a third docker container. So, one NodeJS project and two ASP.NET Core projects, that all live on a single subdomain:
sub.domain.com/
Serves the main website
sub.domain.com/admin
Serves the Admin website
sub.domain.com/api
Serves API endpoints and handles Database connectivity
What I have So Far
So far I have the NGinX reverse proxy set up and a single docker container for the NodeJS application. Currently all traffic on :80 is redirected to 443. All traffic on 443 is directed to the NodeJS docker container, running privately on :5000. I'll admit I'm not great with NGinx and don't fully understand how this works.
The NGinx.conf file
worker_processes 2;
events { worker_connections 1024; }
http {
sendfile on;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
upstream docker-nodejs {
server nodejs_prod:5000;
}
server {
listen 80;
server_name sub.domain.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name sub.domain.com;
ssl_certificate /etc/nginx/ssl/combined.crt;
ssl_certificate_key /etc/nginx/ssl/mysecretkeyfile.key;
location / {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_pass http://docker-nodejs;
}
}
}
Quick Note: In the above code "nodejs_prod:5000" refers to the container who's name is "nodejs_prod" and listens on port 5000. I don't understand how that works, but it is working. Somehow docker is creating DNS entries in the private network, one for each container name. I'm actually using docker-compose.
My Actual Question: How will this NGinx.conf file look when I have 2 more websites (each a docker container). It's important that /admin and /api are sent to the correct docker containers, and not handled by the "catch all" location. I'm imagining that I'll have a "catch all" location which captures all traffic that DOESN'T START WITH /admin OR /api.
Thank you!
In your nginx config you want to add location rules, such as
location /admin {
proxy_pass http://admin_container:6000/;
}
location /api {
proxy_pass http://api_container:5500/;
}
This redirects /admin to admin_container port 6000, and /api to api_container and port 5500.
docker-compose creates a network between all it's containers. Which is why http://api_container:5500/ points to the container named api_container and the port 5500. This can be used for communication between containers. You can read more about it here https://docs.docker.com/compose/networking/

Can one application with one server serve websockets and http traffic?

Is this somehow possible? Is it possible to do something like this in Ruby on top of Rack? I've seen there's websockets-rack but as far as I understand, that is only a rack module to serve ONLY websocket traffic not http also.
So basically, as the question states, is it possible to serve both protocols with just one server on the same port, instead of firing of something like Faye, websockets-rack or em-websockets?
Websockets are just an in-protocol upgrade of HTTP(s), so they are not normal TCP sockets but reuse the existing HTTP(S) connection (and thus use the same port). So, in theory it should work and from what I know it works with the Perl Mojolicious framework. But I don't know if it works work ruby/rack.
The short answer is - (AFAIK) no.
Currently, a ruby HTTP server (like rails or sinatra) and a websocket server are mutually exclusive.
After saying that, you could use a third party to emulate that. Specifically Ngnix. With Nginx you can listen to a single port, but, accroding to a path, decide whether you want to dispatch the request to the HTTP server or the Websocket server.
For example, you can run the HTTP server on port 3000, and the Websocket server on port 3020, and then configure the nginx.conf like this:
upstream http_app {
server 127.0.0.1:3000;
}
upstream websocket_app {
server 127.0.0.1:3020;
}
server {
listen 80;
server_name .example.com;
access_log /var/www/myapp.example.com/log/access.log;
error_log /var/www/myapp.example.com/log/error.log;
root /var/www/myapp.example.com;
index index.html;
location /web {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://http_app;
}
location /socket {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://websocket_app;
}
}
Now any request to http://www.example.com/web/... will reach the HTTP server, and any request to http://www.example.com/socket will reach the Websocket server.

Rails/Unicorn deploy: what creating the Unix socket?

I am deploying a Rails 2.3 // Spree app with Capistrano/Unicorn/Foreman/Upstart.
The part that I cannot figure out is how to have the /myapp/shared/sockets/unicorn.sock be automatically created by the foreman/upstart process management (at least that I think the unix socket should come from them).
What's responsible for creating the unix socket?
Let's say your configuration is nginx + unicorn . As you probably know , in the config dir you should create a file named unicorn.rb . In this file there is a description how to handle non-static requests , something like this :
upstream unicapd {
server unix:/tmp/capd.sock fail_timeout=0;
}
I've named the upstream differently than stated in tutorials , this gives me ability to host a number different applications on the same host .
Then , in your vhosts dir on the Nginx configuration you put something like this (let's say your host file is 'vhosts/myconf.conf':
location #unicorn1 {
proxy_pass http://unicapd;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
}
Here you see the instruction to nginx to serve non-static requests from the place , named "http://unicapd" , which is related to your unicorn.rb config file . This configuration is triggered by a file , which is in your init.d (in case you're running Debian) directory .
Summary : When you say bundle exec unicorn --restart , the script in your init.d triggers the code , forming a "special" file /tmp/capd.sock , which serves the dynamic content from you Rails app.
Path to unix-socket configured in unicorn's config:
...
listen "/home/user/app/shared/sockets/unicorn.sock", :backlog => 64
...
Then in nginx.conf:
location / {
try_files $uri #unicorn;
proxy_cache cache;
proxy_cache_valid 10m;
}
location #unicorn {
proxy_set_header Client-Ip $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://unix:/home/user/apps/shared/sockets/unicorn.sock;
}
When application will be started, unicorn create socket-file in setted path (user must have write access to this path)

Resources