I have a SocketIO server run inside docker swarm + Nginx for load balancing which is working fine.
But I have a small problem with duplicated emit events.
Example: I have 30 replicas (copies) of SocketIO server; 1 Nginx gateway and Everything running in Docker Swarm
My Nginx config:
server {
listen 80;
server_name 10.10.10.212;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $host;
proxy_pass http://loadbalancer;
# enable WebSockets
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
My SocketIO server emit
socketio.emit('warehouse_scan:added', 'HB10790')
Browser result
How can I solve this?
duplicated events on emit is my false.
I set wrong parameter while setting up everything with nginx , match client versions , etc ...
Wrong config in backend:
socketio = SocketIO(app, logger=True, engineio_logger=True, policy_server=True, async_mode='eventlet', manage_session=True, cors_allowed_origins="*", message_queue=socket_io_msg_q)
The correct one that fixed duplication:
socketio = SocketIO(app, logger=True, engineio_logger=True, policy_server=True, async_mode='eventlet', manage_session=True)
The problem is at this line:
message_queue=socket_io_msg_q <-- error!
This will trigger all emit events from backend server to REDIS then get back to client. If i have 50 replicas, clients will receive the amount of message from these replicas.
Related
How to call GRPC Server which is located in docker container on Swarm cluster from NGINX reverse proxy?
GRPC Server in container/service called webui with kestrel development certificate installed
NGINX Proxy which is located outside the stack and routes access to Swarm stacks
GRPC Client is located on a separate virtual machine on another network, the browser page at https://demo.myorg.com is available
part nginx.conf
server {
listen 443 ssl;
server_name demo.myorg.com;
...
location / {
proxy_pass https://namestack_webui;
}
GRPC Client appsetting.json
{
"ConnectionStrings": {
"Database": "Data Source=Server_name;Initial Catalog=DB;User Id=user;Password=pass;MultipleActiveResultSets=True;"
}
...
"GRPCServerUri": "https://demo.myorg.com/",
...
}
}
Problem when connecting GRPC Client to Server, i get error
END] GetOpcDaServerSettingsQuery. Time spent: 7,7166ms
fail: Grpc.Net.Client.Internal.GrpcCall[6]
Error starting gRPC call.
System.Net.Http.HttpRequestException: The SSL connection could not be established, see inner exception.
---> System.Security.Authentication.AuthenticationException: Authentication failed, see inner exception.
---> System.ComponentModel.Win32Exception (0x80090367): No common application protocol exists between the client and the server. Application protocol negotiation failed..
--- End of inner exception stack trace ---
Tried to write and specify a kestrel development certificate (for GRPC Client) that is loaded into the Swarm stack (namestack) through which the other containers in the stack are authenticated, the error is the same.
I understand that it is necessary to specify in appsetting.json the GRPC Server container address (https://namestack_webui), but it is behind NGINX, and I can only specify the GRPC host address (https://demo.myorg.com), tell me what is wrong?
The perfect solution for such a case was not found online.
I finally figured out and found a solution to my question, and I publish it for discussion.
If there are no comments against, then mark it as correct, at least it works for me and will work for YOU.
to proxy grpc connections through NGINX in the configuration, the location section must specify something similar to the url /PackageName.ServiceName/MethodName (This is indicated here by https://learn.microsoft.com/en-aspnetus/aspnet/core/grpc/troubleshoot?view=aspnetcor7.0#unable-to-start-aspnet-core-grpc-app-on-macos )
This URL can be checked with the developer or in the logs when grpc client connects
Should be used to proxy directive grpc_pass grpcs://namecontainer;
Should use http2 protocol.
So the correct configuration file for nginx in my case should look like this
server {
listen 443 ssl **http2**;
server_name demo.myorg.com;
ssl_certificate ...;
ssl_certificate_key ...;
ssl_protocols TLSv1 TLSv1.1 TLSv1.2;
ssl_ciphers RC4:HIGH:!aNULL:!MD5:!kEDH;
add_header Strict-Transport-Security 'max-age=604800';
underscores_in_headers on;
large_client_header_buffers 4 16k;
location / {
proxy_pass https://name_container;
# Configuration for WebSockets
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_cache off;
# WebSockets were implemented after http/1.0
proxy_http_version 1.1;
# Configuration for ServerSentEvents
proxy_buffering off;
# Configuration for LongPolling or if your KeepAliveInterval is longer than 60 seconds
proxy_read_timeout 100s;
proxy_set_header Host $host;
proxy_cache_bypass $http_upgrade;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
proxy_set_header X-URL-SCHEME https;
}
location /App.Name.Api.Contract.ApiService/UpdateOpcDaTags {
grpc_pass grpcs://name_container;
}
}
I have different versions of my web application running in Docker containers. And nginx is running on my host machine.
Is it possible to access the desired deployed version of my web application with the help of sub-domain such as v1.myapp.io, v2.myapp.io without reconfiguring and restarting the nginx?
I also want to access future versions in the same way?
Could anyone tell me if there is any way to achieve it?
Please consider me a newbie to Docker/nginx world.
Thanks in Advance.
Yes, although it can be done but its very difficult to achieve with docker only. kubernetes will make this very easy and everything like dns, service mapping is provided out of the box. I will include both docker and kubernetes approach:
Docker approach:
A first draft will look like this, use regex in nginx server_name and set the docker container names with a pattern. Create a /etc/hosts entry for different containers like:
172.16.0.1 v1.docker.container
172.16.0.2 v2.docker.container
And nginx server conf look like:
server {
listen 80;
server_name "~^(?<ns>[a-z]+.+)\.myapp\.io";
resolver 127.0.0.1:53 valid=30s;
# make sure $ns.docker.container is resolved to container IP
set $proxyserver "$ns.docker.container";
location / {
try_files $uri #clusterproxy;
}
location #clusterproxy {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-IP $clientip;
proxy_set_header X-Forwarded-For $clientip;
proxy_set_header X-Real-IP $clientip;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-FORWARDED-PROTO 80;
proxy_pass http://$proxyserver:80;
}
}
Kubernetes approach:
Create different service and deployment for different versions in a namespace. Lets say namespace is 'app-namespace'. Service names are self explanatory:
APP version v1: v1-app-service
APP version v2: v2-app-service
To make nginx more flexible you can add the service name as namespace to $proxyserver
Nginx rule:
server {
listen 80;
server_name "~^(?<version>[a-z]+.+)\.myapp\.io";
# you can replace this with kubernetes dns server IP
resolver 127.0.0.1:53 valid=30s;
# make sure $ns.docker.container is resolved to container IP
set $proxyserver "$version.app-namespace.svc.kubernetes";
location / {
try_files $uri #clusterproxy;
}
location #clusterproxy {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-IP $clientip;
proxy_set_header X-Forwarded-For $clientip;
proxy_set_header X-Real-IP $clientip;
proxy_set_header X-Forwarded-Host $host;
proxy_set_header X-FORWARDED-PROTO 80;
proxy_pass http://$proxyserver:80;
}
}
I found another solution to this problem after digging a lot. It can be easily be done with Automated Nginx Reverse Proxy for Docker.
Once docker container for ngninx was up and running on my system.
I spun two docker containers of my webapp(diff versions) with the following command:
docker run -e VIRTUAL_HOST=v1.myapp.io --name versionOne -d myapp.io:v1
docker run -e VIRTUAL_HOST=v2.myapp.io --name versionTwo -d myapp.io:v2
and it worked out for me.
Additional notes:
1. I am using dnsmasq for handling all dns queries
My Goal
I want to use a single NGinx docker container as a proxy.
I want it to respond to traffic on my domain: "sub.domain.com" and listen on port 80 and 443.
When traffic comes in on /admin I want it to direct all traffic to one docker container (say... admin_container:6000).
When traffic comes in on /api I want it to direct all traffic to another docker container (say... api_container:5500).
When traffic comes in on any other path (/anything_else) I want it to direct all traffic to another docker container (say... website_container:5000).
Some Helpful Context
Real quick, let me provide some context in case it's helpful. I have a NodeJS website running in a docker container. I'd like to also have an Admin section, created with ASP.NET Core that runs in a second docker container. I'd like both of these websites to share and make use of a single ASP.NET Core Web Api project, running in a third docker container. So, one NodeJS project and two ASP.NET Core projects, that all live on a single subdomain:
sub.domain.com/
Serves the main website
sub.domain.com/admin
Serves the Admin website
sub.domain.com/api
Serves API endpoints and handles Database connectivity
What I have So Far
So far I have the NGinX reverse proxy set up and a single docker container for the NodeJS application. Currently all traffic on :80 is redirected to 443. All traffic on 443 is directed to the NodeJS docker container, running privately on :5000. I'll admit I'm not great with NGinx and don't fully understand how this works.
The NGinx.conf file
worker_processes 2;
events { worker_connections 1024; }
http {
sendfile on;
proxy_buffer_size 128k;
proxy_buffers 4 256k;
proxy_busy_buffers_size 256k;
upstream docker-nodejs {
server nodejs_prod:5000;
}
server {
listen 80;
server_name sub.domain.com;
return 301 https://$host$request_uri;
}
server {
listen 443 ssl;
server_name sub.domain.com;
ssl_certificate /etc/nginx/ssl/combined.crt;
ssl_certificate_key /etc/nginx/ssl/mysecretkeyfile.key;
location / {
proxy_redirect off;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-Proto https;
proxy_set_header X-Forwarded-Ssl on;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Host $server_name;
proxy_pass http://docker-nodejs;
}
}
}
Quick Note: In the above code "nodejs_prod:5000" refers to the container who's name is "nodejs_prod" and listens on port 5000. I don't understand how that works, but it is working. Somehow docker is creating DNS entries in the private network, one for each container name. I'm actually using docker-compose.
My Actual Question: How will this NGinx.conf file look when I have 2 more websites (each a docker container). It's important that /admin and /api are sent to the correct docker containers, and not handled by the "catch all" location. I'm imagining that I'll have a "catch all" location which captures all traffic that DOESN'T START WITH /admin OR /api.
Thank you!
In your nginx config you want to add location rules, such as
location /admin {
proxy_pass http://admin_container:6000/;
}
location /api {
proxy_pass http://api_container:5500/;
}
This redirects /admin to admin_container port 6000, and /api to api_container and port 5500.
docker-compose creates a network between all it's containers. Which is why http://api_container:5500/ points to the container named api_container and the port 5500. This can be used for communication between containers. You can read more about it here https://docs.docker.com/compose/networking/
I have Rails 3.2 application with gem websocket-rails 0.7.
On development machine, all work fine
On production enviroment, I use Nginx/1.6 as proxy server and Unicorn as http server. Thin is used on standalone mode (following https://github.com/websocket-rails/websocket-rails/wiki/Standalone-Server-Mode).
nginx config:
location /websocket {
proxy_pass http://localhost:3001/websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
On backend side, I have the following code for send notification to clients
WebsocketRails[:callback_requests].trigger 'new', call_request
On client side, I got a connection using:
dispatcher = new WebSocketRails window.location.host + ':3001/websocket'
channel = dispatcher.subscribe 'callback_requests'
But notification doesn't come to the client.
Related issue on github - github.com/websocket-rails/websocket-rails/issues/211
Your nginx config is matching requests below /websocket/ with the trailing /. That is the directory component of /websocket/blah.
If you look in your nginx access log file you'll find your requests to /websocket are being 301 redirected to /websocket/.
Remove the trailing /
location /websocket {
proxy_pass http://localhost:3001/websocket;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
Is this somehow possible? Is it possible to do something like this in Ruby on top of Rack? I've seen there's websockets-rack but as far as I understand, that is only a rack module to serve ONLY websocket traffic not http also.
So basically, as the question states, is it possible to serve both protocols with just one server on the same port, instead of firing of something like Faye, websockets-rack or em-websockets?
Websockets are just an in-protocol upgrade of HTTP(s), so they are not normal TCP sockets but reuse the existing HTTP(S) connection (and thus use the same port). So, in theory it should work and from what I know it works with the Perl Mojolicious framework. But I don't know if it works work ruby/rack.
The short answer is - (AFAIK) no.
Currently, a ruby HTTP server (like rails or sinatra) and a websocket server are mutually exclusive.
After saying that, you could use a third party to emulate that. Specifically Ngnix. With Nginx you can listen to a single port, but, accroding to a path, decide whether you want to dispatch the request to the HTTP server or the Websocket server.
For example, you can run the HTTP server on port 3000, and the Websocket server on port 3020, and then configure the nginx.conf like this:
upstream http_app {
server 127.0.0.1:3000;
}
upstream websocket_app {
server 127.0.0.1:3020;
}
server {
listen 80;
server_name .example.com;
access_log /var/www/myapp.example.com/log/access.log;
error_log /var/www/myapp.example.com/log/error.log;
root /var/www/myapp.example.com;
index index.html;
location /web {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://http_app;
}
location /socket {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
proxy_pass http://websocket_app;
}
}
Now any request to http://www.example.com/web/... will reach the HTTP server, and any request to http://www.example.com/socket will reach the Websocket server.