Could not connect websocket using Action Cable in Rails 5.1. HTTP server is Unicorn on nginx and adapter is Redis.
Rails configuration is the following.
# config/environments/production.rb
config.action_cable.disable_request_forgery_protection = true
nginx configuration is the following.
upstream unicorn {
server unix:/rails/current/tmp/sockets/unicorn.sock;
}
server {
listen 80;
charset utf-8;
server_name sub.example.com;
root /rails/current/public;
location / {
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_redirect off;
if (!-f $request_filename) {
proxy_pass http://unicorn;
break;
}
}
location /cable {
proxy_pass http://unicorn;
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto https;
proxy_redirect off;
}
...
}
Errors in console of web browser are the following.
WebSocket connection to 'wss://sub.example.com/cable' failed: Error during WebSocket handshake: Unexpected response code: 404
Errors in Rails are the following.
[ERROR] Failed to upgrade to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: close, HTTP_UPGRADE: )
[INFO] Finished "/cable/"[non-WebSocket] for xxx.xxx.xxx.xxx at 2017-xx-xx
A strangest thing is a error in Rails does not have HTTP_UPGRADE value. But HTTP request headers of web browser include Upgrade key and websocket value. Also setting "websocket" for proxy header in nginx configurations.
What should I do?
On the back of that writeup, I just realized the universal solution is
a simple change to the config/secrets.yml file to reference the
ENV["PORT"] setting
...
# Be sure to restart your server when you modify this file.
development:
secret_key_base: 231bf79489c63f8c8facd7...
action_cable_url : http://localhost:<%= ENV["PORT"] %>
test:
secret_key_base: 1ab8adbcf8410aebb...
action_cable_url : http://localhost:<%= ENV["PORT"] %>
# Do not keep production secrets in the repository,
# instead read values from the environment.
production:
secret_key_base: <%= ENV["SECRET_KEY_BASE"] %>
action_cable_url : <%= ENV["SERVER_PORT"] %>
.
You can also have this in your dot file:
export PORT=3000
source
Related
I have been struggling to get my Rails app deployed correctly for a while now, and have decided it is finally time to consult the community for some help. I have read just about every stackoverflow post on this issues, including the following, with no luck:
Rails 5 ActionCable fails to upgrade to WebSocket on Elastic Beanstalk -> From this post I ensured I was using an Application Load Balancer
ActionCable on AWS: Error during WebSocket handshake: Unexpected response code: 404 -> Configured an nginx proxy, with no change
Problem Description
I am using the following setup:
Ruby 2.7.5
Rails 6.1.0
GraphQL
React Frontend (separate repo)
Elastic Beanstalk
Ruby 2.7 running on 64bit Amazon Linux 2/3.4.1
Application Load Balancer
Postgres ActionCable adapter
My application is deployed to AWS Elasticbeanstalk and all requests to /graphql are successful. However, when attempting to connect to /cable I get this error in my browser console:
WebSocket connection to 'wss://api.redacted.io/cable' failed:
When checking the Elastic Beanstalk logs I see:
/var/app/containerfiles/logs/production.log:
I, [2022-02-20T19:35:25.849990 #32761] INFO -- : [8e1d3e86-81cc-4708-89d3-ebad56470f8f] Started GET "/cable" for <redacted IP> at 2022-02-20 19:35:25 +0000
I, [2022-02-20T19:35:25.850342 #32761] INFO -- : [8e1d3e86-81cc-4708-89d3-ebad56470f8f] Started GET "/cable/"[non-WebSocket] for <redacted IP> at 2022-02-20 19:35:25 +0000
E, [2022-02-20T19:35:25.850384 #32761] ERROR -- : [8e1d3e86-81cc-4708-89d3-ebad56470f8f] Failed to upgrade to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: close, HTTP_UPGRADE: )
I, [2022-02-20T19:35:25.850419 #32761] INFO -- : [8e1d3e86-81cc-4708-89d3-ebad56470f8f] Finished "/cable/"[non-WebSocket] for <redacted IP> at 2022-02-20 19:35:25 +0000
Potentially Relevant Files
cable.yml:
development:
adapter: postgresql
test:
adapter: test
production:
adapter: postgresql
production.rb:
...
config.action_cable.url = 'wss://api.redacted.io/cable'
config.action_cable.allowed_request_origins = ['redacted.io', 'http://redacted.io', 'https://redacted.io']
...
.ebextensions/nginx_proxy.config:
files:
"/etc/nginx/conf.d/websockets.conf" :
content: |
upstream backend {
server unix:///var/run/puma/my_app.sock;
}
server_names_hash_bucket_size 128;
server {
listen 80;
access_log /var/log/nginx/access.log;
error_log /var/log/nginx/error.log;
server_name redacted.elasticbeanstalk.com;
# prevents 502 bad gateway error
large_client_header_buffers 8 32k;
location / {
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header Host $http_host;
proxy_set_header X-NginX-Proxy true;
# prevents 502 bad gateway error
proxy_buffers 8 32k;
proxy_buffer_size 64k;
proxy_pass http://backend;
proxy_redirect off;
location /assets {
root /var/app/current/public;
}
# enables WS support
location /cable {
proxy_pass http://backend/cable;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
}
}
}
container_commands:
01restart_nginx:
command: "nginx -t && service nginx restart"
After posting on reddit, I was able to fix my issue by:
Removing my .ebextensions/nginx_proxy.config file.
Creating a new file, .platform/nginx/conf.d/elasticbeanstalk/websocket.conf with the contents:
location /cable {
proxy_pass http://my_app/cable;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
I'm trying to use Hotwire for Rails, it works perfectly with my development environment, but having issues with websocket connection after deploy to our UAT (staging) environment.
Stack
Ruby: 3.0.0
Rails: 6.0.3.5
Nginx: 1.14.1
Passenger: Phusion Passenger 6.0.4
nginx.conf on my Gateway server
upstream myapp {
server myapp.uat.mycompany.com.au:80;
}
server {
server_name myapp.uat.mycompany.com.au;
location / {
proxy_pass http://myapp;
proxy_redirect off;
proxy_buffering off;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /cable {
proxy_pass http://active_migration;
proxy_http_version 1.1;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "upgrade";
proxy_set_header Host $host;
}
listen 443 ssl; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/myapp.uat.mycompany.com.au/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/myapp.uat.mycompany.com.au/privkey.pem; # managed by Certbot
}
server {
if ($host = myapp.uat.mycompany.com.au) {
return 301 https://$host$request_uri;
} # managed by Certbot
server_name myapp.uat.mycompany.com.au;
listen 80;
return 404; # managed by Certbot
}
nginx.conf on my app server
server {
server_name myapp.uat.mycompany.com.au;
listen 80;
underscores_in_headers on;
passenger_enabled on;
passenger_app_env uat;
root /var/www/myapp/current/public;
location /cable {
passenger_app_group_name myapp_action_cable;
passenger_force_max_concurrent_requests_per_process 0;
}
}
cable.yml
development:
adapter: redis
url: <%= ENV.fetch("ACTION_CABLE_URL") { "redis://localhost:6379/8" } %>
channel_prefix: myapp
test:
adapter: test
uat:
adapter: redis
url: <%= ENV.fetch("ACTION_CABLE_URL") { "redis://localhost:6379/8" } %>
channel_prefix: myapp
production:
adapter: redis
url: <%= ENV.fetch("ACTION_CABLE_URL") { "redis://localhost:6379/8" } %>
channel_prefix: myapp
uat.rb (environment configuration file)
config.action_cable.url = 'wss://myapp.uat.mycompany.com.au/cable'
config.action_cable.allowed_request_origins = ['https://myapp.uat.mycompany.com.au', 'http://myapp.uat.mycompany.com.au']
Errors displayed in Chrome console
connection.js:70 WebSocket connection to 'wss://myapp.uat.mycompany.com.au/cable' failed: WebSocket is closed before the connection is established.
Errors in uat.log
I, [2021-10-29T15:13:33.876173 #668] INFO -- : [25f60292-8a89-4ce5-9f35-fb3ad56bef00] Started GET "/cable" for 10.1.32.123 at 2021-10-29 15:13:33 +1100
I, [2021-10-29T15:13:33.876890 #668] INFO -- : [25f60292-8a89-4ce5-9f35-fb3ad56bef00] Started GET "/cable/" [WebSocket] for 10.1.32.123 at 2021-10-29 15:13:33 +1100
I, [2021-10-29T15:13:33.876966 #668] INFO -- : [25f60292-8a89-4ce5-9f35-fb3ad56bef00] Successfully upgraded to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: upgrade, HTTP_UPGRADE: websocket)
E, [2021-10-29T15:13:33.877255 #668] ERROR -- : [25f60292-8a89-4ce5-9f35-fb3ad56bef00] WebSocket error occurred: wrong number of arguments (given 2, expected 1)
E, [2021-10-29T15:13:33.878046 #668] ERROR -- : WebSocket error occurred: wrong number of arguments (given 2, expected 1)
E, [2021-10-29T15:13:34.234885 #668] ERROR -- : WebSocket error occurred: wrong number of arguments (given 2, expected 1)
E, [2021-10-29T15:13:36.397514 #20846] ERROR -- : WebSocket error occurred: wrong number of arguments (given 2, expected 1)
Note: the above WebSocket error keeps happening ...
I googled around and found someone had the similar problem, so tried with the solution of
commenting out the config.action_cable.url in the uat.rb environment file and put action_cable_meta_tag in the application.html.slim file, but it didn't work.
I've spent lots of time with it, could someone shed some light on this please?
It works fine in my production environment. The only difference I can see is that the passenger version is Phusion Passenger 6.0.4 in my production env.
Will try to upgrade my staging passenger version
I'm having trouble getting ActionCable hooked up in my prod environment, and related questions haven't had a working solution. I'm using an nginx+puma setup with Rails 6.1.3.2 on Ubuntu 20.04. I have confirmed that redis-server is running on port 6379, and that Rails is running as production.
Here's what I'm getting in my logs:
I, [2021-05-25T22:47:25.335711 #72559] INFO -- : [5d1a85f7-0102-4d25-bd4e-d81355b846ee] Started GET "/cable" for 74.111.15.223 at 2021-05-25 22:47:25 +0000
I, [2021-05-25T22:47:25.336283 #72559] INFO -- : [5d1a85f7-0102-4d25-bd4e-d81355b846ee] Started GET "/cable/"[non-WebSocket] for 74.111.15.223 at 2021-05-25 22:47:25 +0000
E, [2021-05-25T22:47:25.336344 #72559] ERROR -- : [5d1a85f7-0102-4d25-bd4e-d81355b846ee] Failed to upgrade to WebSocket (REQUEST_METHOD: GET, HTTP_CONNECTION: close, HTTP_UPGRADE: )
I, [2021-05-25T22:47:25.336377 #72559] INFO -- : [5d1a85f7-0102-4d25-bd4e-d81355b846ee] Finished "/cable/"[non-WebSocket] for 74.111.15.223 at 2021-05-25 22:47:25 +0000
This happens every few seconds. You can see matching output in the browser console:
For one, I'm pretty sure that I need to add some sections to my nginx site config, such as a /cable section, but I haven't figured out the correct settings. Here's my current config:
server {
root /home/rails/myapp/current/public;
server_name myapp.com;
index index.htm index.html;
location ~ /.well-known {
allow all;
}
location / {
proxy_pass http://localhost:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
# needed to allow serving of assets and other public files
location ~ ^/(assets|packs|graphs)/ {
gzip_static on;
expires 1y;
add_header Cache-Control public;
add_header Last-Modified "";
add_header ETag "";
}
listen 443 ssl; # managed by Certbot
ssl_certificate /etc/letsencrypt/live/myapp.com/fullchain.pem; # managed by Certbot
ssl_certificate_key /etc/letsencrypt/live/myapp.com/privkey.pem; # managed by Certbot
include /etc/letsencrypt/options-ssl-nginx.conf; # managed by Certbot
ssl_dhparam /etc/letsencrypt/ssl-dhparams.pem; # managed by Certbot
}
server {
if ($host = myapp.com) {
return 301 https://$host$request_uri;
} # managed by Certbot
listen 80;
server_name myapp.com;
return 404; # managed by Certbot
}
Here's my config/cable.yml:
development:
adapter: async
test:
adapter: test
production:
adapter: redis
url: <%= ENV.fetch("REDIS_URL") { "redis://localhost:6379/1" } %>
channel_prefix: myapp_production
In config/environments/production.rb, I've left these lines commented out:
# Mount Action Cable outside main process or domain.
# config.action_cable.mount_path = nil
# config.action_cable.url = 'wss://example.com/cable'
# config.action_cable.allowed_request_origins = [ 'http://example.com', /http:\/\/example.*/ ]
I have not mounted ActionCable manually in my routes or anything. This is as standard of a setup as you can get. I'm thinking the answer lies centrally in a correct nginx configuration, but I don't know what it should be. Perhaps there are Rails config settings that are needed too, though. I don't remember having to change any when deploying with passenger, but maybe puma is a different story.
Update
I also noticed that a lot of the proposed solutions in other questions, like this one, seem to reference a .sock file in tmp/sockets/. My sockets/ directory is empty, though. Web server's running fine besides ActionCable though.
Update #2
I also noticed that changing the config.action_cable.url to something like ws://myapp.com instead of wss://myapp.com has no effect even after restarting Rails. The browser console errors still say its trying to connect to wss://myapp.com. Possibly due to how I'm set up to force redirect HTTP to HTTPS. I wonder if that has anything to do with it?
I got it working. Here are the settings I needed:
nginx config
The server section from the config in my question must be modified to include the following two sections for the / and /cable locations:
location / {
proxy_pass http://localhost:3000;
proxy_set_header Host $host;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Forwarded-Proto $scheme;
}
location /cable {
proxy_pass http://localhost:3000;
proxy_http_version 1.1;
proxy_set_header Upgrade "websocket";
proxy_set_header Connection "Upgrade";
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
config/environments/production.rb
# Change myapp.com to your app's location
config.action_cable.allowed_request_origins = [ 'https://myapp.com' ]
config.hosts << "myapp.com"
config.hosts << "localhost"
Huge thanks to Lam Phan in the comments for helping me out.
I'm trying to introduce chatting to my Rails app. For this purpose I used gem private_pub and it works perfectly in development mode.
In production I was using Apache + Passenger, but I couldn't configure Faye with it, so I changed Apache to Nginx. My main app is still on Apache server, and this demo on Nginx with port 8080 (just for test).
I'm able to connect to faye.js by entering http://chat.mysite.com:8080/faye.js, but the connection from app throws an error (browser console).
WebSocket connection to 'ws://localhost:9292/faye' failed: Error in connection establishment: net::ERR_CONNECTION_REFUSED
After this error, another error appears every 5 seconds.
faye.js:2 GET http://localhost:9292/faye?message=%5B%7B%22channel%22%3A%22%2Fmeta%2Fhands…22%2C%22callback-polling%22%5D%2C%22id%22%3A%221%22%7D%5D&jsonp=jsonp2 net::ERR_CONNECTION_REFUSED
My private_pub.yml
production:
server: "http://localhost:9292/faye"
secret_token: "mysecret"
signature_expiration: 3600 # one hour
My private_pub.ru
require "bundler/setup"
require "yaml"
require "faye"
require "private_pub"
Faye::WebSocket.load_adapter('thin')
PrivatePub.load_config(File.expand_path("../config/private_pub.yml", __FILE__), ENV["RAILS_ENV"] || "development")
run PrivatePub.faye_app
My nginx site.conf
server {
listen 8080;
server_name www.chat.mysite.com;
passenger_enabled on;
passenger_app_env production;
root /var/www/mysite/public;
location ^~ /faye {
proxy_pass http://127.0.0.1:9292;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection "Upgrade";
proxy_buffering off;
proxy_redirect off;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
#proxy_set_header X-Forwarded-Proto https;
proxy_cache_bypass $http_pragma $http_authorization;
proxy_no_cache $http_pragma $http_authorization;
break;
}
}
If I change private_pub.yml to http://localhost:9292/faye/faye, I saw error like "can't load resource /faye/faye.js".
How should I change my Nginx conf or app yml to resolve websocket error?
I see private_pub is quite similar to ActionCable in its design. Before you go too far down the road, you may want read my blog post on "ActionAcable - The good and bad parts" as it addresses suitable use cases when a system like private_pub is good, and when it's not.
If of course you are aware of the shortcomings already, then good luck!
I tried to configure my private_pub.yml as #niceman said. Now all is working good.
production:
server: "http://my-ip:8080/faye"
Using Nginx 1.6 with private_pub gem
Here are my config files:
private_pub.ru
# Run with: rackup private_pub.ru -s thin -E production
require "bundler/setup"
require "yaml"
require "faye"
require "private_pub"
Faye::WebSocket.load_adapter('thin')
PrivatePub.load_config(File.expand_path("../config/private_pub.yml", __FILE__), ENV["RAILS_ENV"] || "development")
run PrivatePub.faye_app
Private_pub.yml
development:
server: "http://localhost:9292/faye/faye"
secret_token: "secret"
test:
server: "http://localhost:9292/faye/faye"
secret_token: "secret"
production:
server: "http://xxxxx.com/faye/faye"
secret_token: "my secret token"
signature_expiration: 3600 # one hour
in my Nginx.conf
location /faye {
proxy_pass http://0.0.0.0:9292;
break;
}
The service is running but really really slow and I get those errors on safari:
WebSocket connection to 'ws://xxxxx.com/faye' failed: Unexpected response code: 400
Failed to load resource: the server responded with a status of 404 (Not Found)
Failed to load resource: the server responded with a status of 502 (Bad Gateway)
Any thoughts?
OK.. I found the solution for those who ever want to install Faye/Private_pub on Nginx running thin and unicorn.
First:
You have to understand that your upstream server is your localhost:9292 (127.0.0.1:9292)
you set your upstream in your Nginx conf by adding the following:
location /faye {
proxy_pass http://127.0.0.1:9292;
proxy_http_version 1.1;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
proxy_set_header X-Real-IP $remote_addr;
proxy_set_header Upgrade $http_upgrade;
proxy_set_header Connection $connection_upgrade;
proxy_buffering off;
proxy_redirect off;
proxy_connect_timeout 90;
proxy_send_timeout 90;
proxy_read_timeout 90;
proxy_set_header X-Forwarded-Proto https;
break;
}
Also for those who have a 504 after that change the config file in the Nginx and php.fmp (if you have it) so that the timeout is increased.
Don't forget to reload your Nginx. If you still have errors check your Nginx error.log