I just deployed a Rails app to Elastic Beanstalk, and every request is giving me a 502 error.
Here's the contents of /var/logs/nginx/error.log
2015/05/20 16:24:25 [warn] 1535#0: conflicting server name "localhost" on 0.0.0.0:80, ignored
2015/05/20 16:27:12 [crit] 1537#0: *20 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.51.94, server: _, request: "POST /get HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/get", host: "securities-api-prod.elasticbeanstalk.com"
2015/05/20 16:27:17 [crit] 1537#0: *20 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.51.94, server: _, request: "POST /get HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/get", host: "securities-api-prod.elasticbeanstalk.com"
2015/05/20 16:27:19 [crit] 1537#0: *20 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.51.94, server: _, request: "POST /get HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/get", host: "securities-api-prod.elasticbeanstalk.com"
2015/05/20 16:27:22 [crit] 1537#0: *16 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.44.210, server: _, request: "GET /datapoint?tickers=AAPL&datapoints=Ratings HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/datapoint?tickers=AAPL&datapoints=Ratings", host: "securities-api-prod.elasticbeanstalk.com"
2015/05/20 16:27:27 [crit] 1537#0: *20 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.51.94, server: _, request: "POST /get HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/get", host: "securities-api-prod.elasticbeanstalk.com"
2015/05/20 16:27:32 [crit] 1537#0: *20 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.51.94, server: _, request: "POST /get HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/get", host: "securities-api-prod.elasticbeanstalk.com"
2015/05/20 16:28:53 [crit] 1537#0: *52 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.51.94, server: _, request: "GET /datapoint?tickers=AAPL&datapoints=Ratings HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/datapoint?tickers=AAPL&datapoints=Ratings", host: "securities-api-prod.elasticbeanstalk.com"
2015/05/20 16:30:47 [crit] 1537#0: *69 connect() to unix:///var/run/puma/my_app.sock failed (2: No such file or directory) while connecting to upstream, client: 172.31.51.94, server: _, request: "POST /get HTTP/1.1", upstream: "http://unix:///var/run/puma/my_app.sock:/get", host: "securities-api-prod.elasticbeanstalk.com"
I think puma is running -
[ec2-user#ip-172-31-44-135 nginx]$ ps aux | grep puma
root 23299 1.0 0.2 53008 1428 ? Ss 16:38 0:00 su -s /bin/bash -c puma -C /opt/elasticbeanstalk/support/conf/pumaconf.rb webapp
webapp 23314 0.0 2.4 75764 14604 ? Rsl 16:38 0:00 /opt/rubies/ruby-2.1.4/bin/ruby /opt/rubies/ruby-2.1.4/bin/puma -C /opt/elasticbeanstalk/support/conf/pumaconf.rb
ec2-user 23317 0.0 0.1 110284 844 pts/0 S+ 16:38 0:00 grep puma
pumaconf.rb
directory '/var/app/current'
threads 8, 32
workers %x(grep -c processor /proc/cpuinfo)
bind 'unix:///var/run/puma/my_app.sock'
stdout_redirect '/var/log/puma/puma.log', '/var/log/puma/puma.log', true
daemonize false
Anyone know what's going on?
I faced the same error and adding these two lines to config/puma.rb fixed the problem:
bind "unix:///var/run/puma/my_app.sock"
pidfile "/var/run/puma/my_app.sock"
After this step I faced another error to add ENV values, which is very straight forward.
Hope this helps.
Sorry - Super late to the game. But I found this entry when searching for the same error message and could solve it differently.
I am trying to let a Rails 6.1.1 (currently latest and greatest) application run on AWS Beanstalk (Puma with Ruby 2.6), where I got the same error as described in this post: Nginx throwing above connection error / Puma seems not reactive (no entries in puma logs) / When SSHing into the machine it shows Puma is running (initctl status puma -> shows process ID).
After some days of research (and trying all above answers without success) - I switched to Passenger - but still same result. Researching the same problem for Passenger showed the proper solution:
Do not use the gem of the application server in production when you want to use the already provided application server of the AWS platform branch ("Puma with Ruby 2.6..." or "Passenger with Ruby 2.6...") (it somehow collides with the already installed version). Therefore in your Gemfile:
gem 'puma', '~> 5.0', groups: [:development, :test]
respectively:
gem "passenger", ">= 5.3.2", require: "phusion_passenger/rack_handler", groups: [:development, :test]
or use the AWS Beanstalk platform without already installed application server ("Ruby 2.6 running on...)
PS: Seems to be only a solution for higher Rails versions (my old Rails 5.1 app runs perfectly on the "Puma with Ruby 2.6" platform even with the gem "normally" installed.
In my case, this was happening because of a Gem loading error. I was able to reproduce the error by connecting to the server with eb ssh and running pumactl start.
For me, the conflict was with the nio4r gem, although for others it was the puma gem. Setting it to the same version as the Elastic Beanstalk platform's gem fixed the error.
More details here:
https://stackoverflow.com/a/67110462/1852005
and here:
https://forums.aws.amazon.com/thread.jspa?messageID=957426
Changing config.force_ssl = true to false fixed the issue for me.
This line in your logfile means, that you've configured your nginx incorrectly.
upstream: "http://unix:///var/run/puma/my_app.sock:/get"
Actually nginx tries to use unix domain socket as HTTP url.
Can't say exactly what is wrong at your nginx config without it, but you should have something like this:
upstream backend {
server backend1.example.com weight=5;
server 127.0.0.1:8080 max_fails=3 fail_timeout=30s;
server unix:/tmp/backend3;
server backup1.example.com backup;
}
You should also check out this nginx doc http://nginx.org/en/docs/http/ngx_http_upstream_module.html
The AWS doc mandates that puma gem be installed as part of your application.
Add the following to your Gemfile as appropriate:
group :production do
gem 'puma'
end
The error unix:///var/run/puma/my_app.sock failed (2: No such file or directory) will also occur if Elastic Beanstalk cannot detect your Rails application root for whatever reason. In my case, the issue was simply that I was zipping up my Rails application contents inside a subfolder. Instead of creating the zip in the same directory as app, bin, config, db, etc., the application content were being created inside a subfolder called <app_name>.
The only hint was in the Elastic Beanstalk Events section where it had an Informational notification (not warning or error level) which said Gemfile not detected. Custom gems will not be included.
Hope this helps others; took many deployments to figure out this simple problem.
Given you are using Elastic Beanstalk, it starts up puma by running a command very similar to this one (as you can see by running ps):
sudo su -s /bin/bash -c "puma -C /opt/elasticbeanstalk/support/conf/pumaconf.rb webapp"
Try running that command in your eb ssh console. If you get an error like the one in this post:
config/puma.rb:23:in `_load_from': undefined local variable or method `daemonize' for #<Puma::DSL:0x000055596fadb448> (NameError)
you have probably declared Puma 5.x in your Gemfile. The solution would be to use puma < 5.0, as it removed the daemonize method. Note that the configuration Elastic Beanstalk uses is not config/puma.rb but /opt/elasticbeanstalk/support/conf/pumaconf.rb
Add a Procfile
with this content:
web: puma -C /opt/elasticbeanstalk/config/private/pumaconf.rb
Reference:
https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/ruby-platform-procfile.html
Check if you're missing any newly added ENV variables.
To add new Environment variables follow:
Elastic Beanstalk >
Environments >
<Your Environment> >
Configuration >
Software >
EDIT
Under Environment properties Add the new ENV variables
I have the following error in my nginx logs:
2014/09/23 01:56:40 [info] 18250#0: *38 epoll_wait() reported that
client prematurely closed connection, so upstream connection is closed
too while sending request to upstream, client: 49.128.46.126, server:
tm-google-dev.gumi.sg, request: "GET /versions/ping HTTP/1.1",
upstream: "passenger:/tmp/passenger.1.0.18229/generation-0/request:",
host: "tm-google-dev.gumi.sg:443"
It is not exactly an error 500 or 403, so i can't make out what is it trying to say. Please help. Thanks. The app is a rails app.
I am trying to configure nginx for a rails app on DigitalOcean. When I open my IP, I see the BAD GATEWAY (502) message.
In logs is
2014/06/03 09:58:57 [crit] 28612#0: *1 connect() to unix:/tmp/unicorn.myproject-staging.sock failed (2: No such file or directory) while connecting to upstream, client: 81.161.64.26, server: , request: "GET / HTTP/1.1", upstream: "http://unix:/tmp/unicorn.myproject-staging.sock:/500.html", host: "my_ip"
What does error message means and how could I fix it?
Thank you
EDIT:
The application is deployed here:
/home/deployer/apps/myproject-staging
Please check to ensure your Unicorn is running and the path to the unix socket file is right.
It's really tricky in our production environment( Red Hat Enterprise Linux Server release 5.4 (Tikanga) and curl 7.15.5 (x86_64-redhat-linux-gnu) libcurl/7.15.5 OpenSSL/0.9.8b zlib/1.2.3 libidn/0.6.5) that the POST request become to GET request automatically. Below is the relative log provide by nginx.
cache: [GET /login] miss
url->http://localhost:8080/login
About to connect() to localhost port 8080
Expire at 1339680839 / 265363 (300000ms)
Trying 127.0.0.1... * connected
Connected to localhost (127.0.0.1) port 8080 /usr/local/lib/ruby/gems/1.8/gems/typhoeus-0.4.0/lib/typhoeus/multi.rb:141: [BUG] Segmentation fault ruby 1.8.7 (2012-02-08 patchlevel 358) [x86_64-linux]
2012/06/14 21:28:59 [error] 29829#0: *6031 upstream prematurely closed connection while reading response header from upstream, client: 127.0.0.1, server: localhost, request: "POST /login HTTP/1.1", upstream: "passenger:unix:/passenger_helper_server:", host: "127.0.0.1:8081", referrer: "http://127.0.0.1:8081/login"
In fact, we send a POST request to login, but it convert to GET actually.
In our development environment( which is Ubuntu12.04 and curl 7.22.0 (x86_64-pc-linux-gnu) libcurl/7.22.0 OpenSSL/1.0.1 zlib/1.2.3.4 libidn/1.23 librtmp/2.3), everything goes ok.
Who can explain it?
After I upgraded the curl from 7.15 to 7.22. The problem has been solved.
My app generates some image data on the fly and sends it back to the browser with send_data some_huge_blob, :type => 'image/png'. This works well enough in development mode, but in production with nginx/passenger in the mix it appears as if sometimes passenger just crashes. Here is the debug output in my nginx log
[ pid=596 thr=140172782794496 file=ext/common/ApplicationPool/Pool.h:1162 time=2011-07-25 23:15:14.965 ]: Exception occurred while connecting to checked out process 1428: Cannot connect to Unix socket '/tmp/passenger.1.0.589/generation-0/backends/ruby.kJRjXYuZteKoogZIufN8a2cDPdpbIlYmIr1hh3G9UV7GhKDB4pqZ5y0jR': Connection refused (111)
[ pid=596 thr=140172782794496 file=ext/common/ApplicationPool/Pool.h:685 time=2011-07-25 23:15:14.965 ]: Detaching process 1428
[ pid=596 thr=140172782794496 file=ext/common/ApplicationPool/../Process.h:138 time=2011-07-25 23:15:14.969 ]: Application process 1428 (0x2676ee0): destroyed.
[ pid=1405 thr=70178806733240 file=abstract_request_handler.rb:466 time=2011-07-25 23:15:14.982 ]: Accepting new request on main socket
2011/07/25 23:15:16 [error] 642#0: *96 upstream prematurely closed connection while reading response header from upstream, client: 173.8.216.57, server: app.somedomain.com, request: "GET /projects/4e2dee4c106a821bf2000008/revisions/1/assets/Layout2.psd/preview HTTP/1.1", upstream: "passenger:unix:/passenger_helper_server:", host: "app.somedomain.com"
Note that there is nothing in my production.log file that indicates the request even makes it to the app!
Any ideas? Or ideas as to how to debug this further? The connection refused bit is interesting...
For what it's worth, this is an Ubuntu image on a micro instance in AWS.