Using carrier wave to upload a pdf document via CarrierWave. Then using RMagick to save the first page of the pdf as a thumbnail.jpg. I have this working correctly in my development environment (Mac OS X 10.9), however its failing in production (Nginx + Unicorn on Ubuntu 12.04 LTS).
It appears that the upload completes successfully, based on watching the upload percentage in the browser status bar. However, something errors in the processing, and I get Rails error page. The problem is, there is nothing in my production.log.
I get the following in unicorn.log:
I, [2013-12-12T01:22:55.232431 #2935] INFO -- : worker=1 ready
I, [2013-12-12T01:22:55.246992 #2932] INFO -- : worker=0 ready
E, [2013-12-12T01:26:00.666715 #2929] ERROR -- : worker=1 PID:2935 timeout (31s > 30s), killing
E, [2013-12-12T01:26:00.676250 #2929] ERROR -- : reaped #<Process::Status: pid 2935 SIGKILL (signal 9)> worker=1
I, [2013-12-12T01:26:00.676382 #2929] INFO -- : worker=1 spawning...
I, [2013-12-12T01:26:00.681087 #3502] INFO -- : worker=1 spawned pid=3502
I, [2013-12-12T01:26:00.681399 #3502] INFO -- : Refreshing Gem list
SECURITY WARNING: No secret option provided to Rack::Session::Cookie.
This poses a security threat. It is strongly recommended that you
provide a secret to prevent exploits that may be possible from crafted
cookies. This will not be supported in future versions of Rack, and
future versions will even invalidate your existing user cookies.
Called from: /home/deployer/apps/______/shared/bundle/ruby/1.9.1/gems/actionpack-3.2.9/lib/action_dispatch/middleware/session/abstract_store.rb:28:in `initialize'.
And this in my nginx error.log
2013/12/12 01:26:00 [error] 2597#0: *9 upstream prematurely closed connection while reading response header from upstream, client: ###.###.###.###, server: ___.org, request: "POST /admin/explores HTTP/1.1", upstream: "http://unix:/tmp/unicorn.sock:/admin/explores", host: "___.org", referrer: "https://___.org/admin/explores/new"
Not sure how to get a better error description of what's going on. Would appreciate any help.
Thanks!
Ruby 1.9.3 + Rails 3.2.9
The line E, [2013-12-12T01:26:00.666715 #2929] ERROR -- : worker=1 PID:2935 timeout (31s > 30s), killing from your unicorn.log tells you the problem: you have a timeout. Increase unicorn's timeout to 60 seconds by setting timeout 60 in your unicorn.rb. More information: http://unicorn.bogomips.org/Unicorn/Configurator.html#method-i-timeout
Related
I'm new to Ruby on Rails and helping on an existing project. I ran rails server, and received the following output
=> Booting Unicorn
=> Rails 4.2.5.2 application starting in development on http://localhost:3000
=> Run `rails server -h` for more startup options
=> Ctrl-C to shutdown server
I, [2016-12-05T13:44:05.166061 #15228] INFO -- : listening on addr=0.0.0.0:8080 fd=11
I, [2016-12-05T13:44:05.185700 #15228] INFO -- : master process ready
I, [2016-12-05T13:44:05.187404 #15242] INFO -- : worker=0 ready
I, [2016-12-05T13:44:05.187827 #15243] INFO -- : worker=1 ready
I, [2016-12-05T13:44:05.188412 #15244] INFO -- : worker=2 ready
^CI, [2016-12-05T13:45:11.233928 #15228] INFO -- : reaped #<Process::Status: pid 15244 exit 0> worker=2
I, [2016-12-05T13:45:11.234060 #15228] INFO -- : reaped #<Process::Status: pid 15243 exit 0> worker=1
I, [2016-12-05T13:45:11.234227 #15228] INFO -- : reaped #<Process::Status: pid 15242 exit 0> worker=0
Which seems to indicate that the server is up and running. However, I am unable to connect to http://localhost:3000, it gives me an error connection refused. Does anybody know how I can go about debugging this?
With rails 4.2.0, the server binds to localhost by default, instead of 0.0.0.0. When working with a rails in a virtual box, accessing the server from the host computer, the binding address needs to be 0.0.0.0
Start rails server with -b0.0.0.0 to make the rails server accessible from the host computer/browser.
http://guides.rubyonrails.org/4_2_release_notes.html#default-host-for-rails-server https://github.com/samuelkadolph/unicorn-rails/issues/12#issuecomment-60875268
or try passing different port
rails s -p 3001
Try checking your hosts file.
Linux /etc/hosts
Windows c:\Windows\System32\drivers\etc\hosts
and editing the localhosts to serve 127.0.0.1. or move it so it is last. Rails will take the last mention of localhost and use it.
Or you can go even deper and try using telnet localhost 3000 to tell if the port is unavailable at the TCP level.
My hosts file looks something like
127.0.0.1 localhost
127.0.1.1 myPC
# The following lines are desirable for IPv6 capable hosts
::1 ip6-localhost ip6-loopback
I'm making an expensive call to my Heroku Rails server. After 13-15 seconds the console in my browser reports a Service Unavailable 503 error. However when I check my heroku logs, it reports:
Completed 200 OK in 45592ms (Views: 220.3ms | ActiveRecord: 33457.5ms)
Other times the heroku logs report a memory exceeded quota. Here is an example of that below.
2015-06-11T15:17:20.238285+00:00 app[web.1]: Completed 200 OK in 81881ms (Views: 201.6ms | ActiveRecord: 18021.2ms)
2015-06-11T15:17:33.482930+00:00 heroku[web.1]: Process running mem=841M(164.4%)
2015-06-11T15:17:33.482930+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2015-06-11T15:17:53.147570+00:00 heroku[web.1]: Process running mem=841M(164.4%)
2015-06-11T15:17:53.147679+00:00 heroku[web.1]: Error R14 (Memory quota exceeded)
2015-06-11T15:17:59.751540+00:00 app[web.1]: E, [2015-06-11T15:17:59.695813 #3] ERROR -- : worker=2 PID:13 timeout (121s > 120s), killing
2015-06-11T15:17:59.916750+00:00 app[web.1]: E, [2015-06-11T15:17:59.906435 #3] ERROR -- : reaped #<Process::Status: pid 13 SIGKILL (signal 9)> worker=2
2015-06-11T15:18:02.487428+00:00 app[web.1]: I, [2015-06-11T15:18:02.427293 #16] INFO -- : worker=2 ready
Why is it reporting a Completed 200 when the console is reporting a 503?
there are 2 different things here:
your app
the heroku load balancer
In this case the load balancer see that the request is taking too long and sends you the 503. In the back the service processes the request and returns completes with a 200.
See:
https://devcenter.heroku.com/articles/limits
https://devcenter.heroku.com/articles/request-timeout
Heroku does not gives us much information for error-code = H10. Simple put, something is going wrong with your application code / configuration. To see what's going wrong, run heroku run rails console and you will be able to see error details which will be help you in resolving the error. No need to use logs. heroku run rails console is a big help
This happened with me when I used Node. I put a timeout for 30 seconds and I used to get HTTP:503. I then came to know that it's the timeout that caused the issue. I changed the timeout to <30 seconds and it is working fine. Maybe it's because of the loadbalancer that Heroku uses.
Thanks.
I'm developing an AngularJS (front-end) running in an Nginx web server that sends request to a Rails API (backend) running in a Unicorn application server.
I recognize I'm just a developer and I have no idea about servers administration, so I just put the servers and start them.
The application is working, however, Unicorn has a strange behaviour. When I start it I always get this error:
roberto#ubuntu:~/dev/scripts$ ./start_unicorn.sh
I, [2014-06-14T11:46:06.085834 #4258] INFO -- : Refreshing Gem list
I, [2014-06-14T11:46:11.591592 #4258] INFO -- : listening on addr=0.0.0.0:8080 fd=10
I, [2014-06-14T11:46:12.087321 #4258] INFO -- : master process ready
I, [2014-06-14T11:46:12.151320 #4263] INFO -- : worker=0 ready
I, [2014-06-14T11:46:12.150526 #4266] INFO -- : worker=1 ready
E, [2014-06-14T11:46:39.112668 #4258] ERROR -- : worker=0 PID:4263 timeout (16s > 15s), killing
E, [2014-06-14T11:46:39.112898 #4258] ERROR -- : worker=1 PID:4266 timeout (16s > 15s), killing
E, [2014-06-14T11:46:39.118081 #4258] ERROR -- : reaped #<Process::Status: pid 4263 SIGKILL (signal 9)> worker=0
E, [2014-06-14T11:46:39.118634 #4258] ERROR -- : worker=1 PID:4266 timeout (16s > 15s), killing
E, [2014-06-14T11:46:39.121820 #4258] ERROR -- : reaped #<Process::Status: pid 4266 SIGKILL (signal 9)> worker=1
I, [2014-06-14T11:46:39.172067 #4284] INFO -- : worker=1 ready
I, [2014-06-14T11:46:39.172620 #4281] INFO -- : worker=0 ready
It takes some seconds until it responds. And this happens continuosly.
I guess I'm missing some configuration but no idea...
If you need any more details, such as config files, just let me know
Do you have your assets precompiled ? are you in production ?
If not when you start your server and get your first request, rails will try to compile your assets which can take more than 15 seconds and reach the unicorn timeout.
In your start.sh you should have somewhere
export RAILS_ENVIRONEMENT=production
And during you deployment you should have :
rake assets:precompile
I've got a large 2.3 Rails app running on Unicorn. I'm using Unicorn, so that I can have zero downtime deployments. However, I've noticed that the first request after a restart is very slow.
First request:
Completed 304 Not Modified in 2771.8ms (ActiveRecord: 98.6ms)
Second request:
Completed 304 Not Modified in 94.4ms (ActiveRecord: 26.9ms)
I do have preload_app true and I am re-establishing the db-connection in the after-fork.
I have no idea how to explain the 2600ms divergence between these two values.
Does anyone have any thoughts? Really, what I am looking for are ways to debug this issue.
UPDATE
Here is my unicorn.log after a restart:
I, [2014-05-16T13:46:26.529305 #11637] INFO -- : executing ["/data/app/current/ey_bundler_binstubs/unicorn", "-E", "staging", "-c", "/data/app/shared/config/custom_unicorn.rb", "-D", "/data/app/current/config.ru", {12=>#<Kgio::UNIXServer:fd 12>}] (in /data/app/releases/20140516184210)
I, [2014-05-16T13:46:27.566115 #11637] INFO -- : inherited addr=/var/run/engineyard/unicorn_afar.sock fd=12
I, [2014-05-16T13:46:27.566551 #11637] INFO -- : Refreshing Gem list
I, [2014-05-16T13:47:13.036963 #8247] INFO -- : reaped #<Process::Status: pid 8681 exit 0> worker=3
I, [2014-05-16T13:47:14.093196 #8247] INFO -- : reaped #<Process::Status: pid 8670 exit 0> worker=2
I, [2014-05-16T13:47:14.100269 #12047] INFO -- : worker=0 ready
I, [2014-05-16T13:47:15.105249 #12063] INFO -- : worker=1 ready
I, [2014-05-16T13:47:15.114038 #8247] INFO -- : reaped #<Process::Status: pid 8655 exit 0> worker=1
I, [2014-05-16T13:47:15.957970 #8247] INFO -- : reaped #<Process::Status: pid 8638 exit 0> worker=0
I, [2014-05-16T13:47:15.958159 #8247] INFO -- : master complete
I, [2014-05-16T13:47:16.087761 #12082] INFO -- : worker=2 ready
I, [2014-05-16T13:47:16.876129 #11637] INFO -- : master process ready
I, [2014-05-16T13:47:17.102994 #12095] INFO -- : worker=3 ready
And here is the first request on my rails logs:
Started GET "/" for 70.XX.XXX.XXX at 2014-05-16 13:47:51 -0700
Processing by HomeController#index as HTML
(1.1ms) SELECT ..... <regular controller/ActiveRecord queries>
Completed 304 Not Modified in 2724.8ms (ActiveRecord: 98.9ms)
First request to Rails app is very slow may be relevant.
Maybe there is a dependency that is loading / running on the first page load?
Some ideas:
Check the rails log to see if there's anything funky going on
Is this just happening for Unicorn or with other servers too?
Add log statements with time stamps to get a sense for what part of the app is taking a long time
Try using ruby prof
I'm running a Rails 4 app on a Digital Ocean VPS and I'm getting an Nginx 504 gateway time out error on a particular HTTP request. All requests succeed, but this particular one does not. I am running in production mode and there are no errors or exceptions in my Rails log.
The Nginx log looks like this:
2014/04/23 05:08:22 [error] 6946#0: *109805 upstream timed out
(110: Connection timed out) while reading response header from upstream, client:
174.126.176.107, server: new.pilotpro.com, request:
"GET /dropbox/sync?right_now=1398115006.949 HTTP/1.1", upstream:
"http://127.0.0.1:8080/dropbox/sync?right_now=1398115006.949",
host: "new.pilotpro.com"
And the Unicorn log looks like this:
E, [2014-04-23T04:52:06.285058 #5424] ERROR -- : worker=1 PID:5429 timeout (61s > 60s), killing
E, [2014-04-23T04:52:06.297947 #5424] ERROR -- : reaped #<Process::Status: pid 5429 SIGKILL (signal 9)> worker=1
I, [2014-04-23T04:52:06.298207 #5424] INFO -- : worker=1 spawning...
I, [2014-04-23T04:52:06.299481 #5457] INFO -- : worker=1 spawned pid=5457
As far as I can tell nothing is wrong other than the fact that the request is timing out. Is this simply a matter of increasing Nginx's timeout limits?
I'd appreciate any advice. Thanks!