I am using Rails 4.2 and Ruby 2.2.0. Recently, I have noticed that my app will hang for long periods of time when a certain worker is working. This is an app that has been under development for several years and I have never seen this sort of hanging behavior before (nor am I unfortunately completely aware of what changes I've made since it was last not hanging).
What I mean by "hang" is: No pages will load (no controllers can be reached, and even the default homepage which is basically a static page won't load). None of my logs will update. Standard output is not being written to. The application also has many background worker processes (managed by Resque), which also appear to be hung.
I have looked at top and netstat and various other Unix utilities and I don't see anything alarming there, either.
Finally, I'm not sure if it's worth mentioning but this application is running via Foreman on an EC2 instance.
EDIT: I don't believe that it is query related, because nothing abnormal shows up on top. Also, I'm a little inexperienced with rails, but each job has its own entry in the process table, if that helps.
Related
I have a weird issue that just started popping up for our customers. The portal they've been using for years has started freezing on some of the pages that the user navigates to. I tried restarting the IIS Server, the site within and the Application Pool under which the site is site is running. No difference.
In Chrome Dev Tools I can see that it is always one of these three calls that take time to complete:
When it happens, one of those three calls will report that the request is not finished, like this:
When eventually the call completes, I can see that the Content Download took 3.8 minutes. Not sure whether it is relevant or not, but it is always 3.8 minutes:
Did anyone else encounter a similar situation? Is there a suggestion on how to figure out what is happening all of a sudden that triggers these type of behaviours?
TIA,
Ed
Edit: The resource that fails to load after 3.8 minutes always generates a net::ERR_CONNECTION_RESET error:
Edit2: Thanks to all of you trying to help. A little update: I was able to isolate to problem to an issue with the server not serving some of the files. either *.css or *.js. The setting is that of two identical servers placed behind a load balancer. Apparently, the load balancer software was recently updated and right after that we started having these issues. I am working closely with the IT department of our client, trying to figure out what is the impact of the newer version that seems to have triggered all this drama.
We have a web-app here that makes hundreds of calculations with money and other very specific numbers. It has been developed in Rails 3, which I have not much experience.
Recently, I have noticed that even in Digital Ocean server, where it runs for production, whenever an user requests to open some page for example, that does any query to get data from the database, everyone else gets stuck until the processing finishes.
It just looks like rails is running on a "single-threaded" mode, similar to running the same scenario with Tomcat in development mode in Eclipse.
Is there any trick to make it work "multi-threaded"?
I don't see any reason or even logic for it to run like this, there's no sense in making everyone wait until the request of another user finishes.
Thanks in advance to everyone!
Well, I've found the solution.
To enable Rails with WEBrick to work in a "multi-threaded" way, you'll have to simply go to the config file, and add :
# Enable threaded mode
config.threadsafe!
After that, restart the server and the trick is done. It "solved" my problem, but ended up creating a thousand others, cause the application wasn't developed thinking about a "threadsafe" scenario, which means, that everytime I try to run similar requests at the same time, the application mixes up everything in memory, crashing.
Well, one problem at once!
The original from this topic is solved.
I am using Rails 4.2 and Ruby 2.2.0. Recently, I have noticed that my app will hang for long periods of time after a certain controller has been invoked. This is an app that has been under development for several years and I have never seen this sort of hanging behavior before (nor am I unfortunately completely aware of what changes I've made since it was last not hanging).
What I mean by "hang" is: No pages will load (no controllers can be reached, and even the default homepage which is basically a static page won't load). None of my logs will update. Standard output is not being written to. The application also has many background worker processes (managed by Resque), which also appear to be hung. This behavior will persist for minutes at a time.
In order to narrow down the culprit, I removed everything from my app except for one controller. That controller looks like:
puts "Peforming"
respond_to do |format|
format.xml {
render :xml => "<Status>OK</Status>"
}
end
It will accept any POST from a client, and will respond with "OK," after printing a message. The app is hosted on a server, and I am hitting it with an XML POST as fast as I can from my local machine. After less than a minute of hitting it, the app will hang entirely, as described above.
I have also looked at netstat but see nothing out of the ordinary. There are lots of connections in the TIME_WAIT state, but I understand that that is fine. I also looked at top, and the machine is 90% idle, even when hanging.
Finally, this is running on Amazon EC2 in case that is relevant.
I'm new to RoR, and am building a few beginner's projects - but am unfamiliar with the error processing on RoR. For instance, I am working on this project right here, even though the site I found it through warned it had a few errors (if you know of a program that shortens urls based on a domain you own, that has NO errors, let me know and I'll use it). The problem I'm having is on the step:
rails server
It produces the result that the writer shows, then brings up a SECURITY WARNING. Underneath the security warning it has three timestamps with INFO WEBBrick and INFO ruby. However, an hour later, it's still here - and it hasn't brought the application back to the original location of the code (ie: C:\Location).
Is this an error? Or is it supposed to load? I ask because from the article it seems like I could just move on to the next step (after thirty minutes I hit ENTER just to see what happens, but no response), but - unless I open up a new command prompt - I don't see that happening.
Have you made any requests?
After the Rails server starts, it will sit "forever" waiting to service client requests (e.g., from a browser). Under Windows, that command prompt won't be useful until the server is shut down, e.g., with a Ctrl-C.
You can either open a new command window, as you've done, or shut the server down and use the same window. It's worth nothing that sometimes you'll need to restart the server, much of the time you won't. Figuring out when, and under what circumstances, is deterministic, but occasionally confusing.
Does anyone have any experience with Rail development page display erratically slowing down and speeding up (page appears 15-20 seconds after the console says the entire page has been rendered).
My development environment is rails 3.0.17 on Mac (Lion), WEBrick 1.3.1, ruby 1.9.2, with Postgres v11 (app supplied by Heroku) for the development database.
Recently, I've noticed some very long delays loading pages... 15 to 20 seconds at times, and the delay is completely unrelated to the complexity of the page, and a given page might load fast several times, then load slow. It might be pretty bad for several minutes, then go away for an hour.
And whether the page loads slow or fast, the rails log always shows the page has rendered fairly quickly... I'll see something like "Completed 200 OK in 486ms" but then the browser might say "waiting for localhost" for another 15 seconds before displaying the page.
Does not seem to depend on the browser (FF or Safari act the same).
Dos not sem to relate to updating the code base. I cna be testing some UI elements and everything is snappy, then suddenly several pages hang for 15-120 seconds.
Still happens even after I added the gem 'rails-dev-boost' to my Gemfile in development.
I also added to my development machine gem 'http_logger' to log calls to external http requests like S3, but don't see that as a factor (the delays often happen on pages that don't access external APIs).
My layouts have a couple of external .js dependencies such as http://static.twilio.com/libs/twiliojs/1.0/twilio.min.js, but I have added that to the bottom of my layout so I would (resumably) still see the page contents quickly if that were being delayed. (Besides, it is probably cached by my browser).
Of course I have tried rebooting the machine, verified activity monitor seems normal for CPU and memory usage. It does not seem to correlate when timemachine is doing its thing.
MORE INFO: per suggestion in comment from AKG below, I used Firebug's web console and most of the time the stylesheets load in a couple of milliseocnds, but when the slowdown occurs I'm seeing delays of 8-20 seconds for most of the stylesheets... which suggests webrick is failing to serve the pages timely?
It could be that buggy/slow middleware is executing after the render is called, but before the response is sent. I recently ran into this issue with pauses of more than 1 minute in some cases, and the culprit turned out to be the Bullet gem, which is designed to help us detect N+1 queries. I removed the middleware and the issue went away!
I got the same problem a few times. I switched to Mongrel. That automatically resolved the issue. However, I got this with Rails 2.3.8.