I'm running Rails 2.3.3 application which is deployed with passenger/mod_rails with ruby-enterprise-1.8.6-20090610 and apache httpd.
The problem is that whenever I deploy our application, hundreds of httpd processes start dying. I'm getting this error:
[notice] child pid NNNNN exit signal Segmentation fault(11)
After a short period of time 10-20min. those errors pass off.
This problem started after migrating our database to a separate and dedicated machine. So I think it could be a problem of the mysql-db connection pools and management, however I can not define it.
Does anyone could help me with this problem or just give me a clue how to debug it deeper. Thank you in advance.
Start by enabling core dumps on your server.
Then run it to get a core file to get a backtrace and get an initial idea of where the server is core dumping.
I'm going throught the same problem at the moment. Not with Rails though.
HTH
Related
I have a Clojure web application I am running on a free plan on Heroku. The app has been working well for more or less one month, but in the last 3 days the logs have been showing up this error and the app is not working.
I am not able to reproduce this error locally, where everything starts up fine.
I tried restarting the app several times, deploying a new instance and fiddling around with JAVA_OPTS and JAVA_TOOL_OPTIONS, but nothing has helped and I am stuck with the same errors.
The whole code for the application is here. Has anyone experience with this error and possible ways to work around it?
As the error message says, the app consumes more memory than Heroku allocated for it. Heroku allows you to look at app metrics, graph from there might be useful for identifying the cause.
Heroku has a special guide for memory related problems in JVM application java-memory-issues. You might find it useful.
Try setting your max heap size lower by running
$ heroku config:set JAVA_TOOL_OPTIONS="-Xmx256m"
I found the culprit of the exceeded memory.
The command run by Heroku on startup was not using the jar file. What I had before was web: lein ring server-headless and I changed it to execute the jar web: java -jar target/<app-name>-standalone.jar in the Procfile.
Since I am using ring, I also have Heroku run lein ring uberjar instead of lein uberjar before startup: this is as easy as setting LEIN_BUILD_TASK='ring uberjar' as a global config var in Heroku.
I am using Neo4j version 1.8.1.
When I stopped the Neo4j service and started it up again, it will not respond at all.
I am unable to connect to the database using the Neo4j browser even though the service is running.
Any ideas/suggestions on how to troubleshoot this issue? It was working fine until I restarted the service.
Thanks
If you log Java's garbage collection to a file, you can tail this file to get a clue how busy neo4j is during startup, especially, as pointed out, following an unclean shutdown.
The Java startup option you need is:
-Xloggc:neo4j-gc.log -verbose:gc -XX:+PrintGCDetails -XX:+PrintGCDateStamps
My RoR app on windows (xp as well as server 2008) stops responding after maybe a couple of days.
Using guidance from stackoverflow as well, I set up services to run the server automatically at startup. I tried different products: webrick, mongrel with mongrel_service, thin – both standalone or with an apache proxy. They start ok and work fine, but after a certain amount of time (rather than load) it stops responding.
I don't see anything in the logs either, even if I have the server run from the command line I don't get any error messages there nor does the process end, the server just does not answer anymore. Has anybody else come across this?
Andreas
There are so many issues with windows that our whole company switched to ubuntu. I would not advise this as a production solution.
I have managed to get mongrel_cluster running on my Windows server (v 1.0.5) ok, but I am having trouble trying to get it to spawning all of the (4) mongrel instances.
It launches the first one ok, but then does no more. That one instance serves just fine, but the others are not launched. I assume that this is becasue of: "** WARNING: Win32 does not support daemon mode." so its not forking the processes for each instance.
Is there a work arround for this? Even if I have to start 4 'mongrel_rails cluster::start' manually, thats ok (assumming that they start the correct node of the cluster), or am I better off just launching 4 normal (and speerate) mongrel instances?
I have 2 primary questions:
Is there a way to solve this?
What cool features will I loose by not using mongrel_cluster?
It turns out that you simply CANNOT do what I want to do. Instead you need to install the mongrel_service gem and run seperate mongrel instances as system services and balance on those.
I'm having ruby instances from mod_rails go "rogue" -- these processes are no longer listed in passenger-status and utilize 100% cpu.
Other than installing god/monit to kill the instance, can anyone give me some advice on how to prevent this? I haven't been able to find anything in the logs that helps.
If you're using Linux, you can install the "strace" utility to see what the Ruby process is doing that's consuming all the CPU. That will give you a good low-level view. It should be available in your package manager. Then you can:
$ sudo strace -p 22710
Process 22710 attached - interrupt to quit
...lots of stuff...
(press Ctrl+C)
Then, if you want to stop the process in the middle and dump a stack trace, you can follow the guide on using GDB in Ruby at http://eigenclass.org/hiki.rb?ruby+live+process+introspection, specifically doing:
gdb --pid=(ruby process)
session-ruby
stdout_redirect
(in other terminal) tail -f /tmp/ruby_debug.(pid)
eval "caller"
You can also use the ruby-debug Gem to remotely connect to debug sockets you open up, described in http://duckpunching.com/passenger-mod_rails-for-development-now-with-debugger
There also seems to be a project on Github concerned with debugging Passenger instances that looks interesting, but the documentation is lacking:
http://github.com/ddollar/socket-debugger/tree/master
I had a ruby process related to Phusion Passenger, which consumed lots of CPU, even though it should have been idle.
The problem went away after I ran
date -s "`date`"
as suggested in this thread. (That was on Debian Squeeze)
Apparently, the problem was related to a leap second, and could affect many other applications like MySQL, Java, etc. More info in this thread on lklm.
We saw something similar to this with very long running SQL queries.
MySQL would kill the queries because they exceeded the long running limit and the thread never realized that the query was dead.
You may want to check the database logs.
This is a recurring issue with passenger. I've seen this problem many times helping people that ran ruby on rails with passenger. I don't have a fix but you might want to try this http://www.modrails.com/documentation/Users%20guide%20Apache.html#debugging_frozen