I recently restarted POW. On reloading the Rails app I'm working on, it hung.
Checking Rails' logs I get the following repeating every 5-20 seconds:
Connecting to database specified by database.yml
I can interact with the Postgres db without issue through the Rails console, so there is no problem there. I haven't changed anything in my database.yml for weeks, so I don't think the problem is there.
What might be the problem and how can I debug it?
Turns out it was caused by POW
I had set '$powder always_restart` which was causing some sort of loop.
I reset this using `$ powder no_restarts'
Change your production database temporary to your development settings.
Then run in your terminal:
rails console production
Most of the times it is not about your database, but probably a syntax error. Anyways you can debug it like this.
Related
I'm not the most experienced of developers so please excuse me if this is a stupid question.
I am trying to deploy a Sinatra app that uses PostgreSQL as it's database. My application works locally but then when I try to deploy to Heroku, I get this error:
Screenshot of Terminal after I try to push to heroku master
I have no idea what is going on. I don't think I'm requiring anything incorrectly b/c I would think that it wouldn't work locally if that were the case. I tried running the bit of code that says "bundle exec rake -P" but it didn't do anything as far as I know. I thought maybe I'd have to create a database and migrate it since I did that with a Rails app I was working on, but this is in Sinatra, so not sure if you have to do something entirely different.
It apparently deployed to Heroku, but when I run heroku open, it doesn't work. I just get a page that says to try again later and look at the log.
Any ideas on how to solve this issue?
I'm having a real tough time diagnosing a 500 error from my application running in production. I've had it working before, but after re-deploying via Capastrano I am unable to get it going.
Here are the facts:
The server is setup with nginx + passenger, and I'm using
PostgreSQL.
Static assets are working properly, as in I'm able to access them just fine in a browser.
I can access the rails console via RAILS_ENV=production bundle exec rails console and perform Active Record actions (like
retrieving data from the db).
Within console, I can run app.get("/"), which returns a 500 error as well (after first showing the query that was run to load
the model).
The production.log file is never written to. I've set permissions 777 on it just for the hell of it. I've also set the log level to
:debug with nothing to show for it.
The nginx log (which passenger also uses) shows no indication of errors, it just notifies about cache misses.
Because nothing of use is being logged, I have no idea what to do here. I've tried setting full permission on the entire app directory with no help. Restarted the server multiple times, nothing. The database is there and rails can clearly communicate with it. I'm not sure what I did to get it to run the first time around. I just don't know why rails isn't outputting anything to the log.
Okay, I figured this out. The app ran fine in development mode, so I knew something production-specific was screwing it up. I went into config/environments/production.rb and changes these settings:
# Full error reports are disabled and caching is turned on
config.consider_all_requests_local = false # changed from true
config.action_controller.perform_caching = true # changed from false
And then after restarting passenger, rails showed me the error w/ stacktrace. Turns out I was forgetting to precompile the asset pipeline!
Things to check
1) Are you sure you are running in production environment?
Check to see if any entries are in the development.log file
2) Set up your app to email you when a 500 error occurs with a full stack trace. I use the Exception Notifier gem but there are plenty of other solutions for this.
3) When checking your app in the console are you sure you are starting the console in production mode? It is possible that the app is not starting up at all and you just forgot to set the production param thereby thinking that the app runs fine when it doesn't.
4) Are you getting an nginx 500 error or the Rails 500 error? If nginx then it is likely that your app is not starting at all and highly unlikely that you will get any rails error in your log file. The assets are static files and navigating to them proves nothing other than that they exist.
5) Are you sure you are checking the right folder on the server? Sounds really stupid but capistrano could be deploying the app to a folder that is different to the folder that nginx is looking for for your app so double check both the folder capistrano is deploying to and the folder that nginx is looking for are the same.
Just a suggestion, I would use puma rather than passenger. It's awesome with nginx.
My problem is passenger's log file (error.log) has nothing. Then it's a rotation log issue. Run
passenger-config reopen-logs
solved my problem. More.
Have you tried running in development mode to see if the error reports itself?
I have a Rails app installed on a Slicehost server running Apache 2 and Ubuntu LTC 10.04. Things have worked beautifully up until now: I edit a file, do a quick mongrel_rails cluster::restart, and the changes are reflected in production. However, suddenly this process has broken down.
For example, I have a class called Master located in /lib/master.rb. I added a new method to this class that simply runs puts "it works!", then restarted the mongrel cluster. Looking at the production logs, the server throws an error and thinks this method doesn't exist. When I go to the console using ruby script/console production, however, I can use this new method perfectly. I even tried deleting the file containing entire Master class. Once again, the production thought it was still there, but the production console correctly recognized it was missing.
Any ideas? How can the production environment detect a class that doesn't even exist anymore?
Funny, I spend 2 hours debugging this, then post to StackOverflow and figure it out in 20 minutes.
The problem is that I needed to also restart my background jobs as well. They were running the old version of the classes stored in /lib. It's interesting that this problem has never snagged me before.
Here's what I've determined:
Delta indexing works fine in development
Delta indexing does not work when I push to the production server, and no action is logged in searchd.log
I'm running Phusion Passenger, and, as recommended in the basic troubleshooting guide, have confirmed that:
www-data has permission to run indexing rake tasks (ran them from command line manually)
the path to indexer and searchd are correct (/usr/local/bin)
there are no errors in production.log
What on earth could I possibly be missing? I'm running Ruby Enterprise 1.8.6, Rails 2.3.4, Sphinx 0.9.8.1, and Thinking Sphinx 1.2.11.
Thanks!
Last night as I slept it hit me. Unsurprisingly, it was a stupid issue involving bad configuration, though I am rather surprised that it produced the results it did. I guess I don't know much about Thinking Sphinx internals.
Recently I migrated servers. sphinx.yml looked like this:
production:
bin_path: '/usr/local/bin'
host: mysql.mysite.com
On the new server, MySQL was just a local service, but I had forgotten to remove that line. Interestingly, manual rake reindexing still worked just fine. I'm intrigued that Thinking Sphinx didn't throw an error when trying to reload the deltas, since mysql.mysite.com no longer exists, even though that was clearly the source of the issue.
Thanks for all your help, and sorry to have brought up such a silly problem.
Are there any clues in Apache/Nginx's error log?
Here's the next troubleshooting step I would take. Open up the file for the delta indexing strategy you are using (presumably lib/thinking_sphinx/deltas/default_delta.rb). Find the line where it actually generates the indexing command. In mine (v1.1.6) it's line 20:
output = `#{config.bin_path}indexer --config #{config.config_file} #{rotate} #{delta_index_name model}`
Change that so you can log the command itself, and maybe log the output as well:
command = `#{config.bin_path}indexer --config #{config.config_file} #{rotate} #{delta_index_name model}`
RAILS_DEFAULT_LOGGER.info(command)
output = `#{command}`
RAILS_DEFAULT_LOGGER.info(output)
Deploy that to production and tail the log while modifying a delta-indexed model. Hopefully that will actually show you the problem. Of course maybe the problem is elsewhere in the code and you won't even get to this point, but this is where I would start.
I was having this problem and found the "bin_path" solution mentioned above. When it didn't seem to work, it took me a while to realize that I'd pasted in the example code for "production" when I was testing on "stagiing" environment. Problem solved!
This was after making sure that the rake tasks that config, index, and start sphinx are all running as the same user as your passenger instance. If you log into the server as root to run these tasks, they will work in the console but not via passenger.
I had the same problem. Works on the command line, not inthe app.
Turns out that we still had a slave database that we were using for the indexing, but the slave wasn't getting updated.
As above, same issues were faced our side on two machine. The first one we had an issue with mysql which showed in apache2 log. Only seemed to affect local OSX machine..
Second time when we deployed to Ubuntu server, we had same issue. Rails c production was fine, no errors, bla bla bla.
Ended up being a permissions problem. Couldn't figure this out as there were no problems starting, although I guess I was doing so as root.
Using capistrano and passenger, we did this:
Create a passenger user and added to www-data group
Changed user in deploy.rb to be passenger
Manually changed all the /current files to be owned by above
Logged in as passenger user.
Ran rake ts:rebuild RAILS_ENV="production"
Worked a treat for us...
Good luck
I'm working on a Rails site that connects to an Oracle database, and though I didn't build the site from scratch, I'm doing maintenance work. The site uses the delayed_jobs plugin to handle some background tasks and I'd like to be able to run rake jobs:work on the development server to periodically process all jobs in the queue (due to the server's configuration, running a daemonized version of the script on the development server isn't an option). However, whenever I try running the command, I get the following classic Oracle error:
error while trying to retrieve text for error ora-12154
Ordinarily, I'd think this would be an authentication problem (e.g. incorrect credentials in database.yml), but the site is up and running fine (and doing lots of database stuff). I've tried adding RAILS_ENV=production as a parameter to rake to force it to run in in the production environment, but got the same error (there are two separate rails installations for the production and development versions of the site, and I've set the "development" and "production" credentials in development's db config file to be identical).
I'm not sure what could be causing this error, and I don't have a ton of experience using Oracle with rails. Any suggestions?
Thanks a lot!
Justin
EDIT (10/26/09): Still can't figure out what's causing the problem here. The app continues to run (and talk to the database) without a problem, but rake keeps throwing DB errors. So does script/console, which shows a prompt but first complains with the same Oracle error message. I'm going to keep looking, but I'm running out of ideas...
EDIT(10/26/09, later): Following the advice of this link, I set both ORACLE_HOME and TNS_ADMIN to point to the directory where tnsnames.ora lives. Just setting ORACLE_HOME had no obvious effect, but now that TNS_ADMIN points to the right place, I've started getting segmentation faults whenever I try to open the console or run rake:
/usr/local/lib/ruby/site_ruby/1.8/oci8.rb:184: [BUG] Segmentation Fault
and get booted unceremoniously back to the prompt. Any further ideas?
Finally got it...turns out that ORACLE_HOME wasn't being correctly set as an environment variable for my user account. Now rake, script/console, etc. are humming along happily.
The oracle error says the following:
ORA-12154 is generated by the oracle network layer. TNS error message is thrown during the logon process to a database. This error indicates that the communication software in Oracle ( SQL *Net or Net8 ) did not recognize the host/service name specified in the connection parameters. This error almost always indicates a misconfiguration of the oracle tns entries.
Can you connect to your oracle instance using sqlplus or another db tool?
It is odd that the app runs fine though.
Is there an $ORACLE_SID laying around somewhere that could be pointing to a db that doesn't exist?
IN sql server I would probaly run profuiler to see what is actually being sent vice what I think I have set up. I'm sure Oracle aslo has some type of profiling utility. I would try that and see, you may find it isn't using the credentials you tink it is.
Well, as Mike mentioned, ora-12145 means TNS couldn't resolve the database identifier (TNS is Oracle's name-to-database mapping, sorta-kinda like DNS). If you can find your tnsnames.ora file, you can see what databases are configured there and compare that to the database.yml file. The fact that it works as a delayed job but not from the command line is a bit odd, though, and makes me think that perhaps there are some environment variables being set in one context that aren't in the other.
If neither of those pan out, there's a long list of troubleshooting suggestions at http://ora-12154.ora-code.com that are specific to that error code.