I have a Rails app installed on a Slicehost server running Apache 2 and Ubuntu LTC 10.04. Things have worked beautifully up until now: I edit a file, do a quick mongrel_rails cluster::restart, and the changes are reflected in production. However, suddenly this process has broken down.
For example, I have a class called Master located in /lib/master.rb. I added a new method to this class that simply runs puts "it works!", then restarted the mongrel cluster. Looking at the production logs, the server throws an error and thinks this method doesn't exist. When I go to the console using ruby script/console production, however, I can use this new method perfectly. I even tried deleting the file containing entire Master class. Once again, the production thought it was still there, but the production console correctly recognized it was missing.
Any ideas? How can the production environment detect a class that doesn't even exist anymore?
Funny, I spend 2 hours debugging this, then post to StackOverflow and figure it out in 20 minutes.
The problem is that I needed to also restart my background jobs as well. They were running the old version of the classes stored in /lib. It's interesting that this problem has never snagged me before.
Related
Have got new project with production server already running on passenger.
After doing normal deploy as described in 1 page docs left by old team (only cap prod deploy) we are having constant problems. First of all - code we pushed doesn't seem to be working. It should be adding new data on rake tasks to items in database. Physically code is there - in current folder. But it doesnt seems to be triggered.
I ve noticed that there were no rails or gem installed when I ve tried to do simple rails c via ssh. After installing everything and manually launching code with binding.pry added looks like code was triggered. But via normal scheduled rake task it was not.
It looks like passenger is running in daemon since there are no pid in tmp folder (usual for rails app).
1) Is there a chance restarting server will actually help - it was not restarted after deploy and I have no idea how to restart it without pid.
2)passenger-config restart-app returns actually 2 servers. Can they collide and prevent normal app work?(Update: servers are not same - single letter difference in the name)
Still passenger-config restart-app dont seem to restart server
Sorry for the wall of text by I ve spent 2 nights getting it to work and I still lost.
I am using DelayedJob for a long running task in my app. I have the job defined in a class MyJob saved in app/jobs/my_job.rb.
All was well, but I added some code to the file and restarted the server and the changes are not up. Before it was saving one field and now it should be saving two, and I added a logger.debug line to help me, but nothing is coming up in the logs and the models aren't being saved with the field.
This is in 'production' (though still using Thin webserver).
development works.
Since the folder is in the autoload path (at least if I am not misunderstood) I didn't think I needed to do anything special. But since it is not working, something must be off. Help? Let me know if you need more info.
Well, turns out I had a worker who was off the radar.
I run my server in a docker container (should have mentioned that) so even though I did a
RAILS_ENV=production script/delayed_job restart
in both the container and the host (the container has the app from the host as a volume), a worker I probably started some other time continued on. I saw it in the logs when I went back, so I did a
kill {pid}
and that solved my problem. So Flavio was right, I just had to kill the worker myself because the script wasn't picking it up.
I have a working RoR app that's running on Ubuntu Server, served by Nginx and Unicorn. It's using a local postgres database. I also think it's running on Sinatra instead of rails, but I'm too new to RoR to be able to say for certain.
I am attempting to create a development copy of the app to do some testing with. I'm able to do so using the source code, bundle install, and then running webbrick on my desktop. However, my developer wants to duplicate the environment that the actual production copy is running in. I cloned the server hosting the application, but when I browse to the server I get the dreaded "We're sorry, but something went wrong" error.
Here's what I've checked so for. I looked at the log files in the app/logs directory, and those haven't been updated in the last 6 days, so I suspect nothing relevant will be in them. If I run "service nginx status", it appears to be started. The database is running, and I can access it through the psql interface. Again, I'm pretty much an RoR newb... I was hoping one of you kind folks might give me some advice. Where should I look next? I'm sure it's something simple that I'm just overlooking. Thanks in advance!
In your config > environments > production.rb file change the consider_all_requests_local variable to true.
config.consider_all_requests_local = true
FYI, unicorn was not starting along with NGINX. I was able to script unicorn to start on boot, and this fixed my problem.
I wrote a demo HelloWorld Rails app and tested it with WEBrick (it doesn't even use a DB, it's just a controller which prints "hello world"). Then I tried to deploy it to a local Apache powered with Passenger. In fact this test is just to get Passenger working (it's my first deploy on Apache). Now I'm not even sure that Passenger works, but I don't get any error on the Apache side.
When I fire http://rails.test/ the browser shows the Rails 500 error page - so I assume that Passenger works. I want to investigate the logs, but it happens that production.log is empty! I don't think it's a permission problem, because if I delete the file, it is recreated when I reload the page. I tried to change the log level in conf/environments/production.rb, tried to manually write to log file with Rails console production and
Rails.logger.error('asdf')
it returns true but nothing gets written to production.log. The path (obtained per Rails.logger.inspect) is correct, and I remark that the file is recreated if I manually remove it. How can I know what's going on?
(I already checked the Apache logs, plus I set the highest debug level for Passenger but it seems a Rails problem, so is not logged by the server)
Assuming you're running Rails 3.2.1, this is a bug. It was patched in 3.2.2.
If you can't upgrade to 3.2.2 for any reason, this comment on GitHub has a workaround:
# config/initializers/patch_rails_production_logging.rb
Rails.logger.instance_variable_get(:#logger).instance_variable_get(:#log_dest).sync = true if Rails.logger
Setting this works on Rails 3.2.11:
Rails.logger = ActiveSupport::BufferedLogger.new(Rails.root.join("log","production.log"))
Here's what I've determined:
Delta indexing works fine in development
Delta indexing does not work when I push to the production server, and no action is logged in searchd.log
I'm running Phusion Passenger, and, as recommended in the basic troubleshooting guide, have confirmed that:
www-data has permission to run indexing rake tasks (ran them from command line manually)
the path to indexer and searchd are correct (/usr/local/bin)
there are no errors in production.log
What on earth could I possibly be missing? I'm running Ruby Enterprise 1.8.6, Rails 2.3.4, Sphinx 0.9.8.1, and Thinking Sphinx 1.2.11.
Thanks!
Last night as I slept it hit me. Unsurprisingly, it was a stupid issue involving bad configuration, though I am rather surprised that it produced the results it did. I guess I don't know much about Thinking Sphinx internals.
Recently I migrated servers. sphinx.yml looked like this:
production:
bin_path: '/usr/local/bin'
host: mysql.mysite.com
On the new server, MySQL was just a local service, but I had forgotten to remove that line. Interestingly, manual rake reindexing still worked just fine. I'm intrigued that Thinking Sphinx didn't throw an error when trying to reload the deltas, since mysql.mysite.com no longer exists, even though that was clearly the source of the issue.
Thanks for all your help, and sorry to have brought up such a silly problem.
Are there any clues in Apache/Nginx's error log?
Here's the next troubleshooting step I would take. Open up the file for the delta indexing strategy you are using (presumably lib/thinking_sphinx/deltas/default_delta.rb). Find the line where it actually generates the indexing command. In mine (v1.1.6) it's line 20:
output = `#{config.bin_path}indexer --config #{config.config_file} #{rotate} #{delta_index_name model}`
Change that so you can log the command itself, and maybe log the output as well:
command = `#{config.bin_path}indexer --config #{config.config_file} #{rotate} #{delta_index_name model}`
RAILS_DEFAULT_LOGGER.info(command)
output = `#{command}`
RAILS_DEFAULT_LOGGER.info(output)
Deploy that to production and tail the log while modifying a delta-indexed model. Hopefully that will actually show you the problem. Of course maybe the problem is elsewhere in the code and you won't even get to this point, but this is where I would start.
I was having this problem and found the "bin_path" solution mentioned above. When it didn't seem to work, it took me a while to realize that I'd pasted in the example code for "production" when I was testing on "stagiing" environment. Problem solved!
This was after making sure that the rake tasks that config, index, and start sphinx are all running as the same user as your passenger instance. If you log into the server as root to run these tasks, they will work in the console but not via passenger.
I had the same problem. Works on the command line, not inthe app.
Turns out that we still had a slave database that we were using for the indexing, but the slave wasn't getting updated.
As above, same issues were faced our side on two machine. The first one we had an issue with mysql which showed in apache2 log. Only seemed to affect local OSX machine..
Second time when we deployed to Ubuntu server, we had same issue. Rails c production was fine, no errors, bla bla bla.
Ended up being a permissions problem. Couldn't figure this out as there were no problems starting, although I guess I was doing so as root.
Using capistrano and passenger, we did this:
Create a passenger user and added to www-data group
Changed user in deploy.rb to be passenger
Manually changed all the /current files to be owned by above
Logged in as passenger user.
Ran rake ts:rebuild RAILS_ENV="production"
Worked a treat for us...
Good luck