The app is running on Engine Yard. I need to log some data from inside the scripts ran by Whenever. Here's what it looks like:
schedule.rb:
every 1.minute do
rake 'my_tasks:task1 my_tasks:task2', :output => 'log/my_tasks.log'
end
task1 runs the script that has the following lines in it (just for testing purposes):
...
log('test')
...
def self.log(string)
logfile = '/var/log/engineyard/apps/my_app/test_log.log'
File.open(logfile, 'a') { |log| log.puts(string << "\n\n") }
end
The problem is that none of the logs (my_tasks.log, test_log.log) get created, no matter what I tried. When I run bundle exec rake my_tasks:task1 from console it successfully creates the test_log.log file and logs the message. What could be wrong? Is it related to how Engine Yard works?
This is probably related to how Engine Yard works. I am not familiar with Engine Yard itself, but it is of a similar service as Heroku, and I know that in oder to send something to logs in Heroku, you have to use the stdout or stderr.
See here for more details about Heroku, and see if the same works for Engine Yard (keep in mind that this does not allow you to have separate log files, it simply puts the logged line in the same log file as other logged lines).
Related
I'm have a server where I deploy using capistrano and I use delayed_jobs to do some mailing but at my server for some reason the jobs do not execute. The delayed_job process is running (running bin/delayed_job status answers me correctly saying there's a process there by some pid), but I don't know if the process just isn't executing my jobs or even if my jobs aren't being enqueued. Locally it all works fine, but at production staging in the server it does not.
I'd like to know if there's a way I can at least check what jobs are there, since I can't do it by accessing the console
Another gem that works with delayed job is delayed-web which you can find here https://github.com/tatey/delayed-web
you add it to your gemfile
gem 'delayed-web'
then run
rails generate delayed:web:install
this will generate an initializer file delayed_web.rb under config/initializers with the following:
Rails.application.config.to_prepare do
Delayed::Web::Job.backend = 'active_record'
end
and in config/application.rb this will be added for you as well by the generator
# config/application.rb
config.assets.enabled = true
config.assets.precompile << 'delayed/web/application.css'
In routes.rb it may add a route as well but if you are using devise then maybe you want to restrict access to admin user(s) only as follows:
authenticated :user, -> user { user.admin? } do
mount Delayed::Web::Engine, at: '/jobs'
end
Ok so I'm checked my jobs through the database itself, I entered psql through postgres user and did some queries in the delayed_jobs table, you can also try doing RAILS_ENV=production bin/delayed_jobs run (for rails 4, rails 3 use "script/" instead of "bin/") which will show you what the workers are doing while they execute the job.
You can also, as Swards commented above, use a gem to have a web interface for delayed_jobs: https://github.com/ejschmitt/delayed_job_web
If you still wanna see what was my problem with the email sending I've opened another question because it got to far away from what this one was about: What port to use sending email with SMTP (mailgun) in rails app on production server (DigitalOcean)?
I have been working on a Ruby on Rails project (a basic survey application using the 'surveyor' gem) that lets a user download a pdf of the completed survey results. To accomplish, I have written a basic rake task (as below) that calls the wkhtmltopdf command line utility to convert from HTML/CSS to a pdf every 15 minutes via whenever/cron.
Here is the rake task (lib/tasks/pdf_survey_results.rake):
desc "PDF Completed Survey Results for Admin Download"
task :pdf_survey_results => :environment do
# iterate through all the response sets
ResponseSet.all.each do |response_set|
if !response_set.completed_at.blank?
puts "response set #{response_set.id} is complete, checking for pdf..."
if response_set.pdf_url.blank?
puts "no pdf found, now generating..."
# make the pdf
result = system "wkhtmltopdf #{BASE_URL}/surveys/student-questionnaire/#{response_set.access_code} /opt/deployed_rails_apps/survey_app/public/survey-pdfs/survey-#{response_set.access_code}.pdf"
if !result.nil?
response_set.pdf_url = "#{BASE_URL}/survey-pdfs/survey-#{response_set.access_code}.pdf"
response_set.save
end
end
end
end
end
This works totally fine in my development environment (OS X) but when I have deployed to Ubuntu 14.04,
I CAN manually create pdf's via the command line (i.e. wkhtmltopdf http://surveyurl.com sample.pdf)
I CANNOT create pdf's via running the rake task manually (i.e. rake pdf_survey_results will give me the output through "no pdf found, now generating..." but none of the output from the wkhtmltopdf tool even though I DO see this output when running on my local OS X dev environment)
I CANNOT create pdf's via the cron job set up by whenever, which is not surprising as the manual rake task execution is not working.
What I am most stuck with is that I cannot figure out why the rake task is not giving me text output from wkhtmltopdf when it does that in Mac OS X even though the command line utility by itself works fine. So far that aspect has made this a bit difficult for me to debug. By the way, I am using unicorn as an application server in this project.
Any advice is very much appreciated!!! Will answer any relevant questions if anything is unclear.
I have a chain of nginx + passenger for my rails app.
Now after each server restart i need to write in terminal in project folder
rake ts:start
but how can i automatize it?
So that after each server restart thinking sphinx is automatically started without my command in terminal?
I use rails 3.2.8 and ubuntu 12.04.
I can not imagine what can i try ever, please help me.
How can i do this, give some advices?
What I did to solve the same problem:
In config/application.rb, add:
module Rails
def self.rake?
!!#rake
end
def self.rake=(value)
#rake = !!value
end
end
In Rakefile, add this line:
Rails.rake = true
Finally, in config/initializers/start_thinking_sphinx.rb put:
unless Rails.rake?
begin
# Prope ts connection
ThinkingSphinx.search "test", :populate => true
rescue Mysql2::Error => err
puts ">>> ThinkingSphinx is unavailable. Trying to start .."
MyApp::Application.load_tasks
Rake::Task['ts:start'].invoke
end
end
(Replace MyApp above with your app's name)
Seems to work so far, but if I encounter any issues I'll post back here.
Obviously, the above doesn't take care of monitoring that the server stays up. You might want to do that separately. Or an alternative could be to manage the service with Upstart.
If you are using the excellent whenever gem to manage your crontab, you can just put
every :reboot do
rake "ts:start"
end
in your schedule.rb and it seems to work great. I just tested on an EC2 instance running Ubuntu 14.04.
There's two options I can think of.
You could look at how Ubuntu manages start-up scripts and add one for this (perhaps in /etc/init?).
You could set up monit or another monitoring tool and have it keep Sphinx running. Monit should boot automatically when your server restarts, and so it should ensure Sphinx (and anything else it's tracking) is running.
The catch with Monit and other such tools is that when you deliberately stop Sphinx (say, to update configuration structure and corresponding index changes), it might start it up again before it's appropriate. So I think you should start with the first of these two options - I just don't know a great deal about the finer points of that approach.
I followed #pat's suggestion and wrote a script to start ThinkingSphinx whenever the server boots up. You can see it as a gist -
https://gist.github.com/declan/4b7cc4fb4926df16f54c
We're using Capistrano for deployment to Ubuntu 14.04, and you may need to modify the path and user name to match your server setup. Otherwise, all you need to do is
Put this script into /etc/init.d/thinking_sphinx
Confirm that the script works: calling /etc/init.d/thinking_sphinx start on the command line should start ThinkingSphinx for your app, and /etc/init.d/thinking_sphinx stop should stop it
Tell Ubuntu to run this script automatically on startup: update-rc.d thinking_sphinx defaults
There's a good post on debian-administration.org called making scripts run at boot time that has more details.
What is the best way to automatically clear Memcached when I deploy my rails app to Heroku?
I'm caching the home page, and when I make changes and redeploy, the page is served from the cache, and the updates aren't incorporated.
I want to have this be totally automated. I don't want to have to clear the cache in the heroku console each time I deploy.
Thanks!
I deploy my applications using a bash script that automates GitHub & Heroku push, database migration, application maintenance mode activation and cache clearing action.
In this script, the command to clear the cache is :
heroku run --app YOUR_APP_NAME rails runner -e production Rails.cache.clear
This works with Celadon Cedar with the Heroku Toolbelt package. I know this is not a Rake-based solution however it's quite efficient.
Note : be sure you set the environment / -e option of the runner command to production as it will be executed on the development one otherwise.
Edit : I have experienced issues with this command on Heroku since a few days (Rails 3.2.21). I did not have time to check the origin the issue but removing the -e production did the trick, so if the command does not succeed, please run this one instead :
heroku run --app YOUR_APP_NAME rails runner Rails.cache.clear
[On the Celadon Cedar Stack]
-- [Update 18 June 2012 -- this no longer works, will see if I can find another workaround]
The cleanest way I have found to handle these post-deploy hooks is to latch onto the assets:precompile task that is already called during slug compilation. With a nod to asset_sync Gem for the idea:
Rake::Task["assets:precompile"].enhance do
# How to invoke a task that exists elsewhere
# Rake::Task["assets:environment"].invoke if Rake::Task.task_defined?("assets:environment")
# Clear cache on deploy
print "Clearing the rails memcached cache\n"
Rails.cache.clear
end
I just put this in a lib/tasks/heroku_deploy.rake file and it gets picked up nicely.
What I ended up doing was creating a new rake task that deployed to heroku and then cleared the cache. I created a deploy.rake file and this is it:
namespace :deploy do
task :production do
puts "deploying to production"
system "git push heroku"
puts "clearing cache"
system "heroku console Rails.cache.clear"
puts "done"
end
end
Now, instead of typing git push heroku, I just type rake deploy:production.
25 Jan 2013: this is works for a Rails 3.2.11 app running on Ruby 1.9.3 on Cedar
In your Gemfile add the following line to force ruby 1.9.3:
ruby '1.9.3'
Create a file named lib/tasks/clear_cache.rake with this content:
if Rake::Task.task_defined?("assets:precompile:nondigest")
Rake::Task["assets:precompile:nondigest"].enhance do
Rails.cache.clear
end
else
Rake::Task["assets:precompile"].enhance do
# rails 3.1.1 will clear out Rails.application.config if the env vars
# RAILS_GROUP and RAILS_ENV are not defined. We need to reload the
# assets environment in this case.
# Rake::Task["assets:environment"].invoke if Rake::Task.task_defined?("assets:environment")
Rails.cache.clear
end
end
Finally, I also recommend running heroku labs:enable user-env-compile on your app so that its environment is available to you as part of the precompilation.
Aside from anything you can do inside your application that runs on 'application start' you could use the heroku deploy hooks (http://devcenter.heroku.com/articles/deploy-hooks#http_post_hook) that would hit a URL within your application that clears the cache
I've added config/initializers/expire_cache.rb with
ActionController::Base.expire_page '/'
Works sweet!
Since the heroku gem is deprecated, an updated version of Solomons very elegant answer would be to save the following code in lib/tasks/heroku_deploy.rake:
namespace :deploy do
task :production do
puts "deploying to production"
system "git push heroku"
puts "clearing cache"
system "heroku run rake cache:clear"
puts "done"
end
end
namespace :cache do
desc "Clears Rails cache"
task :clear => :environment do
Rails.cache.clear
end
end
then instead of git push heroku master you type rake deploy:production in command line.
To just clear the cache you can run rake cache:clear
The solution I like to use is the following:
First, I implement a deploy_hook action that looks for a parameter that I set differently for each app. Typically I just do this on the on the "home" or "public" controller, since it doesn't take that much code.
### routes.rb ###
post 'deploy_hook' => 'home#deploy'
### home_controller.rb ###
def deploy_hook
Rails.cache.clear if params[:secret] == "a3ad3d3"
end
And, I simply tell heroku to setup a deploy hook to post to that action whenever I deploy!
heroku addons:add deployhooks:http \
--url=http://example.com/deploy_hook?secret=a3ad3d3
Now, everytime that I deploy, heroku will do an HTTP post back to the site to let me know that the deploy worked just fine.
Works like a charm for me. Of course, the secret token not "high security" and this shouldn't be used if there were a good attack vector for taking your site down if caches were cleared. But, honestly, if the site is that critical to attack, then don't host it on Heroku! However, if you wanted to increase the security a bit, then you could use a Heroku configuration variable and not have the 'token' in the source code at all.
Hope people find this useful.
I just had this problem as well but wanted to stick to the git deployment without an additional script as a wrapper.
So my approach is to write a file during slug generation with an uuid that marks the current precompilation. This is impelmented as a hook in assets:precompile.
# /lib/tasks/store_asset_cacheversion.rake
# add uuidtools to Gemfile
require "uuidtools"
def storeCacheVersion
cacheversion = UUIDTools::UUID.random_create
File.open(".cacheversion", "w") { |file| file.write(cacheversion) }
end
Rake::Task["assets:precompile"].enhance do
puts "Storing git hash in file for cache invalidation (assets:precompile)\n"
storeCacheVersion
end
Rake::Task["assets:precompile:nondigest"].enhance do
puts "Storing git hash in file for cache invalidation (assets:precompile:nondigest)\n"
storeCacheVersion
end
The other is an initializer that checks this id against the cached version. If they differ, there has been another precompilation and the cache will be invalidated.
So it dosen't matter how often the application spins up or down or on how many nodes the worker will be distributed, because the slug generation just happens once.
# /config/initializers/00_asset_cache_check.rb
currenthash = File.read ".cacheversion"
cachehash = Rails.cache.read "cacheversion"
puts "Checking cache version: #{cachehash} against slug version: #{currenthash}\n"
if currenthash != cachehash
puts "flushing cache\n"
Rails.cache.clear
Rails.cache.write "cacheversion", currenthash
else
puts "cache ok\n"
end
I needed to use a random ID because there is as far as I know no way of getting the git hash or any other useful id. Perhaps the ENV[REQUEST_ID] but this is an random ID as well.
The good thing about the uuid is, that it is now independent from heroku as well.
In a rake task if I use puts command then I see the output on console. However I will not see that message in log file when app is deployed on production.
However if I say Rails.logger.info then in development mode I see nothing on console. I need to go to log file and tail that.
I would ideally like to use Rails.logger.info and in development mode inside the rake task, the output from logger should also be sent to console.
Is there a way to achieve that?
Put this in application.rb, or in a rake task initialize code
if defined?(Rails) && (Rails.env == 'development')
Rails.logger = Logger.new(STDOUT)
end
This is Rails 3 code. Note that this will override logging to development.log. If you want both STDOUT and development.log you'll need a wrapper function.
If you'd like this behaviour only in the Rails console, place the same block of code in your ~/.irbrc.
You could create a new rake task to get this to work.
desc "switch logger to stdout"
task :to_stdout => [:environment] do
Rails.logger = Logger.new(STDOUT)
end
This way when you execute your rake task you can add to_stdout first to get stdout log messages or don't include it to have messages sent to the default log file
rake to_stdout some_task
Rake tasks are run by a user, on a command-line. Anything they need to know right away ("processed 5 rows") should be output on the terminal with puts.
Anything that needs to be kept for posterity ("sent warning email to jsmith#example.com") should be sent to the Rails.logger.
Code
For Rails 4 and newer, you can use Logger broadcast.
If you want to get both STDOUT and file logging for rake tasks in development mode, you can add this code into config/environments/development.rb :
if File.basename($0) == 'rake'
# http://stackoverflow.com/questions/2246141/puts-vs-logger-in-rails-rake-tasks
log_file = Rails.root.join("log", "#{Rails.env}.log")
Rails.logger = ActiveSupport::Logger.new(log_file)
Rails.logger.extend(ActiveSupport::Logger.broadcast(ActiveSupport::Logger.new(STDOUT)))
end
Test
Here's a small Rake task to test the above code :
# lib/tasks/stdout_and_log.rake
namespace :stdout_and_log do
desc "Test if Rails.logger outputs to STDOUT and log file"
task :test => :environment do
puts "HELLO FROM PUTS"
Rails.logger.info "HELLO FROM LOGGER"
end
end
Running rake stdout_and_log:test outputs
HELLO FROM PUTS
HELLO FROM LOGGER
while
HELLO FROM LOGGER
has been added to log/development.log.
Running rake stdout_and_log:test RAILS_ENV=production outputs
HELLO FROM PUTS
while
HELLO FROM LOGGER
has been added to log/production.log.
I'd say that using Rails.logger.info is the way to go.
You won't be able to see it in the server console because it won't run via the server. Just open up a new console and tail -f the log file, it'll do the trick.
Many users are aware of the UNIX®
command 'tail', which can be used to
display the last few lines of a large
file. This can be useful for viewing
log files, etc.
Even more useful in some situations,
is the '-f' parameter to the 'tail'
command. This causes tail to 'follow'
the output of the file. Initially, the
response will be the same as for
'tail' on its own - the last few lines
of the file will be displayed.
However, the command does not return
to the prompt, and instead, continues
to 'follow' the file. When additional
lines are added to the file, they will
be displayed on the terminal. This is
very useful for watching log files, or
any other file which may be appended
over time. Type 'man tail' for more
details on this and other tail
options.
(via)
In Rails 2.X to redirect the logger to STDOUT in models:
ActiveRecord::Base.logger = Logger.new(STDOUT)
To redirect logger in controllers:
ActionController::Base.logger = Logger.new(STDOUT)
Execute a background job with '&' and open script/console or whatever..
That way you can run multiple commands in the same window.
tail -f log/development.log &
script/console
Loading development environment (Rails 2.3.5)
>> Product.all
2011-03-10 11:56:00 18062 DEBUG Product Load (6.0ms) SELECT * FROM "products"
[<Product.1>,<Product.2>]
note Can get sloppy quickly when there is a lot of logging output.
How about creating an application helper which detects which environment is running and does the right thing?
def output_debug(info)
if RAILS_ENV == "development"
puts info
else
logger.info info
end
end
Then call output_debug instead of puts or logger.info