I'm trying to use delayed_jobs. It works perfectly, locally, when using the rake jobs:work task. However, when I try to use it as a daemon, the tasks fail. I've stopped it from deleting failed tasks. I'm starting the daemon with 'RAILS_ENV=production bin/delayed_jobs start'. When I do status it shows the worker is active.
They seem to be failing with this error 'Job failed to load: undefined class/module ProductAquisition.'
This is a snippet from the module I'm trying to use (lib/product_aquisition.rb).
module ProductAquisition
require 'net/http'
def ProductAquisition.get_updates
.....
end
end
This is from the rakefile I'm trying to use to trigger this
require 'debugger'
require 'date'
require 'thread'
# require 'active_support'
require "#{Rails.root}/lib/product_aquisition.rb"
require "#{Rails.root}/app/helpers/soap_helper"
include SoapHelper
include ProductAquisition
task :get_updates => :environment do
ProductAquisition.delay.get_updates
end
I don't understand how it's unable to load and use the module 'ProductAquisition' when running as a daemon. Can anyone shed some light on this?
Edit: Full error message
Full error output can be seen here: https://gist.github.com/smithmr8/70f1f736fe1c679ceff1/raw/9586759c868adce3797ad9e7697789ef417821a6/gistfile1.rb
I tried posting it here but it wouldn't wrap correctly.
Related
I have recently started using the websocket-rails gem and all is fine in development and even in production, but when I run rspec tests I get the following error:
Failure/Error: Unable to find matching line from backtrace
RuntimeError:
eventmachine not initialized: evma_install_oneshot_timer
The message seems to appear on the second test I run. When I run a previously failing test on it's own by adding :focus => true to the test and then rspec --tag focus its passes OK. If I add the focus to more than one test, usually the first one passes and the second test gives the error.
I am using rspec Capybara with the Selemium web driver. I think this may be to do with the web server in test not being an "eventmachine based web server" but I am not 100% sure. I have tried setting up the websockets server as standalone by adding the following in my websocket initializer config.standalone = true and then starting the server by rake websocket_rails:start_server RAILS_ENV=test.
I solved this by configuring rspec to use the Thin server by adding the following within my spec_helper.rb file:
RSpec.configure do |config|
Capybara.server do |app, port|
require 'rack/handler/thin'
Rack::Handler::Thin.run(app, :Port => port)
end
#other configuration here....
end
I am a beginner when it comes to Rails. I am trying to follow this example:
http://ryanselk.com/2014/09/25/using-background-jobs-in-rails-42-with-active-job/
It says:
"Jobs can be added to the job queue from anywhere. We can add a job to the queue by: ResizeImage.perform_later 'http://example.com/ex.png' "
[UPDATE] Sorry, I am stupid. I came up with this task:
namespace :simple do
# call from command line:
# rake simple:resize_images
desc "Resize images"
task resize_images: :environment do
Dir.foreach('storage') do |next_image|
puts next_image
next if next_image == '.' or next_image == '..'
ResizeImage.perform_later next_image
end
end
end
but now I do:
rake simple:resize_images
and I get:
zacek2_phpP9JGif.jpg
rake aborted!
NameError: uninitialized constant ResizeImage
I've tried:
require ResizeImage
but that did not fix the problem.
I am afraid I don't understand how loading works in Rails. How do I load ResizeImage?
Do I set it up as a cron job?
No.
From the rails guides:
Active Job is a framework for declaring jobs and making them run on a variety of queueing backends.
Active Job is an interface to queueing backends such as sidekiq, delayed_job or resque. It's simply a way for you to write background jobs where you don't have to care about which of the queueing backends will be used.
How do I start ActiveJob?
So ActiveJob doesn't run background jobs on it's own. You're still missing one of the backends. Say you have decided to use delayed_job: Get it installed and then start it via:
script/delayed_job start
I don't understand where "anywhere" is.
That means anywhere in your code, you could write something like:
user.rb
def send_registration_email
UserRegistraionMailJob.perform_later self
end
I'm running a custom rake task...
namespace :import do
desc "Import terms of service as HTML from stdin"
task :terms => :environment do
html = STDIN.read
settings = ApplicationWideSetting.first
settings.terms_and_conditions = html
if settings.save
puts "Updated terms of service"
else
puts "There was an error updating terms of service"
end
end
end
The model ApplicationWideSetting is reported as undefined when running the task in the production environment. However, when running the task on other environments (ie. development, staging, test.) the task runs fine.
Running the process in rails console, in all environments, completes ok.
Does anyone know what's going on, things I could check?
note: I ran the task with
puts Rails.env
To check the shell environment var RAILS_ENV was getting set/read correctly. I've also tried both with and without the square brackets around the :environment dependency declaration.
additional info: Rails v3.2.14
further info: I've setup a completely fresh rails app, and the script works fine in any environment. Since the install in question is a real production environment, I'll have to setup another deploy and check it thoroughly. More info as I find it.
In a nutshell, Rails doesn't eager load models (or anything else) when running rake tasks on Production.
The simplest way to work with a model is to require it when you begin the rake task, and it should work as expected, in this case:
# explicitly require model
require 'application_wide_setting'
It's possible to eager load the entire rails app with:
Rails.application.eager_load!
However, you may have issues with some initializers (ie. devise)
Kinda new to Rails, so please cope with me. What i'm doing now is background processing some Ruby code use Resque. To get the Rescque rake task started, I've been using (on heroku), I have a resque.rake file with that recommended code to attach into heroku's magical(or strange) threading architecture:
require "resque/tasks"
require 'resque_scheduler/tasks'
task "resque:setup" => :environment do
ENV['QUEUE'] = '*'
end
desc "Alias for resque:work (To run workers on Heroku)"
task "jobs:work" => "resque:work"
Since I need access to the Rails code, I reference :environment. If I set at least 1 worker dyno in the background on heroku, my Resque does great, gets cleared, everything is happy. Until i try to automate stuff...
So I wanted to evolve the code and automatically fill the queue with relevant tasks every minute or so. Do that (without using cron, because heroku is not adequate with cron), I declare an initializer named task_scheduler.rb that uses Rufus scheduler to run tasks:
scheduler = Rufus::Scheduler.start_new
scheduler.in '5s' do
autoprocessor_method
end
scheduler.every '1m' do
autoprocessor_method
end
Things appear to work awesome for a while....then the rake process just stops picking up from the queue unexplainably. The queue just gets larger and larger. Even if i have multiple worker dynos running, they all eventually get tired and stop processing the queue. I'm not sure what I am doing wrong, but I suspect the referencing of the Rails environment in my rake task is causing the task_scheduler.rb code to run again, causing duplicate scheduling. I'm wondering how to solve that problem if someone knows, and I'm also curious if that is the reason for the rake task to stop working.
Thank you
You should not be booting the scheduler in an initializer, you should have a daemon process running the scheduler and filling up your queue. It would be something like this ("script/scheduler"):
#!/usr/bin/env ruby
root = File.expand_path(File.join(File.dirname(__FILE__), '..'))
Dir.chdir(root)
require 'rubygems'
gem 'daemons'
require 'daemons'
options = {
:dir_mode => :normal,
:dir => File.join(root, 'log'),
:log_output => true,
:backtrace => true,
:multiple => false
}
Daemons.run_proc("scheduler", options) do
Dir.chdir(root)
require(File.join(root, 'config', 'environment'))
scheduler = Rufus::Scheduler.start_new
scheduler.in '5s' do
autoprocessor_method
end
scheduler.every '1m' do
autoprocessor_method
end
end
And you can call this script as a usual daemon from your app:
script/scheduler start
This is going to make sure you have only one process sending work for the resque workers instead of one for each mongrel that you're running.
First of all, if you are not running on Heroku, i would not recommend this approach. I'd look at Mauricio's answer, or consider using a classic cron job or using Whenever to schedule the cron job.
But if you are in the pain of running on heroku and trying to do this, here is how i got this to work.
I kept the same original Resque.rake code in place, as i pasted in the original question. In addition, i created another rake task that i attached to the jobs:work rake process, just like the first case:
desc "Scheduler processor"
task :scheduler => :environment do
autoprocess_method
scheduler = Rufus::Scheduler.start_new
scheduler.every '1m' do
twitter_autoprocess
end
end
desc "Alias for resque:work (To run workers on Heroku)"
task "jobs:work" => "scheduler"
Couple of notes:
This will be imperfect once you use more than one worker dyno because the scheduler will run in more than one spot. you can solve that by saving state somewhere, but its not as clean as I would like.
I found the original reason why the process would hang. It was this line of code:
scheduler.in '5s' do
autoprocessor_method
end
I'm not sure why, but when I removed that, it never hung again.
I'm using a rufus scheduler to replace cron jobs from the deployment system and get those jobs kick-started when an application loads on deployment.
Now I have this scheduler.rb placed in the config/initializers directory from the application root directory.
The content of the scheduler.rb file is as below:
require 'rufus/scheduler'
require 'rubygems'
require 'rake'
load File.join(RAILS_ROOT,'lib','tasks','tempfile.rake')
temp_files_cleaning_scheduler = Rufus::Scheduler.start_new
temp_files_cleaning_scheduler.cron '*/1 * * * *' do
Rake::Task["tempfile:delete_all"].reenable
Rake::Task["tempfile:delete_all"].invoke
end
Now when I start the application server, I get the error message as below:
scheduler caught exception :
Don't know how to build task 'environment'
/home/karthik/.rvm/gems/jruby-1.5.2/gems/rake-0.8.7/lib/rake.rb:1728:in `[]'
/home/karthik/.rvm/gems/jruby-1.5.2/gems/rake-0.8.7/lib/rake.rb:605:in `invoke_prerequisites'
where 'environment' is a dependant task for the task "tempfile:delete_all" that I'm invoking. And this :environment task is defined in railties/lib/tasks/misc.rake.
I don't want to load this misc.file by hard-coding the path to it. Is there a cleaner way to solve this problem?
Thanks.
It sounds like you need more definitions that are in Rakefiles that aren't getting loaded, probably because "lib/tasks/tempfile.rake" doesn't have any require statements in it.
I assume this works from the command line, and if so, you have two options:
Load your app's main Rakefile, which has all teh necessary includes:
load File.join(RAILS_ROOT,'lib','tasks','tempfile.rake')
Just call it as as if from the console:
system('rake tempfile:delete_all')
Hope that helps!