I am new to Resque. I want to make and send an excel file with the help of a Resque worker. I I have written that code in worker class. Now I have called the worker inside the controller function as:
Resque.redis = Redis.new(:host => "#{Rails.configuration.redis_machine}", :port => Rails.configuration.redis_port)
Resque.enqueue(MakeAndSend,records,email)
Here records is rows of data that are to be printed to the excel file and email is the mail id of the user to which this excel file is to be sent.
But the problem is that my code is get stuck at this line, the following line is getting executed only when the job is over. But as far as I understood from studying resque that the following line should execute immediately.
So if anyone helps me to figure this out, I will be really grateful.
EDIT Here above records is actually collection of objects. So I don't have any option to send ids here. Also, the redis server is not getting hit.
Looking at it, I don't think it would be that line which is the problem - it will likely be something behind that line, such as the worker file or something
I can see one main issue you'll likely have, and there will be others:
-> You're passing variables (ActiveRecord objects) to the Resque queue -- it's convention to send the id's of the variables so the worker file can call the databases when it runs
Here is some live code for our Resque queues:
#app/models/message.rb (this gets called when we run "send_broadcast" function)
def broadcast!
self.subscribers.each do |subscriber|
Resque.enqueue(MailoutQueue, id, subscriber.id)
end
end
#app/workers/mailout_queue.rb
class MailoutQueue
#queue = :mailer
def self.perform(message_id, recipient_id)
MessageMailer.send_message(message_id, recipient_id).deliver
end
end
MessageMailer.rb is just a standard message sender file
If you can post your worker file code, it will help us appreciate where the real issue maybe, so we can fix appropriately!
Related
My counter cache is locking the row under heavy load so I found wanelo/counter-cache gem which seems to be perfect for my problem but I can't set it up and it must be something really simple but I can't see it.
https://github.com/wanelo/counter-cache
I want it to use my already working delayed jobs and Redis.
In my config file
Counter::Cache.configure do |c|
c.default_worker_adapter = here??? DelayedJob ??
c.recalculation_delay = 5.hours
c.redis_pool = Redis.new
c.counting_data_store = Counter::Cache::Redis
end
If I don't put the line c.default_worker_adapter when executing it says
undefined method 'enqueue' for nil:NilClass
Any idea on what's going on? What should I put in the Worker Adapter? Nothing seems to work.
Thank you for your time
default_worker_adapter is the name of the class that will be handling your updates. An example is given on the github page of the gem. For example if you're using sidekiq, you would make sidekiq worker class and name it whatever you want. On the github page, this class is called CounterWorker and you can copy it exactly as its given, though you can use whatever delayed job framework you want. From then on, any counter_cache_on definitions on your models will use that class to make the counter updates.
This question already has answers here:
What's the best way to organize worker processes in Rails?
(4 answers)
Closed 8 years ago.
I am working on a ruby&rails application, where the requirement is following: On receiving the HTTP request, the controller has to store the data in the database and return success to the end user. At this point, I want a background thread to wake up and perform operations on the added data. I am using the WEBRick server in the development mode. How to achieve this functionality?
Since you're looking for background processing libraries lot's of people recommend resque and sidekiq. You can look at this sitepoint tutorial for a good article comparing Delayed Job with the other two (I don't know that one so shan't comment on it here) and you can see a good comparison of these two in this stack overflow answer.
One of the main things to think about is whether your code that you want to run in a background process is theadsafe? if it's not stick to resque (I think it's simpler anyway, matter of opinion) because sidekiq runs parallel jobs (I'm sure you can choose to not do this).
Sample code: Redis (From docs):
To define some work, make a job. Jobs need a work method:
class ImageConversionJob
def work
# convert some kind of image here
end
end
Next, we need to procrastinate! Let's put your job on the queue:
resque = Resque.new
resque << ImageConversionJob.new
Neat! This unit of work will be stored in Redis. We can spin up a worker to grab some work off of the queue and do the work:
bin/resque work
Sidekiq (from linked sitepoint article):
class PrintWorker
include Sidekiq::Worker
def perform(str)
puts str
end
end
Once again, we have to queue up a job somewhere (we’ll do it in the index controller):
class IndexController < ApplicationController
def index
PrintWorker.perform_async(params[:to_print])
end
end
Finally, we have to get the Sidekiq process(es) fired up. Type this into your shell in the root directory of your Rails app:
bundle exec sidekiq
This is just a quick rundown and enough syntax to get a flavour of the answers. For more detail see the linked articles.
I'm integrating a communications api and whenever a text/voice reaches my server(rails controller), I have to send back an OK (200) to the api. I want to send this response before executing my code block because if my code breaks (and is unable to send the OK), the communcations api keeps sending the messages for up to 3 days. Now that just complicates the problem already on my server because it would keep breaking as the same message keeps on coming.
I did some research and found two solutions.
Solution 1: The first solution is below (my current implementation) and it doesnt seem to be working (unless I didnt read the log files properly or I'm hallucinating).
def receive_text_message
head :ok, :content_type => 'text/html'
# A bunch of code down here
end
I thought this should do (per rails doc), but I'm not sure it does.
Solution 2: the second implementation which I'm contemplating is to quickly create a new process/thread to execute the code block and kill off the process that received the message...that way the api gets its OK very quickly and it doesnt have to wait on the successful execution of my code block. I could the spawnling (or spawn) gem to do this. I would go with creating a process since I use passenger (community) server. But new processes would eat up more RAM, plus I think it is harder to debug child processes/thread (i might be wrong on this)
Thanks for the help!
Side question: does rails attempt to restart a process after it just failed?
You could opt for returning a 200 in your controller and start a sidekiq job. That way the 200 will be returned immediately and your controller will be ready to process the next job. So no waste of time and resources in your controller. The let the worker to do the real hard job.
In your controller
def receive_text_message
head :ok, :content_type => 'text/html'
HardWorker.perform_async(params)
end
In your sidekiq worker:
class HardWorker
include Sidekiq::Worker
def perform(params)
# 'Doing hard work'
end
end
I like sidekiq mostly because it is handling the resources more nicely compared to rescue.
In my current database setup I have a user and a server table. I am able to do things like current_user.server from the rails console, but I am not able to do this from inside a resque worker. I am however, able to call current_user.name because of this code in my resque worker:
current_user = User.find(current_id)
I was able to do this by passing in the current_user's id to the worker. I thought I could use the same process for the user's server table; it's a has_one user->server relationship.
So I tried this:
current_server = Server.where(:user_id => current_user.id)
current_server[:cloudserver] = 123 #Just an example number
current_server.save
The first line worked fine, it returned the right server row. However, when I also include the second line I get an error:
TypeError can't convert Symbol into Integer
I know I created the cloudserver column as an integer type, and I made sure I loaded the rails environment in the resque workers. I can set the correct number by using the rails console:
current_user.server[:cloudserver] = 123
current_user.save
Since I can set it outside of the resque worker, I'm pretty sure it's a resque problem.
Oh, and I'm also running rails 3.1rc4 if that helps.
Thanks all!
I'm using delayed_job (tried both tobi's and collective_idea's) on site5.com shared hosting, with passenger as rails environment.
I managed to make jobs done.
However, it seems the plugin ignores any changes in a job class source code after first run.
I have restarted the server on every change (touch tmp/restart.txt) but it still ignores it.
Example:
file: lib/xx_job.rb
class XxJob
def perform
Rails.logger.info "XX START"
TempTest.delete_all
i = 0
10.times {
i+=1
TempTest.create(:name => "XXX")
sleep(1)
}
Rails.logger.info "XX END"
end
end
In a simple controller I call:
Delayed::Job.enqueue(XxJob.new)
Conclusions I have gathered:
If I change xx_job.rb to xx_job1.rb - error on the controller
If I change class XxJob to class XxJob1 - error on the controller
If I delete all the perform method content - the old code old code is executed
New .rb file with class and perform, enqueue this class - works perfectly
If I change something in that new file's perform and run job again - old code is executed
Between every change I made a restart for the server.
It seems like Passenger or something else saves class cache.
How can I delete this cache? Is is stored on the server somewhere? (I hope I have access to it from the shared hosting)
Thanks!
If you run delayed job workers daemonized, then you need to restart them to reload the code. Also, keep in mind that each worker loads its own instance of rails.
Eventually I figured that out - several workers were running in background, each of them caught a job and had their own cache.
I didn't know how to kill them so I changed the table's name for several seconds. That killed them :)
Then I used https://github.com/tobi/delayed_job/wiki/Running-Delayed::Worker-as-a-daemon as worker start, and it works great.