Given an existing rails app with background processes managed by Sidekiq.
How can I migrate calls such as:
Model.delay.some_action()
From Sidekiq's syntax to Active Job's back-end agnostic syntax?
Update:
#craig.karminsky has pointed out the well-written Sidekiq to Active Job Wiki Page. This page addresses mailers.
Old Sidekiq syntax:
MyMailer.delay.send_message()
Active Job syntax:
MyMailer.send_message().deliver_later
That's a good solution for mailers, but how can I migrate non-mailer calls to .delay such as:
NotAMailer.delay.do_something()
Mike Perham, the creator of Sidekiq, has a very good Wiki page up for just such a situation.
I've made a gem for that on top of activejob: https://github.com/cristianbica/activejob-perform_later/. However the gem isn't very tested and this pattern is not a good one. Having code through your application that gets executed delay it will make your app hard to maintain and trigger all kind of bugs
Related
I have built a rails app for storing events. I want to generate a report to find the number of events happened in a day. I want to do it asynchronously. I am new to sidekiq and Redis. Can anyone suggest a good resource to study?
My suggestion for this would be to do this in a rake task that would be run on the server once a day.
You can find good resources on how to create rake tasks online and then use this simple gem to make sure the rake task runs once a day on the server.
https://github.com/javan/whenever
I am assuming you have a Profile model. You could use the timestamps in this model created_at to get all the profiles created on a given day. You could then create a CSV or whatever you like with that data and email it to whoever needs the report (how you handle the data is up to you)
You can do all the above in Sidekiq if you wish, I would recommend reading through the gem docs and this getting started guide from the official wiki https://github.com/mperham/sidekiq/wiki/Getting-Started
It's fairly straightforward and once you get your first process working it will start to make more sense.
I would also highly reccomend this video before you start working with sidekiq and redis, to give you an overall background of how sidekiq works and in what use cases it may be helpful to you.
https://www.youtube.com/watch?v=GBEDvF1_8B8
in my Rails 5 RC1 app write some log entries into a DB table through an ActiveRecord model.
Writing this log entry takes a couple of milliseconds and delays the response for the end user.
I am searching for a mechanism how I can execute the log-writing into the "background" so it is not blocking/delaying the response (kind of "fire-and-forget"). Do you have some hints on how to do that?
I tried to wrap the respective part into
Thread.new { code }
But this even seems to further delay the response a few MS.
I appreciate any hint!
Thanks and regards
Try looking into messaging services like RabbitMQ for asynchronous transactions and activities. I use RabbitMQ in exactly the way you describe.
If you are using ActiveJob, you could use :async, which looks to be the default queue_adapter for ActiveJob. However, this doesn't persist jobs between restarts and is not really recommended for production.
See Rails 5 changed Active Job default adapter from Inline to Async.
I am running a Ruby on Rails application, presently using delayed_job to send emails. But I want to switch to RabbitMQ messaging queue. I am not able to find any useful resource to get started with it. I read many RabbitMQ docs and what not. Please get me some heads up to accomplish this task.
RabbitMQ integrations for Rails heavily depend on eventmachine and callback-based code. This will make your code really complex, and you will need to make all your code aware of the event loop.
For instance, you can't use the same code if you deploy behind thin (which already has a running event loop) or behind unicorn (in which you have to instantiate a reactor and manage its lifecycle).
On the other side, you can abstract this to an actual job queue (Resque is already much faster than DJ, Sidekiq smokes its pants off, and Beanstalkd/Stalker is a very good contender) which is probably going to be compatible to the Rails.queue abstraction in rails 4.
In Rails 4, you have also the option to configure all ActionMailer to be async by default (thereby delegating to any configured job queue). http://reefpoints.dockyard.com/ruby/2012/06/26/rails-4-sneak-peek-async-actionmailer.html
If you are looking to kill 2 birds with one stone, and your sending an email needs are based on active record lifecycle events (which they almost always are), you should check out my gem.
https://github.com/jasonayre/active_pubsub
It was built on the bunny gem and works much like active record observers, except you run the subscribers in a separate process. (this is actually just a bonus functionality, its intended largely to allow multiple rails apps to subscribe to each others events) -- I.E.
#user model
class User < ::ActiveRecord::Base
include ::ActivePubsub::Publishable
publish_as "myapp"
end
#subscriber
class PostSubscriber < ::ActivePubsub::Subscriber
observes "cms"
as "aggregator"
on :created do |user_hash|
user = User.find(user_hash[:id])
#or whatever
WelcomeMailer.deliver(user)
end
end
== async delivery + keeps your controllers slimmer.
If you are looking for a great Ruby background jobs processing library to replace delayed_job you should give Sidekiq a try. There even is a recent Railscast about it here.
You can get an overview on Background Jobs Gems on Knight.io (I started this site).
I use whenever to call rake tasks throughout the day, but each task launches a new Rails environment. How can I run tasks throughout the day without relaunching Rails for each job?
Here's what I came up with, would love to get some feedback on this...?
Refactor each rake task to instead be a method within the appropriate Model.
Use the delayed_job gem to assign low priority and ensure these methods run asynchronously.
Instruct whenever to call each Model.method instead of calling the rake task
Does this solution make sense? Will it help avoid launching a new Rails environment for each job? .. or is there a better way to do this?
--
Running Rails 3
You could certainly look into enqueuing delayed_jobs via cron, then having one long running delayed_job worker.
Then you could use whenever to help you create the delayed_job enqueuing methods. It's probably easiest to have whenever's cron output call a small wrapper script which loads active_record and delayed_job directly rather than your whole rails stack. http://snippets.aktagon.com/snippets/257-How-to-use-ActiveRecord-without-Rails
You might also like to look into clockwork.rb, which is a long-running process that would do the same thing you're using cron for (enqueuing delayed_jobs): http://rubydoc.info/gems/clockwork/0.2.3/frames
You could also just try using a requeuing strategy in your delayed_jobs: https://gist.github.com/704047
Lots of good solutions to this problem, the one I eventually ended up integrating is as such:
Moved my rake code to the appropriate models
Added controller/routing code for calling models methods from the browser
Configured cronjobs using the whenever gem to run command 'curl mywebsite.com/model#method'
I tried giving delayed_job a go but didn't like the idea of running another Rails instance. My methods are not too server intensive, and the above solution allows me to utilize the already-running Rails environment.
Comment this line from schedule.rb
require File.expand_path(File.dirname(__FILE__) + "/environment")
Instead load only required ruby files like models in your case.
Is it possible to use the delayed_job gem outside of Rails? Or put another way, can it be used in a pure Ruby project?
If so, how?
I don't know why this person never posted this.
It's easy. Two steps:
http://brkrd.com/post/45269754283/delayed-job-without-rails
UPDATE Looks like the link is broken.
You will need Active Record, but not Rails. You will have to mock a Rails object, and load your database information, environment, and root into the Rails object so that DelayedJob thinks it's in the Rails environment.
Delayed Job looks heavily rails dependent. https://github.com/defunkt/resque resque is a very good gem that is not rails dependent that accomplishes the same goal. You do however have to have redis on your machine which is very lightweight. Resque also has a cool Sinatra web console to see what is going on.