I have a Rails app that queries an external webservice to update records.
I want to continue polling the web service until the user session expires.
Currently I create an ActiveJob in the show action for the record I want updated.
In the ActiveJob I reschedule it using
self.class.set(wait: 60.seconds).perform_later(record_id)
The problem is that if the user goes to the show action again, it will create another ActiveJob.
Is there anyway to prevent duplicate jobs from being created?
Ideally, there would be a way to search the ActiveJobs to see if one already exists. Something like ActiveJob.find_by(job: JobType, params: record_id). That would allow you to manipulate the jobs before creating a duplicate. I'm going to dig in further and see if that could be created...
The activejob-uniqueness gem is designed to de-dupe jobs. The gem is still relatively new, but it's sponsored by an engineering company (Veeqo) and seems well supported. See also: https://devs.veeqo.com/job-uniqueness-for-activejob/
First set a cookie value when the user visits the show for the first time.
self.class.set(wait: 60.seconds).perform_later(record_id) if cookies[:_aj].nil?
cookies[:_aj] = true
Also maybe create a column in your Record, maybe call it pending_update and set it true whenever you schedule a job to run and set it to false at the end of the scheduled job. That way, even if the user clears the cookies, your program will not create duplicate jobs.
Related
I have an admin dashboard action in a react front end using a ruby on rails API back end. I have a sidekiq job that runs to import users from a third party. I would like to somehow trigger a refresh on the admin panel when the job is complete. What is the best way to go about this? I am using graphql-ruby. I could use start-polling on the front end maybe? Can I poll for completion of a specific job on sidekiq somehow? Any help is appreciated!
Hi Ilovebathroomlights
There are several approaches I can use, one of them I am writing below assuming you are okay to write a state to Redis, Database, or any storage you might be using and can be used to poll.
You can start by assigning a unique id to your sidekiq job, and save this id with status as "pending", at the last line inside the sidekiq job code, update the status of the unique id as "completed".
In meantime, you can poll by the unique id you generated for the particular job, and refresh the data or refresh the page.
I have an application with a model named appointment. On this model, there is a column with the name event_uid and a validation like the following:
validates :event_uid, uniqueness: true, allow_nil: true
The unique validation is only on rails application and not in the database (postgresql).
I am using background job with sidekiq on heroku to sync some remote calendars. I am not sure what happened, but it seems like I got multiple records with duplicate event_uid values. They have been created in the exact same second.
My guess is that something happened on the workers and for some reason they got invoked at the same time or the queue frozen and when it got back it ran the same job twice. I don't understand why rails let the above to pass (maybe because workers run on different threads plays a role?). I added the following migration:
add_index :appointments, [:event_uid], unique: true
With the hope that it won't happen again. Ok so now the questions:
What do you think, will this be enough?
Is it dangerous to allow unique / presence validations to exist only on application level if you are using create / update with background jobs?
Any guess what could have caused the workers to run the same job more than one and exactly the same second?
The Rails uniqueness validation has been reason for confusion a long time.
When you persist a user instance, Rails will validate your model by running a SELECT query to see if any user records already exist with the provided email.
Assuming the record proves to be valid, Rails will run the INSERT statement to persist the user.
https://thoughtbot.com/blog/the-perils-of-uniqueness-validations
This means, if you have several workers / threads selecting at the same time they will all return false and insert the record.
Most of the time it is desirable to have an index on database level to avoid these race conditions too. However, you need to now also handle any ActiveRecord::RecordNotUnique exception.
What do you think, will this be enough?
Yes, adding an index is a good idea but now you need to also handle ActiveRecord::RecordNotUnique.
Is it dangerous to allow unique / presence validations to exist only on application level if you are using create / update with background jobs?
This depends on the application but most of the time you want to have an index on db level too.
Any guess what could have caused the workers to run the same job more than one and exactly the same second?
Most background job libraries only guarantee that at least one job gets enqueued but not exactly one. Your jobs should always be idempotent (can run several times). A good read is this guide about ActiveJob design, especially the part about idempotency.
Usually, validations take place in rails during callbacks only (sometimes before_commit the record on the DB), and yes if you added a unique index this will not happen again because the DB will take charge this time so even if you run into the same flow/issue again the result is likely an error saying that you can't duplicate that index value.
Given the nature of the validator (usually are called during callbacks and there are not thread-safe) meaning that they can run into race conditions, how common this can happens depends on your application, you should add always the validation on the DB as well.
Related to your workers I ran into the same issue due to the retry flow of Sidekiq a few months ago, the solution was to validate on the DB side as well and make a fix to run the workers/jobs after_commit callback (not sure if you are using Sidekiq, but you can always use the after_commit callback, I was using my job after certain operation took place over a particular object).
Hope the above helps! 👍
I have a rails app deployed on Heroku. I want to add a feature that enables users of the app to set a reminder. I need some way for the app to schedule sending an email at the time specified by the user.
I have found numerous posts referring to using delayed_job for this, but none of the write-ups / tutorials / etc. that I have found directly address what I am trying to accomplish (the descriptions I have found seem more geared towards managing long-running jobs that are to be run "whenever").
Am I on the right track looking at delayed_job for this? If so, can somebody point me towards a tutorial that might help me?
If delayed_job is not quite right for the job, does anybody have a suggestion for how I might approach this?
The most typical way of handling this is to use a cron job. You schedule a job to run every 15 minutes or so and deliver any reminders that come up in that time. Unfortunately, heroku only allows cron jobs to run every hour, which usually isn't often enough.
In this case, I'd use delayedjob and trick it into setting up a recurring task that delivers the notifications as often as necessary. For example, you could create a function that begins by rescheduling itself to run in 10 minutes and then goes on to send any reminders that popped up in the previous 10 minutes.
To view delayedjobs send_at syntax to schedule future jobs check here: https://github.com/tobi/delayed_job/wiki
ADDED after comments:
To send the reminder, you would need to create a function that searches for pending reminders and sends them. For example, let's say you have a model called Reminder (rails 3 syntax cause I like it better):
def self.find_and_send_reminders
reminders = Reminder.where("send_at < ? AND sent = ?", Time.now, false).all
reminders.each do |r|
#the following delayed_job syntax is apparently new, and I haven't tried it. came from the collective_idea fork of delayed_job on github
Notifier.delay.deliver_reminder_email(r)
#I'm not checking to make sure that anything actually sent successfully here, just assuming they did. may want to address this better in your real app
r.update_attributes!(:sent => true)
end
#again using the new syntax, untested. heroku may require the old "send_at" and "send_later" syntax
Reminder.delay(:run_at => 15.minutes.from_now).find_and_send_reminders
end
This syntax assumes you decided to use the single reminder entry for every occurence method. If you decide to use a single entry for all recurring reminders, you could create a field like "last_sent" instead of a boolean for "sent" and use that. Keep in mind these are all just ideas, I haven't actually taken the time to implement anything like this yet so I probably haven't considered all the options/problems.
Check out the runt gem, may be useful for you: http://runt.rubyforge.org/
You can use delayed_job's run_at to schedule at a specific time instead of whenever.
If your application allows the users to change the time of the reminders you need to remember the delayed_job to be able to update it or delete it when required.
Here is more details.
It's good to avoid polling if you can. The worker thread will poll at the database level, you don't want to add polling on top of polling.
I'm integrating some non-rails-model tables in my Rails application. Everything works out very nicely, the way I set up the model is:
class Change < ActiveRecord::Base
establish_connection(ActiveRecord::Base.configurations["otherdb_#{RAILS_ENV}"])
set_table_name "change"
end
This way I can use the Change model for all existing records with find etc.
Now I'd like to run some sort of notification, when a record is added to the table. Since the model never gets created via Change.new and Change.save using ActiveRecord::Observer is not an option.
Is there any way I can get some of my Rails code to be executed, whenever a new record is added? I looked at delayed_job but can't quite get my head around, how to set that up. I imagine it evolves around a cron-job, that selects all rows that where created since the job last ran and then calls the respective Rails code for each row.
Update Currently looking at Javan's Whenever, looks like it can solve the 'run rails code from cron part'.
Yeah, you'll either want some sort of background task processor (Delayed::Job is one of the popular ones, or you can fake your own with the Daemon library or similar) or to setup a cronjob that runs on some sort of schedule. If you want to check frequently (every minute, say) I'd recommend the Delayed::Job route, if it's longer (every hour or so) a cron job will do it just fine.
Going the DJ route, you'd need to create a job that would check for new records, process them if there are any, then requeue the job, as each job is marked "completed" when it's finished.
-jon
This is what I finally did: Use Whenever, because it integrates nicely with Capistrano and showed me how to run Rails code from within cron. My missing peace was basically
script/runner -e production 'ChangeObserver.recentchanges'
which is now run every 5 minutes. The recentchanges reads the last looked-at ID from a tmp-file, pulls all new Change records which have a higher ID than that and runs the normal observer code for each record (and saves the highest looked-at ID to the tmp-file, of course).
As usual with monitoring state changes, there are two approaches : polling and notification. You seem to have chose to go the polling way for now (having a cron job look at the state of the database on a regular basis and execute some code if that changed)
You can do the same thing using one of the rails schedulers, there are a few out there (google will find them readily, they have various feature sets, I'll let you choose the one which suits your need if you got that way)
You could also try to go the notification way depending on your database. Some database support both triggers and external process execution or specific notification protocols.
In this case you are notified by the database itself that the table changed. there are many such options for various DBMS in Getting events from a database
How would I go about sending an email to a user, say, 48 hours after they sign up, in Ruby on Rails? Thanks!
As Joseph Daigle mentioned, you need to obviously record the exact date and time the user registered. After that, you need a cron running every certain number of minutes (every hour, for example) checking to see if there's any new users whose registration time is greater than 48 hours, send a mail to said user and mark that user as already emailed, so you don't email them again.
As per the actual mail sending, check out the following documentation page:
http://wiki.rubyonrails.org/rails/pages/HowToSendEmailsWithActionMailer
It has all you need to know to send mails with RoR.
I recommend that you use the latest version of BackgrounDRb to handle this. You can read about BackgrounDRb here: http://backgroundrb.rubyforge.org/
In order to queue a message for later delivery, the BackgrounDRb client code (in your application model's after_create callback, maybe) could look something like this:
MiddleMan(:email_worker).enq_send_email_task(:message => #message,
:job_key => "notify1",
:scheduled_at => Time.now + 48.hours)
You'd have to build a BackgrounDRb worker to handle sending the email:
# RAILS_ROOT/lib/workers/email_worker.rb
class EmailWorker < BackgrounDRb::MetaWorker
set_worker_name :email_worker
def send_email_task(message)
# ... Code to send the email message
end
end
Note that in order to use BackgrounDRb in this way, you have to use persistent job queues, so make sure you run the migration included with BackgrounDRb to set up the persistence table in your application.
BackgrounDRb is started separately from Rails (mongrel, apache, etc) using 'script/backgroundrb start', so make sure that you add the daemon to whatever process monitoring you're using (god, monit, etc) or that you create an /etc/init.d script for it.
First you're going to need a running daemon or background service which can poll your queue (probably from in a database) every few minutes.
The algorithm is pretty simple. Record the time of the user event in the queue. When the daemon checks that item in the queue, and the time difference is greater than 48 hours, prepare the e-mail to send.
You can queue jobs with a delay using async observer. Ideally, anything you have that isn't known to be instant (or very close to it) all the time should pass through something like that.
I wrote a plugin called acts_as_scheduled that may help you out.
acts_as_scheduled allows you to manage
scheduled events for your models.
A good example of this is scheduling
the update of RSS Feeds in a
background process using Cron or
BackgroundRB.
With acts_as_scheduled your schedule
manager can simply call
"Model.find_next_scheduled()" to grab
the next item from the database.
How I would approach this is by creating a scheduling controller, that will query the database for the next_scheduled and then use a mailer to send the message. The you set up a Cron Job to call the controller periodically using WGET or CURL. The advantage of the Cron/Controller approach is that no further infrastructure or configuration is required on the server and you avoid complicated threading code.
I think I'd be inclined to store the need for the email and the earliest time after which it should be sent, somewhere separate, then have my things-to-do task look at that. That way I only have to process as many records as there are emails to be sent, rather than examine every user every time, which would either get tedious or require an otherwise probably unnecessary index. As a bonus, if I had other tasks to be performed on some sort of a diarised basis, the same construct would be useful with little modification.