I have a rufus scheduler in my Rails app that I'm using to record/insert data every 5 minutes from 9:30AM to 4:00PM Mon-Fri. The method containing this action is fired as soon as user is registered. BUT this logging job is intercepted and terminated once user logs out.
Could anyone help in figuring out how to persist the logging method even when the user logs out/session is destroyed?
I am calling the logging method in my Users controller create method and I've set up the logging method in the User model.
Any help would be greatly appreciated!
That sounds convoluted. Why don't you use a single scheduled job that is run every 5 minutes from 9:30AM to 4:00PM Mon to Fri for every User registered in the system.
do_log =
lambda do
User.where(status: 'active').each do |u|
# do the insertion for the user...
end
end
scheduler.cron('30,35,40,45,50,55 9 * * mon-fri America/New_York', &do_log)
scheduler.cron('*/5 10-15 * * mon-fri America/New_York', &do_log)
The User.where(status: 'active') is Sequel, it's probably different in Rails, but I am sure you can translate and adapt to your setting and needs.
Thus, as soon as a User is registered in your system, the logging will begin for him, even if he's logged out.
Yes, I know it's not a single job, it's two jobs, but that takes care of the "0930 to 1630" requirement.
Maybe that's not what you want, but you are not very clear in your question...
Update 2021-01-30 - 1416+0900
#hurricanenara commented:
I'm not certain where the do_log would be called to act as an all-encompassing job that will be applied to all "active" users during the scheduled times
The do_log will be called '30,35,40,45,50,55 9 * * mon-fri America/New_York' and '*/5 10-15 * * mon-fri America/New_York' which corresponds to your requirement:
every 5 minutes from 9:30AM to 4:00PM Mon-Fri
If it is easier to you, the do_log can be "forgotten" if writing
scheduler.cron('30,35,40,45,50,55 9 * * mon-fri America/New_York') do
User.where(status: 'active').each do |u|
# do the insertion for the user...
end
end
scheduler.cron('*/5 10-15 * * mon-fri America/New_York') do
User.where(status: 'active').each do |u|
# do the insertion for the user...
end
end
As you can see, the do_log lambda avoids a repetition (Don't Repeat Yourself, DRY).
This method assumes that when a "User is registered" in your system, they get added to the database in a user table fronted by a User model. The do_log simply iterates over all the user whose status is "active" (not "left", or "revoked", or "whatever"), and records information for that particular user.
Your problem description is vague. I'm guessing that you want to log for each user, because you want to log the information "even when the user logs out/session is destroyed".
I don't know what you mean by "persist the logging method". I'm not sure you mean "persist to disk", so my guess is that you mean "keep on running".
It is your responsibility as a developer to clearly state your problem. Most of the time that effort at clearly stating the problem will yield the answer to you without requiring another developer to listen. See also Rubber Ducking.
Related
I am building an online e-commerce store, and I am trying to use rails with action cable to update a product from being out of stock to in-stock at a certain date time e.g 12:00:00 2020-02-19.
The idea is as soon as the time is reached, I want to push a Websocket that the product is now available.
I have tried a few solutions such as:
Thread.new do
while true do
if **SOMETIME** == Time.now
ActionCable.server.broadcast "product_channel",content: "product-in-stock"
end
end
end
The main issue with this approach is that it creates another thread and makes rails unresponsive. Furthermore, if this value is set for say 1 week from now I do not want every user who queries the endpoint to create a brand-new thread running like this.
You have two option use sidekiq jobs or use whenever job gem
https://github.com/mperham/sidekiq/wiki/Scheduled-Jobs
Whenever allow you to set specific day and time, check the documentation for more info
https://github.com/javan/whenever
Debouncing is a common method to postpone a function/job from executing until after certain time has passed.
Use-case: A conversation with active chatting from multiple users, they should not receive an email notification for each message typed. But more than likely after a few minutes of silence, if the messages are unread, the user should see a notification.
Delayed_Job
Has no solution, has related issues: https://github.com/collectiveidea/delayed_job/issues/72
Sidekiq
https://github.com/hummingbird-me/sidekiq-debounce
Doing yourself is not so bad.
class AdminJob
def self.debounce(job, args={})
handler = YAML.dump(job)
count = Delayed::Job.where(handler: handler).where('locked_at IS NULL').delete_all
Rails.logger.info("deleted: #{count} jobs")
Delayed::Job.enqueue(job, args)
end
end
Instead of writing:
Delayed::Job.enqueue(YourJobName.new(account_id), {run_at: 10.minutes.from_now})
You now write:
AdminJob.debounce(YourJobName.new(account_id), {run_at: 10.minutes.from_now})
Delayed job serializes your job params in YAML and then saves it to the database as handler. So if you call AdminJob.debounce(...) 10 times in a row, it will delete before each.
Make sure to give yourself time (5.minutes, etc) to give users to keep taking actions. If you run your job after 1 second, its likely they'll keep taking actions and trigger again.
Yes i'm answering my own question 3 years later...
What I would do is schedule a periodic task, lets say every 5 minutes that checks if there is someone who has to be notified. Yes it seems an expensive operation, but for your use case I don't see (for now) others solutions. So lets say you have 10 users that uses a chat. Every 5 minutes you could check if there are users that didn't see some messages, and if so you notify them only if they are inactive from N minutes.
To schedule such task you could use crono gem. Check this answer.
Crono lets you do thing like that:
Crono.perform(CheckUsersToBeNotifiedJob).every 5.minutes
If you are using Delayed Job as the implementation for Active Job then take a look at the 'activejob-trackable' gem.
https://github.com/ignatiusreza/activejob-trackable
It uses another table that tracks the jobs and can throttle and debounce.
I want to give my users the option to send them a daily summary of their account statistics at a specific (user given) time ....
Lets say following model:
class DailySummery << ActiveRecord::Base
# attributes:
# send_at
# => 10:00 (hour)
# last_sent_at
# => Time of the last sent summary
end
Is there now a best practice how to send this account summaries via email to the specific time?
At the moment I have a infinite rake task running which checks permanently if emails are available for sending and i would like to put the dailysummary-generation and sending into this rake task.
I had a thought that I could solve this with following pseudo-code:
while true
User.all.each do |u|
u.generate_and_deliver_dailysummery if u.last_sent_at < Time.now - 24.hours
end
sleep 60
end
But I'm not sure if this has some hidden caveats...
Notice: I don't want to use queues like resq or redis or something like that!
EDIT: Added sleep (have it already in my script)
EDIT: It's a time critical service (notification of trade rates) so it should be as fast as possible. Thats the background why I don't want to use a queue or job based system. And I use Monit to manage this rake task, which works really fine.
There's only really two main ways you can do delayed execution. You run the script when an user on your site hits a page, which is inefficient and not entirely accurate. Or use some sort of background process, whether it's a cron job or resque/delayed job/etc.
While your method of having an rake process run forever will work fine, it's inefficient because you're iterating over users 24/7 as soon as it finishes, something like:
while true
User.where("last_sent_at <= ? OR last_sent_at = ?", 24.hours.ago, nil).each do |u|
u.generate_and_deliver_dailysummery
end
sleep 3600
end
Which would run once an hour and only pull users that needed an email sent is a bit more efficient. The best practice would be to use a cronjob though that runs your rake task though.
Running a task periodically is what cron is for. The whenever gem (https://github.com/javan/whenever) makes it simple to configure cron definitions for your app.
As your app scales, you may find that the rake task takes too long to run and that the queue is useful on top of cron scheduling. You can use cron to control when deliveries are scheduled but have them actually executed by a worker pool.
I see two possibilities to do a task at a specific time.
Background process / Worker / ...
It's what you already have done. I refactored your example, because there was two bad things.
Check conditions directly from your database, it's more efficient than loading potential useless data
Load users by batch. Imagine your database contains millions of users... I'm pretty sure you would be happy, but not Rails... not at all. :)
Beside your code I see another problem. How are you going to manage this background job on your production server? If you don't want to use Resque or something else, you should consider manage it another way. There is Monit and God which are both a process monitor.
while true
# Check the condition from your database
users = User.where(['last_sent_at < ? OR created_at IS NULL', 24.hours.ago])
# Load by batch of 1000
users.find_each(:batch_size => 1000) do |u|
u.generate_and_deliver_dailysummery
end
sleep 60
end
Cron jobs / Scheduled task / ...
The second possibility is to schedule your task recursively, for instance each hour or half-hour. Correct me if I'm wrong, but do your users really need to schedule the delivery at 10:39am? I think that let them choose the hour is enough.
Applying this, I think a job fired each hour is better than an infinite task querying your database every single minute. Moreover it's really easy to do, because you don't need to set up anything.
There is a good gem to manage cron task with the ruby syntax. More infos here : Whenever
You can do that, you'll need to also check for the time you want to send at. So starting with your pseudo code and adding to it:
while true
User.all.each do |u|
if u.last_sent_at < Time.now - 24.hours && Time.now.hour >= u.send_at
u.generate_and_deliver_dailysummery
# the next 2 lines are only needed if "generate_and_deliver_dailysummery" doesn't sent last_sent_at already
u.last_sent_at = Time.now
u.save
end
end
sleep 900
end
I've also added the sleep so you don't needlessly hammer your database. You might also want to look into limiting that loop to just the set of users you need to send to. A query similar what Zachary suggests would be much more efficient than what you have.
If you don't want to use a queue - consider delayed job (sort of a poor mans queue) - it does run as a rake task similar to what you are doing
https://github.com/collectiveidea/delayed_job
http://railscasts.com/episodes/171-delayed-job
it stores all tasks in a jobs table, usually when you add a task it queues it to run as soon as possible, however you can override this to delay it until a specific time
you could convert your DailySummary class to DailySummaryJob and once complete it could re-queue a new instance of itself for the next days run
How did you update the last_sent_at attribute?
if you use
last_sent_at += 24.hours
and initialized with last_sent_at = Time.now.at_beginning_of_day + send_at
it will be all ok .
don't use last_sent_at = Time.now . it is because there may be some delay when the job is actually done , this will make the last_sent_at attribute more and more "delayed".
My team is currently using Authlogic for user authentication, which disables a user's account after 6 failed_login_attempts. I want to re-enable such a user's account after a 15-minute time period. The problem is that we are deploying to the Rails cloud host, Heroku, which offers only a one-hour cron job and a daily cron job. I seem to need a cron job that increments in a matter of minutes, which I don't have, or I would have to freeze a thread to sit and wait the time out, which is not even going to be considered for obvious performance reasons.
Do I have any other options to implement this specific user experience?
If you already have the current_user loaded, you can just do this:
if current_user.failed_login_attempts >= 6 && current_user.failed_login_at < 15.minutes.ago
current_user.update_attribute(:failed_login_attempts, 0)
end
Something along those lines.
There's actually a declarative way of doing this. In your UserSession class, among the options available is failed_login_ban_for, which checks the last updated_at (which is touched even for failed attempts) so a successful attempt 15 minutes after a bogus attempt, for example, will now succeed:
class UserSession < Authlogic::Session::Base
logout_on_timeout true
consecutive_failed_logins_limit 5
failed_login_ban_for 15.minutes
end
See the rdoc for more info: http://rdoc.info/projects/binarylogic/authlogic
Rails has a nice set of filters (before_validation, before_create, after_save, etc) as well as support for observers, but I'm faced with a situation in which relying on a filter or observer is far too computationally expensive. I need an alternative.
The problem: I'm logging web server hits to a large number of pages. What I need is a trigger that will perform an action (say, send an email) when a given page has been viewed more than X times. Due to the huge number of pages and hits, using a filter or observer will result in a lot of wasted time because, 99% of the time, the condition it tests will be false. The email does not have to be sent out right away (i.e. a 5-10 minute delay is acceptable).
What I am instead considering is implementing some kind of process that sweeps the database every 5 minutes or so and checks to see which pages have been hit more than X times, recording that state in a new DB table, then sending out a corresponding email. It's not exactly elegant, but it will work.
Does anyone else have a better idea?
Rake tasks are nice! But you will end up writing more custom code for each background job you add. Check out the Delayed Job plugin http://blog.leetsoft.com/2008/2/17/delayed-job-dj
DJ is an asynchronous priority queue that relies on one simple database table. According to the DJ website you can create a job using Delayed::Job.enqueue() method shown below.
class NewsletterJob < Struct.new(:text, :emails)
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
end
Delayed::Job.enqueue( NewsletterJob.new("blah blah", Customers.find(:all).collect(&:email)) )
I was once part of a team that wrote a custom ad server, which has the same requirements: monitor the number of hits per document, and do something once they reach a certain threshold. This server was going to be powering an existing very large site with a lot of traffic, and scalability was a real concern. My company hired two Doubleclick consultants to pick their brains.
Their opinion was: The fastest way to persist any information is to write it in a custom Apache log directive. So we built a site where every time someone would hit a document (ad, page, all the same), the server that handled the request would write a SQL statement to the log: "INSERT INTO impressions (timestamp, page, ip, etc) VALUES (x, 'path/to/doc', y, etc);" -- all output dynamically with data from the webserver. Every 5 minutes, we would gather these files from the web servers, and then dump them all in the master database one at a time. Then, at our leisure, we could parse that data to do anything we well pleased with it.
Depending on your exact requirements and deployment setup, you could do something similar. The computational requirement to check if you're past a certain threshold is still probably even smaller (guessing here) than executing the SQL to increment a value or insert a row. You could get rid of both bits of overhead by logging hits (special format or not), and then periodically gather them, parse them, input them to the database, and do whatever you want with them.
When saving your Hit model, update a redundant column in your Page model that stores a running total of hits, this costs you 2 extra queries, so maybe each hit takes twice as long to process, but you can decide if you need to send the email with a simple if.
Your original solution isn't bad either.
I have to write something here so that stackoverflow code-highlights the first line.
class ApplicationController < ActionController::Base
before_filter :increment_fancy_counter
private
def increment_fancy_counter
# somehow increment the counter here
end
end
# lib/tasks/fancy_counter.rake
namespace :fancy_counter do
task :process do
# somehow process the counter here
end
end
Have a cron job run rake fancy_counter:process however often you want it to run.