In Rails 5, I have this method in my "Aulas" ("Classes" in Portuguese) Controller:
def set_week_classes
classes = Aula.all.to_a
#this_week_classes = classes.shift(2)
end
Considering that "classes" is an array, I would like to have "#this_week_classes = classes.shift(2)" being executed every Monday, at midnight (Brazil's time), getting the next two items of the classes array to be shown on the view. And also, I would like that when it reached the end of the array, it simply started all over, making "#this_week_classes" become again the first two items of the classes array. How could I make this happen? Thank you!
You can use sidekiq with some kind of a scheduling gem (like sidekiq-scheduler or sidekiq-cron). Depending on your installation you could also copy use a rake task and run it periodically using cron. If you use cloud then your provider definitely have some kind of scheduler available.
BTW all of your source code should probably be in English. Mixing some Portuguese class names doesn't look great and can be confusing for other contributors.
But if your only goal is to show some what classes are listed this week it's probably better to do sth like this:
classes = Aula.all.to_a # not the best for the memory
shift = DateTime.current.weeks_since(CONSTANT_TIME) % classes.size
#this_week_classes = ([classes]+[classes])[shift..(shift+2)] # [classes]+[classes] make sure that we won't get too little classes if we reach the and of the `classes` array
It appears you ask two questions at the same time:
Run a job every week at a certain time.
Rails offers different ways to do this. To me the best fit seems to work with an active job library.
Possibilities are:
https://github.com/javan/whenever cron runner. Let's you set up jobs for every week to run.
Another open library for this type of task is the gem delayed_job. It's not very performant but easy to include into small projects.
Cycle through an array of items.
Here a possibility is instead of actually shifting the items out of your array that you store the job run in your database. Keep in mind there are other possibilities that do not need you to change your database. Following code is not tested but should be seen as pseudocode.
def run_job
last_aula_job = AulaJob.all.order(:created_at).last
classes = Aula.all.to_a
total = classes.count
p = last_aula_job.last_pointer % count
#this_week_classes = classes[p..p+1]
# do something with #this_week_classes
AulaJob.create(last_pointer: p + 2)
end
Related
So I've been looking for the simplest way to send an e-mail when X column of Payments table in the database is == 'condition'. Basically what I want is to add a payment and set a date like 6 months. When 6 months have passed I want to send the mail. I've seen many solutions like using Whenever cron jobs and others but I want to know the absolute simplest way (perhaps using Rails only without relying on outside source) to keep my application light and clean. I was thinking I could use the auto generated created_at to evaluate when x time has passed.
Since you have a column in your db for the time to send email, make it a datetime datatype and you can set the email date as soon as the event payment event is created. Then, you can have a rake task where,
range = Time.now.beginning_of_day..Time.now.end_of_day
Payment.where(your_datetime_custom_column: range).each do |payment|
payment.user.send_email
end
and you can run this task everyday from the scheduler.
The "easiest" way is to use Active Job in conjunction with a state machine:
EmailJob.set(wait: 6.months).perform_later(user.id) if user.X_changed?
The problem with this is that the queue will accumulate jobs since jobs don't get handled right away. This may lead to other performance issues since there are now more jobs to scan and they're taking up more memory.
Cron jobs are well suited for this kind of thing. Depending on your hosting platform, there may be various other ways to handle this; for example, Heroku has Heroku Scheduler.
There are likely other ways to schedule repeating tasks without cron, such as this SO answer.
edit: I did use a gem once called 'fist_of_fury', but it's not currently maintained and I'm not sure how it would perform in a production environment. Below are some snippets for how I used it in a rails project:
in Gemfile
gem 'fist_of_fury'
in config/initializers/fist_of_fury.rb
# Ensure the jobs run only in a web server.
if defined?(Rails::Server)
FistOfFury.attack! do
ObserveAllJob.recurs { minutely(1) }
end
end
in app/jobs/observe_all_job.rb
class ObserveAllJob
include SuckerPunch::Job
include FistOfFury::Recurrent
def perform
::Task.all.each(&:observe)
end
end
I have a case scenario where I need to run multiple record updates in the background(using resque) and I want to give user visual indicator of how the task is running(eg started/running/finished).
One way of achieving this(which I can think of) is saving the current state into a table, then showing the state to user by simple page refresh.
Can anyone suggest a better solution of doing it?I want to avoid creating the whole migration, model, controller for this.
Thanks
As I've commented, resque-status gem could be useful for you. I am not sure if that is an answer but since you said that you do not want to create migration, model and controller for this. Thus, a gem might be the way to go.
From the job id you can get the status you are looking for, for example:
status = Resque::Plugins::Status::Hash.get(job_id)
status.working? #=> true
There is also a front-end called resque-web, check that out too.
You may use ruby's global variable $var_name = 'foo'. However I am not sure about it, because they are considered bad practice in rails, but in this case I see them reasonable, as soon as their name is very unique.
It can be done like (in case of resque):
class UpdateJob
#queue = data
def self.perform
$my_job_name_is_running = true
MyJobName.new.run
$my_job_name_is_running = nil
end
end
then you can access them from anywhere in the app:
while $my_job_name_is_running
puts "job is running..." if $my_job_name_is_running
sleep 3 # important to not overload your processor
end
Ruby global vars are not very popular. Check docs for more info https://ruby-doc.org/docs/ruby-doc-bundle/UsersGuide/rg/globalvars.html
I have these two methods in my model. One method looks up a single CatalogItem facebook like count, and another that loops through all active CatalogItems and finds their like counts using the aforementioned.
It takes a while to run through all active facebook likes...it might loop anywhere from 300-1000 objects; so i'd like to move this to some sort of cron, or whatever you guys suggest.
I was thinking I should add a column to CatalogItem called cached_fb_count, and adapt self.facebook_likes to write to that colimn whenever that task runs.
Is this the right approach? What would that task look like if it was running every 2 hours?
def self.facebook_likes
self.active.each_with_index do |i, index|
_likes = i.facebook_like_count
i.update_attribute(:cached_likes, _likes)
# puts "#{index+1} Likes: #{_likes} ########### ID: #{i.id} "
end
end
def facebook_like_count
item_like_count = JSON.parse(open("https://api.facebook.com/method/fql.query?query=select%20like_count%20from%20link_stat%20where%20url='https://www.foobar.com/catalog_items/#{self.id}'&format=json").read).first.flatten[1]
item_like_count = item_like_count + 1 if item_like_count > 0
end
Delayed_job is a perfect tool for doing asynchronous tasks. It runs in a separate process, relation database-based (Active Record) so it saves context of execution as a simple script invokation. And has a rich functionality inculding task's priority and scheldue. but If you tasks assumes huge queues, consider Resque gem. it uses Reddis as a storage for tasks and deals much faster with long queues.
Use whenever its very easy to set up. Here is the link : https://github.com/javan/whenever
My Survey model has about 2500 instances and I need to apply the set_state method to each instance twice. I need to apply it the second time only after every instance has had the method applied to it once. (The state of an instance can depend on the state of other instances.)
I'm using delayed_job to create delayed jobs and workless to automatically scale up/down my worker dynos as required.
The set_state method typically takes about a second to execute. So I've run the following at the heroku console:
2.times do
Survey.all.each do |survey|
survey.delay.set_state
sleep(4)
end
end
Shouldn't be any issues with overloading the API, right?
And yet I'm still seeing the following in my logs for each delayed job:
Heroku::API::Errors::ErrorWithResponse: Expected(200) <=> Actual(429 Unknown)
I'm not seeing any infinite loops -- it just returns this message as soon as I create the delayed job.
How can I avoid blowing Heroku's API rate limits?
Reviewing workless, it looks like it incurs an API call per delayed job to check the worker count and potentially a second API call to scale up/down. So if you are running 5000 (2500x2) jobs within a short period, you'll end up with 5000+ API calls. Which would be well in excess of the 1200/requests per hour limit. I've commented over there to hopefully help toward reducing the overall API usage (https://github.com/lostboy/workless/issues/33#issuecomment-20982433), but I think we can offer a more specific solution for you.
In the mean time, especially if your workload is pretty predictable (like this). I'd recommend skipping workless and doing that portion yourself. ie it sounds like you already know WHEN the scaling would need to happen (scale up right before the loop above, scale down right after). If that is the case you could do something like this to emulate the behavior in workless:
require 'heroku-api'
heroku = Heroku::API.new(:api_key => ENV['HEROKU_API_KEY'])
client.post_ps_scale(ENV['APP_NAME'], 'worker', Survey.count)
2.times do
Survey.all.each do |survey|
survey.delay.set_state
sleep(4)
end
end
min_workers = ENV['WORKLESS_MIN_WORKERS'].present? ? ENV['WORKLESS_MIN_WORKERS'].to_i : 0
client.post_ps_scale(ENV['APP_NAME'], 'worker', min_workers)
Note that you'll need to remove workless from these jobs also. I didn't see a particular way to do this JUST for certain jobs though, so you might want to ask on that project if you need that. Also, if this needs to be 2 pass (the first time through needs to finish before the second), the 4 second sleep may in some cases be insufficient but that is a different can of worms.
I hope that helps narrow in on what you needed, but I'm certainly happy to discuss further and/or elaborate on the above as needed. Thanks!
Rails has a nice set of filters (before_validation, before_create, after_save, etc) as well as support for observers, but I'm faced with a situation in which relying on a filter or observer is far too computationally expensive. I need an alternative.
The problem: I'm logging web server hits to a large number of pages. What I need is a trigger that will perform an action (say, send an email) when a given page has been viewed more than X times. Due to the huge number of pages and hits, using a filter or observer will result in a lot of wasted time because, 99% of the time, the condition it tests will be false. The email does not have to be sent out right away (i.e. a 5-10 minute delay is acceptable).
What I am instead considering is implementing some kind of process that sweeps the database every 5 minutes or so and checks to see which pages have been hit more than X times, recording that state in a new DB table, then sending out a corresponding email. It's not exactly elegant, but it will work.
Does anyone else have a better idea?
Rake tasks are nice! But you will end up writing more custom code for each background job you add. Check out the Delayed Job plugin http://blog.leetsoft.com/2008/2/17/delayed-job-dj
DJ is an asynchronous priority queue that relies on one simple database table. According to the DJ website you can create a job using Delayed::Job.enqueue() method shown below.
class NewsletterJob < Struct.new(:text, :emails)
def perform
emails.each { |e| NewsletterMailer.deliver_text_to_email(text, e) }
end
end
Delayed::Job.enqueue( NewsletterJob.new("blah blah", Customers.find(:all).collect(&:email)) )
I was once part of a team that wrote a custom ad server, which has the same requirements: monitor the number of hits per document, and do something once they reach a certain threshold. This server was going to be powering an existing very large site with a lot of traffic, and scalability was a real concern. My company hired two Doubleclick consultants to pick their brains.
Their opinion was: The fastest way to persist any information is to write it in a custom Apache log directive. So we built a site where every time someone would hit a document (ad, page, all the same), the server that handled the request would write a SQL statement to the log: "INSERT INTO impressions (timestamp, page, ip, etc) VALUES (x, 'path/to/doc', y, etc);" -- all output dynamically with data from the webserver. Every 5 minutes, we would gather these files from the web servers, and then dump them all in the master database one at a time. Then, at our leisure, we could parse that data to do anything we well pleased with it.
Depending on your exact requirements and deployment setup, you could do something similar. The computational requirement to check if you're past a certain threshold is still probably even smaller (guessing here) than executing the SQL to increment a value or insert a row. You could get rid of both bits of overhead by logging hits (special format or not), and then periodically gather them, parse them, input them to the database, and do whatever you want with them.
When saving your Hit model, update a redundant column in your Page model that stores a running total of hits, this costs you 2 extra queries, so maybe each hit takes twice as long to process, but you can decide if you need to send the email with a simple if.
Your original solution isn't bad either.
I have to write something here so that stackoverflow code-highlights the first line.
class ApplicationController < ActionController::Base
before_filter :increment_fancy_counter
private
def increment_fancy_counter
# somehow increment the counter here
end
end
# lib/tasks/fancy_counter.rake
namespace :fancy_counter do
task :process do
# somehow process the counter here
end
end
Have a cron job run rake fancy_counter:process however often you want it to run.