I am trying to run a method that adds the response from an API call to Cache, I decided to use the simple_scheduler gem
Below are snippets of code that I am running
# update_cache_job.rb
class UpdateCacheJob < ActiveJob::Base
def perform
return QueuedJobs.new.update_cache
end
end
And
# simple_scheduler.yml
# Global configuration options. The `queue_ahead` and `tz` options can also be set on each task.
queue_ahead: 120 # Number of minutes to queue jobs into the future
queue_name: "default" # The Sidekiq queue name used by SimpleScheduler::FutureJob
tz: "nil" # The application time zone will be used by default if not set
# Runs once every 2 minutes
simple_task:
class: "UpdateCacheJob"
every: "2.minutes"
And the method I have scheduled to run every 2.minutes
class QueuedJobs
include VariableHelper
def initialize; end
def update_cache
#variables = obtain_software_development_table
# First refresh the project Reviews
puts 'Updating reviews...'
#records = Dashboard.new.obtain_projects_reviews.pluck(
obtain_project_reviews_student_variable,
obtain_project_reviews_id_variable,
'Project'
).map { |student, id, project| { 'Student': student, 'ID': id,
'Project': project } }
Rails.cache.write(
:reviews,
#records,
expires_in: 15.minutes
)
#grouped_reviews = Rails.cache.read(
:reviews
).group_by do |review|
review[:Student]&.first
end
puts 'reviews refreshed.'
# Then refresh the coding challenges submissions
puts "Updating challenges submissions.."
#all_required_submissions_columns = Dashboard.new.all_coding_challenges_submissions.all.map do |submission|
{
id: submission.id,
'Student': submission[obtain_coding_chall_subm_student_var],
'Challenge': submission[obtain_coding_chall_subm_challenge_var]
}
end
#all_grouped_submissions = #all_required_submissions_columns.group_by { |challenge| challenge[:Student]&.first }
Rails.cache.write(
:challenges_submissions,
#all_grouped_submissions,
expires_in: 15.minutes
)
puts "challenges submissions refreshed."
end
end
I have been able to reach these methods from the rails console but when ever I run rake simple_scheduler It just logs the first puts and sometimes it does nothing at all.
What do I need to do here?
Related
I've got pretty old App where I have to create rake task to find all users over 18 and update flags from adult: false to adult: true. I'm wondering what I should use in a rather old version of Rails (I have Rails 5 and Ruby 2.4 on board) to keep the highest performance?
What I have for now is a sidekiq worker with, I think, some syntax error:
class MinorsWorker
include Sidekiq::Worker
def perform
adults = User.where(adults: false).where('date_of_birth >= 18, ?', ((Time.zone.now - date_of_birth.to_time) / 1.year.seconds))
adults.update(adult: true)
end
end
But this code gives me an error:
NameError: undefined local variable or method `date_of_birth' for main:Object
you can do the following. This would update all the matched records in 1 update statement.
If you are concern with db IO, you can batch it.
# in user.rb
class User
scope :adult, -> { where('date_of_birth <= ?', 18.years.ago) }
end
# in your worker file
class MinorsWorker
include Sidekiq::Worker
def perform
update_all
# for update in batches, use #update_all_in_batches
end
private
def update_all
User.adult.where(adult: false).update_all(adult: true)
end
def update_all_in_batches
User.adult.where(adult: false).in_batches(each: 1000) do |users|
users.update_all(adult: true)
sleep 2
end
end
end
ActiveRecord::RecordNotFound in Sidekiq worker when I save object. I don't use rails callbacks.
I run worker from service, and save object in this service.
class LeadSmsSendingService < Rectify::Command
...initialize params
def send_sms_message
sms_conversation = lead.sms_conversations.find_or_create_by(sms_number: sms_number)
attrs = sms_form.to_hash.symbolize_keys.slice(:body, :direction, :from, :to)
.merge(campaign_id: campaign_id)
sms_message = sms_conversation.sms_messages.build(attrs)
sms_message.to ||= lead.phone
sms_message.body = VariableReplacement.new(lead).render(sms_message.body)
# #todo we need to raise an exception here
return unless sms_message.save
DeliverSmsMessageWorker.perform_in(3.seconds, sms_message.id, 'LeadSmsSendingService')
end
end
class DeliverSmsMessageWorker
include Sidekiq::Worker
sidekiq_options queue: 'priority'
def perform(sms_message_id, from_where="Unknown")
sms_message = SmsMessage.find(sms_message_id)
sms_message.deliver!
rescue StandardError => e
Bugsnag.notify(e) do |report|
# Add information to this report
report.add_tab(:worker, { from_where: from_where.to_s })
end
end
end
Seems that the record has still to be commited, even if it sounds strange because of the 3 seconds delay. Does it work if you increase this delay?
This link could be useful: https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting#cannot-find-modelname-with-id12345
So I ran this command rake environment elasticsearch:import:model CLASS='AutoPartsMapper' FORCE=true to index documents in elasticsearch.In my database I have 10 000 000 records=)...it takes (I think) one day to index this...When indexing was running my computer turned off...(I indexed 2 000 000 documents)Is it possible to continue indexing documents?
If you use rails 4.2+ you can use ActiveJob to schedule and leave it running. So, first generate it with this
bin/rails generate job elastic_search_index
This will give you class and method perform:
class ElasticSearchIndexJob < ApplicationJob
def perform
# impleement here indexing
AutoPartMapper.__elasticsearch__.create_index! force:true
AutoPartMapper.__elasticsearch__.import
end
end
Set the sidekiq as your active job provider and from console initiate this with:
ElasticSearchIndexJob.perform_later
This will set the active job and execute it on next free job but it will free your console. You can leave it running and check the process in bash later:
ps aux | grep side
this will give you something like: sidekiq 4.1.2 app[1 of 12 busy]
Have a look at this post that explains them
http://ruby-journal.com/how-to-integrate-sidekiq-with-activejob/
Hope it helps
There is no such functionality in elasicsearch-rails afaik but you could write a simple task to do that.
namespace :es do
task :populate, [:start_id] => :environment do |_, args|
start_id = args[:start_id].to_i
AutoPartsMapper.where('id > ?', start_id).order(:id).find_each do |record|
puts "Processing record ##{record.id}"
record.__elasticsearch__.index_document
end
end
end
Start it with bundle exec rake es:populate[<start_id>] passing the id of the record from which to start the next batch.
Note that this is a simplistic solution which will be much slower than batch indexing.
UPDATE
Here is a batch indexing task. It is much faster and automatically detects the record from which to continue. It does make an assumption that previously imported records were processed in increasing id order and without gaps. I haven't tested it but most of the code is from a production system.
namespace :es do
task :populate_auto => :environment do |_, args|
start_id = get_max_indexed_id
AutoPartsMapper.find_in_batches(batch_size: 1000).where('id > ?', start_id).order(:id) do |records|
elasticsearch_bulk_index(records)
end
end
def get_max_indexed_id
AutoPartsMapper.search(aggs: {max_id: {max: {field: :id }}}, size: 0).response[:aggregations][:max_id][:value].to_i
end
def elasticsearch_bulk_index(records)
return if records.empty?
klass = records.first.class
klass.__elasticsearch__.client.bulk({
index: klass.__elasticsearch__.index_name,
type: klass.__elasticsearch__.document_type,
body: elasticsearch_records_to_index(records)
})
end
def self.elasticsearch_records_to_index(records)
records.map do |record|
payload = { _id: record.id, data: record.as_indexed_json }
{ index: payload }
end
end
end
i am trying to work out how to write a rake tasks that will run daily and find where the days remaining is 0 to update the column amount to zero.
I have the following methods defined in my model, though they don't exactly appear to be working as I am getting the following error in the view
undefined method `-#' for Mon, 27 Jun 2016:Date
def remaining_days
expired? ? 0 : (self.expire_at - Date.today).to_i
end
def expired?
(self.expire_at - Date.today).to_i <= 0
end
def expire_credits
if expired?
self.update(:expire_at => Date.today + 6.months, :amount => 0)
end
end
with the rake tasks i have never written of these and i thought i would be able to call a method of StoreCredit that would expire the points if certain conditions are met but i am not sure how this all works
task :expire_credits => :environment do
puts 'Expiring unused credits...'
StoreCredit.expire_credits
puts "done."
end
# model/store_credit.rb
# get all store_credits that are expired on given date, default to today
scope :expire_on, -> (date = Date.current) { where("expire_at <= ?", date.beginning_of_day) }
class << self
def expire_credits!(date = Date.current)
# find all the expired credits on particular date, and update all together
self.expire_on(date).update_all(amount: 0)
end
end
Since it's a rake task, I think it's more efficient to update all expired ones together
#rake file
result = StoreCredit.expire_credits!
puts "#{result} records updated"
Retrieve Record Count Update
class << self
def expire_credits!(date = Date.current)
# find all the expired credits on particular date, and update all together
records = self.expire_on(date)
records.update_all(amount: 0)
records.length
end
end
You call class method but define instance method. You will need to define class method:
def self.expire_credits
I have the following script which runs once a day on cron on heroku.
However, I realize that I would like the option for the user to be able to press a button from a web page to initiate this same process.
Is there a way to create a 'subroutine' that either cron can call or from a web request? I don't want to use a separate service that runs jobs.
I've just put a snippet to illustrate.....
letter_todos = Todo.current_date_lte(Date.today).asset_is("Letter").done_date_null
unless letter_todos.blank? #check if there any ToDos
# group by asset_id so that each batch is specific to the asset_id
letter_todos.group_by(&:asset_id).each do |asset_id, letter_todos|
# pdf = Prawn::Document.new(:margin => 100) #format the PDF document
html_file = ''
letter_todos.each do |todo| #loop through all Letter_Todos
contact = Contact.find(todo.contact_id) #get associated contact
letter = Letter.find(todo.asset_id) #get associated Letter
redcloth_contact_letter = RedCloth.new(letter.substituted_message(contact, [])).to_html
html_file = html_file + redcloth_contact_letter
html_file = html_file + "<p style='display: none; page-break-after: always'><center> ... </center> </p>"
end
kit = PDFKit.new(html_file)
kit.stylesheets << "#{RAILS_ROOT}/public/stylesheets/compiled/pdf.css"
file = kit.to_pdf
letter = Letter.find(asset_id)
#OutboundMailer.deliver_pdf_email(file)
kit.to_file("#{RAILS_ROOT}/tmp/PDF-#{letter.title}-#{Date.today}.pdf")
# Create new BatchPrint record
batch = BatchPrint.new
batch.pdf = File.new("#{RAILS_ROOT}/tmp/PDF-#{letter.title}-#{Date.today}.pdf")
I've done this by putting the function in question in a file in lib (lib/tasks_n_stuff.rb, say):
module TasksNStuff
def self.do_something
# ...doing something...
end
end
Then I can call if from a Rake task:
desc 'Make sure we depend on :environment, so we can get to the Railsy stuff...'
task :do_something => :environment do
TasksNStuff.do_something
end
Or from a controller (or anywhere, really):
class WhateverController < ApplicationController
def do_something
TasksNStuff.do_something
end
end
And since you can run a rake task as a cron job (cd /my/rails/root; rake do_something), that should be all you need. Cheers!