I've got Sidekiq job (SyncProductsWorker) which fires a class Imports::SynchronizeProducts responsible for a few external API calls.
module Imports
class SyncProductsWorker
include Sidekiq::Worker
sidekiq_options queue: 'imports_sync'
def perform(list)
::Imports::SynchronizeProducts.new(list).call
end
end
end
The Imports::SynchronizeProducts class gives an array of monads results with some comments e.g.
=> [Failure("999999 Product code is not valid"), Failure(" 8888889 Product code is not valid")]
I would like to catch these results to display them on FE. Is it possible to do so? If I do something like:
def perform(list)
response = ::Imports::SynchronizeProducts.new(list).call
response
end
And then inside of the controller:
def create
response = ::Imports::SyncProductsWorker.perform_async(params[:product_codes])
render json: { result: response, head: :ok }
end
I'll have some number as a result
=> "df3615e8efc56f8a062ba1c2"
I don't believe what you want is possible.
https://github.com/mperham/sidekiq/issues/3532
The return value will be GC'd like any other unused data in a Ruby process. Jobs do not have a "result" in Sidekiq and Sidekiq does nothing with the value.
You would need some sort of model instead that keeps track of your background tasks. This is off the cuff but should give you an idea.
EG
# #attr result [Array]
# #attr status [String] Values of 'Pending', 'Error', 'Complete', etc..
class BackgroundTask < ActiveRecord
attr_accessor :product_codes
after_create :enqueue
def enqueue
::Imports::SyncProductsWorker.perform_async(product_codes, self.id)
end
end
def perform(list, id)
response = ::Imports::SynchronizeProducts.new(list).call
if (response.has_errors?)
BackgroundTask.find(id).update(status: 'Error', result: response)
else
BackgroundTask.find(id).update(status: 'Complete', result: response)
end
end
Then just use the BackgroundTask model for your frontend display.
Related
In scheduler.rake I have this method:
task :email_unaccepted_meetings => :environment do
desc "If meeting time is less than 3 days away, and not accepted, send email to requestee"
#date_plus_three_days = Time.now + 3.days
#meetings = Meeting.where("status = ? AND accepted != ? AND meeting_time < ? ", "Active", true, #date_plus_three_days)
if #meetings != nil
#meetings.each do |meeting|
EmailAndPushHelper.unaccepted_lex_request(meeting.id)
end
end
end
and in /controllers/concerns/email_and_push_helper.rb I have this:
module EmailAndPushHelper
def unaccepted_lex_request(meeting_id)
puts meeting_id
#meeting = Meeting.find(meeting_id)
if(check_settings(#meeting.requestee_id, 'lex_email') == true)
MeetingMailer.unaccepted_mtg_request(#meeting).deliver_later
end
if(check_settings(#meeting.requestee_id, 'lex_push'))
#title = "A Lex request needs your response."
#message = "#{#meeting.requestor.preferred_name} is waiting for your response.'"
#data = {
"status": "ok",
"body": #message,
"title": #title,
"meetingId": #meeting.id
}
PushNotificationWorker.perform_async(#meeting.requestee.id, #title, #message, #data)
end
end
end
When I call the task email_unaccepted_meetings I get NoMethodError (undefined method 'unaccepted_lex_request' for EmailAndPushHelper:Module)
In other rake tasks I am calling Mailer methods, Worker methods, so I'm not totally understanding what is going on here. I've tried requiring the file in scheduler.rake but it gives the same error, which makes me believe that it's accessing the file /controllers/concerns/email_and_push_helper.rb, but can not find the method unaccepted_lex_request for some reason?
Thanks in advance for your help!
To declare module methods you need to define the method with self.
module EmailAndPushHelper
def self.unaccepted_lex_request(meeting_id)
# ...
end
end
Instance methods of a module are only available when you include or extend something else with the module. Not as a method of the module itself.
But this really the wrong place/name for this code. controller/concerns is really where you put mixins that are mixed into your controller. Neither is this really a helper which is also an extremely vague term that also generally means a mixin thats used in the view and sometimes the controller in Rails.
This code actually belongs in a job or a service object.
class EmailAndPushJob < ApplicationJob
queue_as :default
def perform(meeting)
# You will have to solve the NoMethodError here
if check_settings(meeting.requestee_id, 'lex_email')
MeetingMailer.unaccepted_mtg_request(meeting).deliver_later
end
if check_settings(meeting.requestee_id, 'lex_push')
title = "A Lex request needs your response."
message = "#{meeting.requestor.preferred_name} is waiting for your response.'"
PushNotificationWorker.perform_async(meeting.requestee.id, title, message, {
# Use strings or symbols!
"status" => "ok",
"body" => message,
"title" => title,
"meetingId" => meeting.id
})
end
end
end
You also want to use batches in your rake task instead of just .each as that will load every record into memory and crash your server once you have a decent amount of data.
task :email_unaccepted_meetings => :environment do
desc "If meeting time is less than 3 days away, and not accepted, send email to requestee"
meetings = Meeting.where(
status: "Active",
accepted: true,
meeting_time: Time.now..3.days.from_now
).find_each do |meeting|
EmailAndPushJob.perform_now(meeting)
end
end
I solved by adding this to the top of scheduler.rake
require "#{Rails.root}/app/controllers/concerns/email_and_push_helper.rb"
include EmailAndPushHelper
And calling it like this:
`EmailAndPushHelper::unaccepted_lex_request(meeting.id)`
ActiveRecord::RecordNotFound in Sidekiq worker when I save object. I don't use rails callbacks.
I run worker from service, and save object in this service.
class LeadSmsSendingService < Rectify::Command
...initialize params
def send_sms_message
sms_conversation = lead.sms_conversations.find_or_create_by(sms_number: sms_number)
attrs = sms_form.to_hash.symbolize_keys.slice(:body, :direction, :from, :to)
.merge(campaign_id: campaign_id)
sms_message = sms_conversation.sms_messages.build(attrs)
sms_message.to ||= lead.phone
sms_message.body = VariableReplacement.new(lead).render(sms_message.body)
# #todo we need to raise an exception here
return unless sms_message.save
DeliverSmsMessageWorker.perform_in(3.seconds, sms_message.id, 'LeadSmsSendingService')
end
end
class DeliverSmsMessageWorker
include Sidekiq::Worker
sidekiq_options queue: 'priority'
def perform(sms_message_id, from_where="Unknown")
sms_message = SmsMessage.find(sms_message_id)
sms_message.deliver!
rescue StandardError => e
Bugsnag.notify(e) do |report|
# Add information to this report
report.add_tab(:worker, { from_where: from_where.to_s })
end
end
end
Seems that the record has still to be commited, even if it sounds strange because of the 3 seconds delay. Does it work if you increase this delay?
This link could be useful: https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting#cannot-find-modelname-with-id12345
How in Rails 4 using Server Sent Events and listen multiple callbakcs (create and destroy)? For example,
Model:
class Job < ActiveRecord::Base
after_create :notify_job_created
after_destroy :notify_job_destroyed
def self.on_create
Job.connection.execute "LISTEN create_jobs"
loop do
Job.connection.raw_connection.wait_for_notify do |event, pid, job|
yield job
end
end
ensure
Job.connection.execute "UNLISTEN create_jobs"
end
def self.on_destroy
Job.connection.execute "LISTEN destroy_jobs"
loop do
Job.connection.raw_connection.wait_for_notify do |event, pid, job|
yield job
end
end
ensure
Job.connection.execute "UNLISTEN destroy_jobs"
end
def notify_job_created
Job.connection.execute "NOTIFY create_jobs, '#{self.id}'"
end
def notify_job_destroyed
Job.connection.execute "NOTIFY destroy_jobs, '#{self.id}'"
end
end
Controller:
class StreamJobsController < ApplicationController
include ActionController::Live
def index_stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new response.stream
begin
Job.on_create do |id|
job = Job.find(id)
stand = Stand.find(job.stand_id)
t = render_to_string(
partial: 'projects/stand',
formats: [:html],
locals: {stand: stand}
)
sse.write(t, event: 'create')
end
Job.on_destroy do |id|
job = Job.find(id)
sse.write(job.stand_id, event: 'destroy')
end
rescue IOError
# When the client disconnects, we'll get an IOError on write
ensure
sse.close
end
end
end
JS code:
$(function () {
var source = new EventSource('/jobs_stream');
source.addEventListener('create', function(e){
console.log('Create stand:', e.data);
$("table.project-stands.table.table-condensed").find("tbody#stand-list").prepend($.parseHTML(e.data));
});
source.addEventListener('destroy', function(e){
console.log('Destroy stand: ', e.data);
var id = e.data;
$("table.project-stands.table.table-condensed").find("tr#stand_" + id).remove();
});
source.addEventListener('finished', function(e){
console.log('Close:', e.data);
source.close();
});
});
As result, I get only LISTEN create_jobs. What's wrong in my controller? Thanks
I don't think anything is wrong with your controller. Your question is more about Postgres notifications than anything else. Do you have tests for the methods you are using in your ActiveRecord models that ensure the NOTIFY/LISTEN combinations function as they should?
As result, I get only LISTEN create_jobs
What do you mean by that? I don't see any LISTEN stuff coming into the SSE output in the example you have posted. I would really suggest you experiment with the model on it's own to see if the blocking waits you are using etc. function in isolation, and then try putting SSE on top as a transport.
I need to mark a collection of messages at the background (I am using delayed_job gem) since it takes some time on the foreground. So I've created an ActiveJob class MarkMessagesAsReadJob, and passed it user and messages variables in order to mark all of the messages read for user.
// passing the values in the controller
#messages = #conversation.messages
MarkMessagesAsReadJob.perform_later(current_user, #messages)
and in my ActiveJob class, I perform the task.
// MarkMessagesAsReadJob.rb
class MarkMessagesAsReadJob < ActiveJob::Base
queue_as :default
def perform(user, messages)
messages.mark_as_read! :all, :for => user
end
end
However, when I tried to perform the task, I got the error
ActiveJob::SerializationError (Unsupported argument type: ActiveRecord::Associations::CollectionProxy):
I read that we can only pass supported types to the ActiveJob, and I think it can not serialize the CollectionProxy object. How can I workaround/fix this?
PS: I considered
#messages.map { |message| MarkMessagesAsReadJob.perform_later(current_user, message) }
however I think marking them one by one is pretty expensive .
I think the easy way is pass message ids to the perform_later() method, for example:
in controller:
#messages = #conversation.messages
message_ids = #messages.pluck(:id)
MarkMessagesAsReadJob.perform_later(current_user, message_ids)
And use it in ActiveJob:
def perform(user, message_ids)
messages = Message.where(id: message_ids)
messages.mark_as_read! :all, :for => user
end
For larger data sets, passing the message_ids is impractical. Instead, pass the SQL for the messages:
#messages = #conversation.messages
MarkMessagesAsReadJob.perform_later(current_user, #messages.to_sql)
then query them from the job:
class MarkMessagesAsReadJob < ActiveJob::Base
queue_as :default
def perform(user, messages_sql)
messages = Message.find_by_sql(messages_sql)
messages.mark_as_read! :all, :for => user
end
end
Because the job will be executed later, I think we should pass ids as parameter instead a collection
ActiveJob need serialize parameter, and SerializationError will be thrown if parameter type isn't supported
http://api.rubyonrails.org/classes/ActiveJob/SerializationError.html
ex:
#message_ids = #conversation.messages.pluck(:id)
# use string if array is not supported
# #message_ids = #message_ids.join(", ")
MarkMessagesAsReadJob.perform_later(current_user, #message_ids)
then query those messages again and mark it
class MarkMessagesAsReadJob < ActiveJob::Base
queue_as :default
def perform(user, message_ids)
# use string if array is not supported
# message_ids = message_ids.split(",").map(&:to_i)
messages = Message.where(id: message_ids) #change this to something else
messages.mark_as_read! :all, :for => user
end
end
not tested , hope that it will be ok
For my API in RAILS I have programmed a code that basically does the following.
class Api::V1::NameController < ApplicationController
skip_before_filter :verify_authenticity_token
def index
end
def create
# Loading data
data_1_W = params[:data1]
data_2_W = params[:data2]
while len > i
# -Here I do some calculations with data_1_W and data_2_W.
# Its not important to show the code here
end
# -Organizing outputs to obtain only one JSON-
# Its not important to show the code here
# Finally HTTP responses
if check_error == 1
render status: 200, json: {
message: "Succesful data calculation",
data_output: response_hash
}.to_json
end
end
end
To test that everything is working I use the cURL command. I notice that loading the data could be a problem and therefore the code would break.
I want to tell the user that it was an error loading the data for some reason (HTTP response), but I don't know where to put it. If I put and else under my success status it would not print it because the code breaks just starting (instead of sending the correct name - d '#data.json' of the data in cURL I send -d '#dat.json').
The data I am loading is a JSON data {"data1":[{"name1":"value1"},{"name2":number2}...],"data2":[{"name1":"value1"},{"name2":number2...}]}. (This data has 70080 rows with 2 columns if we see it as a table, which I divided into two in my CODE for calculations purposes data_1_W and data_2_W)
Could anyone help me where to put it? more or less like this:
render status: 500, json: {
message: "Error loading the data",
}.to_json
Put it in a rescue block around the code that throws the error.
E.g.
def func
# code that raises exception
rescue SomeException => e
# render 422
end
Since you are working in Rails I'd recommend going the rails way. This means that I would create some kind of service and initialize it in the create action.
Now, within the service you do all you funky stuff (which also allows you to clean this controller and make i look prettier) and the moment a condition is not fulfilled in that service return false. So...
# controllers/api/v1/name_controller.rb
...
def create
meaningful_variable_name = YourFunkyService.new(args)
if meaningful_variable_name.perform # since you are in create then I assume you're creating some kind of resource
#do something
else
render json: {
error: "Your error",
status: error_code, # I don't think you want to return 500. Since you're the one handling it
}
end
end
# services/api/v1/your_funky_service.rb
class Api::V1::YourFunkyService
def initiliaze(params)
#params = params
end
def perfom #call it save if you wish
....
return false if check_error == 1
end
end