How in Rails 4 using Server Sent Events and listen multiple callbakcs (create and destroy)? For example,
Model:
class Job < ActiveRecord::Base
after_create :notify_job_created
after_destroy :notify_job_destroyed
def self.on_create
Job.connection.execute "LISTEN create_jobs"
loop do
Job.connection.raw_connection.wait_for_notify do |event, pid, job|
yield job
end
end
ensure
Job.connection.execute "UNLISTEN create_jobs"
end
def self.on_destroy
Job.connection.execute "LISTEN destroy_jobs"
loop do
Job.connection.raw_connection.wait_for_notify do |event, pid, job|
yield job
end
end
ensure
Job.connection.execute "UNLISTEN destroy_jobs"
end
def notify_job_created
Job.connection.execute "NOTIFY create_jobs, '#{self.id}'"
end
def notify_job_destroyed
Job.connection.execute "NOTIFY destroy_jobs, '#{self.id}'"
end
end
Controller:
class StreamJobsController < ApplicationController
include ActionController::Live
def index_stream
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new response.stream
begin
Job.on_create do |id|
job = Job.find(id)
stand = Stand.find(job.stand_id)
t = render_to_string(
partial: 'projects/stand',
formats: [:html],
locals: {stand: stand}
)
sse.write(t, event: 'create')
end
Job.on_destroy do |id|
job = Job.find(id)
sse.write(job.stand_id, event: 'destroy')
end
rescue IOError
# When the client disconnects, we'll get an IOError on write
ensure
sse.close
end
end
end
JS code:
$(function () {
var source = new EventSource('/jobs_stream');
source.addEventListener('create', function(e){
console.log('Create stand:', e.data);
$("table.project-stands.table.table-condensed").find("tbody#stand-list").prepend($.parseHTML(e.data));
});
source.addEventListener('destroy', function(e){
console.log('Destroy stand: ', e.data);
var id = e.data;
$("table.project-stands.table.table-condensed").find("tr#stand_" + id).remove();
});
source.addEventListener('finished', function(e){
console.log('Close:', e.data);
source.close();
});
});
As result, I get only LISTEN create_jobs. What's wrong in my controller? Thanks
I don't think anything is wrong with your controller. Your question is more about Postgres notifications than anything else. Do you have tests for the methods you are using in your ActiveRecord models that ensure the NOTIFY/LISTEN combinations function as they should?
As result, I get only LISTEN create_jobs
What do you mean by that? I don't see any LISTEN stuff coming into the SSE output in the example you have posted. I would really suggest you experiment with the model on it's own to see if the blocking waits you are using etc. function in isolation, and then try putting SSE on top as a transport.
Related
I've got Sidekiq job (SyncProductsWorker) which fires a class Imports::SynchronizeProducts responsible for a few external API calls.
module Imports
class SyncProductsWorker
include Sidekiq::Worker
sidekiq_options queue: 'imports_sync'
def perform(list)
::Imports::SynchronizeProducts.new(list).call
end
end
end
The Imports::SynchronizeProducts class gives an array of monads results with some comments e.g.
=> [Failure("999999 Product code is not valid"), Failure(" 8888889 Product code is not valid")]
I would like to catch these results to display them on FE. Is it possible to do so? If I do something like:
def perform(list)
response = ::Imports::SynchronizeProducts.new(list).call
response
end
And then inside of the controller:
def create
response = ::Imports::SyncProductsWorker.perform_async(params[:product_codes])
render json: { result: response, head: :ok }
end
I'll have some number as a result
=> "df3615e8efc56f8a062ba1c2"
I don't believe what you want is possible.
https://github.com/mperham/sidekiq/issues/3532
The return value will be GC'd like any other unused data in a Ruby process. Jobs do not have a "result" in Sidekiq and Sidekiq does nothing with the value.
You would need some sort of model instead that keeps track of your background tasks. This is off the cuff but should give you an idea.
EG
# #attr result [Array]
# #attr status [String] Values of 'Pending', 'Error', 'Complete', etc..
class BackgroundTask < ActiveRecord
attr_accessor :product_codes
after_create :enqueue
def enqueue
::Imports::SyncProductsWorker.perform_async(product_codes, self.id)
end
end
def perform(list, id)
response = ::Imports::SynchronizeProducts.new(list).call
if (response.has_errors?)
BackgroundTask.find(id).update(status: 'Error', result: response)
else
BackgroundTask.find(id).update(status: 'Complete', result: response)
end
end
Then just use the BackgroundTask model for your frontend display.
ActiveRecord::RecordNotFound in Sidekiq worker when I save object. I don't use rails callbacks.
I run worker from service, and save object in this service.
class LeadSmsSendingService < Rectify::Command
...initialize params
def send_sms_message
sms_conversation = lead.sms_conversations.find_or_create_by(sms_number: sms_number)
attrs = sms_form.to_hash.symbolize_keys.slice(:body, :direction, :from, :to)
.merge(campaign_id: campaign_id)
sms_message = sms_conversation.sms_messages.build(attrs)
sms_message.to ||= lead.phone
sms_message.body = VariableReplacement.new(lead).render(sms_message.body)
# #todo we need to raise an exception here
return unless sms_message.save
DeliverSmsMessageWorker.perform_in(3.seconds, sms_message.id, 'LeadSmsSendingService')
end
end
class DeliverSmsMessageWorker
include Sidekiq::Worker
sidekiq_options queue: 'priority'
def perform(sms_message_id, from_where="Unknown")
sms_message = SmsMessage.find(sms_message_id)
sms_message.deliver!
rescue StandardError => e
Bugsnag.notify(e) do |report|
# Add information to this report
report.add_tab(:worker, { from_where: from_where.to_s })
end
end
end
Seems that the record has still to be commited, even if it sounds strange because of the 3 seconds delay. Does it work if you increase this delay?
This link could be useful: https://github.com/mperham/sidekiq/wiki/Problems-and-Troubleshooting#cannot-find-modelname-with-id12345
How can I send with javascript a global message to all of our subscribed websocket connections without the need of a channel etc. (like the ping message that the actioncable is sending by default globally to all open connections)?
As far as I know, you cannot do it directly from JavaScript without channels (it needs to go over Redis first).
I would suggest you do that as a normal post action, and then send the message in Rails.
I would do something like this:
JavaScript:
$.ajax({type: "POST", url: "/notifications", data: {notification: {message: "Hello world"}}})
Controller:
class NotificationsController < ApplicationController
def create
ActionCable.server.broadcast(
"notifications_channel",
message: params[:notification][:message]
)
end
end
Channel:
class NotificationsChannel < ApplicationCable::Channel
def subscribed
stream_from("notifications_channel", coder: ActiveSupport::JSON) do |data|
# data => {message: "Hello world"}
transmit data
end
end
end
Listen JavaScript:
App.cable.subscriptions.create(
{channel: "NotificationsChannel"},
{
received: function(json) {
console.log("Received notification: " + JSON.stringify(json))
}
}
)
I am using for the first time redis to put chat functionality in my rails app, following this
I have in my javascript`
$(document).ready ->
source = new EventSource('/messages/events')
source.addEventListener 'messages.create', (e) ->
message = $.parseJSON(e.data).message
console.log(message)
$(".chat-messages").append "#some code"
and in my message controller
def create
response.headers["Content-Type"] = "text/javascript"
attributes = params.require(:message).permit(:content, :sender_id, :sendee_id)
#message = Message.create(attributes)
respond_to do |format|
format.js { render 'messages/create.js.erb' }
end
$redis.publish('messages.create', #message.to_json)
end
def events
response.headers["Content-Type"] = "text/event-stream"
redis = Redis.new
redis.subscribe('messages.*') do |on|
on.message do |pattern, event, data|
response.stream.write("event: #{event}\n")
response.stream.write("data: #{data}\n\n")
end
end
rescue IOError
logger.info "Stream closed"
ensure
redis.quit
response.stream.close
end
The problem is that first, nothing is logged in my console, and second I get numbers of random ConnectionTimeoutError errors. Some one hava an idea what's going on
Pre-Reqs:
Ruby 2.0.0+
Rails 4.0.0+
Redis
Puma
Initializer:
Create a redis.rb initializer file in the config/initializers directory, globalizing an instance of redis. It's also a good idea to set up a heartbeat thread (Anything from 5 seconds to 5 minutes is okay, depending on your requirements):
$redis = Redis.new
heartbeat_thread = Thread.new do
while true
$redis.publish("heartbeat","thump")
sleep 15.seconds
end
end
at_exit do
# not sure this is needed, but just in case
heartbeat_thread.kill
$redis.quit
end
Controller:
You need to add two methods to your ChatController, pub and sub. The role of pub is to publish chat events and messages to redis, and sub to subscribe to these events. It should look something like this:
class ChatController < ApplicationController
include ActionController::Live
skip_before_filter :verify_authenticity_token
def index
end
def pub
$redis.publish 'chat_event', params[:chat_data].to_json
render json: {}, status: 200
end
def sub
response.headers["Content-Type"] = "text/event-stream"
redis = Redis.new
redis.subscribe(['chat_event']) do |on|
on.message do |event, data|
response.stream.write "event: #{event}\ndata: #{data}\n\n"
end
end
rescue IOError
logger.info "Stream Closed"
ensure
redis.quit
response.stream.close
end
end
In your routes, make pub a POST and sub a GET, and match the path to something like /chat/publish and /chat/subscribe.
Coffeescript / Javascript:
Assuming your actual webpage for the chat app is at /chat, you need to write some Javascript to actually send and receive chat messages.
For ease of understanding, let's suppose your webpage only has a textbox and a button. Hitting the button should publish the content of the textbox to the chat stream, we can do that using AJAX:
$('button#send').click (e) ->
e.preventDefault()
$.ajax '/chat/publish',
type: 'POST'
data: {
chat_data: {
message: $("input#message").val(),
timestamp: $.now()
}
}
error: (jqXHR, textStatus, errorThrown) ->
console.log "Failed: " + textStatus
success: (data, textStatus, jqXHR) ->
console.log "Success: " + textStatus
Now, you need to be able to subscribe and receive the chat messages as well. You need to use EventSource for this. Using EventSource, open a channel for SSE so that you can receive events, and use that data to update the view. In this example, we will only log them to the javascript console.
The code should look something like this:
$(document).ready ->
source = new EventSource('/chat/subscribe')
source.addEventListener 'chat_event', (e) ->
console.log(e.data)
Enable Parallel Requests:
In your development environment, you'll have to enable parallel requests by adding these two lines to your config/environments/development.rb:
config.preload_frameworks = true
config.allow_concurrency = true
Now fire up your browser, browse to /chat and see the magic. When you type a message and click the button, the message will be received by all instances of that webpage.
Well this is how you make a basic chat application in rails using ActionController::Live and Redis. The final code would obviously be very different depending on your requirements but this should get you started.
Some more resources you should check out:
Tender Love Making - Is it Live?
Railscasts - #401 - ActionController::Live
SitePoint - Mini Chat with Rails and SSEs
Github - mohanraj-ramanujam / live-stream
Thoughtbot - Chat Example using SSEs
Although I've not used redis in this capacity (a mediator for "live" data), I managed to get this functionality working with Pusher
Redis
I don't understand how you're keeping the connection open between your app & Redis. You'll need some sort of web socket or concurrent-connection tech in place to handle the updates -- and to my knowledge, Redis does not handle this directly
If you look at this example, it uses a server called Goliath to handle the asynchronous connectivity:
When tiny-chat connects to the server it sends a GET request to
/subscribe/everyone where everyone is the name of the channel and with
the “Accept” header set to text/event-stream. The streaming middleware
(above) receives this request and subscribes to a redis Pub/Sub
channel. Since Goliath is non-blocking multiple clients can be
listening for events without tying up a Heroku dyno. The payload of a
server sent event looks like this:
That basically uses Middleware to connect you to the redis server -- allowing you to receive updates as required
Code
Although I can't pinpoint any errors specifically, I can give you some code we're using (using Pusher):
#config/initializers/pusher.rb
Pusher.url = ENV["PUSHER_URL"]
Pusher.app_id = ENV["PUSHER_APP_ID"]
Pusher.key = ENV["PUSHER_KEY"]
Pusher.secret = ENV["PUSHER_SECRET"]
#app/controllers/messages_controller.rb
def send_message
id = params[:id]
message = Message.find(id).broadcast!
public_key = self.user.public_key
Pusher['private-user-' + public_key].trigger('message_sent', {
message: "Message Sent"
})
end
#app/views/layouts/application.html.erb
<%= javascript_include_tag "http://js.pusher.com/2.1/pusher.min.js" %>
#app/assets/javascripts/application.js
$(document).ready(function(){
#Pusher
pusher = new Pusher("************",
cluster: 'eu'
)
channel = pusher.subscribe("private-user-#{gon.user}")
channel.bind "multi_destroy", (data) ->
alert data.message
channel.bind "message_sent", (data) ->
alert data.message
});
I have the following code:
list_entities = [{:phone => '0000000000', :name => 'Test', :"#i:type => '1'},{:phone => '1111111111', :name => 'Demo', :"#i:type => '1'}]
list_entities.each do |list_entity|
phone_contact = PhoneContact.create(list_entity.except(:"#i:type"))
add_record_response = api.add_record_to_list(phone_contact, "API Test")
if add_record_response[:add_record_to_list_response][:return][:list_records_inserted] != '0'
phone_contact.update(:loaded_at => Time.now)
end
end
This code is taking an array of hashes and creating a new phone_contact for each one. It then makes an api call (add_record_response) to do something with that phone_contact. If that api call is successful, it updates the loaded_at attribute for that specific phone_contact. Then it starts the loop over.
I am allowed something like 7200 api calls per hour with this service - However, I'm only able to make about 1 api call every 4 seconds right now.
Any thoughts on how I could speed this code block up to make faster api calls?
I would suggest using a thread pool. You can define a unit of work to be done and the number of threads you want to process the work on. This way you can get around the bottleneck of waiting for the server to response on each request. Maybe try something like (disclaimer: this was adapted from http://burgestrand.se/code/ruby-thread-pool/)
require 'thread'
class Pool
def initialize(size)
#size = size
#jobs = Queue.new
#pool = Array.new(#size) do |i|
Thread.new do
Thread.current[:id] = i
catch(:exit) do
loop do
job, args = #jobs.pop
job.call(*args)
end
end
end
end
end
def schedule(*args, &block)
#jobs << [block, args]
end
def shutdown
#size.times do
schedule { throw :exit }
end
#pool.map(&:join)
end
end
p = Pool.new(4)
list_entries.do |list_entry|
p.schedule do
phone_contact = PhoneContact.create(list_entity.except(:"#i:type"))
add_record_response = api.add_record_to_list(phone_contact, "API Test")
if add_record_response[:add_record_to_list_response][:return][:list_records_inserted] != '0'
phone_contact.update(:loaded_at => Time.now)
end
puts "Job #{i} finished by thread #{Thread.current[:id]}"
end
at_exit { p.shutdown }
end