em-websocket send() send from one client to another through 2 servers - ruby-on-rails

I have two websocket clients, and I want to exchange information between them.
Let's say I have two instances of socket servers, and 1st is retrieve private information, filter it and send to the second one.
require 'em-websocket'
EM.run do
EM::WebSocket.run(host: '0.0.0.0', port: 19108) do |manager_emulator|
# retrieve information. After that I need to send it to another port (9108)
end
EM::WebSocket.run(host: '0.0.0.0', port: 9108) do |fake_manager|
# I need to send filtered information here
end
end
I've tried to do something, but I got usual dark code and I don't know how to implement this functionality.

I'm not sure how you would do that using EM.
I'm assuming you will need to have the fake_manager listen to an event triggered by the manager_emulator.
It would be quite easy if you'd be using a websocket web-app framework. For instance, on the Plezi web-app framework you could write something like this:
# try the example from your terminal.
# use http://www.websocket.org/echo.html in two different browsers to observe:
#
# Window 1: http://localhost:3000/manager
# Window 2: http://localhost:3000/fake
require 'plezi'
class Manager_Controller
def on_message data
FakeManager_Controller.broadcast :_send, "Hi, fake! Please do something with: #{data}\r\n- from Manager."
true
end
def _send message
response << message
end
end
class FakeManager_Controller
def on_message data
Manager_Controller.broadcast :_send, "Hi, manager! This is yours: #{data}\r\n- from Fake."
true
end
def _send message
response << message
end
end
class HomeController
def index
"use http://www.websocket.org/echo.html in two different browsers to observe this demo in action:\r\n" +
"Window 1: http://localhost:3000/manager\r\nWindow 2: http://localhost:3000/fake\r\n"
end
end
# # optional Redis URL: automatic broadcasting across processes or machines:
# ENV['PL_REDIS_URL'] = "redis://username:password#my.host:6379"
# starts listening with default settings, on port 3000
listen
# Setup routes:
# They are automatically converted to the RESTful route: '/path/(:id)'
route '/manager', Manager_Controller
route '/fake', FakeManager_Controller
route '/', HomeController
# exit terminal to start server
exit
Good Luck!
P.S.
If you're going to keep to EM, you might consider using Redis to push and subscribe to events between the two ports.

I've found a way how to do it through em-websocket gem! You need just define variables outside of eventmachine block. Something like that
require 'em-websocket'
message_sender = nil
EM.run do
# message sender
EM::WebSocket.run(host: '0.0.0.0', port: 19108) do |ws|
ws.onopen { message_sender = ws }
ws.onclose { message_sender = nil }
end
# message receiver
EM::WebSocket.run(host: '0.0.0.0', port: 9108) do |ws|
ws.onmessage { |msg| message_sender.send(msg) if message_sender }
end
end

Related

Websocket can't connect to ActionController::Live - failed with 200

class MessagesController < ApplicationController
include ActionController::Live
def events
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new response.stream
redis = Redis.new
redis.subscribe(redis_channel) do |on|
on.message do |event, data|
sse.write(data, event: 'messages.create')
end
end
#render nothing: true
rescue IOError
# disco bro!
ensure
redis.quit
sse.close
end
end
// also tried without the SSE.new, so just plain response.stream.write(data)
My Javascript is simple easy
var socket = new WebSocket("ws://localhost:3000/events")
socket.onmessage = function (event) {
console.log(event);
}
When i call /events in my browser and send some messages - its outputting. but if we connect the socket with JS we get
WebSocket connection to 'ws://localhost:3000/petra/events' failed:
Error during WebSocket handshake: Unexpected response code: 200
can anybody light me up? Are we missing something, or do i have a wrong understanding of what i'm trying to do.
WebSockets and Server Send Events are different protocols and can't inoperate in the manner you're trying.
The main js API for SSE is EventSource(url) (introductory article).
If you want bidirectional communication you would switch the server side to use WebSockets instead. For ruby on rails, websocket-rails is a popular library to do this.

How to send a keep-alive packet through websocket in ruby on rails

I want to send a
"Keep alive from client"
message every 30 seconds for my websocket connection. Here's what the code that I have in my websocket initializer looks like:
ws = WebSocket::Client::Simple.connect 'wss://bitcoin.toshi.io/'
ws.on :message do |msg|
rawJson = msg.data
message_response = JSON.parse(rawJson)
end
ws.on :open do
ws.send "{\"subscribe\":\"blocks\"}"
end
ws.on :close do |e|
puts "WEBSOCKET HAS CLOSED #{e}"
exit 1
end
ws.on :error do |e|
puts "WEBSOCKET ERROR #{e}"
end
Without any sort of 'keep alive', the connect closes in about 45 seconds. How should I send the 'heart-beat' packet? It seems that the connection is closed by their server, not mine.
You can use Websocket Eventmachine Client gem to send hearbeat:
require 'websocket-eventmachine-client'
EM.run do
ws = WebSocket::EventMachine::Client.connect(:uri => 'wss://bitcoin.toshi.io/')
puts ws.comm_inactivity_timeout
ws.onopen do
puts "Connected"
end
ws.onmessage do |msg, type|
puts "Received message: #{msg}"
end
ws.onclose do |code, reason|
puts "Disconnected with status code: #{code}"
end
EventMachine.add_periodic_timer(15) do
ws.send "{}"
end
end
You can setup timer for EventMachine with EM::add_periodic_timer(interval_in_seconds), and then send your heartbeat with it.
You can use the auto-ping feature (its default and can't be turned off) if you're using Iodine's Websocket client:
require 'iodine/http'
# prevents the Iodine's server from running
Iodine.protocol = :timer
# starts Iodine while the script is still running
Iodine.force_start!
# set pinging to a 40 seconds interval.
Iodine::Http::Websockets.default_timeout = 40
settings = {}
# set's the #on_open event callback.
settings[:on_open] = Proc.new do
write 'sending this connection string.'
end
# set's the #on_message(data) event callback.
settings[:on_message] = Proc.new { |data| puts "Received message: #{data}" }
# connects to the websocket
Iodine::Http.ws_connect 'ws://localhost:8080', settings
It's a fairly basic client, but also easy to manage.
EDIT
Iodine also includes some cookie and custom header's support, as now seen in Iodine's documentation. So it's possible to use different authentication techniques (authentication headers or cookies).

EventMachine::WebSocketClient.connect causes onclose to trigger on server side

There are three parts in my Rails application:
A part which listens for HTML5 web sockets (em-websocket)
a piece of JavaScript which connects to them
and a part which task is to connect to these sockets from inside of the same web application (em-websocket-client) (Yes, I am trying to do some IPC in Phusion Passenger environment)
JavaScript code connects fine and Web sockets server is happy with such a client, but when I connect from em-websocket-client a strange thing is happening: onclose handler is being called without calling onopen, and moreover - it is called for a socket which had been opened by web browser, not em-websocket-client.
The same em-websocket-client code, when executed in separate Ruby script through command line, works as planned. Here is the sample of em-websocket-client code:
require 'em-websocket-client'
class WebSocketsClient
def initialize
Thread.new do
log 'In a thread'
EventMachine.run do
log 'EM run'
#conn = EventMachine::WebSocketClient.connect("ws://localhost:5050?user_id=1&page_token=JYUTbfYDTTliglififi")
#conn.callback do
log 'Callback'
#conn.send_msg({ message_type: 'phone_call', user_id: 1, order_id: 1}.to_json)
#conn.close_connection
end
#conn.errback do |e|
log 'Errback'
puts "Got error: #{e}"
end
#conn.stream do |msg|
#log 'Stream'
#puts "<#{msg}>"
#if msg.data == 'done'
# #conn.close_connection
#end
end
#conn.disconnect do
puts 'gone'
EventMachine::stop_event_loop
end
end
end
end
def send_phone_call(order_id, user_id)
#conn.send_msg({ message_type: 'phone_call', user_id: user_id, order_id: order_id}.to_json)
end
def log(text)
puts "WebSocketsClient: #{text}\n"
end
end
WebSocketsClient.new
onclose on server side is being callsd as soon as EventMachine::WebSocketClient.connect is executed on client side. It doesn't even come to #conn.disconnect call.
One more thing I can conjecture is that this behaviour is due to using same EventMachine mechanism by server and client inside of same Rails application.

Calling amqp message queque is not asynchronous

Here is my code:
class ParsepdfClient
#!/usr/bin/env ruby
# encoding: utf-8
require 'amqp'
require "rubygems"
require 'mq'
def self.test
EventMachine.run do
connection = AMQP.connect(:host => '127.0.0.1')
puts "Connected to AMQP broker. Running #{AMQP::VERSION} version of the gem..."
channel = AMQP::Channel.new(connection)
queue = channel.queue("amqpgem.examples.helloworld", :auto_delete => true)
exchange = channel.direct("")
queue.subscribe do |payload|
sleep(1.minutes)
puts "Received a message: #{payload}. Disconnecting..."
connection.close { EventMachine.stop }
end
exchange.publish "Hello, world!", :routing_key => queue.name
end
end
end
I'm using Rabbitmq as a broker with rails amqp gem. Now I'm calling :
ParsepdfClient.test from a controller.
From my understanding my call shouldn't sleep for a minute but it waits for a minute and it outputs
"Received a message: Hello, world!. Disconnecting..."
And then it executes rest of my code of controller. Shouldn't the call be asynchronous?
If not how can I make it asynchronous?
What I mean is that shouldn't it execute the rest of the code of my controller after outputting
Connected to AMQP broker. Running #{AMQP::VERSION} version of the gem...
I managed to implement Rabbitmq in rails successfully with the help of my senior. If any one is having any issue I have written a blog post here.

Work with two separate redis instances with sidekiq?

Good afternoon,
I have two separate, but related apps. They should both have their own background queues (read: separate Sidekiq & Redis processes). However, I'd like to occasionally be able to push jobs onto app2's queue from app1.
From a simple queue/push perspective, it would be easy to do this if app1 did not have an existing Sidekiq/Redis stack:
# In a process, far far away
# Configure client
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://redis.example.com:7372/12', :namespace => 'mynamespace' }
end
# Push jobs without class definition
Sidekiq::Client.push('class' => 'Example::Workers::Trace', 'args' => ['hello!'])
# Push jobs overriding default's
Sidekiq::Client.push('queue' => 'example', 'retry' => 3, 'class' => 'Example::Workers::Trace', 'args' => ['hello!'])
However given that I would already have called a Sidekiq.configure_client and Sidekiq.configure_server from app1, there's probably a step in between here where something needs to happen.
Obviously I could just take the serialization and normalization code straight from inside Sidekiq and manually push onto app2's redis queue, but that seems like a brittle solution. I'd like to be able to use the Client.push functionality.
I suppose my ideal solution would be someting like:
SidekiqTWO.configure_client { remote connection..... }
SidekiqTWO::Client.push(job....)
Or even:
$redis_remote = remote_connection.....
Sidekiq::Client.push(job, $redis_remote)
Obviously a bit facetious, but that's my ideal use case.
Thanks!
So one thing is that According to the FAQ, "The Sidekiq message format is quite simple and stable: it's just a Hash in JSON format." Emphasis mine-- I don't think sending JSON to sidekiq is too brittle to do. Especially when you want fine-grained control around which Redis instance you send the jobs to, as in the OP's situation, I'd probably just write a little wrapper that would let me indicate a Redis instance along with the job being enqueued.
For Kevin Bedell's more general situation to round-robin jobs into Redis instances, I'd imagine you don't want to have the control of which Redis instance is used-- you just want to enqueue and have the distribution be managed automatically. It looks like only one person has requested this so far, and they came up with a solution that uses Redis::Distributed:
datastore_config = YAML.load(ERB.new(File.read(File.join(Rails.root, "config", "redis.yml"))).result)
datastore_config = datastore_config["defaults"].merge(datastore_config[::Rails.env])
if datastore_config[:host].is_a?(Array)
if datastore_config[:host].length == 1
datastore_config[:host] = datastore_config[:host].first
else
datastore_config = datastore_config[:host].map do |host|
host_has_port = host =~ /:\d+\z/
if host_has_port
"redis://#{host}/#{datastore_config[:db] || 0}"
else
"redis://#{host}:#{datastore_config[:port] || 6379}/#{datastore_config[:db] || 0}"
end
end
end
end
Sidekiq.configure_server do |config|
config.redis = ::ConnectionPool.new(:size => Sidekiq.options[:concurrency] + 2, :timeout => 2) do
redis = if datastore_config.is_a? Array
Redis::Distributed.new(datastore_config)
else
Redis.new(datastore_config)
end
Redis::Namespace.new('resque', :redis => redis)
end
end
Another thing to consider in your quest to get high-availability and fail-over is to get Sidekiq Pro which includes reliability features: "The Sidekiq Pro client can withstand transient Redis outages. It will enqueue jobs locally upon error and attempt to deliver those jobs once connectivity is restored." Since sidekiq is for background processes anyway, a short delay if a Redis instance goes down should not affect your application. If one of your two Redis instances goes down and you're using round robin, you've still lost some jobs unless you're using this feature.
As carols10cents says its pretty simple but as I always like to encapsulate the capability and be able to reuse it in other projects I updated an idea from a blog from Hotel Tonight. This following solution improves upon Hotel Tonight's that does not survive Rails 4.1 & Spring preloader.
Currently I make do with adding the following files to lib/remote_sidekiq/:
remote_sidekiq.rb
class RemoteSidekiq
class_attribute :redis_pool
end
remote_sidekiq_worker.rb
require 'sidekiq'
require 'sidekiq/client'
module RemoteSidekiqWorker
def client
pool = RemoteSidekiq.redis_pool || Thread.current[:sidekiq_via_pool] || Sidekiq.redis_pool
Sidekiq::Client.new(pool)
end
def push(worker_name, attrs = [], queue_name = "default")
client.push('args' => attrs, 'class' => worker_name, 'queue' => queue_name)
end
end
You need to create a initializer that sets redis_pool
config/initializers/remote_sidekiq.rb
url = ENV.fetch("REDISCLOUD_URL")
namespace = 'primary'
redis = Redis::Namespace.new(namespace, redis: Redis.new(url: url))
RemoteSidekiq.redis_pool = ConnectionPool.new(size: ENV['MAX_THREADS'] || 6) { redis }
EDIT by Aleks:
In never versions of sidekiq, instead of lines:
redis = Redis::Namespace.new(namespace, redis: Redis.new(url: url))
RemoteSidekiq.redis_pool = ConnectionPool.new(size: ENV['MAX_THREADS'] || 6) { redis }
use lines:
redis_remote_options = {
namespace: "yournamespace",
url: ENV.fetch("REDISCLOUD_URL")
}
RemoteSidekiq.redis_pool = Sidekiq::RedisConnection.create(redis_remote_options)
You can then simply the include RemoteSidekiqWorker module wherever you want. Job done!
**** FOR MORE LARGER ENVIRONMENTS ****
Adding in RemoteWorker Models adds extra benefits:
You can reuse the RemoteWorkers everywhere including the system that has access to the target sidekiq workers. This is transparent to the caller. To use the "RemoteWorkers" form within the target sidekiq system simply do not use an initializer as it will default to using the local Sidekiq client.
Using RemoteWorkers ensure correct arguments are always sent in (the code = documentation)
Scaling up by creating more complicated Sidekiq architectures is transparent to the caller.
Here is an example RemoteWorker
class RemoteTraceWorker
include RemoteSidekiqWorker
include ActiveModel::Model
attr_accessor :message
validates :message, presence: true
def perform_async
if valid?
push(worker_name, worker_args)
else
raise ActiveModel::StrictValidationFailed, errors.full_messages
end
end
private
def worker_name
:TraceWorker.to_s
end
def worker_args
[message]
end
end
I came across this and ran into some issues because I'm using ActiveJob, which complicates how messages are read out of the queue.
Building on ARO's answer, you will still need the redis_pool setup:
remote_sidekiq.rb
class RemoteSidekiq
class_attribute :redis_pool
end
config/initializers/remote_sidekiq.rb
url = ENV.fetch("REDISCLOUD_URL")
namespace = 'primary'
redis = Redis::Namespace.new(namespace, redis: Redis.new(url: url))
RemoteSidekiq.redis_pool = ConnectionPool.new(size: ENV['MAX_THREADS'] || 6) { redis }
Now instead of the worker we'll create an ActiveJob Adapter to queue the request:
lib/active_job/queue_adapters/remote_sidekiq_adapter.rb
require 'sidekiq'
module ActiveJob
module QueueAdapters
class RemoteSidekiqAdapter
def enqueue(job)
#Sidekiq::Client does not support symbols as keys
job.provider_job_id = client.push \
"class" => ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper,
"wrapped" => job.class.to_s,
"queue" => job.queue_name,
"args" => [ job.serialize ]
end
def enqueue_at(job, timestamp)
job.provider_job_id = client.push \
"class" => ActiveJob::QueueAdapters::SidekiqAdapter::JobWrapper,
"wrapped" => job.class.to_s,
"queue" => job.queue_name,
"args" => [ job.serialize ],
"at" => timestamp
end
def client
#client ||= ::Sidekiq::Client.new(RemoteSidekiq.redis_pool)
end
end
end
end
I can use the adapter to queue the events now:
require 'active_job/queue_adapters/remote_sidekiq_adapter'
class RemoteJob < ActiveJob::Base
self.queue_adapter = :remote_sidekiq
queue_as :default
def perform(_event_name, _data)
fail "
This job should not run here; intended to hook into
ActiveJob and run in another system
"
end
end
I can now queue the job using the normal ActiveJob api. Whatever app reads this out of the queue will need to have a matching RemoteJob available to perform the action.

Resources