I've got a ruby script that opens a connection to Apple's push server and sends all the pending notifications. I can't see any reason why, but I get broken pipe errors when Apple disconnects my script. I've written my script to accomodate this happening, but I would rather just find out why it's happening so I can avoid it in the first place.
It doesn't consistently disconnect on a specific notification. It doesn't disconnect at a certain byte transfer size. Everything appears to be sporadic. Are there certain limitations to the data transfer or payload count you can send on a single connection? Seeing people's solutions that hold one connection open all the time, I would assume that isn't the issue. I've seen the connection drop after 3 notifications, and I've seen it drop after 14 notifications. I've never seen it make it past 14.
Has anyone else experienced this type of problem? How can this be handled?
The problem was caused by sending an invalid device token to the APNS server. In this specific case it was a development token. When an invalid device token is sent to APNS, it disconnects the socket. This can cause some headaches, and has been addressed by Apple as being something they are going to address in future updates.
I had the same issue for a bit and did two things to tackle it:
Put some auto-reconnect logic in place: I try to keep my connection for as long as possible but Apple will disconnect you every now and then. Be prepared to handle this.
Move to the enhanced interface: Using the simple interface (that's what the APNS gem and many others use) errors will trigger disconnection without any feedback. If you switch to the enhanced format you will receive an integer back every time something happens. Bad tokens will result in a 8 being returned, and I use this to remove the device from my database.
Here's my current connection code, using EventMachine:
module Apns
module SocketHandler
def initialize(wrapper)
#wrapper = wrapper
end
def post_init
start_tls(:cert_chain_file => #wrapper.pem_path,
:private_key_file => #wrapper.rsa_path,
:verify_peer => false)
end
def receive_data(data)
#wrapper.read_data!(data)
end
def unbind
#wrapper.connection_closed!
end
def write(data)
begin
send_data(data)
rescue => exc
#wrapper.connection_error!(exc)
end
end
def close!
close_connection
end
end
class Connection
attr_reader :pem_path, :rsa_path
def initialize(host, port, credentials_path, monitoring, read_data_handler)
setup_credentials(credentials_path)
#monitoring = monitoring
#host = host
#port = port
#read_data_handler = read_data_handler
open_connection!
end
def write(data)
#connection.write(data)
end
def open?
#status == :open
end
def connection_closed!
#status = :closed
end
def connection_error!(exception)
#monitoring.inform_exception!(exception, self)
#status = :error
end
def close!
#connection.close!
end
def read_data!(data)
#read_data_handler.call(data)
end
private
def setup_credentials(credentials_path)
#pem_path = "#{credentials_path}.pem"
#rsa_path = "#{credentials_path}.rsa"
raise ArgumentError.new("#{credentials_path}.pem and #{credentials_path}.rsa must exist!") unless (File.exists?(#pem_path) and File.exists?(#rsa_path))
end
def open_connection!
#connection = EventMachine.connect(#host, #port, SocketHandler, self)
#status = :open
end
end
end
end
end
It separates writes and reads in the connection, using the ID field in the notification to correlated notifications I send with feedback I receive.
Related
I followed this tutorial to create an action cable broadcast but it's not quite working as expected. The channel streams and the web app subscribes successfully, but messages broadcasted from the sidekiq background job are only displayed after refreshing the page. Using the same command on the console does result in an immediate update to the page.
When looking at the frames in chrome's developer mode, I cannot see the broadcasted messages from the background job but can immediately see the ones sent by the console. However, I can confirm that the sidekiq background job is broadcasting those messages somewhere since they do show up upon refresh; however, I don't know where they are being queued.
Are there any additional configuration changes needed to keep the messages from the background job from being queued somewhere? Are there any typos or errors in my code that could be causing this?
Action Cable Broadcast message:
ActionCable.server.broadcast "worker_channel", {html:
"<div class='alert alert-success alert-block text-center'>
Market data retrieval complete.
</div>"
}
smart_worker.rb: -- This is called as perform_async from the controller's action
class SmartWorker
include Sidekiq::Worker
include ApplicationHelper
sidekiq_options retry: false
def perform
ActionCable.server.broadcast "worker_channel", {html:
"<div class='alert alert-success alert-block text-center'>
Market data retrieval complete.
</div>"
}
end
connection.rb:
module ApplicationCable
class Connection < ActionCable::Connection::Base
identified_by :current_user
def connect
self.current_user = current_user #find_verified_user ignored until method implemented correctly and does not always return unauthorized
end
private
def find_verified_user
if current_user = User.find_by(id: cookies.signed[:user_id])
current_user
else
reject_unauthorized_connection
end
end
end
end
worker_channel:
class WorkerChannel < ApplicationCable::Channel
def subscribed
stream_from "worker_channel"
end
def unsubscribed
end
end
worker.js:
App.notifications = App.cable.subscriptions.create('WorkerChannel', {
connected: function() {
console.log('message connected');
},
disconnected: function() {},
received: function(data) {
console.log('message recieved');
$('#notifications').html(data.html);
}
});
cable.yml
development:
adapter: redis
url: redis://localhost:6379/1
test:
adapter: async
production:
adapter: redis
url: <%= ENV.fetch("REDIS_URL") { "redis://localhost:6379/1" } %>
channel_prefix: smarthost_production
Also added
to the view but that didn't make a difference.
I'm not sure this is the entire explanation but this is what I have observed through further testing:
After multiple server restarts, the broadcast started working and would log as expected in the development logger. Console messages where still hit or miss, so I added some additional identifiers to the broadcasted messages and identified that they were being broadcasted before the loading of the next page was completed. This caused two things:
1) A quick flashing of flash messages triggered by the broadcast (in what was perceived to be the old page - i.e. only works after a refresh)
2) A lack of or inconsistent behavior in the browser console: Because the sidekiq worker job finished so quick, sometimes even before the browser started rendering the new page, I believe the console messages are being reset by the page loading actions and are therefore not visible when you check the logs (or even if you stare at it for a while).
It seems as though this is working as expected, and is simply working to quickly in the local environment which makes it seem as though it's not working as intended.
ActionChannel normally does not queue messages and those broadcasted when there's no subscriber should be lost. Observed behaviour can happen if notification actually comes later than you expect.
I'd check:
Run entire job in console, not just notification, and see if it's running slow
Check sidekiq queues latency
Add logging before/after notification in job and check logs if the job is actually run successfully
I'm writing a Rails application that uses WebSockets to communicate with other machines (no browser and client side logic in this process). I have a channel:
class MachinesChannel < ApplicationCable::Channel
def subscribed
...
end
def unsubscribed
...
end
def handle_messages
...
end
end
To receive the data the only way I know about is the JavaScript client:
ActionCable.createConsumer('/cable').subscriptions.create 'MachinesChannel',
received: (message) ->
#perform('handle_messages')
I can call server side methods from JS via #perform() method.
Is there any way to omit the JS part and somehow directly handle the incoming data in MachinesChannel?
The ideal situation would be to have the handle_messages method accept a data argument and have this metod called on incoming data.
After looking into ActionCable source code I got the following solution. You just have to create a method in MachinesChannel that you want to be called, e.g. handle_messages(data). Then, in the client that connects to your websocket, you need to send a message in the following format (example in ruby):
id = { channel: 'MachinesChannel' }
ws = WebSocket::Client::Simple.connect(url)
ws.send(JSON.generate(command: 'message', identifier: JSON.generate(id), data: JSON.generate(action: 'handle_messages', foo: 'bar', biz: 'baz')))
action has to be the name of the method you want to be called in MachinesChannel. The rest of key-values are whatever you want. This the date you can receive in the ActionCable channel.
Recently a gem action_cable_client has been release which seems exactly perfect for this kind of usage. I haven't used it, so I don't know how it really works.
Instead of:
def handle_messages
...
end
This works for me:
def receive(data)
puts data
...
end
I've used Heroku tutorial to implement websockets.
It works properly with Thin, but does not work with Unicorn and Puma.
Also there's an echo message implemented, which responds to client's message. It works properly on each server, so there are no problems with websockets implementation.
Redis setup is also correct (it catches all messages, and executes the code inside subscribe block).
How does it work now:
On server start, an empty #clients array is initialized. Then new Thread is started, which is listening to Redis and which is intended to send that message to corresponding user from #clients array.
On page load, new websocket connection is created, it is stored in #clients array.
If we receive the message from browser, we send it back to all clients connected with the same user (that part is working properly on both Thin and Puma).
If we receive the message from Redis, we also look up for all user's connections stored in #clients array.
This is where weird thing happens:
If running with Thin, it finds connections in #clients array and sends the message to them.
If running with Puma/Unicorn, #clients array is always empty, even if we try it in that order (without page reload or anything):
Send message from browser -> #clients.length is 1, message is delivered
Send message via Redis -> #clients.length is 0, message is lost
Send message from browser -> #clients.length is still 1, message is delivered
Could someone please clarify me what am I missing?
Related config of Puma server:
workers 1
threads_count = 1
threads threads_count, threads_count
Related middleware code:
require 'faye/websocket'
class NotificationsBackend
def initialize(app)
#app = app
#clients = []
Thread.new do
redis_sub = Redis.new
redis_sub.subscribe(CHANNEL) do |on|
on.message do |channel, msg|
# logging #clients.length from here will always return 0
# [..] retrieve user
send_message(user.id, { message: "ECHO: #{event.data}"} )
end
end
end
end
def call(env)
if Faye::WebSocket.websocket?(env)
ws = Faye::WebSocket.new(env, nil, {ping: KEEPALIVE_TIME })
ws.on :open do |event|
# [..] retrieve current user
if user
# add ws connection to #clients array
else
# close ws
end
end
ws.on :message do |event|
# [..] retrieve current user
Redis.current.publish({user_id: user.id, { message: "ECHO: #{event.data}"}} )
end
ws.rack_response
else
#app.call(env)
end
end
def send_message user_id, message
# logging #clients.length here will always return correct result
# cs = all connections which belong to that client
cs.each { |c| c.send(message.to_json) }
end
end
Unicorn (and apparently puma) both start up a master process and then fork one or more workers. fork copies (or at least presents the illusion of copying - an actual copy usually only happens as you write to pages) your entire process but only the thread that called fork exists in the new process.
Clearly your app is being initialised before being forked - this is normally done so that workers can start quickly and benefit from copy on write memory savings. As a consequence your redis checking thread is only running in the master process whereas #clients is being modified in the child process.
You can probably work around this by either deferring the creation of your redis thread or disabling app preloading, however you should be aware that your setup will prevent you from scaling beyond a single worker process (which with puma and a thread friendly JVM like jruby would be less of a constraint)
Just in case somebody will face the same problem, here are two solutions I have come up with:
1. Disable app preloading (this was the first solution I have come up with)
Simply remove preload_app! from the puma.rb file. Therefore, all threads will have their own #clients variable. And they will be accessible by other middleware methods (like call etc.)
Drawback: you will lose all benefits of app preloading. It is OK if you have only 1 or 2 workers with a couple of threads, but if you need a lot of them, then it's better to have app preloading. So I continued my research, and here is another solution:
2. Move thread initialization out of initialize method (this is what I use now)
For example, I moved it to call method, so this is how middleware class code looks like:
attr_accessor :subscriber
def call(env)
#subscriber ||= Thread.new do # if no subscriber present, init new one
redis_sub = Redis.new(url: ENV['REDISCLOUD_URL'])
redis_sub.subscribe(CHANNEL) do |on|
on.message do |_, msg|
# parsing message code here, retrieve user
send_message(user.id, { message: "ECHO: #{event.data}"} )
end
end
end
# other code from method
end
Both solutions solve the same problem: Redis-listening thread will be initialized for each Puma worker/thread, not for main process (which is actually not serving requests).
I am having problem connecting to the AMQP in my rSpec testing. I have code like this:
Module Rabbit
Class Client
def start
EventMachine.run do
connection = AMQP.connect(Settings_object) #it holds host, username and password information
channel = AMQP::Channel.new(connection)
channel.queue("queue_name", :durable => true)
channel.default_exchange.publish("A test message", :routing_key => "queue_name")
end
end
end
Module Esper
Class Server
def start
EventMachine.run do
connection = AMQP.connect(Settings_object) #it holds host, username and password information
=begin
Some code to subscribe to queues
=end
end
end
end
My problem is when I run the rspec:
#client = Rabbit::Client.new
#server = Esper::Server.new
Thread.new do
#client.start
end
Thread.new do
#server.start
end
At first Client is able to connect to the AMQP, and the Server doesn't , but when I run it for the second time, then the Client can't connect to the server. I can't see to overcome this problem. I don't see a reason why would Client stop connecting when I run it on the second time?
The root cause of this problem is that new for every queue for AMQP needs to have it's separate connection. For example:
queue1_connectom = AMQP::Channel.new(connection)
queue2_connectom = AMQP::Channel.new(connection)
And use it like that.
But overall for this whole situation is to use deamon-kit gem. It separates the AMQP into a separate application and the AMQP connections are handled within that "application" or better yet - A Deamon.
It also has a generator for AMQP so good thing would be to use that.
The good folks at LayerVault posted a easy way to add 'real-time' features on existing Ruby application without resorting to client side MCV or rewrite the entire app in Node.js, a while back. Their post show that you can update to all clients whenever your rails Models are updated [Reference 1] through delay_job & Active:Record Observer:
class FileObserver < ActiveRecord::Observer
observer :lv_file
def after_commit(record)
record.delay.report_updated
end
end
and
class LVFile < ActiveRecord::Base
def report_updated
Messenger.publish_message('file_updated', "file/#{self.id}")
end
end
[Reference 1] http://layervault.tumblr.com/post/31462727280/rails-in-realtime-part-2
Connect to socket.io server from client side javascript is easy. however I'm at lost how to implement Messenger.publish_message('file_updated', "file/#{self.id}") on rails server side. In the post they do mentioned that
The report_updated method makes a class method call to a separate Messenger class (our Socket.IO interface on the web app side) and reports that a file has changed to the appropriate room.
But I'm still not very sure how to implement Messager class in Rails so that Active:Record Observer will be able to connect to my socket.io server and send updates.
[PS] A conversation on HN indicates that they utilized this gem
The gem's wiki even included a brief demo:
require 'SocketIO'
client = SocketIO.connect("http://localhost", sync: true) do
before_start do
on_message {|message| puts message}
on_disconnect {puts "I GOT A DISCONNECT"}
end
after_start do
emit("loadLogs", "/var/www/rails_app/log/production.log")
end
end
puts "socket still running"
loop do
sleep 10
puts 'zzz'
end
The Messenger class doesn't have much to it. All you need is to create the connection, fire off the message and then wrap things up. This is the entirety of our messenger.rb:
require 'socketio-client'
class Messenger
def self.send_message!(name, args)
begin
SocketIO.connect(MESSENGER_SOCKETIO_ADDRESS, sync: true) do
after_start do
emit(name, args)
disconnect
end
end
rescue
end
end
end