TLDR;
Why Sneakers worker can't connect to the database or can't query it?
(General advices on "do's" and "dont's" are also welcome in comments)
Full question:
I am able to execute RPC call that returns a simple string, but I can't execute RPC call that is querying the database on the server side. I read the docs, tried many SO posts and blog tutorials, but I am still missing some piece.
I have two services. First service (Client) is using Bunny gem and is making an RPC call to second service (RPCServer) which is listening on workers using Sneakers gem. Both services are Rails apps.
RabbitMQ is serving in a docker container:
docker run -p 5672:5672 -p 15672:15672 rabbitmq:3-management
Postgres database is installed on a local machine.
Client service (mostly from Rabbitbunny docs ):
# app/services/client.rb
class Client
attr_accessor :call_id, :lock, :condition, :reply_queue, :exchange, :params, :response, :server_queue_name, :channel, :reply_queue_name
def initialize(rpc_route:, params:)
#channel = channel
#exchange = channel.fanout("Client.Server.exchange.#{params[:controller]}")
#server_queue_name = "Server.Client.queue.#{rpc_route}"
#reply_queue_name = "Client.Server.queue.#{params[:controller]}"
#params = params
setup_reply_queue
end
def setup_reply_queue
#lock = Mutex.new
#condition = ConditionVariable.new
that = self
#reply_queue = channel.queue(reply_queue_name, durable: true)
reply_queue.subscribe do |_delivery_info, properties, payload|
if properties[:correlation_id] == that.call_id
that.response = payload
that.lock.synchronize { that.condition.signal }
end
end
end
def call
#call_id = "NAIVE_RAND_#{rand}#{rand}#{rand}"
exchange.publish(params.to_json,
routing_key: server_queue_name,
correlation_id: call_id,
reply_to: reply_queue.name)
lock.synchronize { condition.wait(lock) }
connection.close
response
end
def channel
#channel ||= connection.create_channel
end
def connection
#connection ||= Bunny.new.tap { |c| c.start }
end
end
RPCServer service, using this gist (comments here are the "meat" of my question:
# app/workers/posts_worker.rb
require 'sneakers'
require 'sneakers/runner'
require 'byebug'
require 'oj'
class RpcServer
include Sneakers::Worker
from_queue 'Client.Server.queue.v1/filters/posts', durable: true, env: nil
def work_with_params(deserialized_msg, delivery_info, metadata)
post = {}
p "ActiveRecord::Base.connected?: #{ActiveRecord::Base.connected?}" # => true
##### This gets logged
Rails.logger.info "ActiveRecord::Base.connection_pool: #{ActiveRecord::Base.connection_pool}\n\n-------"
##### This never gets logged
Rails.logger.info "ActiveRecord::Base.connection_pool.with_connection: #{ActiveRecord::Base.connection_pool.with_connection}\n\n--------"
### interpreter never reaches this place when ActiveRecord methods like `with_connection`, `where`, `count` etc. are used
ActiveRecord::Base.connection_pool.with_connection do
post = Post.first.to_json
end
##### first commented `publish()` works fine and RPC works when no ActiveRecord is involved (this is, assuming above code using ActiveRecord is commented out)
##### second publish is not working
# publish("response from RPCServer", {
publish(post.to_json, {
to_queue: metadata[:reply_to],
correlation_id: metadata[:correlation_id],
content_type: metadata[:content_type]
})
ack!
end
end
Sneakers::Runner.new([RpcServer]).run
RPCServer sneakers configuration:
# config/initializers/sneakers.rb
Sneakers.configure({
amqp: "amqp://guest:guest#localhost:5672",
vhost: '/',
workers: 4,
log: 'log/sneakers.log',
pid_path: "tmp/pids/sneakers.pid",
timeout_job_after: 5,
prefetch: 10,
threads: 10,
durable: true,
ack: true,
heartbeat: 2,
exchange: "",
hooks: {
before_fork: -> {
Rails.logger.info('Worker: Disconnect from the database')
ActiveRecord::Base.connection_pool.disconnect!
Rails.logger.info("before_fork: ActiveRecord::Base.connected?: #{ActiveRecord::Base.connected?}") # => false
},
after_fork: -> {
ActiveRecord::Base.connection
Rails.logger.info("after_fork: ActiveRecord::Base.connected?: #{ActiveRecord::Base.connected?}") # => true
Rails.logger.info('Worker: Reconnect to the database')
},
timeout_job_after: 60
})
Sneakers.logger.level = Logger::INFO
RPCServer puma configuration:
# config/puma.rb
threads_count = ENV.fetch("RAILS_MAX_THREADS") { 5 }
threads threads_count, threads_count
port ENV.fetch("PORT") { 3000 }
environment ENV.fetch("RAILS_ENV") { "development" }
workers ENV.fetch("WEB_CONCURRENCY") { 2 }
preload_app!
### tried and did not work
# on_worker_boot do
# ActiveSupport.on_load(:active_record) do
# ActiveRecord::Base.establish_connection
# end
# end
before_fork do |server, worker|
# other settings
if defined?(ActiveRecord::Base)
ActiveRecord::Base.connection.disconnect!
end
end
after_worker_boot do |server, worker|
if defined?(ActiveRecord::Base)
ActiveRecord::Base.establish_connection
end
end
plugin :tmp_restart
for completeness, I also have an external Rakefile that is binding queues to exchanges (probably not important in this case)
namespace :rabbitmq do
desc "Setup routing"
task :setup do
conn = start_bunny
rpc_route service: :blog, from: 'v1/filters/posts_mappings', to: 'v1/filters/posts'
conn.close
end
def rpc_route(service:, from:, to:)
...
end
def start_bunny
...
end
end
I tried many sneakers configurations, and many orders of launching rabbitmq, resetting it, deleting queues, connections, etc. All of it is hard to list here and probably not the case.
Why I can't connect to the database or execute ActiveRecord methods? What Am I missing?
Ok I got it. The problem was last line of worker in RPCServer:
Sneakers::Runner.new([RpcServer]).run
It was running worker outside of Rails app. Commenting this out solved my problem of worker not being able to query database.
I'm building an app that supports realtime bid using Faye-websocket. But I got this 200 error and I have no idea what problem it is.
Error:
WebSocket connection to 'ws://localhost/auctions/3' failed: Error during WebSocket handshake: Unexpected response code: 200
SocketConnection.rb
require 'faye/websocket'
require 'websocket/extensions'
require 'thread'
require 'json'
class SocketConnection
KEEPALIVE_TIME = 15 # in seconds
def initialize app
#app = app
end
def call env
#env = env
if Faye::WebSocket.websocket?(env)
socket = Faye::WebSocket.new env
socket.ping 'Mic check, one, two' do
p [:ping, socket.object_id, socket.url]
end
socket.on :open do |event|
p [:open, socket.object_id, socket.url]
p [:open, socket.url, socket.version, socket.protocol]
end
socket.rack_response
else
#app.call(env)
end
end
end
I firgured out the problem. It requires a server to support socket connection. In my case, I use thin server. All errors are fixed
I following Ryan Bates tutorial ActionController Live, and deploying app at heroku. All works fine, except events, where Ryan sad that we should reopen the redis connection, and I cant do it. I using RedisToGo to perform redis on heroku.
Here my events controller action:
def events
response.headers["Content-Type"] = "text/event-stream"
redis = Redis.new(:url => uri)
redis.psubscribe('messages.*') do |on|
on.pmessage do |pattern, event, data|
response.stream.write("event: #{event}\n")
response.stream.write("data: #{data}\n\n")
end
end
rescue IOError
logger.info "Stream closed"
ensure
redis.quit
response.stream.close
end
Also here redis initializer:
uri = URI.parse(ENV["REDISTOGO_URL"])
REDIS = Redis.new(:url => uri)
Can someone help me?
EDIT
I got all to work just initializing the client using Redis.new(url: ENV["REDISTOGO_URL"]) instead of parsing the URI in events controller action.
replace this:
redis = Redis.new(:url => uri)
redis.psubscribe
with this:
REDIS.psubscribe
anywhere you have 'redis' above, replace with the REDIS global.
I am trying to create a thread in Rails to subscribe a message channel of Redis. Is there a way to do this? I am using unicorn.
I have tried to do this in the unicorn configuration like this:
after_fork do |server, worker|
Thread.new do
begin
$redis.subscribe(:one, :two) do |on|
on.subscribe do |channel, subscriptions|
puts "Subscribed to ##{channel} (#{subscriptions} subscriptions)"
end
on.message do |channel, message|
puts "##{channel}: #{message}"
$redis.unsubscribe if message == "exit"
end
on.unsubscribe do |channel, subscriptions|
puts "Unsubscribed from ##{channel} (#{subscriptions} subscriptions)"
end
end
rescue Redis::BaseConnectionError => error
puts "#{error}, retrying in 1s"
sleep 1
retry
end
end
end
But it will make the unicorn server unable to handle any web request. I thought that if I am using a different thread to subscribe to Redis, it won't block the main thread; am I missing something here?
The problem here is the GIL in ruby; and the client library of Redis ruby is using a loop for the subscribe command.
I am working on rabbitmq and trying to push a method to a queue from my ruby on rails app and I am running a server side ruby script to read the queue and execute the method which is send in the payload. Here is my client side code.
module Rabbitesh
require 'amqp'
#debugger
def self.call_rabbits(payload,queue_name)
AMQP.start(:host => "localhost") do |connection|
channel = AMQP::Channel.new(connection)
queue = channel.queue(queue_name)
channel.default_exchange.publish(payload, :routing_key => queue.name)
#EM.add_timer(0.01) do
connection.close do
#end
end
end
end
end
This is now I call the Rabbitmq function
Rabbitesh::call_rabbits(obj,"welcome_mail")
where "welcome_mail" is the queue_name
This is the server side script
require 'rubygems'
require 'amqp'
require 'daemons'
options = { :backtrace => true, :dir => '.', :log_output => true}
Daemons.run_proc('raabbitmq_daemon',options) do
AMQP.start(:host => "localhost") do |connection|
channel = AMQP::Channel.new(connection)
queue = channel.queue("welcome_mail")
Signal.trap("INT") do
connection.close do
EM.stop { exit }
end
end
puts " [*] Waiting for messages. To exit press CTRL+C"
queue.subscribe do |body|
UserMailers.welcome_organic(body).deliver
end
end
end
The problem is when my rails app calls the rabbitmq function the console stops there saying "updating client properties" and though I will be running my server side ruby script, it will not read the queue and take execute the process. I am not able to understand whats wrong with the code, kindly help me out.