We have set up multple Redis instances in Herkou to handle different caches and queues, how do you set up Rails to use different instances?
You can initiate as many redis clients as you want and use the same at different places.
Say something like this,
cache1 = Redis.new(host: 'cache1.redis-server.com', port: 6379)
cache2 = Redis.new(host: 'cache2.redis-server.com', port: 6379)
queues = Redis.new(host: 'queues.redis-server.com', port: 6379)
cache1.set('my_key', 'my_value')
queues.lpush('my_queue', 'my_job')
If you are also using Sidekiq and want a separate connection for it, refer the docs here
Related
Summary: I'm using a single Redis instance for the Rails cache, actioncable and (non-cache) use in my rails code. Should all these uses share a single connection pool and if so how can I config this to make it happen since there seem to be totally different ways to setup the pooling for each?
Details follow since people seem to like to see them.
I'm using redis as the adapter for rails cache using the following config.
config.cache_store = :redis_cache_store, {
url: "redis://XXX.net:6379/0",
pool_size: ENV.fetch('RAILS_MAX_THREADS') { 5 },
password: Rails.application.credentials.dig(:redis, :password),
expires_in: 24.hours,
pool_timeout: 5
}
I've set the expires_in option so that I can set the option in my redis config to evict keys with expiration set so I can use the same redis instance for both cache and non-cache data. Now, I want to also access REDIS directly for non-cache related tasks via something like the example config below
pool_size = ENV.fetch("RAILS_MAX_THREADS", 5)
redis_pool = ConnectionPool.new(size: pool_size) do
Redis.new(
url: "redis://XXX.net:6379/0",
)
end
But I'm not sure if that is correct. Shouldn't I be sharing a connection pool between the cache_store connections and the other connections to Redis? If so, how can I do this?
To complicate matters further I'm also using Redis for actioncable via a config like
production:
adapter: redis
url: <%= ENV.fetch("REDIS_URL") { "redis://XXX.net:6379/0" } %>
password: <%= Rails.application.credentials.dig(:redis, :password) %>
I've seen suggestions that actioncable will automatically handle connection pooling with Redis if I'm using the connection_pool gem (is this right?) but I feel like all these connections should be drawing from the same pool. If so how can I make that happen?
I'm learning & doing SSE for the first time in rails! My controller code:
def update
response.headers['Content-Type'] = 'text/event-stream'
sse = SSE.new(response.stream, event: 'notice')
begin
User.listen_to_creation do |user_id|
sse.write({id: user_id})
end
rescue ClientDisconnected
ensure
sse.close
end
end
Front end:
var source = new EventSource('/site_update');
source.addEventListener('notice', function(event) {
var data = JSON.parse(event.data)
console.log(data)
});
Model pub/sub
class User
after_commit :notify_creation, on: :create
def notify_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
self.class.execute_query(connection, ["NOTIFY user_created, '?'", id])
end
end
def self.listen_to_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
begin
execute_query(connection, ["LISTEN user_created"])
connection.raw_connection.wait_for_notify do |event, pid, id|
yield id
end
ensure
execute_query(connection, ["UNLISTEN user_created"])
end
end
end
def self.clean_sql(query)
sanitize_sql(query)
end
private
def self.execute_query(connection, query)
sql = self.clean_sql(query)
connection.execute(sql)
end
end
I've noticed that if I'm writing to SSE, something trivial like in a tutorial like... sse.write({time_now: Time.now}), everything works great. In command line, CTRL+C successfully shuts down the local server.
However, whenever I need to write something that requires some kind of database action, for example when I'm doing a postgres pub/sub as in this tutorial, then CTRL+C doesn't shut down the local server, it's just stuck and hangs and requires me to manually kill the PID.
On the actual spun up server, sometimes a page refresh will hang forever as well. Other times, it will throw a timeout error:
ActiveRecord::ConnectionTimeoutError (could not obtain a connection from the pool within 5.000 seconds (waited 5.001 seconds); all pooled connections were in use):
Unfortunately this issue persists in production as well, where i'm using Heroku. I just get lots of timeout errors. But I think I have Heroku properly configured, and also local settings... my understanding is I just need to have a sizable pool (I have 5) to pull connections from and allow multiple threads. Below you'll find some config code.
THERE ARE NO ENV VARIABLES, DEFAULTS USED!
# config/database.yml
default: &default
adapter: postgresql
pool: <%= ENV.fetch("RAILS_MAX_THREADS") { 5 } %>
timeout: 5000
development:
<<: *default
database: proper_development
# config/puma.rb
workers Integer(ENV['WEB_CONCURRENCY'] || 1)
threads_count = Integer(ENV['MAX_THREADS'] || 5)
threads threads_count, threads_count
preload_app!
rackup DefaultRackup
port ENV['PORT'] || 3000
environment ENV['RACK_ENV'] || 'development'
on_worker_boot do
# Worker specific setup for Rails 4.1+
# See: https://devcenter.heroku.com/articles/deploying-rails-applications-with-the-puma-web-server#on-worker-boot
ActiveRecord::Base.establish_connection
end
If it's helpful here's the output when I run rails s
=> Booting Puma
=> Rails 5.0.2 application starting in development on http://localhost:3000
=> Run `rails server -h` for more startup options
Puma starting in single mode...
* Version 4.3.3 (ruby 2.4.0-p0), codename: Mysterious Traveller
* Min threads: 0, max threads: 16
* Environment: development
* Listening on tcp://127.0.0.1:3000
* Listening on tcp://[::1]:3000
Use Ctrl-C to stop
The issue here seems to be the lack of consistency between the puma threads and the database connections. If some connection was initiated by middleware etc through AR, the code you have written can lead to two connections being held in the same request cycle until you receive a notification and the thread finishes its job. AR caches connections per thread, so if a request was made and connection was checked out from the pool it will be held by that. Look at this issue for more details. If you end up using the connection pool to check out one more connection and make that connection wait till you get a notification from Postgres, potentially two connections can be held by the same Puma thread that is waiting.
To see this in action, start a new Rails server instance in development and make a request to your SSE endpoint. If you were getting timeouts before you might see two connections to Postgres while you made just one request to a newly launched server. So, even though your number of threads and connection pool size was same, you might run out of free connections from the pool. An easier way might be to just add this line in development after you checkout a connection to see how many cached connections are being held right now.
def self.listen_to_creation
ActiveRecord::Base.connection_pool.with_connection do |connection|
# Print or log this array
p ActiveRecord::Base.connection_pool.instance_variable_get(:#thread_cached_conns).keys.map(&:object_id)
begin
execute_query(connection, ["LISTEN user_created"])
.........
.........
Also, the snippets you have posted seem to indicate you are running up to 16 threads on a connection pool of size 5 in development environment, so that is a different issue.
To fix this, you would want to investigate which thread is holding the connection and if you can reuse it for your notification or just increase the DB pool size.
Now, coming to SSE itself here. Since a SSE connection blocks and reserves a thread in your current setup. If you have multiple requests to this endpoint you might quickly starve out of Puma threads itself making requests wait. This might work in case you are not expecting a lot of requests to this endpoint but if you are, you would need more free threads so you might even want to increase the Puma thread count. Ideally, though a non blocking server would work better here.
Edit:
Also, forgot to add that SSE in rails has an issue of keeping alive dead connections if it doesn't know the connection is dead. You might have threads endlessly waiting this way until some data comes and they realize the connection is no longer valid.
I am using a 3rd party library that sends emails using a HTTP based API.
Is it still possible for me to use ActionMailer to construct the emails and then somehow use ActionMailer to generate the resulting HTML/text for the email which I could then pass to my http email lib to send?
Absolutely. In fact, this is standard practice. You don't need to do anything special, just configure ActionMailer to use the 3rd party of your choice. Here's an example from my production.rb file, using Mailgun:
Rails.application.configure do
# other config stuff goes here...
config.action_mailer.delivery_method = :smtp
config.action_mailer.default_url_options = { host: 'your_domain.com' }
config.action_mailer.smtp_settings = {
authentication: :plain,
address: 'smtp.mailgun.org',
port: 587,
domain: Rails.application.secrets[:mailgun_domain],
user_name: Rails.application.secrets[:mailgun_username],
password: Rails.application.secrets[:mailgun_password]
}
end
Then in your Rails.application.secrets file, you put the info specific to your account.
I have a Rails project and two ruby mini-daemons running in the background. What's the best way to communicate between them?
Communication like below should be possible:
Rails -> Process 1 -> Process 2 -> Rails
Some requests would be sync, other async.
Queues (something like AMQ, or custom Redis based) or RPC HTTP calls?
Check DRb as well.
I implemented a system via RabbitMq + the bunny gem.
Update:
After reading http://blog.brightbox.co.uk/posts/queues-and-callbacks I decided to try out RabbitMQ. There are two gems amqp (async, eventmachine based) or bunny (sync). amqp is great, but if you're using Rails with passenger it can do some weird things.
The system works like this, the daemons listen on a queue for messages:
# The incoming data should be a JSON encoded hash that looks like:
# { "method" => method_to_call, "opts" => [ Array of opts for method ],
# "output" => "a queue where to send the result (optional)" }
# If output is specified it will publish the JSON encoded response there.
def listen_on(queue_name, class)
BUNNY.start
bunny = BUNNY.queue(queue_name)
bunny.subscribe do |msg|
msg = JSON.parse(msg[:payload])
result = class.new.send(msg["method"], *msg["opts"])
if msg["output"]
BUNNY.queue(msg["output"]).publish(result.to_json)
end
end
So once a message is received it calls a method from a class. One thing to note is that it would have been ideal to use bunny for Rails and amqp in the daemons. But I like to use one gem pe service.
I'm using central_logger to store logs from our Rails app in mongodb. When the mongo server went down recently our app started timing out on mongo inserts. How can I prevent Rails from timing out if the mongo server goes down?
The ruby driver supports timeouts like so
#conn = Connection.new("localhost", 27017, :pool_size => 5, :timeout => 5)
But the central_logger gem isn't using that. So you can either fork it to add that in there, or monkey-path the CentralLogger::MongoLogger.connect method
It currently has
def connect
#mongo_connection ||= Mongo::Connection.new(#db_configuration['host'],
#db_configuration['port'],
:auto_reconnect => true).db(#db_configuration['database'])
if #db_configuration['username'] && #db_configuration['password']
# the driver stores credentials in case reconnection is required
#authenticated = #mongo_connection.authenticate(#db_configuration['username'],
#db_configuration['password'])
end
end
You would need to monkey-path in :timeout=>5 (or whatever) to the Mongo::Connection.new
I would bet the author of central-logger would like to have this in there, so a fork and pull request would likely be welcome.
You could use replica sets - so if the master goes down, it can failover automatically to one of the replicas.
Usually the database insert should be fast, so you could work with the ruby timeout:
require 'timeout'
Timeout::timeout(0.2) do
... write to log server
end
this code will timeout and continue after 200 milliseconds in any case.