I've been struggling with this error for quite some time:
Redis::ProtocolError: Got 'i' as initial reply byte.
If you're running in a multi-threaded environment, make sure you pass the :thread_safe
option when initializing the connection. If you're in a forking environment, such as
Unicorn, you need to connect to Redis after forking.
It happens intermittently in apps that use Unicorn and Redis. From this redis-rb Github issue it looks the :thread_safe option is now enabled by default. I am using redis 2.2.2 because redis 3.0.1 is not compatible with the latest version of resque.
In my Unicorn config I'm using Redis.current.quit after the fork.
I'm also connecting to Redis with a gem called ruote-redis which is a storage implementation for the workflow engine Ruote.
How can I ensure that all of my Redis connections are stable and that I don't get this error anymore which is disrupting to the normal use of our app?
Unicorn is not multi-threaded. Are you using threads yourself?
As stated in the docs, the problem you're hitting is that multiple Unicorn workers are sharing the same connection (ie the same underlying file descriptor).
This change, included in version redis-rb 3.0, makes it even clearer.
If you're still hitting this error please post your Unicorn configuration.
Related
I have an app currently on a 48GB (12 vCPU) DO server, which is struggling with connection sizes. We have an API that gets hit ever 5 minutes by 350~ IoT devices and it gets added to a background queue for log processing.
I have tried using both delayed_job and sidekiq for background processing, however both create so many database connections that it exceeds the postgresql default 100 connection limit and causes downtime. I have had to temporarily remove this background processing. Without the background processing, we use around 60-65 postgresql connections.
I am running Dokku 0.6.5
Postgresql was installed using this dokku plugin
App is Ruby on
Rails
My database.yml pool size is set to 5, my puma worker threads are 5 so I followed herokus recommendation that these should match.t
I am not conclusive in how PgBouncer can be installed on a dokku app that is running postgresql through a plugin. Currently, my only idea is to upgrade my dokku to 0.15.0, which then has buildpack support and I can install the heroku PgBouncer buildpack.
I would greatly appreciate any insight on the best way to install PgBouncer with this set-up.
I want to use server-side events with rails. It works locally without any problem. But it does not trigger on production with unicorn.
Is it possible to use SSE with unicorn?
You need a threaded web server such as Puma, as each connection requires a separate thread to keep the long running connection open.
I was wondering if there was a way to use the Puma server (Rails default) JUST on the new Rails 5.1 system tests?
Right now on all our projects we use tiny_tds, but I was trying to experiment with 5.1 system tests with Capybara/Selenium but it fails of course because I do not have Puma installed/running.
I took a look through the documentation and didn't see anything about declaring what type of server you want to use. Were connecting to a SQL Server database so I don't know if Puma is able to do that (And that's probably why were using tiny_tds in the first place).
You're confusing database adapters and rack web servers which are very different things.
Puma (like Webrick, Thin, Unicorn etc) is a general purpose Rack web server. It sits and waits for incoming requests from vistors and dispatches them to an application (like a Rails app) by calling it with a hash containing the environment and request. The application returns an array containing the response. The server sends it back to the visitor.
tiny_tds on the other hand is a database adapter. Its used to connect the application to the database. The web server is almost completly oblivious to how the application creates a response from the request. This includes whatever databases are used.
In Rails 5 most servers don't require configuration. You just add the gem to the gemfile and run bundle install and they plug themselves in. There in no seperate installation step.
That was not the case earlier which is why Webrick - a server with the only real merit being that it does not require configuration was the default. It is now Puma which unlike Webrick is suited for production.
Using different servers in different environments is not a good idea since it violates the idea of dev/prod parity. Your dev & test environment should be as close as possible to what you are deploying to so that you can catch bugs before they end up in production. This means you should be running the same server, same database etc.
Running a seperate test server for different parts of your test suite sounds like a true fools errand - if its even possible without hacking the framework to bits.
For SQL Server there is activerecord-sqlserver-adapter which can use tiny_tds as a backend.
Is it possible to hot deploy Ruby just like PHP?
Normally I used FTP to upload the PHP file, then it will be available automatically.
Can Ruby hot deploy its file like this?
Your comment welcome.
Are you talking about a ruby on rails application ?
If so, when deploying a rails application in production mode, the all application gets loaded in memory. So changing the files won't affect the running application.
For hot restarting a rails application you will need to use solution such as:
Unicorn
Puma
Passenger
For a first time, Puma is the easiest way.
However if you are looking for a zero-downtime, either Unicorn or Passenger enterprise are what you are looking for.
EDIT
Unicorn
Free
Complex configuration
zero-downtime when hot restarting. when hot-restarting unicorn, it keeps the old threads working until the new ones are fully functionnal. So if the new ones fail to start, nothing happens. The old ones just keep going.
Puma
Free
Simple configuration
hot restart but no zero-downtime. When hot-restarting puma, it shuts down the old threads and starts the new ones. Puma keeps the sockets open, so the client are not disconnected, but are waiting to get a response while the new threads restart. However if the new threads fail to start, Puma can't restart the old ones. So connections are lost and the server is down.
Passenger
Free edition
Free
The configuration is easier than unicorn
hot-restart, but no zero-downtime. Like Puma.
Enterprise edition
$29/mo
The configuration is easier than unicorn
zero-downtime when hot restarting. Like Unicorn.
We're running a rails app via heroku that connects to a windows azure VM, where I've set up a redis master/slave to act as a cache (slash quick reference data store). The problem is, we sporadically get redis timeouts. This is with a timeout of 10 seconds (which I know is more than it needs), and reestablishing redis connections on fork. And using hiredis as a driver.
Anyone have a clue why this might be happening? I know heroku and the azure vm are hosted on different coasts, so there's a bit of latency; could there be TCP request drops? I'm fairly out of ideas.