I'm trying to implement a background worker for payments using Sidekiq. I'm using nitrous.io for development and this is probably causing some problems when connecting to redis.
When I try to connect I get the following error:
Redis::CannotConnectError (Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED)):
I assumed that this is the default (localhost).
I've tried to add the following initializer, sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => 'redis://test-box-123456.euw1-2.nitrousbox.com:6379/0'}
end
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://test-box-123456.euw1-2.nitrousbox.com:6379/0'}
end
This however, results in the following error:
Redis::ProtocolError ( Got '<' as initial reply byte. If you're in a forking environment, such as Unicorn, you need to connect to Redis after forking. ):
I haven't worked with background sidekiq/redis before, and I cannot figure out what exactly is going wrong here. How should I configure this connection in both development (nitrous) and production (heroku)?
I was getting the same error locally and figured out that this was because I did not start redis server- type in a new cmd window:
redis-server
Hope that helps at least when testing locally.
Related
I am using Sidekiq on an app hosted on Heroku.
I have been using the redis gem.
Now I am trying to upgrade to sidekiq 7 which uses redis-client instead of redis. And when doing so I get this error:
redis-client requires Redis 6+ with HELLO command available
The Redis instance I have on Heroku is on the version 6.2.6 (so Redis 6+) (picture attached)
Below my redis.rb file (which is in initializers). I replaced Redis with RedisClient but I still get the error in both cases (in the case where I use the redis gem with Sidekiq 7 and the case where I use redis-client).
It means that it is when the Sidekiq inner code is using redis-client that the error is happening.
Any thoughts or ideas would be highly appreciated 🙏🏼 Thanks in advance!
I was expecting this to work as my Redis instance is on a 6+ version
url = ENV["REDISCLOUD_URL"]
if url
Sidekiq.configure_server do |config|
config.redis = { url: url }
end
Sidekiq.configure_client do |config|
config.redis = { url: url }
end
end
Searching the Sidekiq issues for "HELLO" gives you this issue:
https://github.com/sidekiq/sidekiq/issues/5594#issuecomment-1302384905
I have a RoR app with background jobs using whenever and sidekiq gems.
In development environment when I launch sidekiq with local redis instance (on localhost) the job keeps getting executed without problems. But when I switch to a remote redis instance (Heroku add-on) and restart sidekiq, it says it started processing, but nothing happens and workers aren't doing any jobs.
Here's my config/schedule.rb (for whenever gem)
every 2.minutes do
rake "crawler:crawl"
end
Here's my initializers/redis.rb:
Sidekiq.configure_server do |config|
config.redis = { :url => 'redis://user:pass#spinyfin.redistogo.com:9098/' }
end
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://user:pass#spinyfin.redistogo.com:9098/' }
end
If I comment out the content in redis.rb and run a local redis instance, the jobs are processed normally. But when I use this remote redis instance, this shows up and then nothing gets processed:
2013-11-29T15:09:26Z 95156 TID-ov6y7e14o INFO: Booting Sidekiq 2.13.0 using redis://redistogo:user#spinyfin.redistogo.com:9098/ with options {}
2013-11-29T15:09:26Z 95156 INFO: Running in ruby 1.9.3p327 (2012-11-10 revision 37606) [x86_64-darwin11.4.2]
2013-11-29T15:09:26Z 95156 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2013-11-29T15:09:26Z 95156 INFO: Starting processing, hit Ctrl-C to stop
Maybe you connecting to wrong redis database or not connected at all.
In my apps I use redis url without trailing slash. In your case:
This is for database "0"
redis://user:pass#spinyfin.redistogo.com:9098
And this for database "1"
redis://user:pass#spinyfin.redistogo.com:9098/1
I use the environment variable REDIS_URL to ensure that everything is using the same Redis.
Re: Heroku - I just read this here as I was searching for my own solution:
If you're running on Heroku, you can't rely on the config/database.yml as that platform relies on the DATABASE_URL environment variable to determine the database connection configuration. Heroku overwrites the database.yml during slug compilation so that it reads from DATABASE_URL.
I am using Redis with my Rails app. I have sidekiq gem installed too. My redis server runs in the same machine in default port.
I created a initializer which initalizes a redis lient.
config/initializers/redis.rb
$redis = Redis.new(:host => 'localhost', :port => 6379)
I have another initalizer that sets the number of accounts currently active in the system.
config/initializers/z_account_list.rb
$redis.set('accounts',Account.count);
In one of my views i am using this piece of code.
<div class="more-text">and <%= "#{$redis.get('accounts')}" %> more...</div>
When i set the value for accounts manually in redis without using the initializer, everything works fine. But when i add the initializer, i get
ActionView::Template::Error (Tried to use a connection from a child process without reconnecting. You need to reconnect to Redis after forking.):
I searched for the error. But most solutions are for resque and has something to do with after_fork. Since i am new to Rails and Redis and since i am not using Resque i am getting a little confused. Please help me out on this.
In forked environments like pushion passenger, we have to reconnect to redis whenever a worker is forked. My biggest confusion was where to put the reconnection statements. In many blogs it was suggested to put it in config/environments.rb. But it didn't work for me.
I added
if defined?(PhusionPassenger)
PhusionPassenger.on_event(:starting_worker_process) do |forked|
if forked
$redis.client.disconnect
$redis = Redis.new(:host => 'localhost', :port => 6379)
Rails.logger.info "Reconnecting to redis"
else
# We're in conservative spawning mode. We don't need to do anything.
end
end
end
to config/initializers/redis.rb
and everything started working fine.
I am having issues converting from Resque to Sidekiq. I'm not getting any errors though a perform_async(ids) doesn't add anything to Redis. I can add keys directly to the Redis server via the Redis.current.append("test", "key")
Also my Sidekiq worker connects to the Redis server, though I get an empty array when I ask for Sidekiq::Client.registered_workers The web UI shows only the skeleton with no information other than the Redis info. I don't know if this matters but Sidekiq.redis { |conn| conn.info } returns information that is all correct with my local Redis server. Though Sidekiq.server? returns a nil value.
Update: When I perform a perform_async(args) it returns a string.
It sounds like your Sidekiq configuration isn't using the same redis config for the client and server components of Sidekiq.
The client executes within your Rails app server, while the server is a stand-alone separate process where work is performed. If they both aren't using the same redis queue via the redis configuration, work will not be processed.
Here is an example config/initializers/sidekiq.rb config block using the same redis for both components:
redis = { url: (ENV['REDIS_URL'] || 'redis://127.0.0.1'), namespace: 'sidekiq' }
Sidekiq.configure_server do |config|
config.redis = redis
end
Sidekiq.configure_client do |config|
config.redis = redis
end
My guess is that you have a different namespace or configuration between your client and server config.
I was hitting the same issue.
Turns out I had rspec-sidekiq configured in my Gemfile's development group and rspec-sidekiq apparently stubs the async calls for testing.
The solution was amending my Gemfile as such:
gem 'rspec-sidekiq', group: :test, require: false
Finally, I added require 'rspec-sidekiq' in my spec_helper.rb.
I have to use websockets in my rake task and for that I changed my event.rb to
config.synchronize = true
# Uncomment and edit to point to a different redis instance.
# Will not be used unless standalone or synchronization mode
# is enabled.
config.redis_options = {:host => 'localhost', :port => '3000'}
and when I start my rails server I get this error:
! Invalid request
Exiting
/usr/local/rvm/gems/ruby-1.9.3-p194#socialmail/gems/redis-3.0.4/lib/redis/connection/synchrony.rb:115:in `read': Got 'Protocol error, got "H" as reply type byte' as initial reply byte. If you're in a forking environment, such as Unicorn, you need to connect to Redis after forking. (Redis::ProtocolError)
What am I doing wrong?
Thanks
Hey thanks for the question, I personally couldn't get a rake task of mine to post to a websocket channel I had open on my rails server. Your synchronize command helped (along with starting a Redis server locally).
Your problem through - seems like you're pointing to 3000. Is that your rails server or the Redis instance? If you're running it locally, I'd omit that line.