I am using Sidekiq on an app hosted on Heroku.
I have been using the redis gem.
Now I am trying to upgrade to sidekiq 7 which uses redis-client instead of redis. And when doing so I get this error:
redis-client requires Redis 6+ with HELLO command available
The Redis instance I have on Heroku is on the version 6.2.6 (so Redis 6+) (picture attached)
Below my redis.rb file (which is in initializers). I replaced Redis with RedisClient but I still get the error in both cases (in the case where I use the redis gem with Sidekiq 7 and the case where I use redis-client).
It means that it is when the Sidekiq inner code is using redis-client that the error is happening.
Any thoughts or ideas would be highly appreciated 🙏🏼 Thanks in advance!
I was expecting this to work as my Redis instance is on a 6+ version
url = ENV["REDISCLOUD_URL"]
if url
Sidekiq.configure_server do |config|
config.redis = { url: url }
end
Sidekiq.configure_client do |config|
config.redis = { url: url }
end
end
Searching the Sidekiq issues for "HELLO" gives you this issue:
https://github.com/sidekiq/sidekiq/issues/5594#issuecomment-1302384905
Related
I am running sidekiq on a production rails server, and some of its queues workers end up in my development sidekiq instance and, therefore, causes a bunch of errors. The same thing happens with my development sidekiq instance. When I'm in the development version of the application and I click something that kicks off a sidekiq worker, it sometimes tries to run it in the production sidekiq instance and the production database doesn't have the same information.
Is there a proper way to separate these queued jobs so that they only run in the instance which they are initiated?
Sidekiq uses Redis to store enqueued jobs. By default, all sidekiq instances will use the default Redis database. You can change this by creating an initializer in config/initializers/sidekiq.rb
if Rails.env == :production
redis_host = '127.0.0.1'
redis_port = 6379
redis_db = 1
else
redis_host = '127.0.0.1'
redis_port = 6379
redis_db = 2
end
Sidekiq.configure_server do |config|
config.redis = { url: "redis://#{redis_host}:#{redis_port}/#{redis_db}" }
end
Sidekiq.configure_client do |config|
config.redis = { url: "redis://#{redis_host}:#{redis_port}/#{redis_db}" }
end
And change redis_host, redis_port and redis_db with appropriate values. This way each environment will use a separate Redis database. Also don't forget to pass appropriate environment when starting a sidekiq instance with something like:
bundle exec sidekiq -e production
How we should configure action cable in Rails to work in Google App Engine?
I have a Rails application which is using action cable for its messenger.
On my local-host the action cable working well in development and production environments but its not working on Google App Engine.
I have created a Computer Engine instance and installed Redis on it.
Firewall allows all traffic to Redis VM.
Redis has bind 0.0.0.0
When I deploy the app on Google App Engine I get the error:
Firefox can’t establish a connection to the server at wss://mydomain.com/cable.
here is my current configuration:
config/environments/production.rb
config.action_cable.url = 'wss://[MYDOMAIN].com/cable'
config.action_cable.disable_request_forgery_protection = true
config.force_ssl = true
...
config/cable.yml
production:
adapter: redis
url: redis://[IP_OF_REDIS_SERVER]:6379/
app.yaml
entrypoint: bundle exec rackup --port $PORT
env: flex
runtime: ruby
env_variables:
REDIS_PROVIDER: REDIS_URL
REDIS_URL: redis://[IP_OF_REDIS_SERVER]:6379/
SECRET_KEY_BASE: [My_Secret_Key]
I couldn't find anything about actioncable setting in Google documentation of App Engine. I hope this question can help me and everyone with same issue.
You must set create a VPC between your Redis instance and you App Engine instance. This is documented here:
https://cloud.google.com/appengine/docs/standard/python/connecting-vpc
Once you've done this, you'll have:
app.yaml
...
vpc_access_connector:
name: projects/PROJECT_ID/locations/REGION/connectors/CONNECTOR_NAME
...
Secondly, the Google managed Redis server does not allow you to set a client id. I tried forcing it to be nil, but the ActionCable subscription adapter will try to set one for you:
https://github.com/rails/rails/blob/6-0-stable/actioncable/lib/action_cable/subscription_adapter/redis.rb#L18
In order to get it working properly, I had to override this behavior.
I added the following file into my rails app:
config/initializers/decorators/redis.rb
require 'action_cable/subscription_adapter/redis'
ActionCable::SubscriptionAdapter::Redis.class_eval do
cattr_accessor :redis_connector, default: ->(config) do
::Redis.new(config.except(:id, :adapter, :channel_prefix))
end
end
Now everything is working fine. My only concern now is that the client id is somehow required and I'm going to end up with some nasty bug due to it.
Hope this works for you if you try it out!
I'm trying to implement a background worker for payments using Sidekiq. I'm using nitrous.io for development and this is probably causing some problems when connecting to redis.
When I try to connect I get the following error:
Redis::CannotConnectError (Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED)):
I assumed that this is the default (localhost).
I've tried to add the following initializer, sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => 'redis://test-box-123456.euw1-2.nitrousbox.com:6379/0'}
end
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://test-box-123456.euw1-2.nitrousbox.com:6379/0'}
end
This however, results in the following error:
Redis::ProtocolError ( Got '<' as initial reply byte. If you're in a forking environment, such as Unicorn, you need to connect to Redis after forking. ):
I haven't worked with background sidekiq/redis before, and I cannot figure out what exactly is going wrong here. How should I configure this connection in both development (nitrous) and production (heroku)?
I was getting the same error locally and figured out that this was because I did not start redis server- type in a new cmd window:
redis-server
Hope that helps at least when testing locally.
I have a RoR app with background jobs using whenever and sidekiq gems.
In development environment when I launch sidekiq with local redis instance (on localhost) the job keeps getting executed without problems. But when I switch to a remote redis instance (Heroku add-on) and restart sidekiq, it says it started processing, but nothing happens and workers aren't doing any jobs.
Here's my config/schedule.rb (for whenever gem)
every 2.minutes do
rake "crawler:crawl"
end
Here's my initializers/redis.rb:
Sidekiq.configure_server do |config|
config.redis = { :url => 'redis://user:pass#spinyfin.redistogo.com:9098/' }
end
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://user:pass#spinyfin.redistogo.com:9098/' }
end
If I comment out the content in redis.rb and run a local redis instance, the jobs are processed normally. But when I use this remote redis instance, this shows up and then nothing gets processed:
2013-11-29T15:09:26Z 95156 TID-ov6y7e14o INFO: Booting Sidekiq 2.13.0 using redis://redistogo:user#spinyfin.redistogo.com:9098/ with options {}
2013-11-29T15:09:26Z 95156 INFO: Running in ruby 1.9.3p327 (2012-11-10 revision 37606) [x86_64-darwin11.4.2]
2013-11-29T15:09:26Z 95156 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2013-11-29T15:09:26Z 95156 INFO: Starting processing, hit Ctrl-C to stop
Maybe you connecting to wrong redis database or not connected at all.
In my apps I use redis url without trailing slash. In your case:
This is for database "0"
redis://user:pass#spinyfin.redistogo.com:9098
And this for database "1"
redis://user:pass#spinyfin.redistogo.com:9098/1
I use the environment variable REDIS_URL to ensure that everything is using the same Redis.
Re: Heroku - I just read this here as I was searching for my own solution:
If you're running on Heroku, you can't rely on the config/database.yml as that platform relies on the DATABASE_URL environment variable to determine the database connection configuration. Heroku overwrites the database.yml during slug compilation so that it reads from DATABASE_URL.
I am having issues converting from Resque to Sidekiq. I'm not getting any errors though a perform_async(ids) doesn't add anything to Redis. I can add keys directly to the Redis server via the Redis.current.append("test", "key")
Also my Sidekiq worker connects to the Redis server, though I get an empty array when I ask for Sidekiq::Client.registered_workers The web UI shows only the skeleton with no information other than the Redis info. I don't know if this matters but Sidekiq.redis { |conn| conn.info } returns information that is all correct with my local Redis server. Though Sidekiq.server? returns a nil value.
Update: When I perform a perform_async(args) it returns a string.
It sounds like your Sidekiq configuration isn't using the same redis config for the client and server components of Sidekiq.
The client executes within your Rails app server, while the server is a stand-alone separate process where work is performed. If they both aren't using the same redis queue via the redis configuration, work will not be processed.
Here is an example config/initializers/sidekiq.rb config block using the same redis for both components:
redis = { url: (ENV['REDIS_URL'] || 'redis://127.0.0.1'), namespace: 'sidekiq' }
Sidekiq.configure_server do |config|
config.redis = redis
end
Sidekiq.configure_client do |config|
config.redis = redis
end
My guess is that you have a different namespace or configuration between your client and server config.
I was hitting the same issue.
Turns out I had rspec-sidekiq configured in my Gemfile's development group and rspec-sidekiq apparently stubs the async calls for testing.
The solution was amending my Gemfile as such:
gem 'rspec-sidekiq', group: :test, require: false
Finally, I added require 'rspec-sidekiq' in my spec_helper.rb.