I have a RoR app with background jobs using whenever and sidekiq gems.
In development environment when I launch sidekiq with local redis instance (on localhost) the job keeps getting executed without problems. But when I switch to a remote redis instance (Heroku add-on) and restart sidekiq, it says it started processing, but nothing happens and workers aren't doing any jobs.
Here's my config/schedule.rb (for whenever gem)
every 2.minutes do
rake "crawler:crawl"
end
Here's my initializers/redis.rb:
Sidekiq.configure_server do |config|
config.redis = { :url => 'redis://user:pass#spinyfin.redistogo.com:9098/' }
end
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://user:pass#spinyfin.redistogo.com:9098/' }
end
If I comment out the content in redis.rb and run a local redis instance, the jobs are processed normally. But when I use this remote redis instance, this shows up and then nothing gets processed:
2013-11-29T15:09:26Z 95156 TID-ov6y7e14o INFO: Booting Sidekiq 2.13.0 using redis://redistogo:user#spinyfin.redistogo.com:9098/ with options {}
2013-11-29T15:09:26Z 95156 INFO: Running in ruby 1.9.3p327 (2012-11-10 revision 37606) [x86_64-darwin11.4.2]
2013-11-29T15:09:26Z 95156 INFO: See LICENSE and the LGPL-3.0 for licensing details.
2013-11-29T15:09:26Z 95156 INFO: Starting processing, hit Ctrl-C to stop
Maybe you connecting to wrong redis database or not connected at all.
In my apps I use redis url without trailing slash. In your case:
This is for database "0"
redis://user:pass#spinyfin.redistogo.com:9098
And this for database "1"
redis://user:pass#spinyfin.redistogo.com:9098/1
I use the environment variable REDIS_URL to ensure that everything is using the same Redis.
Re: Heroku - I just read this here as I was searching for my own solution:
If you're running on Heroku, you can't rely on the config/database.yml as that platform relies on the DATABASE_URL environment variable to determine the database connection configuration. Heroku overwrites the database.yml during slug compilation so that it reads from DATABASE_URL.
Related
So far I had a simple application that only required the classic rails server to boot.
I have recently added the react_on_rails gem and it requires to boot a nodejs server to handle webpack and javascript stuff.
So I understand I need this foreman gem that is capable of managing several processes. So far so good, but then I'm still having a few problems understanding and deploying this enhanced app to my production environment (Phusion Passenger on Apache/Nginx)
So several questions :
Does passenger handle the transition from rails s to foreman start -f Procfile.dev automatically ?
If no then where do I setup things so passenger works ?
Side question : almost all google results refer to puppet when looking for foreman on passenger. Anyone could explain what puppet does in 1 line and if I really need it in production ? So far everythings runs smoothly on localhost with the foreman start -f Procfile.dev command so I don't know where this is coming from...
I am deploying my application to the Amazon Cloud using Capistrano, and I was expecting to have the rails + nodejs setup on every autoscaled instance (and Passenger would graciously handle all that). Am I thinking wrong ?
In our production environment we use eye to manage other processes related to the rails app. (Passenger will run from mod_passenger while the workers are controlled by eye)
And here is an example of how to start 4 concurrent queue_classic workers:
APP_ROOT = File.expand_path(File.dirname(__FILE__))
APP_NAME = File.basename(APP_ROOT)
Eye.config do
logger File.join(APP_ROOT, "log/eye.log")
end
Eye.application APP_NAME do
working_dir File.expand_path(File.dirname(__FILE__))
stdall 'log/trash.log' # stdout,err logs for processes by default
env 'RAILS_ENV' => 'production' # global env for each processes
trigger :flapping, times: 10, within: 1.minute, retry_in: 10.minutes
group 'qc' do
1.upto(4) do |i|
process "worker-#{i}" do
stdall "log/worker-#{i}.log"
pid_file "tmp/pids/worker-#{i}.pid"
start_command 'bin/rake qc:work'
daemonize true
stop_on_delete true
end
end
end
end
I use Redis caching in my Rails app:
config.cache_store = :redis_store, redis_url
When I push my Rails app straight to Heroku, it is deployed successfully. When using Travis, the Heroku deploy step fails because the asset precompilation attempts to connect to Redis.
Running: rake assets:precompile
rake aborted!
ArgumentError: invalid uri scheme ''
/tmp/build_7c5f167bf750cb2986dbb9c3510ea11e/vendor/bundle/ruby/2.1.0/gems/redis-3.2.0/lib/redis/client.rb:390:in `_parse_options'
I have tried various things: overriding RedisStore methods using rake tasks, moving the cache_store instantiation to the initialization phase, using Docker instead of sudo, changing Heroku build strategy and other travis.yml configurations etc.
I don't want to precompile locally, and I'd rather not change caching solution. Many other apps running on the cedar-14 stack use a very similar setup so the issue seems a bit peculiar.
Any suggestions how to resolve this Travis+Heroku deploy issue?
In my case, I solved this by changing the redis init to:
REDIS = Redis.new(:url => redis_url_string)
where previously I was parsing the URI and passing in the arguments as:
uri = URI.parse(redis_url)
REDIS = Redis.new(:host => uri.host, :port => uri.port, :password => uri.password, :scheme => uri.scheme)
I wonder if the cache store init has a similar init implementation for redis (I haven't checked the source there).
We solved the problem by overriding the Redis::Store initialization. When using Travis to trigger the Heroku deploy, Redis Store tries to connect to Redis. This is probably due to the current version (Nov 2013) of the Redis Store gem not being compatible with the current asset pipeline implementation. The reason why this works when pushing straight to Heroku is unclear. It could be related to the order that assets are compiled, when using the Heroku build strategies as specified in the travis.yml file. Perhaps these issues will be resolved in future Redis Store versions.
This is the rake task used to avoid loading Redis when using Redis Store as cache store (lib/assets/tasks/assets.rake):
pt = Rake::Task['assets:environment']
Rake.application.send(:eval, "#tasks.delete('assets:environment')")
namespace :assets do
task :environment do
class Redis
class Store
def initialize(options = { })
puts ”Do nothing"
end
end
def initialize(options = { })
puts ”Do nothing"
end
end
pt.execute
end
end
This is not a very elegant solution, but it does the trick for now. Consider changing the caching solution instead.
I'm very new to rails and recently I'm trying to use Resque and Rspec to test my codes.
What I'm stocked with is to change Resque environment to test.
This is my state:
Redis-server is lunched: port 6379, PID: 6046
resque-web is lunched:
$ resque-web
[2014-09-19 21:52:43 +0430] Starting 'resque-web'...
[2014-09-19 21:52:43 +0430] trying port 5678...
Couldn't get a file descriptor referring to the console</li>
Now I can see resque web console at http://0.0.0.0:5678/stats/resque
Well, on Stats tab, there's a table and at the first row, environment is set to development.
Resque Client connected to redis://127.0.0.1:6379/0
environment development
failed 0
pending 0
processed 37780
queues 0
servers ["redis://127.0.0.1:6379/0"]
workers 0
working 0
I want to set Resque environment to test, so that I test jobs created by my rspec tests using test database, but I don't know how.Any tips?
Try to set the RAILS_ENV environment variable before running resque-web, e.g.
$ RAILS_ENV=test resque-web
I'm trying to implement a background worker for payments using Sidekiq. I'm using nitrous.io for development and this is probably causing some problems when connecting to redis.
When I try to connect I get the following error:
Redis::CannotConnectError (Error connecting to Redis on 127.0.0.1:6379 (ECONNREFUSED)):
I assumed that this is the default (localhost).
I've tried to add the following initializer, sidekiq.rb
Sidekiq.configure_server do |config|
config.redis = { :url => 'redis://test-box-123456.euw1-2.nitrousbox.com:6379/0'}
end
Sidekiq.configure_client do |config|
config.redis = { :url => 'redis://test-box-123456.euw1-2.nitrousbox.com:6379/0'}
end
This however, results in the following error:
Redis::ProtocolError ( Got '<' as initial reply byte. If you're in a forking environment, such as Unicorn, you need to connect to Redis after forking. ):
I haven't worked with background sidekiq/redis before, and I cannot figure out what exactly is going wrong here. How should I configure this connection in both development (nitrous) and production (heroku)?
I was getting the same error locally and figured out that this was because I did not start redis server- type in a new cmd window:
redis-server
Hope that helps at least when testing locally.
I have to use websockets in my rake task and for that I changed my event.rb to
config.synchronize = true
# Uncomment and edit to point to a different redis instance.
# Will not be used unless standalone or synchronization mode
# is enabled.
config.redis_options = {:host => 'localhost', :port => '3000'}
and when I start my rails server I get this error:
! Invalid request
Exiting
/usr/local/rvm/gems/ruby-1.9.3-p194#socialmail/gems/redis-3.0.4/lib/redis/connection/synchrony.rb:115:in `read': Got 'Protocol error, got "H" as reply type byte' as initial reply byte. If you're in a forking environment, such as Unicorn, you need to connect to Redis after forking. (Redis::ProtocolError)
What am I doing wrong?
Thanks
Hey thanks for the question, I personally couldn't get a rake task of mine to post to a websocket channel I had open on my rails server. Your synchronize command helped (along with starting a Redis server locally).
Your problem through - seems like you're pointing to 3000. Is that your rails server or the Redis instance? If you're running it locally, I'd omit that line.